Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Complete
    • Affects Version/s: 5.0 M1
    • Fix Version/s: 5.0 RC1
    • Component/s: Reactive, Web
    • Labels:
      None
    • Last commented by a User:
      true

      Description

      This needs to be investigated. I don't know if there is anything in the Servlet API for this that's non-blocking.

        Issue Links

          Activity

          Hide
          sdeleuze Sébastien Deleuze added a comment -

          After a meeting with Silvano Riz, the developer of NIO Multipart, we agree on collaborating on a POC that will use Reactor 3 as a foundation to build a truly Reactive Multipart library. The idea is to expose an API like Flux<Part> resolveMultipartRequest(Map<String, List<String>> headers, Publisher<ByteBuffer> body) with Part allowing to get the part body as a Flux<ByteBuffer> or as a Mono<String>.

          Show
          sdeleuze Sébastien Deleuze added a comment - After a meeting with Silvano Riz, the developer of NIO Multipart, we agree on collaborating on a POC that will use Reactor 3 as a foundation to build a truly Reactive Multipart library. The idea is to expose an API like Flux<Part> resolveMultipartRequest(Map<String, List<String>> headers, Publisher<ByteBuffer> body) with Part allowing to get the part body as a Flux<ByteBuffer> or as a Mono<String> .
          Hide
          sdeleuze Sébastien Deleuze added a comment -

          I postpone this issue to RC1 since it will require a significant amount of work and testing.

          Show
          sdeleuze Sébastien Deleuze added a comment - I postpone this issue to RC1 since it will require a significant amount of work and testing.
          Hide
          sdeleuze Sébastien Deleuze added a comment -

          Please find bellow a summary of current status.

          We did not find the time Silvano Riz and me to work on a reactive multipart implementation, so currently we use NIO Multipart API which is event-based BUT use blocking IO (InputStream / OutputStream), see this related discussion. It would be nice to review current bridge between NIO Multipart and Spring Web Reactive to have a better idea of the limitation of the current implementation.

          Based on my discussion with Stéphane Maldini and this RxNetty discussion:

          • Current Netty support for multipart can't be plugged easily in RxNetty or Reactor Netty architecture
          • For long term, it would be nice to work with Norman on a way to get Multipart support from Netty that could be plugged in RxNetty and Reactor Netty
          • Current Multipart support on Reactor Netty is mainly client side oriented
          • Even if we add a Flux<ByteBufFlux> MultipartInbound#from(HttpServerRequest inbound) variant, that would only allow us to get the content of the file (no headers, not filename, no encoding support, etc.) so there would non-trivial improvements to support what we need to expose in Spring multipart API.

          My current proposal for 5.0 would be:

          • Review NIO Multipart to Spring Web Reactive bridge with the team (including Simon and Stéphane) to identify the impact of the part of the API that is blocking
          • Identify if we should use specific schedulers to run NIO Multipart in order to not block too much Netty threads
          • Decide if we should expose both Mono<MultiValueMap<String, Part>> getFormParts() and Flux<Part> getFormPartsAsFlux(), only the Mono based variant (I tend to think we should provide some kind of multipart support in 5.0 even if it comes with some limitation clearly documented)
          • Keep this issue/PR scope as it is (already quite big) and create 2 distinct issues to provide integration in both annotation and functional API

          For 5.1 target, 2 roads could be explored:

          • Try to see if we can move forward on first-class multipart reactive support on RxNetty and Reactor Netty (possibly by refactoring Netty Multipart support, to be discussed with Norman)
          • Collaborate with Silvano Riz on a reactive multipart implementation that could be used independently of the engine used
          Show
          sdeleuze Sébastien Deleuze added a comment - Please find bellow a summary of current status. We did not find the time Silvano Riz and me to work on a reactive multipart implementation, so currently we use NIO Multipart API which is event-based BUT use blocking IO ( InputStream / OutputStream ), see this related discussion . It would be nice to review current bridge between NIO Multipart and Spring Web Reactive to have a better idea of the limitation of the current implementation. Based on my discussion with Stéphane Maldini and this RxNetty discussion : Current Netty support for multipart can't be plugged easily in RxNetty or Reactor Netty architecture For long term, it would be nice to work with Norman on a way to get Multipart support from Netty that could be plugged in RxNetty and Reactor Netty Current Multipart support on Reactor Netty is mainly client side oriented Even if we add a Flux<ByteBufFlux> MultipartInbound#from(HttpServerRequest inbound) variant, that would only allow us to get the content of the file (no headers, not filename, no encoding support, etc.) so there would non-trivial improvements to support what we need to expose in Spring multipart API. My current proposal for 5.0 would be: Review NIO Multipart to Spring Web Reactive bridge with the team (including Simon and Stéphane) to identify the impact of the part of the API that is blocking Identify if we should use specific schedulers to run NIO Multipart in order to not block too much Netty threads Decide if we should expose both Mono<MultiValueMap<String, Part>> getFormParts() and Flux<Part> getFormPartsAsFlux() , only the Mono based variant (I tend to think we should provide some kind of multipart support in 5.0 even if it comes with some limitation clearly documented) Keep this issue/PR scope as it is (already quite big) and create 2 distinct issues to provide integration in both annotation and functional API For 5.1 target, 2 roads could be explored: Try to see if we can move forward on first-class multipart reactive support on RxNetty and Reactor Netty (possibly by refactoring Netty Multipart support, to be discussed with Norman) Collaborate with Silvano Riz on a reactive multipart implementation that could be used independently of the engine used
          Hide
          sdeleuze Sébastien Deleuze added a comment - - edited

          I have rebased the commit on latest master.

          Since it seems we don't have a way to write file in a non blocking way, I think the best we can do is using NIO Mulitpart with small chunks of data in order to avoid blocking too much, and using zero copy where we can (already the case in the code I wrote).

          Rossen Stoyanchev Arjen Poutsma What is the best way we have to control the size of the body coming from the request to make sure we have for example 8K chunks? Do we already configure that at engine level?

          Show
          sdeleuze Sébastien Deleuze added a comment - - edited I have rebased the commit on latest master. Since it seems we don't have a way to write file in a non blocking way, I think the best we can do is using NIO Mulitpart with small chunks of data in order to avoid blocking too much, and using zero copy where we can (already the case in the code I wrote). Rossen Stoyanchev Arjen Poutsma What is the best way we have to control the size of the body coming from the request to make sure we have for example 8K chunks? Do we already configure that at engine level?
          Hide
          sdeleuze Sébastien Deleuze added a comment -

          I have pushed to master the support for Reactive multipart requests available via ServerWebExchange#getMultipartData(), @RequestParam / @RequestPart annotations or BodyExtractors#toMultipartData() functional API. Reader implementation relies on Synchronoss NIO multipart library. Client support for sending multipart request is also provided.

          A demo application is available at https://github.com/sdeleuze/webflux-multipart.

          Rossen Stoyanchev Feel free to review/polish this support when you have the time.

          Show
          sdeleuze Sébastien Deleuze added a comment - I have pushed to master the support for Reactive multipart requests available via ServerWebExchange#getMultipartData() , @RequestParam / @RequestPart annotations or BodyExtractors#toMultipartData() functional API. Reader implementation relies on Synchronoss NIO multipart library . Client support for sending multipart request is also provided. A demo application is available at https://github.com/sdeleuze/webflux-multipart . Rossen Stoyanchev Feel free to review/polish this support when you have the time.

            People

            • Assignee:
              sdeleuze Sébastien Deleuze
              Reporter:
              rstoya05-aop Rossen Stoyanchev
              Last updater:
              Juergen Hoeller
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since last comment:
                33 weeks, 2 days ago