Spring Framework
  1. Spring Framework
  2. SPR-7909

Sending large payloads with RestTemplate results in an OutOfMemoryError

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1 M1
    • Component/s: Web
    • Labels:
      None
    • Last commented by a User:
      true

      Description

      When sending a large file with the RestTemplate (with a POST or PUT), the RestTemplate will throw an OutOfMemoryError:

      com.cat.tcm.jmx.ManagementException: An unexpected error occurred performing the operation. 
      at com.cat.tcm.jmx.JMXTaskTemplate.run(JMXTaskTemplate.java:144) 
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
      at java.lang.Thread.run(Thread.java:619) 
      Caused by: java.lang.OutOfMemoryError: Java heap space 
      at java.util.Arrays.copyOf(Arrays.java:2786) 
      at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94) 
      at org.springframework.util.FileCopyUtils.copy(FileCopyUtils.java:113) 
      at org.springframework.http.converter.ResourceHttpMessageConverter.write(ResourceHttpMessageConverter.java:85) 
      at org.springframework.http.converter.ResourceHttpMessageConverter.write(ResourceHttpMessageConverter.java:1) 
      at org.springframework.http.converter.FormHttpMessageConverter.writePart(FormHttpMessageConverter.java:288) 
      at org.springframework.http.converter.FormHttpMessageConverter.writeParts(FormHttpMessageConverter.java:252) 
      at org.springframework.http.converter.FormHttpMessageConverter.writeMultipart(FormHttpMessageConverter.java:242) 
      at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:194) 
      at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:1) 
      at org.springframework.web.client.RestTemplate$HttpEntityRequestCallback.doWithRequest(RestTemplate.java:588) 
      at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:436) 
      at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:401) 
      at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:279) 
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
      at java.lang.reflect.Method.invoke(Method.java:597) 
      at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309) 
      at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) 
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) 
      at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:50) 
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) 
      at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) 
      at $Proxy17.postForObject(Unknown Source)
      

        Issue Links

          Activity

          Hide
          Arjen Poutsma added a comment -

          This is due to the fact that we buffer all content before sending it over the wire, which is necessary for setting the correct Content-Length header.

          Show
          Arjen Poutsma added a comment - This is due to the fact that we buffer all content before sending it over the wire, which is necessary for setting the correct Content-Length header.
          Hide
          Arjen Poutsma added a comment -

          Fixed, by adding a 'bufferRequestBody' property to the SimpleClientHttpRequestFactory, which by default is true.

          Setting this property to false will enable streaming mode. This will result in a ClientHttpRequest that either streams directly to the underlying HttpURLConnection (if the Content-Length is known in advance), or that will use "Chunked transfer encoding" (if the Content-Length is not known in advance). That is:

          SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
          requestFactory.setBufferRequestBody(false);
          RestTemplate rest = new RestTemplate(requestFactory);
          

          These changes will be in 3.0.6, and in 3.1 M1. Please try a recent 3.1 M1 (as of tonight) to see if it works for you.

          Show
          Arjen Poutsma added a comment - Fixed, by adding a 'bufferRequestBody' property to the SimpleClientHttpRequestFactory, which by default is true. Setting this property to false will enable streaming mode. This will result in a ClientHttpRequest that either streams directly to the underlying HttpURLConnection (if the Content-Length is known in advance), or that will use "Chunked transfer encoding" (if the Content-Length is not known in advance). That is: SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory(); requestFactory.setBufferRequestBody( false ); RestTemplate rest = new RestTemplate(requestFactory); These changes will be in 3.0.6, and in 3.1 M1. Please try a recent 3.1 M1 (as of tonight) to see if it works for you.
          Hide
          Juergen Hoeller added a comment -

          I've removed this from the 3.0.6 backport list since it really seems to be a new 3.1 feature to me. Unless there is a strong need to have this in 3.0.x, I'd rather recommend an upgrade to 3.1 instead.

          Juergen

          Show
          Juergen Hoeller added a comment - I've removed this from the 3.0.6 backport list since it really seems to be a new 3.1 feature to me. Unless there is a strong need to have this in 3.0.x, I'd rather recommend an upgrade to 3.1 instead. Juergen
          Hide
          Serge P. Nekoval added a comment -

          Any chance to get it fixed for HttpComponentsClientHttpRequestFactory?

          Show
          Serge P. Nekoval added a comment - Any chance to get it fixed for HttpComponentsClientHttpRequestFactory?

            People

            • Assignee:
              Arjen Poutsma
              Reporter:
              Arjen Poutsma
              Last updater:
              Serge P. Nekoval
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since last comment:
                1 year, 20 weeks, 3 days ago

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0d
                0d
                Logged:
                Time Spent - 2d
                2d