Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      For a spring-boot 2.0.6 application using spring-data-redis 2.0.11 (via spring-boot-starter-data-redis:2.0.6.RELEASE), the actuator health check will automatically use org.springframework.data.redis.connection.lettuce.LettuceReactiveRedisConnection. Once the health check endpoint has been hit, if you shutdown the application, you get the following leak warning logging from the embedded tomcat server:

      2018-11-23 13:25:11.801  WARN 10454 --- [ost-startStop-2] o.a.c.loader.WebappClassLoaderBase       : The web application [ROOT] appears to have started a thread named [elastic-evictor-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
      ...
      

      For this logging to be output to console, you must be using spring-cloud-config. If you're not, the logging subsystem shuts down and removes the slf4j jul handler before the tomcat leak detector code runs and logs the warning. If you use a debug breakpoint, you can tell that the leak still gets detected, it just doesn't get logged. Something to do with the parent context used when spring-cloud-config is present prevents the logging system from removing the handler, so the warning gets logged to the console. Update; even with spring-cloud-config present, it's a crapshoot whether you see this logging or not.

      I believe this is due to the use of Schedulers.elastic() in org.springframework.data.redis.connection.lettuce.LettuceReactiveRedisConnection.AsyncConnect.AsyncConnect(LettuceConnectionProvider)
      I don't see any call to Schedulers.shutdownNow(). If I create a simple DisposableBean and call Schedulers.shutdownNow() the warning disappears. Would it be the responsibility of spring-data-redis to call that in cases where Schedulers.elastic() has been called (when a reactive connection has been created)?

      simple reproduction repo: https://github.com/gadamsciv/spring-data-redis-leak

      This project doesn't use spring-cloud-config, since even with that the logging is only output under certain circumstances. To reproduce, run `docker-compose up` to start redis locally, then use the IDE of your choice to import and run the cache-api gradle project as a spring-boot project. Place a breakpoint at org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads() line 1687, which is where, if logging worked correctly, tomcat would log a message like

      The web application [{0}] appears to have started a thread named [{1}] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:{2}
      

      Once cache-api has started up, visit http://localhost:8080/actuator/health in a browser. This initializes a reactive redis connection using lettuce. Then cleanly shutdown cache-api in your IDE (SIGINT not SIGKILL). You should hit the WebappClassLoaderBase.clearReferencesThreads breakpoint you set up above.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mp911de Mark Paluch
                Reporter:
                gadamsciv gadamsciv
                Last updater:
                Mark Paluch
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: