Uploaded image for project: 'Spring Data Redis'
  1. Spring Data Redis
  2. DATAREDIS-954

Improve thread management for Redis streams


    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 2.2 M2 (Moore)
    • Fix Version/s: None
    • Component/s: Core



      We at Intuit are working on a project and looking to use Redis Streams. I found that Spring Data Redis provides good support for Redis Streams in it's new upcoming version of 2.2.0. 

      So I worked on a POC to check out 2.2.0.M1 and saw that it has a good feature to have a onMessage functionality for streams. Here is a link to POC. https://github.com/akaskj/redispoc

      However I noticed that a new thread is created and put to waiting stage so that it can poll the stream for every subscription we make to a stream (Attached screenshots). This will not be scalable with our use case as we may be listening to thousands of streams at a given time. So we cannot have that many treads at a given time. Here is the link to StreamPollTask

      My proposal is to have a better threading design to poll streams where we can have limited number(configurable) of threads for polling and we distribute the task of polling among them.

      Could you please let me know if this is something I can make changes and contribute back to Spring-data-redis?

      Or was there is a use case that you had in mind for the design that is done currently?

      Or am I doing anything wrong?

      Anyway please let me know and I can send you more details on the design.


      Thank you,






            • Assignee:
              mp911de Mark Paluch
              ajayachandra Akash Kengunte Jayachandra
              Last updater:
              Mark Paluch
            • Votes:
              0 Vote for this issue
              2 Start watching this issue


              • Created: