Uploaded image for project: 'Spring AMQP'
  1. Spring AMQP
  2. AMQP-748

Consider increasing default prefetch for SimpleListenerContainer

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Complete
    • Affects Version/s: 1.7.3
    • Fix Version/s: 2.0 RC2
    • Component/s: RabbitMQ
    • Labels:

      Description

      With the help of Artem Bilan I've discovered that SimpleListenerContainer uses prefetch value of 1 by default. This is a very conservative value that will affect throughput of relatively efficient consumers (or cases where processing a delivery doesn't take much time).

      Team RabbitMQ published a blog post that includes some numbers that demonstrate how prefetch translates to increased consumer utilization: https://www.rabbitmq.com/blog/2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/.
      Of course, this only works up to a point.

      What's the risk of increasing this value? With consumers that cannot keep up, a certain number of deliveries will sit there waiting processing in TCP buffers and a j.u.c. queue in the Java client. Unless messages are really large or a high (say, 1000 or more) prefetch values is used, those messages should not add up to a dangerous amount of memory (famous last words, I know).

      Would you consider using channel QoS prefetch value of, say, 50 by default?
      I'm happy to assist with finding an optimal default value (if such thing exists given how much
      delivery handling can vary from app to app) and to provide (to a Pivotal staff member) a real world escalation where the default contributed to confusion about why the system wasn't consuming as efficiently as the user expected.

        Attachments

          Activity

            People

            • Assignee:
              abilan Artem Bilan
              Reporter:
              michaelklishin_pivotal Michael Klishin
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: