Spring Integration
  1. Spring Integration
  2. INT-1500

Splitter should not always produce sequenceNumbers on messages

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Complete
    • Affects Version/s: None
    • Fix Version/s: 2.0 RC1
    • Component/s: None
    • Labels:
      None

      Description

      In the sample Kitchen application Iwein and I wrote we first split, do a custom aggregation and then split again, and aggregate the messages the way they were split first. This confuses the sequenceNumbering scheme that we thus need to reset.

      See: http://bit.ly/si-kitchen; file: src/main/resources/grocery-unpacker-flow.xml

        Activity

        Hide
        Mark Fisher added a comment -

        Dave, assigning to you since you recently added the SequenceInfo changes. I am wondering if we need to add an "apply-sequence" boolean flag like what we have on routers and publish-subscribe-channels. I guess the value would be TRUE by default on splitters (unlike those others), but at least it would provide a way to turn it off.

        Or... Dave, do you have a better idea?

        Thanks,
        Mark

        Show
        Mark Fisher added a comment - Dave, assigning to you since you recently added the SequenceInfo changes. I am wondering if we need to add an "apply-sequence" boolean flag like what we have on routers and publish-subscribe-channels. I guess the value would be TRUE by default on splitters (unlike those others), but at least it would provide a way to turn it off. Or... Dave, do you have a better idea? Thanks, Mark
        Hide
        Dave Syer added a comment -

        I'm not sure I understand the problem yet. If a splitter doesn't add sequence numbers, how will you aggregate downstream? Or is it the case that you don't aggregate downstream? In the kitchen sample you quoted, you just set the $sequenceNumber to 0 on all messages. How does that help?

        Show
        Dave Syer added a comment - I'm not sure I understand the problem yet. If a splitter doesn't add sequence numbers, how will you aggregate downstream? Or is it the case that you don't aggregate downstream? In the kitchen sample you quoted, you just set the $sequenceNumber to 0 on all messages. How does that help?
        Hide
        Jeroen van Erp added a comment -

        Hi Dave,

        In the kitchen sample, we start by splitting a Recipe in Ingredients, then we Aggregate these into shoppinglists (custom aggregation). Then we do some work and convert a ShoppingList into a GroceryBag, which is split into products. The products are then aggregated into a Meal again. Let's give a sample:

        Recipe: 1 steak, 20 grams pepper
        --> Split into Message[1steak]

        {seqN:1}

        & Message[20grPepper]

        {seqN:2}

        --> Aggrgated onto 2 ShoppingLists (2 different types of ingredients): Message[ShoppingList<Meat>[1Steak]] & Message[ShoppingList<Grocery>[20grPepper]]
        --> Work to change these to: Message[GroceryBag<Meat>[1Steak]] & Message[GroceryBag<Grocery>[20grPepper]]
        --> GroceryBag<Meat> split into Message[1Steak]

        {seqN:1}

        --> GroceryBag<Grocery> split into Message[20grPepper]

        {seqN:1}

        --> Steak and Pepper need to be aggregated into a Meal, but the SimpleMessageStore decides that the pepper message is already present because it has
        the same sequence number as the steak.

        You will always get this if you do a custom aggregation and split, because the splitter assigns new sequencenumbers to the message.

        Regards,
        Jeroen

        Show
        Jeroen van Erp added a comment - Hi Dave, In the kitchen sample, we start by splitting a Recipe in Ingredients, then we Aggregate these into shoppinglists (custom aggregation). Then we do some work and convert a ShoppingList into a GroceryBag, which is split into products. The products are then aggregated into a Meal again. Let's give a sample: Recipe: 1 steak, 20 grams pepper --> Split into Message [1steak] {seqN:1} & Message [20grPepper] {seqN:2} --> Aggrgated onto 2 ShoppingLists (2 different types of ingredients): Message[ShoppingList<Meat> [1Steak] ] & Message[ShoppingList<Grocery> [20grPepper] ] --> Work to change these to: Message[GroceryBag<Meat> [1Steak] ] & Message[GroceryBag<Grocery> [20grPepper] ] --> GroceryBag<Meat> split into Message [1Steak] {seqN:1} --> GroceryBag<Grocery> split into Message [20grPepper] {seqN:1} --> Steak and Pepper need to be aggregated into a Meal, but the SimpleMessageStore decides that the pepper message is already present because it has the same sequence number as the steak. You will always get this if you do a custom aggregation and split, because the splitter assigns new sequencenumbers to the message. Regards, Jeroen
        Hide
        Mark Fisher added a comment -

        Iwein, I'm assigning this to you, since I think you're most familiar with the situation.

        Show
        Mark Fisher added a comment - Iwein, I'm assigning this to you, since I think you're most familiar with the situation.
        Hide
        Iwein Fuld added a comment -

        This is an example where custom release and correlation are used. In other words, we don't care about sequence numbering, but because the second splitter does apply the sequence the MessageStore gets confused... It's gonna be tricky to reproduce this in a sensible test case

        Show
        Iwein Fuld added a comment - This is an example where custom release and correlation are used. In other words, we don't care about sequence numbering, but because the second splitter does apply the sequence the MessageStore gets confused... It's gonna be tricky to reproduce this in a sensible test case
        Hide
        Mark Fisher added a comment -

        Would it be sufficient if we add "apply-sequence" on splitter? We have that attribute on routers and publish-subscribe-channels. The only difference would be the default value being TRUE for splitter (while it's FALSE for those others).

        Show
        Mark Fisher added a comment - Would it be sufficient if we add "apply-sequence" on splitter? We have that attribute on routers and publish-subscribe-channels. The only difference would be the default value being TRUE for splitter (while it's FALSE for those others).
        Hide
        Iwein Fuld added a comment -

        I've added the apply sequence flag (default to true). This covers all use cases with a custom aggregation that collides with the default sequence headers and correlation id.

        If a custom aggregation needs some of sequence size, sequence number or correlation id, the user must either null out the headers he doesn't want or enrich the headers he does want.

        Show
        Iwein Fuld added a comment - I've added the apply sequence flag (default to true). This covers all use cases with a custom aggregation that collides with the default sequence headers and correlation id. If a custom aggregation needs some of sequence size, sequence number or correlation id, the user must either null out the headers he doesn't want or enrich the headers he does want.

          People

          • Assignee:
            Iwein Fuld
            Reporter:
            Jeroen van Erp
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: