Spring Batch
  1. Spring Batch
  2. BATCH-1531

Restart of a partitioned Step should not call Partitioner

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Complete
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.1, 2.1.3
    • Component/s: None
    • Labels:
      None
    • Environment:
      SpingBatch V2.1

      Description

      For a PartionStep the Partitioner is called on every restart of the failed or stopped step.
      This should only be done if the step is executed the first time or if the flag allowStartIfComplete=true.

      The reasons for this change:
      1) Partitioning is a very time and resource consuming process in our jobs. Those resouces should not be wasted
      2) In our environment the system is not exclusively reserved for the job. Other business may happen and modify the data relevant for the selection of the partition blocks. As a result the data assigned to the blocks may differ. But as only failed steps will be repeated the repeated partioned step might work on other data then expected.

      All information to repeat a failed partition step is already stored in the repository. Therefore the partitioning is not necessary but increases the risk of data missmatch

        Activity

        Hide
        Stéphane Nicoll added a comment -

        We're using a different convention indeed. I will create the issue right the way

        Show
        Stéphane Nicoll added a comment - We're using a different convention indeed. I will create the issue right the way
        Hide
        Stéphane Nicoll added a comment -
        Show
        Stéphane Nicoll added a comment - See BATCH-1616
        Hide
        Dave Syer added a comment -

        Re-opened and re-resolved. the default behaviour changed again in 2.1.3: if you don't want the framework to call your Partitioner on restart, it needs to implement a new interface PartitionNameProvider.

        Show
        Dave Syer added a comment - Re-opened and re-resolved. the default behaviour changed again in 2.1.3: if you don't want the framework to call your Partitioner on restart, it needs to implement a new interface PartitionNameProvider.
        Hide
        Stéphane Nicoll added a comment -

        Dave, wouldn't that make sense to explain the rationale behind this change in the documentation and/or the Javadoc of the PartitionNameProvider?

        Show
        Stéphane Nicoll added a comment - Dave, wouldn't that make sense to explain the rationale behind this change in the documentation and/or the Javadoc of the PartitionNameProvider?
        Hide
        Dave Syer added a comment -

        Good point. Done.

        Show
        Dave Syer added a comment - Good point. Done.

          People

          • Assignee:
            Dave Syer
            Reporter:
            Harald Groß
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 0.25d
              0.25d
              Remaining:
              Remaining Estimate - 0d
              0d
              Logged:
              Time Spent - 1h Time Not Required
              1h