Spring Batch
  1. Spring Batch
  2. BATCH-414

consistent handling of empty input by ItemReaders

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0.m5
    • Fix Version/s: 1.1.0
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      In many cases with ItemReaders it's possible to call open, but have nothing to process. This could be because a file was empty or a query returned no results. This should throw a 'NoWorkFoundException' Throwing a specific exception allows developers to make a domain decision about whether or not finding no work means failure or not. in the majority of cases it will mean failure, however, in a few it might be okay.

        Activity

        Hide
        Robert Kasanicky added a comment -

        I think it makes better sense to handle this concern at the step level?

        Show
        Robert Kasanicky added a comment - I think it makes better sense to handle this concern at the step level?
        Hide
        Robert Kasanicky added a comment -

        Moving to 1.1 - this is not a bug and would change the behavior in a notable way

        Show
        Robert Kasanicky added a comment - Moving to 1.1 - this is not a bug and would change the behavior in a notable way
        Hide
        Lucas Ward added a comment -

        There's been a considerable ammount of noise around this issue, so I'm assigning to myself, and I've given an initial estimate of 3 days. It may potentially need to be handled at the step level, but I'm not quite sure yet. I still feel like it's easy enough to ignore, but I understand the point others have made.

        Show
        Lucas Ward added a comment - There's been a considerable ammount of noise around this issue, so I'm assigning to myself, and I've given an initial estimate of 3 days. It may potentially need to be handled at the step level, but I'm not quite sure yet. I still feel like it's easy enough to ignore, but I understand the point others have made.
        Hide
        Robert Kasanicky added a comment -

        How about using StepExecutionListener.afterStep to check the itemCount value? It is not the same as detecting empty input, but could be good enough. There is also an option to use separate tasklet step to check the input before processing it in the following step. Or even use both maybe.

        Show
        Robert Kasanicky added a comment - How about using StepExecutionListener.afterStep to check the itemCount value? It is not the same as detecting empty input, but could be good enough. There is also an option to use separate tasklet step to check the input before processing it in the following step. Or even use both maybe.
        Hide
        Lucas Ward added a comment -

        There's actually multiple parts to this issue, one, as I think you pointed out Robert was whether or not we should default to a step failure if there is no work found to process. Should we always throw an exception, or should we not an assume anyone that wants to catch it will handle it themselves? Checking the item count seems possible, but it feels like a work around. It seems like it would be easier to ignore an exception than it would be to try and detect that nothing was done.

        The other part of the problem is that, either way, all of the ItemReader implementations should behave the same way, it should be documented in the java docs as part of the contract. I'm afraid some of the readers would probably fail in open if there was no work found. For example, if a file that is configured to be read by a FlatFileItemReader doesn't exist, is that considered 'no work' or a failure? What if the file is empty? Should we dictate that the open method not throw an exception in these scenarios and that the reader simply return null on the first call? That seems somewhat brittle as well. It still seems easier to me to throw a no work found exception and have one point to say 'NoWorkFoundException means the job was successful' via configuration.

        Show
        Lucas Ward added a comment - There's actually multiple parts to this issue, one, as I think you pointed out Robert was whether or not we should default to a step failure if there is no work found to process. Should we always throw an exception, or should we not an assume anyone that wants to catch it will handle it themselves? Checking the item count seems possible, but it feels like a work around. It seems like it would be easier to ignore an exception than it would be to try and detect that nothing was done. The other part of the problem is that, either way, all of the ItemReader implementations should behave the same way, it should be documented in the java docs as part of the contract. I'm afraid some of the readers would probably fail in open if there was no work found. For example, if a file that is configured to be read by a FlatFileItemReader doesn't exist, is that considered 'no work' or a failure? What if the file is empty? Should we dictate that the open method not throw an exception in these scenarios and that the reader simply return null on the first call? That seems somewhat brittle as well. It still seems easier to me to throw a no work found exception and have one point to say 'NoWorkFoundException means the job was successful' via configuration.
        Hide
        Robert Kasanicky added a comment -

        The consensus so far is to accept empty input in the ItemReaders (the DrivingQueryItemReader will need to be adjusted) and provide optional way to fail the step if no work has been done (e.g. StepListener that checks itemCount == 0).

        Show
        Robert Kasanicky added a comment - The consensus so far is to accept empty input in the ItemReaders (the DrivingQueryItemReader will need to be adjusted) and provide optional way to fail the step if no work has been done (e.g. StepListener that checks itemCount == 0).
        Hide
        Dave Syer added a comment -

        Assume closed as resolved and released

        Show
        Dave Syer added a comment - Assume closed as resolved and released

          People

          • Assignee:
            Robert Kasanicky
            Reporter:
            Lucas Ward
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 3d
              3d
              Remaining:
              Remaining Estimate - 0d
              0d
              Logged:
              Time Spent - 1d 5h Time Not Required
              1d 5h