Uploaded image for project: 'Spring Batch'
  1. Spring Batch
  2. BATCH-2309

Partitioner data lose - SqlPagingQueryProviderFactoryBean

    Details

      Description

      I created a project based on "Reference URL" mentioned above that seems to lose data. The configuration where the problem occurs is listed below:

      grid-size=10
      commit-interval=100
      pageSize=1000

      maxActive=11 (org.apache.commons.dbcp.BasicDataSource)
      maxIdle=11 (org.apache.commons.dbcp.BasicDataSource)

      My source table (oracle database 11g) has 10387 records, but when the job process, it only takes 10117 records to the target database (also oracle 11g).

      To conclude, that are 270 records that weren't integrated.

        Activity

        Hide
        mminella Michael Minella added a comment -

        Can you please provide your configuration or a sample project that illustrates the issue?

        Show
        mminella Michael Minella added a comment - Can you please provide your configuration or a sample project that illustrates the issue?
        Hide
        victorjabur Victor Jabur added a comment -

        Sample project is reference url, a sample from spring github

        Show
        victorjabur Victor Jabur added a comment - Sample project is reference url, a sample from spring github
        Hide
        mminella Michael Minella added a comment -

        The reference URL points to our sample. Not the one with your configuration.

        Show
        mminella Michael Minella added a comment - The reference URL points to our sample. Not the one with your configuration.
        Hide
        victorjabur Victor Jabur added a comment -

        I can try prepare a full example with database scripts and reproduce the error with this poc

        Show
        victorjabur Victor Jabur added a comment - I can try prepare a full example with database scripts and reproduce the error with this poc
        Hide
        victorjabur Victor Jabur added a comment -

        I will prepare tomorrow, i am at home now. Thanks

        Show
        victorjabur Victor Jabur added a comment - I will prepare tomorrow, i am at home now. Thanks
        Hide
        victorjabur Victor Jabur added a comment -

        This is not a bug, i was using the partition feature incorrectly. I create a post on my blog to explain my initial problem and help other people with the same problem. Resuming: I was using SqlPagingReader with Rownum to resolve the problem to divide same quantity of records to each thread. But the processing of all records is not guaranteed.

        This is the post: http://wp.me/pVXBl-8R

        Thank you for very fast treatment here in the Jira.

        Victor Jabur

        Show
        victorjabur Victor Jabur added a comment - This is not a bug, i was using the partition feature incorrectly. I create a post on my blog to explain my initial problem and help other people with the same problem. Resuming: I was using SqlPagingReader with Rownum to resolve the problem to divide same quantity of records to each thread. But the processing of all records is not guaranteed. This is the post: http://wp.me/pVXBl-8R Thank you for very fast treatment here in the Jira. Victor Jabur
        Hide
        mminella Michael Minella added a comment -

        Thanks for the update!

        Show
        mminella Michael Minella added a comment - Thanks for the update!

          People

          • Assignee:
            mminella Michael Minella
            Reporter:
            victorjabur Victor Jabur
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: