Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-2810

partition jobs (jdbchdfs) are running in sequence

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: To Do
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Waiting for Triage
    • Component/s: Batch, Ingest
    • Labels:
      None
    • Story Points:
      5
    • Rank (Obsolete):
      9223372036854775807

      Description

      The jdbchdfs jobs that are partitioned are running in sequence rather than in parallel. Our expectation with partition jobs is that they run in parallel.

      Job configuration is:

      jdbchdfs --driverClassName='oracle.jdbc.OracleDriver' --url='jdbc:oracle:thin:@=*********' --username='=********' --password=********* --validationQuery='SELECT CURRENT_TIMESTAMP FROM DUAL' --tableName='HZ_ORGANIZATION_PROFILES' --columns='ORGANIZATION_PROFILE_ID, ..., VERSION_NUMBER' --partitions=10 --partitionColumn='ORGANIZATION_PROFILE_ID' --directory='/ingest/source/oracle11i/ar/hz_organization_profiles' --fileName=hz_organization_profiles --fileExtension=csv --delimiter=| --commitInterval=10000 --rollover=262144000 --dateFormat=yyyy-mmm-dd --partitionResultsTimeout=1800000 --testOnBorrow=false

        Attachments

          Activity

            People

            Assignee:
            mminella Michael Minella
            Reporter:
            bzeyben Buelent Zeyben
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: