Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-3025

SpringXD sqoop module is hanging

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Major
    • Resolution: Complete
    • Affects Version/s: 1.2 M1
    • Fix Version/s: 1.2 RC1
    • Component/s: None
    • Labels:
      None

      Description

      The SpringXD Sqoop module is in execution status until it times out, it is hanging.

      The container logs show:

      2015-05-04 15:15:45,365 1.2.0.M1 INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying job 'sqoop_lookup'
      2015-05-04 15:15:45,536 1.2.0.M1 INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying module [ModuleDescriptor@239b037a moduleName = 'sqoop', moduleLabel = 'sqoop', group = 'sqoop_lookup', sourceChannelName = [null], sinkChannelName = [null], index = 0, type = job, parameters = map['args' -> '--connect=jdbc:oracle:thin:@************:****/******* —username=******** --password-file=/user/zeybeb/workspace/secure-files/gdw.password --table=MASTERDATA.W_LOOKUP_D --target-dir=/user/zeybeb/workspace/ent/masterdata_src/lookup_d -m 1', 'command' -> 'import'], children = list[[empty]]]
      2015-05-04 15:16:17,061 1.2.0.M1 INFO inbound.job:sqoop_lookup-redis:queue-inbound-channel-adapter1 sqoop.SqoopTasklet - Sqoop system.out: /tmp/Sqoop-948322291323951735.out

      The /tmp/Sqoop-948322291323951735.out file content is:

      15:16:17,612 INFO main sqoop.SqoopRunner - Sqoop command: import
      15:16:17,613 INFO main sqoop.SqoopRunner - Using args: [--connect=jdbc:oracle:thin:@************:****/*******, —username=*********, --password-file=/user/zeybeb/workspace/secure-files/gdw.password, --table=MASTERDATA.W_LOOKUP_D, --target-dir=/user/zeybeb/workspace/ent/masterdata_src/lookup_d, -m, 1]
      15:16:17,613 INFO main sqoop.SqoopRunner - Mapreduce home: /opt/pivotal/spring-xd-1.2.0.M1/xd/lib/phd21
      15:16:17,631 INFO main sqoop.SqoopRunner - Setting configuration property: fs.defaultFS=hdfs://ilabphd07.isus.emc.com:8020
      15:16:17,753 INFO main sqoop.SqoopRunner - Setting configuration property: yarn.resourcemanager.address=ilabphd08.isus.emc.com:8050
      15:16:17,753 INFO main sqoop.SqoopRunner - Setting configuration property: yarn.application.classpath=/usr/hdp/2.2.0.0-2041/etc/hadoop/conf.empty,/usr/hdp/2.2.0.0-2041/hadoop/,/usr/hdp/2.2.0.0-2041/hadoop/lib/,/usr/hdp/2.2.0.0-2041/hadoop-hdfs/,/usr/hdp/2.2.0.0-2041/hadoop-hdfs/lib/,/usr/hdp/2.2.0.0-2041/hadoop-yarn/,/usr/hdp/2.2.0.0-2041/hadoop-yarn/lib/,/usr/hdp/2.2.0.0-2041/hadoop-mapreduce/,/usr/hdp/2.2.0.0-2041/hadoop-mapreduce/lib/,/usr/hdp/2.2.0.0-2041/sqoop/,/usr/hdp/2.2.0.0-2041/sqoop/lib/,/usr/hdp/2.2.0.0-2041/flume/,/usr/hdp/2.2.0.0-2041/flume/lib/,/usr/hdp/2.2.0.0-2041/storm/,/usr/hdp/2.2.0.0-2041/storm/lib/
      15:16:17,754 INFO main sqoop.SqoopRunner - Setting configuration property: mapreduce.framework.name=yarn
      15:16:17,837 WARN main tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
      15:16:17,907 INFO main sqoop.Sqoop - Running Sqoop version: 1.4.5
      15:16:18,282 WARN main util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      15:16:19,552 WARN main sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
      15:16:19,657 INFO main oracle.OraOopManagerFactory - Data Connector for Oracle and Hadoop is disabled.
      15:16:19,673 INFO main manager.SqlManager - Using default fetchSize of 1000
      15:16:19,673 INFO main tool.CodeGenTool - Beginning code generation
      15:16:20,639 INFO main manager.OracleManager - Time zone has been set to GMT
      15:16:20,853 INFO main manager.SqlManager - Executing SQL statement: SELECT t.* FROM MASTERDATA.W_LOOKUP_D t WHERE 1=0
      15:16:21,018 INFO main orm.CompilationManager - HADOOP_MAPRED_HOME is /opt/pivotal/spring-xd-1.2.0.M1/xd/lib/phd21
      15:16:23,171 INFO main orm.CompilationManager - Writing jar file: /tmp/sqoop-spring-xd/compile/4e11123a52fa36d6677efdb47bcdc43b/MASTERDATA.W_LOOKUP_D.jar
      15:16:23,191 INFO main manager.OracleManager - Time zone has been set to GMT
      15:16:24,109 INFO main manager.OracleManager - Time zone has been set to GMT
      15:16:24,825 INFO main mapreduce.ImportJobBase - Beginning import of MASTERDATA.W_LOOKUP_D
      15:16:24,848 INFO main manager.OracleManager - Time zone has been set to GMT
      15:16:24,876 WARN main mapreduce.JobBase - SQOOP_HOME is unset. May not be able to find all job dependencies.
      15:16:25,083 INFO main client.RMProxy - Connecting to ResourceManager at ilabphd08.isus.emc.com/10.15.232.191:8050
      15:16:25,977 INFO main db.DBInputFormat - Using read commited transaction isolation
      15:16:26,117 INFO main mapreduce.JobSubmitter - number of splits:1
      15:16:26,361 INFO main mapreduce.JobSubmitter - Submitting tokens for job: job_1429280992648_0019
      15:16:26,717 INFO main impl.YarnClientImpl - Submitted application application_1429280992648_0019 to ResourceManager at ilabphd08.isus.emc.com/10.15.232.191:8050
      15:16:26,782 INFO main mapreduce.Job - The url to track the job: http://http://ilabphd08.isus.emc.com:8088/proxy/application_1429280992648_0019/
      15:16:26,783 INFO main mapreduce.Job - Running job: job_1429280992648_0019

      The logs on the Hadoop side are:

      Showing 4096 bytes. Click here for full log
      mumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:04,026 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:05,033 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:06,042 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:07,049 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:08,056 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:09,062 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:10,069 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:41,093 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:42,100 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:43,107 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:44,113 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:45,120 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:46,129 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:47,136 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:48,143 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:49,150 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      2015-05-04 15:36:50,156 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

      Even though the maxRetries is set to 10 the process is going into several sets of retrials.

        Attachments

          Activity

            People

            Assignee:
            thomas.risberg Thomas Risberg
            Reporter:
            bzeyben Buelent Zeyben
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: