Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-3136

Example hashtag-count MR job fails when running XD on YARN with PHD 3.0

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Minor
    • Resolution: Complete
    • Affects Version/s: None
    • Fix Version/s: 1.2 GA
    • Component/s: Hadoop, Runtime
    • Labels:
      None
    • Story Points:
      5
    • Rank (Obsolete):
      9223372036854775807
    • Sprint:
      Sprint 51

      Description

      Running XD on YARN on PHD 3.0 Ambari install.

      Uploading and submitting a custom job fails with the following:

      2015-06-02 16:54:15,580 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1433273561345_0009_m_000000_0: Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.springframework.xd.examples.hadoop.HashtagCount$TokenizerMapper not found
      	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2076)
      	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
      	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:742)
      	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
      	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      Caused by: java.lang.ClassNotFoundException: Class org.springframework.xd.examples.hadoop.HashtagCount$TokenizerMapper not found
      	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1982)
      	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2074)
      	... 8 more
      

      Same example jar works fine when submitted from XD cluster.

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            thomas.risberg Thomas Risberg
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: