Uploaded image for project: 'Spring for Apache Hadoop'
  1. Spring for Apache Hadoop
  2. SHDP-575

Referenced 'Configuration' (<hadoop:configuration>) is not applied to JarTasklet (<hadoop:jar-tasklet>) in bean definitions xml

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.4.0 GA
    • Fix Version/s: None
    • Component/s: Batch, DI
    • Environment:
      Ubuntu 16.10 amd64; Java 8; Reproduced in Spring MVC application running in Tomcat 8

      Description

      Defining with xml the following:
      ```
      <hadoop:configuration id="myConfig">
      ...
      test.id=test.value
      ...
      </hadoop:configuration>
      ```
      and later referencing it from a jar tasklet :
      ```
      <hadoop:jar-tasklet
      id="myJarTasklet"
      configuration-ref="myConfig"
      main-class="some.hadoop.job.MyHadoopJob"
      >
      ...
      </hadoop:jar-tasklet>
      ```
      The MyHadoopJob class would create its configuration in the main and run the job (as usually jobs would, in this particular case it would be org.apache.nutch.crawl.InjectorJob).
      The referenced Configuration (myConfig) has no chance of being applied, and it is not used in the actual execution. Looking at the source code it confirms it as well:
      JarExecutor.class
      ```
      protected Object invokeTargetObject(Configuration cfg, Object target, Class<Object> targetClass, String[] args)
      throws Exception {
      Method main = ReflectionUtils.findMethod(targetClass, "main", String[].class);
      return ReflectionUtils.invokeMethod(main, null, new Object[]

      { args }

      );
      }
      ```

        Attachments

          Activity

            People

            Assignee:
            thomas.risberg Thomas Risberg
            Reporter:
            raritzu Arthur B
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:

                Time Tracking

                Estimated:
                Original Estimate - 5d
                5d
                Remaining:
                Remaining Estimate - 5d
                5d
                Logged:
                Time Spent - Not Specified
                Not Specified