Uploaded image for project: 'Spring for Apache Hadoop'
  1. Spring for Apache Hadoop
  2. SHDP-554

Missing ability to configure multiple mappers in a single job (MultipleInputs)

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: General Backlog 2.x
    • Component/s: MR
    • Labels:
      None

      Description

      Hi,

      It seems there is no support for configuring hdp:job with multiple mappers.
      Going over JobFactoryBean.java I didn't observe any support for that.

      There are cases where one job contains multiple mappers attached to single reducer. Usually it is done using hadoop MultipleInputs class when configuring a job using java.

      This would be great to have this configuration ability available also from spring. It would be extremely beneficial also to spring-hadoop test module.

        Attachments

          Activity

            People

            Assignee:
            thomas.risberg Thomas Risberg
            Reporter:
            noam.asor Noam Asor
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: