Uploaded image for project: 'Spring for Apache Hadoop'
  1. Spring for Apache Hadoop
  2. SHDP-498

Support TEZ execution type for pig-factory

    XMLWordPrintable

    Details

      Description

      Right now (according to hadoop.xsd) execution type for "pig-factory" can be either LOCAL or MAPREDUCE. So, XML file could look like the following

      <?xml version="1.0" encoding="UTF-8"?>
      <beans:beans
      xmlns="http://www.springframework.org/schema/hadoop"
      xmlns:beans="http://www.springframework.org/schema/beans"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="...">
      ...
      <pig-factory exec-type="MAPREDUCE"/>
      ...
      </beans:beans>

      The Apache community has been released Apache Pig 0.14.0, and the main feature was Pig on Tez. Apache Tez is an alternative execution engine focusing on performance. It offers a more flexible interface so Pig can compile into a better execution plan than is possible with MapReduce.

      Could you please add support of TEZ as possible execution type in the definition of "pig-factory"?

        Attachments

          Activity

            People

            Assignee:
            thomas.risberg Thomas Risberg
            Reporter:
            disciple97 Andrei Yegorov
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: