Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-897

The HDFS Sink should support copying File payloads

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 1.x
    • Component/s: Stream Module
    • Labels:
      None

      Description

      We should support java.io.file payloads in order to support non-textual file and large text file payloads being uploaded to HDFS.

      Currently text file payloads are converted to a text stream in memory and, non-String payloads are converted to JSON first, using an "object-to-json-transformer".

      Ultimately we need to support streams such as "file | hdfs" where the actually payload being copied to HDFS is not necessarily JSON or textual.

      Need to be able to support headers in the message that will indicate which HDFS file the data should be stored in.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              liujiong liujiong
              Reporter:
              hillert Gunnar Hillert
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: