Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-2824

hdfs sink loses messages/data when container killed

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Hadoop, Stream Module
    • Labels:
      None
    • Story Points:
      5
    • Rank (Obsolete):
      9223372036854775807

      Description

      Scenario running a "rabbit | hdfs" stream and killing the xd-container while stream is running.

      Looks like the messages get's acked before the data is flushed to hdfs.

      This results in some data lost due to data either in tmp file or cached in the dfs client.

      Reference: VESC-387

        Attachments

          Activity

            People

            Assignee:
            jvalkeal Janne Valkealahti
            Reporter:
            thomas.risberg Thomas Risberg
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: