Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-3562

Be aware of exceptions thrown in a stream



    • Type: Story
    • Status: To Do
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.2 GA
    • Fix Version/s: None
    • Component/s: Runtime, Stream Module
    • Labels:
    • Story Points:
    • Rank (Obsolete):


      Currently, if a module in a stream throws an exception, the stream flow is broken, but the stream's creator is not notified or warned about the issue, unless a direct monitoring of the logs is performed.

      This could work when an administrator is taking care of the whole life cycle of any stream, but it doesn't work when Spring XD is being used by external services with no human intervention.

      It would be necessary some sort of notification mechanism so when any issue raises in the execution of a stream, its creator can react and decide on how to handle it.

      For instance, I have an application that uses Spring XD to load data from different sources into HDFS. In my application, you simply define a type of source and the destiny in HDFS, without the user knowing that Spring XD is under the hoods. If the user defines, for example, a Twitter source, then I define a "twitterstream" as a source for a new stream with a sink in HDFS. However, if anything goes wrong, I have no way to let my user know about the issue: an authentication issue, the dreaded 420 response from the Twitter API, etc. If I could somehow be notified about the issue, then I could provide that information to my user so she could fix the issue or at least understand what is going on.

      I've seen that the REST API will support an endpoint to check the current status of a stream, but it is not yet available and it would require my application to be polling constantly the stream, with all the disadvantages compared to a pushing mechanism.




            Unassigned Unassigned
            emedina Enrique Medina
            0 Vote for this issue
            1 Start watching this issue