Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-2728

Spark streaming processor module: Dispatcher has no subscribers

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Minor
    • Resolution: Complete
    • Affects Version/s: None
    • Fix Version/s: 1.1.1, 1.2 M1
    • Component/s: None
    • Labels:
      None

      Description

      The spark streaming processor module emits the following exception when there are more messages in the RDD partitions:

      015-02-17 12:45:02,026 1.2.0.SNAP ERROR Executor task launch worker-2 executor.Executor - Exception in task 0.0 in stage 56.0 (TID 142)
      org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'output'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
      at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:81)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:277)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:239)
      at org.springframework.xd.dirt.plugins.spark.streaming.MessageBusSender.send(MessageBusSender.java:105)
      at org.springframework.xd.spark.streaming.java.ModuleExecutor$1$1.call(ModuleExecutor.java:80)
      at org.springframework.xd.spark.streaming.java.ModuleExecutor$1$1.call(ModuleExecutor.java:55)
      at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:195)
      at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:195)
      at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:790)
      at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:790)
      at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1353)
      at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1353)
      at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
      at org.apache.spark.scheduler.Task.run(Task.scala:56)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:200)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
      at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:107)
      at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
      at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
      ... 17 more
      2015-02-17 12:45:02,032 1.2.0.SNAP WARN task-result-getter-1 scheduler.TaskSetManager - Lost task 0.0 in stage 56.0 (TID 142, localhost): org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'output'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
      at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:81)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:277)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:239)
      at org.springframework.xd.dirt.plugins.spark.streaming.MessageBusSender.send(MessageBusSender.java:105)
      at org.springframework.xd.spark.streaming.java.ModuleExecutor$1$1.call(ModuleExecutor.java:80)
      at org.springframework.xd.spark.streaming.java.ModuleExecutor$1$1.call(ModuleExecutor.java:55)
      at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:195)
      at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:195)
      at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:790)
      at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:790)
      at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1353)
      at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1353)
      at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
      at org.apache.spark.scheduler.Task.run(Task.scala:56)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:200)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
      at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:107)
      at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
      at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
      ... 17 more

        Attachments

          Activity

            People

            Assignee:
            iperumal Ilayaperumal Gopinathan
            Reporter:
            iperumal Ilayaperumal Gopinathan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: