Uploaded image for project: 'Spring XD'
  1. Spring XD
  2. XD-2602

trying to import data from MS SQL tables to hadoop using jdbc source and HDFS Sink . HDFS sink is not working

    XMLWordPrintable

    Details

    • Story Points:
      2,567
    • Rank (Obsolete):
      9223372036854775807

      Description

      deploying stream:

      stream create foo2 -definition "jdbc --fixedDelay=600 --split=1 --url='jdbc:sqlserver://ip:port;database=XXXXX' --driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver --username=XXXX --password=XXX-query='select * from abc' | hdfs --fsUri=hdfs://hadopIP:port --directory=xddir" --deploy

      It is throwing error:

      19:17:10,087 1.0.2.RELEASE ERROR inbound.foo1.0-redis:queue-inbound-channel-adapter1 redis.RedisMessageBus$1 - Failed to deliver message; retries exhausted; message sent to queue 'ERRORS:name'
      org.springframework.messaging.MessageHandlingException: failed to write Message payload to HDFS
      at org.springframework.xd.integration.hadoop.outbound.HdfsStoreMessageHandler.handleMessageInternal(HdfsStoreMessageHandler.java:129)
      at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
      at sun.reflect.GeneratedMethodAccessor89.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
      at org.springframework.integration.monitor.SimpleMessageHandlerMetrics.handleMessage(SimpleMessageHandlerMetrics.java:106)
      at org.springframework.integration.monitor.SimpleMessageHandlerMetrics.invoke(SimpleMessageHandlerMetrics.java:86)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
      at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
      at com.sun.proxy.$Proxy100.handleMessage(Unknown Source)
      at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
      at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101)
      at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
      at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:255)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:223)
      at sun.reflect.GeneratedMethodAccessor88.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
      at org.springframework.integration.monitor.DirectChannelMetrics.monitorSend(DirectChannelMetrics.java:113)
      at org.springframework.integration.monitor.DirectChannelMetrics.doInvoke(DirectChannelMetrics.java:97)
      at org.springframework.integration.monitor.DirectChannelMetrics.invoke(DirectChannelMetrics.java:91)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
      at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
      at com.sun.proxy.$Proxy63.send(Unknown Source)
      at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:109)
      at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:44)
      at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:94)
      at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.sendMessage(AbstractReplyProducingMessageHandler.java:260)
      at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.sendReplyMessage(AbstractReplyProducingMessageHandler.java:241)
      at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.produceReply(AbstractReplyProducingMessageHandler.java:205)
      at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleResult(AbstractReplyProducingMessageHandler.java:199)
      at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:177)
      at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
      at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
      at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101)
      at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
      at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:255)
      at org.springframework.xd.dirt.integration.redis.RedisMessageBus$1$1.doWithRetry(RedisMessageBus.java:284)
      at org.springframework.xd.dirt.integration.redis.RedisMessageBus$1$1.doWithRetry(RedisMessageBus.java:280)
      at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:263)
      at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:168)
      at org.springframework.xd.dirt.integration.redis.RedisMessageBus$1.doSend(RedisMessageBus.java:280)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:255)
      at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:223)
      at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:109)
      at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:44)
      at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:94)
      at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:98)
      at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint.popMessageAndSend(RedisQueueMessageDrivenEndpoint.java:211)
      at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint.access$300(RedisQueueMessageDrivenEndpoint.java:50)
      at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint$ListenerTask.run(RedisQueueMessageDrivenEndpoint.java:290)
      at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
      at java.lang.Thread.run(Thread.java:744)
      Caused by: org.springframework.messaging.MessageHandlingException: message not a String
      at org.springframework.xd.integration.hadoop.outbound.HdfsDataStoreMessageHandler.doWrite(HdfsDataStoreMessageHandler.java:75)
      at org.springframework.xd.integration.hadoop.outbound.HdfsStoreMessageHandler.handleMessageInternal(HdfsStoreMessageHandler.java:126)
      ... 60 more

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            neeraj.14 Neeraj Kumar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: