Uploaded image for project: 'Spring Integration'
  1. Spring Integration
  2. INT-3700

FeedEntryMessageSource Improperly Calculates Proper Last Entry Processed Date




      This is different from issue INT-1810

      I'm using SpringXD and i need to process information from RSS feeds. Since SpringXD doesn't come with the feed source, i've compiled the example from the documentation (http://docs.spring.io/spring-xd/docs/1.1.1.RELEASE/reference/html/#creating-a-source-module). when testing, i noticed that after some time, it stops honoring my polling interval and keeps polling in the last feed entry continually.

      After some debugging, i noticed that when the FeedEntryMessageSource reaches the end of the entry set and if it has not meet the maximum number of messages to poll, that it will attempt to fetch the feed again (presuming that it may have updated since) and load any NEW feeds in to the collection for processing. In my case, it always loads the last feed as the feed publish/update date is greater than the feeds publish date or the last processed entries publish date.

      I feel that the setting of the lastTime variable in FeedEntryMessageSource#getNextEntry should use the same logic that is used for comparison. Namely the logic in the method FeedEntryMessageSource#getLastModifiedDate.

      I feel that this is a critical issue as the message source will never return control back to and continually poll the same feed until the feed entries have been added to reach the maximum number of messages to poll count.

      One work around would be to tailor the polling interval and the maximum number of messages to poll count to the update frequency of the feed being read. This is not suitable for me as I do not have control of which feeds are added to my system. The other work around would be to copy the existing class, fix the issues, and wire it in as a source to an inbound channel adapter.

      I have done the second to verify this works; at least on the feed I've been using.




            abilan Artem Bilan
            loesak Aaron Loes
            0 Vote for this issue
            2 Start watching this issue