Uploaded image for project: 'Spring Integration'
  1. Spring Integration
  2. INT-3464

AbstractInboundFileSynchronizer can skip remote files if copy fails after accept filter

    XMLWordPrintable

    Details

      Description

      While developing a custom remote file inbound channel adapter I stumbled across a situation where several files were skipped. Investigation revealed a potential bug in the org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.

      The situation was where an AcceptOnceFileListFilter was configured on the inbound channel adapter and then error occurs between listing the files on the remote file server and copying the files to the temporary local directory.

      The accept once filter was configured something like this:

      <bean id="filter" class="...PersistentAcceptOnceFileListFilter">
      	<constructor-arg><bean class="o.s.i.m.SimpleMetadataStore" /></constructor-arg>
      ...
      </bean>
      
      <int-gd:inbound-channel-adapter
      			channel="inputChannel"
      ...
      			filter="filter">
      		<int:poller fixed-delay="10000"/>
      </int-gd:inbound-channel-adapter>
      

      In the configuration above there is an inbound channel adapter with an accept once filter. It could be the sftp adapter, ftp adapter, or some other remote file transferring inbound adaptor that depends on AbstractInboundFileSynchronizer.

      The issue is exposed when an error occurs at a particular point in the AbstractInboundFileSynchronizer code. The relevant code is in the synchronizeToLocalDirectory method:

      Collection<F> filteredFiles = AbstractInboundFileSynchronizer.this.filterFiles(files);
      for (F file : filteredFiles) {
            if (file != null) {
                 AbstractInboundFileSynchronizer.this.copyFileToLocalDirectory(
                 AbstractInboundFileSynchronizer.this.remoteDirectory, file, localDirectory, session);
            }
      }
      

      In the above code the problem occurs if an exception occurs in copyFileToLocalDirectory, which is after the list of files is retrieved and filtered. When this happens the exception propagates to the poller and the current poll ends. When the next poll occurs this component will get the list of files again and this list will contain the remaining files from the previous attempt. However, because the filter has already seen them (because it is an accept once filter), it will not copy them to the local directory. The files remain on the remote file system unprocessed.

        Attachments

          Activity

            People

            Assignee:
            grussell Gary Russell
            Reporter:
            simontankersley Simon Tankersley
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: