Release Notes - Spring XD - Version M1 - HTML format

Technical task

  • [XD-11] - Initial work that uses the module architecture from DIRT
  • [XD-21] - Create syslog.xml file for module registry
  • [XD-22] - Create Module base abstractions
  • [XD-32] - Create base Channel Registry abstraction
  • [XD-33] - Implement LocalChannelRegistry
  • [XD-39] - Add integration capability to the runtime nodes
  • [XD-42] - RedisChannelRegistry
  • [XD-101] - Add gradle tasks that build and bundle the redis server
  • [XD-114] - Add install script for Redis

Bug

  • [XD-41] - Add JUnit @Rule so Tests Fail Fast with Clear Messaging if Redis Not Available
  • [XD-58] - build.gradle doesn't handle a small handful of libraries
  • [XD-66] - Gauge and Counter hash and equals should not depend on values
  • [XD-122] - XD scripts need to have spring-integration milestone versions updated
  • [XD-123] - XD scripts lib path needs to be dynamic
  • [XD-199] - Ensure the DELETE Operation can Delete a Tap
  • [XD-200] - Creating a tap throws an exception
  • [XD-201] - Fix XD scripts on windows
  • [XD-206] - XD AdminMain & ContainerMain should check xd.home property from scripts

Epic

  • [XD-75] - Add redis bundle to distribution zip file

Improvement

  • [XD-165] - xd-container and xd-admin should log to a file out of the box
  • [XD-212] - Add http port command line option to AdminMain
  • [XD-224] - Reduce necessity for quoting in parameter values in DSL expressions

Story

  • [XD-1] - HDFS ItemWriter
  • [XD-2] - HDFS Core writing helper classes
  • [XD-6] - Channel Registry
  • [XD-7] - Tuple data structure
  • [XD-8] - Syslog Ingestion
  • [XD-9] - Basic implementation of a reactor based tcp server
  • [XD-13] - Tail file channel adapters
  • [XD-20] - DIRT Runtime that deploys an application context across multiple nodes using redis.
  • [XD-23] - add file source and sink modules
  • [XD-24] - Create pipes and filters DSL for ingestion
  • [XD-26] - Basic Performance Test For syslog injestion
  • [XD-27] - Gemfire CQ module for ingestion
  • [XD-28] - Create simple gague service
  • [XD-29] - Create rich gauge service
  • [XD-30] - Create a simple counter service
  • [XD-31] - Create field-value counters
  • [XD-35] - Create CI process for XD build
  • [XD-36] - SI Outbound HDFS Channel Adapter
  • [XD-37] - Gradle based multi-project build
  • [XD-40] - Build script that creates an executable server as an artifact
  • [XD-43] - Metric repositories should support Spring Data CrudRepository interface
  • [XD-44] - Redis based repositories should use a NamingStrategy class to calculate the name of the key to use for persistence
  • [XD-45] - Remove the expiry of keys in Redis based repositories
  • [XD-47] - HDFS sink module
  • [XD-48] - Parameterizable streams
  • [XD-50] - Add tap support to DIRT
  • [XD-51] - Add xd.stream.name property in StreamPlugin
  • [XD-52] - add twitter search source module
  • [XD-54] - XD Metrics backed Message Counter
  • [XD-55] - SI ServiceActivator for an XD Metrics backed Field Value Counter
  • [XD-56] - Switch to use Lettuce driver for Redis
  • [XD-57] - add counter module
  • [XD-59] - Tuple should support storing nested tuples
  • [XD-61] - Create distributable artifact that contains server application and start/stop scripts
  • [XD-62] - Use the tuple data structure to process data in a spring batch step
  • [XD-63] - Document tuple data structure on XD wiki
  • [XD-65] - Gemfire Sink to update a gemfire cache.
  • [XD-67] - Submit a brew-based install for Spring XD
  • [XD-70] - Create general structure for AsciiDoc based wiki and Spring XD guide.
  • [XD-72] - Provide a http source
  • [XD-73] - add test to start/stop stream server
  • [XD-74] - Create XD module for tail file adapter
  • [XD-76] - Add gemfire-server application to the distribution zip of the project spring-xd-gemfire-server
  • [XD-78] - Documentation for "http | file" processing
  • [XD-79] - End user guide for data streams
  • [XD-80] - Documentation for "http | hdfs" processing
  • [XD-81] - Documentation for "syslog | file" processing
  • [XD-82] - Documentation for "tail | file" processing
  • [XD-83] - Documentation for "tail | hdfs" processing
  • [XD-84] - Documentation for "http | gemfire" processing
  • [XD-85] - Documentation for "gemfirecq | file" processing
  • [XD-86] - Documentation for "twittersearch | file" processing
  • [XD-87] - Documentation that introduces taps
  • [XD-88] - Documentation for counter taps
  • [XD-89] - Documentation for gauge taps
  • [XD-90] - Documentation for rich gauge taps
  • [XD-91] - Documentation for field value taps
  • [XD-98] - Documentation on the module system and how to contribute new modules
  • [XD-99] - Sonar build is failing
  • [XD-102] - Create XDContainer class to start stream server
  • [XD-103] - Create XDAdmin server to start container launcher
  • [XD-104] - Add README to be included in root directory of distribution
  • [XD-105] - Add LICENSE to be included in root directory of distribution
  • [XD-106] - Container server does not log a message that it has started or stopped successfully
  • [XD-107] - Clean shutdown of redis in xd-admin
  • [XD-108] - Build script should not package 'spring-xd-dirt' scripts
  • [XD-109] - Documentation for starting Spring XD servers
  • [XD-110] - Remove use of system property xd.home to define location for install location, rely on environment variable XD_HOME
  • [XD-111] - Create final distribution zip across multiple projects
  • [XD-112] - Update XD to Use SI 3.0.0.M2
  • [XD-115] - add SpEL 'transform' processor
  • [XD-116] - add SpEL 'filter' processor
  • [XD-117] - add spring-integration-groovy to container dependencies
  • [XD-118] - replace testsource with time and testsink with log
  • [XD-120] - Find and eliminate package-level cycles across XD projects
  • [XD-124] - Clean shutdown of redis in xd-container
  • [XD-125] - Stream documentation review
  • [XD-126] - Documentation for sources, sinks, modules should define which attributes are required and which optional
  • [XD-127] - Create TCP source module
  • [XD-128] - Create TCP sink module
  • [XD-129] - Documenation for building/starting redis servers
  • [XD-130] - Remove container entry in Redis when the application context event to shutdown the container is fired
  • [XD-131] - Upgrade Lettuce to 2.3.2
  • [XD-133] - Fail Sonar CI build if there are any package tangles violated.
  • [XD-134] - Investigate link checking tool for user guide
  • [XD-135] - Documentation on XD Architecture
  • [XD-136] - Documentation that points on how to install hadoop
  • [XD-138] - Prepare blog post for M1
  • [XD-139] - Update README.txt to include instructions on how to build
  • [XD-140] - Parameterize syslog Source; Add Support for TCP
  • [XD-141] - install-redis script should not use relative path to determine redis source dist
  • [XD-143] - Create externalized property file to support connectivity to redis
  • [XD-147] - Remove use of application plugin for redis project
  • [XD-148] - Improve User Experience when Redis is not running
  • [XD-149] - Create asciidoc toolchain script to create a 'toc2' style html output
  • [XD-150] - Publish Spring XD final distribution zip as part of Bamboo artifactory plugin
  • [XD-152] - Create rich gauge module
  • [XD-153] - create a gauge module
  • [XD-155] - Add a groovy script processor module
  • [XD-156] - Create config support for Redis
  • [XD-161] - Add HTTP Delete Stream Operation
  • [XD-166] - Create config support based on channel registry type
  • [XD-167] - Script to generate reference documentation from wiki and include in .zip distribution
  • [XD-168] - Decide on location to host http reference documentation and automate upload in build scripts
  • [XD-169] - XD should run offline
  • [XD-170] - Home wiki page improvements
  • [XD-171] - Create project home page for SpringXD on springsource.org/spring-xd
  • [XD-172] - Create links to SpringXD on other pages of springsource.org site
  • [XD-173] - Document how to create a custom input/output module for existing SI channel adapters
  • [XD-174] - Document how to create a custom processor module.
  • [XD-175] - Create and document a syslog aggregation example
  • [XD-178] - DefaultContainer should have a default constructor that generates a UUID
  • [XD-179] - Have three startup scripts, xd-singlenode, xd-admin, and xd-container
  • [XD-180] - The command line for xd-admin and xd-container to support an additional option, pipeProtocol, that is used to determine the middleware for sending admin requests and data between processing steps
  • [XD-181] - Update launcher.xml to have protocol independent beans defined and an import statement to load protocol specific defintiions from a system property defined location.
  • [XD-182] - Create redisProtocol.xml that will load all the Redis specific implementations to suppor the XD container runtime and administration
  • [XD-183] - Create localChannelProtocol.xml that will load all the SI specific implementations to suppor the XD container runtime and administration
  • [XD-184] - Add unregistration support to the channel registry
  • [XD-185] - Refactor StreamServer to an interface and create Redis and Local implementations
  • [XD-186] - Create a pipe protocol independent StreamDeployer
  • [XD-187] - Create XD script for xd-single node
  • [XD-188] - Add directory to classpath in server startup scripts so groovy based processors can be easily referenced by name without a resource uri prefix
  • [XD-192] - Update getting started documentation to use xd-singlenode start script.
  • [XD-193] - Need more unique resource locations for XD internal configuration
  • [XD-195] - The {{time}} Source Should Emit String by Default
  • [XD-196] - replace the hacky parser with a good one
  • [XD-197] - Create a scriptProcessor module that allows the execution of a groovy (potentially jruby,jython) based SI Service Activator
  • [XD-202] - Update twittersearch module for Twitter 1.0 API retirement
  • [XD-204] - Document processor modules
  • [XD-207] - Provide configurable properties for hdfs sink.
  • [XD-208] - Document the file sink
  • [XD-211] - Document the log sink
  • [XD-213] - Temporarily add toString() Logic in Local Mode Inter-Module Comms
  • [XD-215] - Add authentication information to twittersearch source doc
  • [XD-216] - Add support for tap foo.bar syntax in the DSL
  • [XD-217] - Add config dir to classpath to support custom properties-locations
  • [XD-219] - Surpress tap WARNING message in local mode
  • [XD-220] - Add twitter oauth properties file to config dir
  • [XD-221] - Links in asciidoctor generated HTML+docbook documentation are broken
  • [XD-222] - Add docs for Deleting a simple stream.
  • [XD-223] - Update Creating a Custom Source Module doc with a different SI adapter due to Twitter issues
  • [XD-225] - field-value-counter should support nested fieldNames

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.