Release Notes - Spring XD - Version M2 - HTML format

Bug

  • [XD-228] - Missing '=' in example of http stream
  • [XD-291] - HTTP Source still listens on port 9000 after removal.
  • [XD-292] - Redis 'install-redis' script fails on Ubuntu64
  • [XD-429] - Document time source
  • [XD-476] - The stream definition is not deleted in redis container when the stream is destroyed
  • [XD-480] - In certain scenarios a job can be redeployed more than once
  • [XD-481] - Running Job with time delay (non cron) launches 2 instances before job is supposed to fire
  • [XD-482] - When creating a job with the fixed-delay parameter in the shell command fails
  • [XD-499] - Cron Jobs stop firing when a named trigger is created and deployed
  • [XD-500] - If a job is created that uses a trigger that has not been created and deployed it throws a 500 error instead of a 400
  • [XD-503] - Jobs are created even though they have an invalid definition
  • [XD-509] - Investigate intermittent failure of RedisStreamDefinitionRepositoryTests
  • [XD-523] - Parser blows on modules names with '-'
  • [XD-556] - CORS support
  • [XD-560] - Ad-hoc Jobs do not start
  • [XD-583] - Dispatcher Has No Subscriber Error when posting a message to a stream
  • [XD-585] - Deploying with twittersearch source throws Jackson ClassDefNotFound exception
  • [XD-592] - Problems with advanced tapping
  • [XD-599] - Gradle Import Broken by Hadoop Pseudo Projects
  • [XD-603] - NPE on stream destroy
  • [XD-608] - Fix text-table rendering
  • [XD-624] - Use External Connection Factory in TCP Syslog Source
  • [XD-626] - Shell: RestTemplate not posting using UTF-8
  • [XD-640] - Cannot start xd-container with the --hadoopDistro option
  • [XD-644] - Connection props in rabbit.properties ignored by xd-admin and xd-container
  • [XD-652] - File Source Name and Duplicates options not working as documented

Epic

  • [XD-177] - Easily switch between a single process that performs all admin and processing tasks to one that has a dedicated admin processes and distributed processing containers.
  • [XD-484] - Create required infrastructure to easily perform integration testing of shell commands

Improvement

  • [XD-190] - Cleanup embedded container story
  • [XD-552] - Add status column for 'stream list' shell command result

Story

  • [XD-119] - HDFS sink should default to hdfs://localhost:8020
  • [XD-137] - Release Spring XD 1.0 M1
  • [XD-142] - StreamServer Context Lifecycle Issues
  • [XD-144] - Remove Log4j Properties/XMLs from Projects
  • [XD-160] - Publish golo themed docs documentation to static.springsource.org as part of nightly build
  • [XD-162] - Create design document for implementation strategy to support message conversion in ChannelRegistry
  • [XD-164] - Validate processing modules declare the required channels
  • [XD-176] - Support exponential moving average in RichGauge
  • [XD-191] - Move Redis connection metadata logging into the code closest to establishing that connection
  • [XD-210] - If output directory does not exist for a file sink, by default allow it to be created
  • [XD-218] - Add support to load a twitter.properties file in the source
  • [XD-226] - Cleanup and Optimize gradle tasks to bundle spring-xd distribution
  • [XD-227] - Add jetty-util-6.1.26.jar and jsr311-api-1.1.1.jar as required jars so they will be on the XD classpath
  • [XD-229] - Add RabbitMQ-based implementation of ChannelRegistry
  • [XD-230] - Add RabbitMQ source module
  • [XD-231] - Add Twitter gardenhose source module
  • [XD-236] - Create an Aggregate Counter
  • [XD-242] - Add JMS source module
  • [XD-244] - Create a Trigger
  • [XD-245] - Deploy Batch Jobs on XD
  • [XD-247] - Need to be able to specify password for Redis
  • [XD-249] - Initial client library for XD REST API
  • [XD-250] - Set up a project for XD REST client library
  • [XD-251] - Add support for stream creation
  • [XD-255] - Set up a project for XD Shell
  • [XD-256] - Create a banner page for XD Shell
  • [XD-257] - Create the base implementation for XDCommands for the shell
  • [XD-258] - Add command for stream creation
  • [XD-259] - Add command for tap creation
  • [XD-260] - Add command for listing streams
  • [XD-261] - Add command for deleting a stream
  • [XD-262] - Convert current REST servlet to Spring MVC
  • [XD-263] - Support GET /streams
  • [XD-264] - Spring MVC infrastructure tests
  • [XD-268] - Streamline command-line arg management
  • [XD-269] - Create JobDefinition repository
  • [XD-294] - Apply JavaDoc HotFix
  • [XD-296] - Add log config file to gemfire in final distro
  • [XD-298] - Refactor gardenhose into more generic twitterstream source
  • [XD-301] - Request to create a repo for Spring XD performance testing
  • [XD-311] - Optionally Add Spring/Integration MBean Exporters to Common ApplicationContext
  • [XD-312] - Add Jolokia Agent Depending on Run Mode
  • [XD-313] - Add Spring/Integration MBean Exporters to Module ApplicationContexts
  • [XD-315] - Package Shell "binary" next to xd-admin and xd-container
  • [XD-317] - Add Documentation Chapter on Executing Batch Jobs
  • [XD-318] - Create Splunk sink module
  • [XD-319] - Add command for deleting a tap
  • [XD-321] - Add create() and deploy() to TapsController
  • [XD-323] - Add command for listing of taps
  • [XD-328] - Retrieve information for a Counter
  • [XD-329] - Retrieve information for a Field Value Counter
  • [XD-330] - Retrieve information for a Gauge
  • [XD-331] - Retrieve information for a Rich Gauge
  • [XD-332] - Retrieve information for an aggregate counter
  • [XD-334] - Document the structure of the REST API
  • [XD-335] - Review DSL
  • [XD-336] - Document Splunk source sink
  • [XD-337] - Test connection pooling on Redis blocking/nonblocking operations
  • [XD-339] - Investigate using profiler when doing the performance testing
  • [XD-342] - Fix classpath error caused by multiple conflicting servlet-api jars
  • [XD-345] - Replace "gardenhose" doc with new "twitterstream"
  • [XD-348] - Trigger - Add support for fixed-delay interval
  • [XD-350] - Support hourly resolution in redis aggregate counter
  • [XD-352] - In-memory implementation of aggregate counter
  • [XD-356] - Test startup scripts on windows
  • [XD-357] - Creating a base class for Plugins
  • [XD-358] - Add support for creating named cron triggers
  • [XD-359] - Add support for creating a spring batch job that has an embedded trigger expression
  • [XD-360] - Add support for creating a spring batch job that references a named trigger
  • [XD-361] - Create a trigger from Shell
  • [XD-363] - Add support for creating fixed delay/ fixed rate triggers
  • [XD-364] - Create TriggerDefinition Repository
  • [XD-365] - Support having multiple property placeholders defined in different modules
  • [XD-366] - Create a XD job definition
  • [XD-367] - Final review of REST API structure document for streams, taps and jobs
  • [XD-368] - Improve connection handling in RedisAggregateCounterService.
  • [XD-369] - Further DSL extensions
  • [XD-370] - Add command for deploying a Stream
  • [XD-371] - Command for creating a job
  • [XD-372] - Command to deploy a job
  • [XD-373] - Command to create a tap
  • [XD-374] - Command to list taps
  • [XD-375] - Command to delete tap
  • [XD-376] - add create() and deploy() methods to JobsController
  • [XD-377] - add create() and deploy() methods to JobDeployer
  • [XD-378] - add create() and deploy() methods to StreamsController
  • [XD-379] - add create() and deploy() methods to StreamDeployer
  • [XD-381] - add create() and deploy() methods to TapDeployer
  • [XD-382] - Create TapRepository
  • [XD-383] - Implement list() method on TapController
  • [XD-384] - Implement list() method on TapDeployer()
  • [XD-388] - Create design document for implementation strategy for ingesting data from twitter into HDFS that can be analyzed by HAWQ
  • [XD-392] - Create a stubbed out job controller
  • [XD-398] - Update Getting Started chapter to use Shell commands instead of curl
  • [XD-399] - Update Getting Started chapter to include a section on starting the shell.
  • [XD-400] - Update Streams Chapter to use shell commands instead of curl
  • [XD-401] - Create a shell command to post data to an http port for use with the http source module
  • [XD-402] - Create a command to browse the HDFS file system
  • [XD-403] - Update Sources section to use Shell commands instead of curl
  • [XD-404] - Update documentation section "Running in Distributed Mode" to show use of RabbitMQ in addition to Redis
  • [XD-405] - Update Sources tail section to use Shell commands instead of curl
  • [XD-406] - Update Sources twitter search section to use Shell commands instead of curl
  • [XD-407] - Update Sources Gemfire CQ section to use Shell commands instead of curl
  • [XD-408] - Update Source Syslog section to use Shell commands instead of curl
  • [XD-409] - Update Sources TCP section to use Shell commands instead of curl
  • [XD-410] - Update Processors Filter & JSon Filed Value Filter section to use Shell commands instead of curl
  • [XD-411] - Update Processors Transform section to use Shell commands instead of curl
  • [XD-412] - Update Processors JSON Field Extractor section to use Shell commands instead of curl
  • [XD-413] - Update Processors Script section to use Shell commands instead of curl
  • [XD-414] - Update Sink's Log section to use Shell commands instead of curl
  • [XD-415] - Update Sink's File section to use Shell commands instead of curl
  • [XD-416] - Update Sink's HDFS section to use Shell commands instead of curl
  • [XD-417] - Update Sink's TCP section to use Shell commands instead of curl
  • [XD-418] - Update Sink's GemFire section to use Shell commands instead of curl
  • [XD-419] - Taps introduction section should show use of shell to create a real stream and a real tap using the shell
  • [XD-420] - Documentation for AggregateCounter
  • [XD-421] - Update Analytics Counter section to use Shell commands instead of curl
  • [XD-422] - Update Analytics Field Value Counter section to use Shell commands instead of curl
  • [XD-423] - Update Analytics Gauge section to use Shell commands instead of curl
  • [XD-424] - Update Analytics Rich Gauge section to use Shell commands instead of curl
  • [XD-425] - Update Samples syslog ingestion section to use Shell commands instead of curl
  • [XD-426] - Update Creating a Processor Module section to use Shell commands instead of curl
  • [XD-427] - Update Creating a Source Module section to use Shell commands instead of curl
  • [XD-428] - Update Creating a Sink Module section to use Shell commands instead of curl
  • [XD-430] - Use a Different Default Jolokia Port for Admin Vs. Container
  • [XD-431] - Make String conversion optional with local transport
  • [XD-435] - Create tests to load the standard runtime app context configurations
  • [XD-437] - Fix Package Tangles
  • [XD-438] - More DSL work: exploiting source/sink channels
  • [XD-439] - More DSL work: hooking up stream directory
  • [XD-443] - Add MQTT Source
  • [XD-444] - Make Redis/Rabbit @Rules Conditional
  • [XD-447] - Add an MQTT Sink
  • [XD-451] - Add a Access-Control-Allow-Origin header to responses in order to support cross-origin requests
  • [XD-465] - Shell should display error messages returned from the server
  • [XD-466] - Add JSON conversion to tuple
  • [XD-469] - Upgrade to spring-data-hadoop 1.0.1.RC1
  • [XD-470] - Create JDBC sink
  • [XD-472] - Add spring-xd-hadoop distro specific sub-projects
  • [XD-473] - Modify startup script of xdadmin/xdcontainer to allow specifying hadoop distro to use
  • [XD-477] - Support pagination in list() command for streams
  • [XD-478] - Add accepted type logic to module
  • [XD-479] - Add conversion support to ChannelRegistrar and ChannelRegistry
  • [XD-483] - Job Delete/Destroy Command for shell
  • [XD-485] - Create stories to enable the use of Spring Shell's 2.0 branch testing facilities
  • [XD-486] - All controllers to return XYZResource objects not the raw domain objects.
  • [XD-487] - StreamsController to return paged results for list()
  • [XD-488] - Rename controllers to have pluralized named (e.g. JobsController)
  • [XD-489] - Introduce distinction between TapDefinition and Tap (the instance)
  • [XD-490] - StreamDeployer to implement ResourceDeployer
  • [XD-491] - Exception Consistency
  • [XD-492] - Rename create to 'save' in ResourceDeployer
  • [XD-493] - Remove duplicate code in ResourceDeployer implementations, create abstract base class.
  • [XD-494] - Implement common set of controller methods
  • [XD-495] - Document MQTT Source and Sink
  • [XD-496] - Disable Collection to Object conversion in DefaultTuple
  • [XD-497] - Fix Sonar build!
  • [XD-498] - Better UX when admin is not running
  • [XD-502] - Delete a trigger from Shell
  • [XD-504] - Add "How to Build Spring-XD" instructions to the documentation
  • [XD-506] - Support pagination in list() command for jobs
  • [XD-507] - Support pagination in list() command for taps
  • [XD-508] - Support pagination in list() command for triggers
  • [XD-510] - Ensure that each controller's list() returns PagedResources
  • [XD-518] - Upgrade to Jackson 2.2.2
  • [XD-524] - Update jobs section to use shell
  • [XD-525] - Provide .settings formatting rules so that they're shared
  • [XD-526] - Make module files classpath aware
  • [XD-533] - Create shell integration tests for stream lifeycle
  • [XD-534] - Create shell integration tests for tap lifeycle
  • [XD-536] - Create shell integration tests for job lifeycle
  • [XD-537] - Refactor analytics to get rid of Services
  • [XD-538] - Develop infrastructure to enable testability of commands
  • [XD-539] - Investigate using Redis txs and pipeline for Inbound/Outbound Q Adapter perf improvements
  • [XD-540] - Broadcast Undeploy Requests
  • [XD-542] - Refactor Module to Encapsulate Group and Index
  • [XD-544] - Fix In-Memory Analytics
  • [XD-545] - Display a counter
  • [XD-546] - Display a Field Value Counter
  • [XD-547] - Display an Aggregate Counter
  • [XD-548] - Display a Gauge
  • [XD-549] - Display a Rich Gauge
  • [XD-551] - Add "trigger list" support to Spring XD Shell
  • [XD-553] - Add additional options to File source
  • [XD-554] - Separate Module Context Refresh from Context Start
  • [XD-555] - make application/json the default output type for the REST API?
  • [XD-559] - Send failing sonar build message to spring-xd mailing list.
  • [XD-561] - Failure when creating/deploying stream leaves invalid stream registry/definitions in the Repository implementations.
  • [XD-563] - Regression test existing functionality of stream/taps based on introduction of new conversion functionality
  • [XD-565] - Documentation for using a specific Hadoop distribution
  • [XD-566] - Document JDBC module
  • [XD-567] - Documentation for fixed rate triggers
  • [XD-568] - Documentation for deleting triggers
  • [XD-569] - Investigate failures to start a stream when using named channels.
  • [XD-571] - Create shell integration test for named chanels
  • [XD-572] - Prepare Blog post for XD M2
  • [XD-574] - Replace usage of 'raw' curl with shell command to post http data in documentation
  • [XD-575] - Change http command to post data by putting 'http' as the main command option
  • [XD-576] - Change banner of shell to say only 'xd'
  • [XD-577] - In documentation, replace usage of 'raw' hadoop command with shell 'hadoop' commands
  • [XD-579] - Modify startup script of xd shell to allow specifying hadoop distro to use
  • [XD-580] - XD Shell needs to support multiple Hadoop distros
  • [XD-581] - configuration conflict when using "--transport", "local", "--store", "redis", "--disableJmx", "true", "--analytics", "redis"
  • [XD-582] - Support named channels when using local transport
  • [XD-587] - Create a abstract base class for rest controllers
  • [XD-588] - RedisAggregateCounterRepository doesn't give proper results back
  • [XD-589] - Create AbstractStreamIntegrationTest that will destory streams that were created during test method execution
  • [XD-591] - Improve build file distribution tasks
  • [XD-593] - Add "counter delete" shell command
  • [XD-594] - Create list/delete commands for all the metrics
  • [XD-595] - Fix wiki documentation to use xd shell command prompt to read "xd:>"
  • [XD-596] - Add CONTRIBUTING.md file
  • [XD-598] - Gemfire cache closed when a gemfire module is undeployed
  • [XD-605] - Rich Gauge doco is outdated
  • [XD-606] - Document JSON quoting behavior in shell
  • [XD-609] - Documentation for rabbit source
  • [XD-610] - Documentation for jms source
  • [XD-611] - Documentation for file source
  • [XD-612] - Create a rabbit sink module and documentation
  • [XD-614] - Conversion Enhancements
  • [XD-617] - Making an http post with json double quoted will hang the shell.
  • [XD-618] - Update to Spring Shell 1.1.0.M1 release
  • [XD-619] - Add list command for AggregateCounter
  • [XD-620] - Refactor test cases to move away from inheritance model of utility methods for streams, counters
  • [XD-622] - Fix JavaDoc Warnings
  • [XD-632] - Shell: HTTP Post - Allow posting of local file contents
  • [XD-633] - Rabbit Source - Make Default QueueName == Stream Name
  • [XD-634] - Fix guava dependency for hadoop20 and phd1
  • [XD-635] - OOTB source modules with poller should use fixed-delay
  • [XD-636] - x-xd-* Transport Content-Type Leakage
  • [XD-642] - Update architecture diagram to show rabbit in addition to redis to communicate between admin and containers
  • [XD-643] - Map column names with underscore to camelCase style keys for JDBC sink
  • [XD-648] - Regression test on file source
  • [XD-649] - Remove 'substream' from the documentation
  • [XD-650] - Eclipse build path error after running gradle -> refresh source folders in Eclipse
  • [XD-653] - Configure Jackson ObjectMappers to Allow Single Quotes
  • [XD-655] - Trim output from http post shell command to two lines
  • [XD-660] - Rename spring-xd-shell to xd-shell
  • [XD-663] - Use correct FS_DEFAULT_NAME_KEY constant based on Hadoop version used
  • [XD-664] - File sink filename should default to having a '.out' suffix.
  • [XD-665] - AggregateCounter display command options with "lastHours" and "lastDays"
  • [XD-666] - Remove Redis Transport Headers from Tapped Stream
  • [XD-670] - TAB completion for existing entities
  • [XD-674] - Add Spring Batch word-count Sample to Spring XD Samples repo
  • [XD-675] - Cannot chain json-field-value-filter & json-field-extractor
  • [XD-678] - change accepted-media-types to accpted-content-types
  • [XD-692] - http source module should copy Content-Type header to SI MessageHeaders.CONTENT_TYPE

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.