Uploaded image for project: 'Spring Framework'
  1. Spring Framework
  2. SPR-5863

Allow for concurrent test execution in the TestContext framework

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Complete
    • Affects Version/s: 3.0 M3
    • Fix Version/s: 5.0 M2
    • Component/s: Test
    • Labels:
    • Last commented by a User:
      false

      Description

      Status Quo

      Newer versions of JUnit support concurrent test execution; however, the Spring TestContext Framework is not designed for concurrency.

      Proposed Solution

      The enclosed fixes sharpen focus on concurrency (including making mutable state much more distinct from immutable state), increasing separation between data for each test method run and the class they are being run on.

      The patch consists of a failing test (patch 1) and a fix (patch 2), including several new tests. If you choose to apply the patch with the failing test, you must revert this before applying the fix (failing test is contained in fix patch). The included failing test may not produce concurrency issues (fail) in all cases and on all hardware platforms. They have been known to fail consistently on 3 different machines, usually upon first run.

      Details of the Patch

      The patch contains minor changes to the ContextLoader interface. The most significant changes have been made to the TestContextManager and TestContext classes.

      Additionally upon completing the functionality, I had multiple deadlocks in the JVM when running my real test suite. I solved this by using a Java 5 ReentrantReadWriteLock in the RequestAttributes.getSessionMutex() method. It really looks to me like the creation of this mutex should be moved to one of the loader filters, since it's always created as of this patch.

      Additionally, the patch contains a MockContextLoader that transfers attributes between threads. I'd really like you guys to check that code out before accepting it; there may be other smarter ways of doing this. It's only a part of the test code-base, but once it's included it sets a standard.

      Real-life Tests

      The patch has been applied to a local version of Spring 3 that has been running stably with multi-core machines and multi-CPU servers too. We have been running a continuous build using parallel classes, methods, and a combination of both. This is a full-scale build that was adaptable to multi-threaded test execution. The application under test uses lots of web-scopes, etc.

      Proposed Documentation

      Parallel Test Execution

      From version 3.0, Spring supports parallel test execution in the Spring TestContext Framework. Executing builds in parallel with JUnit is only supported in later versions of JUnit, and it is recommended to use at least JUnit 4.6 for this feature. Please also note that there's no guarantee your tests will run properly in parallel; a number of general concurrency issues have to be taken into account when executing tests in parallel. Your runner can usually let you choose between classes, methods, and a combination of both. Classes are usually the easiest to get working; "a combination of both" is the hardest. All three modes are supported.

      1. 1concurrencyFix2051.patch
        77 kB
        Kristian Rosenvold
      2. 2tests2051.patch
        32 kB
        Kristian Rosenvold
      3. 3428_SPR5863.patch
        53 kB
        Kristian Rosenvold
      4. 3dirtiesContext2051.patch
        34 kB
        Kristian Rosenvold
      5. completeFix.patch
        190 kB
        Kristian Rosenvold
      6. springFailingTest.patch
        33 kB
        Kristian Rosenvold

        Issue Links

          Activity

          Hide
          sbrannen Sam Brannen added a comment -

          Kristian Rosenvold, since I unfortunately cannot merge your work without it going through the official channels (i.e., Pivotal CLA, etc.)...

          Would you mind submitting a pull request on GitHub that only includes the failing tests (similar to what I've done in my SPR-5863-krosenvold-github branch)?

          Thanks,

          Sam

          Show
          sbrannen Sam Brannen added a comment - Kristian Rosenvold , since I unfortunately cannot merge your work without it going through the official channels (i.e., Pivotal CLA, etc.)... Would you mind submitting a pull request on GitHub that only includes the failing tests (similar to what I've done in my SPR-5863-krosenvold-github branch)? Thanks, Sam
          Hide
          sbrannen Sam Brannen added a comment -

          Kristian Rosenvold, please disregard my request for you to create a PR. I have come up with an alternative means to complete our test suite coverage of concurrent test execution, and I will push that to master in the near future.

          Show
          sbrannen Sam Brannen added a comment - Kristian Rosenvold , please disregard my request for you to create a PR. I have come up with an alternative means to complete our test suite coverage of concurrent test execution, and I will push that to master in the near future.
          Hide
          sbrannen Sam Brannen added a comment -

          For those interested in the details, the following additional refinements have been pushed to master:

          Furthermore, 2699504 replaces the need for the tests originally implemented by Kristian Rosenvold.

          Show
          sbrannen Sam Brannen added a comment - For those interested in the details, the following additional refinements have been pushed to master : https://github.com/spring-projects/spring-framework/commit/fbfad8695ecc9262f23fd31b0b425eca38b58d01 https://github.com/spring-projects/spring-framework/commit/e822e4cbe87b09b901eb2363bc884a635a96a01e Furthermore, 2699504 replaces the need for the tests originally implemented by Kristian Rosenvold .
          Hide
          sbrannen Sam Brannen added a comment -

          A new "Parallel test execution" section has been added to the "Testing" chapter of the reference manual in commit 3e96cab.

          Show
          sbrannen Sam Brannen added a comment - A new "Parallel test execution" section has been added to the "Testing" chapter of the reference manual in commit 3e96cab .
          Hide
          sbrannen Sam Brannen added a comment -

          Resolving this issue as complete.

          For a list of general guidelines and unsupported use cases, please consult the new Parallel test execution section in the Testing chapter of the reference manual.

          Please give Spring Framework 5.0 snapshots a try and let us know how parallel test execution works for you.

          Thanks!

          Show
          sbrannen Sam Brannen added a comment - Resolving this issue as complete . For a list of general guidelines and unsupported use cases, please consult the new Parallel test execution section in the Testing chapter of the reference manual. Please give Spring Framework 5.0 snapshots a try and let us know how parallel test execution works for you. Thanks!

            People

            • Assignee:
              sbrannen Sam Brannen
              Reporter:
              krosenvold Kristian Rosenvold
              Last updater:
              St├ęphane Nicoll
            • Votes:
              42 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since last comment:
                3 weeks, 3 days ago

                Time Tracking

                Estimated:
                Original Estimate - 0d
                0d
                Remaining:
                Remaining Estimate - 0d
                0d
                Logged:
                Time Spent - 0.5h
                0.5h