Spring Framework
  1. Spring Framework
  2. SPR-5863

Fix concurrency issues in the TestContext framework

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0 M3
    • Fix Version/s: 4.1 RC1
    • Component/s: Test
    • Labels:
    • Last commented by a User:
      true

      Description

      Status Quo

      Newer versions of JUnit support concurrent test execution; however, the Spring TestContext Framework is not designed for concurrency.

      Proposed Solution

      The enclosed fixes sharpen focus on concurrency (including making mutable state much more distinct from immutable state), increasing separation between data for each test method run and the class they are being run on.

      The patch consists of a failing test (patch 1) and a fix (patch 2), including several new tests. If you choose to apply the patch with the failing test, you must revert this before applying the fix (failing test is contained in fix patch). The included failing test may not produce concurrency issues (fail) in all cases and on all hardware platforms. They have been known to fail consistently on 3 different machines, usually upon first run.

      Details of the Patch

      The patch contains minor changes to the ContextLoader interface. The most significant changes have been made to the TestContextManager and TestContext classes.

      Additionally upon completing the functionality, I had multiple deadlocks in the JVM when running my real test suite. I solved this by using a Java 5 ReentrantReadWriteLock in the RequestAttributes.getSessionMutex() method. It really looks to me like the creation of this mutex should be moved to one of the loader filters, since it's always created as of this patch.

      Additionally, the patch contains a MockContextLoader that transfers attributes between threads. I'd really like you guys to check that code out before accepting it; there may be other smarter ways of doing this. It's only a part of the test code-base, but once it's included it sets a standard.

      Real-life Tests

      The patch has been applied to a local version of Spring 3 that has been running stably with multi-core machines and multi-CPU servers too. We have been running a continuous build using parallel classes, methods, and a combination of both. This is a full-scale build that was adaptable to multi-threaded test execution. The application under test uses lots of web-scopes, etc.

      Proposed Documentation

      Parallel Test Execution

      From version 3.0, Spring supports parallel test execution in the Spring TestContext Framework. Executing builds in parallel with JUnit is only supported in later versions of JUnit, and it is recommended to use at least JUnit 4.6 for this feature. Please also note that there's no guarantee your tests will run properly in parallel; a number of general concurrency issues have to be taken into account when executing tests in parallel. Your runner can usually let you choose between classes, methods, and a combination of both. Classes are usually the easiest to get working; "a combination of both" is the hardest. All three modes are supported.

      1. 1concurrencyFix2051.patch
        77 kB
        Kristian Rosenvold
      2. 2tests2051.patch
        32 kB
        Kristian Rosenvold
      3. 3428_SPR5863.patch
        53 kB
        Kristian Rosenvold
      4. 3dirtiesContext2051.patch
        34 kB
        Kristian Rosenvold
      5. completeFix.patch
        190 kB
        Kristian Rosenvold
      6. springFailingTest.patch
        33 kB
        Kristian Rosenvold

        Issue Links

          Activity

          Hide
          Frank Prumbaum added a comment -

          So is there any issue why it isn't fixed until 3.2m1?
          With 3.1.RC2 I still seem to have concurrency issues and running
          the Spring tests in parallel mode seems still to be no option. Is
          this correct or is this issues description outdated (since it has not
          been revised for quite a long time now)?

          Show
          Frank Prumbaum added a comment - So is there any issue why it isn't fixed until 3.2m1? With 3.1.RC2 I still seem to have concurrency issues and running the Spring tests in parallel mode seems still to be no option. Is this correct or is this issues description outdated (since it has not been revised for quite a long time now)?
          Hide
          Kristian Rosenvold added a comment -

          I have not bothered resubmitting this patch based on trunk since it's been more or less permanently on the "move forward" list in the issue tracker. This is probably because the patch contains an interface change.

          I think it should be possible to re-work the patch to not contain any breaking changes, especially if this increases the likelihood of getting the patch accepted. The patch is certainly quite stale as applied on current trunk. Let me know if reworking the patch will increase the probability of this getting applied to >0, which seems to be its current value.

          Show
          Kristian Rosenvold added a comment - I have not bothered resubmitting this patch based on trunk since it's been more or less permanently on the "move forward" list in the issue tracker. This is probably because the patch contains an interface change. I think it should be possible to re-work the patch to not contain any breaking changes, especially if this increases the likelihood of getting the patch accepted. The patch is certainly quite stale as applied on current trunk. Let me know if reworking the patch will increase the probability of this getting applied to >0, which seems to be its current value.
          Hide
          Sam Brannen added a comment -

          Hi guys,

          Due to the increasing popularity of this issue, I'm assigning the fix version to 3.2 M2, and I'll take a look at in the M2 time frame.

          Regards,

          Sam

          Show
          Sam Brannen added a comment - Hi guys, Due to the increasing popularity of this issue, I'm assigning the fix version to 3.2 M2, and I'll take a look at in the M2 time frame. Regards, Sam
          Hide
          Juergen Hoeller added a comment -

          Kristian,

          We are currently working on some remaining TestContext framework enhancements for 3.2.2, and have been considering this request as well. We do realize the importance of parallel test execution and will definitely address this over the next few months. Note that we got 3.2.2 upcoming in early March as well as 4.0 M1 in late April.

          It would be great if you could submit a pull request on GitHub since that's much easier to consume for us these days. We might not be able to accept the full set of changes for 3.2.2 due to backwards compatibility issues but are definitely up for doing a thorough revision for the 4.0 line then - even for 4.0 M1 in April already.

          Juergen

          Show
          Juergen Hoeller added a comment - Kristian, We are currently working on some remaining TestContext framework enhancements for 3.2.2, and have been considering this request as well. We do realize the importance of parallel test execution and will definitely address this over the next few months. Note that we got 3.2.2 upcoming in early March as well as 4.0 M1 in late April. It would be great if you could submit a pull request on GitHub since that's much easier to consume for us these days. We might not be able to accept the full set of changes for 3.2.2 due to backwards compatibility issues but are definitely up for doing a thorough revision for the 4.0 line then - even for 4.0 M1 in April already. Juergen
          Hide
          Ian Brandt added a comment -

          The http://github.com/krosenvold/spring-test link above is broken. I'm guessing that https://github.com/krosenvold/org.springframework.test is its replacement? There is no pull request from that repo however.

          Would this fix include context caching across parallel tests? If I recall correctly when I last tried to use JUnit 4.8 parallel testing with Spring 3.0.7 integration tests I ran into two issues: deadlocks as mentioned above, and identical context locations not being reused across tests. No inheritance or profiles were being used. The tests were run via the Maven Failsafe plugin with the default fork mode of "once".

          Show
          Ian Brandt added a comment - The http://github.com/krosenvold/spring-test link above is broken. I'm guessing that https://github.com/krosenvold/org.springframework.test is its replacement? There is no pull request from that repo however. Would this fix include context caching across parallel tests? If I recall correctly when I last tried to use JUnit 4.8 parallel testing with Spring 3.0.7 integration tests I ran into two issues: deadlocks as mentioned above, and identical context locations not being reused across tests. No inheritance or profiles were being used. The tests were run via the Maven Failsafe plugin with the default fork mode of "once".

            People

            • Assignee:
              Sam Brannen
              Reporter:
              Kristian Rosenvold
              Last updater:
              Juergen Hoeller
            • Votes:
              34 Vote for this issue
              Watchers:
              31 Start watching this issue

              Dates

              • Created:
                Updated:
                Days since last comment:
                1 year, 2 weeks, 5 days ago

                Time Tracking

                Estimated:
                Original Estimate - 0d
                0d
                Remaining:
                Remaining Estimate - 0d
                0d
                Logged:
                Time Spent - 0.5h
                0.5h