Uploaded image for project: 'Spring Data MongoDB'
  1. Spring Data MongoDB
  2. DATAMONGO-830

NPE during cache warmup in CustomConversions

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.3 (Babbage SR2)
    • Component/s: Core
    • Labels:
      None

      Description

      The cache seems not to be able to handle heavy load during its warmup phase.
      We use 42 threads in parallel in performance test scenario (a lot of database operations in each thread) and got the NPE in one of five runs.

      Using a ConcurrentHashMap solves the problem.

      java.lang.NullPointerException
           at org.springframework.data.mongodb.core.convert.CustomConversions.getCustomReadTarget(CustomConversions.java:329)
           at org.springframework.data.mongodb.core.convert.CustomConversions.hasCustomReadTarget(CustomConversions.java:283)
           at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:200)
           at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:187)
           at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:183)
           at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:77)
           at org.springframework.data.mongodb.core.MongoTemplate$ReadDbObjectCallback.doWith(MongoTemplate.java:1975)
           at org.springframework.data.mongodb.core.MongoTemplate.executeFindOneInternal(MongoTemplate.java:1626)
           at org.springframework.data.mongodb.core.MongoTemplate.doFindOne(MongoTemplate.java:1446)
           at org.springframework.data.mongodb.core.MongoTemplate.findById(MongoTemplate.java:539)
           at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.findOne(SimpleMongoRepository.java:100)
           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:606)
           at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.executeMethodOn(RepositoryFactorySupport.java:358)
           at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:343)
           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
           at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
           at com.sun.proxy.$Proxy26.findOne(Unknown Source)
           at de.is24.offer.mongodb.evaluation.ReadYourOwnDataRightTask.countNumberOfInvalidResults(ReadYourOwnDataRightTask.java:34)
           at de.is24.offer.mongodb.evaluation.ReadYourOwnDataRightTask$$FastClassByCGLIB$$2b453a69.invoke(<generated>)
           at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
           at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:698)
           at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
           at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:95)
           at java.util.concurrent.FutureTask.run(FutureTask.java:262)
           at java.lang.Thread.run(Thread.java:744)

        Issue Links

          Activity

          Hide
          thomasd Thomas Darimont added a comment -

          We now use a CHM to cache the results of custom read target lookups in order to avoid having to traverse the readingPairs for every lookup. The use of should CHM also prevent potentially NPEs from being thrown if custom conversions are initialised in heavily threaded environments.

          Please revise.

          Show
          thomasd Thomas Darimont added a comment - We now use a CHM to cache the results of custom read target lookups in order to avoid having to traverse the readingPairs for every lookup. The use of should CHM also prevent potentially NPEs from being thrown if custom conversions are initialised in heavily threaded environments. Please revise.
          Hide
          thomasd Thomas Darimont added a comment -

          @Björn Voß would you mind giving this a spin?

          Cheers,
          Thomas

          Show
          thomasd Thomas Darimont added a comment - @Björn Voß would you mind giving this a spin? Cheers, Thomas
          Hide
          bjoern.voss Björn Voß added a comment - - edited

          looks very good:
          version 1.3.3 and 1.4.0-SNAPSHOT (without pull 117) produce the NPE always within 5 runs - mostly within 2 or 3 runs.
          The patch version runs 11 times without the NPE

          thx! + cheers
          Björn

          Show
          bjoern.voss Björn Voß added a comment - - edited looks very good: version 1.3.3 and 1.4.0-SNAPSHOT (without pull 117) produce the NPE always within 5 runs - mostly within 2 or 3 runs. The patch version runs 11 times without the NPE thx! + cheers Björn
          Hide
          olivergierke Oliver Gierke added a comment -

          I think you misunderstood, Björn. Thomas has implemented a more elaborate fix in a feature branch. We discussed this briefly today and I am going to merge this into master to give it a spin. However, I wouldn't really consider this fixed if it occurs after a couple of more iterations. We need this to be gone entirely.

          I'll ping you once we have a snapshot to try.

          Show
          olivergierke Oliver Gierke added a comment - I think you misunderstood, Björn. Thomas has implemented a more elaborate fix in a feature branch. We discussed this briefly today and I am going to merge this into master to give it a spin. However, I wouldn't really consider this fixed if it occurs after a couple of more iterations. We need this to be gone entirely. I'll ping you once we have a snapshot to try.
          Hide
          olivergierke Oliver Gierke added a comment -

          The snapshot is in place. Re-reading your comment I think I misunderstood . Would you mind elaborating on your findings? Were you saying that you still saw NPEs after you built with the patch in the pull request applied? In other words: do you still see the exceptions with a current snapshot?

          Show
          olivergierke Oliver Gierke added a comment - The snapshot is in place. Re-reading your comment I think I misunderstood . Would you mind elaborating on your findings? Were you saying that you still saw NPEs after you built with the patch in the pull request applied? In other words: do you still see the exceptions with a current snapshot?
          Hide
          bjoern.voss Björn Voß added a comment -

          Sorry for confusing you.

          I run our tests with the 1.3.3.RELEASE version and the 1.4.0-SNAPSHOT (yesterday morning github check out) several times and the NPE occurred always within 5 runs - mostly within 2 or 3 runs.

          Than I switch to the branch 'remotes/origin/issue/DATAMONGO-830' and in 11 runs everything was fine, so I think problem solved

          Show
          bjoern.voss Björn Voß added a comment - Sorry for confusing you. I run our tests with the 1.3.3.RELEASE version and the 1.4.0-SNAPSHOT (yesterday morning github check out) several times and the NPE occurred always within 5 runs - mostly within 2 or 3 runs. Than I switch to the branch 'remotes/origin/issue/ DATAMONGO-830 ' and in 11 runs everything was fine, so I think problem solved
          Hide
          olivergierke Oliver Gierke added a comment -

          That's great news! Thanks for the feedback.

          Show
          olivergierke Oliver Gierke added a comment - That's great news! Thanks for the feedback.

            People

            • Assignee:
              thomasd Thomas Darimont
              Reporter:
              bjoern.voss Björn Voß
              Last updater:
              Thomas Darimont
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile