Uploaded image for project: 'Spring Framework'
  1. Spring Framework
  2. SPR-7949

Improve performance of DefaultSingletonBeanRegistry.getSingletonNames(..) for high-concurrent-load applications

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 3.0.4, 3.0.5
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
    • Last commented by a User:
      true

      Description

      Under high-concurrent load situation, in an application using many prototype beans, contention occurs in DefaultSingletonBeanRegistry.getSingletonNames(..)

      This method has uses singletonObjects as a mutex for all map operations.
      Using a ReadWriteLock this contention could be removed, as in our case 99% of the calls here are read-only.

      We see hundreds of thread dumps like in a single jvm:

      java.lang.Thread.State: BLOCKED (on object monitor)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)

      • waiting to lock <0x00007fb8519b11b0> (a java.util.concurrent.ConcurrentHashMap)
        at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:452)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:317)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:296)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:250)
        at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1083)

      java.lang.Thread.State: BLOCKED (on object monitor)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingletonNames(DefaultSingletonBeanRegistry.java:277)

      • waiting to lock <0x00007fb856ab64d8> (a java.util.concurrent.ConcurrentHashMap)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:352)
        at org.springframework.beans.factory.BeanFactoryUtils.beanNamesForTypeIncludingAncestors(BeanFactoryUtils.java:185)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:829)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:744)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:703)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.resolvedCachedArgument(AutowiredAnnotationBeanPostProcessor.java:436)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.access$0(AutowiredAnnotationBeanPostProcessor.java:432)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:469)
        at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:84)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:283)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1074)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireBean(AbstractAutowireCapableBeanFactory.java:295)

      java.lang.Thread.State: BLOCKED (on object monitor)
      at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)

      • waiting to lock <0x00007fb8519b11b0> (a java.util.concurrent.ConcurrentHashMap)
        at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:452)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:317)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:296)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:250)
        at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1083)

        Issue Links

          Activity

          Hide
          ggn ggn added a comment -

          We are also facing the same issue where numerous threads are blocked on the synchronized block in getSingleton method.
          We are using spring 3.0.6 on Tomcat 6.0.35, JDK 1.6
          I downloaded spring 3.0.7 and 3.1 hoping to see some difference in that code, but i could not find any difference. Please help how i can get around with this issue. This issue is causing our server to hang once in a while.

          Please help.
          Thanks
          ggn

          Show
          ggn ggn added a comment - We are also facing the same issue where numerous threads are blocked on the synchronized block in getSingleton method. We are using spring 3.0.6 on Tomcat 6.0.35, JDK 1.6 I downloaded spring 3.0.7 and 3.1 hoping to see some difference in that code, but i could not find any difference. Please help how i can get around with this issue. This issue is causing our server to hang once in a while. Please help. Thanks ggn
          Hide
          rozhok Vladimir Rozhkov added a comment -

          Have same problems in our environment.
          Our application heavily use prototype beans - minimum 15 threads, total 40000 calls to getBean("..") and it becomes to very huge performance gap.

          Any chance to get this fix in 3.2M2? Also, could we raise priority to Normal?

          Thanks.

          Show
          rozhok Vladimir Rozhkov added a comment - Have same problems in our environment. Our application heavily use prototype beans - minimum 15 threads, total 40000 calls to getBean("..") and it becomes to very huge performance gap. Any chance to get this fix in 3.2M2? Also, could we raise priority to Normal? Thanks.
          Hide
          rozhok Vladimir Rozhkov added a comment -

          Also, why we trying to get singleton instance first, even if bean has prototype scope?

          protected <T> T doGetBean(
          			final String name, final Class<T> requiredType, final Object[] args, boolean typeCheckOnly)
          			throws BeansException {
           
          		final String beanName = transformedBeanName(name);
          		Object bean;
           
          		// Eagerly check singleton cache for manually registered singletons.
          		Object sharedInstance = getSingleton(beanName);
          		if (sharedInstance != null && args == null) {

          Is there reasons for it? I guess we should consider bean scope first to prevent execution of unnecessary singleton searching.

          Show
          rozhok Vladimir Rozhkov added a comment - Also, why we trying to get singleton instance first, even if bean has prototype scope? protected <T> T doGetBean( final String name, final Class<T> requiredType, final Object[] args, boolean typeCheckOnly) throws BeansException {   final String beanName = transformedBeanName(name); Object bean;   // Eagerly check singleton cache for manually registered singletons. Object sharedInstance = getSingleton(beanName); if (sharedInstance != null && args == null) { Is there reasons for it? I guess we should consider bean scope first to prevent execution of unnecessary singleton searching.
          Hide
          rozhok Vladimir Rozhkov added a comment -

          Attached patch doesn't actually fixes the problem. We still have tons of threads locked on writeLock insinde getSingleton.

          Also, after moving bean definition resolving to the top of the method, we still have lock in

          org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.registerDependentBean(DefaultSingletonBeanRegistry.java:367)
                  - waiting to lock <0x00002aaac9754d48> (a java.util.concurrent.ConcurrentHashMap)
                  at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:323)
                  at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106)
                  at org.springframework.beans.factory.support.ConstructorResolver.resolvePreparedArguments(ConstructorResolver.java:757)
                  at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:125)
                  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:993)
                  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:885)
                  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
                  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
                  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317)
                  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190)
                  at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1075)

          So Spring cannot handle accurately requests to getting prototype beans in 15 concurrent threads with 4000 invocations of getBean in each. This is huge gap.

          Show
          rozhok Vladimir Rozhkov added a comment - Attached patch doesn't actually fixes the problem. We still have tons of threads locked on writeLock insinde getSingleton. Also, after moving bean definition resolving to the top of the method, we still have lock in org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.registerDependentBean(DefaultSingletonBeanRegistry.java:367) - waiting to lock <0x00002aaac9754d48> (a java.util.concurrent.ConcurrentHashMap) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:323) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106) at org.springframework.beans.factory.support.ConstructorResolver.resolvePreparedArguments(ConstructorResolver.java:757) at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:125) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:993) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:885) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:190) at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1075) So Spring cannot handle accurately requests to getting prototype beans in 15 concurrent threads with 4000 invocations of getBean in each. This is huge gap.
          Hide
          cbeams Chris Beams added a comment -

          This issue duplicates SPR-6870, which is now resolved. Please give it a try (available now in 3.2.0.BUILD-SNAPSHOT).

          Show
          cbeams Chris Beams added a comment - This issue duplicates SPR-6870 , which is now resolved. Please give it a try (available now in 3.2.0.BUILD-SNAPSHOT).

            People

            • Assignee:
              cbeams Chris Beams
              Reporter:
              mck mck
              Last updater:
              Trevor Marshall
            • Votes:
              17 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since last comment:
                5 years, 6 days ago