Details

    • Type: New Feature
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0 M5
    • Fix Version/s: 1.4 M1 (Codd)
    • Component/s: Mapping
    • Labels:
      None
    • Environment:
      Any

      Description

      DbRef's appear to be loaded eagerly.

      Would be nice if there was support for storing DbRef's on a document but being able to lazy (or manually) load them.

        Issue Links

          Activity

          Hide
          ssheehy Steven Sheehy added a comment -

          In addition to lazy loading, I would like to add my support for the manually loading of DBRefs. We are using DBRefs to point to multiple collections and using ObjectIds when the collection name cannot vary as recommended by Mongo (http://www.mongodb.org/display/DOCS/Database+References). As a result, we had to write a custom converter to bypass the automatic fetching of DBRefs that Spring Data defaults to. It would be nice if we didn't have to use a converter and could specify this behavior via the @DbRef annotation.

          Show
          ssheehy Steven Sheehy added a comment - In addition to lazy loading, I would like to add my support for the manually loading of DBRefs. We are using DBRefs to point to multiple collections and using ObjectIds when the collection name cannot vary as recommended by Mongo ( http://www.mongodb.org/display/DOCS/Database+References ). As a result, we had to write a custom converter to bypass the automatic fetching of DBRefs that Spring Data defaults to. It would be nice if we didn't have to use a converter and could specify this behavior via the @DbRef annotation.
          Hide
          ambeth Andrew Bethell added a comment -

          Is another way around this just to not have spring-data map certain fields? If using the MongoRepository - you could do this with the @Query annotation and specify which fields you want returned, and just omit the DBRefs field.

          Show
          ambeth Andrew Bethell added a comment - Is another way around this just to not have spring-data map certain fields? If using the MongoRepository - you could do this with the @Query annotation and specify which fields you want returned, and just omit the DBRefs field.
          Hide
          olivergierke Oliver Gierke added a comment -

          Until we eventually get to solve this issue in a nicer way, you can simply have DBRef properties in your domain objects (which is not ideal of course but get's the job done for now) and resolve them manually.

          class Account {
            @DBRef
            DBRef owner;
          }
           
          Account account = …;
          MongoConverter converter = …;
          User user = converter.read(User.class, account.owner.fetch());

          Show
          olivergierke Oliver Gierke added a comment - Until we eventually get to solve this issue in a nicer way, you can simply have DBRef properties in your domain objects (which is not ideal of course but get's the job done for now) and resolve them manually. class Account { @DBRef DBRef owner; }   Account account = …; MongoConverter converter = …; User user = converter.read(User. class , account.owner.fetch());
          Hide
          titogeo Tito George added a comment -

          Work around with DBRef did not work. It failed while saving.
          My code is like this

          Request.java

          class Request{
          @Id
          private ObjectId id;
          @DBRef
          private com.mongodb.DBRef who;
          @DBRef
          private com.mongodb.DBRef whom;
          @DBRef
          private com.mongodb.DBRef group;
          }

          myservice.java

          AddRequest addRequest = new AddRequest();
          addRequest.setWho(new DBRef(mongoTemplate.getDb(), "users" ,who.getId()));
          addRequest.setWhom(new DBRef(mongoTemplate.getDb(), "users" ,whom.getId()));
          addRequest = addRequestDataRepository.save(addRequest);

          Failed with below ST

          java.lang.StackOverflowError
          	at sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.hashCode(WildcardTypeImpl.java:212)
          	at java.util.Arrays.hashCode(Arrays.java:3655)
          	at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.hashCode(ParameterizedTypeImpl.java:190)
          	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)
          	at org.springframework.data.util.TypeDiscoverer.hashCode(TypeDiscoverer.java:367)
          	at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79)
          	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)
          	at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79)
          	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)

          From Eclipse when StackOverflowError is caught.

          Thread [main] (Suspended (exception StackOverflowError))
          	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 307
          	AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413	                  
          	ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570	                  
          	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283
          	AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413                          
          	ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570	                  
          	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283

          Show
          titogeo Tito George added a comment - Work around with DBRef did not work. It failed while saving. My code is like this Request.java class Request{ @Id private ObjectId id; @DBRef private com.mongodb.DBRef who; @DBRef private com.mongodb.DBRef whom; @DBRef private com.mongodb.DBRef group; } myservice.java AddRequest addRequest = new AddRequest(); addRequest.setWho(new DBRef(mongoTemplate.getDb(), "users" ,who.getId())); addRequest.setWhom(new DBRef(mongoTemplate.getDb(), "users" ,whom.getId())); addRequest = addRequestDataRepository.save(addRequest); Failed with below ST java.lang.StackOverflowError at sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.hashCode(WildcardTypeImpl.java:212) at java.util.Arrays.hashCode(Arrays.java:3655) at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.hashCode(ParameterizedTypeImpl.java:190) at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336) at org.springframework.data.util.TypeDiscoverer.hashCode(TypeDiscoverer.java:367) at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79) at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336) at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79) at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336) From Eclipse when StackOverflowError is caught. Thread [main] (Suspended (exception StackOverflowError)) MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 307 AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413 ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570 MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283 AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413 ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570 MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283
          Hide
          davideanderson David Anderson added a comment -

          Tito,

          Oliver's Account class had an error in it, it should have been:

          class Account {
            @DBRef
            User owner;
          }

          Show
          davideanderson David Anderson added a comment - Tito, Oliver's Account class had an error in it, it should have been: class Account { @DBRef User owner; }
          Hide
          titogeo Tito George added a comment -

          David,
          I have a class called User, belongs to collection users. User contains a list called requests(another collection called request).
          Class request was like below initially.

          Request.java

          class Request{
          @Id
          private ObjectId id;
          @DBRef
          private User who;
          @DBRef
          private User whom;
          @DBRef
          private User group;
          }
           
          User.java
          User{
          @DBRef
          List<Requests> requests;
          }

          Say an instance of User1 can contain list requests and request object can contain User1 (in its fields who or whom). I am able to save User as well as Request. But while querying i am getting https://jira.springsource.org/browse/DATAMONGO-488.

          So I was hoping Oliver's workaround would solve this. But its failing wile saving.

          Show
          titogeo Tito George added a comment - David, I have a class called User, belongs to collection users. User contains a list called requests(another collection called request). Class request was like below initially. Request.java class Request{ @Id private ObjectId id; @DBRef private User who; @DBRef private User whom; @DBRef private User group; }   User.java User{ @DBRef List<Requests> requests; } Say an instance of User1 can contain list requests and request object can contain User1 (in its fields who or whom). I am able to save User as well as Request. But while querying i am getting https://jira.springsource.org/browse/DATAMONGO-488 . So I was hoping Oliver's workaround would solve this. But its failing wile saving.
          Hide
          sebastian-julius Sebastian Julius added a comment -

          Are there any plans when this feature will be integrated? We're currently creating 4-times more requests to the DB then required. Therefore it'd be brilliant to have lazy loading. Thanks in advance

          Show
          sebastian-julius Sebastian Julius added a comment - Are there any plans when this feature will be integrated? We're currently creating 4-times more requests to the DB then required. Therefore it'd be brilliant to have lazy loading. Thanks in advance
          Hide
          regis.leray regis added a comment - - edited

          The morphia project also provided this feature.

          https://github.com/mongodb/morphia/blob/master/morphia/src/main/java/org/mongodb/morphia/annotations/Reference.java

           
          @Entity("employees")
          class Employee {
            // auto-generated, if not set (see ObjectId)
            @Id ObjectId id;
           
           
            //refs are stored**, and loaded automatically
            @Reference(lazy = true)
            List<Employee> underlings = new ArrayList<Employee>();
           
          }

          Why the spring data team doesnt reply ?

          Hey guys the ticket is 3 years OLD....

          Show
          regis.leray regis added a comment - - edited The morphia project also provided this feature. https://github.com/mongodb/morphia/blob/master/morphia/src/main/java/org/mongodb/morphia/annotations/Reference.java @Entity ( "employees" ) class Employee { // auto-generated, if not set (see ObjectId) @Id ObjectId id;     //refs are stored**, and loaded automatically @Reference (lazy = true ) List<Employee> underlings = new ArrayList<Employee>();   } Why the spring data team doesnt reply ? Hey guys the ticket is 3 years OLD....
          Hide
          mikkelbd Mikkel Dan-Rognlie added a comment -

          I see that @thomasd added sprint "The Road to Codd M1" to this issue. According to https://github.com/spring-projects/spring-data-commons/wiki/Release-Train-Codd, it seems we will probable have this feature in the near future. Can you guys confirm that you will be working on it? That would be awesome!

          Show
          mikkelbd Mikkel Dan-Rognlie added a comment - I see that @thomasd added sprint "The Road to Codd M1" to this issue. According to https://github.com/spring-projects/spring-data-commons/wiki/Release-Train-Codd , it seems we will probable have this feature in the near future. Can you guys confirm that you will be working on it? That would be awesome!
          Hide
          olivergierke Oliver Gierke added a comment -

          Yes.

          Show
          olivergierke Oliver Gierke added a comment - Yes.
          Hide
          thomasd Thomas Darimont added a comment - - edited

          How should the lazy loading behaviour be enabled and what should be the default?

          I would propose something like this, with lazy=false as the default:

          @DBRef(lazy = true) List<User> fans;

          Should we support lazy loading for all possible concrete types or just interfaces types or even just for collection types?
          If the raw type has to be an interface we could simply return a JDK Proxy with an appropriate InvocationHandler that checks an "initialised"-flag on every method call. If not yet initialised the proxy will resolve the DBRef, store the result in a field and return that on subsequent invocations.

          If we have to support arbitrary types, then we have to enforce some restrictions, e.g. like the fields of the type needn't be accessed directly - just via accessor methods. We could then generate a custom subclass of the given type (via CGLIB / ASM + (maybe) Objenesis for object construction) where we inject a proper initialisation check into the relevant method bodies and perform the required lazy loading logic if necessary and then delegate to the base class implementation.

          Show
          thomasd Thomas Darimont added a comment - - edited How should the lazy loading behaviour be enabled and what should be the default? I would propose something like this, with lazy=false as the default: @DBRef (lazy = true ) List<User> fans; Should we support lazy loading for all possible concrete types or just interfaces types or even just for collection types? If the raw type has to be an interface we could simply return a JDK Proxy with an appropriate InvocationHandler that checks an "initialised"-flag on every method call. If not yet initialised the proxy will resolve the DBRef, store the result in a field and return that on subsequent invocations. If we have to support arbitrary types, then we have to enforce some restrictions, e.g. like the fields of the type needn't be accessed directly - just via accessor methods. We could then generate a custom subclass of the given type (via CGLIB / ASM + (maybe) Objenesis for object construction) where we inject a proper initialisation check into the relevant method bodies and perform the required lazy loading logic if necessary and then delegate to the base class implementation.
          Show
          thomasd Thomas Darimont added a comment - - edited Added Initial POC: https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348
          Hide
          regis.leray regis added a comment -

          We should stick by default to eager loading (like you propose), we will still compatible with the old behavior. And not breaking apps when they are going to update the version of spring mongo.

          Show
          regis.leray regis added a comment - We should stick by default to eager loading (like you propose), we will still compatible with the old behavior. And not breaking apps when they are going to update the version of spring mongo.
          Hide
          mikkelbd Mikkel Dan-Rognlie added a comment - - edited

          I also think eager loading should be the default. Maybe we could have the possibility to change it globally as a property on MappingMongoConverter. Like for instance the disable-validation attribute? Then one could override it with individual @DBRef(lazy = true|false) on entities.

          If it's not too complicated I think it would be nice to support arbitrary types using CGLIB / ASM. But as you said, maybe it is not desirable with the restrictions on field access (could be perceived as 'magical' etc). If it makes the implementation easier it would be ok to require an interface in order to use JDK proxies.

          I don´t think it should be limited to collection types. If you in a query gets a long list of documents, where each doc has deep nested documents, which at arbitrary levels of nesting has one @DBRef, it is still a lot of extra fetches even though the individual @DBRef fields are not collections.

          Show
          mikkelbd Mikkel Dan-Rognlie added a comment - - edited I also think eager loading should be the default. Maybe we could have the possibility to change it globally as a property on MappingMongoConverter. Like for instance the disable-validation attribute? Then one could override it with individual @DBRef(lazy = true|false) on entities. If it's not too complicated I think it would be nice to support arbitrary types using CGLIB / ASM. But as you said, maybe it is not desirable with the restrictions on field access (could be perceived as 'magical' etc). If it makes the implementation easier it would be ok to require an interface in order to use JDK proxies. I don´t think it should be limited to collection types. If you in a query gets a long list of documents, where each doc has deep nested documents, which at arbitrary levels of nesting has one @DBRef, it is still a lot of extra fetches even though the individual @DBRef fields are not collections.
          Hide
          thomasd Thomas Darimont added a comment - - edited

          Hi,

          could you please give us some feedback on our current POC for lazy loading of MongoDB associations:
          https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348

          At the moment we support lazy loading for interface, concrete types, as far as they provide an appropriate parameterless ctor.
          The @PersistenceConstructor annotation is supported as well.
          If there is a need to support arbitrary constructors, we could also try to back-port the Objenesis stuff from
          Spring Framework 4.0 (https://github.com/spring-projects/spring-framework/pull/327)
          The actual loading is performed only once and is triggered when a method on the proxy is called.

          If you want to give it a try, then please do the following:

          • Change the version of your mongodb dependency to: 1.4.0.DATAMONGO-348-SNAPSHOT
          • Add the spring source snapshot-repository

            <repositories>
                <repository>
                    <id>repository.springsource.snapshot</id>
                    <name>SpringSource Snapshot Repository</name>
                    <url>http://repo.springsource.org/snapshot</url>
                </repository>
            </repositories>

          Cheers,
          Thomas

          Show
          thomasd Thomas Darimont added a comment - - edited Hi, could you please give us some feedback on our current POC for lazy loading of MongoDB associations: https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348 At the moment we support lazy loading for interface, concrete types, as far as they provide an appropriate parameterless ctor. The @PersistenceConstructor annotation is supported as well. If there is a need to support arbitrary constructors, we could also try to back-port the Objenesis stuff from Spring Framework 4.0 ( https://github.com/spring-projects/spring-framework/pull/327 ) The actual loading is performed only once and is triggered when a method on the proxy is called. If you want to give it a try, then please do the following: Change the version of your mongodb dependency to: 1.4.0. DATAMONGO-348 -SNAPSHOT Add the spring source snapshot-repository < repositories > < repository > < id >repository.springsource.snapshot</ id > < name >SpringSource Snapshot Repository</ name > < url >http://repo.springsource.org/snapshot</ url > </ repository > </ repositories > Cheers, Thomas
          Hide
          olivergierke Oliver Gierke added a comment -

          This merged into master. I added an optional dependency to Objenesis to be able to create proxies for classes without a default constructor. Feedback appreciated.

          Show
          olivergierke Oliver Gierke added a comment - This merged into master. I added an optional dependency to Objenesis to be able to create proxies for classes without a default constructor. Feedback appreciated.

            People

            • Assignee:
              thomasd Thomas Darimont
              Reporter:
              smozely Steve Mosley
              Last updater:
              Mark Paluch
            • Votes:
              26 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile