Details

    • Type: New Feature
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0 M5
    • Fix Version/s: 1.4 M1 (Codd)
    • Component/s: Mapping
    • Labels:
      None
    • Environment:
      Any

      Description

      DbRef's appear to be loaded eagerly.

      Would be nice if there was support for storing DbRef's on a document but being able to lazy (or manually) load them.

        Issue Links

          Activity

          smozely Steve Mosley created issue -
          olivergierke Oliver Gierke made changes -
          Field Original Value New Value
          Original Estimate 0d [ 0 ]
          Remaining Estimate 0d [ 0 ]
          Fix Version/s 1.1 M1-Mongo [ 12638 ]
          Hide
          ssheehy Steven Sheehy added a comment -

          In addition to lazy loading, I would like to add my support for the manually loading of DBRefs. We are using DBRefs to point to multiple collections and using ObjectIds when the collection name cannot vary as recommended by Mongo (http://www.mongodb.org/display/DOCS/Database+References). As a result, we had to write a custom converter to bypass the automatic fetching of DBRefs that Spring Data defaults to. It would be nice if we didn't have to use a converter and could specify this behavior via the @DbRef annotation.

          Show
          ssheehy Steven Sheehy added a comment - In addition to lazy loading, I would like to add my support for the manually loading of DBRefs. We are using DBRefs to point to multiple collections and using ObjectIds when the collection name cannot vary as recommended by Mongo ( http://www.mongodb.org/display/DOCS/Database+References ). As a result, we had to write a custom converter to bypass the automatic fetching of DBRefs that Spring Data defaults to. It would be nice if we didn't have to use a converter and could specify this behavior via the @DbRef annotation.
          olivergierke Oliver Gierke made changes -
          Fix Version/s 1.1 M1 [ 12638 ]
          Fix Version/s 1.1 Backlog [ 12837 ]
          Hide
          ambeth Andrew Bethell added a comment -

          Is another way around this just to not have spring-data map certain fields? If using the MongoRepository - you could do this with the @Query annotation and specify which fields you want returned, and just omit the DBRefs field.

          Show
          ambeth Andrew Bethell added a comment - Is another way around this just to not have spring-data map certain fields? If using the MongoRepository - you could do this with the @Query annotation and specify which fields you want returned, and just omit the DBRefs field.
          Hide
          olivergierke Oliver Gierke added a comment -

          Until we eventually get to solve this issue in a nicer way, you can simply have DBRef properties in your domain objects (which is not ideal of course but get's the job done for now) and resolve them manually.

          class Account {
            @DBRef
            DBRef owner;
          }
           
          Account account = …;
          MongoConverter converter = …;
          User user = converter.read(User.class, account.owner.fetch());

          Show
          olivergierke Oliver Gierke added a comment - Until we eventually get to solve this issue in a nicer way, you can simply have DBRef properties in your domain objects (which is not ideal of course but get's the job done for now) and resolve them manually. class Account { @DBRef DBRef owner; }   Account account = …; MongoConverter converter = …; User user = converter.read(User. class , account.owner.fetch());
          Hide
          titogeo Tito George added a comment -

          Work around with DBRef did not work. It failed while saving.
          My code is like this

          Request.java

          class Request{
          @Id
          private ObjectId id;
          @DBRef
          private com.mongodb.DBRef who;
          @DBRef
          private com.mongodb.DBRef whom;
          @DBRef
          private com.mongodb.DBRef group;
          }

          myservice.java

          AddRequest addRequest = new AddRequest();
          addRequest.setWho(new DBRef(mongoTemplate.getDb(), "users" ,who.getId()));
          addRequest.setWhom(new DBRef(mongoTemplate.getDb(), "users" ,whom.getId()));
          addRequest = addRequestDataRepository.save(addRequest);

          Failed with below ST

          java.lang.StackOverflowError
          	at sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.hashCode(WildcardTypeImpl.java:212)
          	at java.util.Arrays.hashCode(Arrays.java:3655)
          	at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.hashCode(ParameterizedTypeImpl.java:190)
          	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)
          	at org.springframework.data.util.TypeDiscoverer.hashCode(TypeDiscoverer.java:367)
          	at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79)
          	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)
          	at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79)
          	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)

          From Eclipse when StackOverflowError is caught.

          Thread [main] (Suspended (exception StackOverflowError))
          	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 307
          	AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413	                  
          	ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570	                  
          	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283
          	AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413                          
          	ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570	                  
          	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283

          Show
          titogeo Tito George added a comment - Work around with DBRef did not work. It failed while saving. My code is like this Request.java class Request{ @Id private ObjectId id; @DBRef private com.mongodb.DBRef who; @DBRef private com.mongodb.DBRef whom; @DBRef private com.mongodb.DBRef group; } myservice.java AddRequest addRequest = new AddRequest(); addRequest.setWho(new DBRef(mongoTemplate.getDb(), "users" ,who.getId())); addRequest.setWhom(new DBRef(mongoTemplate.getDb(), "users" ,whom.getId())); addRequest = addRequestDataRepository.save(addRequest); Failed with below ST java.lang.StackOverflowError at sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.hashCode(WildcardTypeImpl.java:212) at java.util.Arrays.hashCode(Arrays.java:3655) at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.hashCode(ParameterizedTypeImpl.java:190) at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336) at org.springframework.data.util.TypeDiscoverer.hashCode(TypeDiscoverer.java:367) at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79) at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336) at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79) at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336) From Eclipse when StackOverflowError is caught. Thread [main] (Suspended (exception StackOverflowError)) MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 307 AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413 ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570 MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283 AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413 ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570 MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283
          Hide
          davideanderson David Anderson added a comment -

          Tito,

          Oliver's Account class had an error in it, it should have been:

          class Account {
            @DBRef
            User owner;
          }

          Show
          davideanderson David Anderson added a comment - Tito, Oliver's Account class had an error in it, it should have been: class Account { @DBRef User owner; }
          Hide
          titogeo Tito George added a comment -

          David,
          I have a class called User, belongs to collection users. User contains a list called requests(another collection called request).
          Class request was like below initially.

          Request.java

          class Request{
          @Id
          private ObjectId id;
          @DBRef
          private User who;
          @DBRef
          private User whom;
          @DBRef
          private User group;
          }
           
          User.java
          User{
          @DBRef
          List<Requests> requests;
          }

          Say an instance of User1 can contain list requests and request object can contain User1 (in its fields who or whom). I am able to save User as well as Request. But while querying i am getting https://jira.springsource.org/browse/DATAMONGO-488.

          So I was hoping Oliver's workaround would solve this. But its failing wile saving.

          Show
          titogeo Tito George added a comment - David, I have a class called User, belongs to collection users. User contains a list called requests(another collection called request). Class request was like below initially. Request.java class Request{ @Id private ObjectId id; @DBRef private User who; @DBRef private User whom; @DBRef private User group; }   User.java User{ @DBRef List<Requests> requests; } Say an instance of User1 can contain list requests and request object can contain User1 (in its fields who or whom). I am able to save User as well as Request. But while querying i am getting https://jira.springsource.org/browse/DATAMONGO-488 . So I was hoping Oliver's workaround would solve this. But its failing wile saving.
          olivergierke Oliver Gierke made changes -
          Fix Version/s Backlog [ 12836 ]
          Fix Version/s 1.1 Backlog [ 12837 ]
          Hide
          sebastian-julius Sebastian Julius added a comment -

          Are there any plans when this feature will be integrated? We're currently creating 4-times more requests to the DB then required. Therefore it'd be brilliant to have lazy loading. Thanks in advance

          Show
          sebastian-julius Sebastian Julius added a comment - Are there any plans when this feature will be integrated? We're currently creating 4-times more requests to the DB then required. Therefore it'd be brilliant to have lazy loading. Thanks in advance
          tmarshall Trevor Marshall made changes -
          Workflow SPR Workflow [ 51018 ] SPR Workflow - notify Spring Data HipChat [ 82499 ]
          Hide
          regis.leray regis added a comment - - edited

          The morphia project also provided this feature.

          https://github.com/mongodb/morphia/blob/master/morphia/src/main/java/org/mongodb/morphia/annotations/Reference.java

           
          @Entity("employees")
          class Employee {
            // auto-generated, if not set (see ObjectId)
            @Id ObjectId id;
           
           
            //refs are stored**, and loaded automatically
            @Reference(lazy = true)
            List<Employee> underlings = new ArrayList<Employee>();
           
          }

          Why the spring data team doesnt reply ?

          Hey guys the ticket is 3 years OLD....

          Show
          regis.leray regis added a comment - - edited The morphia project also provided this feature. https://github.com/mongodb/morphia/blob/master/morphia/src/main/java/org/mongodb/morphia/annotations/Reference.java @Entity ( "employees" ) class Employee { // auto-generated, if not set (see ObjectId) @Id ObjectId id;     //refs are stored**, and loaded automatically @Reference (lazy = true ) List<Employee> underlings = new ArrayList<Employee>();   } Why the spring data team doesnt reply ? Hey guys the ticket is 3 years OLD....
          thomasd Thomas Darimont made changes -
          Sprint The Road to Codd M1 [ 23 ]
          thomasd Thomas Darimont made changes -
          Rank Ranked higher
          Hide
          mikkelbd Mikkel Dan-Rognlie added a comment -

          I see that @thomasd added sprint "The Road to Codd M1" to this issue. According to https://github.com/spring-projects/spring-data-commons/wiki/Release-Train-Codd, it seems we will probable have this feature in the near future. Can you guys confirm that you will be working on it? That would be awesome!

          Show
          mikkelbd Mikkel Dan-Rognlie added a comment - I see that @thomasd added sprint "The Road to Codd M1" to this issue. According to https://github.com/spring-projects/spring-data-commons/wiki/Release-Train-Codd , it seems we will probable have this feature in the near future. Can you guys confirm that you will be working on it? That would be awesome!
          Hide
          olivergierke Oliver Gierke added a comment -

          Yes.

          Show
          olivergierke Oliver Gierke added a comment - Yes.
          thomasd Thomas Darimont made changes -
          Assignee Oliver Gierke [ olivergierke ] Thomas Darimont [ thomasd ]
          thomasd Thomas Darimont made changes -
          Status Open [ 1 ] Investigating [ 10003 ]
          Hide
          thomasd Thomas Darimont added a comment - - edited

          How should the lazy loading behaviour be enabled and what should be the default?

          I would propose something like this, with lazy=false as the default:

          @DBRef(lazy = true) List<User> fans;

          Should we support lazy loading for all possible concrete types or just interfaces types or even just for collection types?
          If the raw type has to be an interface we could simply return a JDK Proxy with an appropriate InvocationHandler that checks an "initialised"-flag on every method call. If not yet initialised the proxy will resolve the DBRef, store the result in a field and return that on subsequent invocations.

          If we have to support arbitrary types, then we have to enforce some restrictions, e.g. like the fields of the type needn't be accessed directly - just via accessor methods. We could then generate a custom subclass of the given type (via CGLIB / ASM + (maybe) Objenesis for object construction) where we inject a proper initialisation check into the relevant method bodies and perform the required lazy loading logic if necessary and then delegate to the base class implementation.

          Show
          thomasd Thomas Darimont added a comment - - edited How should the lazy loading behaviour be enabled and what should be the default? I would propose something like this, with lazy=false as the default: @DBRef (lazy = true ) List<User> fans; Should we support lazy loading for all possible concrete types or just interfaces types or even just for collection types? If the raw type has to be an interface we could simply return a JDK Proxy with an appropriate InvocationHandler that checks an "initialised"-flag on every method call. If not yet initialised the proxy will resolve the DBRef, store the result in a field and return that on subsequent invocations. If we have to support arbitrary types, then we have to enforce some restrictions, e.g. like the fields of the type needn't be accessed directly - just via accessor methods. We could then generate a custom subclass of the given type (via CGLIB / ASM + (maybe) Objenesis for object construction) where we inject a proper initialisation check into the relevant method bodies and perform the required lazy loading logic if necessary and then delegate to the base class implementation.
          Show
          thomasd Thomas Darimont added a comment - - edited Added Initial POC: https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348
          Hide
          regis.leray regis added a comment -

          We should stick by default to eager loading (like you propose), we will still compatible with the old behavior. And not breaking apps when they are going to update the version of spring mongo.

          Show
          regis.leray regis added a comment - We should stick by default to eager loading (like you propose), we will still compatible with the old behavior. And not breaking apps when they are going to update the version of spring mongo.
          Hide
          mikkelbd Mikkel Dan-Rognlie added a comment - - edited

          I also think eager loading should be the default. Maybe we could have the possibility to change it globally as a property on MappingMongoConverter. Like for instance the disable-validation attribute? Then one could override it with individual @DBRef(lazy = true|false) on entities.

          If it's not too complicated I think it would be nice to support arbitrary types using CGLIB / ASM. But as you said, maybe it is not desirable with the restrictions on field access (could be perceived as 'magical' etc). If it makes the implementation easier it would be ok to require an interface in order to use JDK proxies.

          I don´t think it should be limited to collection types. If you in a query gets a long list of documents, where each doc has deep nested documents, which at arbitrary levels of nesting has one @DBRef, it is still a lot of extra fetches even though the individual @DBRef fields are not collections.

          Show
          mikkelbd Mikkel Dan-Rognlie added a comment - - edited I also think eager loading should be the default. Maybe we could have the possibility to change it globally as a property on MappingMongoConverter. Like for instance the disable-validation attribute? Then one could override it with individual @DBRef(lazy = true|false) on entities. If it's not too complicated I think it would be nice to support arbitrary types using CGLIB / ASM. But as you said, maybe it is not desirable with the restrictions on field access (could be perceived as 'magical' etc). If it makes the implementation easier it would be ok to require an interface in order to use JDK proxies. I don´t think it should be limited to collection types. If you in a query gets a long list of documents, where each doc has deep nested documents, which at arbitrary levels of nesting has one @DBRef, it is still a lot of extra fetches even though the individual @DBRef fields are not collections.
          thomasd Thomas Darimont made changes -
          Fix Version/s 1.4 M1 [ 14249 ]
          Fix Version/s Backlog [ 12836 ]
          olivergierke Oliver Gierke made changes -
          Assignee Thomas Darimont [ thomasd ] Oliver Gierke [ olivergierke ]
          olivergierke Oliver Gierke made changes -
          Status Investigating [ 10003 ] In Progress [ 3 ]
          olivergierke Oliver Gierke made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          Hide
          thomasd Thomas Darimont added a comment - - edited

          Hi,

          could you please give us some feedback on our current POC for lazy loading of MongoDB associations:
          https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348

          At the moment we support lazy loading for interface, concrete types, as far as they provide an appropriate parameterless ctor.
          The @PersistenceConstructor annotation is supported as well.
          If there is a need to support arbitrary constructors, we could also try to back-port the Objenesis stuff from
          Spring Framework 4.0 (https://github.com/spring-projects/spring-framework/pull/327)
          The actual loading is performed only once and is triggered when a method on the proxy is called.

          If you want to give it a try, then please do the following:

          • Change the version of your mongodb dependency to: 1.4.0.DATAMONGO-348-SNAPSHOT
          • Add the spring source snapshot-repository

            <repositories>
                <repository>
                    <id>repository.springsource.snapshot</id>
                    <name>SpringSource Snapshot Repository</name>
                    <url>http://repo.springsource.org/snapshot</url>
                </repository>
            </repositories>

          Cheers,
          Thomas

          Show
          thomasd Thomas Darimont added a comment - - edited Hi, could you please give us some feedback on our current POC for lazy loading of MongoDB associations: https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348 At the moment we support lazy loading for interface, concrete types, as far as they provide an appropriate parameterless ctor. The @PersistenceConstructor annotation is supported as well. If there is a need to support arbitrary constructors, we could also try to back-port the Objenesis stuff from Spring Framework 4.0 ( https://github.com/spring-projects/spring-framework/pull/327 ) The actual loading is performed only once and is triggered when a method on the proxy is called. If you want to give it a try, then please do the following: Change the version of your mongodb dependency to: 1.4.0. DATAMONGO-348 -SNAPSHOT Add the spring source snapshot-repository < repositories > < repository > < id >repository.springsource.snapshot</ id > < name >SpringSource Snapshot Repository</ name > < url >http://repo.springsource.org/snapshot</ url > </ repository > </ repositories > Cheers, Thomas
          thomasd Thomas Darimont made changes -
          Assignee Oliver Gierke [ olivergierke ] Thomas Darimont [ thomasd ]
          thomasd Thomas Darimont made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          thomasd Thomas Darimont made changes -
          Status In Progress [ 3 ] Investigating [ 10003 ]
          thomasd Thomas Darimont made changes -
          Rank Ranked lower
          thomasd Thomas Darimont made changes -
          Status Investigating [ 10003 ] Waiting for Feedback [ 10002 ]
          olivergierke Oliver Gierke made changes -
          Assignee Thomas Darimont [ thomasd ] Oliver Gierke [ olivergierke ]
          olivergierke Oliver Gierke made changes -
          Status Waiting for Feedback [ 10002 ] In Progress [ 3 ]
          olivergierke Oliver Gierke made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Assignee Oliver Gierke [ olivergierke ] Thomas Darimont [ thomasd ]
          Resolution Fixed [ 1 ]
          Hide
          olivergierke Oliver Gierke added a comment -

          This merged into master. I added an optional dependency to Objenesis to be able to create proxies for classes without a default constructor. Feedback appreciated.

          Show
          olivergierke Oliver Gierke added a comment - This merged into master. I added an optional dependency to Objenesis to be able to create proxies for classes without a default constructor. Feedback appreciated.
          olivergierke Oliver Gierke made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          tmarshall Trevor Marshall made changes -
          Workflow SPR Workflow - notify Spring Data HipChat [ 82499 ] DATA Workflow [ 97027 ]
          cstrobl Christoph Strobl made changes -
          Link This issue relates to DATAMONGO-488 [ DATAMONGO-488 ]
          mp911de Mark Paluch made changes -
          Link This issue relates to DATAREDIS-470 [ DATAREDIS-470 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Investigating Investigating
          673d 13h 30m 1 Thomas Darimont 15/Oct/13 7:45 AM
          Investigating Investigating In Progress In Progress
          6d 21h 48m 1 Oliver Gierke 22/Oct/13 5:33 AM
          In Progress In Progress Open Open
          3s 1 Oliver Gierke 22/Oct/13 5:33 AM
          Open Open In Progress In Progress
          6d 1h 26m 1 Thomas Darimont 28/Oct/13 7:00 AM
          In Progress In Progress Investigating Investigating
          34s 1 Thomas Darimont 28/Oct/13 7:00 AM
          Investigating Investigating Waiting for Feedback Waiting for Feedback
          42s 1 Thomas Darimont 28/Oct/13 7:01 AM
          Waiting for Feedback Waiting for Feedback In Progress In Progress
          6d 21h 47m 1 Oliver Gierke 04/Nov/13 4:49 AM
          In Progress In Progress Resolved Resolved
          6h 27m 1 Oliver Gierke 04/Nov/13 11:16 AM
          Resolved Resolved Closed Closed
          16d 19h 6m 1 Oliver Gierke 21/Nov/13 6:23 AM

            People

            • Assignee:
              thomasd Thomas Darimont
              Reporter:
              smozely Steve Mosley
              Last updater:
              Mark Paluch
            • Votes:
              26 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile