[DATAMONGO-348] Lazy Load for DbRef Created: 11/Dec/11  Updated: 26/Feb/16  Resolved: 04/Nov/13

Status: Closed
Project: Spring Data MongoDB
Component/s: Mapping
Affects Version/s: 1.0 M5
Fix Version/s: 1.4 M1 (Codd)

Type: New Feature Priority: Minor
Reporter: Steve Mosley Assignee: Thomas Darimont
Resolution: Fixed Votes: 26
Labels: None
Remaining Estimate: 0d
Time Spent: Not Specified
Original Estimate: 0d
Environment:

Any


Issue Links:
Relate
relates to DATAMONGO-488 Stackoverflow when querying document ... Resolved
relates to DATAREDIS-470 Lazy Load for @Reference Open
Reference URL: http://forum.springsource.org/showthread.php?113321-Support-for-lazy-loading-in-Spring-Data-Document
Last updater: Mark Paluch
Sprint: The Road to Codd M1

 Description   

DbRef's appear to be loaded eagerly.

Would be nice if there was support for storing DbRef's on a document but being able to lazy (or manually) load them.



 Comments   
Comment by Steven Sheehy [ 02/Apr/12 ]

In addition to lazy loading, I would like to add my support for the manually loading of DBRefs. We are using DBRefs to point to multiple collections and using ObjectIds when the collection name cannot vary as recommended by Mongo (http://www.mongodb.org/display/DOCS/Database+References). As a result, we had to write a custom converter to bypass the automatic fetching of DBRefs that Spring Data defaults to. It would be nice if we didn't have to use a converter and could specify this behavior via the @DbRef annotation.

Comment by Andrew Bethell [ 26/Jun/12 ]

Is another way around this just to not have spring-data map certain fields? If using the MongoRepository - you could do this with the @Query annotation and specify which fields you want returned, and just omit the DBRefs field.

Comment by Oliver Gierke [ 26/Jun/12 ]

Until we eventually get to solve this issue in a nicer way, you can simply have DBRef properties in your domain objects (which is not ideal of course but get's the job done for now) and resolve them manually.

class Account {
  @DBRef
  DBRef owner;
}
 
Account account = …;
MongoConverter converter = …;
User user = converter.read(User.class, account.owner.fetch());

Comment by Tito George [ 06/Sep/12 ]

Work around with DBRef did not work. It failed while saving.
My code is like this

Request.java

class Request{
@Id
private ObjectId id;
@DBRef
private com.mongodb.DBRef who;
@DBRef
private com.mongodb.DBRef whom;
@DBRef
private com.mongodb.DBRef group;
}

myservice.java

AddRequest addRequest = new AddRequest();
addRequest.setWho(new DBRef(mongoTemplate.getDb(), "users" ,who.getId()));
addRequest.setWhom(new DBRef(mongoTemplate.getDb(), "users" ,whom.getId()));
addRequest = addRequestDataRepository.save(addRequest);

Failed with below ST

java.lang.StackOverflowError
	at sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.hashCode(WildcardTypeImpl.java:212)
	at java.util.Arrays.hashCode(Arrays.java:3655)
	at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.hashCode(ParameterizedTypeImpl.java:190)
	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)
	at org.springframework.data.util.TypeDiscoverer.hashCode(TypeDiscoverer.java:367)
	at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79)
	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)
	at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79)
	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)

From Eclipse when StackOverflowError is caught.

Thread [main] (Suspended (exception StackOverflowError))
	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 307
	AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413	                  
	ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570	                  
	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283
	AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413                          
	ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570	                  
	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283

Comment by David Anderson [ 06/Sep/12 ]

Tito,

Oliver's Account class had an error in it, it should have been:

class Account {
  @DBRef
  User owner;
}

Comment by Tito George [ 07/Sep/12 ]

David,
I have a class called User, belongs to collection users. User contains a list called requests(another collection called request).
Class request was like below initially.

Request.java

class Request{
@Id
private ObjectId id;
@DBRef
private User who;
@DBRef
private User whom;
@DBRef
private User group;
}
 
User.java
User{
@DBRef
List<Requests> requests;
}

Say an instance of User1 can contain list requests and request object can contain User1 (in its fields who or whom). I am able to save User as well as Request. But while querying i am getting https://jira.springsource.org/browse/DATAMONGO-488.

So I was hoping Oliver's workaround would solve this. But its failing wile saving.

Comment by Sebastian Julius [ 13/Dec/12 ]

Are there any plans when this feature will be integrated? We're currently creating 4-times more requests to the DB then required. Therefore it'd be brilliant to have lazy loading. Thanks in advance

Comment by regis [ 27/Sep/13 ]

The morphia project also provided this feature.

https://github.com/mongodb/morphia/blob/master/morphia/src/main/java/org/mongodb/morphia/annotations/Reference.java

 
@Entity("employees")
class Employee {
  // auto-generated, if not set (see ObjectId)
  @Id ObjectId id;
 
 
  //refs are stored**, and loaded automatically
  @Reference(lazy = true)
  List<Employee> underlings = new ArrayList<Employee>();
 
}

Why the spring data team doesnt reply ?

Hey guys the ticket is 3 years OLD....

Comment by Mikkel Dan-Rognlie [ 08/Oct/13 ]

I see that @thomasd added sprint "The Road to Codd M1" to this issue. According to https://github.com/spring-projects/spring-data-commons/wiki/Release-Train-Codd, it seems we will probable have this feature in the near future. Can you guys confirm that you will be working on it? That would be awesome!

Comment by Oliver Gierke [ 09/Oct/13 ]

Yes.

Comment by Thomas Darimont [ 16/Oct/13 ]

How should the lazy loading behaviour be enabled and what should be the default?

I would propose something like this, with lazy=false as the default:

@DBRef(lazy = true) List<User> fans;

Should we support lazy loading for all possible concrete types or just interfaces types or even just for collection types?
If the raw type has to be an interface we could simply return a JDK Proxy with an appropriate InvocationHandler that checks an "initialised"-flag on every method call. If not yet initialised the proxy will resolve the DBRef, store the result in a field and return that on subsequent invocations.

If we have to support arbitrary types, then we have to enforce some restrictions, e.g. like the fields of the type needn't be accessed directly - just via accessor methods. We could then generate a custom subclass of the given type (via CGLIB / ASM + (maybe) Objenesis for object construction) where we inject a proper initialisation check into the relevant method bodies and perform the required lazy loading logic if necessary and then delegate to the base class implementation.

Comment by Thomas Darimont [ 16/Oct/13 ]

Added Initial POC: https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348

Comment by regis [ 16/Oct/13 ]

We should stick by default to eager loading (like you propose), we will still compatible with the old behavior. And not breaking apps when they are going to update the version of spring mongo.

Comment by Mikkel Dan-Rognlie [ 17/Oct/13 ]

I also think eager loading should be the default. Maybe we could have the possibility to change it globally as a property on MappingMongoConverter. Like for instance the disable-validation attribute? Then one could override it with individual @DBRef(lazy = true|false) on entities.

If it's not too complicated I think it would be nice to support arbitrary types using CGLIB / ASM. But as you said, maybe it is not desirable with the restrictions on field access (could be perceived as 'magical' etc). If it makes the implementation easier it would be ok to require an interface in order to use JDK proxies.

I don´t think it should be limited to collection types. If you in a query gets a long list of documents, where each doc has deep nested documents, which at arbitrary levels of nesting has one @DBRef, it is still a lot of extra fetches even though the individual @DBRef fields are not collections.

Comment by Thomas Darimont [ 28/Oct/13 ]

Hi,

could you please give us some feedback on our current POC for lazy loading of MongoDB associations:
https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348

At the moment we support lazy loading for interface, concrete types, as far as they provide an appropriate parameterless ctor.
The @PersistenceConstructor annotation is supported as well.
If there is a need to support arbitrary constructors, we could also try to back-port the Objenesis stuff from
Spring Framework 4.0 (https://github.com/spring-projects/spring-framework/pull/327)
The actual loading is performed only once and is triggered when a method on the proxy is called.

If you want to give it a try, then please do the following:

  • Change the version of your mongodb dependency to: 1.4.0.DATAMONGO-348-SNAPSHOT
  • Add the spring source snapshot-repository

    <repositories>
        <repository>
            <id>repository.springsource.snapshot</id>
            <name>SpringSource Snapshot Repository</name>
            <url>http://repo.springsource.org/snapshot</url>
        </repository>
    </repositories>

Cheers,
Thomas

Comment by Oliver Gierke [ 04/Nov/13 ]

This merged into master. I added an optional dependency to Objenesis to be able to create proxies for classes without a default constructor. Feedback appreciated.

Generated at Mon Jul 24 02:45:28 UTC 2017 using JIRA 6.4.14#64029-sha1:ae256fe0fbb912241490ff1cecfb323ea0905ca5.