Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Complete
    • Affects Version/s: 1.1.0.M3
    • Fix Version/s: 1.1.0.RC1
    • Component/s: @ CORE
    • Labels:
      None

      Description

      Various low-level metadata enhancements to improve flexibility for add-on development.

        Issue Links

          Activity

          Hide
          Ben Alex added a comment -

          Git commit 03b001f2db3a811dd9152a9e84bb07a15276dab9

          Current Status

          There are four main changes so far caused by the “Project Joey” infrastructure:

          • ”Custom data” can now be added to most classpath-related metadata instances
          • Builders now exist for every classpath-related metadata type
          • Metadata service now detects and gracefully handles infinite metadata loops
          • Member details” scanning and decorator infrastructure automatically creates a complete and customizable representation of classpath-related metadata

          Collectively these changes provide a simple way to assign proprietary data or flags against all of the classpath metadata types that collectively represent a user project. The approach not only permits assignment of custom data, but also easy customisation of data seen by individual metadata providers. The builders enable easy creation of classpath metadata (a frequent requirement of many add-on operations and metadata classes) plus easy customization of classpath metadata (such as via the new member details decorators).

          It is unclear whether a formal JoeyModel object is still necessary in light of these changes. This is explored below and comment is invited.

          Custom Data

          The new CustomData and associated CustomDataAccessor interfaces provide a way to assign arbitrary key-value pairs (Object keys and Object values) against any identifiable Java structure. There is now a new interface called IdentifiableJavaStructure which defines an identifiable Java structure as any member-holding structure (so this typically means ClassOrInterfaceTypeDetails implementations as well as ItsTypeDetails) plus standard Java members (like FieldMetadata, MethodMetadata etc).

          Like the rest of Roo, CustomData is immutable once instantiated. The fact CustomData is stored within classpath metadata instances (which are always immutable) underscores the necessity that CustomData and the keys and values it stores are immutable.

          Builders

          There are now builders for every metadata type. You can easily see the builders by opening org.springframework.roo.model.Builder in Eclipse and hitting F4 (Open Type Hierarchy). Essentially these builders provide vastly simpler ways to create and modify the various org.springframework.roo.classpath metadata types, such as method metadata, field metadata, annotation metadata, class metadata, ITD metadata etc.

          Of course the immutability expectations of the metadata infrastructure have not changed. The builders respect the immutability constraints by allowing a new builder to be constructed that contains the core contents of an existing immutable metadata instance. The existing immutable metadata instance of course remains immutable, but its members are copied into the builder for subsequent modification. It is also possible to create a builder instance by calling a simple constructor that requires minimal initial information used for subsequent validation.

          The builders each provide a build() method that will return some type of immutable object. The build method’s return type is parameterized as part of the Builder interface contract. Builders of course enable convenient modification of CustomData. Builders are also generally more accommodating of illegal input, accepting null values, only accepting new members if it makes logical sense, and providing access to nested builders as appropriate.

          As consequence of the preferable programming model offered by builders, the old technique of using constructors on the default classpath metadata implementation classes is now discouraged. To this end these constructors have been marked as @Deprecated or make invisible outside the declaring package. Users of the old classpath metadata implementations should switch to the builders.

          Metadata Service Infinite Loop Detection

          Historically the Roo architectural approach required add-on developers to carefully identify the dependency relationships between different metadata types. Let us consider a Roo environment with metadata providers A, B and C. “A” provides .java source level information, “B” builds an entity ITD and “C” provides a toString() ITD. Ideally C would know about the members added by a A compilation unit as well as the B ITD. If only these three add-ons were ever to exist, C would depend on B and B would depend on A. But let’s consider if a D add-on was added, which perhaps offered a JavaBean getter/setter ITD. Naturally the C add-on would like to know about the members added by the D ITD, as it would like to call the getters as part of its toString() method. Again we could modify the code in C to directly request D, but C is not particularly easy to maintain by now. As such in Roo 1.0 the ItdRoleAwareMetadataProvider class allowed B and D to indicate they provide “accessor” methods. This enabled C to discover the members added by B and D. This proved adequate enough, but if an add-on E existed which needed access to the toString() method but it also provided an accessor method, we would have a problem. A practical example of an add-on like E that needs to know about all members is the “serializable” add-on. To implement an add-on such as E we would need it to poll every metadata provider. The problem is if polling C, it would poll E, and in inifinite loop would arise.

          To overcome these issues, the ItdRoleAwareMetadataProvider is now removed from Roo. Any add-on can ask for any other metadata item and it will be returned if possible. Null will be returned if an infinite loop would arise. To return to our earlier example, add-on C would ask for data including add-on E. The request to E would cause E to request data for C. The MetadataService will detect this infinite loop and return null. E would still build the rest of its metadata and return a basic representation that would be usable by C. C would continue as normal. Later on when E is building, it would ask C for data. As part of this C will ask E, but MetadataService will detect the infinite loop and return null. C will still build adequately, albeit without the information on E.

          These changes free add-on developers from needing to understand infinite loop possibilities when composing metadata.

          Member Details

          In recognition of the increasing number of add-on development goals that require metadata from numerous metadata providers, a new MemberDetails interface has been introduced. This interface essentially provides immutable access to a a List<MemberHoldingTypeDetails>. In each MemberHoldingTypeDetails is found normal member information such as methods, constructors and fields. A new MemberScanner is provided which can automatically build a MemberDetails instance. The MemberScanner will iterate through all metadata providers in order to build the MemberDetails. Given the high level of usefulness of MemberScanner, it is provided as a field on the commonly-extended AbstractItdMetadataProvider class.

          Interestingly, MemberScanner will not only scan all MetadataProviders and produce a MetadataDetails, but it will also iterate over any MemberDetailsDecorator instances detected in the OSGi container. The MemberDetailsDecorator interface provides an implementation the ability to replace the MemberDetails with a new instance. It also indicates which MetadataProvider is requesting the information, allowing a MemberDetailsDecorator to customize the contents that will incorporated into the MemberDetails. This provides an interesting avenue to customize what one MetadataProvider will see (assuming the MetadataProvider uses MemberScanner, of course).

          Ordering issues are avoided even if multiple MemberDetailsDecorator instances are used. This is achieved by continuously looping through all MemberDetailsDecorators until every one of them returns the same MemberDetails as they were passed. If any MemberDetailsDecorator returns a new MemberDetails instance, all MemberDetailsDecorators will be re-invoked with the new MemberDetails to evaluate. This means one MemberDetailsDecorator can depend on the output of an earlier MemberDetailsDecorator (or series thereof) without needing to formally define a relationship with them. Internally MemberDetailsScanner will perform iteration in alphabetic order of class name, although this is simply to assist debugging through providing a predictible iteration order (as opposed to any MemberDetailsScanner depending on such order or requiring a specific class name to appear at a particular position in the iteration sequence).

          Joey Metadata / Joey Model

          In view of the above, it is unclear whether JoeyMetadata (or JoeyModel as it has also been known) is strictly required. Adding a JoeyMetadata abstraction would further increase conceptual weight for add-on developers, require a new series of builders (assuming the very common immutability guarantee of Roo infrastructure is preserved) and slow down execution time by requiring further MemberDetailsDecorator-like iteration steps.

          The purpose of JoeyMetadata was to reflect the higher-level application model considerations, such as identifiers, persistence methods, human-friendly names, display semantics etc. These are higher-level concepts than classpath metadata provides and typically requires multiple members, compilation units and Java types.

          Nevertheless, this information could be adequately represented with CustomData stored against IdentifiableJavaStructure instances. The ability to continuously iterate MemberDetailsDecorators until they are all satisfied with the resulting MemberDetails also enables resolution of matters such as ordering constraints, hiding of unwanted data, application pluralization information to members and types etc.

          Some utility methods could be offered which work with a MemberDetails instance. These utility methods would use the MemberDetails and locate CustomData stored therein. Most of the methods intended for JoeyMetadata would simply become a utility method that accepts a MemberDetails input.

          It is proposed we identify a list of utility methods that would need to be written to satisfy the common web UI use cases. From there we can identify whether those methods could be adequately implemented via CustomData and one of more MemberDetailsDecorators (or even modification to MetadataItems and thus the storage of CustomData at the time of ITD production).

          Earlier Joey Requirements

          Types:

          • Names (system and human names in singular and plural form; system names prevent reserved word usage)
          • Category (domain object, identifier type, other object)
          • Is an aggregate root

          Properties:

          • FieldMetadata for backing property, MethodMetadata for accessor, MethodMetadata for mutator
          • Is part of identifier
          • Names (system and human names in singular and plural form; system names prevent reserved word usage)
          • MethodMetadata for add(E), add(int,E), remove(E), remove(int), size() for collection delegation
          • JSR 303
          • Cardinality
          • Logical field (possibly with automatic application of defaults based on this)
          • Visibility at different lifecycle stages (create, update, required security role to see field on UI, security role to override this condition)
          • Mutability at different lifecycle stages (create, update, required security role to edit field on UI, security role to override this condition)
          • Allocation model (eg always allocated automatically, automatically allocated if unspecified by user, never automatically allocated)
          • Use for (auto suggest drop-downs, toString methods, toXml methods, toJson methods, list views, summary views, detailed views, admin UI only etc)
          • Preferred constructor

          Next Steps

          • Eliminate all @Deprecated constructor usage (replace with Builder usage instead)
          • Identify utility methods and determine if JoeyMetadata is still required
          Show
          Ben Alex added a comment - Git commit 03b001f2db3a811dd9152a9e84bb07a15276dab9 Current Status There are four main changes so far caused by the “Project Joey” infrastructure: ”Custom data” can now be added to most classpath-related metadata instances Builders now exist for every classpath-related metadata type Metadata service now detects and gracefully handles infinite metadata loops Member details” scanning and decorator infrastructure automatically creates a complete and customizable representation of classpath-related metadata Collectively these changes provide a simple way to assign proprietary data or flags against all of the classpath metadata types that collectively represent a user project. The approach not only permits assignment of custom data, but also easy customisation of data seen by individual metadata providers. The builders enable easy creation of classpath metadata (a frequent requirement of many add-on operations and metadata classes) plus easy customization of classpath metadata (such as via the new member details decorators). It is unclear whether a formal JoeyModel object is still necessary in light of these changes. This is explored below and comment is invited. Custom Data The new CustomData and associated CustomDataAccessor interfaces provide a way to assign arbitrary key-value pairs (Object keys and Object values) against any identifiable Java structure. There is now a new interface called IdentifiableJavaStructure which defines an identifiable Java structure as any member-holding structure (so this typically means ClassOrInterfaceTypeDetails implementations as well as ItsTypeDetails) plus standard Java members (like FieldMetadata, MethodMetadata etc). Like the rest of Roo, CustomData is immutable once instantiated. The fact CustomData is stored within classpath metadata instances (which are always immutable) underscores the necessity that CustomData and the keys and values it stores are immutable. Builders There are now builders for every metadata type. You can easily see the builders by opening org.springframework.roo.model.Builder in Eclipse and hitting F4 (Open Type Hierarchy). Essentially these builders provide vastly simpler ways to create and modify the various org.springframework.roo.classpath metadata types, such as method metadata, field metadata, annotation metadata, class metadata, ITD metadata etc. Of course the immutability expectations of the metadata infrastructure have not changed. The builders respect the immutability constraints by allowing a new builder to be constructed that contains the core contents of an existing immutable metadata instance. The existing immutable metadata instance of course remains immutable, but its members are copied into the builder for subsequent modification. It is also possible to create a builder instance by calling a simple constructor that requires minimal initial information used for subsequent validation. The builders each provide a build() method that will return some type of immutable object. The build method’s return type is parameterized as part of the Builder interface contract. Builders of course enable convenient modification of CustomData. Builders are also generally more accommodating of illegal input, accepting null values, only accepting new members if it makes logical sense, and providing access to nested builders as appropriate. As consequence of the preferable programming model offered by builders, the old technique of using constructors on the default classpath metadata implementation classes is now discouraged. To this end these constructors have been marked as @Deprecated or make invisible outside the declaring package. Users of the old classpath metadata implementations should switch to the builders. Metadata Service Infinite Loop Detection Historically the Roo architectural approach required add-on developers to carefully identify the dependency relationships between different metadata types. Let us consider a Roo environment with metadata providers A, B and C. “A” provides .java source level information, “B” builds an entity ITD and “C” provides a toString() ITD. Ideally C would know about the members added by a A compilation unit as well as the B ITD. If only these three add-ons were ever to exist, C would depend on B and B would depend on A. But let’s consider if a D add-on was added, which perhaps offered a JavaBean getter/setter ITD. Naturally the C add-on would like to know about the members added by the D ITD, as it would like to call the getters as part of its toString() method. Again we could modify the code in C to directly request D, but C is not particularly easy to maintain by now. As such in Roo 1.0 the ItdRoleAwareMetadataProvider class allowed B and D to indicate they provide “accessor” methods. This enabled C to discover the members added by B and D. This proved adequate enough, but if an add-on E existed which needed access to the toString() method but it also provided an accessor method, we would have a problem. A practical example of an add-on like E that needs to know about all members is the “serializable” add-on. To implement an add-on such as E we would need it to poll every metadata provider. The problem is if polling C, it would poll E, and in inifinite loop would arise. To overcome these issues, the ItdRoleAwareMetadataProvider is now removed from Roo. Any add-on can ask for any other metadata item and it will be returned if possible. Null will be returned if an infinite loop would arise. To return to our earlier example, add-on C would ask for data including add-on E. The request to E would cause E to request data for C. The MetadataService will detect this infinite loop and return null. E would still build the rest of its metadata and return a basic representation that would be usable by C. C would continue as normal. Later on when E is building, it would ask C for data. As part of this C will ask E, but MetadataService will detect the infinite loop and return null. C will still build adequately, albeit without the information on E. These changes free add-on developers from needing to understand infinite loop possibilities when composing metadata. Member Details In recognition of the increasing number of add-on development goals that require metadata from numerous metadata providers, a new MemberDetails interface has been introduced. This interface essentially provides immutable access to a a List<MemberHoldingTypeDetails>. In each MemberHoldingTypeDetails is found normal member information such as methods, constructors and fields. A new MemberScanner is provided which can automatically build a MemberDetails instance. The MemberScanner will iterate through all metadata providers in order to build the MemberDetails. Given the high level of usefulness of MemberScanner, it is provided as a field on the commonly-extended AbstractItdMetadataProvider class. Interestingly, MemberScanner will not only scan all MetadataProviders and produce a MetadataDetails, but it will also iterate over any MemberDetailsDecorator instances detected in the OSGi container. The MemberDetailsDecorator interface provides an implementation the ability to replace the MemberDetails with a new instance. It also indicates which MetadataProvider is requesting the information, allowing a MemberDetailsDecorator to customize the contents that will incorporated into the MemberDetails. This provides an interesting avenue to customize what one MetadataProvider will see (assuming the MetadataProvider uses MemberScanner, of course). Ordering issues are avoided even if multiple MemberDetailsDecorator instances are used. This is achieved by continuously looping through all MemberDetailsDecorators until every one of them returns the same MemberDetails as they were passed. If any MemberDetailsDecorator returns a new MemberDetails instance, all MemberDetailsDecorators will be re-invoked with the new MemberDetails to evaluate. This means one MemberDetailsDecorator can depend on the output of an earlier MemberDetailsDecorator (or series thereof) without needing to formally define a relationship with them. Internally MemberDetailsScanner will perform iteration in alphabetic order of class name, although this is simply to assist debugging through providing a predictible iteration order (as opposed to any MemberDetailsScanner depending on such order or requiring a specific class name to appear at a particular position in the iteration sequence). Joey Metadata / Joey Model In view of the above, it is unclear whether JoeyMetadata (or JoeyModel as it has also been known) is strictly required. Adding a JoeyMetadata abstraction would further increase conceptual weight for add-on developers, require a new series of builders (assuming the very common immutability guarantee of Roo infrastructure is preserved) and slow down execution time by requiring further MemberDetailsDecorator-like iteration steps. The purpose of JoeyMetadata was to reflect the higher-level application model considerations, such as identifiers, persistence methods, human-friendly names, display semantics etc. These are higher-level concepts than classpath metadata provides and typically requires multiple members, compilation units and Java types. Nevertheless, this information could be adequately represented with CustomData stored against IdentifiableJavaStructure instances. The ability to continuously iterate MemberDetailsDecorators until they are all satisfied with the resulting MemberDetails also enables resolution of matters such as ordering constraints, hiding of unwanted data, application pluralization information to members and types etc. Some utility methods could be offered which work with a MemberDetails instance. These utility methods would use the MemberDetails and locate CustomData stored therein. Most of the methods intended for JoeyMetadata would simply become a utility method that accepts a MemberDetails input. It is proposed we identify a list of utility methods that would need to be written to satisfy the common web UI use cases. From there we can identify whether those methods could be adequately implemented via CustomData and one of more MemberDetailsDecorators (or even modification to MetadataItems and thus the storage of CustomData at the time of ITD production). Earlier Joey Requirements Types: Names (system and human names in singular and plural form; system names prevent reserved word usage) Category (domain object, identifier type, other object) Is an aggregate root Properties: FieldMetadata for backing property, MethodMetadata for accessor, MethodMetadata for mutator Is part of identifier Names (system and human names in singular and plural form; system names prevent reserved word usage) MethodMetadata for add(E), add(int,E), remove(E), remove(int), size() for collection delegation JSR 303 Cardinality Logical field (possibly with automatic application of defaults based on this) Visibility at different lifecycle stages (create, update, required security role to see field on UI, security role to override this condition) Mutability at different lifecycle stages (create, update, required security role to edit field on UI, security role to override this condition) Allocation model (eg always allocated automatically, automatically allocated if unspecified by user, never automatically allocated) Use for (auto suggest drop-downs, toString methods, toXml methods, toJson methods, list views, summary views, detailed views, admin UI only etc) Preferred constructor Next Steps Eliminate all @Deprecated constructor usage (replace with Builder usage instead) Identify utility methods and determine if JoeyMetadata is still required
          Hide
          James Tyrrell added a comment -

          Is it a bad idea to put a "Builder<T> newBuilder();" in the Builder<T> interface that is used by implementing classes to return a new builder so things can be inlined?

          Show
          James Tyrrell added a comment - Is it a bad idea to put a "Builder<T> newBuilder();" in the Builder<T> interface that is used by implementing classes to return a new builder so things can be inlined?
          Hide
          James Tyrrell added a comment -

          That doesn't makes sense. You couldn't put a static method an interface.

          My rationale was that you could chain your methods calling build at the end to get your built object. But you couldn't do this without having a static method in each concrete builder, right? In which case it is probably not worth it.

          Show
          James Tyrrell added a comment - That doesn't makes sense. You couldn't put a static method an interface. My rationale was that you could chain your methods calling build at the end to get your built object. But you couldn't do this without having a static method in each concrete builder, right? In which case it is probably not worth it.
          Hide
          James Tyrrell added a comment -

          That doesn't even make sense. For method chaining to work you would have to pass back the Builder for each set call.

          Show
          James Tyrrell added a comment - That doesn't even make sense. For method chaining to work you would have to pass back the Builder for each set call.
          Hide
          Alan Stewart added a comment - - edited

          @Deprecated constructor usage has been replaced with Builder usage in Roo internals in Git IDs 463c78718a133849cd490c61b780668e5692e38e, 1827173b2e9b9dce7b79adc61b3d2dac9620c7d1, eac2259d98a1c4e7586c29973f3800d139bc231b, and 191f4ec0aa8281167dd66040c4444e64a2cc1559. The @Deprecated constructors have also now been removed.

          Show
          Alan Stewart added a comment - - edited @Deprecated constructor usage has been replaced with Builder usage in Roo internals in Git IDs 463c78718a133849cd490c61b780668e5692e38e, 1827173b2e9b9dce7b79adc61b3d2dac9620c7d1, eac2259d98a1c4e7586c29973f3800d139bc231b, and 191f4ec0aa8281167dd66040c4444e64a2cc1559. The @Deprecated constructors have also now been removed.
          Hide
          Ricardo García added a comment -

          We were thinking about which improvements could be interesting to be added as new functionalities, and there is a list to be checked:

          Declare Annotation to a method param. i.e. to export class as web service adding @WebParam to each input parameter in MethodMetadata. This is related to AspectJ bug:

          Declare annotation - augmentation/overriding of defined Annotations in class and precedence between AspectJ. This will be released in v1.6.10.

          Metadata (from Class, Method, Field) to update Class/Methods/Constructors/Fields maintaining JavaDoc and Comments inside.

          Thanks!

          Show
          Ricardo García added a comment - We were thinking about which improvements could be interesting to be added as new functionalities, and there is a list to be checked: Declare Annotation to a method param. i.e. to export class as web service adding @WebParam to each input parameter in MethodMetadata. This is related to AspectJ bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=321820 Declare annotation - augmentation/overriding of defined Annotations in class and precedence between AspectJ. This will be released in v1.6.10. https://bugs.eclipse.org/bugs/show_bug.cgi?id=313026 Metadata (from Class, Method, Field) to update Class/Methods/Constructors/Fields maintaining JavaDoc and Comments inside. Thanks!
          Hide
          Ben Alex added a comment -

          The initial Joey infrastructure is in place. We will open new tasks to deal with actually using the infrastructure.

          Show
          Ben Alex added a comment - The initial Joey infrastructure is in place. We will open new tasks to deal with actually using the infrastructure.

            People

            • Assignee:
              Ben Alex
              Reporter:
              Ben Alex
            • Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: