A few further steps taken in the meantime:
We're avoiding our rather expensive annotation retrieval algorithm in common callers of AnnotatedElementUtils when there are no annotations to begin with: e.g. in StandardAnnotationMetadata, AutowiredAnnotationBeanPostProcessor, AnnotationTransactionAttributeSource. StandardAnnotationMetadata also caches the annotation array now, avoiding repeated re-retrieval.
AnnotatedElementUtils consistently operates on the actual annotation type if available, avoiding String name comparisons in favor of annotation type identity checks wherever possible. Also, it avoids double getDeclaredAnnotations/getAnnotations checks on anything other than Classes now - since they'll always deliver the same result for Methods etc anyway, just in a wasteful fresh array.
I've also fixed our new 4.2 order determination check in ConfigurationClassUtils so that it only kicks in for actual configuration candidates; it was running through an unnecessary Order annotation lookup for every single bean in the context before.