Spring Framework's Migration from Jira to GitHub Issues

Engineering | Rossen Stoyanchev | January 15, 2019 | ...

The Spring Framework has migrated its entire history of issues from Jira to GitHub. The goal of this blog post is to provide you with context and details about this move.

Migration Details


The entire 15+ year history of every Spring Framework issue, and every comment, has been imported into GitHub. There is a lot to consider in such a move, so let's take a tour and go over some details.

If you have a link to an existing issue, e.g. https://jira.spring.io/browse/SPR-16751, you'll be redirected to the corresponding GitHub issue. If you actually mean to go to the Jira issue, append the query parameter redirect=false. On the GitHub side, the imported issue has a link back to its Jira issue origin.

Jira issue keys that appear in text, e.g. "SPR-16751", have been replaced with GitHub issue references which has the benefit of inserting links in the timelines of issues on both sides. Another benefit is the mini-preview when you mouse over the link.

Jira issues keys to other Spring projects that appear in text, e.g. "DATAJPA-813", have been replaced with links. For example see #18558 with a link to Spring Data JPA, #20904 to Spring Data MongoDB, and #16906 to Spring Integration Extensions.

Links to other GitHub project issues and pull requests that appear in text, get automatic benefits. After the migration, there are links in the timelines of the referenced issues that point back to the Spring Framework, and you get a preview of the link with a mouse over. For example see #21362 with a link to Spring Boot, #20849 to JUnit, #20256 to Jackson.

Links to commits and source code of any project on GitHub, get automatic benefits too. For example see the link to a source code range in issue #18463.

Jira Details

Every imported issue displays information from Jira in the bottom half of its description. The idea is that all information from Jira is available on GitHub and you don't have to go back and forth between the two. You may see one or more of the following, when available:

  • Affected versions
  • Reference URL
  • Attachments
  • Sub-tasks and related issues
  • Pull requests and commit references
  • Backport versions
  • Vote and watcher counts

Note that votes and watch subscriptions could not be carried over to GitHub. Even if spring-issuemaster has full privileges, it can only vote once. So please visit GitHub issues and re-apply reactions and subscribe to receive updates for specific issues.

Labels

Some Jira fields were converted to GitHub issue labels:

Jira Field Label
Issue Type "type: *"
Status "status: *"
Resolution "status: *"
Component "in: *"

Two extra labels were also applied to imported issues:

Label Description
"has: votes-jira" imported issues with 10+ votes
"has: backports" issues with backport versions

We used the opportunity to streamline the Jira field values, so for example 25 component values in Jira correspond to 5 "in: *" labels on GitHub. The "status: *" and the "type: *" labels have also been given extra through and revised.

Our choice of labels is aligned with the labels used in Spring Boot. The Boot team has given their process and labels a lot of thought, and we know the consistency will be appreciated by many. See the complete set of labels.

In Jira, many fields and labels are modifiable. In GitHub, only contributors can add or remove labels. This makes perfect sense. Reporters simply describe the issue, while contributors categorize it. Both developers and contributors can use the labels to search.

Fix Versions

A Jira issue can have multiple fix versions, while a GitHub issue can have only one target milestone. This feels like a drawback but there is more than is apparent to the eye, and the constraint has forced us to consider some meaningful adjustments.

Take SPR-17226 for example with fix versions "4.3.19", "5.0.9", and "5.1 RC3". While the issue was fixed in all 3 versions, there is no need to pollute the release notes of "5.1 RC3" which was still in development at the time. We can instead say it was fixed in 5.0.9 and backported to 4.3.19, and it's a contributors' concern to make sure the fix is propagated to the next version that's still in development. This is how it shows on the GitHub issue #21759 Could we have made this adjustment in the past? Sure, but the support for multiple fix versions in Jira didn't force us to consider.

This is a small adjustment that will result in cleaner release notes, but it will also impact the way we commit fixes. When the next version is in development on master, we will apply the fix to the current production branch first, then forward merge to master, and then cherry-pick from the older production branch. As opposed to apply the fix to master, and then cherry-pick from the production branches. The result is a cleaner commit history since forward merges help git to be aware or related changes.

As for backports, since an issue can have only one target milestone, we must create separate issues to represent the backporting of a fix. To use the same example, it pays to pick "5.0.9" as the target milestone because we'd have one backport issue only. If we picked "5.1 RC3", we would need two backport issues instead. To give you sense of the huge difference this makes, say the Spring Framework was on GitHub Issues from day 1. If we used the latter approach we'd have about 2,500 backport issues today. If we used the former, we would have about a 1000.

For historic backports, we've created one issue holder per release. There are about 45 of those. Going forward we'll have individual issues for all backported fixes which will be created automatically and used mainly for release tracking. As for discussions, the majority of conversations should happen on the main issue, while backport issues will be used only for discussions specific to the backport.

Markup

The markup was without a doubt the biggest and most painful part of the migration. A 15 year history of issue tracking reflects big shifts in programming styles styles that in turn determine what shows up in comments.

For example lots of XML was pasted in comments in the beginning, and Markdown treats that as HTML blocks which results in the tags not showing at all. Of course if those were surrounded with {code:xml}...{code}, it would look fine, but in those days markup wasn't commonly used and the XML snippets showed up anyway, not forcing the issue, and consequently making it impossible to migrate properly.

There are lots of other intricacies such as the escaping of curly braces, to avoid the effect of monospacing, or the escaping of asterisks to prevent them from disappearing as markup for bold. I'll spare you the details. Suffice to say we put a lot of effort into making sure the quality of markup conversion is reasonably high.

One specific issue to highlight is the use of "@" in plain text (i.e. outside of code blocks). Those are user mentions on GitHub that trigger notifications. You might be surprised that @Bean, @Configuration, @Component are actual GitHub users. First name references like @andy, @arjen, @brian colliding with GitHub user names were also common at some point, and all of those are a huge nuisance in an import of 17K+ issues with comments. This is why we've taken care to escape them. Going forward, when creating new issues or comments, please, be a good GitHub citizen and use backticks, e.g. `@Foo` (yes, https://github.com/foo does exist).

Background


I've used and liked Jira for a long time. The idea of migrating to GitHub Issues did not come to me immediately. It seemed too basic by comparison. What turned me around completely has little to do with a feature-by-feature comparison, although I must admit GitHub Issues has really grown on me since the move. What I'm hinting at are larger forces at work.

GitHub is the home of just about every open source project, including every Spring project, and all users can be reasonably expected to have GitHub credentials. As a result it has become untenable today to expect developers to maintain a separate login for the issue tracker of every open source project they depend on or want to report an issue against.

Then there are the benefits of co-locating source code and issues. I've mentioned many of those earlier, like the autolinked references across issues, pull requests, source code and commits, within a single project and across all projects on GitHub. The ability to mention and notify any GitHub user. All of those are very powerful benefits that are simply not possible with siloed issue tracking. I doubt anyone wants to go back to the days when open source projects were hosted in different places. The same is true for issue tracking.

There are deeper, less obvious benefits for co-located source code and issues. GitHub treats issues and pull requests as equal. They are assigned numbers from the same sequence. They look the same (description, comments, labels, and target milestone). They appear in release notes without distinction. A pull request is nothing but an issue with commits attached.

Historically in the Spring Framework, we've insisted on a Jira issue for every pull request. We didn't like the burden either but we needed a single place of record for all issues. As a consequence of this split situation, it was never too clear what should be discussed under the pull request and how much belongs in the Jira issue.

Going forward this is no longer a problem. We expect either an issue or a pull request, not both. If you need to start with a discussion first, which we do encourage, create an issue, and later, if you submit a pull request, the PR will supersede the issue. The two are linked still and nothing is lost. The conversation simply follows the action.

Not to be overlooked is the markup question. There is no doubt in my mind that Wiki markup is painful for code related discussions. I've used it for years on a daily basis. I'm used to it but some things are just hard, and require too much effort. Here is a reminder of what it takes to display something as common as curly braces and asterisks in code snippets: {{/endpoint/\{server-id\}/\{session-id\}/\{transport/\*\}}}.

There is no question that Markdown is easier for code related comments. It requires less typing and it just works when it comes to formatting code because it's simpler and does not clash with symbols that commonly appear in code. This has looked obvious to me from the start, since I've also used GitHub and Markdown in parallel for years too. I never understood why Jira still doesn't support Markdown. To be clear this wasn't a decisive factor. It's just one of those things you learn to live with that later can become extra incentive for change.

Last but not least, today most developers use Spring through Spring Boot which has always used GitHub Issues. From that perspective alone, there is enough incentive for the Spring Framework to migrate, since Spring Boot isn't about to migrate to Jira, and that would be the only other way to create a more consistent experience for Spring users.

The Actual Migration


Despite a lot of preparation there is nothing like the day of the actual migration. We used GitHub's unofficial import API which is documented to not trigger any notifications. We hadn't noticed any problem with that during testing. Once the actual migration started, notifications for every issue and every comment started pouring in.

We began to reach out to GitHub support using every available channel available. Fortunately for us they noticed too. How could they not? By my estimate, the 2,600 issues that were imported before we pulled the breaks must have generated tens of millions of emails, not surprisingly causing notification disruptions.

A day later, after GitHub support had corrected the issues and turned off all notifications for the Spring Framework project to be on the safe side, we were onto a smoother path to importing all issues over the course of 8-9 hours. It took another few hours to make a second pass over all issues and comments, to replace Jira issue keys with GitHub reference numbers, and then several more days to check and clean up markup conversion issues.

All of that is now complete, and it is my pleasure to announce we are now open for business on GitHub Issues.

Get the Spring newsletter

Thank you for your interest. Someone will get back to you shortly.

Get ahead

VMware offers training and certification to turbo-charge your progress.

Learn more

Get support

Tanzu Spring Runtime offers support and binaries for OpenJDK™, Spring, and Apache Tomcat® in one simple subscription.

Learn more

Upcoming events

Check out all the upcoming events in the Spring community.

View all