Migrating MESA to GitHub
07 Jun 2021Earlier today, I was pleased to announce that the MESA stellar evolution code is now publicly hosted on GitHub.
The MESA stellar evolution code began its life in 2007 as a Subversion repository hosted on SourceForge.
MESA is somewhat unusual in that a checkout of the code is multiple GB in size. This is because it versions its key binary assets (e.g., equation of state data files) along with the code. In addition, there are also numerous large text files (containing stellar models) that can frequently change, but whose diffs are not meaningful.
The SVN proved satisfactory for MESA’s early life, especially as its development was primarily due to a single person (Bill Paxton).
Around 2016-2017, the developer base had expanded to include several other regular contributors and SourceForge was becoming increasingly unreliable.
At that time, we considered moving to git, but solutions for versioning large binary files with git were in their infancy. Instead, we migrated the SVN to a paid hosting plan at Assembla, immediately resolving our performance problems.
Developers who wanted to interface via git (including myself), were able to do so using git-svn. This was an acceptable stop-gap measure, but the large binary files meant that the git repository constructed in this way was in excess of 50 GB and generally suffered from poor performance when doing complex operations.
With an understanding that a git migration remained desirable, sustained effort was applied to reduce the number of large files tracked in the repository. Some simple improvements were realized by removing various types of cruft. Others required more involved structural changes. For example, redesigning the testing infrastructure so that solutions could be compared between different machines via a central server removed the necessity to version-control reference stellar model files.
By 2020, the MESA developer base had continued to grow, GitLFS was mature, and there was a desire to more effectively engage with a new generation of users and developers. With the help of my colleagues, especially Rob Farmer and Bill Wolf, we put together a plan to migrate MESA to GitHub. In advance of the migration, documentation was written to provide a basic introduction to git/GitHub for those who were unfamiliar.
In the end, we chose not to include the complete MESA development history in the git repository. Instead, the git repository was initialized to the most recent release (r15140) and a modest number of patches originally committed to the SVN were applied to the git repository.
The MESA developers moved to git with minimal interruption in mid-December 2020. We took about 6 months to get comfortable and establish and document new workflows, and then finally made the repository public. This should pave the way for a new era of collaborative MESA development and interaction with the astrophysics community.