If you previously have experience with ATLAS’ previous source code system, Subversion, along with all of its associated workflow (SVN repository layout, package tags, Tag Collector, projects, etc.) then git can seem intimidatingly different at first. Don’t worry, it will pass… and most people do end up really appreciating git’s power and flexibility.
Here we provide a guide to thinking like a git (pun intended) when you work with ATLAS code that should help map the old workflow as much as is possible to the new one, alongside some explanations as to why the new workflow is the way it is.
We go through the workflow top to bottom, from a developer’s point of view:
In SVN there was one single repository and everyone worked from that. However, this was inflexible and led to committing code to the single main repository being the only general way to let others look at it or test it.
With git we use personal or group copies of the main repository (aka the upstream repository). So there is an extra step here to take your own fork of the main repository. However, this only needs to be done once - the cost is small and the benefits are great.
Assuming that your basic account was setup correctly then in SVN developers would take a copy of the repository directly from the single SVN master:
svn co $SVNOFF/Tools/PyJobTransforms/trunk Tools/PyJobTransforms
(Of course, as that was tedious,
svnco was used as a wrapper.)
Instead with git you clone the repository:
git clone https://:@gitlab.cern.ch:8443/YOUR_USER_NAME/athena.git git remote add upstream https://:@gitlab.cern.ch:8443/atlas/athena.git
The second git command run is to make sure your git repository knows about the main repository - it’s important to be able to synchronise with changes there directly.
The git checkout is a fully functioning repository on its own - so many operations like diffing code, searching or viewing history become extremely fast. The default checkout is also of all packages, so you can see the whole of the code at once (and, of course, if you don’t want that then just use a sparse checkout).
Making a local copy of the repository does take up more space - about 200MB for a sparse checkout, 800MB for a complete one…
…however, it is possible to keep a lot of development fed from one checkout by using branches and fetching updates - so reuse it.
Actual code development is pretty independent of the SCM that ATLAS uses. However, a fully functioning local repository is more powerful.
You can save local snapshots of your code as a work in progress as often as you like (just
git commitwhenever). You can even use side branches of your main topic branch for isolating different sub-developments. You can also trivially
diffyour code against any version in the repository. This is really powerful when you do incremental cycles of local development and testing.
In git you also create a branch for your development at this point, which is equivalent to a set of package tags plus a Tag Collector bundle. However, it’s hugely easier and faster to do:
git fetch upstream # Sync with latest changes in main repo git checkout -b my-new-development upstream/[PARENT_BRANCH] --no-track
The SVN plus Tag Collector equivalent we don’t even try to show…
The git topic branch is far more versatile and better supported by tools.
svn commit ...
git add ... git commit ... git push -u origin ...
Why the difference? Git distinguishes between your local repository
and the GitLab one - so you can keep everything local (
until it’s fully ready (
git push). Git also has the concept of a
staging area, which is why
git add is needed. (Although there are
more steps, git gives you a lot more flexibility.)
In SVN updating to HEAD is pretty easy:
In git the process is just to apply the changes from the upstream branch onto your topic branch:
git fetch upstream # Get all upstream changes # (does not touch your checkout!) git merge upstream/[PARENT_BRANCH]
If there is a line-by-line conflict you must resolve it by hand.
git has much more powerful facilities for rolling back out of
a failed merge (see
git status during the merge).
svn cp -r 123456 $SVNOFF/Tools/PyJobTransforms/trunk $SVNOFF/Tools/PyJobTransforms/tags/PyJobTransforms-01-02-03
becomes… nothing at all. Because this change is already identified uniquely in git by the commits made in the development and by the topic branch that was used.
Just creating a branch in git is easier, faster and less error prone than the fiddly SVN equivalent.
In the old workflow a developer would request that a package tag be incorporated into a release by either
In git this is done from GitLab by just clicking on the
button in the GitLab web page. Handling merge requests in GitLab is
vastly superior, not just because the user interface is a thing of joy
compared to Tag Collector:
Faster, easier and more functional in GitLab.
When packages needed to be changed in concert they needed to be handled specially in Tag Collector using a bundle. This was an awkward step and caused a lot of problems, especially if packages needed to be swept from one release to another as the bundle identity was lost when the bundle was accepted.
However, in git, a merge request will encapsulate coherently all changes, regardless of package boundaries. The merge request can be coherently cherry picked between release branches as well. (In fact, one can think of a topic branch in git as being inherently like a bundle, implemented correctly.)
Faster, easier and more functional with merge requests of a topic branch.
Probably the single most unsettling thing for developers used to the old workflow is the lack of package tags. These were used in SVN to snapshot a particular version of a particular package, then collected by Tag Collector to assemble a release.
We already discussed the superiority of GitLab merge requests to a Tag Collector workflow, but still, people will probably still ask where are my package tags?
git diff release/21.0.1..release/21.0.8
This probably shows more changes than you want, so give a path to only see the changes for a package, e.g.,
git diff release/21.0.1..release/21.0.8 Tools/PyJobTransforms
git diff nightly/21.0/2016-12-01T2230..nightly/21.0/2016-12-07T2230 Tools/PyJobTransforms
git log PATHto see only commits that affected a particular PATH (which can be a package, but more powerfully can be single files or domain areas). Then follow up with a
git showto see the commit log and diff for a particular commit.
git tag my_package/some_label
(These tags will not be in the main repository, although other developers can get a copy of them if they add your private fork as a remote.)
We have a collection of hints that will provide more recipes that can be used.
The import of ATLAS code from SVN was only made for SVN tags that
were part of a release (or for
dev, at least validated). If there
is code you need to take from SVN into git then you can use the
svnpull.py script to achieve that.
Usage is very simple: giving a package name will import the current
SVN trunk; giving a package tag will import that tag. Use
svnpull.py --help for full usage, which allows you to import any
SVN path into git, also restricting to only some files or using an
arbitrary SVN revision.
The script will only copy the code from SVN into your git checkout.
It is then up to you to
commit when you have reviewed
the change and are ready to make a merge request.
Some very important notes when importing from SVN:
svnpull.py has been added to
lsetup, but it requires python 2.7
lsetup git python. The code is maintained on the
svnpull branch in this GitLab repo.
This table tries to summarise what was described above. However, it’s just not possible to map a git command to an SVN one and plug this into the new workflow (which was possible with the CVS to SVN migration), because SVN and git are just not two beasts from the same stable. So, instead, we try to map some of the key concepts and give commands or procedures that are similar.
It’s also useful to keep our git cheat sheet close at hand.
|Identify code patch for a release||SVN package tag:
||git topic branch:
|Request tags||Tag Collector or Email||GitLab merge request|
|Tag bundles||Tag Collector||unnecessary|