Moving to Git

This page describes how I (Joel) initially set up the e-Labs repository in Git. Ideally, nothing here should have to be repeated. For documentation on the day-to-day use of the repository, see [[][ADD LATER]]

Architecture

A central Git server from which developers will pull and push code will be installed on i2u2-wiki. Developers will access it by SSH with passphrase.

To begin with, we will maintain the division between development and production filesets by maintaining two repositories on i2u2-wiki, called 4.0-dev and 4.0-prod. Ultimately, though, it is expected that we will move to a single repository with development and production branches in the usual Git design. I intend to drop large amounts of unused code in the transition, such as the CMS test beam e-Labs, the Adler i-Lab, things dealing with OGRE, etc. This trimming may be sufficient to warrant moving to a "5.0" label.

On deployment, the quarkcat user on i2u2-prod and i2u2-dev will pull from the appropriate repo on i2u2-wiki via passphraseless SSH. For production deployments, quarkcat will also instruct the repository to push to a public repository hosted on GitHub. This will allow us to maintain an open and publicly-accessible codebase, assuming that the repository we end up with satisfies GitHub's storage space requirements for free accounts. Developers will not typically interact with the GitHub repository.

Notes:

• I used i2u2-wiki to host the repo because a Git repo isn't a big enough thing to require its own VM, and i2u2-wiki has become the standard VM to host "miscellaneous" e-Lab development applications.
• I looked at but declined to use gitolite because with only three developers, we can do everything we need to without it.
• I installed the Git repository under /srv/ on i2u2-wiki because that seems to be what /srv/ is for. It could go pretty much anywhere.

Server installation

Developers and the website deployment scripts will access the e-Lab code repository on i2u2-wiki as the user git over SSH. The steps for enabling this are

1. Establish the git user on i2u2-wiki
2. Initialize a bare repository on i2u2-wiki
3. Install RSA public keys from each developer and from quarkcat on i2u2-wiki
4. Push the e-Lab code from a fresh repository on a developer's machine into the bare repository on i2u2-wiki as its first commit

In detail:

The git user

First, make sure git-shell is listed in /etc/shells. It was installed by default on i2u2-wiki (Ubuntu 14.04 LTS) at /usr/bin/git-shell, but it was not in /etc/shells.

$ sudo echo $(which git-shell) >> /etc/shells

This shell is non-interactive (that is, no direct login is allowed) and can only be used to issue Git commands. Using it limits the security risks of adding a new user with SSH access.

Now add the git user

$ sudo adduser --shell /usr/bin/git-shell --firstuid 901 --lastuid 999 --disabled-password git
Full Name: git
Room Number: 
Work Phone: 
Home Phone: 
Other: 
jgriffi8@i2u2-wiki:~$ sudo groupmod -g 901 git

The flags --firstuid 901 --lastuid 999 are used to assign git the lowest available user ID (UID) between 901 and 999. If you don't specify a UID, adduser defaults to the lowest available UID withover 1000, which is the block conventionally reserved to local (i.e., non-system) users. On i2u2-wiki, though, that block is also subject to Kerberos authentication, so if you try to create a user with UID >= 1000, adduser will demand a Kerberos password that can't possibly exist yet. It's annoying, but circumventable by keeping the UID <1000 (quarkcat is 999 on i2u2-dev and i2u2-prod, for reference).

The --disabled-password flag is another security element that prevents anyone from logging in as git using a password.

adduser creates the git group automatically when it creates the git user, but I didn't realize that the group ID (GID) of the new group wouldn't default to the same as that set for the UID, if available. Instead, it independently defaulted to 1000, like the UID tried to. So, after confirming that GID 901 was available ($ cat /etc/group | grep 901) I changed it to match the UID using groupmod. There was probably a better way to do that.

Acting as that user, the next step is to create /home/git/.ssh and give it the permissions SSH expects

$ sudo -u git mkdir ~git/.ssh
$ chmod 700 ~git/.ssh

Finally, create an authorized_keys file for git and set permissions

$ sudo -u git touch ~git/.ssh/authorized_keys
$ chmod 600 ~git/.ssh/authorized_keys

The server repository

Install a bare repository under the /srv/ directory on i2u2-wiki.

$ sudo mkdir -p /srv/git/elabs/4.0-dev.git
$ sudo chown git:git /srv/git/elabs/4.0-dev.git
$ cd /srv/git/elabs/4.0-dev.git
$ sudo -u git git init --bare

Do the same for 4.0-prod.

This bare repository is currently empty. To make it usable, we must prepare the repository files elsewhere and push them into it.

RSA keys

To use the repository, each developer must supply the public key of an RSA keypair and add it to the git user's keychain on i2u2-wiki; see the developer instructions. This section covers only the initial setup for the quarkcat users of i2u2-dev and i2u2-prod, though note that establishing the repository requires one developer to also arrange his or her own keypair in advance in order to make the initial commit to the i2u2-wiki repository.

Both i2u2-dev and i2u2-prod had existing id_rsa keypairs in ~quarkcat/.ssh/ dating from 8Dec2015. This was about the time of the transition from the Argonne servers; either Edit generated them or they were copied over in the transition. Since I can't vouch for their provenance, I generated new git_rsa keys on both VMs for connecting to i2u2-wiki.

Acting as root (to satisfy .ssh/ folder permissions), generate keys:

root@i2u2-dev:/home/quarkcat/.ssh# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): /home/quarkcat/.ssh/git_rsa

No passphrase - quarkcat does not need a passphrase because anyone who has access to its private keys stored on the dev/prod servers has already compromised our servers. Fix ownership:

root@i2u2-dev:/home/quarkcat/.ssh# sudo chown quarkcat:quarkcat git_rsa*

Transfer the public key to i2u2-wiki:

$ scp ~/.ssh/id_rsa.pub jgriffi8@i2u2-wiki.crc.nd.edu:/tmp/git_rsa.quarkcat-dev.pub

On i2u2-wiki add this to the git user's authorized_keys file:

root@i2u2-wiki:/home/git# cat /tmp/git_rsa.quarkcat-dev.pub >> ~git/.ssh/authorized_keys

You must act as root (not just sudo) to use the shell built-in redirect >> to a permissions-protected file.

TODO: Fix quarkcat permissions to pull-only from i2u2-wiki

Preparing the repository files

On a developer's computer, one must create a new Git repository out of the e-Lab code base. The easiest way to do this is to create a new directory (i.e., ~/git/elabs/4.0-dev/) and svn export into it from your SVN working copy:

$ svn export -q --force /path/to/your/working/copy/4.0-ND-dev ~/git/elabs/4.0-dev

If you happen not to have a working copy, you can export directly from the repo at Fermilab, but this takes longer:

$ svn export -q --force http://cdcvs.fnal.gov/subversion/quarknet/branches/4.0-ND-dev ~/git/elabs/4.0-dev

This creates an ordinary file tree of the e-Lab code, completely untouched by SVN or Git. In the initial construction of the repo, this is the point where dead weight in the code is excised.

Once the code is the way you want it to look in the initial commit to i2u2-wiki, initialize a Git repo in the root directory, then add and commit all files :

~/git/4.0-dev$ git init
~/git/4.0-dev$ git add --all
~/git/4.0-dev$ git commit -m "initial commit of 4.0-dev"

Candidates for Deletion

• config/www13/
• config/www17/
• config/www18/

-- Main.JoelG - 2017-09-18

Comments