Self hosting git repositories

Posted on 2016-02-10

Most Free/Open Source Software projects these days are hosting their repositories on github or gitlab or such centralized hosting “services”. And then these centralized services go down, once in a while and every one starts complaining about it, again on centralized social media services like twitter. The noise settles down after a day or two, people go back to their old ways of doing things (i.e. pushing code into these centralized servers).

Now, don’t get me wrong. Github significantly reduces the barrier to contribute code into a project. There is no denial of that fact and that’s one of the reasons why it is very popular.

Free Software developers (or anyone who wants to share their code or data over any version control system) need not depend on these services. First of all, Free Software needs free tools. There are no two rules for “developers” and “users”. As Free Software proponents who release code as free software, we keep telling our users that Freedom is important. But then we, as developers use these propreitary services and push code to it. Isn’t that contradictory? Why is it that we, the developers, need propreitary tools, but the “lowly” users don’t?

We have had distributed version control systems for a while now. We could host code on any server and let people clone the code. We may not have fancy features like a web API or a way to send Pull Requests or have fancy statistics about the developer activity etc. But we can mostly achieve collaboration with free software tools. Pull Requests can be replaced with git-format-patch(1) and git-email (which provides git-send-email(1) command). In fact, that’s how it is intended to be used and that’s how the linux kernel developers still use it and it works really really well for them.

I have used a similar setup in a large organization which works via a mailing list and the afore mentioned tools around git to collaborate over email and patches. It makes code reviews quite easy. For those who like to have fancy web UI for code reviews, one could setup a phabricator instance. But for my personal repositories which does not attract too many contributors, I felt, simple hosting is good enough and I can easily collaborate over email.

I had been running my own private git server (over ssh) for a long time and had been using it to store private data. I had also been mirroring my repositories there, but putting it behind ssh makes it hard for others to read it. But I was also using github for a long time. I decided that this has to change. Now onwards, I will only use github to work with other github projects (I respect the decisions of the owners of those projects). For my own little projects, I will be hosting my own repositories.

I also decided to throw in a git sub-domain and put it behind a TLS pad lock. Most of the things I needed were pieced together from the information others have put in on the Internet.

I decided to host all the bare repositories under the user ‘git’. The username ‘git’ does not have a password and use git-shell as its login shell. This is mostly out of the Pro Git book.

# useradd --create-home --skel /dev/null --home-dir /home/git --shell /usr/bin/git-shell git
# mkdir /home/git/projects

Now, we need commands to create repositories, add SSH public keys (for access control), to list the repositories we have etc. To do that, we need to add scripts under the directory git-shell-commands and the name of the script will be the name of the command. On a Debian system, there are a few example scripts in /usr/share/doc/git/contrib/git-shell-commands/ like help and list. We copy over these commands into /home/git/git-shell-commands directory to start with.

The additional commands we need are: create and addkey. I decided to split the create command into two: create command that creates a private repository and a public command that takes a given repository and makes it public. These scripts (create, addkey) are mostly identical to the ones found in a blog post. I added a new command public that makes a new repo public.

public

#!/bin/sh

# If the user is not root
if [ "$USERNAME" != "root" ]
then

	# Dislpay a notice and stop
	echo "Sorry, only root can use this command."
	exit 1

fi

# If no project name is given
if [ $# -eq 0 ]
then

	# Display usage and stop
	echo "Usage: public <project.git>"
	exit 1

fi

# Set the project name, adding .git if necessary
project=$(echo "$*" | sed 's/\.git$\|$/.git/i')

if [ ! -d "$project" ]
then

    # project does not exist
    echo "given repository does not exist. Please see 'create' command"
    exit 1
fi


cd "$project" && \
touch git-daemon-export-ok && \
cd .. && \
cd projects && \
ln -sf ../"$project" "$project"

Be sure to make these scripts executable.

A user session would look like this:

$ sudo su git
Run 'help' for help, or 'exit' to leave.  Available commands:
addkey
create
list
public
git> create foo.git 
Initialized empty Git repository in /home/git/foo.git/
git> public foo.git
git> 

Web interface

I chose to use gitweb for the web interface for no particular reason. On a Debian system, these can be installed with:

$ sudo apt-get install gitweb fcgiwrap

Gitweb has one config file to modify, this is how my config file look like.

$ cat /etc/gitweb.conf
# path to git projects (<project>.git)
$projectroot = "/home/git/projects";

# directory to use for temp files
$git_temp = "/tmp";

# target of the home link on top of all pages
#$home_link = $my_uri || "/";

# html text to include at home page
#$home_text = "indextext.html";

# file with project list; by default, simply scan the projectroot dir.
$projects_list = $projectroot;

# stylesheet to use
@stylesheets = ("static/gitweb.css");

# javascript code for gitweb
$javascript = "static/gitweb.js";

# logo to use
$logo = "static/git-logo.png";

# the 'favicon'
$favicon = "static/git-favicon.png";

# git-diff-tree(1) options to use for generated patches
#@diff_opts = ("-M");
@diff_opts = ();

@git_base_url_list = qw(https://git.rkrishnan.org);

git sub domain

I wanted to host my repositories under git.rkrishnan.org. This needs a bit of work.

First, I had to go to the DNS dashboard of my DNS provider and add a CNAME record for the subdomain. I am hosting the regular root domain – rkrishnan.org – also on the same host. So, the webserver configs are adjusted to have two virtual hosts. I use nginx as my webserver. Here is how the configuration look like. Be sure to “enable” the virtual host, by creating a symbolic link in the sites-enabled directory.

The configuration is mostly a copy of this nice blog post except for more explicit ciphers and the dhparams configuration. The configurations are mostly self explanatory. It uses the git-http-backend to serve the git files over http.

server {
        listen 443;
	server_name git.rkrishnan.org;
	root /usr/share/gitweb;
 	index index.html index.htm;

	ssl on;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	ssl_certificate /etc/letsencrypt/live/git.rkrishnan.org/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/git.rkrishnan.org/privkey.pem;

	ssl_stapling on;
	ssl_stapling_verify on;

	ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';

	ssl_prefer_server_ciphers on;
	ssl_dhparam /etc/nginx/dhparams.pem;

	# static repo files for cloning over https
	location ~ ^.*\.git/objects/([0-9a-f]+/[0-9a-f]+|pack/pack-[0-9a-f]+.(pack|idx))$ {
	    root /home/git/projects/;
	}

	# requests that need to go to git-http-backend
	location ~ ^.*\.git/(HEAD|info/refs|objects/info/.*|git-(upload|receive)-pack)$ {
	   root /home/git/projects;

	   fastcgi_pass unix:/var/run/fcgiwrap.socket;
	   fastcgi_param SCRIPT_FILENAME   /usr/lib/git-core/git-http-backend;
	   fastcgi_param PATH_INFO         $uri;
	   fastcgi_param GIT_PROJECT_ROOT  /home/git/projects;
	   include fastcgi_params;
       }

       # send anything else to gitweb if it's not a real file
       try_files $uri @gitweb;
       location @gitweb {
           fastcgi_pass unix:/var/run/fcgiwrap.socket;
	   fastcgi_param SCRIPT_FILENAME   /usr/share/gitweb/gitweb.cgi;
	   fastcgi_param PATH_INFO         $uri;
	   fastcgi_param GITWEB_CONFIG     /etc/gitweb.conf;
	   include fastcgi_params;
      }
}

and finally, I also added https by using the new excellent let’s encrypt initiative by EFF and others and created two separate certificates – one for the regular root and one for the git subdomain – as letsencrypt does not support wildcard domains yet.

After the changes, nginx will need a restart.

sudo service fcgiwrap restart
sudo service nginx restart

That’s about it and one can use the web interface that gives a clone url, browse through the code, see the logs etc..