Monday, April 14th, 2008...9:12 am

Hosting Git Repositories on Dreamhost

Jump to Comments

Over the weekend I went searching for a way to host remote git repositories on my Dreamhost (DH) account. I do a fair bit of development year round and up to this point have been hosting all my code on my DH account using Subverision. Over the past few weeks I’ve been making the move from Subversion to Git on my local dev machines, and figured it was time to start hosting my git repositories remotely as well.

It took a lot of searching around online and some tinkering, but I was able to get everything setup properly and in the hopes of saving everyone else some time, I’ll post my results here.

First, if you take a look at the DH wiki entry for Git, you will see the very first method described is to run Git on your DH account using WebDAV. If you’re like me, thinking about doing Git over WebDAV probably just left you scratching your head. I think the reason why this is advocated, is because of the way DH setups SSH accounts. By doing git over SSH to a DH account, only one person would be able to remotely push changes, which is fine for my needs. But, if you need to give multiple users access for pushes, then WebDAV would technically work for you. But, I opted not to go down this route.

I chose to simply compile Git locally into my DH account and connect to it over SSH. This blog post on Autopragmatic.com got me about 90% of the way toward completion, so the next few steps are taken directly from his post.

Step 1. Compile and Install Git on Dreamhost

The first thing that needs to be done is to compile Git locally into your DH account. I personally have a ~/src directory where I download and compile all my code and I have a ~/packages directory where I install my binaries. You may have a different setup so if so, just substitute those paths accordingly. Also, make sure to substitute $USER with your DH username. (Also note the current stable version of git might be different from what you see here. At the time of this writing, 1.5.4.rc4 was the most recent version, but feel free to install a newer version if available.)

$ cd ~/src
$ wget http://www.kernel.org/pub/software/scm/git/git-1.5.4.rc4.tar.gz
$ tar xvzf git-1.5.4.rc4.tar.gz
$ cd git-1.5.4.rc4
$ ./configure --prefix=/home/$USER/packages NO_CURL=1 NO_MMAP=1
$ make
$ make install
$ git --version

There are a few things to be aware of. First, the NO_MMAP=1 parameter is necessary to keep the DH process police from killing your Git process. Git normally creates two 250MB memory map files, and DH will kill your Git process since it sees it consuming over 500MB of memory. I also used the NO_CURL=1 parameter as I will not be needing my git repositories to have the ability to pull from other remote locations. If you do need this functionality, then you will also need to download, compile and install libcurl, but that is beyond the scope of this post. Also, make sure that you have the path of your installed binaries in your $PATH or that last command of git –version will fail. In my case, I have added /home/$USER/packages/bin to my $PATH in my ~/.bashrc file.

Step 2. Create a new bare Git repository

Now that you have installed git, you can start making new repositories. You can, of course, store these repositories anywhere you want. I chose to create a whole new sub-domain under the Dreamhost Domains Panel. Some people prefer to instead create a git subdirectory under an existing domain. Any way is fine, the choice is yours. For the rest of this post I’ll simply be refering to git.example.com as the <PATH> of my git repositories. You will change this to your path.

If you plan on making multiple repositories, the easiest thing to do is simply automate the process by adding the following function to your ~/.bashrc file.

newgit()
{
    if [ -z $1 ]; then
        echo "usage: $FUNCNAME project-name.git"
    else
        gitdir="/home/$USER/$PATH/$1"
        mkdir $gitdir
        pushd $gitdir
        git --bare init
        git --bare update-server-info
        chmod a+x hooks/post-update
        touch git-daemon-export-ok
        popd
    fi
}

This way you can simply login to your DH account and type:

$ newgit my-new-repos.git

and a new bare repository will be created for you in your <PATH>. Or if you prefer to create a new repository from your local box you could even do it over SSH:

$ ssh $USER@$MACHINE.dreamhost.com newgit my-new-repos.git

Step 3. Pull and push to your new repository

Theoretically, you should now be able to pull and push from this bare repository using the following command:

$ git clone ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/repo-name.git
$ git push origin

However, I found this did not work for me and resulted in the following errors:

Initialized empty Git repository in ~/src/newrepo/.git/
Bad port ”
fatal: no matching remote head
fetch-pack from ’ssh://$USER@$MACHINE.dreamhost.com:/home/$USER/$PATH/my-new-repo.git’ failed.

The problem is that the new repository is bare and has absolutely no data in it at all. That led me on another search that brought me to this excellent writeup that got me the rest of the way home. What I did was simply changed directories into my local git repository. If you haven’t even started your codebase locally yet, I would recommend going ahead and doing that.

Create a new directory and create or add in your files, then do the following to setup a local git repos:

$ cd /my/code/path
$ git init
$ git add .
$ git commit -m "initial import"

That will create a local git repository in your working directory and commit the initial files. Next follow the next set of commands to add your DH repos as a remote and push your initial codebase up to it.

### assuming ssh://server/remote.git resolves to an empty, bare git repo
### and that we are chdir()'d to the local repository:
$ git push --all ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/my-new-repos.git
$ git remote add origin ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/my-new-repos.git
$ git config branch.master.remote origin
$ git config branch.master.merge refs/heads/master
$ git fetch
$ git merge master
$ git branch -a
* master
  origin/master

Now you can simply work locally and do a

$ git push

to push to your DH remote. And if you ever need to you can simply execute a

$ git pull

to pull down and merge remote changes.

Step 4. Setup GitWeb (optional)

If you want to be able to access your git repositories via a web browser, you need to setup GitWeb. GitWeb ships with the git source code so you already have it available. You just need to copy the GitWeb files into your git repository directory and make them executable.

cd <PATH>;
cp ~/src/git-1.5.4.rc4/gitweb/git* .
rm gitweb.perl
chmod 755 gitweb.cgi

Its a good idea to create a configuration file as well:

touch gitweb_config.perl
vim gitweb_config.perl

and put this as the

gitweb_config.perl

content:

# where is the git binary?
$GIT = "/home/$USER/packages/bin/git";

# where are our git project repositories?
$projectroot = "/home/$USER/git.example.com";

# what do we call our projects in the gitweb UI?
$home_link_str = "My Git Projects"

#  where are the files we need for gitweb to display?
@stylesheets = ("gitweb.css");
$logo = "git-logo.png";
$favicon = "git-favicon.png";

# what do we call this site?
$site_name = "My Personal Git Repositories";

Then to get it to actually run in the browser you need to tell apache to serve the files up via CGI. Edit your

.htaccess

file with the following info:

Options +ExecCGI
RewriteEngine On
RewriteRule ^$ gitweb.cgi
RewriteRule ^([?].*)$ gitweb.cgi$1

Now you should be able to access your git repositories from you web browser at http://git.example.com.

I personally went one step further and password protected my repositories. If you don’t want your git repositories exposed to the whole world, you might want to do the same thing. I used simply HTTP authentication by adding the following to the bottom of my

.htaccess

file:

AuthType Basic
AuthName "Private Git Repository Access"
AuthUserFile /home/$USER/.htpasswd
Require valid-user

This, of course, requires that you have proper

.htpasswd

file setup which is beyond the scope of this tutorial.

31 Comments

  • Excellent write up. I followed this straight through steps 1-3 with my hosting company Site5 and only ran into one minor snag which was having to ask them to enable PermitUserEnvironment so that ~/.ssh/environment could set the path to my git binaries in ~/bin. Again, nice helpful write up.

    –Garrick West

  • Thanks I used this to install Git on my webhost.
    I believe Git is no longer dependent on libcurl, so the NO_CURL=1 option isn’t needed anymore.

    Also I had to write some php scripts to install since I didn’t have SSH access to my webhost. If anyone is in the same boat I posted the scripts here:
    http://sharpideas.ca/Software/Dev/SourceControl/Git/InstallGit.html

  • Wow. THANK you!

  • Any idea on how to compile Git with Curl support (so I can git clone remote sources)? I cannot find out how to do this without being able to install debs (which I cannot do without root access on DreamHost).

  • David, if you also install libcurl (http://curl.haxx.se/) you should be able to leave the NO_CURL=1 option off of the git compile command to get curl support. Look in Step 1 of the tutorial above.

    I compiled git without curl support on Dreamhost and I can clone other git repos without a problem. Are you having problems cloning repositories or doing git pull requests? Because I can do both on my version of git without curl compiled in.

  • [...] Hosting Git Repositories on Dreamhost | tail -f development.log I chose to simply compile Git locally into my DH account and connect to it over SSH. This blog post on Autopragmatic.com got me about 90% of the way toward completion, so the next few steps are taken directly from his post. (tags: git dreamhost hosting repository repo) [...]

  • Nice write up, this worked perfectly on my bluehost account. Also, you can use this with multiple developers, they just have to add their ssh public keys to the ~/.ssh/authorized_hosts file and have an author configured via ‘git config –global user.name’. This method does have the drawback that it also grants them unrestricted shell access to your web host, which you may not want.

  • agreed. If you are going to allow multiple developers access to your git repositories, I’d look into using a tool like gitosis instead of doing this method. It’s not smart at all to allow shell access to other users (in my opinion).

  • Craig:

    Couldn’t figure out CURL, but I was luckily wrong about needing it! I think my earlier issue with cloning a remote git repository was that I was trying to clone a GitHub repo without already providing GitHub with my public key, via their website (their documentation was lacking until my friend added this to their wiki just the other day!). Seems I can do it just fine now.

    Nik, Craig:

    Why not do what I did and simply restrict access via your system’s access control model? I’ve configured a user group for each project and assigned it to the applicable repository directory.

  • Yeah, that would work nicely as well.

  • Just wanted to warn that, in case you go the route I did, make sure you setgid of the repo directory (chmod g+s /path/to/repodir) so that newly created files will be owned by the repo dir’s group, instead of whichever primary group (pg*******) that DreamHost assigned to each of your users. Otherwise, you might run into this problem when other users create new objects, as I have!:

    http://kerneltrap.org/mailarchive/git/2008/11/26/4243324/threadt

  • [...] this to work, you need to install git on your dreamhost account. Nice step by step walkthrough Official Wiki [...]

  • commenting usually isnt my thing, but ive spent an hour on the site, so thanks for the info

  • As a note, I have been able to get this working without rebuilding git. I did download the source to get gitweb though. It would appear that the git version currently installed is newer than the one mentioned.

  • Hi great tutorial!!!

    I don’t seem to have this file gitweb.cgi in git src/gitweb folder, where can I find it? Version 1.5.6.5

    Thanks,

    Peter.

  • I believe the implementation of gitweb changed a bit after I made this post. I really need to update this post for the latest 1.6.x version of git.

    Either way, you now need to generate the gitweb.cgi file yourself from the gitweb.perl file. The gitweb.cgi file is no longer included by default.

    View the INSTALL file in your gitweb/ directory to see how this is done.

  • [...] http://www.gogloom.com/FreeNode/git/ – most useful ! [...]

  • [...] Hosting Git Repositories on Dreamhost | tail -f development.log (tags: git) Posted by Nick on Wednesday, December 30, 2009, at 8:03 am. Filed under del.icio.us. Follow any responses to this post with its comments RSS feed. You can post a comment or trackback from your blog. [...]

  • I did not get past Step #1. It seems Dreamhost has already installed git, though it is an older version of what is available today (git 1.4.4.4 as opposed to the git 1.7.0.5 I tried to install).

    I also had trouble getting a PATH setting in .bashrc to take hold. Even if you logout and login via ssh, the changes don’t happen. You need to run this command:

    . ${HOME}/.bashrc

    to get the changes to immediately take. Even then, since the ‘packages/bin’ path comes after ‘/usr/bin’ in printenv, when you type ‘git –version’ you get the /usr/bin one, not the one you just installed.

    I guess it’s a moot point, given Dreamhost installed git for me – so I don’t need to install it myself. Still, I wonder how I can override their version with the one I installed. I’m sure I can figure this out with a few minutes of searching, but thought I’d throw it out there.

  • you probably need to source your .bashrc file in your .bash_profile file. Meaning add the following to your ~/.bash_profile file:

    source ~/.bashrc

    Then if you set “PATH=~/packages/bin:$PATH” in your .bashrc file, your local packages/bin binaries will always be found before the Dreamhost installed binaries. Alternatively, you could also just put all this directly into your .bash_profile file and not in your .bashrc.

    But, if you are happy with the version 1.4.4.4 installed by git, feel free to use it instead of installing your own version. Given that git is now at version 1.7.0.5, I’d highly recommend installing the latest version if you can though.

  • Ahhh, yes. Put the $PATH variable AFTER the new PATH value …. that makes sense!

    Thanks! I was going to go forward with 1.4.4.4, but I’d of course rather use latest and greatest. Much appreciated!

  • I used the newgit method in my .bashrc file, but when I initialize a new repository I git a failed error message due to this line:

    chmod a+x hooks/post-update

    message:
    chmod: cannot access `hooks/post-update’: No such file or directory

    The hooks/post-update dirs are not there. Are they supposed to get made? And why make them world readable?

  • @b01

    What version of git did you install?

  • I used your instructions to install git 1.7.1 on dream host, but I didn’t setup my bash to load my local bin folder before the PATH like you suggested:

    “PATH=~/packages/bin:$PATH”

    So that probably means I’m just using the DH git installed.

    Anyway, I read about git-hooks here:
    http://www.kernel.org/pub/software/scm/git/docs/githooks.html

    and I now have an idea of what that line does, when I run newgit, everything seems good, so I went to the hooks directory in one of my repositories and noticed that the script “post-update” is still named “post-update.sample.” SO the error was self explanatory. I need to modify the script to rename the update script then chmd a+w. But is it safe to make it writable by all on this script, I don’t even know what it does!?

    Thanks for the write-up :)

  • @bo1

    You aren’t making it world writeable, just executable (a+x).

    If I recall correctly, this is because of how DH handles user (UID) access to these file during a push request

  • O.K., newgit runs without errors now. I added the following line to the newgit method (just before the chmod on the update script):

    “mv hooks/post-update.sample hooks/post-update”

    now when I push from my computer to my remote DH account the update script will be run so that I will see my push immediately when I access my repository via web http://git.mydomain.net

  • [...] might be a good idea to setup a private git repository. Following pointers from atwo year old post here, I was successfully able to setup a personal web hosted git [...]

  • [...] If you don’t have git installed in you box, have an old version or if for some other reason your need to compile git, follow Craig’s instructions in Hosting Git Repositories on Dreamhost. [...]

  • [...] http://wiki.dreamhost.com/Git http://craigjolicoeur.com/blog/hosting-git-repositories-on-dreamhost http://priodev.blogspot.com/2010/02/hosting-your-git-repository-on.html [...]

  • [...] While, I’ve certainly followed a few cool projects on there, and I’ve wrestled to get git working on Dreamhost, and I’ve even done some command line work with the repository, I haven’t yet set up a [...]

  • [...] so. I followed the basic outline in “Step 4. Setup GitWeb (optional)” of this excellent blog post to get it up and [...]

© 2014 Craig P Jolicoeur.