Monday, April 14th, 2008...9:12 am
Hosting Git Repositories on Dreamhost
Over the weekend I went searching for a way to host remote git repositories on my Dreamhost (DH) account. I do a fair bit of development year round and up to this point have been hosting all my code on my DH account using Subverision. Over the past few weeks I’ve been making the move from Subversion to Git on my local dev machines, and figured it was time to start hosting my git repositories remotely as well.
It took a lot of searching around online and some tinkering, but I was able to get everything setup properly and in the hopes of saving everyone else some time, I’ll post my results here.
First, if you take a look at the DH wiki entry for Git, you will see the very first method described is to run Git on your DH account using WebDAV. If you’re like me, thinking about doing Git over WebDAV probably just left you scratching your head. I think the reason why this is advocated, is because of the way DH setups SSH accounts. By doing git over SSH to a DH account, only one person would be able to remotely push changes, which is fine for my needs. But, if you need to give multiple users access for pushes, then WebDAV would technically work for you. But, I opted not to go down this route.
I chose to simply compile Git locally into my DH account and connect to it over SSH. This blog post on Autopragmatic.com got me about 90% of the way toward completion, so the next few steps are taken directly from his post.
Step 1. Compile and Install Git on Dreamhost
The first thing that needs to be done is to compile Git locally into your DH account. I personally have a ~/src directory where I download and compile all my code and I have a ~/packages directory where I install my binaries. You may have a different setup so if so, just substitute those paths accordingly. Also, make sure to substitute $USER with your DH username. (Also note the current stable version of git might be different from what you see here. At the time of this writing, 1.5.4.rc4 was the most recent version, but feel free to install a newer version if available.)
$ cd ~/src $ wget http://www.kernel.org/pub/software/scm/git/git-1.5.4.rc4.tar.gz $ tar xvzf git-1.5.4.rc4.tar.gz $ cd git-1.5.4.rc4 $ ./configure --prefix=/home/$USER/packages NO_CURL=1 NO_MMAP=1 $ make $ make install $ git --version
There are a few things to be aware of. First, the NO_MMAP=1 parameter is necessary to keep the DH process police from killing your Git process. Git normally creates two 250MB memory map files, and DH will kill your Git process since it sees it consuming over 500MB of memory. I also used the NO_CURL=1 parameter as I will not be needing my git repositories to have the ability to pull from other remote locations. If you do need this functionality, then you will also need to download, compile and install libcurl, but that is beyond the scope of this post. Also, make sure that you have the path of your installed binaries in your $PATH or that last command of git –version will fail. In my case, I have added /home/$USER/packages/bin to my $PATH in my ~/.bashrc file.
Step 2. Create a new bare Git repository
Now that you have installed git, you can start making new repositories. You can, of course, store these repositories anywhere you want. I chose to create a whole new sub-domain under the Dreamhost Domains Panel. Some people prefer to instead create a git subdirectory under an existing domain. Any way is fine, the choice is yours. For the rest of this post I’ll simply be refering to git.example.com as the <PATH> of my git repositories. You will change this to your path.
If you plan on making multiple repositories, the easiest thing to do is simply automate the process by adding the following function to your ~/.bashrc file.
newgit()
{
if [ -z $1 ]; then
echo "usage: $FUNCNAME project-name.git"
else
gitdir="/home/$USER/$PATH/$1"
mkdir $gitdir
pushd $gitdir
git --bare init
git --bare update-server-info
chmod a+x hooks/post-update
touch git-daemon-export-ok
popd
fi
}
This way you can simply login to your DH account and type:
$ newgit my-new-repos.git
and a new bare repository will be created for you in your <PATH>. Or if you prefer to create a new repository from your local box you could even do it over SSH:
$ ssh $USER@$MACHINE.dreamhost.com newgit my-new-repos.git
Step 3. Pull and push to your new repository
Theoretically, you should now be able to pull and push from this bare repository using the following command:
$ git clone ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/repo-name.git $ git push origin
However, I found this did not work for me and resulted in the following errors:
Initialized empty Git repository in ~/src/newrepo/.git/ Bad port ” fatal: no matching remote head fetch-pack from ’ssh://$USER@$MACHINE.dreamhost.com:/home/$USER/$PATH/my-new-repo.git’ failed.
The problem is that the new repository is bare and has absolutely no data in it at all. That led me on another search that brought me to this excellent writeup that got me the rest of the way home. What I did was simply changed directories into my local git repository. If you haven’t even started your codebase locally yet, I would recommend going ahead and doing that.
Create a new directory and create or add in your files, then do the following to setup a local git repos:
$ cd /my/code/path $ git init $ git add . $ git commit -m "initial import"
That will create a local git repository in your working directory and commit the initial files. Next follow the next set of commands to add your DH repos as a remote and push your initial codebase up to it.
### assuming ssh://server/remote.git resolves to an empty, bare git repo ### and that we are chdir()'d to the local repository: $ git push --all ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/my-new-repos.git $ git remote add origin ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/my-new-repos.git $ git config branch.master.remote origin $ git config branch.master.merge refs/heads/master $ git fetch $ git merge master $ git branch -a * master origin/master
Now you can simply work locally and do a
$ git push
to push to your DH remote. And if you ever need to you can simply execute a
$ git pull
to pull down and merge remote changes.
Step 4. Setup GitWeb (optional)
If you want to be able to access your git repositories via a web browser, you need to setup GitWeb. GitWeb ships with the git source code so you already have it available. You just need to copy the GitWeb files into your git repository directory and make them executable.
cd <PATH>; cp ~/src/git-1.5.4.rc4/gitweb/git* . rm gitweb.perl chmod 755 gitweb.cgi
Its a good idea to create a configuration file as well:
touch gitweb_config.perl vim gitweb_config.perl
and put this as the
gitweb_config.perl
content:
# where is the git binary?
$GIT = "/home/$USER/packages/bin/git";
# where are our git project repositories?
$projectroot = "/home/$USER/git.example.com";
# what do we call our projects in the gitweb UI?
$home_link_str = "My Git Projects"
# where are the files we need for gitweb to display?
@stylesheets = ("gitweb.css");
$logo = "git-logo.png";
$favicon = "git-favicon.png";
# what do we call this site?
$site_name = "My Personal Git Repositories";
Then to get it to actually run in the browser you need to tell apache to serve the files up via CGI. Edit your
.htaccess
file with the following info:
Options +ExecCGI RewriteEngine On RewriteRule ^$ gitweb.cgi RewriteRule ^([?].*)$ gitweb.cgi$1
Now you should be able to access your git repositories from you web browser at http://git.example.com.
I personally went one step further and password protected my repositories. If you don’t want your git repositories exposed to the whole world, you might want to do the same thing. I used simply HTTP authentication by adding the following to the bottom of my
.htaccess
file:
AuthType Basic AuthName "Private Git Repository Access" AuthUserFile /home/$USER/.htpasswd Require valid-user
This, of course, requires that you have proper
.htpasswd
file setup which is beyond the scope of this tutorial.
31 Comments
October 20th, 2008 at 12:59 am
Excellent write up. I followed this straight through steps 1-3 with my hosting company Site5 and only ran into one minor snag which was having to ask them to enable PermitUserEnvironment so that ~/.ssh/environment could set the path to my git binaries in ~/bin. Again, nice helpful write up.
–Garrick West
November 2nd, 2008 at 4:49 pm
Thanks I used this to install Git on my webhost.
I believe Git is no longer dependent on libcurl, so the NO_CURL=1 option isn’t needed anymore.
Also I had to write some php scripts to install since I didn’t have SSH access to my webhost. If anyone is in the same boat I posted the scripts here:
http://sharpideas.ca/Software/Dev/SourceControl/Git/InstallGit.html
November 21st, 2008 at 12:44 pm
Wow. THANK you!
December 24th, 2008 at 1:23 pm
Any idea on how to compile Git with Curl support (so I can git clone remote sources)? I cannot find out how to do this without being able to install debs (which I cannot do without root access on DreamHost).
December 24th, 2008 at 1:56 pm
David, if you also install libcurl (http://curl.haxx.se/) you should be able to leave the NO_CURL=1 option off of the git compile command to get curl support. Look in Step 1 of the tutorial above.
I compiled git without curl support on Dreamhost and I can clone other git repos without a problem. Are you having problems cloning repositories or doing git pull requests? Because I can do both on my version of git without curl compiled in.
January 24th, 2009 at 7:04 pm
[...] Hosting Git Repositories on Dreamhost | tail -f development.log I chose to simply compile Git locally into my DH account and connect to it over SSH. This blog post on Autopragmatic.com got me about 90% of the way toward completion, so the next few steps are taken directly from his post. (tags: git dreamhost hosting repository repo) [...]
January 29th, 2009 at 12:00 pm
Nice write up, this worked perfectly on my bluehost account. Also, you can use this with multiple developers, they just have to add their ssh public keys to the ~/.ssh/authorized_hosts file and have an author configured via ‘git config –global user.name’. This method does have the drawback that it also grants them unrestricted shell access to your web host, which you may not want.
January 29th, 2009 at 12:03 pm
agreed. If you are going to allow multiple developers access to your git repositories, I’d look into using a tool like gitosis instead of doing this method. It’s not smart at all to allow shell access to other users (in my opinion).
January 30th, 2009 at 2:26 pm
Craig:
Couldn’t figure out CURL, but I was luckily wrong about needing it! I think my earlier issue with cloning a remote git repository was that I was trying to clone a GitHub repo without already providing GitHub with my public key, via their website (their documentation was lacking until my friend added this to their wiki just the other day!). Seems I can do it just fine now.
Nik, Craig:
Why not do what I did and simply restrict access via your system’s access control model? I’ve configured a user group for each project and assigned it to the applicable repository directory.
January 30th, 2009 at 2:31 pm
Yeah, that would work nicely as well.
February 5th, 2009 at 7:14 pm
Just wanted to warn that, in case you go the route I did, make sure you setgid of the repo directory (chmod g+s /path/to/repodir) so that newly created files will be owned by the repo dir’s group, instead of whichever primary group (pg*******) that DreamHost assigned to each of your users. Otherwise, you might run into this problem when other users create new objects, as I have!:
http://kerneltrap.org/mailarchive/git/2008/11/26/4243324/threadt
February 18th, 2009 at 4:08 pm
[...] this to work, you need to install git on your dreamhost account. Nice step by step walkthrough Official Wiki [...]
March 1st, 2009 at 5:00 pm
commenting usually isnt my thing, but ive spent an hour on the site, so thanks for the info
March 14th, 2009 at 12:59 am
As a note, I have been able to get this working without rebuilding git. I did download the source to get gitweb though. It would appear that the git version currently installed is newer than the one mentioned.
June 26th, 2009 at 12:58 am
Hi great tutorial!!!
I don’t seem to have this file gitweb.cgi in git src/gitweb folder, where can I find it? Version 1.5.6.5
Thanks,
Peter.
June 26th, 2009 at 1:31 pm
I believe the implementation of gitweb changed a bit after I made this post. I really need to update this post for the latest 1.6.x version of git.
Either way, you now need to generate the gitweb.cgi file yourself from the gitweb.perl file. The gitweb.cgi file is no longer included by default.
View the INSTALL file in your gitweb/ directory to see how this is done.
September 29th, 2009 at 12:09 pm
[...] http://www.gogloom.com/FreeNode/git/ – most useful ! [...]
December 30th, 2009 at 8:05 am
[...] Hosting Git Repositories on Dreamhost | tail -f development.log (tags: git) Posted by Nick on Wednesday, December 30, 2009, at 8:03 am. Filed under del.icio.us. Follow any responses to this post with its comments RSS feed. You can post a comment or trackback from your blog. [...]
April 21st, 2010 at 4:05 pm
I did not get past Step #1. It seems Dreamhost has already installed git, though it is an older version of what is available today (git 1.4.4.4 as opposed to the git 1.7.0.5 I tried to install).
I also had trouble getting a PATH setting in .bashrc to take hold. Even if you logout and login via ssh, the changes don’t happen. You need to run this command:
. ${HOME}/.bashrc
to get the changes to immediately take. Even then, since the ‘packages/bin’ path comes after ‘/usr/bin’ in printenv, when you type ‘git –version’ you get the /usr/bin one, not the one you just installed.
I guess it’s a moot point, given Dreamhost installed git for me – so I don’t need to install it myself. Still, I wonder how I can override their version with the one I installed. I’m sure I can figure this out with a few minutes of searching, but thought I’d throw it out there.
April 22nd, 2010 at 8:35 am
you probably need to source your .bashrc file in your .bash_profile file. Meaning add the following to your ~/.bash_profile file:
source ~/.bashrc
Then if you set “PATH=~/packages/bin:$PATH” in your .bashrc file, your local packages/bin binaries will always be found before the Dreamhost installed binaries. Alternatively, you could also just put all this directly into your .bash_profile file and not in your .bashrc.
But, if you are happy with the version 1.4.4.4 installed by git, feel free to use it instead of installing your own version. Given that git is now at version 1.7.0.5, I’d highly recommend installing the latest version if you can though.
April 22nd, 2010 at 9:09 am
Ahhh, yes. Put the $PATH variable AFTER the new PATH value …. that makes sense!
Thanks! I was going to go forward with 1.4.4.4, but I’d of course rather use latest and greatest. Much appreciated!
June 15th, 2010 at 4:06 pm
I used the newgit method in my .bashrc file, but when I initialize a new repository I git a failed error message due to this line:
chmod a+x hooks/post-update
message:
chmod: cannot access `hooks/post-update’: No such file or directory
The hooks/post-update dirs are not there. Are they supposed to get made? And why make them world readable?
June 16th, 2010 at 8:06 am
@b01
What version of git did you install?
June 16th, 2010 at 11:15 am
I used your instructions to install git 1.7.1 on dream host, but I didn’t setup my bash to load my local bin folder before the PATH like you suggested:
“PATH=~/packages/bin:$PATH”
So that probably means I’m just using the DH git installed.
Anyway, I read about git-hooks here:
http://www.kernel.org/pub/software/scm/git/docs/githooks.html
and I now have an idea of what that line does, when I run newgit, everything seems good, so I went to the hooks directory in one of my repositories and noticed that the script “post-update” is still named “post-update.sample.” SO the error was self explanatory. I need to modify the script to rename the update script then chmd a+w. But is it safe to make it writable by all on this script, I don’t even know what it does!?
Thanks for the write-up :)
June 16th, 2010 at 11:19 am
@bo1
You aren’t making it world writeable, just executable (a+x).
If I recall correctly, this is because of how DH handles user (UID) access to these file during a push request
June 16th, 2010 at 11:57 am
O.K., newgit runs without errors now. I added the following line to the newgit method (just before the chmod on the update script):
“mv hooks/post-update.sample hooks/post-update”
now when I push from my computer to my remote DH account the update script will be run so that I will see my push immediately when I access my repository via web http://git.mydomain.net
August 25th, 2010 at 7:52 pm
[...] might be a good idea to setup a private git repository. Following pointers from atwo year old post here, I was successfully able to setup a personal web hosted git [...]
November 8th, 2010 at 7:58 pm
[...] If you don’t have git installed in you box, have an old version or if for some other reason your need to compile git, follow Craig’s instructions in Hosting Git Repositories on Dreamhost. [...]
December 17th, 2010 at 4:10 pm
[...] http://wiki.dreamhost.com/Git http://craigjolicoeur.com/blog/hosting-git-repositories-on-dreamhost http://priodev.blogspot.com/2010/02/hosting-your-git-repository-on.html [...]
March 20th, 2011 at 3:06 pm
[...] While, I’ve certainly followed a few cool projects on there, and I’ve wrestled to get git working on Dreamhost, and I’ve even done some command line work with the repository, I haven’t yet set up a [...]
July 9th, 2011 at 10:49 am
[...] so. I followed the basic outline in “Step 4. Setup GitWeb (optional)” of this excellent blog post to get it up and [...]