tail -f development.log

I ran into a situation today while working on a client project that was resulting in some “funky” output.

I’ve been using Bob Aman’s standard FeedTools library to grab and parse some RSS feeds for output in a WordPress site.

Now, it was obvious to me this had something to do with character encoding issues so I dug in to find out the problem. It turns out FeedTools converts the feeds it parses to UTF-8 encoding, which is great, but can result in some strange characters in the string depending on what the original feed encoding was.

Viewing the native feeds in a feed reader I could see they were simply quotes, apostrophes and ellipses characters. I needed a way to either convert these character strings into ASCII printable characters or simply discard them altogether.

I didn’t want to go through the process of setting up a translation table for every possible character code that might possibly show up in a feed title, so I simply chose to discard the “unprintable” strings completely. I came across this post by Obie Fernandez and decided his simple method would work for me.

require 'iconv'

class String
  def to_ascii_iconv
    converter = Iconv.new('ASCII//IGNORE//TRANSLIT', 'UTF-8')
    converter.iconv(self).unpack('U*').select{ |cp| cp < 127 }.pack('U*')
  end
end

Ruby’s built-in Iconv library can be used for charset conversions. I extended the String class to include a new to_ascii_iconv method. This method creates a new Iconv converter that will convert from UTF-8 encoding to ASCII encoding. The IGNORE and TRANSLIT flags tell the converter to ignore errors and transliterate accented chracters to an appropriate charcter in the ASCII charset. Next, I use the newly created converter to do the conversion and then strip out any characters with decimal values higher than 127 (non-ASCII printable characters).

The results are exactly what I needed. No “funky” characters in my output.

While everyone is busy talking about the new applications available from the AppStore and the new iPhone 3G, I think some of the best iPhone 2.0 OS enhancements have yet to be talked about.

So far, in my book, two of the best new features include multiple calendar support in iCal and the ability to bulk manage emails in Mail.app.

This may be a “small enhancement” in the grand scheme of things regarding the 2.0 OS release, but in my book, it’s one of the best.

Over the past two days I have made several pretty significant updates to the FeedCache plugin for WordPress.

Version 1.0.4 is now available at wordpress.org for you to download.

The most significant changes are related to the plugin configuration and Ruby CRON script processing. FeedCache no longer accesses the file system for storing configuration and RSS feed caches. There is now a single master configuration file and a YAML file with all the feed group listings. I’m using the spyc php yaml class library for the YAML processing and it works as promised.

The Ruby CRON script now uses the more standard FeedTools gem instead of the old feedparser library I was using. FeedTools seems to be much more stable and memory-friendly then feedparser and I’m pretty happy with it.

Another HUGE improvement for the Ruby CRON script is the fact that all RSS feeds are now cached directly into your WordPress database instead of in plaintext cache files like previous versions. You will need the ActiveRecord gem available on your server (along with Rubygems, of course).

If you are hosted on Dreamhost, you already have all the necessary gems installed automatically so you won’t need to install them yourself. If you are like man and host on your own dedicated server or even a VPS, just install the gems with the usual commands:

sudo gem install

I also have FeedCache hosted up on GitHub now if you are interested and are a GitHub user. If you want to help work on FeedCache or expand/enhance the plugin, fork the code on GitHub and submit any patches to me there.

Over the weekend I went searching for a way to host remote git repositories on my Dreamhost (DH) account. I do a fair bit of development year round and up to this point have been hosting all my code on my DH account using Subverision. Over the past few weeks I’ve been making the move from Subversion to Git on my local dev machines, and figured it was time to start hosting my git repositories remotely as well.

It took a lot of searching around online and some tinkering, but I was able to get everything setup properly and in the hopes of saving everyone else some time, I’ll post my results here.

First, if you take a look at the DH wiki entry for Git, you will see the very first method described is to run Git on your DH account using WebDAV. If you’re like me, thinking about doing Git over WebDAV probably just left you scratching your head. I think the reason why this is advocated, is because of the way DH setups SSH accounts. By doing git over SSH to a DH account, only one person would be able to remotely push changes, which is fine for my needs. But, if you need to give multiple users access for pushes, then WebDAV would technically work for you. But, I opted not to go down this route.

I chose to simply compile Git locally into my DH account and connect to it over SSH. This blog post on Autopragmatic.com got me about 90% of the way toward completion, so the next few steps are taken directly from his post.

Step 1. Compile and Install Git on Dreamhost

The first thing that needs to be done is to compile Git locally into your DH account. I personally have a ~/src directory where I download and compile all my code and I have a ~/packages directory where I install my binaries. You may have a different setup so if so, just substitute those paths accordingly. Also, make sure to substitute $USER with your DH username. (Also note the current stable version of git might be different from what you see here. At the time of this writing, 1.5.4.rc4 was the most recent version, but feel free to install a newer version if available.)

$ cd ~/src
$ wget http://www.kernel.org/pub/software/scm/git/git-1.5.4.rc4.tar.gz
$ tar xvzf git-1.5.4.rc4.tar.gz
$ cd git-1.5.4.rc4
$ ./configure --prefix=/home/$USER/packages NO_CURL=1 NO_MMAP=1
$ make
$ make install
$ git --version

There are a few things to be aware of. First, the NO_MMAP=1 parameter is necessary to keep the DH process police from killing your Git process. Git normally creates two 250MB memory map files, and DH will kill your Git process since it sees it consuming over 500MB of memory. I also used the NO_CURL=1 parameter as I will not be needing my git repositories to have the ability to pull from other remote locations. If you do need this functionality, then you will also need to download, compile and install libcurl, but that is beyond the scope of this post. Also, make sure that you have the path of your installed binaries in your $PATH or that last command of git –version will fail. In my case, I have added /home/$USER/packages/bin to my $PATH in my ~/.bashrc file.

Step 2. Create a new bare Git repository

Now that you have installed git, you can start making new repositories. You can, of course, store these repositories anywhere you want. I chose to create a whole new sub-domain under the Dreamhost Domains Panel. Some people prefer to instead create a git subdirectory under an existing domain. Any way is fine, the choice is yours. For the rest of this post I’ll simply be refering to git.example.com as the <PATH> of my git repositories. You will change this to your path.

If you plan on making multiple repositories, the easiest thing to do is simply automate the process by adding the following function to your ~/.bashrc file.

newgit()
{
    if [ -z $1 ]; then
        echo "usage: $FUNCNAME project-name.git"
    else
        gitdir="/home/$USER/$PATH/$1"
        mkdir $gitdir
        pushd $gitdir
        git --bare init
        git --bare update-server-info
        chmod a+x hooks/post-update
        touch git-daemon-export-ok
        popd
    fi
}

This way you can simply login to your DH account and type:

$ newgit my-new-repos.git

and a new bare repository will be created for you in your <PATH>. Or if you prefer to create a new repository from your local box you could even do it over SSH:

$ ssh $USER@$MACHINE.dreamhost.com newgit my-new-repos.git

Step 3. Pull and push to your new repository

Theoretically, you should now be able to pull and push from this bare repository using the following command:

$ git clone ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/repo-name.git
$ git push origin

However, I found this did not work for me and resulted in the following errors:

Initialized empty Git repository in ~/src/newrepo/.git/
Bad port ”
fatal: no matching remote head
fetch-pack from ’ssh://$USER@$MACHINE.dreamhost.com:/home/$USER/$PATH/my-new-repo.git’ failed.

The problem is that the new repository is bare and has absolutely no data in it at all. That led me on another search that brought me to this excellent writeup that got me the rest of the way home. What I did was simply changed directories into my local git repository. If you haven’t even started your codebase locally yet, I would recommend going ahead and doing that.

Create a new directory and create or add in your files, then do the following to setup a local git repos:

$ cd /my/code/path
$ git init
$ git add .
$ git commit -m "initial import"

That will create a local git repository in your working directory and commit the initial files. Next follow the next set of commands to add your DH repos as a remote and push your initial codebase up to it.

### assuming ssh://server/remote.git resolves to an empty, bare git repo
### and that we are chdir()'d to the local repository:
$ git push --all ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/my-new-repos.git
$ git remote add origin ssh://$USER@$MACHINE.dreamhost.com/home/$USER/$PATH/my-new-repos.git
$ git config branch.master.remote origin
$ git config branch.master.merge refs/heads/master
$ git fetch
$ git merge master
$ git branch -a
* master
  origin/master

Now you can simply work locally and do a

$ git push

to push to your DH remote. And if you ever need to you can simply execute a

$ git pull

to pull down and merge remote changes.

Step 4. Setup GitWeb (optional)

If you want to be able to access your git repositories via a web browser, you need to setup GitWeb. GitWeb ships with the git source code so you already have it available. You just need to copy the GitWeb files into your git repository directory and make them executable.

cd <PATH>;
cp ~/src/git-1.5.4.rc4/gitweb/git* .
rm gitweb.perl
chmod 755 gitweb.cgi

Its a good idea to create a configuration file as well:

touch gitweb_config.perl
vim gitweb_config.perl

and put this as the

gitweb_config.perl

content:

# where is the git binary?
$GIT = "/home/$USER/packages/bin/git";

# where are our git project repositories?
$projectroot = "/home/$USER/git.example.com";

# what do we call our projects in the gitweb UI?
$home_link_str = "My Git Projects"

#  where are the files we need for gitweb to display?
@stylesheets = ("gitweb.css");
$logo = "git-logo.png";
$favicon = "git-favicon.png";

# what do we call this site?
$site_name = "My Personal Git Repositories";

Then to get it to actually run in the browser you need to tell apache to serve the files up via CGI. Edit your

.htaccess

file with the following info:

Options +ExecCGI
RewriteEngine On
RewriteRule ^$ gitweb.cgi
RewriteRule ^([?].*)$ gitweb.cgi$1

Now you should be able to access your git repositories from you web browser at http://git.example.com.

I personally went one step further and password protected my repositories. If you don’t want your git repositories exposed to the whole world, you might want to do the same thing. I used simply HTTP authentication by adding the following to the bottom of my

.htaccess

file:

AuthType Basic
AuthName "Private Git Repository Access"
AuthUserFile /home/$USER/.htpasswd
Require valid-user

This, of course, requires that you have proper

.htpasswd

file setup which is beyond the scope of this tutorial.

For some current updates we are doing at wis.dm, I came across a need (or desire) for a Growl type effect for Prototype.

I was very surprised that no one had implemented one yet. I found an implementation for jQuery, YUI, and mooTools. I actually did find a decent implementation called protoGrowl, but it was much too overweight for what I needed since it was originally developed to be used with the Midgard CMS system.

In the end, I really liked the mooTools implementation and decided to try and port it to Prototype. Unfortunately, my Prototype 1.6 skills aren’t as solid as I would like so I sought out help online and came across Thomas Reynolds on gitHub. He agreed to help and so the project began. A few hours later we had the first working version of prototypeGrowl. After a nights rest and some more coding we have a pretty nice implementation of growl for prototype.

Check it out and feel free to fork the repository on gitHub and contribute any tweaks or updates you want.