Posts tagged with “tier”


Sun 11 Nov

Deploying TIER's Village Base Station in Papua

I'm in Papua, Indonesia right now working on a deployment of a system one of my TIER colleagues, Kurtis Heimerl, developed. This is a little blurb I wrote on the TIER blog about our first week here.


Fri 2 Mar

Gitorious on Ubuntu Server 11.10

I set up a Gitorious installation for my research group today, following the quite good instructions from Lucas here. I had to make a couple minor adjustments to his setup steps, which I'll document below.

I started with a fresh installation of Ubuntu Server 11.10. To get started, run the following:

sudo apt-get install build-essential tcl-dev libgeoip-dev postfix apache2 mysql-server mysql-client apg libsqlite3-dev imagemagick libpcre3-dev zlib1g-dev libyaml-dev libmysqlclient-dev apache2-dev libonig-dev ruby-dev rubygems libmysql-ruby libdbd-mysql-ruby libmagick++-dev zip unzip memcached git-core git-svn git-doc git-cvs irb git sphinxsearch libcurl4-openssl-dev libxslt1-dev libxslt-ruby

You'll be asked to set up a mysql root password; remember what you enter.

sudo gem install rake daemons rmagick stompserver passenger bundler

cd /var/www
git clone git://gitorious.org/gitorious/mainline.git gitorious
git submodule init
git submodule update

cd /var/www/gitorious/doc/templates/ubuntu/ && sudo cp git-daemon git-poller git-ultrasphinx stomp /etc/init.d/ && cd /etc/init.d/ && sudo chmod 755 git-daemon git-poller git-ultrasphinx stomp
sudo update-rc.d git-daemon defaults && sudo     update-rc.d git-poller defaults &&   sudo   update-rc.d git-ultrasphinx defaults && sudo    update-rc.d stomp defaults

sudo ln -s /usr /opt/ruby-enterprise

sudo $(gem contents passenger | grep passenger-install-apache2-module)

At this point, the installer the last script runs will ask you to copy some configuration code into a file. It will look something like what's in the echo below:

echo "LoadModule passenger_module /var/lib/gems/1.8/gems/passenger-3.0.11/ext/apache2/mod_passenger.so
    PassengerRoot /var/lib/gems/1.8/gems/passenger-3.0.11
    PassengerRuby /usr/bin/ruby1.8" | sudo tee /etc/apache2/mods-available/passenger.load

Let's move on.

sudo a2enmod passenger &&  sudo a2enmod rewrite && sudo a2enmod ssl

# NOTE: This step requires thinking. Replace the server name appropriately.
echo "<VirtualHost *:80>
    ServerName your.website.com
        DocumentRoot /var/www/gitorious/public
        </VirtualHost>" | sudo tee /etc/apache2/sites-available/gitorious

# Clearly, use a real SSL cert/key... 
echo "<IfModule mod_ssl.c>
    <VirtualHost _default_:443>
        DocumentRoot /var/www/gitorious/public
        SSLEngine on
        SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
        SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
        BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
    </VirtualHost>
</IfModule>" | sudo tee /etc/apache2/sites-available/gitorious-ssl

sudo a2dissite default && sudo a2dissite default-ssl && sudo a2ensite gitorious && sudo a2ensite gitorious-ssl

mysql -u root -p

# In mysql, enter:
# mysql> GRANT ALL PRIVILEGES ON *.* TO 'gitorious'@'localhost' IDENTIFIED BY '<your password>' WITH GRANT OPTION;
# mysql> FLUSH PRIVILEGES;

cd /var/www/gitorious/ && sudo bundle install && sudo bundle pack

adduser --system --home /var/www/gitorious/ --no-create-home --group --shell /bin/bash git
chown -R git:git /var/www/gitorious

Alright. At this point we're almost done. The next couple steps have some room for creativity (aka, pick options and paths that make sense for how you want to deploy Gitorious).

sudo su git
cd /var/www/gitorious
mkdir .ssh && touch .ssh/authorized_keys && chmod 700 .ssh && chmod 600 .ssh/authorized_keys && mkdir tmp/pids && mkdir repositories && mkdir tarballs

cp config/database.sample.yml config/database.yml && cp config/gitorious.sample.yml config/gitorious.yml && cp config/broker.yml.example config/broker.yml

Follow the configuration instructions provided by Lucas. The next step, quoting him: "Because of an incompatibility of RubyGems with Rails < 2.3.11 you need to add the following line at the top of config/boot.rb:"

require 'thread'

(I can verify this is still necessary in 11.10). Back to Lucas, let's wrap up:

export RAILS_ENV=production && bundle exec rake db:create && bundle exec rake db:migrate && bundle exec rake ultrasphinx:bootstrap

# NOTE: The path to bundle has changed in 11.10! This is an update.
crontab -e * * * * * cd /var/www/gitorious && /usr/local/bin/bundle exec rake ultrasphinx:index RAILS_ENV=production

env RAILS_ENV=production ruby1.8 script/create_admin

Now, a couple small changes before you're ready to run. Some paths have changed in 11.10, as noted by a very helpful commenter on Lucas' article. First, we need to update /etc/init.d/stomp. Change GEMS_HOME="/usr/local" to GEMS_HOME=”/usr/local”. Next, we need to edit /etc/init.d/git-daemon and /etc/init.d/git-poller. As provided, these are each run with /usr/bin/ruby; I had to modify each to run with bundle exec. So make the following changes to the two respective files:

# /etc/init.d/git-daemon
GIT_DAEMON="$RUBY_HOME/local/bin/bundle exec $GITORIOUS_HOME/script/git-daemon -d"

# /etc/init.d/git-poller
GIT_POLLER="$RUBY_HOME/local/bin/bundle exec $GITORIOUS_HOME/script/poller"

At this point, I believe you should be able to reboot and have a working installation of Gitorious. Please let me know if I missed anything so I can update this (I'm actually re-enabling comments on this blog just for this). A couple gotchya's I ran into while I was setting this up:

  • Gitorious relies on several background scripts, most of which are located in the script directory. It also relies on stompserver for message passing. If the site is very slow, especially if pushes don't work, repository creation takes a long time, etc., check that both stompserver and poller are running. Note also that you can see the log output of the gitorious scripts under $GITORIOUS_HOME/tmp/pids.
  • If you can't push to a repo, stompserver is probably not running.

I'm pretty excited about having a simple way to host our projects internally. While I'm a big fan of Github, and our group pushes most of our public code there, having a private hosting location will also be helpful for things like in-submission papers, configuration files, and sensitive data. Our particular installation takes advantage of the very reliable file storage system that our department offers, so by using it all of our work can take advantage of their replication and automated backup/recovery systems. This is really important to me after hearing a harrowing tale from our sysadmin of a grad student who lost 3 years of work due to the theft of the only server that contained his data and code. Eek!