Its time to have some bug tracking tools - Installing Trac

We're in the stage where simple notes.txt isn't enough to track our project. I think we need a better bug/feature tracking tools. So, I'm going hunting for a few of the common bug tracking tools that's available on the market.

I narrow down my choice into two, Jira and Trac. I've used Bugzilla before, but it never clicked with me. And also Bugzilla and Mantis have their own horror story. Jira actually caught my attention with the $10 starter pack. But the next packet, 25 user for $1,200 is way out of my league.

So finally my decision goes to Trac. It was simple enough. People won't be distracted or hinder by the many fields they need to fill to submit a bug. Both Jira and Trac support mercurial. Just perfect. And although simple and minimalistic Trac have two very cool feature :

• Timeline – shows current and past project events in order
• Roadmap – shows the upcoming milestones to be achieved in the project

The only drawback in Trac is the interface, it has its way to annoy you when you look at it too long :P

Installation

I wont be installing this from scratch. I will be installing this at the last state of  my infastructure post.

You can install Trac with mercurial support using this three simple steps :

sudo apt-get install python-setuptools
sudo apt-get install python-genshi
sudo apt-get install trac-mercurial

But when i write this blog post the trac version in ubuntu repository is 0.11.7. Since i want to use trac 0.12, I'll be doing some of the things manually.

First I installed the setup tools using apt-get.

sudo apt-get install python-setuptools

Then I'm using setuptools easy_install to install Genshi and Trac

sudo easy_install Genshi
sudo easy_install Trac

And for the trac-mercurial plugin, you need to check out and build it manually. But first you need to install subversion.

sudo apt-get install subversion

check out the code, build the egg, and install it

mkdir trac-plugin
cd trac-plugin
svn co http://svn.edgewall.com/repos/trac/plugins/0.12/mercurial-plugin
cd mercurial-plugin
python setup.py bdist_egg
sudo easy_install dist/TracMercurial-0.12.0.28dev_r10657-py2.6.egg

After finished installing Trac, lets do some test. But first you need to create a database for Trac in mysql. Using mysql root user :

-- It is better to set the innodb as the default storage engine
mysql> create database trac;
mysql> grant all privileges on trac.* to 'trac'@localhost identified by 'mypassword';
mysql> flush privileges

Trying logging out and then loggin in using the new account.

Next, you need to install python library for connecting to mysql from python.

sudo apt-get install python-mysqldb

And the basic setup is complete.

Create A Trac Project

Now we need to create a trac project. Since i want it to be integrated with my mercurial server. I've created the project in my hguser account.

sudo su - hguser

create the directory for your project

mkdir -p ~/trac/myproject

init the project

trac-admin ~/trac/myproject initenv

If you're having a problem with trac-admin not in your path, try logging out and then relogin to your system.

The initenv command will ask some question. On the database connection string input this :

mysql://username:password@hostname:port/databasename

After finish with the environment, we need to modify the configuration file to support mercurial.

vi /home/hguser/trac/myproject/conf/trac.ini

modify the file with the line below :

[components]
tracext.hg.* = enabled[repositories]

[trac]
repository_dir = path_to_default_repository
repository_type = hg

You can start trac server and see the result by accessing http://yourserver:8000/myproject

tracd --port 8000 ~/trac/myproject

Integrating with apache

Mod_python is dead, so I'm choosing mod_wsgi to integrate trac with my apache server. First you need to install mod wsgi and restart the apache2 server

sudo apt-get install libapache2-mod-wsgi
sudo service apache2 restart

sudo to the hguser and create egg-cache directory under ~/trac/myproject

to create the wsgi file you can create it manually, or use the trac-admin deploy method like below :

trac-admin trac/myproject deploy /tmp/deploy

it will create the .cgi .fcgi .wsgi file under the cgi-bin directory. Then you can copy the file to the real cgi-bin directory

sudo mkdir /usr/lib/cgi-bin/trac
sudo cp /tmp/deploy/cgi-bin/trac.wsgi /usr/lib/cgi-bin/trac/myproject.wsgi
sudo chmod +x /usr/lib/cgi-bin/trac/myproject.wsgi

now you need to add the new location to your apache configuration file. Usually it's under site-enabled/000-default-file . I'm using port 8008 for this purpose but later will create a subdomain for trac :

ServerAdmin webmaster@localhost
DocumentRoot /var/www

#WSGI scrip for trac
WSGIScriptAlias /trac/myproject /usr/lib/cgi-bin/trac/myproject.wsgi

        <Directory /home/hguser/trac/myproject>
            WSGIApplicationGroup %{GLOBAL}
            Order deny,allow
            Allow from all
        </Directory>

After restarting apache, you should be able to access your trac installation in http://yourlocalhost:8008/trac/myproject

While trying to integrate trac with apache, I keep stumbling to an error that said :

TimeoutError: Unable to get database connection within 0 seconds. (TracError: Database "/usr/www/trac/db/trac.db" not found.)

I'm using mysql so trac shouldn't need to create the embedded db. It seems that trac couldn't read the content of my trac.ini files. It turn out that I've stumble to this bug. As a work around I added group read-write permission ,

sudo chmod g+rw ~/trac/myproject/conf/trac.ini

but i wonder, will this create a security problem. Since the trac.ini contains the db access URL ?

Configuring Authentication

To add authentication method, I just added my mercurial hgusers file

        <location /trac/myproject/login>
                AuthType Basic
                AuthName "Trac"
                AuthUserFile /home/hguser/hgusers
                AuthGroupFile /home/hguser/hggroups
                Require group myproject
        </Location>

To be able to use the admin web page you need to add admin privileges to your user

trac-admin ~/trac/myproject permission add frodo TRAC_ADMIN

it is recomended to install the AccountManager plugin. To install it run

sudo easy_install https://trac-hacks.org/svn/accountmanagerplugin/trunk

and then go to the admin tab > plugin , and enable the features

Removing anonymous user access

You can disallow anonymous user access by

trac-admin ~trac/myproject permission remove anonymous '*'

Mercurial : Separating two groups

In my mercurial setup a new requirement has come up. I need to separate two groups of user, lets call the first group research and the second one project. A user can belong to one of the groups or both. My mercurial server is setup like this, and then I modified it a little to enable the virtual server name right here. To implement this requirement I'm following the public and private repositories and modified it a little.

Configuration

First, what i do is i log-in as the hgusers (user I've setup specially for mercurial, see the first blog post). At the hgusers home directory I've created a new folder called project. And then I move hgusers and hgweb.config from the hg folder to the hgusers main folder.

We need to create a different set of file for our repositories. So copy each of the config files :

$> cp hgweb.config hgweb_project.config

and then modify hgweb_project.config content so it will point to the new project folder. The file content should look like this :

[collections]
/home/hguser/project/ = /home/hguser/project/

now go to our hgweb.cgi directory and copy the file

$> cd /usr/lib/cgi-bin/hg/
$> cp hgweb.cgi hgweb_project.cgi

modify the config variable to point to the project repo folder

config = "/home/hguser/hgweb_project.config"

and finally modify the apache settings

vi /etc/apache2/sites-enabled/000-default

copy the previous ScriptAlias and Location div, and modify it to point to the project location. Another that you need to do is to add the AuthGroupFile, and change The Require line. I will go into detail about this in the next section. The file should look like this :

ScriptAlias /project /usr/lib/cgi-bin/hg/hgweb_project.cgi


AuthType Basic
AuthName "Mercurial repositories"
AuthUserFile /home/hguser/hgusers
AuthGroupFile /home/hguser/hggroups
Require group project


ScriptAlias /repo /usr/lib/cgi-bin/hg/hgweb.cgi


AuthType Basic
AuthName "Mercurial repositories"
AuthUserFile /home/hguser/hgusers
AuthGroupFile /home/hguser/hggroups
Require group research

Creating Group
I already have user sam and frodo from my previous settings. Now i'm adding two more user using htpasswd command. The users are golum and sauron.

Now we need to create the group for these users. To do this go to the hguser home directory and create a new file called hggroups. You can use VI or your favorite text editor to create it. The file content should look like this :

research: frodo sam golumn
project: golumn sauron

the word before the colon is the group name, like the one you specify on the Require group line. And the words after that is the users that belong to the group. In my setup sauron would only able to see the project repository (limiting his access). And so does frodo and sam, they can only see the research repository. While golumn would be able see both repo. Just what i wanted to.

If you wanted to have a lot of user that can access both group, you could create a new group and than add that group to the Require line

Setting new dedicated server

I'm migrating my VPS server to a new dedicated server. So i need to repeat the same proses I've done for the old server to this one. For future reference , I'm going to document my steps :)


Setting swappiness

To set the vm.swappiness permanently you need to edit the /etc/sysctl.conf file and add the line

vm.swappiness=10

The default number usually 60. The smaller the number means that your system is unlikely to swap process from memory to the swap space. To see the change, use :

cat /proc/sys/vm/swappiness

Installing Sun JDK package

I'm still very fond to use the JDK from SUN, to do that in ubuntu you need to follow this steps.

You can add the repository manually by editing /etc/apt/sources.list
or installing the package python-software-properties and run

sudo apt-get install mysql-server-5.1

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"

Then, update the repository information and install the jdk

sudo apt-get update
sudo apt-get install sun-java6-jdk

Installing MySQL

sudo apt-get install mysql-server-5.1
vi /etc/mysql/my.cnf #change the default port

Installing glassfish

wget -b -c -i http://download.java.net/glassfish/3.0.1/release/glassfish-3.0.1.zip -o log.out
sudo apt-get install unzip
unzip glassfish-3.0.1.zip

Start glassfish

./glassfish/bin/asadmin start-domain domain1

Connect to the admin console and change the admin password

http://yourip.net:4848/
enterprise server > administrator password
logout and re-login

If you want a more secure and thorough set-up you can follow this guide. I usually just change the port in the admin console to 80.

Installing Apache

sudo apt-get install apache2

enable reverse proxy

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo /etc/init.d/apache2 restart

add reverse proxy context

sudo vi /etc/apache2/sites-enabled/000-default
ProxyPass /itsme/ http://www.mysite.net:8080/itsme/
ProxyPassReverse /itsme/ http://www.mysite.net:8080/itsme/

if you have a problem "client denied by server configuration proxy" you need to modify the proxy.conf to allow access to your site. So far the only solution i know is by adding allow all or commenting deny all in the proxy.conf file. To edit the conf file, use this command :

sudo vi /etc/apache2/mods-enabled/proxy.conf

if you wan't to pass the original ip through your proxy server follow this link.

Setting mercurial server
To set the mercurial server you can see my previous post.

Using subdomain for mercurial server
To do this you need to add a CNAME entry to your DNS Server. For example I ask my hosting support to add CNAME entry for hg.myserver.net.

After that, you just need to copy the virtual host from the 000-default files and add

ServerName hg.myserver.net

And you should be able to access your hg server using http://hg.myserver.net/hg

I've edited the location and path for this virtual server so i can access it using http://hg.myserver.net/repo. Better than the redundant hg for the url :)

Start glassfish at reboot time
Create a glassfish.sh file at your home directory, copy the code below as the content

#!/bin/sh
#
### BEGIN INIT INFO
# Provides:          scriptname
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     3 4 5
# Default-Stop:      0 1 2 6
# Short-Description: Start daemon at boot time
# Description:       Enable service provided by daemon.
### END INIT INFO

GLASSFISH_HOME="/home/mreunion/glassfish/"

case "$1" in
start)
    $GLASSFISH_HOME/bin/asadmin start-domain
    ;;
stop)
    $GLASSFISH_HOME/bin/asadmin stop-domain
    ;;
restart)
    $GLASSFISH_HOME/bin/asadmin restart-domain
    ;;
force-reload)
    $GLASSFISH_HOME/bin/asadmin restart-domain
    ;;
*)
    echo "usage: $0 (start|stop|restart|help)"
esac

sudo cp glassfish.sh /etc/init.d/
sudo chmod +x glassfish.sh
sudo update-rc.d glassfish.sh defaults

Setting your own private Mercurial server

update : the steps in this post has been updated for using mercurial 1.7.2

First of all I want to say that this wiki link is amazing, it's complete and robust. But even with the best guide sometime you've made mistakes. Never hurt to have many reference :)

So to simply put it,  this is my steps in implementing that guide. As a background i have an ubuntu server that i will be using as my own private mercurial server. I already have some code that works with google mercurial server. So its a matter switching repositories. And I also already have an apache server running on my server.

Now, Lucid lynx comes with mercurial 1.4. If you want to install Mercurial 1.5, 1.6, or 1.7 you need a new repo :

sudo add-apt-repository ppa:mercurial-ppa/releases
sudo apt-get update
sudo apt-get install mercurial

Don't forget to check the version using

hg -v

The steps

Pre-install

Installing mercurial in ubuntu is as easy as running

$>sudo apt-get install mercurial

Finding the hgweb.cgi file

First we need to find the hgweb.cgi file. Usually it resides in /usr/share/doc/mercurial-common/examples/hgweb.cgi . If you can't find it using locate or executing this command should do it.

$>find / -iname hgweb.cgi

go to the /usr/lib/cgi-bin/ and create hg directory. Copy the hgweb.cgi to /usr/lib/cgi-bin/hg/. Then make it executbale using this command

chmod  755

Creating the user and directories

Create user hguser with www-data as its group. it should have the /home/hguser as its default home. Now login as the hguser, change the .bashrc for hguser. You can use

vi ~/.bashrc

Add this line at the end of the file and save it:

umask 002

With the umask set, the file and directories created by the hguser will have group write permission. So the www-data (default apache user) can create and write any file in the repository. Now create hg directory for our repositories, so now we have /home/hguser/hg for our repositories yay !

for my setup I'm using 2 repositories, one for development and one for release. So under the hg directory, I created 2 other directory,

/home/hguser/hg/dev and
/home/hguser/hg/stable

Now you can logout from the hguser and back to your default user.

Setting the apache server

As I mention already have an apache2 server running. My server 80 is already used by something else, so i'm using another port for apache. Lets say port 9000. To do this you must edit the ports.conf file and make the changes accordingly. You can use this command :

sudo vi /etc/apache2/ports.conf

You also need to change the port on the default apache file to 9000. You can use this command :

sudo vi /etc/apach2/sites-enabled/000-default

still in the 000-default files, activate the cgi script. The easiest way is by using Script Alias, add the lines bellow to the 000-default files:

ScriptAlias /hg /usr/lib/cgi-bin/hg/hgweb.cgi

Setting Mercurial Repositories

To enable mercurial for multiple repository, it is easier to use the hgweb.config file.

First login as the hguser.
Create the hgweb.config under the hg directory

The hgweb.config file should contain :

[collections]
/home/hguser/hg/ = /home/hguser/hg/

edit the hgweb.cgi and update the config line to the line bellow :

config = "/home/hguser/hg/hgweb.config"

Restart the apache server, than you should be able to access http://yourserver/hg directory

Allowing Push

I'm allowing all authenticated user to push to the repository, and since the most important thing is make the repo up and running, I'm making SSL implementation none mandatory. So to allow push you need to edit the /etc/mercurial/hgrc and add the following line below

[web]
allow_push = *
push_ssl = false

Adding Authentication

To add the authentication to your mercurial server you need to add the line bellow to the 000-default file, just under the script alias should suffice.

<location /hg>
                AuthType Basic
                AuthName "Mercurial repositories"
                AuthUserFile /home/hguser/hg/hgusers
                Require valid-user
        </Location>

Now login as the hguser and go to the hg directory. To create the hgusers authentication file and create the first user you can use the comand bellow :

htpasswd -c /home/user/hg/hgusers frodo

to add another user you can remove the -c flag

htpasswd /home/user/hg/hgusers sam

restart apache and you should be able to to access the repo at :

http://www.yourserver.com:9000/hg/

Trouble Shooting

The trouble shooting section in the wiki is very good. I just highlight some here since I've encountered them in my installation. 

If you find the error message bellow in your apache2 error log. 

/var/lib/python-support/python2.6/mercurial/hgweb/common.py:24: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6

you're using and older mercurial that work quite right with the latest python implementation. To suppress it add the following lines to the hgweb.cgi immediately after any sys.path adjustment :

import warnings

warnings.simplefilter("ignore", DeprecationWarning)

And if you're able to create a new repo and synch it to your local komputer but  failing when pushing or cloning your existing repo to the server. Try  changing the hguser/hg directory permission recursively to 777. If that did work than you didn't set your permission correctly.

Good luck