Tags: soyouwant
In the last installment we set up Trac, and I hinted at a couple of its features. But what should we actually be using it for?
Well, let's take a look inside.
When you first visit Trac, you're presented with a standard intro page that tells you about Trac. It's useful to start, but it's not the key information we're going to want every time we visit the Wiki. As such, I tend to either shift it to another page, or just add new content above it.
This new content should help developers to work with the new systems you've just set up, so useful content to add includes:
As users may be quite new to these tools, it's usually worth adding a couple of quick reference guides to things like the unix command line or svn commands - either links or you own summaries. Just pasting in the output of svn help is pretty useful.
Beyond this (and while we've not covered these topics yet) it's useful to have notes on:
OK, so much for the wiki functionality. What else is in here? The nav bar at the top of the page gives us a list of Wiki, Timeline, Roadmap, Browse Source, View Tickets, New Ticket and Search.
Wiki we've seen. Timeline we briefly mentioned last time as a summary of changes made to the code and wiki; it tends to be worth a glance occasionally to sanity check changes and commit frequency and messages. Search is useful too; it lets you hunt for info within all areas of the wiki - and if your documentation is searchable, people use it more and add to it. See below for a way to extend this search and add it to a user's browser for even greater ease of use.
Let's take a look at one more option for now. Browse Source is a link to an SVN repository browser. As is generally the way in Trac, it's fairly lightweight and fairly easy to use. It uses repository connection data that's supplied when you initialise your trac project, and gives you an easy way to browse the repository contents, finding out what's where, who changed it, when, and how. You can do all of this with the SVN command line tools, but it's far easier in here, and a repository browser is invaluable for getting the full historical power of version control.
The browser also has the useful feature that, being part of Trac, it integrates smoothly into the wiki and ticketing systems. By typing r1234 (or any SVN revision number) in a wiki page, you'll get an automatic link to that changeset in the repository browser. You can also link to related files using Trac's source: syntax - check out the TracLinks page in your copy of Trac for more info. Also, any wiki links you add to your subversion commit messages will be linked in the Timeline and Changelogs within Trac.
Next time, we'll take a look at task allocation within a project, with the help of the ticketing system within Trac.
Before we do that though, here's how to add Trac search directly into your Firefox or IE7 browser. It's not the most elegant solution, but it works.
First, we need to create and host an Open Search specification document that can act as a plugin for both Firefox and IE. We'll host this file in the root folder of the webserver which we've set up to host subversion and trac, which in our example is /var/www/html for devbox.example.com
Copy the following into /var/www/html/trac-search.xml:
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName>Devbox Trac Search</ShortName>
<Description>Search Devbox Trac</Description>
<Contact>webmaster@devbox.example.com</Contact>
<Image height="16" width="16"
type="image/x-icon">http://devbox.example.com/trac/uberproject/chrome/common/trac.ico</Image>
<Url type="text/html"
template=
"http://devbox.example.com/trac/uberproject/search?q={searchTerms}&noquickjump=1&wiki=on" />
</OpenSearchDescription>
Obviously, adjust hostnames, paths and descriptions to your own system. Note that the URL constructed here only searches the wiki - I find that's usually what I want to do anyway.
Next, locate the template files for your Trac project. In our example, they'll be at /opt/trac/uberproject/templates. We want to edit the header template site_header.cs. This file starts off pretty empty, and we'll add the link to the search plugin, plus some standard javascript that seems to be required to install a search engine. The resulting file looks like this:
<?cs
####################################################################
# Site header - Contents are automatically inserted above Trac HTML
?>
<link rel="search" type="application/opensearchdescription+xml"
title="Devbox Trac search" href="/trac-search.xml">
<script type="text/javascript">
/**
* Install a search engine (opensearch)
* Returns false in case of success (sic!) because that will keep the file link
* from being followed.
*/
function addEngine(engineURL) {
if (window.external && ("AddSearchProvider" in window.external)) {
window.external.AddSearchProvider(engineURL);
return false;
} else {
alert(error_opensearch_unsupported);
return true;
}
}
</script>
<div style="position: absolute; right: 20px; color: #bbb; cursor: pointer">
<a onclick="addEngine('http://devbox.example.com/trac-search.xml');">Install trac search plugin for IE7
& Firefox</a>
</div>
And that's it. Each page of your trac project should now contain an install link for the plugin, which will appear in your search box with the trac 'paw' icon.
Tags: soyouwant
So, everyone's got their sandboxes and dev servers, and they can all code merrily away and share their code. So far, so good. There are two things we now need: planning and documentation. I'll concentrate on the latter for the moment, by means of a Team Wiki.
There's a lot of odds and ends of documentation that needs to be shared in a dev team; things like Where The Code Lives, How The Servers Work and Those Important Planning Decisions. One way of doing this is just to stick all the docs in a shared drive on the server, but I've never seen that work so well; people can't find the docs they need and don't spot new documents and changes - and equally can't comment on any document or change. This strategy also tends to discourage minor hints and notes which can be invaluable. I find that a wiki works well to solve this problem, and the one I tend to use is Trac.
To be fair, calling Trac a wiki is selling it a bit short; it's also a ticket-tracking system and repository browser, and it ties these elements together well. The Trac site can fill you in on a bit more detail, but for the moment we'll just consider it as as useful location for project documentation.
Installing Trac, thanks to the magic of Ubuntu, is dead simple; just type (as root) aptitude install trac. Ain't Linux great?
Actually, in this case, no. Trac can be awkward to fully set up; the ubuntu and debian packages are (at the time of writing) at least a little broken, and you have to do a bit of magic to finish the setup. Here's the full process that currently works for my Ubuntu servers:
Most of trac's installation really is achieved with aptitude install trac, but there's still a fair bit of configuration to do.
Skip this block for now and come back to it if the next steps throw errors. You may get lucky and find it's all been fixed.
One common error is that Clearsilver doesn't properly install its python library in the Ubuntu package. The solution (usually in response to an error in neo_cgi.so) is to install the clearsilver library from source and ensure that you add the –with-python=/python/2.5/dir option when you configure it. Full details on that (and other possibe solutions) at https://launchpad.net/ubuntu/+source/clearsilver/+bug/86685
(you can have multiple projects within one trac install)
This is done with a command-line tool called trac-admin. You need to chose a location for your trac's data files; I tend to go with /opt/trac/whatever.
As trac will be operated via apache, the apache user needs to have full read/write access to all files. So, as root:
root@devbox:/opt/trac# trac-admin uberproject initenv
and answer the questions it asks; for more info, trac-admin –help
then
root@devbox:/opt/trac# chown -R www-data:www-data uberproject
Add the following stanzas to an apache virtual server's config file, eg that for our devbox example:
<Location /trac> SetHandler mod_python PythonInterpreter main_interpreter PythonHandler trac.web.modpython_frontend PythonOption TracEnvParentDir /opt/trac/ PythonOption TracUriRoot /trac </Location>
Note that there are two ways of specifying the location of the trac files. You can set a path to a single trac as TracEnv /full/path/to/trac (eg TracEnv /opt/trac/uberproject) or have the potential for Trac to automatically find multiple trac sets under one parent directory with the newer TracEnvParentDir, which we use in the above example. Note that getting the wrong directory, or using the wrong directive, will lead to some very confusing errors
to allow developers to access the appropriate tools
This is done in a couple of places, and depends on how much public access you want to allow. We'll assume you only want known users to access your system and have no public read access. If you want fully public access, just skip the next steps.
Firstly, you need to create HTTP basic auth protection to your trac URI, as follows:
<LocationMatch "/trac/[^/]+"> AuthType Basic AuthName "Trac" AuthUserFile /etc/apache2/trac.htpasswd Require valid-user </LocationMatch>
You'll also need to set up the htpasswd file as covered previously.
You can then, for each user requiring particular rights, go into trac-admin's interactive mode (root user will allow you to manage the files now owned by the web browser):
root@devbox:/opt/trac# trac-admin uberproject [...] Type: '?' or 'help' for help on commands. Trac [/opt/trac/uberproject]>
Again, I'm not going to duplicate the trac manual here; do as it suggests and type help; look in particular at the 'permission add' command.
Having done all this, you can restart apache and visit your new trac system in a browser.
Once you've got Trac running, all can join in. The features you'll initially find the most useful are the Wiki and the Timeline. I'll introduce the timeline first, because it's simple as well as useful. Simply, it gives you a summary of who changed what and when; it's a combination of subversion's change log and the wiki's revision history, and provides a good way of working out both what you did (or when you did it) and keeping up with other developers' changes. This includes keeping track of who's creating good documentation and change messages, and who isn't.
The wiki is rather more basic than (say) Wikipedia and the syntax is quite old-fashioned (although there's a reasonable internal guide). Page names are generally CamelCase and new pages are created simply by clicking a link to, or visiting the address of, a page that doesn't yet exist. However it's lightweight and fairly easy to use.
In the next installment, I'll go into a little more detail on these, as well as some hints on using the system.
Tags: soyouwant
A mere three posts in, and I think there's going to be enough content to keep me going for a while. I'll be revising the content as I go to make it more coherent and readable, and to add newly discovered tricks and ideas as I go. Comments and questions are welcome - I've fixed the comment system, but you can also mail me (richard@phase.org).
Tags: soyouwant
Last time we got as far as checking out the code from a local repository, and I promised we'd take a quick look at some ways of managing the repository as a network server.
There are three benefits to this:
The first that you don't have to make the whole repository writable by (and therefore susceptible to accidental damage by) all of your developers (and conversely you don't have to manage all the user and group privs and file permissions that all of them might use, which ends up locking someone out of part of the repository).
Secondly, you can give access to developers (or reviewers, managers etc) who don't have direct access to the server or need to work remotely. Even if you're using the shared drive strategy previously covered, network access also lets you use desktop SVN clients rather than ssh'ing in to the linux shell to run commands there. (Personally, I generally find the shell friendly and easier to use, but YMMV...)
Thirdly, you have the possibility of more finely-grained and easier to manage access control; you can set up groups to read and right, and even limit access to certain parts of the repository.
There are two server systems you can use: svnserve and apache (http). We'll use apache here as it works well alongside our test webserver, and has fewer problems with firewalls than svnserve.
To use apache as an svn server, you'll need to install its DAV and SVN modules. In ubuntu, this is simple: sudo apt-get install libapache2-svn (I'll assume you're on apache 2; if not, check your calendar).
Ubuntu will then create a file called /etc/apache2/mods-available/dav_svn.conf, which you'll most likely want to completely comment out and replace with a more useful configuration (but do take a read of it).
(As an aside, you may need to execute a2enmod dav_svn; I can't recall if it's enabled on install. In any case, you'll need to restart apache for it to work).
Your config, inside a standard vhosts configuration, will look much like the following:
<VirtualHost *>
ServerAdmin webmaster@example.com
DocumentRoot /var/www/html
ServerName devbox.example.com
<Location /svn/uberproject >
DAV svn
SVNPath /opt/subversion/uberproject/repository
AuthType Basic
AuthName "Uberproject Subversion Repository"
#<LimitExcept GET PROPFIND OPTIONS REPORT>
Require valid-user
#</LimitExcept>
AuthUserFile /etc/apache2/svn.htpasswd
satisfy all
</Location>
</VirtualHost
So what's going on in there? Well, we're adding a svn path to a (possibly pre-existing) virtual host - in this case, our devbox.example.com server. We then add a special rule for the URI http://devbox.example.com/svn/uberproject to use the SVN DAV adapter to link to the repository at /opt/subversion/uberproject/repository
The rest of it (from AuthType downwards) sets up access control. Anyone who wants access will have to be listed in the /opt/subversion/uberproject/repository file.
If you want public read-only access, remove the # marks to uncomment the LimitExcept lines.
You can create and edit this file with the htpasswd command:
root@devbox:/etc/apache2# htpasswd -c svn.passwd someuser New password: Re-type new password: Adding password for user someuser
Note that you only need the -c switch the first time you run the command; once the file exists you can drop it.
Note that for your respository to be usable, the repository files themselves will need to be read-writable by the user than apache runs as. On Ubuntu and Debian, that's user and group www-data. See the last installment for how that worked - if you move all your users to http access, they won't need direct write access.
You can now use the URI http://devbox.example.com/svn/uberproject to refer to your repository where we used the file:// URI before.
I'm not going to cover svnserve access here; take a look at the chapter on "svnserve, a custom server" in the SVN book if you're interested.
In fact the SVN Book (aka the Red Bean book) is an excellent reference resource for SVN, so keep it bookmarked.
I'm not going to go through the full functionality here, as the SVN book will cover it all and even the built in help is very good. To use that, just type svn help at the command line. However, here are the most common commands:
This is obviously a very short list, and there are many more commands. Use the svn help or red bean for more info; we will probably touch on some (much) later in this series.
Tags: soyouwant
In the last article, we set up development areas for our developers. Now we need to put some code in there and set up a system to share that code between our users.
Let's presume we've got a project structure that looks like this:
uberproject /frontend /application /libraries /config /tests
What this actually means doesn't really matter for now, but it's useful to have a reference to what we're working with.
Let's also assume that we've got this code structure built in one developer's working area, as ~bob/projects/uberproject. We want any developer to be able to take an up-to-date copy of this code, work with it, and share their changes. To avoid accidentally missing changes, deleting other people's work and thereby throwing team morale through the floor, we'll use version control software; in this case, subversion.
First, we need to install the subversion client and server on our dev box, if it's not already there. In ubuntu, that means (as root) typing apt-get install subversion subversion-tools.
Now we need to create a repository which will act as the central storage location for all of our code. On a shared server, my preference is to put these in /opt/subversion/projectName. So, still working as root:
root@devbox# mkdir -p /opt/subversion/uberproject/repository root@devbox# svnadmin create /opt/subversion/uberproject/repository
That's it - you've got a repository. Easy, wasn't it?
We need to make sure our developers can all write to this:
root@devbox# addgroup svnusers root@devbox# chgrp -R svnusers /opt/subversion/uberproject/repository root@devbox# chmod -R g+wx /opt/subversion/uberproject/repository root@devbox# adduser bob svnusers
OK, now we actually need to put the code in it. Bob's working area contains the latest version, so let's use that as our source. We'll work as bob now, not root. He may need to log back in now he's got his new group privs.
bob@devbox:~$ cd ~/projects/uberproject bob@devbox:~/projects/uberproject$ svn import . file:///opt/subversion/uberproject/repository -m \
'Initial Import' Adding frontend Adding tests Adding application Adding config Adding libraries Committed revision 1.
OK, what happened there? Well, we told subversion to import all the code in '.' (the current directory) to our repository, which we identified via the URI of file:///opt/subversion/uberproject/repository. It took everything in that directory (which isn't much yet) and stuck it in the repository.
You're probably wondering about that file:// URI. Why couldn't we just put the repository path? And if we can have file://, what else can we have?
The first answer is just "That's how it works". SVN commands take URIs as their repository arguments, if they need them.
The second answer is: You have a choice of file://, svnserve://, and http:// (or https). We're using file:// for now purely for illustrative purposes, as it's the easiest to set up. It has limitations though, so we'll do something smarter later.
For now, however, we've got code in our repository, but none of our sandboxes (including the area we just checked the code in from) is under version control. We need to replace that sandbox with an svn-controlled version, and check code out from the repository to any other sandbox.
First, move the original work area out of the way:
bob@devbox: ~/projects$ mv uberproject uberproject-pre-svn
and now check out the repository code:
bob@devbox:~/projects$ svn co file:///opt/subversion/uberproject/repository/ uberproject A frontend A tests A application A config A libraries Checked out revision 1.
ie, svn checkout code-at-this-repository-uri to this-target-directory. The precise syntax of that command frequently fools me, and I check out the code a level too high or low. The thing to remember is that, when an SVN repository path ends with a slash, it means "contents of this directory", without the slash it's the directory itself.
If the required target directory already exists, you can check out the code inside it as follows:
bob@devbox:~/projects/uberproject$ svn co file:///opt/subversion/uberproject/repository/ .
Repeat as required for any other developers' sandboxes, and set up web servers as in the last article.
In the next article, we'll learn a little more about subversion, including how to set up remote access to the repository.
Tags: soyouwant
The plan is to make this into the first article in a series on software team management. Since I'm remarkably bad at keeping to plans as regards blogging, please send your feedback and comments to assure me that you're reading and/or interested.
There are numerous things you need to get right to get a software team running smoothly. The first, even before a line of code is written, is to ensure that multiple developers can actually work on the same project without tripping over each other.
This boils down to a need for:
Or in other words, sandboxes, virtual servers, and version control.
Now, there are dozens of ways of laying out sandboxes, a range of web servers, and quite a few version control systems.
I'm not going to try and cover all these combinations. In this series, I'm going to try and provide one working solution to each of the project team's requirements.
The solution I'm going to cover here uses Linux, Apache, Samba, and Subversion.
You will also need some dedicated hardware which, given the flexibility of Linux, can just be a spare desktop machine, wiped and with Linux installed. Most distributions give you a "Server", "Web Server" or "Internet Server" option at install time which generally installs all these requirements. Personally, I tend to use Ubuntu as I'm most familiar with it, and it makes most of the installation procedures I encounter pretty easy.
I'm not going to tell you (right now, at least) how to install the LAMP stack or recompile PHP, as those are fairly well-documented and generally procedures. (OK, PHP recompilation can be non-trivial, but you can usually just install a package. While I will include some config fragments and instructions, you'll usually need some familiarity with the tools, or at least access to their help files.
What I will do is suggest how to set up your sandboxes, virtual hosts and development tools.
In this model, each of your developers will keep a copy of the project code in a common location in their home directories. They can then map, or mount, their home directories to their desktop (Windows, Mac or Linux) machines to use their favourite editors™ in the OS of their choice.
I suggest you lay down the location in which the project lives; keeping it the same for all developers makes debugging a lot easier. I suggest a path of ~/projects/projectName (in unix systems, '~' is shorthand for "a user's home directory".
To get started, and for testing purposes, I also suggest you create the folder ~/public_html, and the file ~/public_html/index.html which can just contain the text "Hello World, this is username's directory."
First, you need to mount the drives to your developers' desktops. On the server side, this means Samba needs to be running, and the [homes] stanza in /etc/samba.conf is enabled (ie, not commented out). Let's assume that your server is devbox.example.com, known on the network as devbox (you'll also need to set this in samba.conf as "netbios name", I believe).
To mount a Samba share in Window, go to the Explorer (the file explorer, not IE), then tools => Map Network Drive. Choose a drive letter (eg, H: for Home), find your network on the server, and select its "homes" share. Assuming your linux usernames and passwords aren't the same as on your windows network, you'll also need to click the "Connect as a different user" link and fill out the details. (It's possible to co-ordinate accounts and passwords, but it's far too complex to cover here). The user's home directory should now be their H: drive.
To mount the share as a volume in OS X, go to the Finder, then Go => Connect To Server..., and type in smb://devbox/homes/ . Hit the [+] button before connecting to store that link as a favourite. Then hit connect, and supply username and password. The remote share will now be available (usually) as /Volumes/Homes/
To connect in Linux, find someone who uses Linux as a desktop machine, 'cos I haven't done so in years.
Now, find (or create) the public_html folder, and see if you can open index.html in a text editor. If you can, and can save it, you've got a dev area. It's not a sandbox yet though...
For the next step, you'll need apache running with mod_userdir enabled. If you're at the linux shell as root, and you're on an ubuntu or debian server, take a look in /etc/apache2/mods-available . userdir.conf and userdir.load should be in there. Assuming it's not already running, you can enable the module by typing "a2enmod userdir" then "apache2ctl graceful" (translated into english: apache2-enable-module "userdir", then apache2-control-gracefully-reload-configuration).
You should now be able to visit the server in your web browser at (eg) http://devbox.example.com/~username and see the index.html you were just editing.
I'm also assuming, for the purpose of this setup, that you have control over DNS records for your domain or subdomain. Why? Because being able to assign a wildcard DNS record to your dev server makes setting up new virtual hosts and sandboxes much easier. For example, if you assign *.devbox.example.com to point to your dev box's IP address, then username.project.devbox.example.com will point to that box for all variations on username and project. You can then use these dynamic hostnames in your apache configuration for each user's virtual development server.
Once you've got this set up, you'll need to create the apache virtual hosts that map bob.uberproject.devbox.example.com (yes, the hostnames get big) to /home/bob/projects/uberproject/wherever/the/webroot/is. To do this in ubuntu, you create a new config file in /etc/apache2/sites-available/ that contains the following:
<VirtualHost *> ServerName bob.uberproject.devbox.example.com DocumentRoot /home/bob/projects/uberproject/wherever/the/webroot/is </VirtualHost *>
Usually, there's a few more lines in there, but those are the critical ones to illustrate what we're doing.
If that file is /etc/apache2/sites-available/bob.uberproject.conf, you'll need to enable it: as root, do a2ensite bob.uberproject.conf, then apache2ctl graceful. If all's worked out, you can then visit http://bob.uberproject.devbox.example.com/ in your browser.
What you see there will, of course, depend on the contents of your project / document root. "Hello world" is always a good start.
Having got our developer's personal web server running, we'll pause there for now. I'll return fairly soon for the second part of this article, in which I'll cover setting up a subversion repository and using it to synchronise changes between users.
Tags: soyouwant