Getting started

This chapter will lead you through an example use of darcs, which hopefully will allow you to get started using darcs with your project.

Creating your repository

Creating your repository in the first place just involves telling darcs to create the special directory (called _darcs) in your project tree, which will hold the revision information. This is done by simply calling from the root directory of your project:

% cd my_project/
% darcs initialize

This creates the _darcs directory and populates it with whatever files and directories are needed to describe an empty project. You now need to tell darcs what files and directories in your project should be under revision control. You do this using the command darcs add^3.1:

% darcs add *.c Makefile.am configure.ac

When you have added all your files (or at least, think you have), you will want to record your changes. ``Recording'' always includes adding a note as to why the change was made, or what it does. In this case, we'll just note that this is the initial version.

% darcs record --all
What is the patch name? Initial revision.

Note that since we didn't specify a patch name on the command line we were prompted for one. If the environment variable `EMAIL' isn't set, you will also be prompted for your email address. Each patch that is recorded is given a unique identifier consisting of the patch name, its creator's email address, and the date when it was created.

Making changes

Now that we have created our repository, make a change to one or more of your files. After making the modification run:

% darcs whatsnew

This should show you the modifications that you just made, in the darcs patch format. If you prefer to see your changes in a different format, read Section , which describes the whatsnew command in detail.

Let's say you have now made a change to your project. The next thing to do is to record a patch. Recording a patch consists of grouping together a set of related changes, and giving them a name. It also tags the patch with the date it was recorded and your email address.

To record a patch simply type:

% darcs record

darcs will then prompt you with all the changes that you have made that have not yet been recorded, asking you which ones you want to include in the new patch. Finally, darcs will ask you for a name for the patch.

You can now rerun whatsnew, and see that indeed the changes you have recorded are no longer marked as new.

Making your repository visible to others

How do you let the world know about these wonderful changes? Obviously, they must be able to see your repository. Currently the easiest way to do this is typically by http using any web server. The recommended way to do this (using apache in a UNIX environment) is to create a directory called /var/www/repos, and then put a symlink to your repository there:

% cd /var/www/repos
% ln -s /home/username/myproject .

As long as you're running a web server and making your repository available to the world, you may as well make it easy for people to see what changes you've made. You can do this by running make installserver, which installs the program darcs_cgi at /usr/lib/cgi-bin/darcs. You also will need to create a cache directory named /var/cache/darcs_cgi, and make sure the owner of that directory is the same user that your web server runs its cgi scripts as. For me, this is www-data. Now your friends and enemies should be able to easily browse your repositories by pointing their web browsers at http://your.server.org/cgi-bin/darcs.

Getting changes made to another repository

Ok, so I can now browse your repository using my web browser... so what? How do I get your changes into my repository, where they can do some good? It couldn't be easier. I just cd into my repository, and there type:

% darcs pull http://your.server.org/repos/yourproject

Darcs will check to see if you have recorded any changes that aren't in my current repository. If so, it'll prompt me for each one, to see which ones I want to add to my repository. Note that you may see a different series of prompts depending on your answers, since sometimes one patch depends on another, so if you answer yes to the first one, you won't be prompted for the second if the first depends on it.

Of course, maybe I don't even have a copy of your repository. In that case I'd want to do a

% darcs get http://your.server.org/repos/yourproject

which gets the whole repository.

I could instead create an empty repository and fetch all of your patches with pull. Get is just a more efficient way to clone a whole repository.

Get, pull and push also work over ssh. Ssh-paths are of the same form accepted by scp, namely [username@]host:/path/to/repository.

Moving patches from one repository to another

Darcs is flexible as to how you move patches from one repository to another. This section will introduce all the ways you can get patches from one place to another, starting with the simplest and moving to the most complicated.

All pulls

The simplest method is the ``all-pull'' method. This involves making each repository readable (by http, ftp, nfs-mounted disk, whatever), and you run darcs pull in the repository you want to move the patch to. This is nice, as it doesn't require you to give write access to anyone else, and is reasonably simple.

Send and apply manually

Sometimes you have a machine on which it is not convenient to set up a web server, perhaps because it's behind a firewall or perhaps for security reasons, or because it is often turned off. In this case you can use darcs send from that computer to generate a patch bundle destined for another repository. You can either let darcs email the patch for you, or save it as a file and transfer it by hand. Then in the destination repository you (or the owner of that repository) run darcs apply to apply the patches contained in the bundle. This is also quite a simple method since, like the all-pull method, it doesn't require that you give anyone write access to your repository. But it's less convenient, since you have to keep track of the patch bundle (in the email, or whatever).

If you use the send and apply method with email, you'll probably want to create a _darcs/prefs/email file containing your email address. This way anyone who sends to your repository will automatically send the patch bundle to your email address.

If you receive many patches by email, you probably will benefit by running darcs apply directly from your mail program. I have in my .muttrc the following

macro pager A "<pipe-entry>darcs apply --verbose --mark-conflicts \
        --reply droundy@abridgegame.org --repodir ~/darcs"

which allows me to apply patches directly from mutt, sending a confirmation email to the person who sent me the patch.

Push

If you use ssh (and preferably also ssh-agent, so you won't have to keep retyping your password), you can use the push method to transfer changes (using the scp protocol for communication). This method is again not very complicated, since you presumably already have the ssh permissions set up. Push can also be used when the target repository is local, in which case ssh isn't needed. On the other hand, in this situation you could as easily run a pull, so there isn't much benefit.

Note that you can use push to administer a multiple-user repository. You just need to create a user for the repository (or repositories), and give everyone with write access ssh access, perhaps using .ssh/authorized_keys. Then they run

% darcs push repouser@repo.server:repo/directory

Push --apply-as

Now we get more subtle. If you like the idea in the previous paragraph about creating a repository user to own a repository which is writable by a number of users, you have one other option.

Push --apply-as can run on either a local repository or one accessed with ssh, but uses sudo to run a darcs apply command (having created a patch bundle as in send) as another user. You can add the following line in your sudoers file to allow the users to apply their patches to a centralized repository:

ALL   ALL = (repo-user) NOPASSWD: /usr/bin/darcs apply --all --repodir /repo/path*

This method is ideal for a centralized repository when all the users have accounts on the same computer, if you don't want your users to be able to run arbitrary commands as repo-user.

Sending signed patches by email

Most of the previous methods are a bit clumsy if you don't want to give each person with write access to a repository an account on your server. Darcs send can be configured to send a cryptographically signed patch by email. You can then set up your mail system to have darcs verify that patches were signed by an authorized user and apply them when a patch is received by email. The results of the apply can be returned to the user by email. Unsigned patches (or patches signed by unauthorized users) will be forwarded to the repository owner (or whoever you configure them to be forwarded to...).

This method is especially nice when combined with the --test option of darcs apply, since it allows you to run the test suite (assuming you have one) and reject patches that fail--and it's all done on the server, so you can happily go on working on your development machine without slowdown while the server runs the tests.

Setting up darcs to run automatically in response to email is by far the most complicated way to get patches from one repository to another... so it'll take a few sections to explain how to go about it.

Security considerations

When you set up darcs to run apply on signed patches, you should assume that a user with write access can write to any file or directory that is writable by the user under which the apply process runs. Unless you specify the --no-test flag to darcs apply (and this is not the default), you are also allowing anyone with write access to that repository to run arbitrary code on your machine (since they can run a test suite--which they can modify however they like). This is quite a potential security hole.

For these reasons, if you don't implicitly trust your users, it is recommended that you create a user for each repository to limit the damage an attacker can do with access to your repository. When considering who to trust, keep in mind that a security breach on any developer's machine could give an attacker access to their private key and passphrase, and thus to your repository.

Installing necessary programs

You also must install the following programs: gnupg, a mailer configured to receive mail (e.g. exim, sendmail or postfix), and a web server (usually apache). If you want to be able to browse your repository on the web you must also configure your web server to run cgi scripts and make sure the darcs cgi script was properly installed (by either a darcs-server package, or `make install-server').

Granting access to a repository

You create your gpg key by running (as your normal user):

% gpg --gen-key

You will be prompted for your name and email address, among other options. Of course, you can skip this step if you already have a gpg key you wish to use.

You now need to export the public key so we can tell the patcher about it. You can do this with the following command (again as your normal user):

% gpg --export "email@address" > /tmp/exported_key

And now we can add your key to the allowed_keys:

(as root)> gpg --keyring /var/lib/darcs/repos/myproject/allowed_keys \
               --no-default-keyring --import /tmp/exported_key

You can repeat this process any number of times to authorize multiple users to send patches to the repository.

You should now be able to send a patch to the repository by running as your normal user, in a working copy of the repository:

% darcs send --sign http://your.computer/repos/myproject

You may want to add ``send sign'' to the file _darcs/prefs/defaults so that you won't need to type --sign every time you want to send...

If your gpg key is protected by a passphrase, then executing send with the --sign option might give you the following error:

darcs failed:  Error running external program 'gpg'

The most likely cause of this error is that you have a misconfigured gpg that tries to automatically use a non-existent gpg-agent program. GnuPG will still work without gpg-agent when you try to sign or encrypt your data with a passphrase protected key. However, it will exit with an error code 2 (ENOENT) causing darcs to fail. To fix this, you will need to edit your ~/.gnupg/gpg.conf file and comment out or remove the line that says:

use-agent

If after commenting out or removing the use-agent line in your gpg configuration file you still get the same error, then you probably have a modified GnuPG with use-agent as a hard-coded option. In that case, you should change use-agent to no-use-agent to disable it explicitly.

Setting up a sendable repository using procmail

If you don't have root access on your machine, or perhaps simply don't want to bother creating a separate user, you can set up a darcs repository using procmail to filter your mail. I will assume that you already use procmail to filter your email. If not, you will need to read up on it, or perhaps should use a different method for routing the email to darcs.

To begin with, you must configure your repository so that a darcs send to your repository will know where to send the email. Do this by creating a file in /path/to/your/repo/_darcs/prefs called email containing your email address. As a trick (to be explained below), we will create the email address with ``darcs repo'' as your name, in an email address of the form ``David Roundy $<$droundy@abridgegame.org$>$.''

% echo 'my darcs repo <user@host.com>' \
      > /path/to/your/repo/_darcs/prefs/email

The next step is to set up a gnupg keyring containing the public keys of people authorized to send to your repository. Here I'll give a second way of going about this (see above for the first). This time I'll assume you want to give me write access to your repository. You can do this by:

gpg --no-default-keyring \
    --keyring /path/to/the/allowed_keys --recv-keys D3D5BCEC

This works because ``D3D5BCEC'' is the ID of my gpg key, and I have uploaded my key to the gpg keyservers. Actually, this also requires that you have configured gpg to access a valid keyserver. You can, of course, repeat this command for all keys you want to allow access to.

Finally, we add a few lines to your .procmailrc:

:0
* ^TOmy darcs repo
|(umask 022; darcs apply --reply user@host.com \
    --repodir /path/to/your/repo --verify /path/to/the/allowed_keys)

The purpose for the ``my darcs repo'' trick is partially to make it easier to recognize patches sent to the repository, but is even more crucial to avoid nasty bounce loops by making the --reply option have an email address that won't go back to the repository. This means that unsigned patches that are sent to your repository will be forwarded to your ordinary email.

Like most mail-processing programs, Procmail by default sets a tight umask. However, this will prevent the repository from remaining world-readable; thus, the ``umask 022'' is required to relax the umask. (Alternatively, you could set Procmail's global UMASK variable to a more suitable value.)

Checking if your e-mail patch was applied

After sending a patch with darcs send, you may not receive any feedback, even if the patch is applied. You can confirm whether or not your patch was applied to the remote repository by pointing darcs changes at a remote repository:

darcs changes --last=10 --repo=http://abridgegame.org/repos/darcs

That shows you the last 10 changes in the remote repository. You can adjust the options given to changes if a more advanced query is needed.

Reducing disk space usage

A Darcs repository contains the patches that Darcs uses to store history, the working directory, and a pristine tree (a copy of the working directory files with no local modifications). For large repositories, this can add up to a fair amount of disk usage.

There are two techniques that can be used to reduce the amount of space used by Darcs repositories: linking and using no pristine tree. The former can be used on any repository; the latter is only suitable in special circumstances, as it makes some operations much slower.

Linking between repositories

A number of filesystems support linking files, sharing a single file data between different directories. Under some circumstances, when repositories are very similar (typically because they represent different branches of the same piece of software), Darcs will use linking to avoid storing the same file multiple times.

Whenever you invoke darcs get to copy a repository from a local filesystem onto the same filesystem, Darcs will link patches whenever possible.

In order to save time, darcs get does not link pristine trees even when individual files are identical. Additionally, as you pull patches into trees, patches will become unlinked. This will result in a lot of wasted space if two repositories have been living for a long time but are similar. In such a case, you should relink files between the two repositories.

Relinking is an asymmetric operation: you relink one repository (to which you must have write access) to another repository, called the sibling. This is done with darcs optimize --relink, with -the --sibling flag specifying the sibling.

  $ cd /var/repos/darcs-unstable
  $ darcs optimize --relink --sibling /var/repos/darcs

The --sibling flag can be repeated multiple times, in which case Darcs will try to find a file to link to in all of the siblings. If a default repository is defined, Darcs will try, as a last resort, to link against the default repository.

Additional space savings can be achieved by relinking files in the pristine tree (see below) by using the --relink-pristine flag. However, doing this prevents Darcs from having precise timestamps on the pristine files, which carries a moderate performance penalty.

Alternate formats for the pristine tree

By default, every Darcs repository contains a complete copy of the pristine tree, the working tree as it would be if there were no local edits. By avoiding the need to consult a possibly large number of patches just to find out if a file is modified, the pristine tree makes a lot of operations much faster than they would otherwise be.

Under some circumstances, keeping a whole pristine tree is not desirable. This is the case when preparing a repository to back up, when publishing a repository on a public web server with limited space, or when storing a repository on floppies or small USB keys. In such cases, it is possible to use a repository with no pristine tree.

Darcs automatically recognizes a repository with no pristine tree. In order to create such a tree, specify the --no-pristine-tree flag to darcs initialize or darcs get. There is currently no way to switch an existing repository to use no pristine tree.

The support for --no-pristine-tree repositories is fairly new, and has not been extensively optimized yet. Please let us know if you use this functionality, and which operations you find are too slow.