This book was envisioned as a collaborative project. I wanted to hear how other people have collaborated on projects, find out what works and what doesn’t.

In developing this book I have to thank several people, First of all Simon St Lawrence and Andy Oram at O’Reilly Media who have supported me.

Matthew McCullough of the github training group has offered me a number of suggestions as well as promoted this book within github.

In addition a number of people have sent me pull requests with changes. Contributors are listed here in alphabetical order by github user name.

The quote in French means "Good accounts make for good friends" and is fitting. As business venture needs a good system to track its money, and a project needs a good system to keep track of the work done by its members.

A project can be any form of creative work, a computer program, a Novel, this book, a set of knitting patterns and so forth. Most projects will consist of one or more files that will reside on the computers of the project members.

As an example take this book. It consists of around 20 or so files that contain the actual text of the book, one for each chapter or section, and then a bunch of other files that contain things like the images that are included and a few random others.

As the book is being written we want to be able to share those documents between the writers, reviewers, editors, and in this case the world at large. Other projects will be shared in different ways.

In order to make this collaboration work we need two things. First a way to share the documents between the project members, and secondly a set of rules on how those contributors will work together.

It is all fine and good to say that it is possible to know if two users have the same version of a file, or how their copies differ, but if you have 20 people attempting to assemble a complex project you need a way to organize people so that tasks can be managed.

This book will attempt to lay out how to construct a system that will work for your team, by showing how different teams have faced these problems and mapping out their solutions.

Thankfully various computer programmers over the years have faced this problem and realizing that it was a problem, built tools to try and solve it. These tools collectively are known as "Version Control Systems" Or "Revision Control Systems". The history of Version Control systems dates back to the 1970’s and has generated significant progress.

The current major Version Control system is called "git" and was created by Linus Torvolds, when he was unable to find that existing systems did not meet the needs of the team that was developing the Linux kernel.

GIT is a very powerful tool that has a lot of features, but most of what is needed to work and collaborate with others can be done with a small enough to learn pretty quickly. In addition while many git tutorials show how to use git from a command line there are now many very nice graphical tools to enable the use of git without having to ever see a command line if you don’t want too.

The way version control systems work is they track files and changes to those files. So it is possible to see the history of changes to a file and when each line was last changed and by whom.

This allows project members to say that they all the same version of a file, and to ensure that everyone’s changes have been accounted for. Git allows users to ensure that they have the same version of the project files by applying a mathematical function called SHA1 to the files in a project. At any given time the files committed to a project will have a unique SHA1 key.

Furthermore, if two members of project team have a project directory with the same SHA1 key they can be sure that the files in the project are the same.

Every time files are changed in a project git will create a new SHA1 key for the project. So if Bob and Lisa want to know if they have the same files they do not have to check every file for changes they can just compare their keys. If they match they know that the files are the same.

However if they are not the same git can look at the history of the files and figure out what has changed. So if Bob has sent Lisa file for editing and she made some changes, git will use the SHA1 keys to compare the history of the files. In this case it will see that Bob’s files are an ancestor to Lisa’s. It can then know exactly what changes Lisa made and allow Bob to safely merge her changes. We will see an example of how to do this in Pull Request.

The use of git is greatly enhanced by Github a company that was founded in 2008 to facilitate using git to collaborate on software projects. Since the Github has grown to become the most popular site for building collaborations in the open source software community.

Github is itself a distributed company with its team spread out all over the place so they not only develop the tools for working with distributed teams, they use them ever day.

To create a repository use the command git init which will initialize an empty git repository. Once a repository has been created files can be added to it and all other operations performed.

The other way to create a repository is by doing it with Github or other remote hosting service. Once you create the repository on Github you then clone the repository onto your local computer.

The first thing you will wish to do after creating a new repository is add some files for it to track. A repository without any files is not particularly useful, anymore than any other file system without files.

To add files use the command git add <file> you can also add more than one file or a directory structure in one command line.

What makes git so useful is that it has such wonderful features for working with remote repositories. There are for major commands for working with a remote repository. In this case a remote repository can be a different directory on the same computer or on a different computer somewhere else, or even on a thumb drive.

In order to move a set of changes from one repository to another there are two options. You can push or pull the changes. A push takes the changes in the current repository and merges them into a foreign repository, which can be on a remote computer, or just a second directory on the same computer.

The other option is to do a pull which takes changes from another repository and moves them into yours.

In both cases these operations preserve the complete history of commits from the source repository.

In addition these operations can be be repeated. So Ann can pull changes from Bob who has pulled changes from Charlie, the final repository will have all three sets of changes. This is very different from sending a patch file where there is no change history.

Of course if Ann is going to pull changes from Bob she needs a way to know that she should pull his changes. If Ann and Bob are sitting next to each other Bob may just say to Ann "Ok pull my changes" and let her know the URL of the repository and which branch the changes are on. You can push or pull from any local branch to any remote branch.

However if you are more than three or four people and are not sitting all in one place you will probably want a more formal way to do this. To support this Github has created a formal idea of a "Pull Request" which is a formal way for a user to let another user know that there are changes to be integrated.

If Ann has changes that she wishes Bob to take into his repository via Github (or bitbucket) she will go to the web site for her repository on Github and then select which branch contains her changes and of course who she wishes to send the change request too.

Bob will then be notified by Github that he has a pull request pending. Github notifications can be configured to run over email, threw the web site or via the API or a mobile app.

Bob will then have a chance to review the changes that Ann made and integrate them into his three or reject them.

All of this can also be done via the Github API, but that is somewhat beyond the scope of this chapter.

To illustrate let us use an example from the writing of this book. I have published this book on github and user breskeby noticed a typo in one of these files. He had forked the repository and committed his change. You can see this in this graph. His change is the lower line in blue.

After committing his changes breskeby sent me a pull request with his change. In this case you see that the change he has sent to line 3. My text is in red. While his corrected text is bellow it in green.

In this case I chose to merge the changes into the text and now the network graph looks like this. Notice that in this graph the commit from breskeby (in blue) is now in the zkessin band of the graph as those commits are now also in my repository.

A while latter the graph looks like this graph. Notice here that there are further commits from me. In addition there is a new commit in green from user kirean which has not yet been merged into my repository.

If I look at the graph from kirean’s repository it will look like this. There are a few things to note here. First of all his changes are in the top band in black, while mine are bellow in blue. Secondly the trunk up until where the two repositories diverge is shown in his repository. Even thought I made those commits since they have been copied into his copy of the repository they are shown there.

If we want to share that repository we can run the command hg serve which will open a web server on port 8000 which we can use the interact with the hg repository.

The hg serve command will produce a web page that looks a bit like the following image, with a history of the repository.

The nice this about this web page is that for free you get a number of nice features, that can help you manage a repository.

If you want to see a graphical version of the network of commits you can use the graph option which will give you a screen that looks like this image. Here you can see the results of a repository where there was a branch and a merge. With the side branch in green.

You can also use the server to clone a repository by simply typing hg clone URL which will pull the changes.

In terms of how to add files, like in git you use hg add <files> followed by hg commit and like in git the -m option will let you set the commit message.

If you made some changes that you don’t want to keep you can use the hg revert <files> to roll one or more files back to the last commit. When you do this hg will save any files you rolled back with the ending .orig so that they are not lost.

The 10,000 pound gorilla of the DVCS hosting world is github. This is for a good reason, github has been leading the effort to create new and innovative features for collaborative development. In addition github has become the social network for programmers, and the central place for open source development, having taken that crown from Soruceforge several years ago.

Github has become a leader in the field for a number of reasons. But one of them is that they practice what they preach. Github is very much a distributed company with people all over the world. So before they release tools to the world, they have used them internally. In addition their internal use of these tools is not just the standard run threw QA testing but actual use when things have to get done.

Github has an amazing set of features, and between when I write this and when this book is published they will probably introduce a bunch more. So this section will not be able to cover all of the features of github. However there are some highlights to know about.

First of all if you are viewing a file in Markdown or Asciidoc it will render the file for you. You can see raw text by clicking the Raw button.

It will also render most programming languages in a way that programmers will be used to seeing them in their editors. So keywords will be highlighted and so on.

If you want to make a simple edit to a file in a github repository and do not wish to clone it to your local computer, edit the file and push the result back up. You can use the Edit button make a change and then commit it to the repository directly from the web page. This is not the best way to write code, it is very useful if you want to do something simple like fix a typo in a comment.

If you want to know where each line of text comes from click the Blame button which will show the file with the commit information. So that for each chunk of text you can see who committed it most recently and when.

The History button will allow you to see a chronological view of the commits to a file.

In some ways Bitbucket is a clone of Github, however it has a few options that github lacks. First of all it support HG in addition to GIT so if your team wishes to use HG it is a strong possibility. In addition while github offers free public repositories it does require you to pay for private ones. With Bitbucket you can have as many private repos as you want, however you can only share them with up to five other people before you have to pay for hosting. In addition Bitbucket allows unlimited repos with all plans but limits how many people you can share with. If you don’t need community or to share with more than a few people Bitbucket can be a good option. For the case of a lone developer who just wants off site backups this can be a good option.

The Linux kernel was the project for which GIT was created, so it is a good place to start talking out how teams use git to get work done. (Pretty much everything here would also work with HG). It is important to understand that GIT was built for large teams, a linux kernel release may contain thousands of commits from hundreds of contributers. GIT was built to handle that kind of usage. Thankfully it scales down quite well for small projects.

At the center of the Linux Universe in Linus Torvalds. He maintains a central authoritative copy of the kernel source tree. Around him are a number of lieutenants each of whom maintain a part of component. This central group of people is probably pretty stable in terms of membership. However they are a distributed team. They do not all work for the same company, nor do they sit in the same place. As such they needed a tool that would let them work in this way.

The way the system works is that if a developer wishes to submit a patch he will send a pull request to the maintainer of whichever component is involved. That maintainer will review those changes and if he or she approves them forward them up the tree until they eventually end up with Linus who will release them to the world.

Now the central team is probably very stable. Linus has been in charge of the Linux kernel for over 20 years, and most of the senior lieutenants have probably been doing it for years. However the folks on the edges may come and go. It is quite possible to think that a company would write a device driver for some piece of hardware submit it to the correct person and then only change it once in a while.

If you have a large team some variation on this work flow may make sense. In this case what would have to happen is that the project must be able to decompose into smaller components each of which would be the responsibility of a component leader. Those component leaders will have to forward their changes up to the global leader.

If you are developing a project solo many of the complexities of a group are greatly simplified, As there is only one team member she does not need to coordinate with team members. fThere are two aspects of how to work on a project solo. How to best take advantage of the features of git such as branches and how to take best advantage of cloud hosting.

I have found that while working solo or in a group using frequent small branches makes life simple. For each new feature I create a new branch and do the development I want to do. When I have fished the feature I merge it back into the main branch.

I tend to use Bitbucket for solo projects. Unlike github Bitbucket allows you to have have unlimited private repositories. As I don’t care about community in this case giving up github’s features doesn’t bother me as I am working solo.

I do however want to use a cloud host as I may be using the project from several computes. I may wish to have access to the files from my desktop, my laptop and a cloud server for deployment.

One reason to prefer github is that many online build tools can take advantage of the github API. SO example if I am building an application for mobile phones with PhoneGap then I may wish to use their API to build my application. {TODO link PhoneGap} this will enable me to take advantage of those build tools. In that case I could also push my changes onto the PhoneGap git archive.

If you are a free lance software consultant (which I am) and you need to work with clients a lot on projects then you will need to vary the previous pattern for working with your clients.

In this case the team is well defined, you the consultant and the client. The scope is probably limited to one short project, and the leadership structure is pretty simple based on just you and the client.

In this case, generally the developer will probably be committing most of the changes to the code or content and the client will be reviewing them, but possibly also submitting some content from time to time, which could involve setting configuration options or uploading artwork or the like. (Clearly this will change based on the exact nature of the consultant client relationship).

In this case the easiest way to structure the project is probably to open a repository at Github or Bitbucket and share it with the client so that you both have read / write access, it would be possible to fork a repo for the client and send pull requests back and forth but it is probably more trouble than it is worth.

I find that when working in this small a group many of the notification features that are present in github or Bitbucket can be dealt with by simply picking up my phone and calling my client, or by using some form of IM. Of course that does not mean that project notifications are not useful, so much as that using them may not be required in such a small team.

At the opposite end of the spectrum from the Linux Kernel team is the Internet cafe or hackathon session. In this case we can assume 2-4 people are going to be sitting down over a coffee or pizza to develop something. This would also apply to an early stage startup where people are writing code on laptops around a table, or a group of students working on their homework. In this case we should assume that the project will only last a few hours or a few days, so the work flow should be informal.

In this case the project probably does not have a formal leadership, and planning is probably done on a whiteboard. In addition a "Pull Request" is probably done by turning to your friend and saying "Hey grab my code" not via a formal Github Pull request. (though you can do that too if you wish).

On the plus side you have the advantage that everyone involved is probably in the same room and may well have known each other before the project started. This can make the interaction between team members simpler.

In this case a deep system like the Linux Kernel team uses makes no sense what so ever. What we want here is a limited number of peers each of whom can pull from the others. This can be done with the hg server option if you are using HG, or by creating a project on Github and having everyone fork it. When it comes time to integrate code it will be as simple as doing a pull from the correct repository and branch.

To adapt this work flow is as easy as it could be. Create a repository where everyone can access it then have everyone clone it. Each member of the team will commit to his or her own repository. When features need to be integrated changes are pushed to a repository then changes can be shared between developers. In this case there is not a central authoritative version of the source, as in some of the other work flows, instead it will be up to members of the team to integrate reasonably often and not let things get too disorganized.

This work flow also takes little or no infrastructure. Each user needs only his own computer and a way to share repositories. This can be done in several ways. First of all if you are using HG you can use the built in http server with the hg --serve command. If you are using git there is not quite as easy an option, however you can use a cloud hosting service like github or even share the repositories with a file server or dropbox.

What would probably work is to setup a repository on github and give everyone read/write access to it and push all changes there.

What will probably be more work is knowing when you have outgrown this workflow. At some point three guys sitting around a table with Pizza can turn into a real company which may require other types of work flow.

A public project, including many open source programs, is one where you are trying to leverage the community at large to build something. This may be a book (like this one), or a open source software project or even a families collection of recipes.

In this case Github is a clear win just by its large community and its tools to create and support community.

In terms of how to work it is pretty simple, the project leader will publish a repository on github and then announce it (normally via twitter, blogs, mailing lists or other social media sites).

At this point anyone else who is interested in the project will then fork the repository and add their own content. This will be sent back to the original leader via a pull request.

If the project gets large then it may evolve into something that resembles the Linux Kernel team work flow. This is in effect a simplified version of that pattern. While the Linux kernel team has several layers of gate keeper between a contributer and the authoritative version of the project, in a smaller project there may only be one or two core people who accept all the pull requests and integrate them.

When you have outgrown the previous work flow, which resembles a fully connected graph then the question is what to replace it with. There is not one correct way to do things at this stage but clearly there are some wrong ways. The Linux Kernel model is clearly overkill for a team of 6-10 developers.

The simplest model is to have one central repository where everyone can push changes to integrate. It could also be linked into a CI system via hooks if that is what is desired. This is a good base but is probably not the full solution.

Unlike many of the other models in this book this one relies on pushing to a central repository. In this case each developer must be given write access to that repository. When he has changes he does a push and they are integrated.

As a variation it would also be possible to have a central integration machine, and he will send that account a pull request. Then move to the integration machine merge the code and integrate the changes. In order for this to work there should not be a backlog of changes and people should integrate them as soon as possible.

One place that this can lead to problems is that if changes are not integrated on a regular basis then integration can become a nightmare, so make sure that code is integrated on a regular basis.

In addition a DVCS also will allow developers to pull changes from each other without going threw the central server. This is very useful for things like doing a code review. If Adam has some code that he has written that he want Bonnie to review he can ask her to pull that code directly from him for review. When she is done it can be pushed into the central server.

As with the Linux Kernel model it is important to have one central point of "truth" for the project, so that everyone knows what is going to be shipped. It would be way to easy for a project to end up with the situation where there are a number of repos each with a slightly different version of the code and have no one realize it until two days before a major deadline. In the previous work flow that risk is mitigated by close communication and the limited scope of the project. A hack-a-thon or a typical homework project will probably only last a few hours to a few days, while a small company might be working on its code base for a period of years.

One way to work with a small group is to have one central repository and then have all new development happen in a series of branches. Each feature is implemented in its own branch and when code review and testing are complete the code is merged into the master branch from which the final build is constructed and shipped.

In order for this work flow to work then your team must have good discipline as anyone can integrate code into the master branch and ship that code.

If you are using github for hosting your product you may wish to do Continous Integration with your github repository. There are a number of ways to do this but one easy hosted solution is to use Travis CI (http://travis-ci.org) which is a hosted CI solution that can run tests with a number of different languages.

Travis-Ci was started by the ruby guys, but has since expanded to allow testing in C, C++, Clojure, Erlang, Go, Groovy, Haskell, Java, JavaScript (Node.js), Perl, PHP, Python and Scala. As Travis is open source if you need a language that is not on that list I am sure they would be happy to get a pull request from you that would make it work.

To setup travis-ci you need to add a yml file to the repository called .travis.yml which should contain the options for how to build your software.

When Travis goes to build your project it will create a new virtual machine and then build your project based on the instructions in the .travis.yml file. At the end of the build cycle it will destroy the virtual machine. Thus any changes to the VM made by one run will never be kept.

An automated build tool…

If you have a comprehensive set of tests it can be possible to build a tool to allow you to very easily zero in on a fault if one gets introduced into the system. If someone has pushed a set of changes into a project then those changes may have affected dozens of files and hundreds of tests. In addition there may have been 50 or more commits in that change set.

If you know that tests start failing then it would be possible to roll back to the last commit which all tests pass then by moving forward one commit at a time unit a test fails. At that point we can see which lines in which files have changed, and what lines of code are covered by the failing tests.

What we want is is to know is which lines are in common between the changes and the coverage of the failing tests. If the commits are small changes and the coverage of each test is tight then in many cases the developer can be directed to an area of 1-2 lines of code where the bug exists.

There are a number of graphical git tools for the mac. You can of course download the command line git via homebrew (and probably should) but if you want to do some things with a graphical client there are a few free options.