2.13 Network Service Revisited

Up until now, Jim has been using his laptop and database as a sort of “central server” for the company; Abe and Beth have been syncing with Jim, and learning of each other’s work via Jim’s database. This has worked fine while the product has been in early development; Jim has good network connectivity in Japan, and has been staying home concentrating on programming. He has been able to leave his laptop connected and running all the time, while his employees in different time-zones work and sync their databases. This is now starting to change, and two problems are starting to cause occasional difficulties.

• First, Jim is finding that he has to spend more of his time travelling, demonstrating the new juicebot features to customers; thus his laptop is spending more time disconnected from the network, or connected at dynamic addresses where it’s not convenient for Abe and Beth to find him and sync.

  This doesn’t prevent them doing any work, but it does have some uncomfortable consequences: they’re more likely to have to manually merge conflicting changes when they finally sync up and discover they’ve both come up with slightly different fixes for the same bug in the meantime, and they’re more exposed to loss of work if one of them suffers a disk failure before they’ve had a chance to sync that work with another database.

• Second, because Jim has been using the one database file both for his own local work, and for serving to the others in the team, he occasionally finds that the monotone serve process (busy syncing with Abe or Beth) has a lock on the database, while he’s trying to do local work like updates or commits.

  The level of project activity is picking up, and there are more and more changes to be synced in the narrower window of time while Jim is connected. He finds he sometimes needs to take down the server process to do this local work, further exacerbating the first problem.

The juicebot team are resourceful, and by now quite used to working independently. While Jim has been away travelling, Abe and Beth have come up with their own solution to the first problem: they’ll run servers from their databases, setting them up just like Jim did previously. That way, if Jim’s database is offline, either Beth or Abe can run the serve command and provide access for the other to sync with. Beth also has the idea to create a second database for the serve process, and to sync her development database with that server locally, avoiding locking contention between multiple monotone processes on the one database file.

When Jim reappears, the next person to sync with him will often pass him information about both employees’ work that they’ve sync’ed with each other in the meantime, just as he used to do. In fact, Jim now finds it more convenient to initiate the sync with one of the other servers when he has a spare moment and dynamic connectivity from a hotel room or airport. Changes will flow between servers automatically as clients access them and trade with one another.

This gets them by for a while, but there are still occasional inconveniences. Abe and Beth live in very different time-zones, and don’t always have reliable network connectivity, so sometimes Jim finds that neither of them is online to sync with when he has the chance. Jim now also has several customers interested in beta-testing the new code, and following changes as the bugs and issues they report are addressed.

Jim decides it’s time for a permanent server they can all sync with; this way, everyone always knows where to go to get the latest changes, and people can push their changes out without first calling their friends and making sure that they have their servers running.

Jim has rented some web server space on a service provider’s shared system for the JuiceBot Inc. public website, www.juicebot.co.jp; he thinks this server will be a good place to host the central monotone server too. He sets up a new monotone database on the server, generates a new key specially for the server (so he doesn’t have to expose his own development private key on the shared system), and loads in the team-members’ keys:

$ mtn --db=server.mtn db init
$ mtn genkey monotone-server@www.juicebot.co.jp
enter passphrase for key ID [monotone-server@www.juicebot.co.jp] (...): <Jim enters a new passphrase>
confirm passphrase for key ID [monotone-server@www.juicebot.co.jp] (...): <Jim confirms the passphrase>
mtn: generating key-pair 'monotone-server@www.juicebot.co.jp'
mtn: storing key-pair 'monotone-server@www.juicebot.co.jp' in /home/jim/.monotone/keys
mtn: key 'abe@juicebot.co.jp' has hash '78be08f7a2a316a9f7c6b0db544ed20673ea2190'
$ cat abe.pubkey beth.pubkey jim.pubkey | mtn --db=server.mtn read
mtn: read 3 packets

For the team members, he sets up the permissions files on the server much like before — except that of course he needs to also grant his jim@juicebot.co.jp key permission to access the new server. For the beta-testers, Jim wants to allow them read-only access just to the main JuiceBot 7 development line, but not to any of the sub-branches where other experimental development is going on. He adds some lines at the top of the ~/.monotone/read-permissions on the server, above the broader permissions given to team-members. See get_netsync_read_permitted for more details; the resulting file looks like this:

comment "Provide beta-testers with specific read-only access"
pattern "jp.co.juicebot.jb7"
allow "beta1@juicebot.co.jp"
allow "beta2@juicebot.co.jp"
continue "true"

comment "Fall-through, and allow staff access to all branches"
pattern "*"
allow "abe@juicebot.co.jp"
allow "beth@juicebot.co.jp"
allow "jim@juicebot.co.jp"

Jim could log in and start the monotone process manually from his shell account on the server, perhaps under a program like screen to let it stay running while he’s away. This would be one way of giving it the server-key’s passphrase each startup, but he wants to make sure that the server is up all the time; if the host reboots while he’s travelling and the monotone server is down until he next logs in, things aren’t much better than before. For the server to start automatically each time, he’ll need to use the get_passphrase hook in the server’s monotonerc file again.

Because he’s running on a shared server, Jim needs to be a little more restrictive about which interfaces and addresses his new server process will listen on. He should only accept connections at the address used for his website, because some of the provider’s other customers might also want to publish their own monotone projects on this host. Jim uses the --bind=address:port argument like so:

$ mtn --db=server.mtn --bind=www.juicebot.co.jp serve

This will start monotone listening on the default port (4691), but only on the IP address associated with www.juicebot.co.jp. Jim can do this because his hosting provider has given him a dedicated IP address for his website. If the hosting provider offered only a single shared IP address belonging to the server, each customer could bind a different port number on that address.

While he’s first testing the setup, Jim uses --bind=localhost:1234. This causes the monotone process to listen only to port 1234 on the loopback interface 127.0.0.1, which is not accessible from the network, so Jim doesn’t expose an open port to the rest of the world until he’s satisfied with the permissions configuration. You can cause monotone to listen on all interfaces on port 1234 by leaving out the address part like --bind=:1234.

When he’s satisfied the server is set up correctly, Jim does an initial sync with the new database, filling it with all the revision history currently on his laptop. While Jim has been busy setting up the server, Abe and Beth have kept working; the server will catch up with their latest changes when they next sync, too.

All of the team members now want to sync with the new monotone server by default. Previously, they had been syncing with Jim’s laptop by default, even if they occasionally specified another team-member’s server on the command line when Jim was away, because monotone had remembered the first server and branch patterns used in database vars (see Vars). These vars can be seen as follows:

$ mtn list vars
database: default-exclude-pattern
database: default-include-pattern jp.co.juicebot.jb7*
database: default-server jim-laptop.juicebot.co.jp
known-servers: jim-laptop.juicebot.co.jp 9e9e9ef1d515ad58bfaa5cf282b4a872d8fda00c
known-servers: abe-laptop.juicebot.co.jp a2bb16a183247af4133621f7f5aefb21a9d13855
known-servers: www.juicebot.co.jp 120a99ch93b4f174432c13d3e3e9f2234aa92612

The team members can reset their local database vars accordingly:

$ mtn set database default-server www.juicebot.co.jp

With their new server, the juicebot team have gained the convenience of a readily available common point of reference for syncs. However, they also know that this is there only as a convenience, and doesn’t prevent them working as they did before:

• The team members can still sync with each other if needed.

  Hopefully, their new server won’t ever be down, but sometimes they might be working together while away from ready network access — fixing up the last few issues and finalising presentation materials while travelling to a sales conference, for example. The server will learn of these changes on the next sync.

• The team members continue to discover multiple heads and changes that need merging, as before. Each team member can merge the heads, and will produce the same revision id if they merge to the same result.

  They now develop a new habit out of courtesy, though — they try not to leave multiple heads and unmerged changes on the server, at least not for long. This saves them from repeating work, and also helps prevent confusion for the beta-testers. When each team member is ready to sync, they develop the habit of doing a pull from the server first. If new revisions were received from the server, they first merge their new revisions with the head(s) from the server, and finally sync to publish their merged changes as one. If the last sync happens to pull in new revisions again from the server, it means someone else has deposited new work at the same time, and another merge and sync would probably be polite.

• Jim knows he doesn’t have to keep a special backup of the new server’s contents; if the server should fail, all the contents of its database can be found amongst the other team members (especially because no commits are done on the server itself).

  He does, however, take a copy of the server’s private key, so he can restore that if necessary.

• In fact, Jim realises that he can now commit a copy of the web site’s current contents into monotone on a new branch, jp.co.juicebot.www, and keep a backup of that content too.

  Now he can use monotone to work on the website offline, and let other team members add and edit the content; he can also preview changes locally before updating the production content. He keeps a workspace checkout of this content in the webroot on the server, and runs a monotone update in there when he wants to bring the public web site up to date. Later, he’ll think about using monotone’s Quality Assurance mechanisms and Event Notification Hooks, so that the web server can update itself automatically when appropriate new revisions are received.

• Jim also knows that even if someone should break into the shared hosting server and tamper with the database, they won’t be able to inject malicious code into the project, because all revisions are signed by the team members, and he has set his Trust Evaluation Hooks so he doesn’t trust the server key for signing revisions.

  In monotone, the important trust consideration is on the signed content, rather than on the replication path by which that content arrived in your database.