Peer to peer synching with TouchDB

Updated 2012-06-05: Incorporated Jens's suggestions and corrections.

TouchDB is a lean CouchDB-compatible database framework that can be embedded in iOS applications (or more generally, mobile or desktop applictions but this post is about iOS). Jens Alfke, its author, describes it this way: “If CouchDB is MySQL, then TouchDB is SQLite.” The project is available on github.

TouchDB is CouchDB-compatible with respect to its replication API when initiated on the device against another ‘regular’ CouchDB. You can create push and pull replication tasks on TouchDB. However, out of the box, TouchDB does not offer an HTTP interface for other TouchDB (or CouchDB) instances to connect to. This means that initially, you are limited to a “star” topology with a regular CouchDB at its center and iOS devices with TouchDB connecting to it as a synchronization hub.

However, with a little extra work, it is quite easy to turn this into a peer to peer setup, thanks to the Listener framework Jens has included in TouchDB.

In order to get this to work, you first need to build the listener framework. To do so, clone the git repository, pull the submodules and build the “Listener iOS Framework” target as follows:

git clone https://github.com/couchbaselabs/TouchDB-iOS
cd TouchDB-iOS
git submodule init
git submodule update
xcodebuild -target "Listener iOS Framework"
open build/Release-ios-universal

The open command will open a Finder window with the framework, which you need to add to your existing project.

After you have done that, you need to start the listener. One place where you might want to do that could be application:didFinishLaunchingWithOptions:. Add the following code to start the listener:

CouchTouchDBServer *server = [CouchTouchDBServer sharedInstance];
[server tellTDServer:^(TDServer *tdServer) {
  NSLog(@"Starting listener");
  _listener = [[TDListener alloc] initWithTDServer:tdServer port:59840];
  [_listener start];
}];

NB: Make sure _listener is retained outside the block and lives on, otherwise your listener goes out of scope and stops listening immediately. And as you can tell from the unbalanced alloc message: these code snippets are assuming ARC.

This is basically all you need to do to connect to your TouchDB instance via HTTP. For example, you could use curl to query it for documents. However, peer to peer benefits from advertising and discovering your service via Bonjour and the rest of this article briefly describes how to achieve this.

First off the advertising part. Add the following to a startup section of your application, for example right after creating the listener:

UIDevice *device = [UIDevice currentDevice];
self.netService = [[NSNetService alloc] initWithDomain:@"local" type:@"_myapp._tcp" name:device.name port:59840];
NSData *data = [NSNetService dataFromTXTRecordDictionary:[NSDictionary dictionaryWithObject:conf.localDbname forKey:@"path"]];
[self.netService setTXTRecordData:data];
[self.netService publish];

Replace myapp and 59840 with values of your choosing and note that it is advisable to choose a better service name than simply the device name as I have done in this example.

For discovery, you create an NSNetServiceBrowser and search for hosts of your service type:

self.browser = [[NSNetServiceBrowser alloc] init];
self.browser.delegate = self;
[self.browser searchForServicesOfType:@"_myapp._tcp" inDomain:@"local"];

You will be notified of any matches by implementing the following NSNetServiceBrowserDelegate protocol callback:

- (void)netServiceBrowser:(NSNetServiceBrowser *)netServiceBrowser didFindService:(NSNetService *)netService moreComing:(BOOL)moreServicesComing
{
  [self.services addObject:service];
  if (! moreServiceComing) {
    [self.tableView reloadData];
  }
}

In this example, I’ve added the service to an array. This could be an array that is driving a UITableView for example. (There’s a complete bonjour browser example available on the iOS Dev Center that includes a browsing UI and discovery and resolution for bonjour services that these code examples are based on.)

As Jens Alfke correctly points out in the comments, it is important to implement the companion method netServiceBrowser:didRemoveService:moreComing: as well in order to remove a service from the list when it disappears:

- (void)netServiceBrowser:(NSNetServiceBrowser *)netServiceBrowser didRemoveService:(NSNetService *)netService moreComing:(BOOL)moreServicesComing
{
  [self.service removeObject:service];
  if (! moreServiceComing) {
    [self.tableView reloadData];
  }
}

Once a service is selected in this table view, we try to resolve it:

- (void)tableView:(UITableView *)tableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath
{
  NSNetService *service = [self.services objectAtIndex:indexPath.row];
  [service setDelegate:self];
  [service resolveWithTimeout:0.0];
}

Finally, we implement the relevant part of the NSNetServiceDelegate protocol to handle the resolved address. This is where we would then update the sync settings for our app, which is encapsulated in the [self updateSync:url] in this example. This would be the same updateSync: present in the TouchDB example apps.

- (void)netServiceDidResolveAddress:(NSNetService *)sender {
  // Construct the URL including the port number
  // Also use the path, username and password fields that can be in the TXT record
  NSDictionary* dict = [NSNetService dictionaryFromTXTRecordData:[service TXTRecordData]];
  NSString *host = [service hostName];
  NSString* user = [self copyStringFromTXTDict:dict which:@"u"];
  NSString* pass = [self copyStringFromTXTDict:dict which:@"p"];
  NSString* portStr = @"";

  // Note that [NSNetService port:] returns an NSInteger in host byte order
  NSInteger port = [service port];
  if (port != 0 && port != 80) {
    portStr = [[NSString alloc] initWithFormat:@":%d",port];
  }

  NSString* path = [self copyStringFromTXTDict:dict which:@"path"];
  if (!path || [path length]==0) {
    path = [[NSString alloc] initWithString:@"/"];
  } else if (![[path substringToIndex:1] isEqual:@"/"]) {
    NSString *tempPath = [[NSString alloc] initWithFormat:@"/%@",path];
    path = tempPath;
  }

  NSString *ipAddress = nil;
  for (NSData* data in [service addresses]) {
    char addressBuffer[100];
    struct sockaddr_in* socketAddress = (struct sockaddr_in*) [data bytes];
    int sockFamily = socketAddress->sin_family;
    if (sockFamily == AF_INET /* || sockFamily == AF_INET6 */) {
      const char* addressStr = inet_ntop(sockFamily,
                                         &(socketAddress->sin_addr), addressBuffer,
                                         sizeof(addressBuffer));
      int port = ntohs(socketAddress->sin_port);
      if (addressStr && port) {
        NSLog(@"Found service at %s:%d", addressStr, port);
        ipAddress = [NSString stringWithCString:addressStr encoding:NSASCIIStringEncoding];
      }
    }
  }

  NSString* url = [[NSString alloc] initWithFormat:@"http://%@%@%@%@%@%@%@",
                   user?user:@"",
                   pass?@":":@"",
                   pass?pass:@"",
                   (user||pass)?@"@":@"",
                   ipAddress?ipAddress:host,
                   portStr,
                   path];

  NSLog(@"service: %@", service);
  NSLog(@"url: %@", url);
  [self updateSyncURL:url];
}

The method above references one simple helper method to access bonjour data from the service:

- (NSString *)copyStringFromTXTDict:(NSDictionary *)dict which:(NSString*)which {
  // Helper for getting information from the TXT data
  NSData* data = [dict objectForKey:which];
  NSString *resultString = nil;
  if (data) {
    resultString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
  }
  return resultString;
}

As mentioned above, this bonjour code is mostly from the Apple example code of BonjourWeb but it required some minor changes. I’ve added the path component to broadcast which database to replicate with. I’ve also commented out the AF_INET6 socket family part, because it did not work with the replication and for the same reason I’m using the IP address for the URL rather than the clear name, because this also did not yield a working connection.

Hopefully this post will help people getting started with TouchDB peer-to-peer replication!