Configuration Handling#

The configuration handling in SyncEvolution was originally designed for a SyncML client. This article explains how it is to be extended to cover SyncML servers and other use cases.

Terminology and Items#

source: A data source is a local database which is to be synchronized.
backend: Access to data sources in SyncEvolution is done via backends which are either compiled into the main executable or loaded dynamically. Exactly one backend is responsiple for each data source.
server: A SyncML server implements the more complex role in a SyncML session. It has to remember meta information about all its clients.
client: A SyncML client typically has to store less or at least, different meta information. Client and server cannot switch their roles without starting over from scratch.
peer: Used instead of client and server in cases when the specific role does not matter.
property: A single configuration value. Identified by its location in the configuration tree and its key. A property has a value, a comment and an attribute which determines whether the attribute is internal (not meant to be edited by users) or user-configurable.
configuration tree: A hierarchy of nodes. Each node contains a set of configuration properties with unique keys plus one or more additional key/value stores. Many different implementations of a configuration tree are possible; the one currently used in SyncEvolution uses plain .ini files inside a directory hierarchy that is rooted in $XDG_CONFIG_ROOT/syncevolution (typically ~/.config/syncevolution). User-visible properties are stored in config.ini, hidden ones in .internal.ini. Each backend gets access to a key/value store for its own, for example change tracking (.other.ini). In a server, the glue code for a backend and the Synthesis engine uses .server.ini for the Synthesis meta information.
binfiles: Utility code in a Synthesis client engine which stores meta information automatically in a configured directory. This code simplifies change tracking (no need to report deleted items; no extra work for suspend/resume state). This data is stored in a .synthesis directory at the root of the configuration tree of a client.

History#

The initial implementation of the SyncEvolution configuration was provided by the Funambol C++ client library. It was not meant to be accessed by users directly and therefore was not very user-friendly (deep directory hierarchy with unusual names).

SyncEvolution 0.8 reimplemented this using its own code and a simpler file layout. All data was stored as properties or in the key/value store. In SyncEvolution 0.9, the configuration was kept more or less the same despite the migration to the Synthesis engine. The .synthesis directory was added. The main new feature in SyncEvolution 1.0, the support for running as a SyncML server, requires more fundamental changes. So far, each configuration contained a complete set of source settings for each server. Different configurations were completely independent, despite accessing the same local data. Because a client typically only talks to one server, this was intuitive and acceptable despite duplicating some settings when configuring multiple servers. A server talks to many different clients. We have operations on the local data (backup/restore, viewing logs) which are independent of the peer. Given these new use cases, the main drawbacks of the old scheme are:

Settings for sources and logging are duplicated and might become intentionally or unintentionally inconsistent between peers.
Finding all old sessions involving a certain local data source is hard: one has to check multiple different peer configurations and cannot be sure that source “addressbook” of peer A really is the same “addressbook” as in peer B.
When we are contacted for the first time by a peer, it is unclear which local source and logging configuration we are supposed to use.

Configuration Handling SyncEvolution >= 1.0#

Before 1.0, users and GUIs only had to deal with a very simple configuration schema: a server configuration had sync properties describing the server and parameterizing the sync and one set of source properties for each data source. This directly mapped to user-visible config files like this:

.config/syncevolution/memotoo/
    config.ini
    sources/
        addressbook|calendar|.../
            config.ini

We decided to keep it that simple as far as the user and the APIs are concerned. SyncEvolution >= 1.0 provides a view which only distinguishes between sync properties and source properties. Some sync properties are shared between these views. Changing them in one view also updates all other views. Each property is stored in exactly one config node. Sync properties are now

global: the same in all configurations, for example “which peer is the default”
per source set: apply to all peer configurations using the same set of local data sources, for example logging settings
per peer: specific to a certain peer, like device ID or sync URL

Source properties are:

per source: the same for all users of a certain source, like backend and database selection
per peer and source: meta information, URIs at the peer’s side

A view is selected via a config name. A fuzzy matching is used to select the relevant properties:

empty string “”: A view with global and per source properties and no properties specific to a peer.
<peer name>: A view with global and per source properties and properties specific to the peer. This is the same as the traditional “server configuration” and can be used like one.
<peer name>@<source set>: Instead of the default set of sources and peers, a completely different configuration can be used. It only has the global sync properties in common with other configurations.

The layout looks like this:

.config/syncevolution/
   config.ini          global sync properties
   default/
      config.ini       source set config
      .synthesis       shared (!) Synthesis binfiles
      sources/
         addressbook|calendar|.../
            config.ini        source config
      peers/
         scheduleworld/
            config.ini        peer configuration
            sources/
               addressbook|calendar|.../
                  config.ini  per peer source config
                  .other.ini  per peer and per source storage for backend
         phone_xyz/
            ...
   alternate/
      ....             a complete set of configuration files as above

For each config.ini there’s also an .internal.ini which is not exposed via command line or D-Bus API. The exact location of each property is defined in this table:

Property Type	Properties
global	defaultPeer
per source set	logdir, maxlogdirs
peer	syncURL, username, password, useProxy, proxyHost, proxyUsername, proxyPassword, RetryDuration, RetryInterval, deviceId, enableWBXML, maxMsgSize, maxObjSize, SSLServerCertificates, SSLVerifyServer, SSLVerifyHost, loglevel, printChanges, WebURL, ConsumerReady, IconURI
source	type, evolutionsource, evolutionuser, evolutionpassword
per peer source properties	sync, uri

In the new scheme the Synthesis binfiles are shared between peers, as intended by Synthesis. This is not necessary for us at the moment, but some of the binfile features (simplified change tracking via a single “updated” flag in the local database) might be useful for other backends. When asking for a peer template, that template is already populated with existing shared properties, so that it can be written back without unintentionally overwriting those.

Synthesis Data Storage#

Meta data for sync clients is stored in the .synthesis directory using the binfile support which is compiled into the engine. The only configurable parameter here is the location of these files. Meta data for sync servers is handled differently:

Per source local to remote ID mapping is written into the .server.ini key/value stores if a backend decides to use the SyncSourceAdmin class. A backend can decide to provide a different implementation, but one has to be provided.
Per source admin data (for example, sync anchors) are written as a text blob into the internal, per source, per peer “adminData” property by SyncSourceAdmin. The content of this blob is opaque for SyncEvolution.
Per peer data is stored as internal sync properties by SyncContext: lastNonce, remoteDeviceID, adminData. SyncContext also handles passwords and session login.

Multiple Transports per Peer#

This was discussed in the context of Moblin. In particular we talked about how to specify multiple ways of contacting a peer. This is still a bit in flux, but in general the idea is that one peer configuration documents all methods available for contacting a peer.

There’s one problem with that: how does the GUI create such a configuration correctly when it only knows that “Bluetooth device with MAC address xyz is available for sync”? There might be a peer configuration for this device already, using USB as transport. But because the only common element between the two peers (the SyncML Device ID) isn’t available for Bluetooth at this point, the existing peer configuration cannot be identified automatically.

Do we rely on the user to add the Bluetooth transport to the right existing configuration? If he gets that wrong, he will end up with two different local sets of meta data (source/addressbook/.other.ini), which will break syncs when switching back and forth between the two transports.

The new configuration handling should be tolerant of this. This means that the location of the .other.ini files can only be determined once the peer’s Device ID is known (internal backend API change).

A separate directory for .other.ini files would make the configuration layout more complex in all cases. Therefore the .other.ini file is kept in the normal/peer/*/sources/* directories and duplicated peer configurations are handled as follows:

when we learn about the Device ID of a peer, we check which of the currently configured peers has the same ID and use an existing .other.ini file for it
when removing a peer configuration, we have to check whether the .other.ini is still needed and if so, move the file because it is still needed