Data Conversion + Testing

SyncEvolution comes with a CPPUnit based test suite. The “client-test” test runner executes the tests, which consist of both traditional unit tests as well as integration tests. Compilation determines which tests are available. See the HACKING document for details.

This page provides background information about some of the tests.

Data Conversion

PIM data passes through several components on the way from a local data source like Evolution to another device like a mobile phone. Each component must work correctly to avoid data loss or mangling. The chain of data processing is this: native data source <– native format –> SyncEvolution plugin for that source (EvolutionSyncSource) <– native format –> Synthesis DB API <– Synthesis internal format –> Synthesis Engine <– SyncML peer format –> SyncML peer <– peer internal format –> storage on peer.

In releases <= 0.8.1, SyncEvolution more or less uses the same format for “native” and its “peer”. In 0.9 beta 1, it became possible to let the Synthesis engine do conversions (e.g. iCalendar 2.0 <-> vCalendar 1.0), but the configuration of the engine still uses the same “flavor” of iCalendar 2.0 for both Evolution and the peer.

In future releases, SyncEvolution will be changed so that both sides of the Synthesis engine can use different configurations for their data formats. With that change it will become possible to let the engine handle properties on the Evolution side which the other side cannot represent (for example, X-EVOLUTION extensions which only make sense locally) or represents differently (X-EVOLUTION-SPOUSE vs. X-SPOUSE).

The following table describes the different components and which tests cover them. The * wildcard stands for a data source name which uniquely identifies both the SyncEvolution backend and the corresponding server configuration. For example, “vcard21” and “vcard30” both access Evolution contacts using the internal vCard 3.0 format, but are meant to be used with servers which require vCard 2.1 resp. vCard 3.0.

Data comparison is done with the “synccompare” Perl script. It normalizes data before comparing it, so allowed variations in the data representation don’t show up as failures. It also simplifies the test data to suppress known differences; this is currently hard-coded in the script.

Component	Test Data	Tests
local import/export of single items: EvolutionSyncSource::insertItem/createItem()	defined via RegisterSyncSourceTest; must match native format	Client::Source::*::testImport
The “testcase” files used by the Evolution data sources are vcard30.vcf, ical20.ics, itodo20.ics, imemo20.ics, respectively. Simple items are also generated automatically based on templates hard-coded in the source code. The test imports each of the items into the local data storage, then exports them again. Comparison is done between the original testcase file and a local dump called Client_Source_*_testImport.B.test.dat. Differences found in this test are caused by either the EvolutionSyncSource or the local data storage.
conversion native <-> internal format: Synthesis Engine, XML Configuration in SyncEvolution, sysync::DataConversion API	same test cases as before	Client::Sync::*::testConversion
This test runs locally without contacting the server, like the Client::Source tests. But the involved components require a session context, which is why this test is implemented as a Client::Sync test. It uses a preliminary sysync::DataConversion() call which was introduced specifically for this purpose. Data comparison is done between the original test case file and Client_Sync_*_testConversion.convertedvcard30.dat. The test ensures that the internal format, which is defined by the XML configuration provided by SyncEvolution, is capable enough to store all data that the local data storage can handle. Differences can be caused by an incomplete XML configuration, bugs and/or limitations in the conversion engine.
conversion native <-> internal format: same components as before	no specific test cases right now	no dedicated test, done as part of round-trip test below
Similar to the Client::Sync::*::testConversion test this conversion could be covered by automated tests without really talking to a server. One would have to gather test cases in the format used by different peers and corresponding meta information (in particular capabilities information), then ensure that the conversion engine is in exactly the same state as it would be during a normal sync. The sysync::DataConversion API is not yet complete enough for that.
client A -> server -> client B	same as for testImport	Client::Sync::*::testItems
The simpler Client::Sync tests just test synchronization between one client and the server. The more advanced ones involved a simulated client A and client B which both run on the same host and access the same remote databases, but use different local ones. That way it is possible to test more complex scenarios (incremental sync of changes, collisions) and for the purpose of data conversion testing, sending of the test case items from one client to the other. Comparison of data is between data exported by both clients after importing it into client A, sending to the server, and receiving it into client B. The comparison is intentionally not between the original test cases and the data exported by client B: that way errors introduced by the import/export step only show up in Client::Source::*::testImport and not again in Client::Sync::*::testItems. Differences found by this test can be due to the same reasons as for Client::Sync::*::testConversion, the untested conversion towards the server, and limitations/bugs of the server. Analyzing such differences usually involves an analysis of what is really sent to the server and what the server sends back. If it sends back less data, then either the server simply doesn’t support a certain property (server needs to be improved) or the data wasn’t sent to it in a way that it understands (local data conversion and/or server need to be improved).
client A -> server -> client B client A <- server <-	test data that contains properties only supported by Evolution	none at the moment
One interesting feature of the Synthesis Engine is that when receiving an update for an existing item, it can merge the old data with the new data so that properties which are only supported locally are preserved. Simpler implementations would replace the local item with the remote data completely, thus losing information if the server is less capable than the client. SyncEvolution <= 0.8.1 had this problem. Another example are file attachements: up to and including 0.9 beta 1, calendar events are not supported by SyncEvolution’s Evolution calendar backend and thus would fail the testImport test if they were part of the test cases. In 0.9 support should be added, but then it is likely that some servers don’t support them. This will show up in the testItems comparison. In such a case, sending a reduced item back to client A should not remove the additional data from the item on client A: if the server properly declares its capabilities, then the Synthesis Engine should be able to preserve the additional data.