Bad configuration file handling: proposal

[Adam Williamson]

Hi, all. I only just signed up for the list but most of you probably
know me from IRC, I'm the guy packaging synce for Mandriva.

I realized today that synce's handling of its configuration file is
pretty bad. Quick overview of how it's handled at present:

synce installs a default configuration file to /usr somewhere. This copy
of the file is never actually used in operation. It's just used for the
next bit...

When you run sync-engine, it checks if ~/.synce/config.xml exists. If it
doesn't, it copies the default configuration file from /usr to
~/.synce/config.xml .

The default file is 'active' - that is, it actually sets all the values
it contains, they're not commented out by default.

This is fairly crack-addled in many ways. Easy to illustrate with a
single value from the file:

<OpensyncXMLFormat>OS20</OpensyncXMLFormat>

okay, so we're defaulting to opensync 0.2 format. Great, that's
perfectly sensible right now. But it won't be forever. Some day,
opensync 0.3 / 0.4 will be stable, and we'll want to default to OS30 /
OS40 / whatever it's called by then.

How do we that?

Well, we...er...can't, for existing users. Everyone who's already using
synce will have a ~/.synce/config.xml . We can't make synce's
installation routine edit that file, that'd just be horrible. So anyone
who's already used synce is basically stuck with all the default
configuration values that happened to exist whenever they first
installed synce. Forever. Even if those defaults stop making sense in
future. Even if they never actively set any of those values.

This is how it should be done:

The default config file should install in /etc somewhere, and if it
exists, sync-engine should take it into account when running. It should
*not* actually be 'active' by default: it should list all the default
settings, but they should be commented out. All the defaults should be
written into synce itself.

Whether we create ~/.synce/config.xml by default or not is essentially a
moot point. We may as well, because it aids discoverability, but even if
we don't, everything would work. (As a corollary, sync-engine should not
fail to run if no user configuration file exists and it can't copy the
original from /usr for some reason - that's just crack. It should go
ahead and run, but print a warning). If we do, it should obviously be
the same as the copy in /etc : it should not actually set any values at
all by default.

This would work far better. Now if we want to change a default value in
a new version of synce, it's no problem: just change it in synce's code,
and change the commented-out default in the copy of config.xml that
installs to /etc . Anyone who didn't choose to change any default values
themselves will get the new default automatically.

It's also far better for distribution packagers, like me. The current
situation gives us absolutely no way to set distro-specific defaults.
There *must* be a system-wide configuration file, which has priority
over the written-in defaults, but itself is overridden by the user's own
config file, if it exists. This is the only sane way for distributors,
as packages can touch system-wide configuration files but cannot touch
files in a user's home directory.

Thanks for reading, hope this can be changed soon :)

[John Carr]

On Fri, May 16, 2008 at 9:16 PM, Adam Williamson

Post by Adam Williamson
Hi, all. I only just signed up for the list but most of you probably
know me from IRC, I'm the guy packaging synce for Mandriva.
I realized today that synce's handling of its configuration file is
synce installs a default configuration file to /usr somewhere. This copy
of the file is never actually used in operation. It's just used for the
next bit...
When you run sync-engine, it checks if ~/.synce/config.xml exists. If it
doesn't, it copies the default configuration file from /usr to
~/.synce/config.xml .
The default file is 'active' - that is, it actually sets all the values
it contains, they're not commented out by default.
This is fairly crack-addled in many ways. Easy to illustrate with a
<OpensyncXMLFormat>OS20</OpensyncXMLFormat>
okay, so we're defaulting to opensync 0.2 format. Great, that's
perfectly sensible right now. But it won't be forever. Some day,
opensync 0.3 / 0.4 will be stable, and we'll want to default to OS30 /
OS40 / whatever it's called by then.
How do we that?
Well, we...er...can't, for existing users. Everyone who's already using
synce will have a ~/.synce/config.xml . We can't make synce's
installation routine edit that file, that'd just be horrible. So anyone
who's already used synce is basically stuck with all the default
configuration values that happened to exist whenever they first
installed synce. Forever. Even if those defaults stop making sense in
future. Even if they never actively set any of those values.
The default config file should install in /etc somewhere, and if it
exists, sync-engine should take it into account when running. It should
*not* actually be 'active' by default: it should list all the default
settings, but they should be commented out. All the defaults should be
written into synce itself.
Whether we create ~/.synce/config.xml by default or not is essentially a
moot point. We may as well, because it aids discoverability, but even if
we don't, everything would work. (As a corollary, sync-engine should not
fail to run if no user configuration file exists and it can't copy the
original from /usr for some reason - that's just crack. It should go
ahead and run, but print a warning). If we do, it should obviously be
the same as the copy in /etc : it should not actually set any values at
all by default.
This would work far better. Now if we want to change a default value in
a new version of synce, it's no problem: just change it in synce's code,
and change the commented-out default in the copy of config.xml that
installs to /etc . Anyone who didn't choose to change any default values
themselves will get the new default automatically.
It's also far better for distribution packagers, like me. The current
situation gives us absolutely no way to set distro-specific defaults.
There *must* be a system-wide configuration file, which has priority
over the written-in defaults, but itself is overridden by the user's own
config file, if it exists. This is the only sane way for distributors,
as packages can touch system-wide configuration files but cannot touch
files in a user's home directory.
Thanks for reading, hope this can be changed soon :)
--
adamw

Fully agree, +1. Though in the OpenSync 0.3 vs OpenSync 0.2 - i would
prefer if the opensync plugin asked for a specific version over dbus
rather than the user having to manually set anything.

John

[Dr J A Gow]

Post by Adam Williamson
Hi, all. I only just signed up for the list but most of you probably
know me from IRC, I'm the guy packaging synce for Mandriva.
I realized today that synce's handling of its configuration file is
synce installs a default configuration file to /usr somewhere. This copy
of the file is never actually used in operation. It's just used for the
next bit...
When you run sync-engine, it checks if ~/.synce/config.xml exists. If it
doesn't, it copies the default configuration file from /usr to
~/.synce/config.xml .
The default file is 'active' - that is, it actually sets all the values
it contains, they're not commented out by default.

Comment them out if you wish - sync-engine will work with the same
defaults. It has to pick up runtime values from somewhere. If config.xml
is not present, it will use its own internal defaults (which are the
same as the values contained in the config file).

Post by Adam Williamson
This is fairly crack-addled in many ways. Easy to illustrate with a

Thank you. Just for the record I do not, and never have, used crack.

I agree that leaving the defaults uncommented in the XML is probably not
a good idea from the point of view of user upgrades. I'll change this in
the SVN tree tonight. It's a simple change, all that is needed is to
comment out the defaults.

Post by Adam Williamson
Everyone who's already using
synce will have a ~/.synce/config.xml . We can't make synce's
installation routine edit that file, that'd just be horrible. So anyone
who's already used synce is basically stuck with all the default
configuration values that happened to exist whenever they first
installed synce. Forever.

Err, no. Not forever. This file is read at _runtime_. Nothing is preventing
a user, (or any other program for that matter) from editing that file. That
is why I used XML for the format - many programs can sensibly handle this
format.

Post by Adam Williamson
Even if those defaults stop making sense in
future. Even if they never actively set any of those values.
The default config file should install in /etc somewhere, and if it
exists, sync-engine should take it into account when running. It should
*not* actually be 'active' by default: it should list all the default
settings, but they should be commented out. All the defaults should be
written into synce itself.

The defaults _are_ in sync-engine itself.

Post by Adam Williamson
Whether we create ~/.synce/config.xml by default or not is essentially a
moot point. We may as well, because it aids discoverability, but even if
we don't, everything would work. (As a corollary, sync-engine should not
fail to run if no user configuration file exists and it can't copy the

It should not fail to run. Sync-engine will run just fine using sensible
defaults without requiring a ~/.synce/config.xml. I do not know what the
packages do, but the development tree copies the file from a location in
the tree itself, so it should be guaranteed to be there as part of the
install. If the packages do it in a fashion where it can break, this needs
to be addressed.

However, I fully appreciate the value of leaving the defaults commented
out in the XML, and this will be done shortly.

Post by Adam Williamson
original from /usr for some reason

Only if the install is broken.

Post by Adam Williamson
- that's just crack. It should go
ahead and run, but print a warning). If we do, it should obviously be
the same as the copy in /etc : it should not actually set any values at
all by default.

If it fails, it is because the sync-engine install is broken. That file
should be there as part of a complete sync-engine install.

Post by Adam Williamson
This would work far better. Now if we want to change a default value in
a new version of synce, it's no problem: just change it in synce's code,

That is exactly what happens at the moment - but I agree that the
defaults shoudl be commented out in the XML.

Post by Adam Williamson
It's also far better for distribution packagers, like me. The current

I can see this to be true.

Post by Adam Williamson
Thanks for reading, hope this can be changed soon :)

I'll put it in SVN tonight (don't know how long it will take to reach
the packages.

John.

[Adam Williamson]

Post by Dr J A Gow

Post by Adam Williamson
This is fairly crack-addled in many ways. Easy to illustrate with a

Thank you. Just for the record I do not, and never have, used crack.

Just a turn of phrase :)

Post by Dr J A Gow
I agree that leaving the defaults uncommented in the XML is probably not
a good idea from the point of view of user upgrades. I'll change this in
the SVN tree tonight. It's a simple change, all that is needed is to
comment out the defaults.

Post by Adam Williamson
Everyone who's already using
synce will have a ~/.synce/config.xml . We can't make synce's
installation routine edit that file, that'd just be horrible. So anyone
who's already used synce is basically stuck with all the default
configuration values that happened to exist whenever they first
installed synce. Forever.

Err, no. Not forever. This file is read at _runtime_. Nothing is preventing
a user, (or any other program for that matter) from editing that file. That
is why I used XML for the format - many programs can sensibly handle this
format.

Yes, obviously, I meant _without manual intervention_. But no point
belabouring the issue, you fixed it. :)

Post by Dr J A Gow

Post by Adam Williamson
Whether we create ~/.synce/config.xml by default or not is essentially a
moot point. We may as well, because it aids discoverability, but even if
we don't, everything would work. (As a corollary, sync-engine should not
fail to run if no user configuration file exists and it can't copy the

It should not fail to run. Sync-engine will run just fine using sensible
defaults without requiring a ~/.synce/config.xml. I do not know what the
packages do, but the development tree copies the file from a location in
the tree itself, so it should be guaranteed to be there as part of the
install. If the packages do it in a fashion where it can break, this needs
to be addressed.

It's an unnecessary failure case. Sure, it indicates something wrong
with the packaging / user installation / whatever. But why fail when you
don't have to? There's no reason that sync-engine should refuse to run
just because it can't create a config file it does not, strictly
speaking, need. I think it's a decent principle to never fail to run
unless you actually *can't* run...

Post by Dr J A Gow
If it fails, it is because the sync-engine install is broken. That file
should be there as part of a complete sync-engine install.

Sure. But as I said, it's just not necessary to fail that hard. :)

[Dr J A Gow]

Post by Adam Williamson

Post by Dr J A Gow

Post by Adam Williamson
This is fairly crack-addled in many ways. Easy to illustrate with a

Thank you. Just for the record I do not, and never have, used crack.

Just a turn of phrase :)

Oops. I forgot the smiley! I do tend to overdo the caffeine though :-)

Post by Adam Williamson
It's an unnecessary failure case. Sure, it indicates something wrong
with the packaging / user installation / whatever. But why fail when you
don't have to? There's no reason that sync-engine should refuse to run
just because it can't create a config file it does not, strictly
speaking, need. I think it's a decent principle to never fail to run
unless you actually *can't* run...

See below. Speaking generally there is another category where a broken
install may allow the code to run, but changes in config or similar
could result in data corruption.

Post by Adam Williamson

Post by Dr J A Gow
If it fails, it is because the sync-engine install is broken. That file
should be there as part of a complete sync-engine install.

Sure. But as I said, it's just not necessary to fail that hard. :)

I've always been ambivalent about this issue. In some ways I know that I
would like to have some obvious indication that either the install was
either broken or that I had corrupted it inadvertently _before_ runtime,
so that it would fail rather than risk munging vital data.

If the program won't start, the user is alerted to the defect in the
install and can fix it immediately. If it runs, but logs an error, then
a user who is not regularly looking at the logs (say; someone who runs
sync-engine --detached) will not see the error until data transfer
breaks.

An example, again, would be the OpensyncXMLFormat. It's not a good
example, but the only one I can think of at the moment. The default is
currently OS20, as this is the mainstream version. But let's say that a
user had OS0.3x installed, and had reset this element in their
config.xml appropriately. Now the config.xml gets lost, and sync-engine
run subsequently. Firing OS20 data at OS30 is guaranteed to generate a
load of empty contacts! Not good, and time-consuming to fix!

John.

[Adam Williamson]

Post by Dr J A Gow
An example, again, would be the OpensyncXMLFormat. It's not a good
example, but the only one I can think of at the moment. The default is
currently OS20, as this is the mainstream version. But let's say that a
user had OS0.3x installed, and had reset this element in their
config.xml appropriately. Now the config.xml gets lost, and sync-engine
run subsequently. Firing OS20 data at OS30 is guaranteed to generate a
load of empty contacts! Not good, and time-consuming to fix!

That's true, but ironically, not actually covered by synce at
present. :) AFAIK, sync-engine will happily run if ~/.synce/config.xml
does not exist, even if it *previously* existed. It'll just copy it
afresh from /usr - and, naturally, everything then gets reset to
defaults. OS30 gets reset to OS20, and chaos ensues, as you point out.
The only case that actually causes sync-engine to fail to run at present
is if ~/.synce/config.xml doesn't exist, and it can't find the
'original' in /usr . The case where a user inadvertently (or
maliciously, or whatever...) deletes ~/.synce/config.xml , but the
'original' is still available in /usr, isn't 'protected' against at all.

[Dr J A Gow]

Post by Adam Williamson

Post by Dr J A Gow
An example, again, would be the OpensyncXMLFormat. It's not a good
example, but the only one I can think of at the moment. The default is
currently OS20, as this is the mainstream version. But let's say that a
user had OS0.3x installed, and had reset this element in their
config.xml appropriately. Now the config.xml gets lost, and sync-engine
run subsequently. Firing OS20 data at OS30 is guaranteed to generate a
load of empty contacts! Not good, and time-consuming to fix!

That's true, but ironically, not actually covered by synce at
present. :) AFAIK, sync-engine will happily run if ~/.synce/config.xml
does not exist, even if it *previously* existed. It'll just copy it
afresh from /usr - and, naturally, everything then gets reset to
defaults. OS30 gets reset to OS20, and chaos ensues, as you point out.
The only case that actually causes sync-engine to fail to run at present
is if ~/.synce/config.xml doesn't exist, and it can't find the
'original' in /usr . The case where a user inadvertently (or
maliciously, or whatever...) deletes ~/.synce/config.xml , but the
'original' is still available in /usr, isn't 'protected' against at all.

Very true. So perhaps we should continue to allow sync-engine to fail
only in the case where the 'original' is missing (it now has no defaults
active in it). This would be a broken install or a user too handy with
the rm command.

Or am I misunderstanding you slightly? Were you saying that we should
place the 'original' in /etc, rather than in the source tree, and then
copy from this to ~/.synce if the ~/.synce/config.xml is not present?

This would retain the per-user config, while placing the
'original' (empty except for comments) config.xml in /etc? The file
in /etc would never be read at runtime, but used as a template if the
user's ~/.synce/config.xml was missing?

If you can clarify this it would help.

John.

[Mark Ellis]

Post by Dr J A Gow

Post by Adam Williamson

Post by Dr J A Gow
An example, again, would be the OpensyncXMLFormat. It's not a good
example, but the only one I can think of at the moment. The default is
currently OS20, as this is the mainstream version. But let's say that a
user had OS0.3x installed, and had reset this element in their
config.xml appropriately. Now the config.xml gets lost, and sync-engine
run subsequently. Firing OS20 data at OS30 is guaranteed to generate a
load of empty contacts! Not good, and time-consuming to fix!

That's true, but ironically, not actually covered by synce at
present. :) AFAIK, sync-engine will happily run if ~/.synce/config.xml
does not exist, even if it *previously* existed. It'll just copy it
afresh from /usr - and, naturally, everything then gets reset to
defaults. OS30 gets reset to OS20, and chaos ensues, as you point out.
The only case that actually causes sync-engine to fail to run at present
is if ~/.synce/config.xml doesn't exist, and it can't find the
'original' in /usr . The case where a user inadvertently (or
maliciously, or whatever...) deletes ~/.synce/config.xml , but the
'original' is still available in /usr, isn't 'protected' against at all.

Very true. So perhaps we should continue to allow sync-engine to fail
only in the case where the 'original' is missing (it now has no defaults
active in it). This would be a broken install or a user too handy with
the rm command.
Or am I misunderstanding you slightly? Were you saying that we should
place the 'original' in /etc, rather than in the source tree, and then
copy from this to ~/.synce if the ~/.synce/config.xml is not present?
This would retain the per-user config, while placing the
'original' (empty except for comments) config.xml in /etc? The file
in /etc would never be read at runtime, but used as a template if the
user's ~/.synce/config.xml was missing?
If you can clarify this it would help.

I'll throw in my two penneth for a laugh :)

I've always considered copying the default file to ~/.synce excessive.
In fact I would consider the _application_ copying the file anywhere to
be excessive, that is a distro or an install thing. How about this...

1) sensible defaults in the app (which they are)

2) (maybe) check for a config file in /etc, if it doesn't exist, fine

3) check for a config file in ~/.synce, if it doesn't exist, fine

You then get defaults, or system config, or user config, but it's all
done knowingly ie. if someone changes or places or removes a config
file, they know it's been done.

If we need a template config.xml, stick it in /usr/share/doc/examples.

Mark

[Dr J A Gow]

Post by Mark Ellis
I'll throw in my two penneth for a laugh :)
I've always considered copying the default file to ~/.synce excessive.
In fact I would consider the _application_ copying the file anywhere to
be excessive, that is a distro or an install thing. How about this...
1) sensible defaults in the app (which they are)
2) (maybe) check for a config file in /etc, if it doesn't exist, fine
3) check for a config file in ~/.synce, if it doesn't exist, fine
You then get defaults, or system config, or user config, but it's all
done knowingly ie. if someone changes or places or removes a config
file, they know it's been done.
If we need a template config.xml, stick it in /usr/share/doc/examples.

This seems quite a sensible change, except that I am not sure about
checking /etc (sync-engine is a user-based app and config could well be
different for each user, e.g. file sync paths).

I honestly can't remember why I put the file copy in there in the first
place but on reflection it does seem daft. I'll change it in a day or
so.

John.

[Mark Ellis]

Post by Dr J A Gow

Post by Mark Ellis
I'll throw in my two penneth for a laugh :)
I've always considered copying the default file to ~/.synce excessive.
In fact I would consider the _application_ copying the file anywhere to
be excessive, that is a distro or an install thing. How about this...
1) sensible defaults in the app (which they are)
2) (maybe) check for a config file in /etc, if it doesn't exist, fine
3) check for a config file in ~/.synce, if it doesn't exist, fine
You then get defaults, or system config, or user config, but it's all
done knowingly ie. if someone changes or places or removes a config
file, they know it's been done.
If we need a template config.xml, stick it in /usr/share/doc/examples.

This seems quite a sensible change, except that I am not sure about
checking /etc (sync-engine is a user-based app and config could well be
different for each user, e.g. file sync paths).
I honestly can't remember why I put the file copy in there in the first
place but on reflection it does seem daft. I'll change it in a day or
so.
John.

Agree with the /etc thing, it doesn't really fit.

The copy is reasonable in the context of putting the config file
somewhere as a template, since looking in the python tree isn't the
first thing most people would think of.

I've recently come across some ideas on installing this kind of thing
outside of the tree with setuptools (in the Vodafone Mobile Connect Card
software out of interest). I'll try and get around to doing something
with this, should make access to the template a bit easier.

Mark

[Adam Williamson]

Post by Mark Ellis
Agree with the /etc thing, it doesn't really fit.

The point of having a system-wide config file is to allow distributions
to change defaults if necessary. For instance, if I decided to start
shipping opensync 0.3 in Mandriva Cooker, I would want to change synce's
default to 0.3 format, not 0.2. Without a system-wide config file, this
can only be done by patching the code (messy!), because distro packaging
should never ever touch anything in a user's home directory.

The standard system (and the one I'd recommend for synce) is for a
user's config file to override the system wide config file, if it exists
and has an explicit setting for the given option. So, say there are
three options - OPTION_A , OPTION_B and OPTION_C - that can all be set
to 0 or 1. The default in the software is 0. If you have this:

/etc/app.conf:

OPTION_B = 1
OPTION_C = 1

~/.app.conf:

OPTION_B = 0

What you will get is 0 for OPTION_A (app default), 0 for OPTION_B
(because the 0 in ~/.app.conf overrides the 1 in /etc/app.conf), and 1
for OPTION_C (because /etc/app.conf overrides the app default, and it's
not set in ~/.app.conf at all). That way it's still perfectly simple to
have per-user configuration where required.

[Mark Ellis]

Post by Adam Williamson

Post by Mark Ellis
Agree with the /etc thing, it doesn't really fit.

The point of having a system-wide config file is to allow distributions
to change defaults if necessary. For instance, if I decided to start
shipping opensync 0.3 in Mandriva Cooker, I would want to change synce's
default to 0.3 format, not 0.2. Without a system-wide config file, this
can only be done by patching the code (messy!), because distro packaging
should never ever touch anything in a user's home directory.
The standard system (and the one I'd recommend for synce) is for a
user's config file to override the system wide config file, if it exists
and has an explicit setting for the given option. So, say there are
three options - OPTION_A , OPTION_B and OPTION_C - that can all be set
OPTION_B = 1
OPTION_C = 1
OPTION_B = 0
What you will get is 0 for OPTION_A (app default), 0 for OPTION_B
(because the 0 in ~/.app.conf overrides the 1 in /etc/app.conf), and 1
for OPTION_C (because /etc/app.conf overrides the app default, and it's
not set in ~/.app.conf at all). That way it's still perfectly simple to
have per-user configuration where required.

Indeed, this is the way to go for config files in general. What I meant
was I can't see anywhere a system file would be useful for sync-engine,
it's all very user specific (except opensync version, see below). I
suppose of course it wouldn't hurt to make allowance for one anyway.

The point about the opensync version is good, but I think we need to
clarify something first. John, is there any problem with using opensync
0.2x to do a sync, replace opensync and the plugin with 0.3x, and then
do another sync ? If this works, then I think opensync versions in /etc
would be a good thing for distros. Of course it would be better worked
out programmatically, but I don't know if that is possible.

Mark

[Adam Williamson]

Post by Mark Ellis
Indeed, this is the way to go for config files in general. What I meant
was I can't see anywhere a system file would be useful for sync-engine,
it's all very user specific (except opensync version, see below). I
suppose of course it wouldn't hurt to make allowance for one anyway.

Yeah, I take that point, although I could argue that in some cases you
might want to make a site-wide change to AuthMethod. And, of course,
other settings may be added in future that need sitewide changes. It's
just good practice and doesn't really hurt anything.

[Mark Ellis]

Post by Adam Williamson

Post by Mark Ellis
Indeed, this is the way to go for config files in general. What I meant
was I can't see anywhere a system file would be useful for sync-engine,
it's all very user specific (except opensync version, see below). I
suppose of course it wouldn't hurt to make allowance for one anyway.

Yeah, I take that point, although I could argue that in some cases you
might want to make a site-wide change to AuthMethod. And, of course,
other settings may be added in future that need sitewide changes. It's
just good practice and doesn't really hurt anything.

I'm just discovering how much I miss autoconf and the C preprocessor :)
Python has grown on me lately, but the trouble with this is taht you
have to make certain assumptions about the install system that I
wouldn't dream of doing in C.

Anyway, apart from that, I've nearly got a patch for this, just needs a
bit of a test.

Mark

[Dr J A Gow]

Post by Mark Ellis
clarify something first. John, is there any problem with using opensync
0.2x to do a sync, replace opensync and the plugin with 0.3x, and then
do another sync ? If this works, then I think opensync versions in /etc
would be a good thing for distros. Of course it would be better worked
out programmatically, but I don't know if that is possible.

I strongly suspect this might not work, though I haven't delved into
Opensync internals enough to be 100% sure. I think that Opensync's sync
database is different between OS0.2x and OS0.3x - meaning that the files
and Opensync sync partnership generated by OS0.2x would not be
compatible with OS0.3x (all in the ~/.opensync directory). So we can not
use old 0.2x sync data with 0.3x.

I can see the point now of putting a systemwide file in /etc, and this
would not be difficult to implement. Not only would AuthMethod and
OpensyncXMLFormat benefit from this, but some of the file sync config
settings could be hardware (i.e. system) dependent rather than user
dependent.

I would be happy to take the approach of scanning /etc, then ~/.synce,
and if unset use the app defaults.

Alternatively, packagers could put the template file in somewhere
sensible, such as /usr/share/synce/examples or similar, where it could
be found by users who want to customize their configuration.

John.

[Adam Williamson]

Post by Dr J A Gow
I would be happy to take the approach of scanning /etc, then ~/.synce,
and if unset use the app defaults.
Alternatively, packagers could put the template file in somewhere
sensible, such as /usr/share/synce/examples or similar, where it could
be found by users who want to customize their configuration.

I think it's actually fine to install a copy (as discussed earlier, with
all values commented out) to ~/.synce/config.xml when first running
synce - as we all understand it's purely to hint to the user that they
can indeed use a file in that location to change sync-engine's
configuration.

The usual behaviour is that an application creates the per-user config
file when you first make a change to the configuration within the app
itself, but this doesn't apply to sync-engine, as you can't change any
of the settings in the config file from 'within' the app.

So given that, having it created on first run is sensible enough, and
probably better than expecting people to learn from documentation, or
guess, that they can copy the system-wide file there and edit it. The
thing I was really objecting to initially in the this thread was never
the creation of the file per se, but the fact that it had values
explicitly set by default.

BTW, for what it's worth, I currently patch sync-engine in Mandriva to
install the template config file to
self.sepath,"SyncEngine/config/config.xml" rather than
self.path,"config.xml" .

[Mark Ellis]

Post by Adam Williamson

Post by Dr J A Gow
I would be happy to take the approach of scanning /etc, then ~/.synce,
and if unset use the app defaults.
Alternatively, packagers could put the template file in somewhere
sensible, such as /usr/share/synce/examples or similar, where it could
be found by users who want to customize their configuration.

I think it's actually fine to install a copy (as discussed earlier, with
all values commented out) to ~/.synce/config.xml when first running
synce - as we all understand it's purely to hint to the user that they
can indeed use a file in that location to change sync-engine's
configuration.
The usual behaviour is that an application creates the per-user config
file when you first make a change to the configuration within the app
itself, but this doesn't apply to sync-engine, as you can't change any
of the settings in the config file from 'within' the app.
So given that, having it created on first run is sensible enough, and
probably better than expecting people to learn from documentation, or
guess, that they can copy the system-wide file there and edit it. The
thing I was really objecting to initially in the this thread was never
the creation of the file per se, but the fact that it had values
explicitly set by default.

See your point, not sure what's best really. The main thing I don't like
is that we copy it there with no way to update it for new options, or
anything else, whereas other apps that write a config do have this
capability.

Post by Adam Williamson
BTW, for what it's worth, I currently patch sync-engine in Mandriva to
install the template config file to
self.sepath,"SyncEngine/config/config.xml" rather than
self.path,"config.xml" .

John, I'm attaching a patch for you. I would've committed it myself but
I'm still not that familiar with most of sync-engine, so thought it best
that you had a look.

1) installs config.xml into sys.prefix/share/doc/sync-engine

2) doesn't copy the file

3) Reads the system config file first, which is expected to be in /etc
if sys.prefix is /usr, or sys.prefix/etc otherwise, then reads the user
file

Works ok for me.

I did think we should probably change the name of config.xml if it's
going in /etc. Could go in a subdirectory but that seems excessive, how
about synceng-conf.xml ?

Mark

[Dr J A Gow]

Post by Mark Ellis
John, I'm attaching a patch for you. I would've committed it myself but
I'm still not that familiar with most of sync-engine, so thought it best
that you had a look.
1) installs config.xml into sys.prefix/share/doc/sync-engine
2) doesn't copy the file
3) Reads the system config file first, which is expected to be in /etc
if sys.prefix is /usr, or sys.prefix/etc otherwise, then reads the user
file

The patch seems basically OK from looking at it. I'm not going to get a
chance to check it physically until tomorrow night - can you try one
thing? Just delete the entirety of ~/.synce, not just its contents. Then
see if it gets correctly recreated when sync-engine is run and new
partnerships are created.

Reason is that you have removed the line that creates config_dir. Now I
can't remember whether the partnership code will also create it without
looking at the code, and I have a feeling it may fail when no ~/.synce
exists.

I will submit it to a more thorough evaluation tomorrow if you can wait
that long :)

John.

[Mark Ellis]

Post by Dr J A Gow

Post by Mark Ellis
John, I'm attaching a patch for you. I would've committed it myself but
I'm still not that familiar with most of sync-engine, so thought it best
that you had a look.
1) installs config.xml into sys.prefix/share/doc/sync-engine
2) doesn't copy the file
3) Reads the system config file first, which is expected to be in /etc
if sys.prefix is /usr, or sys.prefix/etc otherwise, then reads the user
file

The patch seems basically OK from looking at it. I'm not going to get a
chance to check it physically until tomorrow night - can you try one
thing? Just delete the entirety of ~/.synce, not just its contents. Then
see if it gets correctly recreated when sync-engine is run and new
partnerships are created.
Reason is that you have removed the line that creates config_dir. Now I
can't remember whether the partnership code will also create it without
looking at the code, and I have a feeling it may fail when no ~/.synce
exists.

Partnership.__init__ checks to see if the partnerships dir exists, and
if not does an os.makedirs(), which is recursive, on the full path,
looks fine.

Post by Dr J A Gow
I will submit it to a more thorough evaluation tomorrow if you can wait
that long :)
John.

Cool, I'm not going anywhere :)

[John Carr]

Post by Mark Ellis

Post by Dr J A Gow

Post by Mark Ellis
John, I'm attaching a patch for you. I would've committed it myself but
I'm still not that familiar with most of sync-engine, so thought it best
that you had a look.
1) installs config.xml into sys.prefix/share/doc/sync-engine
2) doesn't copy the file
3) Reads the system config file first, which is expected to be in /etc
if sys.prefix is /usr, or sys.prefix/etc otherwise, then reads the user
file

The patch seems basically OK from looking at it. I'm not going to get a
chance to check it physically until tomorrow night - can you try one
thing? Just delete the entirety of ~/.synce, not just its contents. Then
see if it gets correctly recreated when sync-engine is run and new
partnerships are created.
Reason is that you have removed the line that creates config_dir. Now I
can't remember whether the partnership code will also create it without
looking at the code, and I have a feeling it may fail when no ~/.synce
exists.

Partnership.__init__ checks to see if the partnerships dir exists, and
if not does an os.makedirs(), which is recursive, on the full path,
looks fine.

Post by Dr J A Gow
I will submit it to a more thorough evaluation tomorrow if you can wait
that long :)
John.

Cool, I'm not going anywhere :)

config.xml in trunk is broken... specifically:

<AutoSyncCommand>xterm -e msynctool --sync wm5sync</AutoSyncCommand>

is commented out. Unfortunately, XML doesn't like the use of -- within
an XML comment. I can't see anyway around this. Other than "Please see
/usr/share/blah/config.xml.sa

[Dr J A Gow]

Post by Mark Ellis
Cool, I'm not going anywhere :)

It's in r3465 with a slight mod to remove some redundancy, and I have
also renamed config.xml to syncengine.conf.xml to avoid any confusion in
future. The double-hyphen in the template I have dealt with simply by
removing the example sync command and just inserting some text to tell
the user to insert their sync command at this point (a better approach,
I think, as a user's sync group name is almost guaranteed to be
different to the one I had in the file!

Seems OK here with my usual batch of stress-tests across three different
handheld types, so we can see how it flies in the field....

John.

[Mark Ellis]

Post by Dr J A Gow

Post by Mark Ellis
Cool, I'm not going anywhere :)

It's in r3465 with a slight mod to remove some redundancy, and I have
also renamed config.xml to syncengine.conf.xml to avoid any confusion in
future. The double-hyphen in the template I have dealt with simply by
removing the example sync command and just inserting some text to tell
the user to insert their sync command at this point (a better approach,
I think, as a user's sync group name is almost guaranteed to be
different to the one I had in the file!
Seems OK here with my usual batch of stress-tests across three different
handheld types, so we can see how it flies in the field....
John.

Good stuff John. I've got another sync-engine tweak, but I think that's
for another thread...

[Mark Ellis]

Post by Dr J A Gow

Post by Mark Ellis
clarify something first. John, is there any problem with using opensync
0.2x to do a sync, replace opensync and the plugin with 0.3x, and then
do another sync ? If this works, then I think opensync versions in /etc
would be a good thing for distros. Of course it would be better worked
out programmatically, but I don't know if that is possible.

I strongly suspect this might not work, though I haven't delved into
Opensync internals enough to be 100% sure. I think that Opensync's sync
database is different between OS0.2x and OS0.3x - meaning that the files
and Opensync sync partnership generated by OS0.2x would not be
compatible with OS0.3x (all in the ~/.opensync directory). So we can not
use old 0.2x sync data with 0.3x.
I can see the point now of putting a systemwide file in /etc, and this
would not be difficult to implement. Not only would AuthMethod and
OpensyncXMLFormat benefit from this, but some of the file sync config
settings could be hardware (i.e. system) dependent rather than user
dependent.

I was going to say at this point that it would be best to not allow
users to override the OS version, but I've just noticed that my
~/.opensync is actually ~/.opensync-0.22 (Ubuntu hardy), whilst on
Debian it was just ~/.opensync, so it is probably safe in certain
circumstances.

Mark

Post by Dr J A Gow
I would be happy to take the approach of scanning /etc, then ~/.synce,
and if unset use the app defaults.
Alternatively, packagers could put the template file in somewhere
sensible, such as /usr/share/synce/examples or similar, where it could
be found by users who want to customize their configuration.
John.

[John Carr]

Post by Mark Ellis
I was going to say at this point that it would be best to not allow
users to override the OS version, but I've just noticed that my
~/.opensync is actually ~/.opensync-0.22 (Ubuntu hardy), whilst on
Debian it was just ~/.opensync, so it is probably safe in certain
circumstances.
Mark

azeem did that because there was no way to safely migrate from
opensync 0.19 to opensync 0.22. I believe they plan to be able to
migrate opensync 0.22 to 0.4 when it is released.

John

[Dr J A Gow]

-------- Forwarded Message --------
From: Dr J A Gow <***@furrybubble.co.uk>
To: John Carr <***@unrouted.co.uk>
Subject: Re: Bad configuration file handling: proposal
Date: Fri, 16 May 2008 22:54:00 +0100

Post by John Carr
Fully agree, +1. Though in the OpenSync 0.3 vs OpenSync 0.2 - i would
prefer if the opensync plugin asked for a specific version over dbus
rather than the user having to manually set anything.

I'll change the config.xml stuff tonight - but I'll look at the d-bus
Opensync version detect soon - I agree this would be much slicker.

All done in r3462.

John.

[Adam Williamson]

Post by Dr J A Gow
-------- Forwarded Message --------
Subject: Re: Bad configuration file handling: proposal
Date: Fri, 16 May 2008 22:54:00 +0100

Post by John Carr
Fully agree, +1. Though in the OpenSync 0.3 vs OpenSync 0.2 - i would
prefer if the opensync plugin asked for a specific version over dbus
rather than the user having to manually set anything.

I'll change the config.xml stuff tonight - but I'll look at the d-bus
Opensync version detect soon - I agree this would be much slicker.

All done in r3462.

That's great, thanks. Just leaves the issue of having a system-wide
config file (/etc/synce/config.xml , or something) and having
sync-engine not fail if it cannot copy config.xml out of /usr for
whatever reason. Thanks again :)

[Dr J A Gow]

Post by Adam Williamson

Post by Dr J A Gow
-------- Forwarded Message --------
Subject: Re: Bad configuration file handling: proposal
Date: Fri, 16 May 2008 22:54:00 +0100

Post by John Carr
Fully agree, +1. Though in the OpenSync 0.3 vs OpenSync 0.2 - i would
prefer if the opensync plugin asked for a specific version over dbus
rather than the user having to manually set anything.

I'll change the config.xml stuff tonight - but I'll look at the d-bus
Opensync version detect soon - I agree this would be much slicker.

All done in r3462.

That's great, thanks. Just leaves the issue of having a system-wide
config file (/etc/synce/config.xml , or something) and having
sync-engine not fail if it cannot copy config.xml out of /usr for
whatever reason. Thanks again :)

A system-wide config.xml may not be as wise. The reason for this is down
to the fact that sync-engine runs as a user. Although only one device is
usually plugged in at any one time, each user on a system may have their
own config.xml with their own desired settings for their instance of
sync-engine.

The only truly system-wide setting is the Opensync version setting - but
this option can disappear if we can automatically detect it.

We could have a systemwide config.xml overridden by a
user/.synce/config.xml but this seems unnecessarily complex and messy
with no real advantages.

John.