[Synce-devel] Information on Pocket Outlook Inbox Database format

Discussion:

Sven Oliver Moll

2006-08-08 18:19:14 UTC

Hello,

I bought myself a second-hand HP Jornada 720, and I'm pretty amazed on how
nice it works together with Linux as a host. A pretty nice job you've done
on SynCE and the other stuff.

Right now I'm going through the sourcecode of synce-tools and rra-tools.
Based upon pls and pcp I managed to build myself a small fetch-updated-
documents tool that also converts from Pocket Word to RTF.
(I'm just changing the header from {\pwd2 to {\rtf1, but it works so that
I can finish the RTFs with OpenOffice.)

Once that's done, I was thinking about accessing the Pocket Outlook Inbox
ActiveSync folders via rra. Playing around with the rra-tools, I managed
to download a testmessage to a file on the Linux side. But now I'm stuck
since I don't know anything on the format. I know it's my message, since
I can see the UTF-16 encoded text of it, but can anyone point me out to
a resource on how to interpret the binary data around it? It doesn't
matter, if it's a format description or some sourcecode.

Thanks in advance.

Greetings from Germany,
SvOlli

David Eriksson

2006-08-08 20:14:44 UTC

Permalink

Post by Sven Oliver Moll
Hello,
I bought myself a second-hand HP Jornada 720, and I'm pretty amazed on how
nice it works together with Linux as a host. A pretty nice job you've done
on SynCE and the other stuff.

Thanks for the kind words!

Post by Sven Oliver Moll
Right now I'm going through the sourcecode of synce-tools and rra-tools.
Based upon pls and pcp I managed to build myself a small fetch-updated-
documents tool that also converts from Pocket Word to RTF.
(I'm just changing the header from {\pwd2 to {\rtf1, but it works so that
I can finish the RTFs with OpenOffice.)

See also http://synce.sourceforge.net/synce/task_pwi.php and
http://www.synce.org/index.php/ISF is probably used for the
hand-drawings.

Please keep us posted with your progress!

Post by Sven Oliver Moll
Once that's done, I was thinking about accessing the Pocket Outlook Inbox
ActiveSync folders via rra. Playing around with the rra-tools, I managed
to download a testmessage to a file on the Linux side. But now I'm stuck
since I don't know anything on the format. I know it's my message, since
I can see the UTF-16 encoded text of it, but can anyone point me out to
a resource on how to interpret the binary data around it? It doesn't
matter, if it's a format description or some sourcecode.

Actually "Khamael" in #synce mentioned this today. My reply was:

"Personally I wouldn't bother with mail support for pre-WM5 devices;
it's a mess to reverse engineer, IIRC"

But I'm a lazy dude with other projects, so don't pay to much attention
to the above... :-)

If you really want to dig into this, I suggest you look att Microsoft's
MAPI (Messaging Application Programming Interface) because it's quite
probable that the data transmitted via RRAC is a serialization of the
data used by the IMessage et.al COM objects.

Good luck!

\David

Sven Oliver Moll

2006-08-08 22:10:32 UTC

Permalink

Post by David Eriksson
Thanks for the kind words!

I got it running one step at a time, and had the advantage of using a rather
old device. Toughest part was finding the useful packages in the Ubuntu
repository. ;-)

[...]

Post by David Eriksson

Post by Sven Oliver Moll
Based upon pls and pcp I managed to build myself a small fetch-updated-
documents tool that also converts from Pocket Word to RTF.
(I'm just changing the header from {\pwd2 to {\rtf1, but it works so that
I can finish the RTFs with OpenOffice.)

See also http://synce.sourceforge.net/synce/task_pwi.php and
http://www.synce.org/index.php/ISF is probably used for the
hand-drawings.

That's not what I needed. I just wanted to convert the ".pwd" files to
".rtf" and that's just replacing the reader as mentioned above. ".pwi" is
not important to me.

Post by David Eriksson
Please keep us posted with your progress!

I've done a bit cleaning up on the code and made some features optional.
The result is included as an attachment. It's not intended to be suitable
for a full backup, rather an one-way substitute for MS's synchonisation
folder.

Post by David Eriksson

Post by Sven Oliver Moll
Once that's done, I was thinking about accessing the Pocket Outlook Inbox
ActiveSync folders via rra. Playing around with the rra-tools, I managed

[...]

Post by David Eriksson

Post by Sven Oliver Moll
a resource on how to interpret the binary data around it? It doesn't
matter, if it's a format description or some sourcecode.

[...]

Post by David Eriksson
If you really want to dig into this, I suggest you look att Microsoft's
MAPI (Messaging Application Programming Interface) because it's quite
probable that the data transmitted via RRAC is a serialization of the
data used by the IMessage et.al COM objects.

Actually rra-decode, which I 'discovered' five minutes after my first post,
did me a LOT of favors. Only thing it can't handle are "Multi Strings",
they are shown as blobs. But the format is simple (same is used in the
registry):
"String1\0"
"String2\0"
[..some more..]
"\0\0" <- endmarker

Most other stuff should be solveable, since only very few 'magic' numbers
are used. Biggest question to me right now is: how is stored in which folder
(deleted, sent, outbox, inbox) the message belongs?

As far as I'm concerned, the final result of these efforts are two small
tools:
- One that pulls out the messages from Pocket Outlook's Outbox in a format
that can be forwarded to an smtp server, or if that's easier to provide,
sends them out themself.
- One that stuffs /var/spool/mail/$USER into Pocket Outlook's Inbox.
Of cause there are some problems to solve, not only CE related ones. For
example, I don't want to access the mailfile in a raw mode, but with a
library that helps me dissolving that large text file.

For analysing the Format of the messages, I'll put up a case where all four
folders of PO's "ActiveSync" will contain several messages, dump these
and put it up somewhere (maybe the wiki?) for discussion.

Anyway, bedtime now!
SvOlli

Sven Oliver Moll

2006-08-21 18:22:43 UTC

Permalink

I thought there might be someone here who would like to get an update...

Post by David Eriksson

"Personally I wouldn't bother with mail support for pre-WM5 devices;
it's a mess to reverse engineer, IIRC"

This statement seems to be half true from what I've encountered so far.
I've got three WinCE devices: a Handheld PC 3.0, a Handheld PC 2000 and
a Pocket PC 2003 Second Edition (my cellphone).

On the Pocket PC reverse engineering the Merlin Mail data seems to be no
fun at all.

But on the Handheld PCs it doesn't seem to be such a big hassle to get
the data of the emails. I've written a small tool that checks for the
changed mails and prints the information about the mails found to stdout.
The output looks something like this:

type_id: 00002713, object_id: 07001f6d, field count: 13
ID: 0x0000000e
Flags: 0x0250fdfd
MsgLen: 0x0000000d
Totallen: 0x0000000d
BodyLen: 0x000d
Date: Thu, 10 Aug 2006 22:05:22 +0200 (1155240322)
From: ""
Presubject: ""
Subject: "Outbox 1"
Header: "To"="<***@example.com>"
Header: "Cc"=""
Header: "Bcc"=""
Body: "Outbox 1 Text"
Svcoid: 0x05001e73

The source is available on request, if anyone would like to join in. It's
also lacking a lot of functionality, but it's a start. What's been lacking
is (probably amonst other stuff, but these are the next topics):
- handling of attachments, it seems that I've got to pull them out of the
filesystem
- handling of mailbodys > 0x7fe0 characters, since they don't fit in the
database and will be stored on the filesystem as well
- putting these data together in structure for easy processing

I'm not sure on how to get the pathnames from the CE device for filesystem
access. First attempt would be to read the parts from the registry and put
them together myself, if noone would come up with a more elegant way of
doing this, that is.

Greetings from Germany,
SvOlli