Email Migration Saga and Solutions
August 5th, 2009 at 2:31 pm
Email: We take it for granted until the day it threatens to leave us. Anyone who has ever lost messages, or even just experienced a temporary power or internet outage knows this. Unfortunately, due to a few factors, I am also all too familiar.
This is a narrative tale of woe and triumph, but also contains advice for those who may be experiencing the nightmare of importing messages to Entourage from Apple’s Mail (and possibly other email clients, particularly those who use mbox files for storing messages). So read on!
Our tale starts long ago, before I even worked at this institution. Employees have always, as far back as I’ve ever known, had simple POP email access. No IMAP, no Exchange, just good ol’ squirrel mail on the web and POP-ing into the client of your choice. A few years back the topic of converting to Exchange came up, though without any timeline discussed. Since that was a possibility and Eudora (ack) was the current client of choice, we all went to Outlook 2003 at the time – around 2005 or so I believe? Mind you, we only used Outlook as a client, still just with POP.
Time chugs along and I eventually get a shiny new MacBook Pro in 2007 to replace my aging Dell desktop. With a new computer and OS came a new choice of email client. We still weren’t on Exchange (look at that non-timeline!) and I knew “Mac people” whined about Entourage, so Mac Mail / Mail.app ended up being my plan. However, at least at the time, there were major issues getting messages from Outlook into Mail. Since it was so long ago I can’t even tell you what all I tried and what all failed. I have the feeling there was a Thunderbird somewhere in there, a million executions of export and import commands in various programs, and maybe more. I do know what finally worked, though, as its influence still shows on my emails today… More on that later. It was this little $10 program, Outlook2Mac. Yes, out of desperation I went with a pay solution. It was a good deal though, the product worked for me when nothing else would, no question.
Let’s skip ahead through nearly two years of happily using Mail. Finally the institution is ready for the great Exchange switchover. This means in lieu of 10.6 Snow Leopard’s Mail Exchange support I had to get ready for Entourage. I was optimistic, it couldn’t be as bad as Mac fanboys made it out to be! So I sat down with our Help Desk manager and we began to import my messages – all 12,000+ of them (heh heh). Things were going smoothly at first, other than lots of waiting for import and sync processes. But then I noticed something weird: about 1/2 of my messages had a received date of (drumroll) the date of import. And that just won’t do. There’s no way I could find anything in that mess!
After poking at the message list with our Help Desk guy for a little bit, I realized what the pattern was. All of the messages I imported from Outlook to Mail in 2007 had that day’s (the import) date as the received date. It was past time to go home for the day so we sighed and decided to attack it the next day.
First thing the next morning, I realized a couple of things:
- I can show the “Sent” column in Entourage rather than the “Received” column and sort by that in order to get the messages to behave and appear in their correct date order.
- However, when they sync with the Exchange server, it does grab the incorrect “Received” date, so when using web access the column sort trick is totally pointless.
- The headers of these Outlook -> Mail messages contained far less information than traditional headers which was probably the problem. Outlook2Mac seems to have stripped off much of that information, leaving only a “Date” field, not a full path of transit, as it were.
Perhaps the most painful part is that I had backups of my Outlook messages in several formats up until a month or so ago, when I said to myself, “ah, my email works fine, I can get rid of these!” Otherwise I would have tried to import them to Entourage in like .pst or some other format. *sigh*
So instead I talked to our Exchange admin to see if he had any ideas. He found and sent over an Applescript called CorrectTheDate. I didn’t even really know that such things existed – editable scripts you can run against your messages? Cool. Windows would never do that for me. I found that this script, however, did not work. We tried editing it slightly, which only resulted in it setting a couple of message’s dates to 2012. Not so helpful.
Well, the Exchange admin went on vacation, and he probably would have gotten sick of me anyway, so I decided to follow the scripting lead and see what I could do. Bear in mind I know some PHP but have never worked with Applescript at all, and wasn’t really prepared to train myself on the entire language just to fix my emails. Therefore I mostly geared my web searches towards pre-written scripts or just descriptions of similar problems, hoping someone would have made a solution already. I was about running out of steam – turning up lots of results about similar problems but no solutions that would work for me – when I found another script. This one is called Modify Date Received and, when applied to a message, prompts you to enter a new date and time for the message to appear to be sent at. I tested this on a message and indeed, it worked. But there was no way I was going through and changing the dates manually on 6000 messages.
So I got an idea: mash the scripts together! Perhaps I could do this without really having to know Applescript! See, CorrectTheDate is good at looping through multiple messages and (maybe) finding out what the date should be from the “Date” field in the header, what Entourage calls the “Sent” date. And Modify Date Received is good at changing the correct dang field! After an hour or so of fiddling with the scripts, inserting “display” commands (the equivalent of echo, print or more like alert in Javascript) at every turn, and admittedly, learning a bit of Applescript in the process, I had a really rough thing that was… drumroll… working! I was able to select big groups of badly dated messages and fix them, and the script doesn’t take too long to execute even on big batches. As always, though, it then took a while to sync those changes back to Exchange.
For those who are interested/plagued by the same problem, you can download my hacked I-don’t-know-what-I’m-doing code in a minute, after disclaimers:
- I don’t know what I’m doing. I know it’s not the best way to code this, but don’t throw eggs please!
- It worked for me, it may not work for you. No guarantees.
- Code heavily taken from CorrectTheDate and Modify Date Received. Thanks to those coders!
If you’re going to use this, place it in your Entourage Script Menu Items folder, usually in Documents/Microsoft User Data. Please run this script on the messages BEFORE you sync with the Exchange server (as in, either while not connected to the Internet or run it on the folders in On My Computer > Mail Import before moving the imported folders to the Exchange account area).
Download Emily’s messy but working Entourage “Received Date” fix script
If you used this script or experienced similar problems/solutions, please comment!
Tags: applescript, email, entourage, exchange, work
Word count: 1,277