This post is the ultimate, mobile-site-friendly, migration guide from Outlook to Thunderbird. If you want the short version, go immediately to the section To wrap it all up. The author describes in detail the design flaws in Microsoft’s Outlook and how these shortcomings are avoided in Thunderbird. If you need a Windows program to handle a large number of email messages coming from various accounts the open source and free Thunderbird should be your choice. With this guide Outlook will be history.
Table of content
For scientists communicating and connecting to the outside world through email is the most important communication channel. My email messages are also my to-do list. Dealing with my email messages has been a struggle for the last 10 years. I receive typically 30 to 40 messages a day. These numbers are quite common for scientists, I guess. Here are my requirements for a digital solution to my problem:
- I want to be able to store my messages in a hierarchical, collapsible tree. My set of messages contains in addition to recent messages, all access details to Internet sites where I have an account, all my software licenses, and all other email messages that contain information that I have to or want to keep. I am quite bad in throwing emails away and still keep 75% of all messages I receive, going back more than 10 years. On a number of occasions – including in a legal question – this storage mania of mine came in quite handy. I want my whole email archive to be arranged and stored in a systematic way, like we do in a filing cabinet, but then with folders in folders in folders etc.. I want to have all of the stored items available and visible within three or four mouse clicks. As these mails contain for instance a number of passwords I do not want to them to be retrievable when I check my email from a public computer, such as present at scientific conferences. Key logging software is too easily installed there.
- I want to be able to search all of those messages with a convenient search engine. This engine does not necessarily need to be part of the email client. As a matter of fact Google Desktop did a wonderful job. So good that Google decided to abandon it. To develop a general program searching through email messages is difficult as the various email clients like Outlook, do not store their messages in an ASCII formatless text file. I wrote an article why this is really bad practice.
- I want to be able to retrieve any piece of information from my email store when I do not have a LAN or Internet connection.
- I want the collection of emails to be up to date and synchronized from wherever in the world I am checking and handling my mail.
- I would like a solution where I can use a mobile phone app to peek at (not necessarily download) the most recent email messages and send an occasional message or reply
Up to a few months ago I used Outlook, partly dictated by the fact that my various employers used a Microsoft Exchange Server
for their mail server, and up to a few years ago there were no acceptable web interfaces for this server.
Why not using Gmail?
Many of my colleagues manage all their email traffic from Gmail. Gmail certainly satisfies my criterion 4 (world-wide synchronization) and criterion 3 (searchable) and obviously satisfies criterion 6 (mobile-phone app) as could be expected from the driving force behind the Android
operating system. But it fails dramatically on my most important criterion 1: a collapsible navigation tree. Google will never ever introduce such a navigation three because it violates Google’s philosopy and raison d’être. The search giant has become big with this philosopy: “Dump anything digital everything you got in a structureless container, and our search machine will find it. Google cannot afford a navigation tree.
Gmail fails also on requirement 3 (no need of Internet connection). One could make a local copy of emails stored at the server, but as this is most conveniently done with an email client on the local computer this solution begs the question.
Using a local client like Outlook or Thunderbird
For the last ten years I used Outlook as local email client and saved my email messages, as pst files
, on a fast USB drive, a drive that I take with me wherever I go (In my case a Western Digital
500 GB 100$ drive). Everyday I have to synchronize my files at most four times: (i) in the morning at work, (ii) at work just before I leave for home, (iii) early in the evening at home and (iv) late at home or early in the morning before I go to work. To be a success this synchronization process requires discipline, but with a simple utility as Beyond Compare
2 (its newer version Beyond Compare 3 is not nearly as good) or Microsoft’s free command-line utility Robocopy (download here
, it contains a good help command) this task becomes light. Maintaining this discipline is not difficult for me as I have to synchronize a lot of other stuff as well, like edited scientific papers, grant proposals and presentations.
On all my computers I have a made a data-only partition (drive d:) that is also fully mapped on my USB disk. The synchronization procedure could be totally automated, but I like to see what files are going to be refreshed before I overwrite them.
Whatever email program one uses in this approach one has to carry all synchronized data on a disk wherever one goes. When going abroad a more convenient solution, without bringing the USB disk will be described later here. Also further on in this article I will indicate that using the cloud, like Dropbox
, is not a good solution for parking your email archive.
Problems with Outlook as email client
Outlook suffers from the Microsoft philosophy and as a result is built to optimize incompatibility with non-Microsoft products and is built to maximize the hurdle for non-Microsoft developers to produce useful add-ons to Outlook. Writing add-ons for any Microsoft program requires the complexity of Microsoft-only COM
objects. Another way is to write snail’s pace macros with the hopelessly unstructured, outdated Visual Basic
language. Both type of solutions make Outlook slower.In the old days an email client as the free Eudora
saved its emails in a large number of flat text files. Such a design choice is a pleasure from the point of view of synchronization. A few new emails will only lead to a few new or changed files to be synchronized.
Surprise, surprise Microsoft has chosen for a different design. Its design choice is a logical consequence of its closed-shop philosophy, but is steadily getting a major inconvenience in the era of the cloud computing. Outlook saves all its emails in large database files, so-called pst files, files of a very complicated structure. Such a file can easily become of gigabytes size. To keep the pst files manageable the user can create his own pst files, with default name “Personal Folder” showing itself in the navigation tree of Outlook and where one can drag mail folders and mail messages. Here comes the weird Microsoft design: if you open Outlook and not even downloads or send any email message all the pst files of that Outlook profile are changed and get a new file modification time. This is a synchronization nightmare: everyday you have to synchronize a few gigabytes. In my case my social life started to suffer from it. I had to synchronize for minutes and minutes per synchronization cycle, while colleagues waiting in my office to take me out for dinner or while my my wife was waiting in the car wondering what was keeping me. Outlook was keeping me.
The pst files of Outlook are so complex and so fragile that they get easily corrupted. Microsoft realizes this shortcoming and supplies a repair utility for pst files scanpst.exe, shipping standard with the Microsoft Office suite. I had to use this repair facility at least once per month as Outlook warned me regularly that a certain pst file was corrupt and could not be used any longer.
There are quite a number of other less important inconveniences with Outlook. It is quite complex to change the view of newly created folders (Change Current View) . It is quite difficult to change defaults, like showing number of items in a folder rather than showing the number of unread items. I wrote a Visual Basic
macro to accomplish this. What normal user masters Visual Basic? Another shortcoming is that there is no connection to the favorite folder list and the navigation tree (there is in Thunderbird). Outlook has the annoying habit of not storing the email addresses of people it knows the email address of. It just stores the name of that person. This gives problems when you forward a mail under the condition that the address book is not available on the computer you work or hen you want to unsubscribe and you cannot find out to which of your accounts the mail was sent..
Start-up times of Outlook were very long, getting longer with each newer version and getting longer due to the growth of my pst files.
Problems with Microsoft Exchange Server
Amusing in the battle with the pst files is that Microsoft itself acknowledges that the structure of its pst files makes synchronization a disaster. Even if you and your employer or other provider are Microsoft-abiding customers using on the server side Microsoft Exchange Server and on the client side Outlook. If you really let the Microsoft server interact directly with your computer your whole computer’s bandwidth will be taken by an unstoppable synchronization process: the server is continuously trying to send these gigabyte over your Internet connection. This is not very pleasant when you are in a hotel with your laptop fitted with a slow connection and trying to find on the Internet the address of a garage to repair your car.
Microsoft has found the solution: cache mode. In this mode Outlook tells the server basically to shut up. Outlook stores all the data on your local computer, on a file of the type .ost which of course can also become huge. You better be sure that before you shut down your computer and you want to continue to work somewhere else you switch off cache mode. Otherwise you are in big trouble. One solution is not get it out of cache mode before you shut down but synchronizes the ost file as well. But even so you can have big problems. As Microsoft is quite good in inventing new file structures that easily get corrupted the company also supplies a repair tool for the ost files: scanost.exe. If you are able to master all this you can apply for a vacancy in an IT department.
For the above reasons in this section I always configure Exchange Servers that I have to use in a downgrade mode: that of a the good old POP
server (it means just adding your account as a POP server and not as an Exchange Server in the setting up of the account in Outlook. Then and all my mails end up in a pst file. In such a solution there are no ost files. The profile of Outlook is configured to store its pst data files on the USB disk directly. In case I forgot to synchronize there is no harm done as the USB disk always has the newest data. Great, the IT supporting people continuously were warning me against using the Exchange Server in POP mode, but I knew what I did and it always worked perfectly. My point here is that the necessity of using a server in cache mode demonstrates the failure of the pst file model of Microsoft.
Why not everything on server of employer?
A convenient solution could be to keep all your emails on the Exchange Server of your provider. This is a cloud solution avant la lettre. The first problem that arises in such an approach is what do you do with your email messages of other accounts. If you also let them sitting with their server you will end up with the chaos of having to check different mail servers manually. You will have to choose one “master” account and redirect all your mails from the other accounts to the master account. The redirection will work fine as long as you never log in tot he non-master servers to prevent ending up with massive numbers of duplicate mails or the disaster of lost mails.
A serious complication is that “master” mail server providers always limit the storage capacity you are allowed to keep. This constraint might go away in the future. The decisive factor in my case not using this solution is that web interfaces do not allow to creation of personal folders. This means that you end up with one unmanageable fat navigation branch, rather than a rich tree with many roots. And in case of lost emails I must get into contact with the IT department of my employer. Be my guest.
Why not Outlook in the Cloud?
An ideal seems to have one’s email in the cloud. Forced by the competition Microsoft is now offering all its office applications in the cloud
, thereby robbing quite a number of its certified partners from its clients. I studied the solution in quite detail. First of all it is not very expensive. On the order of 10$/month for the lite version. Email works through the Outlook web interface, so that is fine. One can create all kinds of personal folders, so that is also fine. The big problem occurs when you decide you want to have some local copy a mail messages. Then the cloud server of Microsoft sends the whole pst file again. At that time is best you ask your colleagues for a coffee break.
Enter Thunderbird and goodbye Outlooks
After all this grieve with Outlook I decided to try Thunderbird. I downloaded
and installed. The normal procedure of account details. My first positive experience: Thunderbird allows to store email in a navigation tree. Fantastic.
Thunderbird allows you tell it where to store its emails. This location is called to profile location. As soon as possible you must set this location to a suitable empty directory on your USB disk. Close Thunderbird and find the Thunderbird file profiles.ini. Use search to find it. It is somewhere in c:\Users\your_username\AppData \Roaming\Thunderbird . The content looks like like:
The Path variable tells Thunderird where during start up it can find the email tree. The drive “g:” denotes my USB disk. In your case you must put your location of where you want to store your emails on your USB disk. Because you always want to synchronize the email tree to your desktop, necessary in case of a mishap, you also must make a similar directory on your hard disk. The more the two directies have comparable names the easier the synchronization. In my case their names are identical except for the name of the drive.
What to do with all my email history in the form Outlook pst files that I want to import in Thunderbird. Next task is to import all these pst files from the Outlook profile.
Importing pst files into Thunderbird
Thunderbird has an Outlook import facility. If you have more profiles for Outlook Thunderbird will only import the default profile. The following steps will import all your pst files flawlessly.
- Configure Outlook to start up automatically in the default profiles.
- Configure Outlook not to send or receive any email. You might even disconnect the Internet.
- Run the scanpst utility on all the pst files. You must do this.
- Open Outlook and move all files out of the inbox account file into your pst files. As a result all your default pst file (that is the first root in the navigation bar) is empty
- Close all pst files except the default account . Open one pst file and close Outlook. The procedure of importing them one by one is safe and in case of error you know exactly what pst is not (fully) imported.
- Use Thunderbird to import Outlook pst files. It will import two files, the default file (containing the Inbox) and the other pst files you had opened. You will get an error of your pst file has a very deep tree (7 levels deep or so). If you get this error you have to go back to Outlook and move some items to make the tree less deep.
- Thunderbird will now show the two imported files. Rename them or put them in a local folder you like.
- Repeat step 5 to 7 for all pst files you want to import in Thunderbird. That is to say you open Outlook, close the pst file you just imported and open the other pst file you need
You could import more pst files simultaneously but it gives easily rise to import errors (you are importing gigabyte files)
Add-ons make Thunderbird the best email client
Thunderbird is open source and has a css-oriented approach to its looks. That is to say on start-up Thunderbird loads its css files. Developers can easily make add-ons and tailor how they look. I found the following add-ons very useful:
- Silvermel: this add-on is a skin for Thunderbird and changes the whole look and feel. Superb skin.
- Remove Duplicate Messages (Alternate) delete replicates. When your employer uses IMAP servers or your pop servers are are leaving messages on the server you end up with duplicate messages. With this add-on one click removes them from a folder
- Extra Folder Columns I always want to know how many items a folder contains. With this add-on this is possible
- QuickFolders: this add-on really great. It enables a toolbar with favorite folders, with drag and drag and drop and sub-folder display. Absolute superior to Outlook’s favorites list.
At first I found the looks of Thunderbird a little old-fashioned, but the Silvermel skin changed Thunderbird in a beautifully looking application. Together with all the add-ons described above I come to the conclusion that Thunderbird looks as nice as Outlook, has a much better User Interface. How about the synchronization speed?
Thunderbird saves its emails in a large number of files – not a file for each message because that would slow down the program too much, but the fragmentation is good enough to speed up synchronization tremendously. Synchronization of all my emails, again gigabytes, now takes 10 seconds. Ten to twenty times faster than Outlook.
Checking recent email without your PC.
When you go on a trip you might find it inconvenient to use your USB disk when your are sitting with your laptop somewhere in the back of a conference room. There is an easy solution to this.
- Right before you leave home synchronize the USB disk with your laptop.
- Edit the profiles.ini file and change the location to the location where you keep the copy of your emails on your hard disk. Thunderbird will now start without using the USB disk
- When you return home, the first thing you do is to synchronize by updating the USB disk and change back in the profiles.ini file the location of your email archive to the location on your USB disk.
To simplify the editing of profiles.ini you could put a shortcut on the desktop pointing to the ini file. Even better is to make two batch files (.bat) that do changing of the path variable automatically and then one mouse click is sufficient.
Using your smart phone
For this we need a Gmail account that we we will only use for occasional reading the most recent messages and for sending an occasional emergency reply. As we want to minimize traffic to our Gmail account, you have to configure the “reply-to” address of my Gmail to one of my other accounts. People that reply to your Gmail message will not send that reply to your Gmail account.
We configure all our mail accounts through the web-mail interface of our providers, except our Gmail account, to redirect a copy of all incoming messages on the server to our Gmail account. In case of a Microsoft Exchange Server you better set this redirection filter while running Outlook (so do not uninstall Outlook).
For the real work we use Thunderbird as email client. We supply Thunderbird with all our account data, except my Gmail account. All incoming fetched mails are filtered – using Thunderbird filters – to automatically arrive in one Inbox. We also tell Thunderbird to send a copy of each sent email to our GMail account.
Some emails are necessarily directly sent to a Google email account, for instance Google Calendar remainder emails. On the web interface of Gmail I filter these message to send a copy (redirection is unfortunately not possible) to one of my email accounts. Redirection and forwarding can give rise to infinite loops but Gmail and Thunderbird recognizes these and stop any loop.
Now we end up with a situation that all emails, received or sent, are registered by Thunderbird as well as Gmail.
I activate my Gmail account on my Android phone. Now when I do not have my laptop available, but want to check my recent messages I check my Gmail app. Regularly I have to clean (meaning delete) my Google inbox. I only send emergency emails from there and then always with sending in addition a copy to one of my other accounts (Google does not let you automate this). In this way Thunderbird will always tell me what I did with my phone. If you know you will need some important email messages while you are at a conference, you keep them on your Gmail account.
To wrap it all up
Here I summarize the steps to take to migrate form Outlok to Thunderbird.
- If you do not have a Gmail account get one, or get an equivalent one.
- Use Thunderbird as email client with a number of useful add-ons and the Chameleon skin. All your mail accounts, except your Gmail account, are included in the accounts managed by Thunderbird. All incoming mails are filtered within Thunderbird to automatically arrive in one Inbox. Of all sent email messages a copy is sent to your Gmail account.
- At the web interface of your employer or professional relation you should set filters to redirect any incoming message to your Gmail account.
- Some emails are necessarily sent to your Google email account, for instance Google Calendar’s reminder emails. On the web interface of Gmail I filter these message to send a copy (redirection is unfortunately not possible) to one of your other accounts managed by Thunderbird.
- You make two batch files of which you set shortcuts on your desktop to switch from a Thunderbird profile on your USB disk (standard situation) to a profile residing locally on the computer. This trick is useful at conferences, but beware when you return home and do not forget to synchronize from the laptop to USB disk and do not forget to switch the profile back to the USB disk.
- Activate your Gmail account on your Android phone or iPhone.
If you have both your computer and your USB disk connected, you have everything to manage all of your email business. If there is no Internet connection Thunderbird will wait fetching and sending messages until the connection becomes available. Not having synchronized your emails for a view days is not a problem as everything is run from the disk. But do not wait too long to get them back into sync.
You can check all your recent emails on your mobile phone and even send mails (do not forget to cc. yourself to a non-Gmail account as otherwise Thunderbird will never know you sent that mail. Clean your Gmail Inbox regularly.
Some technical terms are explained here.
- mail server: a computer program run on a computer of the Internet Service Provider (ISP). The ISP could be your employer or a commercial provider. The mail server receives mail messages from the Internet meant for its domain and stores them until the local recipient fetches the message. In addition the mail server is responsible for relaying email messages sent by a local user in its domain to the Internet.
- Exchange Server: This is a commercial mail server program developed and maintained by Microsoft and a possible option for the ISP to run as a mail server on his computer.
- email client: a program that runs an a local personal computer, examples are Thunderbird and Outlook. It allows the user to get his emails from the mail server and send emails to the mail server that are to be relayed to the domain of the recipient.
- web-mail client: this is a website of the ISP dedicated to allowing the user, if he navigates with his browser to this site, to communicate directly with the mail server of the ISP. Normally this communications is taken care of by the email client in an invisible way for the user. A user can read and compose and send his email messages through the browser. No action is taking place at the site of the computer. Email messages remain stored on the mail server of the ISP, unless later the user starts an email client on his computer to fetch the mails from the mail server. Traditional examples of web-mail interfaces are those of Hotmail and Yahoo.
- pst file: this is the file format used by Microsoft to store email messages. The internal structure of this file is that of a full database. The ost files, also invented by Microsoft, are used to mirror the email messages on the Exchange Server. The pst and ost files are closely related and can be converted into each other.
- redirection: an action where a received email message is sent unaltered, that is with the same recipients and cc.’s and content and attachments, to a new email address. The new recipient will not notice this redirection – unless he checks the full message headers, something 99% of the people will never do. Redirection can be set at the mail server or at the email client, but as redirection is usually set once and for all, redirection set-up is preferably done at the mail server of your ISP.
- forwarding: an action where a new email message is sent to a new recipient, containing in the body the content of the forwarded email message, including – as text – the original recipients, cc.’s and original sender, and with as attachments the original attachments.
The Mozilla foundation is a not-for-profit organization developing open source software. Their flagship product is the browser Firefox.
The developers of the add-ons Silvermel and of QuickFolders will get each $7.50. If you also move from Outlook to Thunderbird I would be very happy if you would also donate to these developers.
- - - - - -
If you like this post why don't you email
to our new posts. Or subscribe to our