Moving mail from another server

When people move to Nixt Server, their old mail can come with them. versealx-server migrate imap signs in to the old server as the person, reads every folder, and copies the mail into their mailbox here — with the folders, the read and flagged marks, and the date each message originally arrived.

It works with any server that offers IMAP: Dovecot, Exchange and Microsoft 365 with IMAP enabled, Gmail, Zimbra, cPanel hosts and the rest. From Microsoft 365 and Google Workspace you can also move mail without anybody’s password: see From Microsoft 365 or Google Workspace.

This page starts with one mailbox. To move many people together from the console, see Moving everybody at once.

Before you start

• The account must already exist here. See Domains and accounts.
• You need the person’s password on the old server, or an app password if the old server requires one for IMAP.
• Run the command on the Nixt Server machine, with the same configuration file the server uses. The server can keep running while you migrate.

Copy one mailbox

Put the old password in a file only you can read:

install -m 600 /dev/null /root/old-password
nano /root/old-password

Then run:

sudo -u versealx versealx-server migrate imap alex@example.com \
  --from imap.oldhost.example \
  --password-file /root/old-password

alex@example.com from alex@example.com@imap.oldhost.example:993
  folders: 6 seen, 6 complete
  messages: 4812 copied (1893442011 bytes), 0 already here, 0 failed, 0 gone at the source
  the source password is destroyed

Delete the password file when you are done. The command never keeps it: it is read, used, and overwritten in memory when the copy ends.

The password is never accepted as a command-line argument: anyone on the machine could read it from the process list.

Where folders go

• Inbox goes to the inbox.
• Sent, Drafts, Trash, Junk and Archive go to the matching folder here, whatever the old server calls them — Sent Items, Gesendete Elemente or [Gmail]/Sent Mail all land in Sent, as long as the old server marks them as such, which current servers do.
• Every other folder keeps its name and its place in the hierarchy.

From Gmail

Gmail shows every label as a folder and keeps a copy of every message in All Mail. Leave that out, or everything arrives twice:

sudo -u versealx versealx-server migrate imap alex@example.com \
  --from imap.gmail.com \
  --user alex@gmail.com \
  --password-file /root/old-password \
  --except "[Gmail]/All Mail"

Gmail needs an app password for this, not the account password.

If it stops part-way

Run exactly the same command again. It carries on from where it stopped, and it never copies a message twice — even if the stop happened in the middle of a message. A large mailbox over a slow connection can take several runs, and that is fine.

The same is true if people keep using the old server while you migrate: run the command again just before you switch over, and it copies only what arrived since.

If the old server renames or rebuilds a folder in the meantime, the messages already copied are recognised and not copied again.

Reading the result

After migrating

• People see the copied mail straight away in apps that are open; an app that was idle picks it up the next time it checks.
• Once everyone is copied, point your MX records at this server and run each migration one last time to pick up anything that arrived in between.

From Microsoft 365 or Google Workspace

When your organisation is leaving Microsoft 365 or Google Workspace, you do not need each person’s password. The organisation’s administrator there gives Nixt Server read access to the mailboxes once, and you then move each person’s mail with one command. The mail keeps its folders, its read and flagged marks, and the date each message arrived. Their contacts and calendars can come too.

As with IMAP, the account must already exist here, and you run the command on the Nixt Server machine while the server keeps running.

Microsoft 365

In the Microsoft Entra admin center, a Microsoft 365 administrator:

1. Opens App registrations and chooses New registration. Any name will do, such as Nixt Server migration.
2. On the new registration’s API permissions, adds Microsoft Graph › Application permissions › Mail.Read, Contacts.Read to bring people’s contacts too, and Calendars.Read to bring their calendars, then chooses Grant admin consent.
3. On Certificates & secrets, creates a New client secret and copies its value.
4. Notes the Application (client) ID and the Directory (tenant) ID from the registration’s overview.

Put the secret in a file only you can read, then run:

sudo -u versealx versealx-server migrate m365 alex@example.com \
  --tenant contoso.onmicrosoft.com \
  --client-id 0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0 \
  --secret-file /root/m365-secret

alex@example.com from alex@example.com at Microsoft 365
  folders: 9 seen, 9 complete
  messages: 6120 copied (2210458331 bytes), 0 already here, 0 failed
  contacts: 340 into the address book Contacts
  calendars: 2215 events into the calendar Calendar
  the app registration's secret is destroyed

Every folder comes across with its subfolders. The Inbox, Sent Items, Drafts, Deleted Items, Junk Email and Archive go into this account’s own folders of that kind.

With Contacts.Read, the person’s contacts come after their mail: the contacts in Outlook’s own Contacts go into their address book called Contacts, joining any contacts already there, and each contact folder becomes an address book of its name. Without it, the result says the contacts were not read, and the mail moves all the same.

With Calendars.Read, the calendars the person owns come after their contacts: their Outlook calendar goes into their default calendar here, and each other calendar of theirs into a calendar of the same name. Calendars other people shared with them stay behind: they come with their owners. Each event keeps the time zone it was made in, so a weekly meeting at ten stays at ten when the clocks change, and its repeats, changed and cancelled occurrences, attendees and rooms and their answers, reminder, categories, and whether it is private or shows as free. Moving an event invites nobody again. Without the permission, the result says the calendars were not read, and the mail moves all the same.

Google Workspace

In the Google Cloud console, with a project of your organisation’s:

1. Enable the Gmail API, the People API to bring people’s contacts too, and the Google Calendar API to bring their calendars.
2. Under IAM & Admin › Service accounts, create a service account, then on its Keys tab add a key of type JSON. The key file downloads to your computer; copy it to the Nixt Server machine where only you can read it.
3. Note the service account’s Unique ID (its client ID).

Then in the Google Admin console, a super administrator opens Security › Access and data control › API controls › Manage Domain Wide Delegation, chooses Add new, and gives the client ID this scope:

https://www.googleapis.com/auth/gmail.readonly

To bring people’s contacts and calendars too, give all three, separated by commas:

https://www.googleapis.com/auth/gmail.readonly,https://www.googleapis.com/auth/contacts.readonly,https://www.googleapis.com/auth/calendar.readonly

Then run:

sudo -u versealx versealx-server migrate google alex@example.com \
  --key-file /root/google-key.json

alex@example.com from alex@example.com at Google
  folders: 7 seen, 7 complete
  messages: 5304 copied (1604377812 bytes), 0 already here, 0 failed
  contacts: 212 into the address book Contacts
  calendars: 1480 events into the calendar alex@example.com
  calendars: 36 events into the calendar Team offsite
  the service account's key is destroyed

Gmail has labels rather than folders, and here each label becomes a folder. A message with several labels is in each of those folders, but its space is counted only once. The inbox, Sent, Drafts, Spam and Trash go into this account’s own folders of that kind; mail that was archived in Gmail, with no label, goes into Archive. Unread and starred mail stays that way, and Gmail’s categories (Social, Promotions, Updates and Forums) come across as keywords that apps and filters can use. Chats are not mail and stay behind.

With the contacts scope, the person’s contacts come after their mail, into their address book called Contacts, joining any contacts already there. Each contact group they made comes as a group of the same name. Without the scope, the result says the contacts were not read, and the mail moves all the same.

With the calendar scope, the calendars the person owns come after their contacts. Their main Google calendar goes into their default calendar here, and each of their other calendars into a calendar of the same name. Calendars other people shared with them stay behind: they come with their owners. Each event keeps its time zone, so a weekly meeting at ten stays at ten when the clocks change, and its repeats, changed and cancelled occurrences, guests and their answers, reminders, and whether it is private or shows as free. Moving an event invites nobody again. Birthdays Google makes from contacts, and working-location entries, stay behind. Without the scope, the result says the calendars were not read, and the mail moves all the same.

To move many people at once, start one migration from the console instead: see From Microsoft 365 or Google, for everybody.

Carrying on, and the final pass

Both work like an IMAP migration. If a run stops, run the same command again: it carries on and copies nothing twice. Once you have pointed your MX records here, run it once more for each person, and it copies only what arrived since the last run. When Microsoft or Google asks it to slow down, it waits and carries on.

Calendars work the same way. A later run brings only the events added or changed since the last one, including a single meeting of a repeating series moved or cancelled, and removes here the events deleted there. The result says how many events each calendar added and removed. If an event was changed on both sides in between, the old service’s version wins. When Google no longer recognises where the last run stopped, which it does for runs far apart, that calendar is read again from the start, and nothing is duplicated.

The secret and the key file are never kept: each is read when the command starts and overwritten in memory when it ends. Once everybody has moved, delete the secret at Microsoft, or the service account’s key and its delegation at Google.

Importing mail from files

Old mail often lives in files rather than on a server: an Outlook data file (PST), a Thunderbird profile, an export from Apple Mail, or the Maildir of a server that has been switched off. An administrator uploads a PST or an MBOX file from the console, and the server’s operator imports files that are already on the server machine, including Maildirs.

From the console

An administrator can import a PST or MBOX file without access to the server machine. Open the person’s page in the console, find Import mail, choose the file, and name the folder it goes into. The file is uploaded from your browser; keep the page open until the upload finishes. The server then imports it in the background, and the mail appears in the folder as it is imported. The card shows how far each import has got, and what became of its messages. Cancel stops an import that has not finished; what was already imported stays.

From the command line, the same upload is:

vsx admin mailbox import ada@example.com archive.mbox --into "Imported 2019"
vsx admin mailbox imports ada@example.com

A file uploaded this way can be up to 4 GB, and one import runs for a mailbox at a time. Until it is imported, the uploaded file is kept encrypted with the organisation’s key, and it is deleted once the import finishes or is cancelled. An upload that is never finished is dropped after a day.

The organisation’s administrators can import for anybody, and a domain’s administrators for the people in their domains. An auditor sees the imports. A custom role can be given Import somebody’s old mail from a file. Making, starting and cancelling an import are in the audit log, which names the file but never what it holds.

On the server

Copy the files to the Nixt Server machine, then import them into a person’s mailbox:

sudo -u versealx versealx-server migrate mbox /srv/import/ada-archive.mbox ada@example.com --into "Imported 2019"
sudo -u versealx versealx-server migrate maildir /srv/import/ada-maildir ada@example.com
sudo -u versealx versealx-server migrate pst /srv/import/ada-2019.pst ada@example.com --into "Outlook 2019"

With --into, everything goes into that folder, and the files’ own folders become its subfolders. Without it, the folders keep their names: the old inbox goes into the inbox, and folders named like Sent, Drafts, Trash, Junk or Archive go into this account’s own folders of that kind.

Outlook data files (PST)

A PST keeps its folders: the Inbox goes into the inbox, and Sent Items, Deleted Items, Drafts and Junk E-mail into this account’s own folders of that kind (or under --into, when given). Each message keeps the headers it arrived with, its text and formatting, its attachments and pictures, and whether it was read, answered, flagged or forwarded. Its calendar and its contacts come too. The PST’s calendar goes into the person’s default calendar and its contacts into their Contacts address book; other calendar and contacts folders become calendars and address books of the same name. Each appointment keeps its time zone, attendees, reminder and whether it shows as busy, and a weekly series keeps its days, its end, and the meetings deleted from it or moved. A series that repeats daily, monthly or yearly comes in as its first meeting, and the result names it. Contacts keep their names, email addresses, phone numbers, company, addresses, birthday and notes. Importing the same PST again adds nothing twice. Tasks, notes and journal entries are counted but not imported.

Outlook’s newer offline files (.ost) are not PST files: export a PST from Outlook first.

MBOX and Maildir

migrate mbox takes one MBOX file or a directory of them:

migrate maildir takes a Maildir: the directory that holds cur and new. Its folders (.Sent, .Work.2019, or folders inside the directory) come too.

Each message keeps its read, answered, flagged and draft marks, the date it arrived, and a Maildir’s keywords. Mail deleted in the old app but still in the files is left out. The account’s quota is checked before anything is copied, so an import that would not fit is refused rather than stopped half way.

Like a migration, an import can be run again. It carries on where it stopped and never imports a message twice.

The result:

Moving everybody at once

To move a whole organisation, or a group of people, start one migration for all of them. The server copies several people’s mail at a time in the background, and you watch it from the console or the command line. Nothing has to run on the server machine.

On the console, open Mail flow › Moving in and choose New migration…. Give it:

• The old server’s address, with its port and whether it uses TLS or STARTTLS. It must be on the public internet. An old server inside your own network, on a private address, is reached only once the server’s operator sets [migrations] allow_private; until then the migration stops and says so.
• Whose mail moves: everybody with a mailbox here, the members of a group, or a list of people. Their passwords at the old server come from a CSV file saved from a spreadsheet, one person a line.
• What it moves: every folder, or every folder but junk and deleted mail.
• Since when: all mail, or only mail from a date on. Mail from before that date is left behind for good, also by later migrations of the same accounts.
• How many at once: four people unless you choose otherwise, up to twenty.

From the command line, keep the passwords in a file only you can read, never on the command line itself. A people file has one line per person: their address at the old server, their address here, and their password there. When both addresses are the same, give it once.

vsx admin migrate start --source imap.old.example --people people.csv
vsx admin migrate start --source imap.old.example --group sales@example.com --passwords passwords.csv --skip-junk-and-trash
vsx admin migrate start --source imap.old.example --everyone --passwords passwords.csv --since 2024-01-01 --at-once 8

A passwords file has each person’s address here and their password at the old server. For a server that offers only STARTTLS on port 143, add --starttls.

From Microsoft 365 or Google, for everybody

A migration can also read from Microsoft 365 or Google Workspace, with nobody’s password. Set up the app registration or the service account first, as in Microsoft 365 or Google Workspace above. Then, in New migration…, choose Microsoft 365 and give the tenant, the application (client) ID and the client secret, or choose Google Workspace and choose the service account’s key file. The key file is read in your browser.

Whose mail moves is chosen as for any migration: everybody, a group, or a list. A list for Microsoft 365 or Google has each person’s address there and their address here, or one address when it is the same, and no passwords.

From the command line:

vsx admin migrate start --source m365 --tenant contoso.onmicrosoft.com \
  --client-id 0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0 --secret-file m365-secret --everyone
vsx admin migrate start --source google --key-file google-key.json --people people.csv

The secret or the key is kept encrypted under the organisation’s key only while the migration needs it, and destroyed once every move is complete or the migration is cancelled. It never appears in an answer, a report or the audit log. The migration’s page shows whether it is still kept. Once the migration is complete, delete the secret at Microsoft, or the service account’s key and its delegation at Google.

Leaving junk and deleted mail behind, and Only mail from, work as for any migration. From Google, junk and deleted mail are Gmail’s Spam and Trash.

Watching it

Moving in lists every migration. A migration’s page shows each person: waiting, copying (with how many of their messages and how much of their mail is done), copied, failed or complete.

vsx admin migrate list
vsx admin migrate status 4

Before anything is copied, each person’s old mailbox is counted. One that would not fit their quota here fails at once, saying so, and nothing of it is copied.

When somebody fails

A person fails when the old server refuses their password or stops answering, and their line shows the old server’s own words. Retry everyone who failed runs them again, and New password… retries one person with a new password. Nothing already copied is copied twice.

vsx admin migrate retry 4
vsx admin migrate retry 4 ada@example.com --new-password

--new-password asks for the password at the terminal.

The final pass

Once everybody is copied, point your MX records at this server, then choose Final pass…. Every move runs once more, copying only what arrived at the old server in between, and is then complete.

vsx admin migrate final-pass 4

Passwords

Each person’s password at the old server is kept encrypted under the organisation’s key, only for as long as retries and the final pass need it. It is destroyed as soon as that person’s move is complete, or when the migration is cancelled. It never appears in an answer, a report or the audit log. When the migration is complete, delete your own passwords file too.

Stopping, and the report

Cancel… stops the migration: whoever is being copied stops, and every password still kept is destroyed. What was already copied stays.

Download report gives a CSV file with one line per person: what was copied, what was already here, what was older than the date you chose, what failed, and each message or folder left behind, with why.

vsx admin migrate cancel 4
vsx admin migrate report 4 > report.csv

Starting, retrying, the final pass and cancelling are for the organisation’s administrators, and each is in the audit log. An auditor can see migrations and their reports.