Database Export and Import in AxioDB: Move a Whole Database, Index and All

AxioDB can now export a whole database into a single file, and import that file on any other machine running AxioDB.

This post covers what the export actually contains, why that matters for how fast your database is when it lands, and the three safety checks that sit between an uploaded file and your real data.

What AxioDB is, in one paragraph

AxioDB is a small database for Node.js apps. Normally when your app needs a database you install a separate one and keep it running. AxioDB skips that. You run npm install, and you have a database. No separate server, no native build step, nothing to compile. It suits desktop apps, CLI tools, local-first software and side projects, where asking a user to set up a database server is not a reasonable thing to ask.

How your data is laid out on disk

Everything lives in plain folders, and it is worth knowing the shape because export is built directly on it.

AxioDB/                  <- the main folder
  MyDatabase/            <- a database
    Users/               <- a collection
      <records>          <- your actual documents
      indexes/           <- the hash index for this collection
    Orders/
      ...
  AnotherDatabase/
    ...

One main folder. Your databases inside it. Each database holds its collections. Inside a collection sit the records, plus a hash index.

A hash index is a lookup table. Instead of reading every record to find one, it takes the ID and jumps straight to it. One step, no matter how many records the collection holds. Values are kept sorted as well, so a range query like "everything between these two dates" stays fast instead of turning into a full scan.

What export actually packs

Export takes a database folder and compresses the whole thing into one file. Records, hash index, collection structure, all of it.

That last part is the bit people usually assume works the other way. Plenty of systems export the raw data and rebuild the index on the other side, which means a large import ends with a long, silent rebuild before anything is fast again.

AxioDB does not do that, because it does not have to. The index is a file in the folder like everything else, so it gets packed with the folder. Hand the export to a different machine running AxioDB, import it, and the database is immediately ready to query at full speed. Nothing gets rebuilt.

Concretely, that means you can:

• keep a real backup of a running database

• move data between a dev machine and a server

• hand a working, pre-populated database to a teammate

• ship a database with a desktop app, already indexed

The three things import has to survive

An import takes a file from outside and writes it into your data directory. That is exactly the kind of operation that deserves paranoia, so here is what it does about it.

1. A file that is not really a backup

There is an old trick where a file is only 1 MB, and the moment you start unpacking it, it becomes 10 GB. It is small because it is largely the same byte repeated, and compression handles that extremely well. Unpack it blindly and your disk fills up and the process dies.

The obvious defence is a size limit, and the obvious defence does not work. The dangerous file is already small, so it passes a size check without effort. Meanwhile the limit blocks a legitimate user with a genuinely large database. It fails in both directions at once.

The size of the upload was never the signal. The growth was.

So import measures as it unpacks, watching how much data goes in against how much comes out. A real database backup compresses. It does not multiply. If a few megabytes are turning into hundreds, unpacking stops immediately and everything written so far is deleted.

This was tested against a real archive of that kind, one that expands roughly a thousand times over. It was caught and cleaned up.

The same pass rejects anything trying to write outside the folder it belongs in, and refuses symbolic links, so an archive cannot reach into the rest of your filesystem on the way out.

2. Two imports at the same time

Two people press import at the same moment on the same server.

If they are importing two different databases, both should complete, they have nothing to do with one another. If they are importing the same database, one should complete and the other should be told, because both writing into the same folder produces a result nobody asked for.

Which means import has to answer a question first: are these two uploads the same database?

Comparing file names does not answer it. The same export can be saved under two different names, so you would let both through and let them collide. And two entirely unrelated exports can both be named backup.tar.gz, so you would block one for no reason.

A file name is a label. It is not identity.

So import reads the real database name from inside the archive and holds a lock on that. Different databases, both proceed. Same database, the second request gets a clear message that it is already being imported, rather than quietly landing on top of the first.

3. A failure halfway through

If an import unpacks straight into the live folder and something goes wrong at the halfway mark, you are left with half a database. Part old, part new. That is worse than a failed import, because a failed import that leaves your data intact costs you nothing.

Import now runs in three steps: stage, validate, promote.

Stage. The upload is written to a temporary folder outside the data directory and unpacked there. The live database is untouched and unaware.

Validate. All checks run against the staged copy. Is this an AxioDB export at all? Is the folder structure right? Did anything try to escape its directory? All of it happens off to the side, where a bad archive can only damage a temporary folder.

Promote. Only after every check passes does the staged database move into place, as a single move.

If anything fails at any point, the temporary folder is deleted and the live data was never opened. There is no partial state to recover from, because the real folder only ever receives a database that has already passed every check.

Using it

Export and import are both available from the AxioDB Dashboard on port 27018. Export gives you a file. Import takes one.

The full documentation is at axiodb.in, and AxioDB is on npm and GitHub.

A closing note

None of these three protections is complicated. They are small pieces of code. What took the time was asking the questions in the first place, because a feature that works on a good file and a feature that survives a bad one look identical right up until the day they do not.

---

This post is adapted from a longer write-up on the decisions behind it, published on Ankan's blog.