FAQ
How to run the server, how a document sits on disk, and how a backup is built.
Running the server
What do I connect to?
A MongoDB driver, pointed at 127.0.0.1 port 27018. The listener speaks the MongoDB wire. The same process answers find, aggregate, and db.runCommand({ sql }). Install and the first insert are on Getting started.
Do I install MongoDB?
The Kova process is the database. A MongoDB server on port 27017 can stay beside it. You need a MongoDB driver or shell only as the client that sends commands.
Who can connect?
Security is off in the default kova.toml. Any client that can reach the port can read and write. Turn on [security] when you want a user and a password. Both must be sent. One of them alone is refused. See Security.
How do I use the crates.io crate?
The package name is kovadb. The library name is kova. After 0.1.0 is published, cargo install kovadb installs the binaries kova and kovadump. A program that embeds the server depends on the package and calls the library:
kovadb = "0.1"#[tokio::main]
async fn main() {
kova::serve().await;
}serve is the same process the kova binary starts: it reads kova.toml, opens data/, and listens. Version 0.1.0 is published from GitHub Actions when a merge into main changes the crate sources. It is not on crates.io yet. The install commands are on Getting started.
Can SQL and document commands see the same rows?
Yes. A collection is the table. db.emp.find({ sal: { $gt: 2000 } }) and SELECT ename, sal FROM emp WHERE sal > 2000 read the same pages. SQL runs in DataFusion. The statement is one FROM. A join in SQL is outside that statement. Use $lookup in the pipeline, or write the result with $out and select from that collection. See SQL.
How do I create a table, insert, and update?
The collection is the table. The first insert creates it. db.createCollection("orders") creates it empty, which is also how you attach an autoincrement field. insertOne and insertMany add rows. updateOne and updateMany change them. deleteOne removes one. createIndex adds an index on a field you look up. A later SELECT reads those rows.
db.createCollection("orders")
db.orders.insertOne({ sku: "kettle", qty: 2 })
db.orders.updateOne({ sku: "kettle" }, { $set: { qty: 3 } })
db.orders.createIndex({ sku: 1 })CREATE TABLE orders (sku VARCHAR, qty INT);
INSERT INTO orders (sku, qty) VALUES ('kettle', 2);
UPDATE orders SET qty = 3 WHERE sku = 'kettle';
CREATE INDEX sku_1 ON orders (sku);The SQL statements are stored. They become the same log records as the document commands above. Details are on Documents, Indexes, and SQL.
What survives a restart?
Committed documents, indexes, views, users, and autoincrement counters. They are in the page files under data/. An open transaction that had not committed is gone: its writes never reached a page. Start the process again from the same directory and the listener answers with the same data.
What happens when I add a shard?
Add the shard to the router's list and restart the router. Documents whose _id now falls on the new arc move. A document that stays on its current shard is left where it is. During the move, a find by _id returns the document once. Removing a shard does not move documents. See Sharding.
How a document is stored
Where does a document live?
On a page. The default page is 16 KiB. A collection is a list of page ids. The document body is BSON in a slot on one of those pages. A document larger than the page payload is a chain: a head page and continuation pages. The head id is the one the collection lists. RAM does not hold a second copy of the collection. The concepts page is the short map of this layout.
What happens on an insert?
The listener hands the command to the worker that owns the collection. That worker appends the change to the write-ahead log and flushes it when wal_fsync is true, which is the default. Then it dirties the page. The reply means the log has the change. A checkpoint, every 128 log records by default, writes the dirty pages, records how far the log has been applied, and deletes the log segments behind that point.
What is the buffer pool?
A fixed cache of pages, 4096 per partition by default. A read pins a page, copies the slots it needs, and unpins. A miss reads the page file. When the pool is full, the least recently used unpinned page is written back if it is dirty, then dropped. The collection can be larger than RAM. The cache is the only place a hot page stays resident.
Why four workers?
One process runs four partition workers. The collection name is hashed, and every document of that collection on this process lives on one worker. Writes to that collection stay on that worker, so the log for the collection stays in one order. A scan of the collection can decode pages on more than one core. shop.orders and shop.customers can run at the same time because they can hash to different workers. This split is inside the process. Sharding is the split of one collection across processes, by _id.
What does startup do after a crash?
It opens the catalog page. When that catalog is current, startup does not walk user documents to learn the schema. It opens the B-tree and hash index from their own pages. It then replays the log records that landed after the last checkpoint. A write that reached the log and not yet the page is applied. A transaction that did not commit has no page image and no log record to replay.
Where do indexes live?
The B-tree and the hash index are pages in the same pool. Each entry records the page that holds the document. A hit reads that page. A missing id reads no page. When an update moves the document to another page, the entry's page moves with it. A text index is Tantivy, updated on write. A view can ask for that text index to stay on disk. See Indexes and Text search.
What is the admin catalog?
Database names, collections, field types, index declarations, users, roles, views, and autoincrement counters are rows in admin. They are written at checkpoint from the memory the process already keeps. You can find them. An autoincrement counter stores last, the last number already handed out. The next insert that omits the field stores last + 1. See Catalog and Autoincrement.
How backup works
What does kovadump read?
kovadump opens a MongoDB wire connection to a server that is already running. It does not copy the page files and it does not copy the log. Backup asks the server for the collections in one database, then for their documents, index declarations, views, and autoincrement counters. The server answers through the same commands a client uses. A checkpoint can run while the dump runs. The dump sees committed documents.
kovadump backup --db shop shop.kdmpwrote 1 collections, 3 documents, 0 views to shop.kdmpWhat is inside the file?
A Kova dump starts with the bytes KOVADMP1. After that magic, the file is a sequence of BSON documents. Each one has a kind:
collectioncarriesname. Thedocument,index, andcounterrecords that follow belong to it.documentcarries the stored document indoc.indexcarries onecreateIndexesspec inspec. The generated_id_index is omitted. The server adds it on restore.countercarriesfieldandlastfor one autoincrement field.viewcarriesname,viewOn,pipeline, andkova.
The file is a logical copy. Page size, compression, and which worker owned the collection are not in it. Restore can load it into a server with a different page_size.
What does restore do?
Restore reads those records and sends ordinary commands: create, insert, createIndexes. Each insert appends the log and dirties a page, the same path as a client insert. Indexes are built from the declarations. A replica primary proposes those writes, so followers receive them through the oplog. A view that still reads a collection being replaced is dropped first. The collection is dropped and created again, then the views are created. A collection the dump does not name stays. When the collection is already there, restore asks before replacing it. --yes skips the question.
The same command reads a pg_dump plain SQL file, a mysqldump SQL file, or a mongodump directory or archive, and writes them through that insert path. Binary COPY and custom-format archives are refused. Details and the flags are on Backup and restore.
Router, one shard, or one process?
Point kovadump at the address you want to read. Against one process, the file has that process's database. Against a router, the file has the whole database when every shard answers. If a shard is down, the dump fails and no partial file is written. Against one shard, the file has that piece.
Copies, and what is still planned
What is a replica set?
Photocopies of one process. Every voter stores every document of that set. The primary appends the log. Followers apply the same records. OpenRaft runs the election. The router does not join the set. See Replica sets.
Is there a change feed?
The feed is planned. A committed insert, update, or delete will publish one JSON record, with before and after, to Apache Iggy running as a separate process. Kova will not embed that process. Until the feed is in the server, read with find, or tail local.oplog.rs on a replica set. The record shape is on the Feed page. The libraries the server links today are on Powered by.