A table is a schema plus an index spec: which columns are full-text- and
vector-indexed (see Indexing). Create it once, then append rows.
Create and append
import infino
import pyarrow as pa
# Durable storage (a path or s3:// URI), required if you'll update/delete later.
db = infino.connect("./data")
schema = pa.schema([
pa.field("doc_id", pa.large_utf8(), nullable=False),
pa.field("body", pa.large_utf8(), nullable=False),
pa.field("embedding", pa.list_(pa.float32(), 384), nullable=False),
])
docs = db.create_table(
"docs", schema,
infino.IndexSpec().fts("body").vector("embedding", 384, 64, "cosine"),
)
# One append is one atomic commit, durable when the call returns.
docs.append([
{"doc_id": "1", "body": "To cancel, open Settings then Billing.", "embedding": embed("...")},
])
One append == one atomic commit, durable on return. Append in batches rather than
row-by-row for throughput.
Open an existing table
A table persists, so you create it once and open it on later runs with open_table.
docs = db.open_table("docs")
Update and delete
In Python and Node.js, update and delete match rows by a SQL predicate
string. In Rust, they take a DataFusion Expr (col(...).eq(lit(...))), where col
and lit come from the datafusion-expr crate (see the Quickstart for the
companion crates Rust needs). update replaces matched rows 1:1 with the new rows you
supply.
# Replace the row(s) matching the predicate (1:1 with the new rows):
docs.update("doc_id = '1'", [
{"doc_id": "1", "body": "Refunds go to the original payment method.", "embedding": embed("...")},
])
# Delete rows matching a predicate:
stats = docs.delete("doc_id = '1'")
print(stats.matched, stats.n_tombstoned)
update and delete require durable storage: a path or s3:// URI, not
memory://. update is 1:1, so the matched row count must equal the replacement count.
Compact, list, and drop
docs.optimize() # merge small / underfilled superfiles
db.list_tables() # ["docs"]
db.drop_table("docs", purge=True) # purge=True also deletes the table's storage
optimize accepts options (omit for engine defaults):
| Option (Python / Node.js / Rust) | Description |
|---|
max_memory_mb / maxMemoryMb | build-time memory budget, in MB |
min_fill_percent / minFillPercent | only compact superfiles below this fill percent (0 to 100) |
target_superfile_size_mb / targetSuperfileSizeMb | target merged-superfile size, in MB |
Limitations
- One writer per table at a time. Appends/updates/deletes are serialized through a
single atomic commit (concurrent writers from other processes retry).
update is 1:1. The matched row count must equal the replacement-row count.update / delete need durable storage, not memory://.- Schema is fixed at creation. Adding or changing columns means a new table.
See also
