Boring Rust bindings for zvec (Alibaba's in-process vector database) that don't eat your data.
zvec's C API has two very different teardown functions: zvec_collection_close
releases the handle, while zvec_collection_destroy deletes the collection's
on-disk storage — it's the DROP TABLE of zvec.
The existing zvec crate calls destroy from Drop. Every time a
Collection goes out of scope — every graceful shutdown — the index is
silently wiped from disk. We shipped it in production at
Cerul and lost our users' vector indexes on
every app restart before tracing it back
(oly-wan-kenobi/zvec-rs#45,
minimal repro included).
This crate is the fork we now run in production: same core API, corrected teardown semantics, and CI that proves the contract on every commit.
Dropcloses the handle. So does the explicitclose(). Neither ever touches data on disk.- Deleting data requires typing the word.
collection.destroy()is a consuming method whose doc comment starts with a warning. - Durability is tested, not assumed. CI runs, on every commit:
- write → flush → drop → reopen → the data is still there;
destroy()actually deletes, and nothing else does;- a SIGKILL crash test: a child process writes, flushes, and is killed with no destructors running — the parent must read the data back.
- Pinned upstream binaries. The
bundledbuild downloads Alibaba's official PyPI wheel and verifies a pinned SHA-256 before extracting.ZVEC_BUNDLED_WHEEL_PATHsupports fully offline builds.
[dependencies]
zvec-safe = { git = "https://github.com/cerul-ai/zvec-safe", tag = "v0.0.1", features = ["bundled"] }use zvec_safe::{
Collection, CollectionSchema, DataType, Doc, FieldSchema, IndexParams,
IndexType, MetricType, VectorQuery,
};
fn main() -> zvec_safe::Result<()> {
let mut schema = CollectionSchema::new("docs")?;
let id = FieldSchema::new("id", DataType::String, false, 0)?;
schema.add_field(&id)?;
let mut hnsw = IndexParams::new(IndexType::Hnsw)?;
hnsw.set_metric_type(MetricType::Cosine)?;
hnsw.set_hnsw_params(16, 200)?;
let mut embedding = FieldSchema::new("embedding", DataType::VectorFp32, false, 3)?;
embedding.set_index_params(&hnsw)?;
schema.add_field(&embedding)?;
schema.validate()?;
let collection = Collection::create_and_open("./docs.zvec", &schema, None)?;
let mut doc = Doc::new()?;
doc.set_pk("a")?;
doc.add_string("id", "a")?;
doc.add_vector_fp32("embedding", &[0.1, 0.2, 0.3])?;
collection.insert(&[&doc])?;
collection.flush()?;
let mut query = VectorQuery::new()?;
query.set_field_name("embedding")?;
query.set_query_vector_fp32(&[0.1, 0.2, 0.3])?;
query.set_topk(5)?;
for hit in collection.query(&query)?.iter() {
println!("{:?} score={}", hit.pk_copy(), hit.score());
}
Ok(())
}Short versions below; the full runnable program is
examples/recipes.rs
(cargo run --example recipes --features bundled).
Open a collection if it exists, create it otherwise:
let collection = match Collection::open(path, None) {
Ok(collection) => collection,
Err(_) => Collection::create_and_open(path, &schema, None)?,
};Write or overwrite documents, then make them durable:
collection.upsert(&[&doc_a, &doc_b])?;
collection.flush()?; // data is not durable until flushedVector search restricted by a metadata filter:
let mut query = VectorQuery::new()?;
query.set_field_name("embedding")?;
query.set_query_vector_fp32(&[1.0, 0.0, 0.0])?;
query.set_topk(10)?;
query.set_filter("category = 'animals'")?; // needs an inverted index on the field
let hits = collection.query(&query)?;Read back by primary key:
let docs = collection.fetch(&["a", "b"])?;Delete by primary key, or everything matching a filter:
collection.delete(&["a"])?;
collection.delete_by_filter("category = 'plants'")?;
collection.flush()?;Tear down without losing data — and the one call that does delete:
drop(collection); // closes the handle; data stays on disk
// collection.destroy()?; // the DROP TABLE — deletes the data directoryCargo dependency renaming means your use zvec::... paths keep working:
[dependencies]
zvec = { package = "zvec-safe", git = "https://github.com/cerul-ai/zvec-safe", tag = "v0.0.1", features = ["bundled"] }Two behavior changes to be aware of:
Dropandclose()no longer delete your data. Inzvec0.1.0 both ranzvec_collection_destroy, wiping the collection's on-disk storage. If you were (perhaps unknowingly) relying on that to clean up, call the explicitcollection.destroy()instead.- The hybrid-search / reranker / builder / derive / serde-json / tokio layers
were removed. If you need async, wrap calls in
tokio::task::spawn_blocking— that is what we do in production.
examples/basic.rsis a complete walkthrough — schema with inverted + HNSW indexes, insert, flush, stats, vector query:cargo run --example basic --features bundledexamples/recipes.rs— the "Common operations" above as one runnable program, compiled in CI so they can't rot.- Full API docs:
cargo doc --open --features bundled(not on docs.rs yet; the crate isn't published to crates.io while it soaks in production). - The durability suite doubles as documentation of the teardown contract.
build.rs locates a prebuilt libzvec_c_api, in order:
ZVEC_LIB_DIR— explicit directory containing the library.ZVEC_ROOT— install prefix; uses$ZVEC_ROOT/lib(+lib64).--features bundled— download the official PyPI wheel (SHA-256 verified). Overrides:ZVEC_BUNDLED_WHEEL_PATH(local wheel, offline builds),ZVEC_BUNDLED_WHEEL_URL+ZVEC_BUNDLED_WHEEL_SHA256.pkg-config(with thepkg-configfeature).- System linker defaults.
Set ZVEC_STATIC=1 for static linking.
| Feature | Adds |
|---|---|
bundled |
Fetches upstream's PyPI wheel at build time (SHA-256 verified). |
half |
fp16 vector helpers taking &[half::f16]. |
pkg-config |
Locate libzvec_c_api via pkg-config. |
| zvec-safe | zvec (upstream engine) |
|---|---|
| 0.0.x | wheel pinned in build.rs (SHA-256 verified per platform) |
Upgrading the engine wheel is a PR that must pass the full durability suite.
Deliberately minimal: schema definition, collection lifecycle, DML, vector +
filtered queries, stats, flush/optimize, raw FFI in sys. No query builders,
no rerankers, no kitchen sink. PRs adding surface area need a use case.
Cerul — a local-first video memory app. Every vector Cerul indexes goes through this binding.
Forked from zvec 0.1.0 by
oly-wan-kenobi (Apache-2.0),
after diagnosing the data-loss bug above; trimmed to the core API and
hardened. See NOTICE.
Licensed under Apache-2.0. zvec itself is developed by Alibaba and distributed under its own license via PyPI.