LanceDB

LanceDB is an open-source database for vector search, built with persistent storage, which greatly simplifies retrieval, filtering, and management of embeddings. Designed for multimodal AI, LanceDB supports various types of data including text, images, videos, and more.

LanceDB offers production-scale vector search capabilities with no servers to manage, making it easy to deploy and scale.

Store, query, and filter vectors, metadata, and multimodal data such as text, images, videos, and point clouds.

Supports vector similarity search, full-text search, and SQL, providing a flexible and powerful query interface.

LanceDB offers native support for Python and JavaScript/TypeScript, enabling seamless integration into existing projects.

Manage versions of your data without needing extra infrastructure. LanceDB supports zero-copy and automatic versioning, making data management efficient and straightforward.

Leverage GPU support for building vector indexes, enhancing performance for large-scale operations.

LanceDB integrates with popular tools and frameworks such as LangChain 🦜️🔗, LlamaIndex 🦙, Apache Arrow, Pandas, Polars, DuckDB, and more.

The core of LanceDB is written in Rust, providing high performance and safety. It is built using Lance, an open-source columnar format designed for performant ML workloads.

Install the vectordb package:

npm install vectordb

Example usage:

const lancedb = require("vectordb")
const db = await lancedb.connect("data/sample-lancedb")
 
const table = await db.createTable({
  name: "vectors",
  data: [
    { id: 1, vector: [0.1, 0.2], item: "foo", price: 10 },
    { id: 2, vector: [1.1, 1.2], item: "bar", price: 50 },
  ],
})
 
const query = table.search([0.1, 0.3]).limit(2)
const results = await query.execute()
 
// You can also search for rows by specific criteria without involving a vector search.
const rowsByCriteria = await table
  .search(undefined)
  .where("price >= 10")
  .execute()

Install the lancedb package:

pip install lancedb

Example usage:

import lancedb
 
uri = "data/sample-lancedb"
db = lancedb.connect(uri)
table = db.create_table("my_table",
                         data=[{"vector": [3.1, 4.1], "item": "foo", "price": 10.0},
                               {"vector": [5.9, 26.5], "item": "bar", "price": 20.0}])
result = table.search([100, 100]).limit(2).to_pandas()

Explore various resources to learn more about LanceDB and its capabilities:

• 📈 2000x better performance with Lance over Parquet
• 🤖 Build a question and answer bot with LanceDB