Vector Databases

Vector Databases are designed to ensure that all data uploaded by users is vectorized, allowing for efficient and effective data retrieval and utilization of public datasets uploaded by users in the community. Using peer-hosted compute resources host embedding models to embed data for high performing agents equipped with specialized memory.

Vectorization of Data

Any data uploaded by users is automatically vectorized, transforming it into a format suitable for efficient querying and retrieval in a vector space. The Vector DB is built on Chroma DB, optimized for handling large-scale vectorized data.

Embedding Models

High-quality embedding models are used to vectorize the data, ensuring that the vectors accurately represent the information's semantic content. The process of embedding text-based datasets utilizes compute resources provided by peers in the network.

Query Processing

When a user makes a query, it is embedded into a relevant vector space using the same embedding models. The embedded query is then matched against a cluster of vectorized information relevant to the context of the query, enabling precise and relevant data retrieval.

Re-Ranking

The Vector DB includes a re-ranker mechanism that is currently under development. This re-ranker will refine the search results by re-evaluating the relevance of the retrieved vectors, ensuring that the most pertinent information is presented to the user.