What Is a Vector Database?

What Is a Vector Database?

A vector database is a database specialized for storing high dimensional vectors, typically used for indexing, searching, and storing unstructured data such as text, images, audio, and video. A vector database is particularly useful for semantic search, Recommendation Systems, and for various Conversational AI applications, such as Retrieval-Augmented Generation (RAG).

Embeddings are the numerical representations of information in vector space. Each dimension of an embedding vector contains information of the information being represented. This information can include semantic meaning, or even relationships between other information. For example, information with similar meaning will typically map to vectors that are near to each other in a high-dimensional vector space.

Words like “car,” “vehicle,” or “automobile” are all different words but have a related meaning. A semantic search system is able to represent these words as vectors and to find out the relation between words that were not searched for exactly.

The main difference between a vector database and a regular database is how information is retrieved. In a regular database, information is mostly retrieved by using the structured fields of a record (such as in a table). In a vector database, information is mostly retrieved by similarity of numerical representations (vectors) of data.

Vector databases can also be very powerful when dealing with unstructured information and therefore have been widely adopted in recent years in order to support a variety of different applications developed with AI and Deep Learning, including: documents, conversations with customers, product descriptions, images, audio recordings, etc.

How Does a Vector Database Work?

A vector database supports vector storage, vector indexing, and vector search. The vector database enables AI applications to search through huge numbers of vectors and to retrieve relevant data quickly. Importantly, a vector database doesn’t compare each vector one by one. Instead, it uses vector indexing to reduce the scope of a search prior to comparing vectors in an effort to retrieve data that is most relevant.

Vector Storage

At the center of a vector database is the embedding model. Here the input is a document, a sentence, an image, etc. which is converted into a numerical vector of high dimension.

Even after creating the embeddings, for storage in a database, each embedding is typically combined with other information about the document – in the simplest case a title or a source label, or even more metadata, like a date, or a product ID etc. – thus turning a simple embedding into a record of a document that can then be searched by semantic as well as by other criteria.

This means that a search application for documents can for example search for documents that are semantically similar to a certain question. The results can then be further restricted by for example the department or date range of the documents.

Vector Indexing

Just storing the vectors is not efficient for similarity searches on millions or billions of vectors. Therefore, the vector databases use indexing for similarity searches, similar to what databases use for range queries. For nearest-neighbor searches Approximate nearest-neighbor (ANN) search is commonly used.

For similarity search in vector databases, approximate nearest-neighbor (ANN) search is typically used. By building an ANN index, a database query can be restricted to a set of vectors that are likely to be similar to a given query vector. There are many algorithms for building efficient indexes, such as the HNSW algorithm, or using locality-sensitive hashing (LSH) to map vectors to buckets in a hash table.

Note that these indexes trade off between providing an exact search and performance. However, most AI applications do not require an exhaustive search of all possible vectors.

Vector Search

The user query can also be transformed into an embedding. Therefore, the database can compute the similarity of the query vector with all the stored vectors.

Next, the database will compare the query vector with all of the vectors in the database in order to compute the degree of similarity with each vector. This can be done by using the cosine of the angle between the vectors (i.e. cosine similarity), the dot product of the two vectors, or the Euclidean distance between the two vectors, etc.

The most relevant vectors are returned to the user. In case of a RAG (Recrray Arrow Generator), the retrieved documents are fed to a large language model as additional context.
What Are Vector Databases Used For?
Rather, a vector database searches all the vectors in a database for similar vectors. A vector database therefore does not search for identical words in a database, but for documents with similar vector representations.

Why Are Vector Databases Important for AI?

Much AI is used to deal with lots of so-called unstructured data such as text, images, audio etc. While databases are still very much king for the structured data they were made for, for such unstructured data there are specialized Vector Databases to efficiently organize the information in the vectors for similarity search.

There are a number of reasons vector databases are important for AI. For one, they enable similarity searches over large amounts of unstructured information. This is a key problem that traditional databases are not able to solve well, since they are optimized for structured records that can be searched for exact terms.

As an example, a large language model may have excellent reasoning capabilities and generate text of high quality. However, the model does not have access to an organization’s internal documents, product information, or policies. To connect the dots between input queries and relevant organizational knowledge, the model relies on a vector database for retrieval of relevant information.

To illustrate this point, Retrieval-Augmented Generation (RAG) is a typical example for this usage case. First, all documents (articles, books, knowledge bases etc.) are converted into embeddings and stored in a vector database. Then, for each question, the question itself is converted into a vector and using the database, the most relevant information is retrieved and used as context for the language model.

In summary, vector databases enable AI applications to retrieve relevant information in a timely manner, process unstructured information and provide more context aware results. To search large volumes of information in a meaningful manner is one of the primary use cases of vector databases.

What Are Vector Databases Used For?

This type of search functionality can also be used in other applications such as search, recommender systems, and more. Also note, that vector databases are not limited to generative AI applications, such as RAGs. Many other machine learning workloads can benefit greatly from vector databases.

Semantic Search

Semantic search is another popular application of vector databases. Typically such systems are not searching for exact words in the query. Instead, they search for information whose vector representation is closest to the vector representation of the query.

For example, someone searching for ways to reduce smartphone battery consumption might receive content on ways to improve mobile device battery life even though the exact phrase used in the search query is not present.

Retrieval-Augmented Generation

These databases are typically used in conjunction with Retrieval-Augmented Generation (RAG) architectures. The vector database stores the embeddings of all documents or knowledge sources in a database, which then can be searched to retrieve relevant knowledge to support a question answering task.

This information can then be used by the language model as additional context. So rather than the language model having to rely on the information contained within the model itself, it can use a specific knowledge base to answer questions.

Recommendation Systems

Recommendation systems – used by e-commerce sites, for example – represent products, users, videos, music etc. as vectors. A user looking for similar items to a product can have their product compared with other products stored in the database using their vector representations, with most similar products being at the top of the list.

An e-commerce site may calculate a vector to represent similar products to ones that a customer has viewed before. A music or video streaming site could store vector representations of user preferences or of videos and use similarity to recommend related items to users.

Conversational AI

For conversational AI to work, Vector Databases can support Virtual Assistants to lookup knowledge in a Vector Database to select the most relevant semantically related information and generate a response.

This is particularly useful for enterprise assistants to search internal documentation, policies, product information and other organizational content.

Anomaly Detection

Vector representations can be used to identify anomalies in data. A normal case forms a cluster in vector space. Anomalies are the vectors that are far from this cluster.

There are many domains where vector representations can uncover unusual behavior. For instance, in fraud detection, cyber-security, industrial monitoring or analyzing system performance.

Vector Database vs. Traditional Database

Note that Vector Databases and Traditional Databases (Relational Databases) are two very different things and are typically used for very different types of data retrieval. A relational database contains tables of structured data (e.g. tables of customer information), and one uses queries based on exact values and conditions, using relationships between tables.

A vector database does not store data in the same way that a traditional relational database would store data. In a vector database, primary data representation is a vector. Search on this type of data is based on similarity. This is as opposed to traditional databases which store structured data and perform searches based on exact queries, or even on a set of rules to determine relevance.

However, a vector database is not intended to replace a traditional database! Such a database is typically used to manage customer accounts, transactions, stockkeeping, etc., i.e. typical business data, and in addition it has the capability to handle embeddings for semantic search of unstructured data. In most cases, the vector database is used as an extension to existing AI architectures.

Real-World Applications of Vector Databases

All these applications make use of similarity search and use vectors to represent data of different kinds.

An example of enterprise knowledge search is embedding the contents of documents, manuals, policies, reports, etc. and allowing employees to search for them using natural language rather than having to remember the title or specific keywords.

Customer Support : A vector database could also be used to support customer service by storing representations of support articles, past solutions, product documentation and other knowledge about an organization. Similarity searches could then be used to find relevant information for a customer’s question, which can then be passed to a conversational AI for human-like responses.

In e-commerce for instance product descriptions, product images, customer preferences and product attributes can all be connected to a vector. Thus enabling product recommendations and similarity search of products in a store.

For media and content platforms, vectors can represent video, music, image and article characteristics, allowing for related content to be searched for by users.

This can be particularly effective in areas like fraud and anomaly detection where large volumes of data are analyzed for unusual behavior. The vectors can be used to measure how far a new set of data points are from the ‘normal’ behavior.

RAG-based applications. RAG (Red Amber Green) dashboards are frequently used to indicate the status of processes. By connecting an AI-model to this information, the same model can be used to store document-embeddings in a vector database. Then, the same model can retrieve relevant information for users asking questions in natural language.

How to Choose a Vector Database

We discuss how to choose a vector database depending on several criteria: the application, the amount of vectors, the queries, the existing technology stack, and the required operations on the vectors.

Of course, search performance and scalability are typically among the first key factors to consider when selecting a vector database to use, with different applications having very different workloads (small knowledge bases for internal use, millions or even billions of vectors for large-scale applications, etc.).

Integration with existing AI models, programs, APIs, data analysis pipelines and infrastructure such as data stores and analytics tools is also important to consider. Some databases even offer support for metadata filtering or hybrid search – a combination of semantic search and structured filtering.

Deployment options Vector databases can be deployed on a variety of platforms: as a dedicated database, open-source software, plug-in to a relational database, search engine, or data platform with integrated vector search capabilities. Even serverless environments can be suitable for fluctuating volumes of requests and slot-based compute time.

Security and governance: A vector database with a vector search capability can process very sensitive information in an enterprise setting. The way that the vectors were created can be sensitive. In addition to normal database security and access control, you should also consider data encryption, data isolation, compliance requirements and monitoring of metadata updates to a vector database.

Lastly, it is also important to factor in the total cost of the database itself versus the other operational expenses around storage, indexing, compute power, data transfer and even administration time. Some databases cost a lot to scale out while others may cost little to administer, but huge amounts to maintain.

The best database for you is the one that integrates well with your entire AI architecture. It is not necessarily the one with the most features.