Vector databases and their use in Generative AI

Generative AI (GAI) has been a dominant technology in data (and beyond) over the past couple of years. Accordingly, many organisations have spent considerable time and resources building systems that are able to leverage it, either to use internally or to license externally (or both). GAI engines are built on Large Language Models (LLMs) which are in turn built on vector data. Therefore, vector databases (which is to say, databases with vector support, whether dedicated or multi-model) form an important part of GAI solutions.

What’s more, there are basically two ways to supply data to an LLM. The first is to use a public LLM then enhance it using RAG (Retrieval Augmented Generation). This augments the LLM with your own vector data to add context and rigour. This is relatively quick and easy to set up, but has limits in terms of performance overheads and how much context RAG can actually provide. The second is to use a private LLM (whether open source, licensed, or homegrown) that is trained directly on your vector data. This approach will be more difficult and expensive, but has commensurately greater outcomes. Note that vector databases are key components in both of these use cases, and it follows that vector databases (at least in a GAI context) must be judged on their degree of support for each of them.

Vector databases themselves are not a new phenomenon, but have come into their own with the advent of GAI.  More specifically, arbitrary data, including text and images, can be stored as vector data via vector embeddings, which are lists of floating-point numbers that represent a normalised, multidimensional vector that, in turn, represents the original asset. The distance between two vector embeddings can be calculated just as with any other two vectors, which in this case translates to similarity of meaning. This forms the basis of semantic search for both words and images. Moreover, vector embeddings can be readily processed by an LLM, which can then power GAI as described above.

One of the key distinctions in the vector database market, as alluded to above, is between dedicated vector databases and multi-model databases with vector support. The former may offer performance and feature advantages, but come with the inherent cost and complexity of expanding your data infrastructure. In addition, they typically provide little to no support for other data types, that you may need to build your GAI applications. Combining multiple types of data (for, say, training purposes) will also usually be harder in a dedicated vector database, as it will involve data integration or similar technologies. It is also important to note that you will want to search, compare, and otherwise operate on your vector data. This is where the presence (or lack thereof) of various built-in algorithms to this effect can make a difference. On a similar note, it may also make sense to find a versatile database that has a vector capability if you want to perform hybrid searches across different kinds of data. Various other qualities, less specific to vector databases – high performance, flexibility, integration (especially with various GAI-oriented tools and libraries), scalability, and so on – are also important things to consider when picking out a vector database.

Fuller thoughts on the interplay between generative AI and vector databases can be found in our white paper, Architecture Patterns and Roadmap for Generative AI in the Enterprise.