What is the difference between SQL and vector database?

SQL databases store structured data in rows and columns and are optimised for exact-match queries — finding a record where customer_id equals 4471, for example. Vector databases store high-dimensional numerical representations (embeddings) of data and are optimised for similarity search — finding items that are semantically close to a query, even if no keywords match. SQL uses indexes like B-trees for precise lookups. Vector databases use approximate nearest neighbor (ANN) algorithms like HNSW to find the closest vectors in high-dimensional space. The key difference is intent: SQL answers 'find the exact match,' while a vector database answers 'find the closest meaning.'

What are the top 5 vector databases?

The top five vector databases widely adopted in production AI systems are: (1) Pinecone — a fully managed cloud-native vector database with strong developer experience and minimal operational overhead. (2) Milvus — an open-source, highly scalable vector database designed for billion-scale datasets. (3) Weaviate — an open-source vector database with built-in vectorisation modules and hybrid search. (4) Qdrant — a Rust-based open-source vector database known for performance and advanced filtering. (5) pgvector — a PostgreSQL extension that adds vector similarity search to existing Postgres deployments, ideal for teams that want to avoid introducing a new database into their stack.

What is the difference between a vector database and a normal database?

A normal (relational or document) database stores data as structured records and retrieves them using exact-match queries, filters, and joins. A vector database stores data as high-dimensional numerical vectors — mathematical representations of meaning generated by embedding models — and retrieves them using similarity metrics like cosine similarity or Euclidean distance. Normal databases answer the question 'give me exactly this.' Vector databases answer the question 'give me the most similar things to this.' This makes vector databases essential for AI applications like semantic search, recommendation engines, and retrieval-augmented generation (RAG), where understanding meaning matters more than matching keywords.

Does Netflix use a vector database?

Yes. Netflix uses vector similarity search extensively for its recommendation system. When a user watches a film, Netflix generates embedding vectors that capture the semantic properties of that content — genre, mood, pacing, cast, visual style — and then searches for the nearest vectors to recommend similar titles. This is the technology behind 'Because you watched...' recommendations. Netflix has publicly discussed using approximate nearest neighbor search at scale, and their engineering blog documents their use of vector-based approaches for content personalisation, search ranking, and artwork selection.

Is the vector database dead?

No. The vector database market is growing rapidly, not declining. What has changed is the competitive landscape. Traditional databases like PostgreSQL (via pgvector), Elasticsearch (via k-NN), and MongoDB (via Atlas Vector Search) have added vector search capabilities, which means teams no longer always need a standalone vector database. However, purpose-built vector databases like Pinecone, Milvus, and Qdrant still outperform these add-ons at scale, particularly for workloads exceeding millions of vectors where indexing strategy, memory management, and ANN algorithm tuning become critical. The technology is not dead — it is maturing and being absorbed into the broader database ecosystem.

What are the 4 types of databases?

The four primary types of databases are: (1) Relational databases (SQL) — store data in structured tables with rows and columns, using SQL for queries. Examples include PostgreSQL, MySQL, and Oracle. (2) Document databases (NoSQL) — store data as flexible JSON-like documents without a fixed schema. Examples include MongoDB and CouchDB. (3) Key-value databases — store data as simple key-value pairs for ultra-fast lookups. Examples include Redis and DynamoDB. (4) Vector databases — store data as high-dimensional embedding vectors and retrieve results using similarity search. Examples include Pinecone, Milvus, and Weaviate. Each type is optimised for different query patterns, and modern AI systems often use multiple types together.

What are 5 examples of vectors?

In the context of AI and vector databases, five practical examples of vectors are: (1) Text embeddings — a sentence like 'How do I reset my password?' is converted by a model like OpenAI's text-embedding-ada-002 into a 1536-dimensional vector that captures its meaning. (2) Image embeddings — a product photo is converted by a vision model like CLIP into a vector that represents its visual features. (3) Audio embeddings — a spoken phrase is converted into a vector that captures acoustic and semantic properties. (4) User preference vectors — a user's viewing or purchase history is encoded as a vector representing their tastes for recommendation systems. (5) Document embeddings — an entire PDF or article is chunked and each chunk is converted into a vector for retrieval-augmented generation (RAG) systems.

Why does AI use a vector database?

AI uses vector databases because large language models (LLMs) have a fixed knowledge cutoff and limited context windows. A vector database solves both problems by storing external knowledge as embeddings and retrieving only the most relevant information at query time. This is the foundation of retrieval-augmented generation (RAG). When a user asks a question, the system converts the query into a vector, searches the vector database for the closest matches, and feeds those results to the LLM as context. Without a vector database, the LLM can only rely on its training data — which may be outdated, incomplete, or entirely wrong for domain-specific questions. Vector databases give AI systems access to current, specific, and verifiable knowledge.

What is the easiest vector database to use?

For most teams, the easiest vector database to start with is Chroma DB. It is open-source, runs locally with a single pip install, requires minimal configuration, and integrates directly with popular frameworks like LangChain and LlamaIndex. For teams already running PostgreSQL, pgvector is the easiest path because it adds vector search to an existing database without introducing new infrastructure. For teams that want zero operational overhead in production, Pinecone is the easiest managed option — it handles indexing, scaling, and infrastructure entirely, so engineers only need to send vectors and queries via an API. The right choice depends on whether the priority is local development speed, infrastructure simplicity, or production scalability.

Vector Databases · Technical Briefing

Scale AI Without Breaking Trust

Production controls that prevent chaos.

Leonard S Palad Senior AI Engineer · Master of AI (2025–present) · MBA · cloudhermit.com.au

Scale AI Without Breaking Trust — Vector Databases

Picture this. A CTO — sharp, experienced, the kind who reads the papers and actually understands them — stands in front of their board and confidently announces that their new AI system does semantic search. It understands meaning, not just keywords. It finds what the user actually needs, not just what they literally typed. The board nods. The budget is approved. The engineers ship.

And then — quietly, invisibly, in production — the system starts returning garbage. Not obvious garbage. Plausible garbage. The kind of garbage that looks correct until the day a user asks a critical question, the retrieval layer pulls the wrong context, the LLM confidently fabricates an answer from that wrong context, and nobody knows why because the logs show a successful query.

This is not a hypothetical. This is what happens when an AI system is built on a retrieval architecture that was never designed for the job.

"The problem is not that their AI model is wrong. The problem is that it is retrieving the wrong information — and doing it with complete confidence."

What Is Actually Happening Under the Hood of Every AI System Built on the Wrong Foundation

Traditional databases were built for one thing: exact matches. A CTO asks their system to find a record where the customer ID equals 4471. The database finds it. Done. Clean. Deterministic. But an AI system does not operate on exact matches. It operates on meaning. And meaning cannot be stored in a B-tree index.

When a user types "the new action movie with that one actor who was also in the movie with green falling numbers" — they are not searching for a keyword. They are searching for a concept. A traditional database returns nothing. A properly built AI system understands they are asking for John Wick, retrieves it, and answers correctly. The difference between those two outcomes is the retrieval architecture — and most engineering teams are quietly running the first one while telling their CTO they built the second.

The questions every VP Engineering should be asking right now — but most are not.

When their AI system fails to retrieve the right document — does the engineering team even know it failed? Or does it just return the closest approximate match and move on?

When the dataset crosses a scale threshold — does the retrieval architecture degrade gracefully, or does it silently collapse into returning whatever is computationally convenient?

When their LLM produces a confident, articulate, completely wrong answer — can the team trace it back to a retrieval failure? Or does debugging begin and end with "the model hallucinated"?

When a regulator, an auditor, or an angry enterprise customer asks why the AI said what it said — is there a complete audit trail of what was retrieved, what was indexed, and what similarity metric made that decision?

The Brutal Mathematics of Getting This Wrong at Scale

Here is what the research actually says, and a CTO ignores it at their peril. The naive approach to vector search — computing the distance between a query and every single vector in the database — scales at O(dN) operations, where d is the number of dimensions and N is the number of vectors. In plain language: as the dataset grows, the search time grows proportionally. Every document added makes every query slower. There is no ceiling. There is no plateau. It just keeps getting worse.

100,000

The hard boundary. Below this number, exact vector search can work. Above it, exact search becomes computationally prohibitive and a production AI system must use approximate nearest neighbor algorithms — or it will buckle under real-world load.

Most CTOs building AI systems right now are sitting comfortably below that threshold in development. Their search feels fast. Their demos look clean. Their engineers say everything is fine. And then the system scales. The documents pile up. The queries compound. And the architecture that worked perfectly at 50,000 vectors begins to crack visibly at 500,000 — because nobody made the hard architectural decision before it was cheap to make it.

The engineering team is not lying. They are not cutting corners deliberately. They are simply running a retrieval architecture that was never designed for the velocity of data that production AI systems actually produce. And by the time the CTO discovers the problem, re-indexing, re-ingesting, and re-architecting the entire retrieval layer has become a six-figure problem that could have been a two-day decision.

Free Technical Report — Immediate Access

The exact retrieval architecture that fixes this — before it becomes a crisis.

The full report hands CTOs and VPs of Engineering the specific algorithms, trade-offs, and 7-step implementation plan to build a retrieval layer that holds up at production scale.

What the full report reveals:

→Why HNSW is the industry standard for production systems over 100,000 vectors — and the exact parameter ranges engineers must tune or face silent accuracy collapse

→How IVF clustering slashes search complexity to O(DN/k) — and the two configuration mistakes that wipe out every speed gain

→The cosine similarity vs. Euclidean distance decision that most teams get wrong — and what happens to retrieval quality when they do

→The document chunking strategy (500–1000 tokens, 20% overlap) that preserves semantic coherence — and the chunking anti-patterns that silently destroy RAG accuracy

→How to evaluate retrieval quality before it fails in front of a customer — including the minimum evaluation dataset size and the metric most teams skip entirely

→The complete vector database selection framework — when to use FAISS, Pinecone, Milvus, Weaviate, or pgvector, and when each one will quietly fail the team

→The 7-Step Production Action Plan — sequenced, specific, and annotated with the implementation warnings that save engineering teams from the mistakes already made by others

Get the full report — no cost, immediate access.

For CTOs making architectural decisions before they become expensive · VPs of Engineering who need to brief up and brief down with confidence · Engineering leads building production RAG systems

No spam. No sales calls. One report, delivered once. Unsubscribe anytime.

✓

Good. Keep reading — the solution is right below.

The full report is also on its way to your inbox. Scroll down to continue reading now.

The Architecture That Actually Holds: How Vector Databases Work at Scale

The reason vector databases exist is not because traditional databases are bad. It is because traditional databases were built to answer a different question. SQL finds exact matches. Vector databases find meaning. And the mechanism that makes that possible — high-dimensional embeddings searched with approximate nearest neighbor algorithms — is specific, learnable, and entirely actionable. Here is what every CTO needs to understand before their next architectural decision.

Step 1 — Establish Requirements Before Touching a Database

The single most expensive mistake a CTO can make is selecting a vector database based on what is trending in the engineering community rather than what the actual system requires. Before any selection decision, the team needs answers to five questions: Is the dataset above or below 100,000 vectors? Does the system need full CRUD operations or read-only access? What are the memory constraints? Does the team have operational expertise for self-hosted deployment, or does the system need managed cloud infrastructure? And critically — what metadata filtering does the system require, and how will that interact with vector search performance?

These questions have different answers for different organisations. The answers determine the entire architecture. Getting them wrong at the start costs months at the rebuild.

Step 2 — Choose the Embedding Model First, Everything Else Second

The embedding model is not a configuration detail. It is the foundation. The model determines vector dimensionality, the correct similarity metric, and the semantic quality of every single retrieval result the system will ever produce. The catastrophic mistake — and it happens regularly — is changing the embedding model after ingestion. Every vector in the database must be regenerated and re-ingested from scratch, because vectors from different models cannot be compared. This is not a minor inconvenience. This is a full re-architecture under production pressure.

Step 3 — HNSW for Production Systems Over 100,000 Vectors

For production systems above the 100,000-vector threshold, HNSW — Hierarchical Navigable Small World — is the industry standard. It builds a multi-layer graph structure, analogous to a highway system: the top layer connects major hubs with few edges, lower layers add progressively finer connections, and search starts at the top to rapidly narrow the region of interest before descending for precision. The result is search speed that scales to millions of vectors without proportional degradation.

The parameter decisions that matter: M, the number of connections per node, should be set between 16 and 64. efConstruction, which controls index quality at build time, should be set between 200 and 500. efSearch should be tuned based on the accuracy requirements of the specific application. Set these too low and retrieval quality silently degrades. Set them too high and memory consumption becomes a production constraint. The tuning is not optional — it is the difference between a system that works and a system that looks like it works.

Step 4 — Chunk Documents for Semantic Coherence, Not for Convenience

Retrieval quality lives or dies on chunking strategy. Chunks of 500 to 1,000 tokens with 20 percent overlap between consecutive chunks preserve the semantic thread of a document while keeping individual chunks focused enough to be retrievable. Too small and the chunk lacks the context needed to answer the question. Too large and the semantic signal is diluted across too many concepts, reducing retrieval precision. Section boundaries should be preserved wherever possible, because splitting a concept across two chunks is one of the most reliable ways to produce confident, wrong LLM outputs.

Step 5 — Metadata Filtering Architecture Before Ingestion

Metadata filtering — by source, timestamp, section type, document ID — is not a post-launch feature. It must be designed before ingestion begins, because the schema decisions made at ingestion time determine what filtering is possible later. Indexes must be created on frequently filtered fields. The team must decide whether to filter before or after vector search, because both approaches carry performance implications that vary dramatically across vector database products. This decision cannot be retrofitted without re-ingesting the entire dataset.

Step 6 — Monitor Before Production, Not After

Gradual performance degradation is the silent killer of vector database systems. As indexes grow, query latency slowly worsens — until it suddenly doesn't. Monitoring must capture p50, p95, and p99 query latency with alerts configured before p99 exceeds the acceptable threshold. RAM, CPU, and disk usage alerts should fire at 80 percent capacity. The team that deploys monitoring after their first production incident has already paid the tuition.

Step 7 — Evaluate Retrieval Quality Independently of LLM Output

A CTO who evaluates their RAG system by asking whether the final answer looks correct is measuring the wrong thing. The LLM can compensate for poor retrieval by drawing on its training data — producing answers that seem accurate but are entirely disconnected from the system's actual knowledge base. Retrieval quality must be evaluated independently, using a dataset of at least 100 query-document pairs where the correct documents are known. Precision and recall must be measured at the retrieval layer before the LLM is ever involved. This is the evaluation step most engineering teams skip — and the one that makes every other step meaningful.

"The CTO who understands their retrieval architecture is the one whose AI system keeps working when everyone else's starts failing."

The decision is not whether to build on vector databases. That decision is already made. The question is whether the architecture is built correctly the first time, or rebuilt expensively after the first serious production failure. The framework above makes the first outcome available to any team that chooses to use it.

Leonard S Palad Senior AI Engineer specialising in production RAG and multi-agent systems. Currently completing a Master of AI. He writes practical, engineering-first AI content at cloudhermit.com.au and connects with practitioners on LinkedIn.