Vector databases have become one of those technologies people add to project diagrams before they have a project.

The pitch is seductive. Your AI assistant will understand company documents, find the right paragraph instantly and answer questions like a colleague who has read every Slack thread since 2019. So the team creates embeddings, opens an account with a vector database provider and starts moving data around.

Three weeks later, somebody asks where the travel policy is.

The assistant confidently returns an outdated reimbursement rule from a document that should have been deleted last year.

Vector search can be extremely useful. It is also not a magical memory system, and treating it like one creates very polished mistakes.

The Useful Part Happens Before the Database

Suppose you have 20,000 support messages. A customer writes, “My payment went through but the order still says pending.”

A plain keyword search may look for those exact words. That can miss an older ticket saying, “Card charged, checkout never completed.” The wording is different, but the situation is nearly the same.

An embedding model turns each piece of text into a list of numbers—a vector—designed so related material sits closer together in that numerical space. A search query gets converted in the same way, and the system looks for nearby vectors. Pinecone’s documentation describes dense vectors as fixed-length numerical representations that encode the meaning of text, images or other data.

That conversion is the clever part.

The database stores those vectors and finds the nearest ones quickly. It does not know which refund policy is current or notice that finance changed a rule yesterday unless your application gives it that context.

Worth remembering.

Similar Does Not Mean Correct

This is where demos and real systems separate.

A vector search can find text that feels close to the question. But closeness is not truth. A paragraph about cancelling Netflix may be semantically close to one about cancelling business software, even though the billing rules are different.

The system needs more than similarity.

It may need filters for country, product, date, department or document status. Pinecone supports metadata filtering to restrict results to records matching conditions, and pgvector can combine nearest-neighbour searches with ordinary SQL filters.

Imagine an employee asking, “How much can I claim for a hotel in Tokyo?” The answer depends on travel grade, currency, policy version and perhaps whether the trip is domestic or international. Returning the paragraph that sounds most similar is not enough.

This is why a good search system often feels less like AI magic and more like careful filing with some clever maths added.

Exact Words Still Matter

The industry got slightly carried away with the idea that meaning would replace keywords.

It won’t.

If someone searches for an invoice number, a GitHub error code, a product SKU or the exact name of a compliance form, semantic similarity may be the wrong tool. You want the exact thing. Not a spiritually related invoice.

Pinecone recommends full-text search when relevance depends on exact phrases, product names, technical identifiers or specialist jargon. Its indexing model can combine dense vectors, sparse signals, text and metadata rather than pretending one method solves every query.

This mixed approach is usually called hybrid search. The name sounds grand. The idea is ordinary: use meaning when meaning helps, and exact matching when exact matching helps.

A support engineer searching for ERR_AUTH_041 probably wants that code. Someone searching for “users keep getting logged out after changing devices” wants related incidents even if nobody used those exact words.

Most useful systems need both.

The Small-Team Mistake I Keep Seeing

A common side-project story goes like this.

Someone builds an AI assistant over a folder of PDFs. They split every file into chunks, create embeddings and put them into a specialist vector database. The first few questions work. Encouraged, they add more documents.

Then the answers become worse.

The chunks are too small, so sentences lose context. Or they are too large, so one result contains unrelated topics. Old and new policy versions sit beside each other. Scanned PDFs contain broken text. A table that made sense visually becomes a row of confusing fragments.

The database gets blamed because it is the visible new technology.

But the real problem is document preparation.

Vector search cannot repair bad source material. It cannot decide which version should win unless that information is stored and used. It cannot recover a heading that disappeared during extraction.

Anyway, data cleaning remains stubbornly unfashionable.

You May Already Have Enough Database

A separate vector database is not always necessary.

PostgreSQL users can add vector similarity search through pgvector while keeping vectors beside ordinary application data. It supports exact and approximate nearest-neighbour search, multiple distance measures and standard Postgres features such as joins and transactions.

For a small internal assistant or a product with modest data, that may be enough. One system is easier to operate than two. Backups, access controls and deletion rules stay in a place the team already understands.

A dedicated vector service makes more sense when the collection is large, search traffic is heavy, low latency matters or the team needs specialised scaling and indexing features. Even then, choose it because the workload demands it—not because the architecture diagram looked empty.

The funny part is that many early AI projects worry about searching ten million vectors before they have ten regular users.

The Answer Still Needs Evidence

Most people meet vector databases through retrieval-augmented generation, or RAG. The application retrieves a few relevant pieces of information, gives them to a language model and asks it to answer using that material.

The weak point is obvious: if retrieval brings back the wrong passages, the model can produce a beautifully written wrong answer.

So show sources. Keep the retrieved text available. Test with awkward questions, not only the examples used during development. Ask questions whose answer does not exist and make sure the system can admit that.

Measure whether the right document appeared near the top. Do not judge the whole system only by whether the final paragraph sounded sensible.

A vector database is valuable because it helps software find related material at useful speed. That is a practical engineering job, not artificial memory and certainly not understanding.

Sometimes it is exactly what an application needs.

Sometimes a normal database, decent keyword search and cleaner documents would have solved the problem by Friday.