Why lexical retrieval for Chinese cannot use tsvector

One Chinese sentence, one token

Hybrid retrieval needs a lexical arm. The obvious pick is PostgreSQL's built-in tsvector and ts_rank — a few lines of SQL and you are done.

Except it cannot segment Chinese:

SELECT to_tsvector('simple', '國內出差住宿費核實報支上限為新臺幣二千八百元');
-- → '國內出差住宿費核實報支上限為新臺幣二千八百元':1     ← the whole sentence, one token

SELECT to_tsvector('english', 'lodging expenses are reimbursed up to 2800 dollars');
-- → '2800':7 'dollar':8 'expens':2 'lodg':1 'reimburs':4  ← works fine

The English line is what full-text search is supposed to look like: tokenised, stemmed, matchable. The Chinese line treats a forty-character string as a single word — so unless the user types the entire sentence back verbatim, it never matches anything.

This is not an oversight in PostgreSQL. It follows from Chinese not delimiting words with spaces. Fixing it means bolting on a Chinese segmenter (zhparser, pg_jieba), which means an extension to install, a dictionary to maintain, and segmentation errors that turn directly into retrieval errors.

Character trigrams instead

pg_trgm takes a completely different route: ignore language, cut the string into overlapping three-character groups, compare similarity. "Ignore language" is the entire value here — it treats Chinese and English identically, because it has no concept of a word at all.

The cost is that on Chinese it behaves very close to exact substring matching. Measured: a question compared against the chunk that actually contains its answer scored word_similarity between 0.50 and 0.71; against every other chunk it scored exactly 0.0.

Not small. Zero.

Two consequences of that sparseness

An all-or-nothing distribution creates two problems you have to handle.

First, filter out the zeros before fusing. A pile of rows all scoring 0 is ordered arbitrarily among themselves. Feed that arbitrary order into a rank fusion and you inject pure noise — it will confidently name a top result that is only there because it happened to come out first.

Second, lower the default threshold. pg_trgm defaults word_similarity_threshold to 0.6, and one of my measured correct hits scored 0.50 — on the default, that question is silently discarded. So search_chunks_hybrid drops the threshold to 0.25 with SET LOCAL, scoped to that transaction so nothing else in the database is affected.

You hit this because 0.6 was tuned for fuzzy matching in English. Change the language and the same constant means something different.

Fusing the two arms

The vector arm produces a ranking, the lexical arm produces a ranking, and they have to become one.

I use Reciprocal Rank Fusion: score = Σ 1/(60 + rank), with k = 60 from the original paper.

Why not just blend the scores? Because cosine distance and trigram similarity are unrelated scales with unrelated distributions. Adding them requires normalisation, every normalisation is an arbitrary choice, and either side changing its model or parameters means retuning it. RRF reads ranks only, so the scale problem never arises.

That choice has one consequence worth naming: the distance ceiling can only apply to the vector arm. A chunk found purely by literal match may sit far away in embedding space — which is fine, since semantics is not how it was found. Filtering the fused result on distance would delete the lexical arm's entire contribution in one go.

The arm is precise, and brittle

The lexical arm finds what embeddings blur: form codes, figures, statute numbers, proper nouns. It also finds nothing whatsoever for a paraphrase — query 特休天數 against a chunk that says 特別休假 and the score is zero.

Which is exactly right. The two arms are not redundant backups for each other, they are complements: one handles the literal, one handles the meaning, each working in the other's blind spot.

Then I measured it

If you build it, measure it. The eval set is 20 documents and 36 questions: 6 documents hold the answers, and the other 14 are deliberately chosen distractors — same topic, shared vocabulary, no answer. The questions come in three flavours: paraphrased, sharing wording, and hinging on a figure or form code (that last one being dense retrieval's classic weak spot).

Ground truth is defined as "the result must contain this substring" rather than a specific chunk id, so changing the chunking strategy does not invalidate the eval set.

Mode	recall@1	recall@3	recall@5	MRR	ms/query
vector	94.4%	100%	100%	0.972	7
hybrid	97.2%	100%	100%	0.986	9

Read that table honestly: the win is small and the benchmark is saturated.

bge-m3 is a strong multilingual retriever that already handles exact identifiers well — probing IR-001, 千分之一 and 百分之四十 directly landed the right chunk at rank 1 or 2 on the vector arm alone. What hybrid actually accomplished was moving one question from rank 2 to rank 1. With 31 chunks in the whole corpus, recall@3 and recall@5 are at the ceiling for both modes, so the only numbers still carrying signal are recall@1 and MRR.

So why keep it

It stays because it is never worse, it costs about 2ms, and it covers a failure mode dense retrieval is known to have.

It does not stay because the benchmark proved it transformative. It did not, and this eval set is currently incapable of proving it either way. Making it discriminate needs a corpus an order of magnitude larger. That is on the roadmap.

Which is what the benchmark is actually for right now: a regression guard, not an argument. An eval that tells you "there is no visible difference here" is worth considerably more than one that always produces a flattering number.

Source

This is part of Enterprise RAG Assistant — FastAPI, PostgreSQL + pgvector, any OpenAI-compatible endpoint, with multi-tenant isolation enforced in SQL rather than in the handlers. The decisions this post did not open up — why the tenant can only come from the key, why SHA-256 rather than bcrypt, why migrations bypass the connection pool — are all written down in docs/architecture-decisions.md.