Beyond vector search: Nova, an AI chatbot built on a knowledge graph
Some questions vector search simply cannot answer. How Nova combines a Neo4j knowledge graph with pgvector retrieval into GraphRAG, achieving both accuracy and safety in Vietnamese securities and regulation.
"How many times has this company's largest shareholder increased their stake this year, and how did the share price move each time?" Ask a typical RAG chatbot that and you get an answer that sounds right and isn't. However well it retrieves document chunks, counting along a chain of relationships is not what vector search does.
Nova is an AI chatbot platform specialised in Vietnamese securities and financial regulation. We solved this problem with GraphRAG — a knowledge graph combined with vector search. This is an account of why vectors alone weren't enough, and what we built to get accuracy and safety at the same time.
Why vector search alone falls short
The basic idea of RAG (retrieval-augmented generation) is simple: find document chunks semantically close to the question by vector similarity, and hand them to the LLM along with the query. For conceptual questions like "what's the dividend policy?" it works beautifully.
The trouble starts with questions that require relationships and aggregation. Vector search is good at finding what is similar; it cannot follow what is connected.
- "What's the total value of bonds held by A Corp's subsidiaries?" — you have to walk ownership relationships several hops.
- "Which other tickers filed the same type of disclosure as this one?" — you have to group by a structural property.
- "Among the top foreign net-buys over the last three months, which have the highest dividend yield?" — filtering and sorting, i.e. exact computation.
For these, vector search returns plausible sentences without guaranteeing a correct answer. In finance and law, where being wrong is not an option, that is fatal.
GraphRAG — a graph-and-vector hybrid
Our answer was to run two retrieval engines side by side.
- A Neo4j knowledge graph stores entities — tickers, news, disclosures, financials, legal provisions — as nodes with relationships as edges. Relationship traversal and exact aggregation and filtering are the graph's job.
- PostgreSQL + pgvector stores document embeddings in an HNSW index. Conceptual and descriptive questions go to vector search.
Depending on the nature of the question, evidence is gathered from the appropriate engine — or both — before the LLM composes an answer. Structural facts come precisely from the graph; context and explanation come richly from the vectors.
Three-tier query generation — balancing precision and flexibility
Asking the graph anything ultimately means translating into a Cypher query. The common mistake here is to let the LLM do Text2Cypher for every question. LLM-written queries are flexible, but unpredictable and occasionally dangerous.
Nova layers query generation into three tiers.
- Templates (~80%) — frequently seen question shapes are handled by vetted, parameterised query templates. Fast, safe and deterministic.
- DSL (~15%) — questions that don't fit a template but still follow a pattern are expressed in a restricted domain-specific language and compiled safely into Cypher.
- Text2Cypher (~5%) — only genuinely free-form questions the first two can't cover go to the LLM to write Cypher directly. This path is the most powerful and the most dangerous, so it must pass the validation layer described below.
Most traffic flows down the safe paths, and the LLM's creativity is permitted only on the 5% where it is actually needed. Performance and safety at once.
Nine-stage orchestration — a pipeline that doesn't collapse
Between question and answer, Nova runs a nine-stage orchestration pipeline: language detection → intent routing → query generation → retrieval (graph/vector) → evidence merging → answer generation → safety filtering → grounding verification → logging.
The key property is that no single failing stage takes the whole thing down.
- Circuit breaker: when an external dependency (graph DB, LLM API) fails repeatedly, the circuit opens and the system falls back fast.
- LLM fallback: if the answer-generation model times out, a lighter model or a predefined safe response takes over.
- Intent routing uses fast, cheap Claude Haiku; final answer generation uses higher-quality Claude Sonnet. Multilingual embeddings come from
multilingual-e5-large.
Three safeguards against being wrong
The scariest failure mode for a finance or legal chatbot is being confidently wrong. Nova blocks that from three directions.
1. Three layers against investment advice
- Explicit blocking: direct requests for investment advice ("should I buy this?") are filtered by rule.
- Disclaimers on ambiguous phrasing: wording that could be read as a recommendation gets a disclaimer attached.
- Output filtering: the generated answer is checked once more at the end and advisory phrasing is stripped.
2. Hallucination detection
- Grounding score: how far the generated answer actually rests on retrieved evidence is scored. Ungrounded sentences are filtered out.
- False ticker matches removed: ticker codes or names that don't exist or don't fit the context are caught.
- Claim verification: factual claims in the answer are checked against the evidence.
3. Cypher injection defence
The Text2Cypher path carries a threat comparable to SQL injection. To stop user input from modifying the graph or reading data beyond its permissions, generated queries are restricted to read-only whitelist patterns and destructive operations are blocked.
Living data — the pipeline keeps filling it
A knowledge graph is only valuable if it keeps being updated. Nova's data pipeline automates:
News ingestion: CafeF · VnExpress RSS crawling → LLM extraction → graph update
Market data: 5-second polling during trading hours → Redis streaming
Scheduler: 6 automated Celery Beat jobs (collect, clean, index, prune)
When news arrives, an LLM extracts tickers, events and relationships and links them into the graph while the document embedding is indexed into the vector store. Graph and vectors grow together from the same source.
Compliance and multi-tenancy
Because the service targets the Vietnamese market, compliance with the country's personal data protection decree (PDPD) is built in: automatic deletion after 90 days, PII masking (field removal, patterns and role-based rules), audit logs and consent management, plus Redis sliding-window rate limiting.
Nova is also designed as multi-tenant SaaS rather than a single deployment. For a new customer's domain, the path is customise the ontology, load the data, validate — with a target of launching within two weeks. Because the ontology defines the domain, expanding into a new industry requires no code changes.
Scale and stack
- 169 Python files, 32 REST API endpoints, 18 database tables
- 131 automated tests (running in about 31 seconds)
- Automatic language detection across Korean, Vietnamese and English
Python 3.12 + FastAPI
Neo4j 5 (knowledge graph) · PostgreSQL 16 + pgvector (HNSW)
Redis 7 + Celery (async and scheduling)
Claude Haiku / Sonnet (LLM) · multilingual-e5-large (embeddings)
Docker Compose (7 containers) · Nginx reverse proxy
What we believe
RAG is powerful but not universal. Vector search, which finds what is similar, and graph search, which follows what is connected, don't replace each other — they complement each other. GraphRAG folds both into one pipeline so the system answers precisely where precision matters and flexibly where flexibility matters.
On top of that sit three safeguards: don't be wrong, don't give dangerous answers, and always leave a record of the reasoning. For AI to be genuinely usable in domains where trust is everything, those three matter more than an impressive-sounding answer. Nova is built on that principle.