Asking a Database in Natural Language — IRIS Ontology-First Design, Operations Edition

If the previous post Not Schema, But Concepts was about design, this post is a record of what we decided while actually running IRIS.

Problem

For executives and practitioners trying to make data-driven decisions at work, the biggest barrier is not technical limitation but lack of accessibility. To answer simple questions like "Which department has the highest turnover rate this year?", one must write SQL queries. Users who cannot do this directly must send requests to the IT team every time. This process hinders immediate insights and causes days of waiting time for answers. The bottleneck in data retrieval is a major cause of reduced business agility.

Why It Is Difficult

Database schemas only define column names and data types; they do not contain business concepts like "turnover rate." To bridge this gap, someone must perform the complex task of translating business terms into data structures. Additionally, sending both schemas and actual data to Large Language Models (LLMs) increases costs and raises privacy concerns. A bigger issue is that LLMs may generate different SQL for the same question each time, making it difficult to guarantee reproducibility of results.

What We Did

We designed an ontology in YAML format to separate business concepts from DB schemas. When the domain changes, only the ontology needs to be modified, improving system maintainability. We divided query processing into three stages to maximize efficiency. Tier 1 performs golden query normalization matching, Tier 1.5 uses embedding similarity, and only Tier 2 uses an LLM to parse intent and generate SQL.

We controlled the flow so that only the list of ontology concepts and parsing results are sent to the LLM, ensuring actual data does not leak externally. In the response formatter stage, only aggregated values are passed after PII masking, minimizing security risks. Users can ask questions directly in their existing work channels via the Microsoft Teams bot (Adaptive Card). We introduced an ontology self-reinforcement loop to approve or reject new aliases and expressions, continuously accumulating vocabulary during operations. We built SQL generators for comparison, trend, cohort, and ranking analysis, along with a 4-step chain HR agent, to support complex analyses.

There is a reason for the order of the 3 Tiers. Frequently asked questions are fixed with golden queries so that the same question always yields the same SQL. Questions with different phrasings are attached to golden queries via embedding similarity. Only new questions that still aren't caught are sent to the LLM. This reduces LLM calls and makes answers reproducible.

By the Numbers

Metric	Value	Notes
Test Pass Rate	65/65	Passed all phases A~D
Actual Data Sent to LLM	0 cases	Only ontology concepts and aggregated values sent
Pilot Period	4 weeks	Based on proposal
Teams Bot Deployment Date	2026-03-11	Teams integration completed

What This Post Says

The core of a natural language data query system is not model performance but the sophistication of the ontology. If you build a solid ontology first, the LLM only handles intent parsing in the final stage, ensuring stability and security. Readers should collect the 20 most frequently asked questions by executives tomorrow and create a table mapping the nouns in each question to the corresponding database columns. This table will be the first version of your ontology. As you create the table, you will find cases where the same noun refers to different columns in different teams. Resolving these conflicts is essentially ontology design.

Problem

Why It Is Difficult

What We Did

By the Numbers

What This Post Says

Further Reading