In many organizations, making data-driven decisions requires writing precise SQL queries. However, many users lack the technical background to do this comfortably. This challenge creates slowdowns in analyzing data. Our multi-agent system aims to solve this issue by automatically turning plain-language questions into validated SQL queries, and then displaying the answers as text and visuals.

Imagine someone asking, “Which products had the highest sales last quarter?” Instead of learning SQL, they can simply type the question. The system:

Interprets the question using natural language understanding.

Generates an SQL query automatically.

Validates the query using BigQuery checks.

Presents the outcome in an easy-to-follow chat interface.

Involves a human-in-the-loop step for confirmation.

By connecting everyday language with large datasets, our system makes data exploration easier without compromising on accuracy.

Achieving this objective necessitated an intricate orchestration of cutting-edge AI technologies, including automatic speech recognition (ASR), natural language processing (NLP), and text-to-speech (TTS) synthesis. However, constructing an enterprise-grade AI voicebot demanded far more than the simple aggregation of these tools; it required a meticulously structured workflow capable of handling dynamic customer interactions while ensuring an intuitive, responsive, and naturalistic user experience. Additionally, the system needed to be scalable, adaptable to future enhancements, and seamlessly integrate into the client’s existing infrastructure, avoiding disruption to business operations.

Collaborative Query Processing

Agent Specialization: Different agents handle steps like parsing, optimization, or running the query. This focus makes handling complex queries more efficient.

Parallel Execution: Agents can work at the same time on different parts of the query, speeding up responses.

Dynamic Query Optimization

Real-Time Adaptation: The system tracks how users interact with data and adjusts its approach, continuously improving performance.

Feedback Loop: Machine learning tools let the system learn from past queries, refining methods over time.

Enhanced Natural Language Processing (NLP)

Contextual Understanding: The system uses NLP to grasp user intent better, transforming everyday questions into accurate SQL.

Multi-Turn Conversations: Users can ask follow-up questions without re-explaining their entire request.

Intelligent Data Discovery

Automated Insights: The system may suggest trends or anomalies in the data.

Schema Exploration: Users can ask about database structures to learn how tables relate to each other.

At the core, our system combines multiple technologies, each addressing a specific requirement in the data exploration process:

LangGraph: 

Coordinates a multi-agent workflow. 

It channels user questions to the correct agents and maintains a structured record of each conversation in its memory. 

This ensures that the system can reference past user queries, align them with the current request, and maintain consistent context across interactions.

Streamlit: 

The front-end layer provides a user-friendly, conversational interface. 

Because Streamlit allows fast, interactive pages, we can embed real-time chat elements and visual results within the same interface. 

This significantly reduces friction, as users can issue queries, view data, and request new visualizations—all within a single page.

BigQuery: 

Serves as the central data warehouse, handling large-scale SQL operations efficiently. 

BigQuery’s serverless model means it can process substantial volumes of data without manual infrastructure management. 

Plotly: 

Specializes in transforming raw query outputs into engaging visual representations—bar charts, scatter plots, time-series graphs, and more. 

Plotly’s interactive elements enable users to zoom in on data points, hover for more details, or switch among multiple visualization types on the fly.

SQLite: 

Stores historical conversations, agent decisions, and other system state details. 

Since it is lightweight and local, SQLite speeds up lookups for past chat contexts and query references.

This allows quick resumption of sessions without depending on remote servers.

By combining these technologies, we establish a holistic environment where natural language inputs seamlessly transform into rich, validated data insights.

Below is a high-level block diagram illustrating how each agent or tool interacts within our system. Each component represents a specific function, and the arrows show the data flow between them.

Below is an overview of the primary agents and their responsibilities within our multi-agent system. Each agent operates with distinct tasks and models, ensuring a streamlined workflow and precise SQL query handling from start to finish.

Master Agent (GPT-4o)

Coordinates incoming user requests and routes them to the most suitable specialized agent. 

It interprets user objectives—whether they involve creating a new query, modifying an existing one, or updating visual output—and ensures that the appropriate component handles each request.

SQL Agent (o3-mini)

Responsible for generating and testing SQL queries. 

It queries the BigQuery schema to understand table structures and writes the SQL query. 

Once satisfied that the SQL query is correct, it passes the query to be critiqued

Critique Agent (o3-mini)

Validates the SQL command against the user’s stated requirements, flagging any mismatches or incomplete parameters. 

Proactively identifies logical or semantic errors and reduces the likelihood of running flawed queries in production.

Run Query (o3-mini)

Once the Critique Agent approves a query and the user confirms it, this component executes the final SQL in BigQuery. 

It also safeguards against unintentional large-scale queries or inaccurate operations by offering a human-in-the-loop confirmation step.

Visualizer Agent (o3-mini)

Transforms raw query output into interactive data visualizations through Plotly. 

Capable of producing everything from basic bar plots and scatter charts to complex time-series or comparative dashboards, it helps users derive insights more easily from the returned data.

Cost Estimates: Shows expected OpenAI and BigQuery charges so users can decide if a query is worth running.

Chat Management: Allows users to rename, remove, or revisit past discussions, keeping their workspace organized.

Session Resumption: SQLite saves entire conversations, making it easy to jump back into old queries.

Adaptive Visualizations: Users can simply say which chart type they want, and the system generates it automatically.

We assessed our agent from both technical and user-oriented perspectives to ensure it meets enterprise needs.

Technical Accuracy & Performance

SQL Validity: Achieved a 98% success rate for syntactically correct SQL.

The SQL agent was rigorously tested using a combination of functional testing to verify expected behavior and ad-hoc testing to assess its performance in unpredictable, real-world scenarios.

The Critique Agent adds an extra layer of validation to catch potential mismatches, ensuring that only queries that fully satisfy the user’s intent are executed.

The human-in-the-loop mechanism in the Run Query step adds a layer of assurance, helping prevent accidental execution of incorrect or large-scale queries.

User-Centric Metrics

Natural Language Understanding: Resolved 89% of ambiguous queries via multi-turn clarifications, with an F1 score of 0.76.

Response Time: Empirical measurements indicate the system can return fully processed queries and visualizations within 40 seconds to 2 minutes, even for more intricate requests. 

Performance and Scalability

Data Scalability: Tested the SQL agent on a large Shopify dataset containing over 50 million rows and more than 100 tables with complex joins. Despite the dataset’s size and complexity, the agent correctly identified the necessary tables, columns, and joins, generating valid queries with minimal latency.

Efficient Data Handling: BigQuery’s serverless architecture enables seamless processing of large-scale datasets with minimal latency.

Modular and Extensible Design: LangGraph’s graph-based framework allows for flexible integration of new agents and supports complex, multi-stage workflows.

Well-Defined Agent Roles: Clear separation of responsibilities among agents makes the system easier to maintain, update, and troubleshoot.

Persistent Conversation Memory: Using SQLite to store interaction history simplifies auditing and supports continuous refinement of system performance.

Model Testing and Comparison

4o Model:
Our tests revealed that the 4o model often struggled to follow all system instructions accurately. This led to additional iterations as the Critique Agent had to rewrite queries, increasing query generation time.

o3-mini Model:
The o3-mini model performed exceptionally well. Its advanced reasoning capabilities allowed it to follow instructions more accurately, resulting in fewer revisions. In our evaluations, more than 98% of the queries generated by our system are accurate.

Gemini Flash 2.0:
We tested the Gemini Flash 2.0 model and found that while it offers lower latency, it shows lower accuracy in query editing and instruction following.

Claude Sonnet 3.6:
The 3.6 version did not perform well, primarily due to missing parameters in tool calls, which negatively impacted its performance.

Claude Sonnet 3.7:
Although the 3.7 version handled tool calls effectively and achieved similar accuracy to the o3-mini, considering cost and rate limits, the o3-mini proved to be the more practical choice.

Deepseek R1:
This model was found unsuitable for tool calling because it repeated similar tool calls, affecting overall performance.

Open Source Models:
We evaluated several open-source models as well, but after weighing accuracy, cost, and rate limits, o3-mini emerged as the best choice at the time of this blog.

Accuracy of Results: The system’s multi-agent workflow consistently produces queries that align with user intent, achieving high precision and reliability.

Usability for Everyone: Designed for technical and non-technical users, the interface simplifies data querying through natural language, making it easy for anyone to explore data.

Cost Estimates: Real-time projections for AI model usage and BigQuery queries inform users about potential expenses, supporting better cost management and decision-making.

Overall, these results underscore the system’s ability to maintain a strong balance between accuracy, efficiency, and usability for both technical and non-technical users.

This system addresses the most pressing needs of non-technical users who require meaningful data insights but lack direct SQL expertise. Combining multi-agent collaboration, intuitive design, and robust validation mechanisms streamlines the entire analytics process, allowing users to focus on interpreting results rather than wrestling with query syntax.

Core Advantages

Natural Language Interaction: Eliminates the need for manual SQL query writing, making data analysis accessible to everyone.

Automated Validation: Ensures queries are correct before execution, reducing errors and wasted resources.

Interactive Visualizations: Translates raw data into intuitive charts and graphs, delivering immediate insights.

Scalable & Modular: Provides a flexible foundation that can easily integrate additional data sources or specialized agents.

Enhanced User Control: Facilitates renaming, deleting, and resuming chat sessions, granting users greater autonomy in managing their exploratory process.

Cost Transparency: Supplies estimated costs for both OpenAI calls and BigQuery queries, enabling more informed decision-making.

Ultimately, these features converge to empower teams across varying skill levels, driving more efficient and inclusive data exploration. The result is a powerful framework that removes traditional barriers to analytics, ensuring that actionable insights remain within reach for every stakeholder.

Elevate your projects with our expertise in cutting-edge technology and innovation. Whether it’s advancing data capabilities or pioneering in new tech frontiers such as AI, our team is ready to collaborate and drive success. Join us in shaping the future—explore our services, and let’s create something remarkable together. Connect with us today and take the first step towards transforming your ideas into reality.

Drop by and say hello! Medium LinkedIn Facebook Instagram X GitHub