From JSON to Vector Search: What I Learned Storing Structured User Data for AI

Description and use

The article explains how SpendWise integrated structured user data into a vector database for semantic search while keeping a relational database for deterministic queries.

Problem

How to make structured user data retrievable for an AI when the user asks context‑dependent questions?

Outcome

The hybrid architecture allowed SpendWise to provide contextually relevant user information to the LLM without over‑loading the vector store with data that is better served by SQL queries.

Workflow

• Collect structured user data during onboarding
• Convert relevant fields to a natural‑language paragraph
• Generate embeddings with an embedding model
• Store embeddings in a vector database with metadata linking back to the original JSON
• When a user asks a question, determine if it requires semantic similarity or exact match
• If semantic, query vector store for relevant context
• If deterministic, query PostgreSQL directly
• Feed retrieved context to the LLM for response

Evidence

When we were building SpendWise, one of the things we wanted was pretty simple: The AI should know something about the user before answering them.

Source

The AI might need context such as: {"investment_experience":"Beginner","risk_preference":"Moderate","financial_goal":"Buy a house in 5 years"}

Source

If I need: expenses > ₹10,000 ... SQL is extremely good at that.

Source