2-week AI Audit
AI AuditVisibility, risk, outcomes.AI TransformationCapture AI value across workflows.AI GovernanceContinuous evidence, not paperware.AI FluencyWorkforce that compounds.AI engineeringBuild AI inside your product.EvalsContinuous evaluation in production.All servicesThe full service map.
Not sure where to start?Take the 5-min maturity assessment →Maturity modelOpen assessment
By industry
Private equityPortfolio-wide value capture.Regulated financeThe finance umbrella.Banks & capital marketsRegulator-defensible under SR 11-7.FintechShip trusted AI fast.Asset & wealthFor advisors and ops.REITs / real estateAI-driven NOI uplift.InsuranceUnderwriting + claims, instrumented.
Guides, templates, frameworks
Field notesEditorial reads on where AI work stalls.GuidesFinance AI systems, controls, and evidence.FrameworksLevers, maturity models, and control maps.TemplatesAudit memos and reusable eval artifacts.ScorecardsFast reads on adoption, strategy, and governance.Case studiesProof from AI work in finance.Golden datasetsReusable test sets for AI evaluation.NL-to-SQL evalsAudit-grade evidence for finance data agents.AI Audit resourcesInventory, Shadow AI, and board evidence.All resourcesBrowse the complete editorial library.
Featured field note
Why Bank AI Budgets Get Approved and Don't Ship.
Why approved AI spend still fails to produce owned outcomes.
Read more →

Evaluate the semantic layer before SQL.

Most NL-to-SQL errors are not purely SQL errors. They start when the system retrieves the wrong business meaning for the question.

Evaluate the semantic layer ->Read the finance guide ->

Semantic layer evaluation measures whether an AI system maps finance schema objects to the right business meaning: metric definitions, synonyms, sample values, tenant overlays, time logic, role context, and permitted data slices.

Why it matters

Schema accuracy is not business accuracy.

A model can select the right table and still use the wrong meaning. Finance metrics vary by tenant, workflow, close state, policy, and role.

Revenue can mean bookings, recognized revenue, or cash received depending on context.
Headcount can include contractors in one workflow and exclude them in another.
Vendor spend can depend on mapping tables that are not visible in the raw schema.
A metric definition can be right globally and wrong for one tenant overlay.
Eval dimensions

The semantic layer needs its own scorecard.

Do not wait for the final SQL answer to discover semantic drift. Evaluate retrieval quality before the SQL writer gets the context.

DimensionWhat to checkEvidence
Metric meaningDoes the system retrieve the right definition?Definition ID, version, owner, and expected usage.
SynonymsDoes natural language map to the right field?Accepted terms, rejected terms, and examples.
Tenant overlayDoes the tenant-specific logic override the generic definition?Overlay version and tenant-specific examples.
Role contextDoes the retrieved meaning match the user's permission slice?Role replay and filtered result expectation.
Operating model

Treat definitions as controlled evidence.

The semantic layer should be editable by the right domain owner, versioned like code, and replayed against the golden set after every material change.

Owner

Every material metric needs a named business owner, not only a column description.

Version

Metric definitions should carry version history so old answers can be reconstructed.

Replay

Semantic edits should trigger golden-set replay for affected personas, tenants, and question tiers.

FAQ

Evaluate the semantic layer before SQL, answered plainly.

No. The schema names tables and columns. The semantic layer maps those objects to business meaning, metric definitions, synonyms, tenant logic, and role context.

Finance metrics often vary by business model or customer. A generic definition can be directionally right and still wrong for a specific tenant.

Create examples where the correct answer depends on a specific metric definition, synonym, tenant overlay, or role slice, then evaluate whether retrieval selects the expected context before SQL generation.

Further reading

Keep the evidence trail connected.

If a finance AI answer can move an operating decision, the evidence behind it needs to be readable after the answer is gone.

Bring one workflow, vendor, or AI portfolio. We will map the evidence needed for finance leaders to fund, ship, or stop it.

Evaluate the semantic layer ->