Observability
Observability is used both for incident reconstruction and for continuous qualification. Production monitoring compares current behavior with the release baseline and slices failures by workflow, tool, rule, warehouse, exception class and model/policy version.
| Telemetry layer | Required fields | Operational use |
|---|---|---|
| Identity & intent | request_id, timestamp, user_id, role, intent, order/customer scope | Who asked, for what purpose and under which authority |
| Runtime & version | model_version, prompt_policy_version, workflow_version, tool_versions, rule_version | Compare behavior across releases and regressions |
| Evidence | source IDs, source versions, retrieved_at, effective_at, freshness state | Prove which operational facts supported the answer |
| Reasoning controls | rule IDs/results, calculation trace, abstention reason, recommendation set | Explain blocker classification and why options were permitted/withheld |
| Human control | proposed_action, approval_id, approved_by, approval_time | Tie protected action to accountable decision |
| Execution | pre-execution versions, execution result, verification result, errors | Detect stale state, partial failure and post-action mismatch |
| Performance | latency, tool latency, retry count, token/cost telemetry | Track reliability, cost and service performance |
| Outcome | user disposition, escalation, final exception state, reopen/rework signal | Connect AI behavior to business outcome |
| Monitoring view | Signal |
|---|---|
| Quality drift | Groundedness, blocker accuracy, abstention and evidence completeness by release/version |
| Data drift | Exception mix, warehouse/SKU patterns, missing fields and source freshness |
| Operational drift | Tool failures, latency, retries, API/schema changes |
| Policy drift | Customer-rule and authorization-policy version changes |
| Security | Denied tools/actions, approval failures, stale-state blocks, suspicious request patterns |
| Business | Investigation time, rework, escalation, exception ageing and adoption vs baseline |
Correlation trace records user role, order, SKUs, warehouses, tools, source versions, rule results, evidence references, recommendations, approval state and action status.
SOAI-REQ-004817SO-12547ORDER_LINE_SHORTAGE4NONENOT_REQUESTEDPrimewayz AI enablement for enterprise operations