# Metergrade documentation

> The Economic Control Check: an Azure Workbook and six KQL queries that
> run inside your own Log Analytics workspace against Azure API Management
> diagnostics. Nothing is transmitted to Metergrade, no account is required,
> and no credential is issued.

This file is generated from the same Markdown the HTML pages render, so the
two cannot disagree. Only capabilities that exist are documented; see
"What is not built yet" for an accounting of the rest.

## Contents

### Get started

- [Overview](https://metergrade.com/docs/overview) — What the Economic Control Check establishes, and what it deliberately does not.
- [Architecture](https://metergrade.com/docs/architecture) — The two Azure tables the check reads, why both are required, and where the boundary sits.
- [Quickstart](https://metergrade.com/docs/quickstart) — From nothing to a result, in the order that avoids the two failures people actually hit.

### Assess

- [Prerequisites](https://metergrade.com/docs/prerequisites) — The one diagnostic category that is off by default, and how to turn it on.
- [Reading the results](https://metergrade.com/docs/reading-the-results) — What each of the six queries reports, and the two conventions that keep a missing value from reading as a real one.
- [Supported schemas](https://metergrade.com/docs/supported-schemas) — Every column the kit reads, and the ones it will never read.

### Reference

- [Evidence rules](https://metergrade.com/docs/evidence-rules) — How the kit decides what a number is, and when it refuses to produce one.
- [Verifying the kit](https://metergrade.com/docs/verifying-the-kit) — Establish that the kit is what Metergrade published, without trusting anything Metergrade wrote.
- [What is not built yet](https://metergrade.com/docs/not-built) — The documented product architecture, and an honest line on every part of it that does not exist today.

---
# Overview

The Economic Control Check answers one question: **where does your Azure AI
spend go, and how much of it can you account for?**

It is an Azure Workbook and six KQL queries. They run inside your own Log
Analytics workspace, against diagnostics your API Management gateway already
emits. Nothing is transmitted to Metergrade, there is no account to create,
and no credential is issued.

## What it establishes

| Question | Answered by |
|---|---|
| Can this workspace answer the question at all? | Preflight |
| What is being consumed, and when? | Observed consumption by day |
| Where does consumption concentrate? | Model and deployment distribution |
| How much has no accountable owner? | Attribution gaps |
| How much was spent on work that failed? | Failure and retry waste |
| What is worth testing before changing anything? | Validation candidates |

## What it does not do

It does not change anything. Every query is read-only, and read access to the
workspace is sufficient throughout.

It does not identify retries. Proving that request B is a retry of request A
needs client-side correlation that no gateway log carries. What the log
supports is the waste retries produce — consumption under failed and
throttled requests — and that is what is reported.

It does not recommend a change. The validation-candidates query ranks; it
does not conclude. No threshold is applied, because the check does not know
your quality, latency or reliability requirements. Turning a candidate into a
verdict means testing it against those requirements, which is a deliberate
step and not something a workbook tile should imply it has done.

It does not price anything. The queries report tokens, not currency. Applying
your negotiated rates is a separate step, and a query that guessed at list
prices would produce a number that looks authoritative and is not.

## Where your data goes

Nowhere. The check has no upload step and makes no outbound call to
Metergrade. The kit is plain text you can read before you run it, and the
[verification procedure](/docs/verifying-the-kit) lets you confirm the files
are the ones Metergrade published without trusting anything we wrote.

Saving a baseline over time is a different mode with a different data
boundary, and it is [not built yet](/docs/not-built).

---

# Architecture

## The data path

```
your application
      │
      ▼
Azure API Management  ──emits──▶  Log Analytics workspace
      │                              │
      ▼                              │  ApiManagementGatewayLogs
Azure OpenAI                         │  ApiManagementGatewayLlmLog
                                     │
                                     ▼
                          the workbook and queries
                          (run here, results stay here)
```

Metergrade appears nowhere in that diagram. The check is a set of queries you
run against your own workspace; the kit is how you obtain them.

## Why two tables

Neither table alone can answer what a workload cost.

**`ApiManagementGatewayLogs`** carries request identity — which API, which
operation, which product and subscription, the response code and the timing.
It does not know anything about models or tokens.

**`ApiManagementGatewayLlmLog`** carries the economics — model, deployment,
prompt tokens, completion tokens. It identifies the request only by
`CorrelationId`. **It does not contain `ApiId` or `OperationId`.**

So attribution — connecting consumption to an accountable workload — is a
property of the join between them, on `CorrelationId`. Any query that tries
to read an API identity directly from the LLM table does not return an empty
result; it fails to resolve and does not run.

## One row per request, not per log record

`ApiManagementGatewayLlmLog` does not emit one record per request. Request,
response and streamed-chunk records are written separately under a single
`CorrelationId`.

Counting records therefore overstates traffic — on a streaming-heavy estate,
by a multiple — and summing token columns across those records double-counts
any value reported more than once.

Every query in the kit collapses the table to one row per `CorrelationId`
before it counts or sums anything. See [Evidence rules](/docs/evidence-rules)
for how a value is chosen when records disagree.

## The boundary, stated precisely

- The queries read metadata: identity, status, timing, model, deployment and
  token counts.
- They never read `RequestMessages`, `ResponseMessages`, `RequestBody` or
  `ResponseBody`. Prompt and completion content is not touched, and a test in
  the kit's build enforces that rather than documenting it.
- Results render in your Azure portal. There is no egress.

---

# Quickstart

Five steps. Steps 1 and 2 exist because they are where this goes wrong.

## 1. Confirm you can read the workspace

You need `Log Analytics Reader` on the workspace your API Management instance
sends diagnostics to. Nothing in this kit needs write access to anything.

## 2. Run the preflight query first

Download [`00-preflight.kql`](/setup/download/economic-control-check/queries/00-preflight.kql),
paste it into Log Analytics, and run it. It reports, for each of the two
required tables: `READY`, `PRESENT BUT EMPTY`, `PRESENT WITHOUT TOKEN DATA`,
or `NOT PRESENT`.

**Do this before anything else.** A workspace that has never received the LLM
diagnostic category does not hold an empty table — it holds no table, so
every later query fails to resolve rather than returning zero rows. That
reads as a broken kit and is not one. See [Prerequisites](/docs/prerequisites)
for how to switch the category on.

## 3. Import the workbook

Download [`workbook.json`](/setup/download/economic-control-check/workbook.json).

In the Azure portal: **Monitor → Workbooks → New → Advanced Editor** (the
`</>` icon), replace the contents with the file, then **Apply** and **Save**.

Pick your workspace and a time range in the parameter row at the top. Thirty
days is the default: long enough for weekday and weekend shape to appear,
short enough to sit inside common retention.

## 4. Work down the page

The tiles run in a deliberate order — baseline, then concentration, then the
three findings. Each states what it measured, not what you should do.

## 5. Read the results correctly

Two conventions run through every number. Read
[Reading the results](/docs/reading-the-results) before you quote any figure
to anyone, in particular the usage-coverage column — it tells you what share
of requests the totals actually rest on.

## Running without the workbook

Every query is also a standalone file that runs unchanged when pasted
straight into Log Analytics. Each declares its own 30-day window at the top,
which you can edit:

```kusto
let _startTime = ago(30d);
let _endTime   = now();
```

The workbook runs the same query bodies with the time range supplied by its
parameter instead. The workbook is generated from these files, so the two
cannot disagree.

## Handing it to an assistant

The kit ships `AGENT-INSTRUCTIONS.md`, written as an operator procedure
rather than a chat prompt: detect the environment, verify the data is there,
reveal what can be established, name what is missing, and act only on what
you approved. It is explicitly forbidden from changing routing, altering
endpoints, collecting prompts, or uploading anything.

[Download the agent instructions](/setup/download/instructions/AGENT-INSTRUCTIONS.md)

---

# Prerequisites

## What has to be true

| Requirement | Detail |
|---|---|
| API Management in front of your models | The check reads APIM gateway diagnostics. Traffic that bypasses the gateway is invisible to it. |
| Diagnostics sent to Log Analytics | Both categories below, pointed at a workspace. |
| `Log Analytics Reader` | On that workspace. Read access is sufficient throughout. |

## The two diagnostic categories

| Table | Category | Default |
|---|---|---|
| `ApiManagementGatewayLogs` | Gateway logs | Commonly already on |
| `ApiManagementGatewayLlmLog` | **LLM logs** | **Off** |

**The LLM category is the one that is usually missing.** It is enabled
separately from gateway logging, and without it there is no token data, so
there are no economics to establish.

## Turning on LLM logging

In the Azure portal: **your API Management instance → Monitoring →
Diagnostic settings → Add (or edit) a setting → select the LLM logs category
→ send to your Log Analytics workspace → Save.**

Then wait. Diagnostic data is not retroactive: the check can only see traffic
from the moment the category was switched on. If you enable it today, run the
preflight tomorrow and give it a few days before drawing conclusions about
weekly shape.

## Why the check reports this instead of assuming it

A workspace that has never received a category does not contain an empty
table. It contains no table at all — so a query against it fails to resolve
and returns an error rather than zero rows.

That error looks like a broken vendor kit. It is a configuration fact, it is
knowable before anything else runs, and reporting it is the preflight query's
entire job.

## What it costs

**Running the queries costs nothing.** Log Analytics does not charge for
queries on the Analytics plan — you are billed for what you ingest and
retain, not for reading it.

**Enabling the LLM logs category adds ingestion**, billed at Azure's
published Log Analytics rates like any other diagnostic data. These are
metadata rows — model, deployment and token counts — and they stay small
because capture of prompt and completion content is a separate setting the
kit tells you to leave off.

This page will not quote you a monthly figure. It would be wrong for your
commitment tier, your region and your retention, and a number we cannot
stand behind is what this kit exists not to produce. Measure it against your
own workspace:

```kusto
Usage
| where TimeGenerated > ago(30d)
| where IsBillable
| where DataType in ("ApiManagementGatewayLogs", "ApiManagementGatewayLlmLog")
| summarize IngestedGB = sum(Quantity) / 1024 by DataType, bin(TimeGenerated, 1d)
| order by TimeGenerated asc
```

Multiply the daily figure by your workspace's rate in the Azure pricing
calculator.

## If your estate is very large

Preflight reports how many requests each table holds in the selected window.
On a high-volume estate, narrow the window before running the heavier
queries. A query that reaches the Log Analytics result limit returns a
truncation error, and that error reads as a broken kit rather than as an
estate too big for a thirty-day pass.

Start at seven days, confirm the shape, then widen. Nothing in the check
depends on a particular window length: every query declares its own at the
top, and the workbook takes it from the time range you pick.

## Removing it afterwards

Delete the imported workbook. If you enabled a diagnostic setting only for
this check, revert it. Nothing else was changed.

---

# Reading the results

## The two conventions

Both exist for the same reason: a number you cannot support is worse than no
number.

### One row per request, not per log record

The LLM table writes request, response and streamed-chunk records separately
under a single `CorrelationId`. Every query collapses to one row per request
before aggregating. A "request" in any result below means a logical request,
not a log line.

### Disagreement reads as absent

Where several records report different values for the same field, the queries
return **nothing** for it rather than picking one. Where no record reported a
value, the result is likewise absent, not zero.

This is why `UsageCoveragePct` exists. Below 100%, the token totals rest on
that share of requests and are **a floor, not an estimate of the remainder**.
Quote the coverage whenever you quote a total.

## The queries

### 00 · Preflight

Reports per table: `READY`, `PRESENT BUT EMPTY`, `PRESENT WITHOUT TOKEN DATA`,
or `NOT PRESENT`. Anything other than `READY` on both means the later results
are not trustworthy yet. See [Prerequisites](/docs/prerequisites).

### 01 · Observed consumption by day

Daily request volume and token consumption.

`UsageCoveragePct` is the share of that day's requests the totals are derived
from. A day showing high volume and low coverage is a day where the token
telemetry is incomplete — not a cheap day.

### 02 · Where consumption concentrates

Requests and tokens by model and deployment, with each row's share of total
volume. Values that were never reported, or that records disagreed on, group
under `(not reported)` rather than being dropped — dropping them would shrink
the denominator and overstate everything else.

### 03 · Attribution gaps

How much observed consumption cannot be tied to an accountable API and
operation. Five outcomes:

| Result | Meaning |
|---|---|
| `ATTRIBUTED — API, operation and product` | Fully accounted for |
| `ATTRIBUTED — API and operation, no product` | Accountable, no product dimension |
| `PARTIAL — API known, operation not identified` | Half attributed |
| `UNATTRIBUTED — gateway record carries no API or operation identity` | The gateway saw it and could not name it |
| `UNATTRIBUTED — no gateway record for this model call` | A model call with no matching gateway record at all |

The unattributed rows stay in the denominator. That is the point of the
query: the share you cannot account for is the finding.

### 04 · Failure and retry waste

Consumption grouped by request outcome — succeeded, 4xx, 429 throttled, 5xx,
and indeterminate where gateway records disagree on status.

Tokens consumed under a 5xx were paid for and thrown away. That is exact.

429s are reported as throttling pressure, which is what drives client retries
— **not** as a retry count. `TokensConsumed` is absent, not zero, for any
outcome where no request reported usage: a throttled request that never
reached a model has no consumption to report, and "we have no telemetry" must
not render as "this cost nothing".

### 05 · Validation candidates

Operations ranked by input context paid for relative to output produced,
highest total input first.

A high input-to-output ratio is the signature of context that may not be
earning its cost — retrieved documents that go unused, prompts that
accumulated, history replayed every turn.

**It is a candidate and nothing more.** Whether that context is waste or the
reason the output is correct cannot be established from a log. No threshold
is applied. Ranking is by total input tokens rather than by ratio, so a
trivial operation with an extreme ratio does not outrank a dominant one.

Operations that produced no output at all show an absent ratio rather than
infinity.

---

# Supported schemas

The kit reads two Azure Monitor tables. The columns below are the complete
set it touches — verified against the published Azure Monitor table
reference by a test that parses every shipped query against those schemas.

## `ApiManagementGatewayLogs`

Request identity, status and timing.

| Column | Used for |
|---|---|
| `TimeGenerated` | Window filter, request time |
| `CorrelationId` | Join key; the identity of one logical request |
| `ApiId` | Attribution |
| `OperationId` | Attribution |
| `ProductId` | Attribution depth |
| `ApimSubscriptionId` | Export only |
| `UserId` | Export only |
| `WorkspaceId` | Export only |
| `ResponseCode` | Outcome classification |
| `TotalTime` | Latency, export only |
| `BackendUrl` | Backend identity, export only |
| `_ResourceId` | Service identity, export only |

## `ApiManagementGatewayLlmLog`

Model, deployment and token counts.

| Column | Used for |
|---|---|
| `TimeGenerated` | Window filter |
| `CorrelationId` | Join key |
| `ModelName` | Concentration, candidates |
| `DeploymentName` | Concentration |
| `PromptTokens` | Input consumption |
| `CompletionTokens` | Output consumption |
| `TotalTokens` | Total consumption |
| `SequenceNumber` | Export only |
| `RequestId` | Provider-side request id, export only |

**This table has no `ApiId` and no `OperationId`.** The closest thing it
carries is `OperationName`, which is not the operation identity used for
attribution. See [Architecture](/docs/architecture).

## Never read

`RequestMessages`, `ResponseMessages`, `RequestBody`, `ResponseBody`.

Prompt and completion content is never touched by any query in the kit. This
is enforced by a test over the shipped query files rather than left as a
promise in prose.

## Four identifiers, never collapsed into one

The export path distinguishes these deliberately, because conflating them
produces confident nonsense:

| Identifier | What it is |
|---|---|
| `CorrelationId` | Azure/APIM correlation — the identity of one logical request |
| `ProviderRequestId` | The language model's own request id. Provider-side evidence, not the APIM request |
| `source_group_key` | Metergrade's analysis grouping, derived inside Metergrade |
| `ai_workload_id` | Confirmed business attribution. Never present in a log |

---

# Evidence rules

Four rules govern every figure the check reports. They exist because a wrong
number is worse than a missing one: a missing number prompts a question, and
a wrong one gets acted on.

## 1. One row per request

The LLM table emits several records per logical request. Every query collapses
to one row per `CorrelationId` before counting or summing.

Without this, `count()` returns log lines rather than requests — inflated by
a multiple on a streaming estate — and `sum()` double-counts any token value
that appears on more than one record.

## 2. A value is taken only where the records agree

Collapsing uses `make_set(x, 2)`, which collects up to two distinct non-null
values:

| Distinct values | Meaning | Result |
|---|---|---|
| 1 | The records agree | The value is taken |
| 0 | Nothing was reported | Absent |
| more than 1 | The records conflict | Absent, conflict flagged |

Conflict is never resolved by choosing. In the normal streaming case only the
terminal record carries usage, so exactly one distinct value exists and it is
taken unambiguously. If several records report *different* counts, no
aggregate is defensible — summing double-counts a cumulative counter, and
taking the maximum understates true deltas — so the row's tokens are refused
rather than invented.

## 3. Absent is not zero

Kusto's `sum()` returns `0`, not null, for a group in which every value is
null. Left alone, that turns "no request here reported usage" into "this
consumed nothing".

For a throttled request that never reached a model, zero happens to be true.
For a group whose requests predate the LLM diagnostic category being switched
on, it is false in the worst direction — it reads as *these failures cost
nothing* instead of *we have no telemetry for these failures*.

So a token total resting on zero reporting requests is returned absent. A bar
at zero reads as measured absence of spend; a gap reads as absence of
evidence, which is the fact.

## 4. The denominator keeps what it cannot explain

Requests that cannot be attributed stay in the total. Values that were never
reported group under `(not reported)` rather than being dropped. Dropping the
inconvenient rows shrinks the denominator and silently improves every
percentage above it.

## Reading coverage

`UsageCoveragePct` is the share of requests in a group whose token totals
could be established. Below 100%, the totals are **a floor**. They are not an
estimate of the remainder, and they should not be scaled up to guess at it.

---

# Verifying the kit

The kit is published as a signed release. The signature is **Sigstore
keyless**: there is no Metergrade key to trust, and the signing identity is
the release workflow itself, recorded in the certificate.

## The trust chain, in the only direction it works

```
Sigstore identity  authenticates  manifest.json
manifest.json      authenticates  the verifier and every kit artifact
the verifier       checks         the complete kit policy
```

The bundled verifier is convenience. **Sigstore and the signed manifest are
the authority.**

## Normal use

```
./verify              (macOS, Linux)
verify.cmd            (Windows)
```

Needs Node 18+ and [cosign](https://github.com/sigstore/cosign).

| Exit | Meaning |
|---|---|
| `0` | Verified. Signed by the Metergrade release, and the files match the signed manifest. |
| `1` | **Do not run this kit.** It is not what it claims to be. |
| `3` | Contents are internally consistent, but the signature was not checked. Nothing shows Metergrade produced it. |

A missing cosign is a failure, not a skip. A check that turns "could not run"
into success is how a green result comes to mean nothing.

## High assurance: do not start by running our code

Establish the manifest first, then use it to establish the verifier, then run
it.

**1. cosign authenticates the manifest.**

```
cosign verify-blob \
  --signature manifest.json.sig \
  --certificate manifest.json.pem \
  --certificate-identity-regexp '^https://github\.com/Metergrade/metergrade-platform/\.github/workflows/release-setup-kit\.yml@refs/' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  manifest.json
```

Do not relax the identity to a wildcard. That accepts a signature from
anyone, and proves only that a signature exists.

**2. The authenticated manifest establishes the verifier.** Compare the
`metergrade-verify.mjs` entry in `manifest.json` with the file on disk:

```
sha256sum metergrade-verify.mjs
```

**3. Now run it.**

```
node metergrade-verify.mjs .
```

Or skip step 3 entirely — `manifest.json` lists every artifact with its
`sha256`, and `checksums.sha256` is the same data in `sha256sum -c` form. Two
things that shortcut does not do, and the verifier does: confirm that no file
is present which the manifest does **not** list, and recompute `content_hash`
from the files rather than reading it back out of the manifest.

## Two digests, two different claims

| Digest | What it proves |
|---|---|
| `content_hash` | The kit's identity. A digest over sorted path and hash pairs, so it reproduces on any machine from the contents alone. This is what the signature binds. |
| `archive_sha256` | Only that the download arrived intact. A ZIP embeds timestamps and ordering, so two honest builds of the same contents differ. |

Conflating them would let a correct-looking checksum vouch for the wrong
contents.

## What the signature covers is what this site serves

Every file offered on [/setup](/setup) is covered by the signed manifest at
the same `sha256`. The publishing step refuses to ship a release whose
manifest disagrees with the queries the site renders, and a test asserts the
same thing against the live site — because a signature over something other
than what you just read would be worse than no signature at all.

## Download

- [Signed archive](/setup/download/release/metergrade-setup-kit-v1.0.0.zip)
- [manifest.json](/setup/download/release/manifest.json)
- [manifest.json.sig](/setup/download/release/manifest.json.sig)
- [manifest.json.pem](/setup/download/release/manifest.json.pem)
- [checksums.sha256](/setup/download/release/checksums.sha256)

---

# What is not built yet

Metergrade's architecture describes five capabilities — Observe, Analyze,
Validate, Deploy, Govern — and a documentation structure covering all of
them. **Most of it is not built.**

This page exists so that gap is stated in one place rather than discovered
one dead link at a time. If a capability is not listed as available below, it
does not exist yet, whatever a diagram elsewhere might suggest.

Nothing here carries a date. A date would be a commitment, and the honest
position is a status.

## Available today

| Capability | State |
|---|---|
| Economic Control Check | **Available.** No account, nothing transferred. Documented in these pages. |
| Signed, verifiable kit distribution | **Available.** Sigstore keyless signing, published at metergrade.com. |
| Assisted Assessment (service) | **Available.** A US$5,000 fixed engagement — people, not software. |

## Not built

### Saving a baseline · Metergrade Free

An account, an Organization, and analysis that persists across sessions.
Requires you to upload an export you produce locally, after signing in.

**State:** not built. There is no console and no sign-in. `/start` says so
rather than linking to a host that does not resolve. The export query exists
and runs, but there is nowhere to send its output.

### Connect — continuous ingestion

Event Hub, the forwarder, authentication, reconciliation.

**State:** not built. The check is a point-in-time query you run yourself.
Nothing watches your estate continuously. The kit's agent instructions
explicitly forbid installing continuous ingestion, because it is a later
milestone and not part of setup.

### Control — workloads, ownership, economic standards

Naming a workload, assigning an accountable owner, attaching business
outcomes, and defining the economic standard a workload is expected to
satisfy.

**State:** not built. The check reports attribution *gaps* — consumption with
no accountable API or operation behind it — but there is nowhere to resolve
one. Resolving it today means changing how your callers identify themselves
at the gateway.

### Prove — validation and verdicts

Testing a candidate change against your own quality, latency and reliability
requirements, returning PASS / FAIL / INCONCLUSIVE with the evidence.

**State:** not built. This is the centre of the product architecture and the
point at which Metergrade charges, and it does not exist yet. Query 05 ranks
candidates worth testing; it applies no threshold and reaches no verdict,
because there is no validation engine behind it.

### Deploy — configuration artifacts

APIM policy, Bicep, Terraform, CI/CD integration, attestation, rollback.

**State:** not built. Metergrade emits nothing deployable today.

### Maintain — drift, invalidation, revalidation

Continuous monitoring of whether a validated result still holds.

**State:** not built. Depends on continuous ingestion and on validation,
neither of which exists.

### Administer — organizations, roles, retention, audit, billing

**State:** not built. There is no multi-user surface, so there is nothing to
administer. Team and Foundation plans cannot be bought; `/pricing` says so.

### Reference — API, events, errors

**State:** not built. There is no public API and no event stream.

## Documentation that follows from that

The ratified documentation structure names Get started, Assess, Connect,
Control, Prove, Deploy, Maintain, Administer and Reference.

**Get started, Assess and Reference are written**, because the Economic
Control Check is real and you can run it today.

The rest are absent rather than stubbed. A page reading "coming soon" tells a
reader nothing they could not infer from its absence, and thirty of them make
a product look unfinished while adding no information. When a capability
ships, its documentation ships with it.

## Where the boundary sits commercially

Discovering the economic state of an estate is free and always will be. The
Economic Control Check is that surface, and it is complete.

Payment begins at Validate — which is the part that is not built. Metergrade
is not currently able to charge for the thing it intends to charge for, and
saying so plainly is cheaper than being found out.

---