Read it before you run it.

Before you start

The check reads two Azure Monitor tables, both produced by Azure API Management diagnostic settings pointed at a Log Analytics workspace.

ApiManagementGatewayLogs

Request identity, operation, status, latency. Produced by the gateway logs category.

ApiManagementGatewayLlmLog

Model, deployment, token counts. Produced by the LLM logs category, which is separate and off by default. This is the one that is usually missing.

A workspace that has never received the LLM category does not hold an empty table — it holds no table, so a query against it fails to resolve rather than returning zero rows. That reads as a broken kit and is not one. Run the preflight query first: it reports, for each table, whether it is absent, empty, present without token data, or ready.

Read access is enough throughout. Nothing in this kit needs write access to anything.

What this costs

Running the queries costs nothing. Log Analytics does not charge for queries on the Analytics plan. You are billed for what you ingest and retain, not for reading it.

Enabling the LLM logs category adds ingestion, billed at Azure’s published Log Analytics rates like any other diagnostic data. These are metadata rows — model, deployment and token counts. They stay small because capture of prompt and completion content is a separate setting, and the kit tells you to leave it off.

We are not going to quote you a monthly figure. It would be wrong for your commitment tier, your region and your retention, and a number we cannot stand behind is the thing this kit exists not to produce. Measure it against your own workspace instead:

Usage
| where TimeGenerated > ago(30d)
| where IsBillable
| where DataType in ("ApiManagementGatewayLogs", "ApiManagementGatewayLlmLog")
| summarize IngestedGB = sum(Quantity) / 1024 by DataType, bin(TimeGenerated, 1d)
| order by TimeGenerated asc

Multiply the daily figure by your workspace’s rate in the Azure pricing calculator. It is reversible: turn the category off and the ingestion stops.

If your estate is very large

Preflight reports how many requests each table holds in the window you selected. On a high-volume estate, narrow the window before running the heavier queries. A query that reaches the Log Analytics result limit returns a truncation error, and that error reads as a broken kit rather than as an estate too big for a thirty-day pass.

Start at seven days, confirm the shape, then widen. Nothing in the check depends on a particular window length — every query declares its own at the top, and the workbook takes it from the time range you pick.

The queries

Each file runs unchanged when pasted straight into Log Analytics — it declares its own 30-day window at the top, which you can edit. The workbook runs the same query bodies with the time range supplied by its parameter instead, and is generated from these files, so the two cannot disagree.

How to read the numbers

Two conventions run through every query, and both exist to keep a missing value from being reported as a real one.

One row per request, not per log record. The LLM table writes request, response and streamed-chunk records separately under a single correlation id. Counting records overstates traffic — on a streaming-heavy estate, by a multiple — so every query collapses the table to one row per request before it counts or sums anything.

Disagreement reads as absent. Where several records report different values for the same field, the queries return nothing for it rather than picking one. A usage-coverage figure below 100% means the totals rest on that share of requests and are a floor, not an estimate of the remainder.

The validation-candidates query ranks; it does not conclude. No threshold is applied, because the kit does not know your quality, latency or reliability requirements — and a query that invented one would be recommending a change nobody tested.

Verifying this kit

This kit is published as a signed release. The signature is Sigstore keyless — there is no Metergrade key to trust, and the signing identity is the release workflow itself, recorded in the certificate.

content_hash

edee6acd37c6225e2cdcc709d780c8dbd7d2dc884e45d50bc233d2a04d5755bc
The kit’s identity — a digest over sorted path and hash pairs, so it reproduces on any machine from the contents alone. This is what the signature binds.

archive_sha256

cdfda7b3316d582f0a1e7c606c99f0025add5734a3bc964ea9b4ed7de20f1ed4
Verifies only that the download arrived intact. It is not the kit’s identity: a ZIP embeds timestamps and ordering, so two honest builds of the same contents differ.

Every file the signature covers is a file this page serves. The build refuses to publish a release whose manifest disagrees with the queries above, because a signature over something other than what you just read would be worse than no signature at all.

Verify without running anything of ours — establish the manifest first, then use it to establish the verifier:

cosign verify-blob \
  --signature manifest.json.sig \
  --certificate manifest.json.pem \
  --certificate-identity-regexp '^https://github\.com/Metergrade/metergrade-platform/\.github/workflows/release-setup-kit\.yml@refs/' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  manifest.json

Do not relax the identity to a wildcard. That accepts a signature from anyone, and proves only that a signature exists. The full procedure, including the checksum-only path that runs none of our code, is in the setup instructions below.

Setup instructions

Shipped in the kit as instructions/SETUP.md. Rendered here from the same file, so this page cannot describe the kit differently from how the kit describes itself.

# Metergrade Setup Kit

This kit checks the economics of your Azure API Management AI traffic. It
is plain text, KQL, and a workbook definition — read it before running it.

## What this is not

No installer. No daemon. No agent runtime. No outbound network calls to
Metergrade. No credentials.

## Two ways to use it

**Economic Control Check** (no account, nothing transferred) — import the
workbook, run the queries, read the results in your own Azure environment.

**Metergrade Free** (saved baseline) — produce a supported export locally,
sign in to Metergrade, and upload it yourself.

## Before you start: what has to be switched on

The check reads two Azure Monitor tables, and both are produced by APIM
diagnostic settings pointed at a Log Analytics workspace:

| Table | Carries | Diagnostic category |
|---|---|---|
| `ApiManagementGatewayLogs` | request identity, operation, status, latency | Gateway logs |
| `ApiManagementGatewayLlmLog` | model, deployment, token counts | **LLM logs — separate, and off by default** |

**The LLM category is the one that is usually missing.** It is enabled
separately from gateway logging, and a workspace that has never received it
does not contain an empty table — it contains no table at all, so a query
against it fails to resolve rather than returning zero rows.

That is a configuration fact, not a broken kit, and it is knowable before
you run anything. `queries/00-preflight.kql` reports it directly. Run that
first. It tells you, for each table: not present, present but empty,
present without token data, or ready.

You also need read access to the workspace (`Log Analytics Reader` is
sufficient). Nothing here needs write access to anything.

## What you run

`economic-control-check/workbook.json` is an Azure Workbook. Import it,
pick your workspace and a time range, and work down the page.

Every query is also a standalone file under
`economic-control-check/queries/`, and each one runs unchanged when pasted
straight into Log Analytics — they declare their own 30-day window at the
top, which you can edit. The workbook runs the same query bodies with the
time range supplied by its parameter instead. The workbook is generated
from these files, so the two cannot disagree.

| Query | Answers |
|---|---|
| `00-preflight.kql` | Can this workspace answer the question at all? |
| `01-observed-spend.kql` | What is the daily request volume and token consumption? |
| `02-model-distribution.kql` | Where does consumption concentrate, by model and deployment? |
| `03-attribution-gaps.kql` | How much consumption has no accountable workload behind it? |
| `04-retry-and-failure-waste.kql` | How much was consumed by requests that did not succeed? |
| `05-validation-candidates.kql` | Which operations are worth testing before anything changes? |

### How to read the numbers

Two conventions run through every query, and both exist to keep a missing
value from being reported as a real one.

**One row per request, not per log record.** `ApiManagementGatewayLlmLog`
writes request, response and streamed-chunk records separately under a
single `CorrelationId`. Counting records overstates traffic — on a
streaming-heavy estate, by a multiple — so every query collapses the table
to one row per request before it counts or sums anything.

**Disagreement reads as absent.** Where several records report different
values for the same field, the queries return nothing for it rather than
picking one. A `UsageCoveragePct` below 100 means the totals rest on that
share of requests and are a floor, not an estimate of the remainder.

`05-validation-candidates.kql` ranks; it does not conclude. No threshold is
applied, because this kit does not know your quality, latency or
reliability requirements — and a query that invented one would be
recommending a change nobody tested.

## With an AI agent

Hand `AGENT-INSTRUCTIONS.md` to the assistant you already use. It runs in
your environment with your credentials.

## Without an AI agent

Every step is human-followable: `economic-control-check/` holds the
workbook and queries, `metergrade-free/export-apim.md` describes the
export.

## Verifying this kit

The trust chain, in the only direction it works:

    Sigstore identity  authenticates  manifest.json
    manifest.json      authenticates  the verifier and every kit artifact
    the verifier       checks         the complete kit policy

The verifier bundled here is convenience. **Sigstore and the signed
manifest are the authority.** There is no Metergrade key, no Metergrade
endpoint, and nothing to trust that you cannot check with standard tools.

### First, check what this copy actually contains

    ls manifest.json manifest.json.sig manifest.json.pem

A published Metergrade release contains all three. **If the `.sig` and
`.pem` are absent, this copy did not come from the signing release**, and
`./verify` will FAIL it — exit 1, "do not run this kit" — rather than pass
it with a caveat. That is deliberate. An unsigned kit is an unauthenticated
kit, and a verifier that waved one through would be worth nothing.

If you are holding such a copy, its contents can still be checked for
internal consistency with `./verify --no-signature` (exit 3), but nothing
about that check demonstrates Metergrade produced it. Obtain a signed
release instead.

### Normal use

    ./verify              (macOS, Linux)
    verify.cmd            (Windows)

Needs Node 18+ and [cosign](https://github.com/sigstore/cosign). It prints
a report and exits:

| Exit | Meaning |
|---|---|
| `0` | Verified. Signed by the Metergrade release, and the files match the signed manifest. |
| `1` | **Do not run this kit.** It is not what it claims to be. |
| `3` | Contents are internally consistent, but the signature was not checked (`--no-signature`, cosign missing, or an unsigned copy). Nothing here shows Metergrade produced it. |

A missing cosign is a failure, not a skip. A check that turns "could not
run" into success is how a green result comes to mean nothing.

Optional: `./verify --archive path/to/kit.zip` also checks the downloaded
archive against the published `archive_sha256`.

### High assurance

If you are auditing, do not start by running our code. Establish the
manifest first, then use it to establish the verifier, then run it.

**1. cosign authenticates the manifest.** The identity is pinned to the
release workflow. Do not relax it to a wildcard — that accepts a signature
from anyone, and proves only that a signature exists.

    cosign verify-blob \
      --signature manifest.json.sig \
      --certificate manifest.json.pem \
      --certificate-identity-regexp '^https://github\.com/Metergrade/metergrade-platform/\.github/workflows/release-setup-kit\.yml@refs/' \
      --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
      manifest.json

**2. The authenticated manifest establishes the verifier.** Compare the
`metergrade-verify.mjs` entry in `manifest.json` with the file on disk:

    sha256sum metergrade-verify.mjs

**3. Now run it.**

    node metergrade-verify.mjs .

Or skip step 3 entirely: `manifest.json` lists every artifact with its
`sha256`, and `checksums.sha256` is the same data in `sha256sum -c` form.
Two things that check does not do for you, and the verifier does — check
that no file is present which the manifest does **not** list (every
declared hash still matches when something undeclared has been added), and
recompute `content_hash` from the files rather than reading it back out of
the manifest.

### What the archive checksum is not

`archive_sha256` in `RELEASE.json` verifies only that the ZIP bytes arrived
intact. It is not the kit's identity — a ZIP embeds timestamps and
ordering, so two honest builds of the same contents produce different ZIP
bytes. `RELEASE.json` is published beside the archive rather than inside
it, since it records that archive's own digest.

`content_hash` in `manifest.json` is the identity. It is a digest over
sorted path and hash pairs, so it reproduces on any machine from the
contents alone.

## Removing it

Delete the imported workbook and, if you enabled diagnostic settings only
for this check, revert them. Nothing else was changed.

Handing this to an assistant

The kit ships agent instructions written as an operator procedure: detect the environment, verify the data is there, reveal what can be established, name what is missing, and act only on what you approved. It runs in your environment with your credentials, and it is explicitly forbidden from changing routing, altering endpoints, collecting prompts, or uploading anything.