Set up guards

Guards are synchronous checks that run on the request path before an LLM call. Use them to block harmful prompts, redact sensitive data, or filter dangerous tool calls — all before the model sees the request.

Guards reuse the same evaluator types as online evaluation (LLM judge, regex, JSON schema, heuristic) but execute synchronously and return an allow/deny decision to the SDK.

Before you begin

Agent Observability is enabled on your Grafana Cloud stack and receiving generation data.
If you plan to use LLM judge guards, choose a provider and model available in your stack. Refer to Choose a judge provider.
You have the Agent Observability Admin role, or a custom Grafana role with grafana-agento11y-app.guards:write and the read access needed for evaluator testing.

How guards work

Your application SDK calls Agent Observability before each LLM invocation.
Agent Observability evaluates the request against enabled guard rules in priority order.
Each matching rule can redact the input with regex patterns, check tool calls against a block list, or run evaluators against the content.
The response tells the SDK whether to proceed (allow) or abort (deny), and optionally includes a sanitized copy of the input.

If the guard service is unreachable, the SDK proceeds by default (fail-open). You can change this to fail-closed in your SDK configuration.

Create a guard

Navigate to Evaluations > Guards in Agent Observability.
Click New guard.
Choose Create manually, Use a template, or Use Assistant.
Enter a guard ID (for example, pii_redaction or block_dangerous_tools).
Choose a guard type:

Evaluator guard

Runs one or more evaluators on the request content and denies or warns based on the result.

Select Evaluator as the guard type.
Attach one or more evaluators (existing evaluators from the Evaluators tab).
Set Action on fail to deny (blocks the request) or warn (logs but allows).

Redact guard

Applies regex patterns to redact content before the request reaches the model.

Select Redact as the guard type.
Add one or more patterns:
- Regex — the pattern to match (for example, \b\d{3}-\d{2}-\d{4}\b for US SSNs).
Matches are always replaced with the fixed token [REDACTED].
Redactions run before evaluators, so you can combine a redact guard with an evaluator guard on the same rule.

Ingest redaction

When the sigil.guards-ingest-redaction feature flag is enabled for your tenant, Agent Observability also applies enabled redact rules to generations and workflow steps before storing them. When viewing a redact guard in Evaluations > Guards, a note confirms that stored conversations are redacted too.

This keeps matched text out of storage even when callers don’t use the sanitized transformed_input payload, for example:

Coding-agent hosts, for example, Claude Code, Codex, or Cursor, that can’t rewrite prompts before the model sees them.
SDK adapters and integrations that record generation data before guard evaluation runs.
Orchestrators and applications that write custom workflow step state.

Ingest redaction includes the following behavior:

Redacted fields — Message text, thinking blocks, tool results, tool-call and tool-result JSON, system prompts, tool descriptions and schemas, conversation-title metadata, generation error messages, raw artifacts, and workflow step input state, output state, metadata, and error messages. Patterns match JSON string values after escapes are decoded. If redacting a value inside a JSON field would break JSON syntax, the field is replaced whole.
Broader rule scope — Ingest applies a redact rule more broadly than the request path does. It ignores the rule’s selector, so every turn is redacted, and it treats match conditions it can’t check at ingest as matching. For example, a condition on the tools used in a conversation always matches.
Untouched fields — Tags other than the marker tags, record IDs, agent and model identifiers, token usage, timestamps, and media parts.
Visual indicator — When ingest redacts content, it sets the tags agento11y.redaction.ingest=true and agento11y.redaction.rule_ids (listing the matched rule IDs). Only ingest sets these tags: if your application sends tags with these names, ingest removes them. In conversation explore, the generation header and workflow step views display a purple Redacted at ingest badge with an eye-slash icon. Hover over the badge to see which redact guards applied.
Rule caching and fail-closed protection — Redact rules are cached for 10 seconds, so rule changes apply to new data within about 10 seconds. Turning ingest redaction on or off can take up to 40 seconds. If the ingester can’t load your rules, it keeps applying the last rules it loaded. If it has none to apply, it fails closed: generations are rejected with the transient reason redaction_unavailable (returning a retryable HTTP 503 or gRPC Unavailable), and workflow steps are rejected per item. Unredacted content is never stored.
Scope limits — Ingest redaction doesn’t modify records stored before the feature was enabled, can’t change text the model provider already received, and never redacts tags (for example, a user email stored in a user.id tag).

Tool filter guard

Blocks specific tool calls by name using glob patterns.

Select Tool filter as the guard type.
Add blocked tool name patterns (for example, shell_exec, file_delete, Bash(*rm*)).
Patterns use * and ? wildcards. Patterns containing ( also match against tool call arguments, enabling argument-level filtering.

Configure priority and matching

Priority — guards run in ascending priority order. Lower numbers run first.
Selector — determines which requests the guard applies to. Default is all.
Match filters — narrow the guard to specific agents, models, or tags (for example, agent_name: ["my-agent"] or model.provider: ["openai"]). By default, a guard starts with no filters and displays Matches all agent traffic. Click Add filter to add conditions.

Configure your SDK

Enable hooks in your SDK configuration to start calling guards:

Setting	Default	Description
`hooks.enabled`	`false`	Enable guard evaluation.
`hooks.endpoint`	Agent Observability URL	Override the hooks endpoint.
`hooks.timeoutMs`	`5000`	Client-side timeout in milliseconds.
`hooks.failOpen`	`true`	Proceed with the LLM call if the guard request fails.
`hooks.phases`	`["preflight"]`	Which phases to evaluate.

When transformed_input is present in the guard response, the SDK uses the sanitized messages for the LLM call.

Example: Block PII and moderate content

A common setup combines a redact guard for PII redaction with an evaluator guard for content moderation:

Guard 1 (priority 1): Redact guard with regex patterns for SSN, email, and phone number redaction.
Guard 2 (priority 10): Evaluator guard with an LLM judge that checks whether the (now redacted) input is appropriate.

Guard 1 runs first and sanitizes the input. Guard 2 then evaluates the sanitized content. If the content is inappropriate, the request is denied.

Monitor guard activity

In Evaluations > Guards, open an evaluator or tool filter guard to review its activity over the last 24 hours. The page shows counts of matches, denials, warnings, and redactions, with a Denied conversations or Warned conversations list.

The list shows up to 100 recent decisions linked to conversations, including the agent, reason, and last activity. Click a row to inspect the conversation. You need permission to read conversations to view these links. Denials without a conversation ID still contribute to the counts, so an empty list doesn’t mean the guard hasn’t blocked requests.

Click Edit guard to change its configuration, or View guard if you have read-only access. Redact guards open their configuration directly.

Guard execution emits Prometheus metrics:

Request rate and latency — track how guards affect your request path latency.
Allow/deny rates — monitor how often guards block requests.
Per-rule evaluations — see which rules trigger most frequently.
Ingest redactions — when ingest redaction is active, monitor records modified at storage:
- agento11y_ingest_redactions_total — Count of records redacted at ingest, partitioned by kind (generation or workflow_step) and rule_id.
- agento11y_ingest_redaction_collapsed_json_total — Count of JSON fields replaced whole to preserve valid syntax.
- agento11y_ingest_redaction_duration_seconds — Latency of ingest redaction processing.
- agento11y_ingest_redaction_rule_load_errors_total — Failures loading redact rules during ingest.