Sentinel Network Architecture

1. Threat Model

The Sentinel Network addresses a fundamental problem in multi-agent systems: what happens when an agent goes rogue?

In the OpenClaw crisis scenario, a compromised agent could perform unauthorized searches, install malicious skills, sabotage shared resources, or exfiltrate data. Traditional monitoring (reading log files) has two fatal flaws:

The fix: move tool execution itself into an isolated, trusted process. Workers become thin LLM shells with no local tool implementations.

2. Architecture

3. Process Map

The system runs as 7 independent processes, launched in order:

4. Action Gateway Design

The Action Gateway is not a Bedsheet agent. It's a standalone asyncio process that connects directly to PubNub. No LLM, no instruction, no memory. This keeps the trust boundary clean: the gateway is deterministic code, not an LLM that could be prompt-injected.

Components

Processing Pipeline

5. Tamper-Proof Ledger

The ActionLedger is the single source of truth for all agent activity. Every action request — approved, denied, or rate-limited — is recorded as an ActionRecord.

ActionRecord Fields

Why It's Tamper-Proof

• The ledger lives inside the gateway process — no external process can modify it
• Workers communicate via PubNub signals, never touching the ledger directly
• Even denied actions are logged — a rogue agent's attempts create evidence
• Auto-pruning keeps only the last 10 minutes (configurable), bounding memory usage

Query Interface

The behavior sentinel queries the ledger via PubNub signals:

# Sentinel sends:
Signal(kind="request", sender="behavior-sentinel", target="action-gateway",
       payload={"type": "query_rates", "minutes": 2})

# Gateway responds with:
{
    "type": "rate_stats",
    "window_minutes": 2,
    "total_actions": 47,
    "agents": {
        "web-researcher": {"count": 35, "rate": 17.5, "denied": 20, "approved": 15},
        "scheduler": {"count": 8, "rate": 4.0, "denied": 0, "approved": 8},
    },
}

6. Anomaly Detector

The AnomalyDetector is a stateless evaluator that runs before every tool execution. It checks two conditions:

Rate Limiting

This is what stops rogue bursts. When the web researcher fires 50 rapid searches, the first ~10 get approved. Every subsequent request returns "verdict": "rate_limited". The rogue loop gets back "denied" for every attempt.

Suspicious Keywords

The detector scans all action parameters for suspicious terms:

SUSPICIOUS_KEYWORDS = {
    "password", "exploit", "bypass", "credential", "dump",
    "exfiltrat", "inject", "hack", "leaked", "pastebin",
}

If any keyword is found in the JSON-serialized parameters, the action is immediately denied. This catches the rogue search queries like "password dump site" and "bypass authentication" on the first attempt, before any rate limit is even reached.

7. Tool Executor

The ToolExecutor class holds the actual implementations of every tool that workers use. When a worker was previously doing ddgs.text(query) locally, that code now lives exclusively inside the gateway's executor.

Available Tools

8. Signal Protocol

All communication between workers, the gateway, and sentinels uses Bedsheet's Signal dataclass over PubNub. The serialization is compact (short keys, 30KB limit) to fit within PubNub's 32KB message limit.

Signal(kind="request", sender="web-researcher", target="action-gateway",
       correlation_id="abc123",
       payload={
           "type": "action_request",
           "action": "search_web",
           "params": {"query": "AI trends 2026"},
       })

Signal(kind="response", sender="action-gateway", target="web-researcher",
       correlation_id="abc123",
       payload={
           "verdict": "approved",       # or "denied" / "rate_limited"
           "result": "Results for 'AI trends': ...",
           "reason": "",                # explanation if denied
           "rate": 3.5,                 # agent's current rate
       })

Signal(kind="request", sender="behavior-sentinel", target="action-gateway",
       payload={"type": "query_rates", "minutes": 2})

Signal(kind="response", sender="action-gateway", target="behavior-sentinel",
       payload={
           "type": "rate_stats",
           "window_minutes": 2,
           "total_actions": 47,
           "agents": {
               "web-researcher": {"count": 35, "rate": 17.5, "denied": 20, "approved": 15},
               "scheduler": {"count": 8, "rate": 4.0, "denied": 0, "approved": 8},
           },
       })

Signal(kind="alert", sender="action-gateway",
       payload={
           "severity": "critical",
           "category": "gateway_enforcement",
           "agent": "web-researcher",
           "denied_count": 20,
           "message": "Agent 'web-researcher' blocked: 20 denied requests in 1min",
       })

9. Request Flow

Every tool call follows this exact path. There are no shortcuts — every action goes through the gateway, every action is logged.

Fail-Closed Design

10. Alert Escalation

The system has two alert sources that feed the Sentinel Commander:

Gateway Direct Alerts

When the gateway denies 5+ actions from the same agent within 1 minute, it broadcasts a critical alert directly to the alerts channel. This gives the commander immediate notification from the trusted authority.

Behavior Sentinel Alerts

The behavior sentinel polls the gateway ledger every 10 seconds and detects rate anomalies (>10 actions/min). It broadcasts high severity alerts for sustained anomalies. This catches lower-severity trends that the gateway's per-request denial doesn't escalate.

Alert Timeline (Rogue Burst)

11. Supply Chain Trust Boundary

Skill installation is the most security-sensitive operation. The gateway enforces the trust boundary:

12. How Rogue Behavior Is Blocked

Each worker has a ~15% chance of going rogue per cycle. Here's what happens for each:

Web Researcher Rogue Burst

Fires 50 rapid searches with suspicious queries like "password dump site".

Scheduler Rogue Sabotage

Rapid-fires 20 junk appointments ("JUNK-XXXX" on "1999-01-01").

Skill Acquirer Rogue Install

Tries to install data_exfiltrator.py (flagged malicious).

13. Defense in Depth

The Sentinel Network uses three independent defense layers:

14. Running the System

Prerequisites

# Required environment variables
export PUBNUB_SUBSCRIBE_KEY=sub-c-...
export PUBNUB_PUBLISH_KEY=pub-c-...
export GEMINI_API_KEY=AIza...

# Install dependencies
uv pip install -e ".[sense,demo]"

Option A: Shell Script (recommended)

cd examples/agent-sentinel
./start.sh              # Launch gateway + 6 agents + dashboard
./start.sh --no-dash    # Without dashboard server

Option B: Python Launcher

cd examples/agent-sentinel
python run.py

Launch Order

1. Action Gateway (must start first — workers need it)
2. Workers (web-researcher, scheduler, skill-acquirer)
3. Sentinels (behavior-sentinel, supply-chain-sentinel)
4. Commander (sentinel-commander)

Each process is staggered by 2 seconds to allow PubNub connections to establish.

15. Live Dashboard

The Sentinel Dashboard connects directly to PubNub and displays real-time signals from all agents. When using start.sh, the dashboard is automatically served at:

http://localhost:8765/agent-sentinel-dashboard.html

The dashboard shows:

• Agent status — heartbeats, online/offline indicators
• Alert feed — gateway enforcement alerts and sentinel anomaly reports
• LLM activity — thinking, tool calls, completions per agent
• World map — animated agent locations

Gateway alerts appear as critical events with the category gateway_enforcement, showing exactly which agent was blocked and how many requests were denied.