Skip to main content

FAQ

Documented for Python SDK token-sentinel 1.0.3.

Product-facing answers (keys, outages, vs observability / gateways) also appear on the marketing site: tokensentinel.dev/faq.

Open source SDK vs paid cloud​

Open-source SDK (token-sentinel)TokenSentinel Cloud (separate product)
LicenseApache-2.0Proprietary
What it isIn-process rules + provider wrappersHosted API, dashboard, policy, Pro features
CostFree foreverPaid tiers (Team / Pro / Enterprise)
NetworkNothing leaves the process by defaultOnly if you set cloud_endpoint + api_key
Where docs livedocs.tokensentinel.dev (this site) + SDK GitHubProduct site + tokensentinel.dev (Cloud docs coming soon)

Strategy in one line: detection logic is open so teams can audit and self-host the agent-side library; dashboards, multi-tenant policy, retention, and calibration are the commercial cloud.

What the free SDK includes​

  • All 15 deterministic waste rules (chat/agent, vision, audio, voice, rerank, repair)
  • Native wrappers: Anthropic, OpenAI (+ compatible), Gemini/Vertex, Bedrock, Voyage, Cohere V2, Replicate, Deepgram, ElevenLabs
  • Modes log / alert / block
  • Model-aware burn estimates + optional [tiktoken] when usage is missing
  • Optional LangChain + OpenTelemetry enrichers
  • Zero-dependency core; failure-isolated instrumentation

What paid cloud adds (not in this package)​

Typical Team+ / Pro surfaces (hosted Cloud; exact packaging may change before GA):

  • Event ingestion, retention, hosted dashboard, webhooks (email / Slack / PagerDuty / HTTP)
  • Intervention Pack: session USD budgets, tokens/min velocity caps, operator kill-switch (SDK polls policy when configured)
  • Pro: LLM-as-judge on gray-zone confidences, drift / stability index, trace consolidation, composite signals (lost_agent, runaway_retrieval, zombie_loop), RBAC / audit, chargeback by tags

Cloud waitlist and product surfaces live on tokensentinel.dev (public list pricing is not published yet). Capability detail is on the features page — not re-litigated in this OSS tree.

# OSS-only (default)
Sentinel(project="my-agent", mode="log")

# Opt into paid cloud (any mode ships events when both are set)
Sentinel(
    project="my-agent",
    mode="log",
    cloud_endpoint="https://api.tokensentinel.dev",
    api_key="ts_live_...",
)

Why not OpenTelemetry alone?​

OTel spans are observational and usually fire after the call. TokenSentinel sits on the client call path so your handler (or block exception) can stop the next call. We also ship an OTel span processor enricher for frameworks that only emit gen_ai.* spans — that path is still post-call observation, not a substitute for wrap when you need tight control.

How is this different from Langfuse / LangSmith / Helicone / Datadog LLM?​

Those products excel at traces, analytics, and cost reporting after the fact. TokenSentinel is in-process detection (and optional mid-session halt) for waste patterns. They compose: wrap with TokenSentinel and keep your observability stack.

How is this different from Alephant / Bifrost / Portkey?​

Those are proxy / gateway layers: traffic (and often keys) go through their hop for routing, fallbacks, and BYO-key management. TokenSentinel is in-process — it wraps your existing LLM client, does not route or hold provider keys, and the rule engine works offline (e.g. log mode with no network). Different layer of the stack; you can use a gateway and still wrap the client with TokenSentinel.

Will TokenSentinel send my provider keys to your cloud?​

No. The SDK runs in-process. Your Anthropic / OpenAI / Gemini / Bedrock (and other) keys never leave your process. Optional Cloud receives only the structured waste events your SDK chooses to send, after the default redaction pipeline — and only when you set both cloud_endpoint and api_key.

What happens during a cloud outage?​

The SDK is failure-isolated. If Cloud is unreachable, event shipping and the dashboard degrade; your agents keep running. Detection and log / block on the local path do not depend on Cloud availability. Policy fail-mode is configurable — open by default (safer for prod), closed if you need it.

Can I self-host today?​

Yes for the SDK: it is Apache-2.0, self-contained, and works offline in log mode with no account. Hosted Cloud is the optional commercial path for Team/Pro. Self-hosted Docker/Helm for the Cloud control plane is an Enterprise SKU for design partners — contact us via the product site.

What are composite signals?​

Pro Cloud multi-rule patterns on the event stream: lost_agent, runaway_retrieval, and zombie_loop. They are not part of the free in-process SDK rule set — Free and Team still run the same 15 deterministic rules; Pro layers composites (and judge/drift) on top in the cloud.

Does it work with self-hosted LLMs?​

Yes — OpenAI-compatible servers (vLLM, Ollama, TGI, LM Studio, LocalAI). Point openai.OpenAI(base_url=...) and wrap. Waste signals still apply; estimated_burn is not a real invoice for GPU-amortized tokens.

Timing: does it stop spend on the current call?​

No. Wrappers call the provider first, then rules / block / policy. The current request is already billed. Value is stopping runaway loops and giving operators a signal in real time. Documented in Modes.

How accurate is estimated_burn?​

It is a FinOps signal, not a billing export. As of 1.0.3 the SDK uses a static per-model price table (input vs output, with prompt-cache discount when providers report cache-read tokens). Unknown model IDs fall back to a flat average. Override with Sentinel(pricing_table=...) or call estimate_usd(...) directly. Details: Cost estimates.

How do I change tool_loop to need more than 3 calls?​

Sentinel(
    project="my-agent",
    config={"tool_loop.min_calls": 5, "tool_loop.cosine_threshold": 0.80},
)

All rule knobs use the flat key form "rule_name.param". Tables: Waste rules.

Is there a TypeScript or Go SDK?​

Not yet. Python is first. Open an issue if you need another language for a production use case.

How do I contribute?​

  • Bugs / features: GitHub Issues on this repository
  • PRs: welcome; non-trivial changes should start with an issue
  • Tests: pytest tests/ -q
  • Lint: ruff check token_sentinel tests examples
  • Types: mypy token_sentinel

New providers: follow wrappers/anthropic.py patterns. New rules: deterministic, no I/O, pure function of the session buffer.

License​

Apache-2.0 for the SDK. Use it in commercial products. Cloud remains proprietary.

Why “TokenSentinel”?​

“Sentinel” = watching the call path. Marketing speaks “token waste” (cost); the Python API keeps LeakEvent / on_leak for compatibility and adds WasteEvent / on_waste aliases.

Does alert mode send data to the cloud?​

Only if you configure cloud_endpoint and api_key. Those flags enable the sink for all modes, including log. Without them, alert is local-only like log.

Policy exceptions without mode="block"?​

Yes. BudgetExceeded, VelocityExceeded, and KillSwitchActive are raised from the policy client when cloud policy is active, independent of mode. They require a paid cloud policy plane.

Version​

python -c "import token_sentinel; print(token_sentinel.__version__)"