Troubleshooting
Symptom-first index for common Solwyn SDK problems in production
This page is a symptom-first index. Find the symptom you are seeing in your application or logs, then follow the resolution.
My calls are not appearing in the dashboard
The most common cause is the metadata reporter being shut down before its background batch flushes.
Resolution:
- Confirm
client.close()is called when your application exits, or that you used a context manager (with Solwyn(...) as client:/async with AsyncSolwyn(...) as client:). - Look for
reporter.ingest_events_rejectedwarnings (SDK v0.1.7+). If you see them, your batches are reaching Solwyn Cloud but specific events were refused — usually a model Solwyn cannot price yet. See the next section. - Look for the warning
Cloud API budget check failedin your logs. If you see it, the SDK could not reach Solwyn Cloud — check your network andSOLWYN_API_URLvalue. - Verify
SOLWYN_PROJECTnames the project you expect, and thatSOLWYN_API_KEYreaches it. The dashboard shows usage per project. - SDK v0.4.0+: look for
reporter.spend_events_dropped. That line names exactly how much spend was written off and why (retry_exhausted,shutdown_deadline,overflow, and so on). It is the definitive answer to "how much is missing" — see Undeliverable spend. - SDK v0.4.0+: if a short-lived process is losing spend at exit, raise
reporter_shutdown_deadline(default5.0seconds).close()counts and drops whatever is still queued when that deadline expires.
If using AsyncSolwyn, you must await client.close() (not client.close()).
Since 0.4.0, delivery is at-least-once — a transient failure is retried rather than dropped — so persistent gaps now point at configuration or connectivity rather than a single unlucky flush. See Spend delivery.
reporter.ingest_events_rejected warnings in my logs
Solwyn Cloud accepted the metadata batch but refused to record specific events in it — this is not a connectivity failure, and the other events in the batch were recorded normally. Logged at WARNING level by the solwyn.reporter logger, one aggregated line per distinct rejection code and model per batch (SDK v0.1.7+):
reporter.ingest_events_rejected: code=unknown_model model=vendor-x-1 count=2 message=Solwyn does not have pricing for model 'vendor-x-1'. File an issue at https://github.com/solwyn-ai/solwyn-python-sdk/issues or contact support — we typically add new models within 24h.What the codes mean:
unknown_model— Solwyn has no pricing entry for that model identifier. Those events were dropped, so their cost is missing from the dashboard.unknown_service_tier— the event carried a service tier Solwyn Cloud does not recognize, so the cost could not be computed at the correct rate.invalid_tags— the event's spend tags broke the per-event bounds (more than 10 keys, a key outside 1–64 characters, a value over 256, or a NUL). The SDK enforces the same bounds before dispatch, so this normally indicates a version skew.tag_cardinality_exceeded— the event would have introduced a tag key beyond the account's 100 active keys, or a value beyond the 1,000 values allowed for one key. Retire unused keys in the dashboard, or stop putting high-cardinality data (ids, timestamps) in tag values.unsupported_modality— a known model whose media configuration Solwyn cannot price (for example a resolution or quality that matches no variant).- Rejection codes are server-owned and the set can grow without an SDK release; an unfamiliar code is logged the same way. The
messagefield carries the server's guidance verbatim.
Resolution:
- Rejected events are terminal: the SDK logs and drops them, never retries — resubmission would be rejected identically until a pricing entry lands server-side. The corresponding calls stay absent from the dashboard.
- For
unknown_model, open an issue at github.com/solwyn-ai/solwyn-python-sdk with themodel=value from the log line. Pricing lives server-side: once Solwyn adds the entry, subsequent traffic is priced automatically — no SDK upgrade or redeploy on your side. The already-rejected calls are not recovered retroactively. - For
unknown_service_tier, include the full log line in the issue — the server'smessagenames the tier it could not match, and support for it is likewise a server-side fix. - No action is needed for the accepted events in the same batch — they are already durable.
For the line-by-line format reference, see Logging.
ConfigurationError: Invalid API key format at construction
Your api_key did not match the required pattern solwyn_sk_<64 lowercase hex chars>.
Common causes:
- An old project key. Keys that start with
sk_proj_stopped working when keys became account keys (SDK v0.11.0). The SDK says so:This is an old Solwyn project key (sk_proj_). Create a new key in the dashboard or with solwyn keys create.See API keys. - Trailing whitespace or newline (often from copy-paste). Strip with
os.environ["SOLWYN_API_KEY"].strip(). - Using the key's name or prefix instead of the raw key.
- Quoting the value in your
.envfile with extra characters.
The error's field attribute is set to "api_key" and the message includes the first 12 characters of what the SDK saw.
ConfigurationError: No Solwyn project configured at construction
SDK v0.11.0+. Every client names the project it spends against. Set SOLWYN_PROJECT, or pass project=, to a project name or its proj_ id. The argument wins over the environment.
If you still set SOLWYN_PROJECT_ID, rename it: the SDK no longer reads it, and the error says so. project_id= is not an option either; use project=.
If you set up the project with the CLI, .solwyn.env in your repository names it, and solwyn keys env exports SOLWYN_PROJECT with the key. See Give the key to your app.
ApiKeyRefusedError on the first call or from validate()
SDK v0.11.0+. Solwyn refused the key or the project. e.code says which:
e.code | Fix |
|---|---|
project_not_found | The key does not reach that project, or no project has that name. Check SOLWYN_PROJECT. A renamed project's old name stops working at the next start; its proj_ id always works. To give the key another project, add it in the dashboard under Settings > API keys. |
key_not_dedicated | The project only accepts keys that hold no other project. Create a key for this project alone. |
api_key_expired | Keys expire a year after they are created or rotated. Rotate it in the dashboard or with solwyn keys rotate KEY_ID (solwyn keys list shows the id). |
api_key_invalid | The key is unknown or was revoked. Create a new one. |
read_only_key | The SDK reports spend, so it needs a full-scope key. |
project_selector_required, project_selector_conflict | Solwyn refused the project header the SDK sent. That is an SDK bug: report it with the SDK version. Calls are blocked. |
A refusal is never treated as an outage: fail_open=True does not let calls past it. The one exception is an expired or revoked key on an alert-only project with no hard rule or run cap, after the client has validated once: calls keep running unmetered and the SDK logs solwyn.api_key_refused at error on each one. Replace the key; until then that spend is missing from the dashboard. See ApiKeyRefusedError.
To catch a bad key or project at startup instead of on the first call, call client.validate() (await client.validate() on AsyncSolwyn).
Cloud API budget check failed warnings in my logs
Solwyn Cloud was unreachable for one budget check. This is logged at WARNING level by the solwyn logger.
When to ignore: Single occurrences during transient network issues. With the default fail_open=True, your LLM calls proceeded, and each one was tallied for the next budget check that reaches Solwyn.
When to investigate: Persistent failures — every call. Check:
SOLWYN_API_URLis correct (defaulthttps://api.solwyn.ai).- The host running your code can reach the API URL (
curl -I $SOLWYN_API_URL). - Your API key has not expired or been revoked. A refused key raises
ApiKeyRefusedErrorinstead of this warning on SDK v0.11.0+.
If you set fail_open=False, calls that no live lease covers are denied outright (SDK v0.8.0+): the SDK raises BudgetExceededError with deny_reason="control_plane_unreachable" and logs budget.fail_closed_unreachable at most every 30 seconds. It no longer meters against the last limit Solwyn returned. To keep run-scoped calls flowing, keep lease_enabled=True and make the calls inside solwyn.run(...).
budget.fail_closed_unreachable in my logs
You set fail_open=False, Solwyn Cloud is unreachable, and a call on the per-call check path was refused. Since 0.8.0 the SDK does not estimate cost locally, so there is no local ledger to meter against — the call is denied, with budget_limit and current_usage of 0.0 on the exception. The line is rate-limited to one every 30 seconds however many calls were refused.
Resolution:
- Restore connectivity (the checks under
Cloud API budget check failedabove apply). - If run-scoped calls must survive an outage, make them inside
solwyn.run(...)withlease_enabled=True: a live lease admits them locally through the outage ladder. - If availability matters more than metering, set
fail_open=True; admitted calls are then tallied and reported on the next successful check.
budget.uncounted_report_dropped in my logs
Under fail_open=True the SDK tallied calls it admitted during an outage and tried to report them on the next budget check, but Solwyn Cloud answered that check with a 4xx other than 408, 409 or 429. The tally is dropped rather than re-sent forever, because Solwyn records the tally before it evaluates the check, so any other 4xx means the report itself was refused. The usual cause is a control plane that predates the uncounted_* fields (accepted on /budgets/check since 2026-09-17), or the same 401/403 that would fail any check from that key. The dropped figures are advisory — never priced and never part of current_usage — so nothing in your spend history changes.
BudgetExceededError raised when I did not expect it
Solwyn Cloud denied the call's pre-flight check in hard_deny mode. That mode comes from Cloud: the project's budget mode, or a hard-deny scoped rule or per-run cap that applies to the call. The constructor's budget_mode= setting does not decide it; it only labels results the SDK builds itself while Cloud is unreachable. (With fail_open=False, an unreachable Cloud also raises; see When Solwyn Cloud is unreachable.)
Diagnose:
- Inspect
e.budget_limitande.current_usageon the exception. Ifcurrent_usageis at or nearbudget_limit, the budget genuinely is exhausted. - Check the dashboard for the project's current period spend.
- Confirm you are looking at the right project — multi-agent setups often have separate projects per agent.
- SDK v0.10.0+: if
current_usageis well belowbudget_limit, the call's output bound may not fit. The check reserves input plus the call's largest possible output, so a largemax_tokenson an expensive model can be refused before the budget is spent. Lower the cap (orlease_output_bound_default, for calls that set none). See What hard_deny bounds.
To let calls through while you investigate, change the mode in Solwyn Cloud: set the project's budget mode, or the mode of the scoped rule or per-run cap that denied (e.budget_period names it), to alert_only, or raise its limit. Setting budget_mode="alert_only" on the client does not bypass a Cloud hard denial.
OutputBoundUnenforceableError on a hard-deny admission
SDK v0.10.0+. On a hard_deny admission (including a call on an alert_only project that a hard-deny scoped rule or per-run cap applies to), the SDK refuses a call locally when it cannot hold the call's output to the bound the budget check priced. Usually nothing was sent to the provider. A lease-funded call hands its lease claim back, but a per-call reservation stays held in Solwyn Cloud until it expires. The exception is an Anthropic call refused after an earlier send in the same call succeeded: that generation already ran, the call reports possibly_succeeded, and the reservation is kept at its bound. Read e.reason and e.hint:
reasoning_without_budget: set a positivethinking_budgetorenable_thinking=Falsefor Qwen, use a non-reasoning xAI model, or setprovider=explicitly if your gateway counts reasoning inside the cap.unsupported_override: remove the extra key fromextra_body(or Googlehttp_options.extra_body, or BedrockadditionalModelRequestFields) and use the request's own cap parameters.multi_generation_loop: turn off Google automatic continuation and automatic function calling, and remove Anthropicfallbacks.azure_api_version: set a cap on the call or move to an Azure api-version from 2024-09-01 on.bound_exceeded,malformed_cap_container,malformed_candidate_count: set the cap and candidate count on the call itself, as plain integers in a plain dict. A zero or negative cap insideextra_bodyis not replaced by the default; remove it.
See the full table in What hard_deny bounds.
My calls keep raising ProviderUnavailableError
The provider's circuit breaker is open — the dashboard shows this provider as Down — and the SDK is refusing to send calls. Look for Circuit breaker [provider] opened due to failures in your logs (the [name] label is new in 0.4.0; grep for Circuit breaker to match every version). That is the usual cause, but not the only one: if the message reads failover deadline expired, the failover_total_timeout window (default 30 s) ran out before any candidate was dispatched — raise it or shorten the chain.
Resolution:
- The circuit closes automatically after the recovery probe succeeds. With defaults, this is roughly 60 seconds (±20% jitter, so up to 72 seconds) + 2 successful probe calls.
- To recover faster, restart the process — circuit-breaker state is per-process and resets on restart.
- Investigate why the provider was failing — Solwyn does not catch provider exceptions, so the original errors should be visible in your logs from the calls that opened the circuit.
To require more failures before opening (less aggressive), increase circuit_breaker_failure_threshold. To recover faster, decrease circuit_breaker_recovery_timeout. A same-provider model fallback that fails on both the primary and the fallback model still records only one failure against that provider's breaker — failures are deduplicated per provider within a single request — so with the default threshold of 3, the circuit opens after three failed requests.
If you configure a fallback= chain, ProviderUnavailableError is not raised just because one provider is down — a single unhealthy provider should fail over rather than raise. If it raises with a fallback configured, check e.attempted to see which candidates the router had selected when it gave up (it is empty when every breaker was already open).
My dashboard shows failover usage I didn't expect
Some calls failed on the requested provider/model and were served by a fallback entry. The dashboard tags these events: is_model_fallback=true for a same-provider model swap, is_provider_fallback=true for a cross-provider hop, with failover_reason and requested_provider / requested_model describing what changed.
Resolution:
- Filter the dashboard by
is_model_fallback/is_provider_fallbackto see how often each kind of failover fires. - Inspect
failover_reason(circuit_open,primary_error,model_fallback, orcost_routed) andfailover_error_classto understand why the primary was bypassed.cost_routedis not a failure at all: you configuredCostPolicyand a healthy, cheaper provider served the call by design — see Cost-aware routing. - Check application logs for the original primary errors — the SDK surfaces them only when the whole chain is exhausted, so successful failovers won't reach your exception handlers.
- If failover fires more than expected, fix the underlying primary issue, or shorten the
fallback=chain to surface the original errors directly.
A cross-provider call raises UntranslatableRequestError
A failover hop to a different provider could not translate the request shape (for example an OpenAI response_format that has no Anthropic equivalent). The chain aborts before any network call, so no duplicate request is sent.
Resolution:
- Read
e.feature,e.source, ande.target— they name what could not be translated, structurally (never the value). - Either keep that call's request shape within what every provider in the chain can serve, or drop the cross-provider entry for calls that rely on a provider-specific feature.
UntranslatableModelErrormeans a fallback entry has no model set for its provider — add a concrete model to that tuple.
My async tests hang on shutdown
AsyncSolwyn started a background reporter task that was never closed.
Resolution:
Always use async with or call await client.close():
# Correct
async with AsyncSolwyn(AsyncOpenAI()) as client:
...
# Also correct
client = AsyncSolwyn(AsyncOpenAI())
try:
...
finally:
await client.close()
# Wrong — hangs on shutdown
client = AsyncSolwyn(AsyncOpenAI())
# ... no close()SDK v0.4.0+: close() is now bounded by a single wall-clock deadline (reporter_shutdown_deadline, default 5.0 seconds) covering the worker join, the final flush, and the last breaker-report cycle. An unreachable Solwyn can no longer hold shutdown open indefinitely — undelivered work is counted and dropped at the deadline instead. If teardown is still slow, lower that deadline; if it is dropping spend you care about, raise it.
Two related 0.4.0 changes worth knowing here:
- An
AsyncSolwynbuilt withoutasync withused to queue events and settlements silently untilclose(). Its flush loop now auto-starts on first enqueue. If you seereporter.enqueue_without_event_loop, the client was used outside a running event loop. - A process that exits without calling
close()at all now flushes on the way out via an exit hook, bounded the same way. It is a safety net, not a substitute forclose().
SDK v0.6.0+: the opt-in pytest fixtures close the client for you, in a finally, even when the test body raises. Enable them with pytest_plugins = ["solwyn.testing.pytest_plugin"] and request solwyn_test_client — see Testing budget enforcement.
Streaming tokens are not being counted
The provider's stream did not emit usage data, or your code aborted the stream before the usage chunk arrived.
Resolution:
- For OpenAI: Solwyn injects
stream_options={"include_usage": True}automatically. If you pass your ownstream_optionsdict, Solwyn preserves your other keys and force-setsinclude_usage=True— even an explicitinclude_usage=Falseis overridden — so missing usage on a direct OpenAI stream is not caused bystream_options. - For OpenAI-compatible providers: Solwyn injects
include_usageonly where the provider documents support for it. It never injects for xAI, Mistral, Together, Fireworks, Perplexity, OpenRouter, or the genericopenai_compatiblecatch-all — those endpoints reject, deprecate, or simply do not document the parameter, and most deliver usage automatically in the final chunk. If no usage arrives at all, the SDK reports length-based estimated counts flaggedtoken_details.is_estimated=truewith a one-time WARNING — counts are present but approximate, not missing. See OpenAI-compatible providers. - For Anthropic and Google: usage events are emitted by default — no action needed.
- For Bedrock: usage arrives in the stream's terminal
metadataevent. Fully consumeresult["stream"], or on early abandonment callresult["stream"].close()(or usewith result["stream"]:) so the reservation settles with the observed usage. A stream that settles at zero tokens after real traffic logsBedrock stream settled at zero tokens— see Streaming. - For early aborts on the other providers: wrap the stream in
with stream:(sync) orasync with stream:(async) so Solwyn settles with the tokens it observed before the break. See Streaming.
The SDK is logging a lot — how do I quiet it?
All Solwyn log messages are emitted under the solwyn logger. Set its level to ERROR to silence WARNING-level messages:
import logging
logging.getLogger("solwyn").setLevel(logging.ERROR)For the full list of messages and what each one means, see Logging.
RunStoppedError raised inside a run
SDK v0.6.0+. The active solwyn.run(...) was stopped — not budget-denied. Read e.source:
"server"withreason="manual_kill"— an operator pressed Stop on the dashboard's Agents tab. The stop is permanent for that run id and survives a Solwyn outage. Start a new run; do not retry inside the old one."local_velocity"withreason="velocity:repeat_size"or"velocity:monotonic_growth"— the SDK's own detector decided the run was looping andvelocity_mode="deny"is set. Inspect the loop, or lift the stop for that run id withsolwyn.clear_run_termination(run_id)if it was a false positive.
RunStoppedError is not a BudgetExceededError, so an except BudgetExceededError block will not catch it. See Run control.
velocity.flagged warnings in my logs
SDK v0.6.0+. Local velocity detection flagged the active run: velocity.flagged: rule=repeat_size run=run_.... In the default velocity_mode="warn" this is advisory — the call proceeds and the run shows a Flagged badge on the Agents tab. The rules are repeat_size (the same model with the same-sized prompt, repeatedly), monotonic_growth (prompts growing steadily, as when a loop re-feeds its own output), and rate_acceleration (a call-rate spike; advisory only).
Resolution:
- Decide whether the pattern is a bug. A retry loop that re-sends an identical prompt, or an agent that appends every turn to its context, is exactly what the rules exist to catch.
- If it is expected for this workload, raise the thresholds (
velocity_repeat_count,velocity_growth_streak, ...) or setvelocity_mode="off". See Velocity settings. - To make the flag a stop, set
velocity_mode="deny"— the run then raisesRunStoppedErroron its next call.
"exposes untracked surface" warnings, or UntrackedSpendSurfaceError
SDK v0.6.0+. Your code reached a provider capability Solwyn does not meter — files, batches, moderations, models.list, a with_raw_response helper, or something new in the provider SDK. Under the default on_unmetered="warn" the SDK logs once per surface per process and forwards the call; under "raise" it refuses with UntrackedSpendSurfaceError before any provider I/O. Before 0.6.0 most of these surfaces passed in silence, which is why the warning is new to you after upgrading.
Resolution:
- If the surface carries no spend you care about, acknowledge it:
acknowledge_untracked={"models.list"}(orSOLWYN_ACKNOWLEDGE_UNTRACKED=models.list). The exception'stokenattribute is the exact string to add. Tokens are validated at construction, so a typo fails immediately withConfigurationError(field="acknowledge_untracked"). - If you want the old silence for everything, set
on_unmetered="allow"— that silences the log only; addreport_untracked_surfaces=Falseif you also want no advisory report. - If the surface should be metered, tell Solwyn — the advisory report the SDK already sent is what the Providers tab's "Surfaces Solwyn does not meter yet" card lists.
See Coverage controls.
ConfigurationError from a Responses API call
SDK v0.6.0+. Native OpenAI and Azure responses.create / parse / stream are metered, and three request shapes are refused before the budget check because they cannot be metered:
field="background"—background=Truequeues the response and exposes no create-time usage. Use the raw OpenAI client for background responses.field="stream"— a streamingresponses.parse. Useresponses.create(stream=True)or thestream()helper.field="extra_body"—extra_bodyoverridesmodel,input,instructions,max_output_tokens, orstream. Pass those as top-level arguments so preflight and dispatch see the same values.
See The Responses API.
TypeError: ... cannot be pickled
SDK v0.6.0+. Something tried to pickle the wrapper — multiprocessing with the spawn start method, a task queue serializing its arguments, or a framework snapshotting configuration. The client holds live reporter and budget state and refuses. Construct the client in the target process instead; copy.copy and copy.deepcopy return the same wrapper rather than a clone. See Copying and pickling the client.
CostPolicy selected but this budget check carried no price hints
SDK v0.6.0+. You configured selection_policy=CostPolicy() and a budget check answered with price_hints: null, so the policy used health-based order for that call. Logged once per process by solwyn._base. The usual cause is benign: lease-backed calls inside a solwyn.run(...) scope carry no price hints yet, so CostPolicy keeps configured order there. If you see it outside runs, the control plane was unreachable or answered a legacy shape. An explicit empty map ({}) — the answer for non-text modalities and for a primary provider Solwyn cannot price — is silent by design. See Cost-aware routing.
AttributeError: 'OpenAI' object has no attribute 'update_price_hints'
Solwyn.update_price_hints and the client-wide hint store were removed in 0.6.0. Price hints now arrive on every budget check and apply only to the call they were priced for; there is nothing to update. Remove the call. (The error names the provider class because unknown attributes forward to the wrapped client.)
SolwynTagsClampedWarning in my test output
The merged spend tags for a call — client defaults, run scope, and solwyn_tags together — exceeded 10 keys. The SDK kept the 10 highest-priority keys, dropped the rest from that call's attribution, and proceeded. Trim the lowest-priority layer (usually client-wide defaults). Each mapping on its own is still limited to 10 keys and raises ValueError if it exceeds them. See Bounds, validation, and clamping.
lifecycle.run_surrender_failed warnings in my logs
SDK v0.7.0+. A run scope exited — or RunHandle.finish() was called — and the SDK's attempt to hand that run's budget lease back raised inside a lease holder, before the release ever reached the network. The failure is swallowed: your scope exit completed, the run's spend is unaffected, and a release that was already parked still rides close(). This is not a connectivity symptom — a release that fails on the wire logs lease.surrender_failed at DEBUG instead. A single line is harmless; recurring lines are a bug worth reporting with the exc_type. See When a lease is handed back.
Still stuck?
- Check the SDK version with
from solwyn import __version__; print(__version__)and include it in any bug report. - Open an issue at github.com/solwyn-ai/solwyn-python-sdk with the version, the symptom, and any relevant log lines.