A model 4xx reports as "model backend unavailable", so a refused request invites a retry that cannot work #894

New issue

Closed

opened 2026-08-17 05:33:08 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-08-17 05:33:08 +00:00

Member

Split from #875 by Angie (ENG, claude seat), so that issue's P0 can be about the rollout that actually fixes it.

A 4xx reports as an outage

turnFailureNotice has no case for a modelHTTPError carrying a 4xx, so it falls through to the stageModel fallback:

case stageModel:
    return noticeModelFailed   // "model backend unavailable, retry shortly"

A backend that answered and refused the request this service built therefore reads identically to a backend that was down.

Why it is worth its own fix

Under #875 the member was told to retry shortly while DeepSeek was answering every call. Retrying rebuilt the same malformed message array and drew the same 400, so the advice could not work in the case that generated it. That is the sirens-echo#449 family - a true sentence pointing somewhere useless.

This is wrong independently of #875. After that rollout lands, a 4xx from any cause still reports as an outage, and any future malformed request would hide in the same way.

It also hid the class. stage_failed is what every model failure shares, so a malformed-request rejection was invisible in every aggregate until someone parsed LiteLLM's logs.

Scope

Classify a 4xx as a refusal and give it its own notice and cause.
429 and 408 stay retryable, because those are the two 4xx a wait does fix. Trading one misdirection for another is the obvious way to get this wrong.
Availability statuses keep reporting as availability.

The wording itself is member-facing and Content Creator's to set. Matching the existing phrasing is enough for this issue.

Acceptance

A 400 does not render the unavailable notice.
502, 503 and 504 still do, and stay retryable.
429 and 408 are not classified as refusals.
A rejection is never retried, since the retry rebuilds the same request.
The cause is countable on its own rather than collapsing into stage_failed.

Related - #875 (where this surfaced), #449 (the misdirection family), coilyco-bridge/deploy#621 (the rollout that fixes #875's cause).

Split from `#875` by Angie (ENG, `claude` seat), so that issue's P0 can be about the rollout that actually fixes it. ## A 4xx reports as an outage `turnFailureNotice` has no case for a `modelHTTPError` carrying a 4xx, so it falls through to the `stageModel` fallback: ```go case stageModel: return noticeModelFailed // "model backend unavailable, retry shortly" ``` A backend that **answered and refused** the request this service built therefore reads identically to a backend that was down. ## Why it is worth its own fix Under #875 the member was told to retry shortly while DeepSeek was answering every call. Retrying rebuilt the same malformed message array and drew the same 400, so the advice could not work in the case that generated it. That is the sirens-echo#449 family - a true sentence pointing somewhere useless. **This is wrong independently of #875.** After that rollout lands, a 4xx from any cause still reports as an outage, and any future malformed request would hide in the same way. It also hid the class. `stage_failed` is what every model failure shares, so a malformed-request rejection was invisible in every aggregate until someone parsed LiteLLM's logs. ## Scope * Classify a 4xx as a refusal and give it its own notice and cause. * **`429` and `408` stay retryable**, because those are the two 4xx a wait does fix. Trading one misdirection for another is the obvious way to get this wrong. * Availability statuses keep reporting as availability. The wording itself is member-facing and Content Creator's to set. Matching the existing phrasing is enough for this issue. ## Acceptance * A 400 does not render the unavailable notice. * 502, 503 and 504 still do, and stay retryable. * 429 and 408 are not classified as refusals. * A rejection is never retried, since the retry rebuilds the same request. * The cause is countable on its own rather than collapsing into `stage_failed`. **Related** - #875 (where this surfaced), #449 (the misdirection family), coilyco-bridge/deploy#621 (the rollout that fixes #875's cause).

coilyco-ops added the

labels

2026-08-17 05:33:23 +00:00

coilyco-ops referenced this issue from a pull request that will close it,

2026-08-17 05:33:27 +00:00

fix(notice): a refused request is not an unavailable backend #893