Resilience reference¶

httpware's resilience middleware live in httpware.middleware.resilience and plug into the same middleware chain as your own:

AsyncRetry and Retry retry transient failures with full-jitter exponential backoff.
RetryBudget is a token bucket that caps the share of traffic spent on retries, so retries can't pile onto an outage. One instance can be shared by sync and async clients.
AsyncBulkhead and Bulkhead cap concurrent requests and reject a request that can't get a slot in time.
AsyncCircuitBreaker and CircuitBreaker stop sending requests to a downstream that keeps failing.
AsyncKeyedCircuitBreaker and KeyedCircuitBreaker keep a separate circuit per upstream.
AsyncTimeout bounds the total time of a call, retries included.

Order matters. Composition gives the recommended order and the reason for each position.

See it under load

The interactive demos run each pattern through an outage next to a client without it.

`AsyncRetry`¶

from httpware.middleware.resilience import AsyncRetry

Parameter	Default	Effect
`max_attempts`	`3`	Total tries, including the first. `1` disables retries; `<1` raises `ValueError`.
`base_delay`	`0.1` (s)	Floor for the full-jitter exponential backoff. Must be `>= 0`; a negative or NaN value raises `ValueError`.
`max_delay`	`5.0` (s)	Ceiling for backoff and the longest `Retry-After` honored. Must be finite and `>= 0`; a negative, NaN or infinite value raises `ValueError`.
`retry_status_codes`	`frozenset({408, 429, 502, 503, 504})`	Status codes considered retryable.
`retry_methods`	`frozenset({"GET", "HEAD", "OPTIONS", "PUT", "DELETE"})`	Idempotent methods only. To retry POST, pass a frozenset that includes `"POST"`. Matching ignores case, so `"post"` works too.
`respect_retry_after`	`True`	When a retryable response has a `Retry-After` header, wait that long instead of the jittered backoff. If the value is over `max_delay`, `AsyncRetry` gives up, emits `retry.giving_up` with `reason="retry_after_exceeds_max_delay"`, and re-raises the `StatusError` with a PEP 678 note: `httpware: Retry-After (Ns) exceeded max_delay (Ms); giving up`. To avoid this, set `respect_retry_after=False` or raise `max_delay`.
`budget`	`RetryBudget()`	The token bucket. Pass one shared `RetryBudget` to several clients to give them a joint budget.

To bound the total time across all attempts, put AsyncTimeout first in the chain. To bound a single request, set httpx2.Timeout on the client or pass timeout= per request.

Retry-After parsing¶

Retry-After can be either:

integer seconds: Retry-After: 30 waits 30 seconds.
an HTTP date (RFC 5322): Retry-After: Wed, 21 Oct 2026 07:28:00 GMT waits until that time, or not at all if it has passed.

Either form over max_delay makes AsyncRetry give up as described above. An integer too large to fit in a float counts as an infinite wait, so it always exceeds max_delay. Negative integers count as 0. Malformed values are ignored and the jittered backoff is used instead.

Streaming-body refusal¶

If the request body was an async iterable, AsyncRetry does not retry, because the first attempt consumed the iterator. It re-raises the original exception with a PEP 678 note:

httpware: not retrying because the request body is a stream that cannot replay across attempts

The same applies to a request you build yourself and pass to send() or send_with_response(), whether it comes from client.build_request() or httpx2.Request(). send() marks it as streaming when its body is not held in memory and is not a multipart upload, so content= with a generator is marked, while content=b"...", json=, data= and files= are not. A body built from a list of chunks also counts as streaming here, because send() cannot tell it from a generator; client.post() and the other verb methods see the original argument and don't mark it.

The note, and the retry.streaming_refused event on httpware.retry, appear only when the failure would otherwise have been retried. A POST with a streaming body, for example, is re-raised without the note because POST is not retried in the first place. See Observability.

Exhaustion behavior¶

When attempts run out, AsyncRetry re-raises the last exception, such as ServiceUnavailableError or NetworkError, with its original class, so except ServiceUnavailableError still works. It adds the note httpware: gave up after N attempts.

If the budget refuses a retry before max_attempts is reached, AsyncRetry raises RetryBudgetExhaustedError instead, with last_response, last_exception and attempts set. See Errors.

`RetryBudget`¶

from httpware.middleware.resilience import RetryBudget

A token bucket in the style of Finagle's retry budget. Each request deposits a token and each retry tries to withdraw one. Retries are allowed up to percent_can_retry of recent deposits, plus a floor of min_retries_per_sec * ttl.

Parameter	Default	Effect
`ttl`	`10.0` (s)	Sliding window over which deposits and withdrawals count. Must be finite and `> 0`.
`min_retries_per_sec`	`10.0`	Floor: at least this many retries per second are allowed, whatever the deposit rate. Must be finite and `>= 0`.
`percent_can_retry`	`0.2`	Fraction of recent deposits that can convert to retries (above the floor). Must be finite and `>= 0`; values above `1.0` allow more retries than requests.

An out-of-range value raises ValueError when the budget is constructed.

The token-bucket formula¶

ceiling = ceil(len(deposits_in_window) * percent_can_retry) + int(min_retries_per_sec * ttl)

The percent term rounds up (math.ceil); the floor term truncates (int). A withdrawal fails when len(withdrawn_in_window) >= ceiling.

The window moves in steps of ttl / 10, so a deposit or withdrawal stops counting between 0.9 * ttl and ttl after it was made.

Why a floor matters¶

With no traffic yet, the percent term is zero, and without the floor the first retry would be refused. The floor lets low-traffic clients retry occasional failures. For high-traffic clients the percent term is much larger and the floor stops mattering.

Clients that call the same downstream can share one RetryBudget:

import asyncio

from httpware import AsyncClient
from httpware.middleware.resilience import AsyncRetry, RetryBudget


shared = RetryBudget()


async def main() -> None:
    async with (
        AsyncClient(base_url="https://api.example.com", middleware=[AsyncRetry(budget=shared)]) as users,
        AsyncClient(base_url="https://api.example.com", middleware=[AsyncRetry(budget=shared)]) as orders,
    ):
        await asyncio.gather(users.get("/users/1"), orders.get("/orders/1"))

Thread safety¶

RetryBudget guards its state with a threading.Lock, so one instance can be shared across threads, across coroutines, and between a Client and an AsyncClient in the same process.

`AsyncBulkhead`¶

from httpware.middleware.resilience import AsyncBulkhead

Limits concurrent requests with an asyncio.Semaphore. Each request waits up to acquire_timeout for a slot and releases it when it finishes, fails or is cancelled.

Parameter	Default	Effect
`max_concurrent`	required	Maximum requests in flight. `<1` raises `ValueError`. There is no default because the right cap depends on the downstream's capacity.
`acquire_timeout`	`1.0` (s)	How long to wait for a slot before raising `BulkheadFullError`. `None` waits forever; `0` fails fast. `<0` raises `ValueError`.

Slot release contract¶

The slot is released in a try/finally around await next(request), so a success, an exception or a CancelledError all release it.

Like RetryBudget, one bulkhead can be shared by several clients:

shared_bulkhead = AsyncBulkhead(max_concurrent=10)

async with (
    AsyncClient(base_url="https://api.example.com", middleware=[shared_bulkhead, AsyncRetry()]) as a,
    AsyncClient(base_url="https://api.example.com", middleware=[shared_bulkhead, AsyncRetry()]) as b,
):
    ...  # combined in-flight across a + b is capped at 10

Rejection¶

If no slot opens within acquire_timeout, AsyncBulkhead raises BulkheadFullError with the configured max_concurrent and acquire_timeout (see Errors) and emits bulkhead.rejected on httpware.bulkhead (see Observability).

`AsyncCircuitBreaker` / `CircuitBreaker`¶

from httpware.middleware.resilience import AsyncCircuitBreaker  # async
from httpware.middleware.resilience import CircuitBreaker  # sync

A circuit breaker counts consecutive failures and, past a threshold, stops sending requests to the downstream for a while.

States¶

CLOSED: normal operation. After failure_threshold counted failures in a row, the circuit opens.
OPEN: until reset_timeout has passed, every request fails at once with CircuitOpenError, whose retry_after says how many seconds remain. The first request after that moves the circuit to HALF_OPEN and becomes the probe.
HALF_OPEN: one probe at a time is let through. After success_threshold successful probes in a row the circuit closes; one failed probe opens it again.

An outcome only counts against the state its request was admitted under. A request that is still in flight when the circuit changes state, such as a slow call sent before the circuit opened, finishes without touching the circuit. It cannot reopen a HALF_OPEN circuit, count toward success_threshold, or free the probe slot.

Constructor¶

Parameter	Default	Effect
`failure_threshold`	`5`	Consecutive counted failures required to open. `<1` raises `ValueError`.
`reset_timeout`	`30.0` (s)	Seconds to stay OPEN before admitting a probe. `<0` raises `ValueError`.
`success_threshold`	`1`	Consecutive probe successes required to close. `<1` raises `ValueError`.
`failure_status_codes`	`None`	Status codes that count as failures. `None` means every 5xx.
`failure_rate_threshold`	`None`	Setting it switches to rate mode: the share of failures in the rolling window that opens the circuit. `None` keeps the consecutive-failure mode.
`window_seconds`	`30.0` (s)	Rate mode only: the length of the rolling window.
`minimum_calls`	`20`	Rate mode only: how many outcomes the window needs before the rate is checked.

Failure classification¶

A counted failure is a NetworkError, an httpware TimeoutError, or a StatusError whose status is in failure_status_codes. Other exceptions pass through without changing the circuit.

An expired AsyncTimeout deadline counts only when the breaker comes before AsyncTimeout, as in the composition order. With the breaker after it, the deadline cancels the breaker's call, and a cancellation is not a counted failure.

A RetryBudgetExhaustedError from a retry placed after the breaker is classified by its last_exception. If the last attempt failed with a counted failure, the breaker counts one; if it failed with a 4xx, that is a success.

4xx responses, 429 included, count as successes. A 429 means the service is up and throttling you; opening the circuit on it would add circuit-open rejections on top of the throttling.

`CircuitOpenError`¶

Raised while the circuit is OPEN, with a positive retry_after, or while it is HALF_OPEN and a probe is already in flight, with retry_after=None. It subclasses httpware.ClientError; see Errors.

Observability¶

Emitted on logger httpware.circuit_breaker:

Event	When
`circuit.opened`	The failure threshold was reached and the circuit went from CLOSED to OPEN.
`circuit.rejected`	A request failed fast because the circuit was OPEN or a HALF_OPEN probe was already running. Emitted at most once per second per circuit, and always for the first rejection of an outage. Its `suppressed` attribute counts the rejections in the same outage that were not emitted since the previous `circuit.rejected`.
`circuit.half_open`	The reset timeout passed and the circuit went from OPEN to HALF_OPEN.
`circuit.closed`	The success threshold was reached and the circuit went from HALF_OPEN to CLOSED. Its `rejected` attribute is the number of requests rejected during the outage, emitted or not.

An outage runs from the moment the circuit opens until it closes again; a probe failure that reopens it stays in the same outage.

The breaker emits its events after releasing its internal lock, so a logging handler may send requests through a client that uses the same breaker.

Time-based failure-rate mode¶

By default the breaker opens after failure_threshold failures in a row. A downstream that fails every other request never produces a long enough streak, so the circuit stays closed at a 50% error rate.

Passing failure_rate_threshold switches to rate mode, which opens on the share of failures in a rolling window (parameters in the constructor table):

from httpware.middleware.resilience import AsyncCircuitBreaker


breaker = AsyncCircuitBreaker(
    failure_rate_threshold=0.5,  # open at ≥50% failures
    window_seconds=30.0,  # over a rolling 30s window
    minimum_calls=20,  # but only once 20+ calls are observed
)

Once the window holds at least minimum_calls outcomes, the rate is checked after every outcome, successes included. A success can therefore be the call that opens the circuit. With minimum_calls=4 and a threshold of 0.5, three failures leave the circuit closed because the window holds only three outcomes; a fourth call that succeeds brings it to four, the rate is 3/4, and the circuit opens. That call still returns its response to the caller, and the next request is rejected with CircuitOpenError.

The window moves in steps of window_seconds / 10, so an outcome stops counting between 0.9 * window_seconds and window_seconds after it was recorded.

In rate mode failure_threshold is ignored. Recovery through HALF_OPEN works the same in both modes, and the sync CircuitBreaker takes the same parameters.

State introspection¶

AsyncCircuitBreaker and CircuitBreaker have a read-only state property that returns a CircuitState:

from httpware import CircuitState
from httpware.middleware.resilience import AsyncCircuitBreaker

breaker = AsyncCircuitBreaker(failure_threshold=5)
# ... later, in a health/readiness handler:
if breaker.state is CircuitState.OPEN:
    ...  # report the dependency as degraded

Assigning to state raises AttributeError. The move from OPEN to HALF_OPEN happens when the first request arrives after reset_timeout, not when the timeout passes, so state reports OPEN until then. Reading state never changes it.

Pass the same instance to several clients to give them one circuit. A sync CircuitBreaker and an AsyncCircuitBreaker cannot share state, because they use different locking primitives.

Example¶

from httpware import AsyncClient
from httpware.middleware.resilience import AsyncCircuitBreaker


breaker = AsyncCircuitBreaker(failure_threshold=3, reset_timeout=60.0)

async with AsyncClient(
    base_url="https://api.example.com",
    middleware=[breaker],
) as client:
    response = await client.get("/users/1")

For sync code, use Client and CircuitBreaker without await.

`AsyncKeyedCircuitBreaker` / `KeyedCircuitBreaker`¶

from httpware.middleware.resilience import AsyncKeyedCircuitBreaker  # async
from httpware.middleware.resilience import KeyedCircuitBreaker  # sync

Use a keyed breaker when one client sends requests to more than one upstream. A plain AsyncCircuitBreaker holds a single circuit, so one failing upstream fast-fails requests to all the healthy ones. The keyed breaker keeps a separate circuit for each circuit key and opens only the failing upstream's circuit.

Each circuit behaves exactly like an AsyncCircuitBreaker built with the same arguments: the same states, failure classification, rate mode, half-open probe and events. Each circuit has its own probe slot, so two upstreams that recover at the same time are probed independently.

Constructor¶

Every AsyncCircuitBreaker parameter, with the same defaults, plus:

Parameter	Default	Effect
`key`	the request's origin	Maps a request to its circuit key. Any hashable value works. The default is the origin, built from scheme, host and port, which events show as `https://a.example` or `http://a.example:8080`. Hosts are lowercased and default ports dropped, so `https://A.example/x` and `https://a.example:443/y` share a circuit, while `http://a.example` and `https://a.example:8443` each get their own. Userinfo is never part of it.

Circuit lifetime¶

A circuit is created on the first request for its key and kept for as long as the breaker lives. Nothing is evicted or pruned, so memory grows with the number of distinct keys. key must therefore map requests to a small, bounded set, such as your configured upstreams. Never derive it from user input. A key function that returns something unbounded, such as the full URL, grows the map forever.

Observability¶

The keyed breakers emit the same events as AsyncCircuitBreaker on the same httpware.circuit_breaker logger. Each event has one extra attribute, circuit_key: the origin string for the default key, otherwise the str() of the request's circuit key. httpware redacts url but not circuit_key, which reaches log records and span events exactly as the key function returned it. A custom key must not return credentials, tokens or other sensitive values. The default origin contains none.

Example¶

from httpware import AsyncClient
from httpware.middleware.resilience import AsyncKeyedCircuitBreaker, AsyncRetry


async with AsyncClient(
    middleware=[AsyncKeyedCircuitBreaker(failure_threshold=5, reset_timeout=60.0), AsyncRetry()],
) as client:
    await client.get("https://suggest-a.example/v1/suggest")
    await client.get("https://suggest-b.example/v1/suggest")  # unaffected if suggest-a is down

In the composition order, the keyed breaker takes the circuit breaker's place, before AsyncRetry, so it counts one outcome per retry sequence. For sync code, use Client and KeyedCircuitBreaker without await.

`AsyncTimeout`¶

from httpware.middleware.resilience import AsyncTimeout

Bounds the total time of everything after it in the chain. Put it right after the circuit breaker and before AsyncBulkhead and AsyncRetry, so a call, with all its retries and backoff, finishes within timeout seconds and the breaker counts an expired deadline as a failure. When time runs out it raises httpware.TimeoutError.

Parameter	Default	Effect
`timeout`	required	Overall deadline in seconds. Must be finite and greater than 0, or `ValueError` is raised.

AsyncTimeout does not replace per-request timeouts. httpx2's connect, read, write and pool timeouts bound each request; AsyncTimeout bounds the whole retry sequence, which httpx2 can't.

There is no sync Timeout, because sync Python cannot interrupt a blocking httpx2 call midway. In sync code, set httpx2.Timeout on the client or pass timeout= per request.

On expiry it emits timeout.exceeded on httpware.timeout. Composition has an example with AsyncTimeout in the recommended place.

Composition¶

httpware doesn't enforce an order, but this one is recommended:

AsyncCircuitBreaker → AsyncTimeout → AsyncBulkhead → AsyncRetry → terminal

AsyncCircuitBreaker comes before AsyncTimeout, so an expired deadline reaches the breaker as an httpware TimeoutError, which it counts as a failure, and an upstream that hangs opens the circuit. In the reverse order the deadline fires while the breaker is still waiting on its inner call: the breaker sees a CancelledError, which it doesn't count, and the circuit stays closed however long the upstream hangs. The breaker itself never waits, so the deadline loses nothing by starting after it. An open circuit still rejects at once, and the half-open probe runs under the deadline like any other call.
AsyncTimeout comes before AsyncBulkhead and AsyncRetry, so the deadline covers the wait for a slot, every retry and every backoff.
AsyncCircuitBreaker comes before AsyncRetry, so an open circuit rejects the call without starting the retry loop, and the breaker counts one outcome per retry sequence instead of one per attempt. Putting it before AsyncBulkhead too means a rejected call never takes a slot.
AsyncBulkhead comes before AsyncRetry, so one slot covers every attempt of a call. In the reverse order each retry takes a new slot, and under load the bulkhead admits more work than its cap intends.

from httpware import AsyncClient
from httpware.middleware.resilience import (
    AsyncBulkhead,
    AsyncCircuitBreaker,
    AsyncRetry,
    AsyncTimeout,
)


async def main() -> None:
    async with AsyncClient(
        base_url="https://api.example.com",
        middleware=[
            AsyncCircuitBreaker(),
            AsyncTimeout(timeout=30.0),
            AsyncBulkhead(max_concurrent=10),
            AsyncRetry(),
        ],
    ) as client:
        await client.get("/users/1")

Your own middleware that sets something per call, like the request-ID middleware in Middleware, also belongs before AsyncRetry, so every attempt of a call shares one ID.

One event loop per instance¶

AsyncBulkhead, AsyncCircuitBreaker and AsyncKeyedCircuitBreaker are bound to the event loop they first run on. Using the same instance from another loop raises RuntimeError, and so does a later asyncio.run() in the same thread, because each asyncio.run() starts a new loop. httpx2.AsyncClient can't be reused across asyncio.run() calls either. Create the async resilience middleware, and the AsyncClient holding it, once per event loop, for example inside the coroutine you pass to asyncio.run():

import asyncio

from httpware import AsyncClient
from httpware.middleware.resilience import AsyncCircuitBreaker, AsyncRetry


async def main() -> None:
    async with AsyncClient(
        base_url="https://api.example.com",
        middleware=[AsyncCircuitBreaker(), AsyncRetry()],
    ) as client:
        await client.get("/users/1")


asyncio.run(main())

The usual way to hit the error is in tests: pytest-asyncio gives each test function its own loop by default, so an instance created at module level or in a session-scoped fixture fails in the second test that uses it. Create it in a function-scoped fixture or in the test.

To share a cap or a circuit across threads, use the sync classes with Client.

Sync Retry and Bulkhead¶

Use these with the sync Client.

`Retry`¶

from httpware.middleware.resilience import Retry

Retry takes the same parameters as AsyncRetry and sleeps with time.sleep between attempts. Retry-After handling, streaming bodies, giving up, and RetryBudgetExhaustedError all work the same way.

There is no sync Timeout (see AsyncTimeout); set httpx2.Timeout on the client or pass timeout= per request.

`Bulkhead`¶

from httpware.middleware.resilience import Bulkhead

Bulkhead is AsyncBulkhead on a threading.Semaphore. The slot is released in the same try/finally, including on KeyboardInterrupt and other interrupts.

One bulkhead cannot cap sync and async clients together, because Bulkhead and AsyncBulkhead use different semaphores. If you create one of each with the same max_concurrent, each is enforced separately, so together they admit up to twice that many requests.

Composition with sync `Client`¶

The composition order applies with the sync classes, minus the timeout: CircuitBreaker, then Bulkhead, then Retry.

Resilience reference¶

AsyncRetry¶

Retry-After parsing¶

Streaming-body refusal¶

Exhaustion behavior¶

RetryBudget¶

The token-bucket formula¶

Why a floor matters¶

Sharing across clients¶

Thread safety¶

AsyncBulkhead¶

Slot release contract¶

Sharing across clients¶

Rejection¶

AsyncCircuitBreaker / CircuitBreaker¶

States¶

Constructor¶

Failure classification¶

CircuitOpenError¶

Observability¶

Time-based failure-rate mode¶

State introspection¶

Sharing¶

Example¶

AsyncKeyedCircuitBreaker / KeyedCircuitBreaker¶

Constructor¶

Circuit lifetime¶

Observability¶

Example¶

AsyncTimeout¶

Composition¶

One event loop per instance¶

Sync Retry and Bulkhead¶

Retry¶

Bulkhead¶

Composition with sync Client¶

See also¶

`AsyncRetry`¶

`RetryBudget`¶

`AsyncBulkhead`¶

`AsyncCircuitBreaker` / `CircuitBreaker`¶

`CircuitOpenError`¶

`AsyncKeyedCircuitBreaker` / `KeyedCircuitBreaker`¶

`AsyncTimeout`¶

`Retry`¶

`Bulkhead`¶

Composition with sync `Client`¶