Package middleware
Go module github.com/bide-ai/bide; all packages.
github.com/bide-ai/bide/middleware · pkg.go.dev · Stability: core
Package middleware provides composable func(Handler) Handler batteries for the model call, the "LiteLLM for Go" layer at the semantic level (handlers see messages, tool calls, and token usage, not raw bytes).
Functions
func Cost(m *CostMeter, r Rates) agent.Middleware: Cost is a model middleware that records token usage and computes running cost, wherever it sits in the chain.func ErrorSummary(err error) string: ErrorSummary describes err by what failed, never by what its text says: the control-flow signal it is (pending approval, resume halted, saga aborted), the provider's HTTP status and error type or code (from *agent.APIError) or a rate limit, and the condition or category sentinel it wraps.func Hedge(delay time.Duration, backups ...agent.Model) agent.Middleware: Hedge races the model call against one or more backup models and returns the FIRST successful response, cancelling the rest.func RateLimit(r *RateLimiter) agent.Middleware: RateLimit throttles model requests through the shared limiter, blocking (respecting context cancellation) until a token is available.func Retry(n int, opts ...RetryOption) agent.Middleware: Retry retries the model call up to n additional times on error, respecting context cancellation.func Retryable(err error) bool: Retryable is a ready-made classifier for WithRetryIf: it retries transient failures and fails fast on terminal ones.func ToolCache() agent.ToolMiddleware: ToolCache memoizes successful tool results by (name, args) in a process-local map and short-circuits a repeat call, the tool never runs on a hit.func ToolLog(logf func(format string, args ...any), opts ...ToolLogOption) agent.ToolMiddleware: ToolLog logs one line per tool call, name, duration, and ok/error, via logf (pass log.Printf, t.Logf, or a structured logger's Printf-shaped method).func ToolRateLimit(r *RateLimiter) agent.ToolMiddleware: ToolRateLimit throttles tool calls through the shared limiter.func ToolRetry(n int, opts ...RetryOption) agent.ToolMiddleware: ToolRetry retries a failing tool call up to n additional times with exponential backoff + jitter (honoring an *agent.RateLimited RetryAfter), respecting context cancellation.
type CostMeter
type CostMeter struct {
// contains filtered or unexported fields
}CostMeter accumulates token usage and computed cost across model calls, in two views: the answers, the responses the calls returned, which a run records; and the spend, every request the calls sent, including failed attempts a Retry repeated and losing Hedge targets, which the provider bills too.
Methods
func (m *CostMeter) Snapshot() CostSnapshot: Snapshot returns the meter's totals, read together under one lock, so the answer and spend views are always of the same moment.
type CostSnapshot
type CostSnapshot struct {
Answer, Spend agent.Usage
AnswerUSD, SpendUSD float64
}CostSnapshot is a CostMeter's totals at one moment.
type RateLimiter
type RateLimiter struct {
// contains filtered or unexported fields
}RateLimiter is a token-bucket limiter shared across calls: it allows one token every interval with a burst up to burst.
Constructors and functions
func NewRateLimiter(interval time.Duration, burst int) *RateLimiter: NewRateLimiter allows one call everyinterval(so rate = 1/interval), bursting up toburstcalls.
type Rates
type Rates struct {
InputPer1M float64
OutputPer1M float64
CacheReadPer1M float64
CacheWritePer1M float64
}Rates holds the billing rates for a model in USD per 1,000,000 tokens.
Methods
func (r Rates) Cost(u agent.Usage) float64: Cost returns the USD cost of the token usage u at these rates.
type RetryOption
type RetryOption func(*retryConfig)RetryOption configures optional behavior of Retry.
Constructors and functions
func WithBackoff(base, max time.Duration) RetryOption: WithBackoff sets the initial backoff duration and the maximum cap.func WithRetryIf(pred func(error) bool) RetryOption: WithRetryIf classifies which errors are worth retrying: pred returns true to retry, false to fail fast.func WithTimeout(d time.Duration) RetryOption: WithTimeout bounds each individual attempt with its own deadline: the call gets a context that is cancelled after d, so a hung model or tool call fails that attempt (and is retried) instead of blocking the run forever.
type ToolLogOption
type ToolLogOption func(*toolLogConfig)ToolLogOption configures ToolLog.
Constructors and functions
func LogErrorText() ToolLogOption: LogErrorText makes ToolLog log a failed call's error text instead of its ErrorSummary: the text the agent journals for the call and sends to the model (agent.ToolCall.ErrorText), which is the agent's WithToolErrorRedactor text if one is set, with every URL in it redacted.