Wire providers

provider.Provider is the streaming-completion abstraction. Adapters share a request shape, a chunk state machine, and a conformance suite. Pick one — switching is a one-line change.

The interface

type Provider interface {
    Info() Info
    Stream(ctx context.Context, req *Request) (EventStream, error)
}

type Capabler interface {
    Capabilities() Capabilities
}

type Capabilities struct {
    Tools         bool
    ToolChoice    bool
    Reasoning     bool
    StopSequences bool
    CacheControl  bool
    RequestID     bool
}

Adapters that don't support a feature report false from Capabilities(). The conformance suite skips capability-gated assertions when an adapter reports false, so adapter authors don't have to fake support.

In-tree adapters

Each New returns (provider.Provider, error) — the interface, not a struct pointer. Use the error path: missing API keys fail at construction, not Run.

OpenAI

import "github.com/jerkeyray/starling/provider/openai"

prov, err := openai.New(
    openai.WithAPIKey(os.Getenv("OPENAI_API_KEY")),
)

OpenAI-compatible endpoint examples:

// Groq
prov, _ := openai.New(
    openai.WithBaseURL("https://api.groq.com/openai/v1"),
    openai.WithAPIKey(os.Getenv("GROQ_API_KEY")),
    openai.WithProviderID("groq"),
)

// Local Ollama
prov, _ := openai.New(
    openai.WithBaseURL("http://localhost:11434/v1"),
    openai.WithProviderID("ollama"),
)

// Azure OpenAI
prov, _ := openai.New(
    openai.WithBaseURL("https://YOUR-RESOURCE.openai.azure.com/openai"),
    openai.WithAPIKey(os.Getenv("AZURE_OPENAI_KEY")),
    openai.WithAPIVersion("2024-08-01-preview"),
)

Anthropic

import "github.com/jerkeyray/starling/provider/anthropic"

prov, err := anthropic.New(
    anthropic.WithAPIKey(os.Getenv("ANTHROPIC_API_KEY")),
)

Capabilities reported by Anthropic: tool use, extended thinking with per-block signatures, prompt caching metadata. Cache-control hints are attached via the Params field on the Request (CBOR blob).

Gemini

import "github.com/jerkeyray/starling/provider/gemini"

prov, err := gemini.New(
    gemini.WithAPIKey(os.Getenv("GEMINI_API_KEY")),
)

Amazon Bedrock

import (
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/jerkeyray/starling/provider/bedrock"
)

awsCfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("us-east-1"))
if err != nil {
    return err
}

prov, err := bedrock.New(bedrock.WithAWSConfig(awsCfg))

The adapter calls native ConverseStream, so it accepts every model id Bedrock Converse accepts: foundation model ids, inference profiles, prompt ARNs, and provisioned-throughput ARNs. Bedrock-specific request fields — additionalModelRequestFields, additionalModelResponseFieldPaths, requestMetadata, performanceConfig, serviceTier, promptVariables, outputConfig, guardrailConfig — are passed through Request.Params. Unknown keys are rejected, so misspelled fields don't silently no-op.

Capabilities reported by Bedrock: tool use (no none tool-choice), reasoning content with signatures, redacted thinking, cache-aware usage counters, top_k via additionalModelRequestFields.

OpenRouter

import "github.com/jerkeyray/starling/provider/openrouter"

prov, err := openrouter.New(
    openrouter.WithAPIKey(os.Getenv("OPENROUTER_API_KEY")),
    openrouter.WithHTTPReferer("https://your-app.com"),
    openrouter.WithXTitle("Your App"),
)

OpenRouter is a thin wrapper over the OpenAI adapter - same chunk contract, plus attribution headers. Use OpenAI-style model strings.

Error classification

Adapters wrap their underlying SDK / HTTP errors with one of four sentinels so retry policy lives in caller code, not in vendor-string parsing. Categories are non-overlapping; an error wraps at most one.

var (
    provider.ErrRateLimit  // 429 / quota
    provider.ErrAuth       // 401 / 403
    provider.ErrServer     // 5xx
    provider.ErrNetwork    // DNS / dial / TLS / broken stream
)

The two helpers used by every in-tree adapter:

provider.WrapHTTPStatus(err, status int) error
provider.ClassifyTransport(err error) error

WrapHTTPStatus annotates by HTTP status code; status == 0 delegates to ClassifyTransport, which wraps net.Error and *url.Error with ErrNetwork. Statuses outside the four categories pass through unmodified - 4xx errors that are neither auth nor rate-limit (invalid request, model not found) reflect a caller bug, not a transient condition, so they stay unwrapped on purpose.

Caller code:

_, stream, err := prov.Stream(ctx, req)
switch {
case errors.Is(err, provider.ErrRateLimit):
    backoff(); retry()
case errors.Is(err, provider.ErrAuth):
    return fmt.Errorf("starling: bad credentials: %w", err)
case errors.Is(err, provider.ErrServer), errors.Is(err, provider.ErrNetwork):
    backoff(); retry()
case err != nil:
    return err  // 4xx caller bug; surface unmodified
}

If you write your own provider adapter, call WrapHTTPStatus from the entry point that receives the upstream response and your callers automatically get the same retry contract.

The streaming chunk contract

Adapters yield StreamChunk values. Kinds:

ChunkText
ChunkReasoning
ChunkRedactedThinking
ChunkToolUseStart
ChunkToolUseDelta
ChunkToolUseEnd
ChunkUsage
ChunkEnd

step.LLMCall enforces the state machine:

• No EOF before ChunkEnd.
• No duplicate ChunkToolUseStart for the same call id.
• No chunks after ChunkEnd.
• ChunkUsage is optional; budgets enforce mid-stream when it arrives.

If an adapter violates the contract, step.ErrInvalidStream is the typed error.

Capability-gated features

Check what an adapter supports before enabling a feature:

if c, ok := prov.(provider.Capabler); ok {
    caps := c.Capabilities()
    if caps.CacheControl {
        // attach cache markers via Params
    }
    if !caps.Reasoning {
        // don't ask for chain-of-thought
    }
}

The conformance suite (provider/conformance/) is the contract test adapter authors run against fixtures. If you write your own adapter, plug it into the suite — it's the cheapest way to get the chunk ordering, tool id stability, and cancellation right.

Switching providers

Mid-fleet swap is one line:

// Before
prov, _ := openai.New(openai.WithAPIKey(os.Getenv("OPENAI_API_KEY")))
a.Config.Model = "gpt-4o-mini"

// After
prov, _ := anthropic.New(anthropic.WithAPIKey(os.Getenv("ANTHROPIC_API_KEY")))
a.Config.Model = "claude-haiku-4-5-20251001"

Note: changing the provider or model invalidates replay fixtures. RunStarted records ProviderID, ModelID, and a hash of the Params blob. Replay against an old fixture with a new provider fails fast on RunStarted payload divergence.

Anti-patterns

• Hard-coding the API key in source. Always read from env.
• Ignoring the error from New. Construction validates the config; treating it as infallible hides typos.
• Using one provider for production but a different one for replay fixtures. Replay only works against the recorded provider; pick one and stick to it per fixture.
• Setting WithProviderID to a value that varies per process. The id is hashed into RunStarted. Make it stable.

Where to next