gateway

js/ai/gateway.ts

fino:ai/gateway — per-key model policy and credential-safe realm facades.

GatewayPolicy uses a generic atomic store counter to smooth requests over a fixed window. gatewayModel() applies it immediately before provider work. modelFacade() keeps the real model and its credentials in the parent realm while exposing only generation methods to a child.

import { memoryStore } from 'fino:store';
import { gatewayModel, GatewayPolicy } from 'fino:ai/gateway';

const policy = new GatewayPolicy({
  store: memoryStore(),
  requests: 60,
  windowMs: 60000,
});
const tenantModel = gatewayModel(parentModel, { policy, key: 'tenant-42' });

Interfaces

interface GatewayPolicyOptions {

Fixed-window gateway policy options.

Properties

store: AtomicExpiringStore

Generic store used for atomic, expiring counters.

requests: number

Requests allowed for each key in one window.

windowMs: number

Window duration in milliseconds.

clock?: () => number

Deterministic clock hook. Defaults to Date.now.

interface GatewayModelOptions {

Options for gatewayModel().

Properties

policy: GatewayPolicy

Policy checked before every generation.

key: string | (() => string)

Static key or parent-side resolver for the current tenant/API key.

interface ModelFacadeOptions {

Options for a child-realm model facade.

Properties

specifier: string

Synthetic module specifier imported by the child.

Classes

class GatewayRateLimitError extends Error {

Error returned when a gateway key has exhausted its current window.

Readonly Properties

readonly key: string

Rejected tenant/API key.

readonly retryAfterMs: number

Milliseconds until the counter window resets.

readonly retryAt: number

Absolute reset timestamp.

readonly limit: number

Configured request limit.

Constructors

constructor(key: string, retryAt: number, now: number, limit: number)

Create structured retry metadata for a rejected key.

class GatewayPolicy {

Atomic per-key request counter backed directly by fino:store.

Constructors

constructor(options: GatewayPolicyOptions)

Create a policy. Counters are isolated by the exact key passed to acquire().

Methods

async acquire(key: string): Promise<{ remaining: number; resetAt: number }>

Consume one request from key, retrying optimistic-store conflicts.

Throws GatewayRateLimitError before provider invocation when the current window is full.

Functions

function gatewayModel(base: Model, options: GatewayModelOptions): Model

Wrap a model with per-key gateway policy.

function modelFacade(resolve: () => Model | null, options: ModelFacadeOptions): Facade

Expose model generation without exposing the model object or credentials.

resolve runs in the parent for every call. Return a newly configured model to rotate credentials, or null to revoke access immediately.