LinkedInGitHub

Use cases & examples

stubfetch replaces live HTTP with deterministic in-process handlers—ideal for agent evals, integration tests, and recording traffic.

Automated tests: Vitest (Node) and pytest (Python). Source: github.com/vgulerianb/stubfetch.

Python vs Node: fetch() in Python returns (status: int, body: str). The npm openai() chat preset exists only on the TypeScript side today; Python covers GitHub, Stripe, S3, Slack, Anthropic, Postgres, etc.

Testing agents with stubfetch

stubfetch is for tests and evals: you control what HTTP looks like so the agent’s behavior is repeatable. Common patterns:

  1. Inject env.fetch (or wrap globalThis.fetch) so the agent’s code never hits the real network.
  2. Seed providers (GitHub, Stripe, …) with the exact JSON your scenario needs.
  3. After the agent finishes, wasCalled / was_called and calls() prove it hit the right APIs with the right methods/paths.
  4. Use runEval / run_eval to run many scenarios in one script; add chaos from Testing & chaos to stress error handling.

Inject deterministic fetch into your agent

Your agent module should accept a fetch implementation (dependency injection). In tests, pass GhostEnv#fetch.

import { GhostEnv, github } from "stubfetch";

type AgentFetch = typeof fetch;

/** Example: agent that lists issues using injected fetch */
export async function agentListIssues(fetchImpl: AgentFetch, repo: string) {
  const res = await fetchImpl(`https://api.github.com/repos/${repo}/issues`);
  if (!res.ok) throw new Error(`github ${res.status}`);
  return res.json();
}

// Test: no network — stubbed issues only
export async function testAgentUsesGithubStub() {
  const env = new GhostEnv({
    seed: 1,
    providers: [
      github({
        issues: [{ repo: "acme/api", title: "Stub issue", body: "test" }],
      }),
    ],
  });

  const data = await agentListIssues(env.fetch.bind(env) as AgentFetch, "acme/api");
  console.assert(Array.isArray(data) && data[0].title === "Stub issue");
  console.assert(env.wasCalled("github", { method: "GET" }));
}

If you cannot change the agent API, wrap globalThis.fetch in test setup (restore after) or use your runtime’s hook for outbound HTTP.

End-to-end style: run the agent inside runEval

Use run to invoke your agent entrypoint; use assert / check to validate env after it returns.

import { runEval, defineScenario, github, stripe } from "stubfetch";

const report = await runEval([
  defineScenario({
    name: "agent lists stubbed issues",
    config: {
      providers: [
        github({ issues: [{ repo: "acme/api", title: "Eval ticket" }] }),
        stripe({ customers: [{ email: "[email protected]" }] }),
      ],
    },
    run: async (env) => {
      // Replace with: await runMyAgent(env) where your agent uses env.fetch
      const res = await env.fetch("https://api.github.com/repos/acme/api/issues");
      console.assert(res.status === 200);
      await res.json();
    },
    assert: (env) => {
      if (!env.wasCalled("github", { method: "GET" }))
        throw new Error("expected GitHub list");
    },
  }),
]);

console.log(report.passRate, report.results);

(my_agent is illustrative—point the import at your real agent module.)

Golden recordings for regression

After a successful agent run, exportRecordingJSON / export_recording_json can be committed as a baseline; diff when behavior changes—see §3 Record calls below.

1. Stub GitHub for an agent test

import { GhostEnv, github, exportRecordingJSON } from "stubfetch";

const env = new GhostEnv({
  seed: 7,
  providers: [
    github({
      issues: [{ repo: "acme/api", title: "P0 outage", body: "details here" }],
    }),
  ],
});

const res = await env.fetch("https://api.github.com/repos/acme/api/issues");
console.assert(res.status === 200);
const data = await res.json();
console.assert(data[0].title === "P0 outage");

console.log(exportRecordingJSON(env.calls()));

2. Stripe + GitHub together (integration-style)

import { GhostEnv, github, stripe } from "stubfetch";

const env = new GhostEnv({
  providers: [
    github({ issues: [{ repo: "acme/billing", title: "Invoice bug" }] }),
    stripe({ customers: [{ email: "[email protected]" }] }),
  ],
});

const customers = await env.fetch("https://api.stripe.com/v1/customers");
console.assert(customers.status === 200);

const issues = await env.fetch("https://api.github.com/repos/acme/billing/issues");
console.assert(issues.status === 200);

Providers are tried in order; the first match wins.

3. Record calls for a fixture or LLM transcript

After exercising fetch, export a stable JSON trace.

import { writeFileSync } from "node:fs";
import { GhostEnv, github, exportRecordingJSON } from "stubfetch";

const env = new GhostEnv({ providers: [github({ issues: [] })] });
await env.fetch("https://api.github.com/repos/acme/api/issues");

writeFileSync("fixture.json", exportRecordingJSON(env.calls()));

4. Eval harness (sequential scenarios)

import { runEval, defineScenario, github } from "stubfetch";

const report = await runEval([
  defineScenario({
    name: "lists issues",
    config: { providers: [github({ issues: [{ repo: "a/b", title: "t" }] })] },
    run: async (env) => {
      await env.fetch("https://api.github.com/repos/a/b/issues");
    },
    assert: (env) => {
      if (!env.wasCalled("github", { method: "GET" })) throw new Error("expected GET");
    },
  }),
]);

console.log(report.passRate, report.results);

Python uses check= instead of assert (keyword conflict).

5. TypeScript-only: canned OpenAI chat completions

import { GhostEnv, openai } from "stubfetch";

const env = new GhostEnv({
  providers: [
    openai({
      responses: [
        {
          match: { model: "gpt-4" },
          response: { choices: [{ message: { content: "pong" } }] },
        },
      ],
    }),
  ],
});

const res = await env.fetch("https://api.openai.com/v1/chat/completions", {
  method: "POST",
  body: JSON.stringify({ model: "gpt-4", messages: [] }),
});
const body = await res.json();
console.log(body.choices[0].message.content);

Next steps