Skip to content
needle.js

Local tool calling, down to the last bit

A 13.7 MB Needle 2 runtime for TypeScript, with direct, Vercel AI SDK, and pi integrations.
Get startedRun the browser demoView on GitHub
45M parameters13.7 MB CQ weightsNo API keyCPU + WebGPU

One model, three interfaces

Section titled “One model, three interfaces”

needle.js

Load the official .cact archive, define JSON Schema tools, and run the complete execution loop without a framework.

Start with the core library →

Vercel AI SDK

Use a standard LanguageModelV4 with generateText(), streamText(), AI SDK tools, and structured output.

Use the AI SDK provider →

pi-ai + Agent

Register a native, zero-cost pi-ai provider or install it directly into pi as a local model extension.

Build a pi agent →

Build the engine

Follow a token through the model, decode Cactus Quants, and implement a reference inference loop and byte grammar from scratch.

Start the model tutorial →

Why Needle?

Section titled “Why Needle?”

Needle 2 is purpose-built for selecting tools and extracting their arguments. The model combines grouped-query attention, four-lane hyper-connections, a fixed Hadamard MLP, and hashed n-gram memory. Cactus Quants keeps 45 million parameters in a compact archive that can be read without expanding every matrix.

needle.js implements that stack in TypeScript and keeps the deployment contract simple:

  1. Load embedded, local, or cached weights.
  2. Describe a small set of tools with JSON Schema.
  3. Let a continuous byte grammar make malformed calls unreachable.
  4. Execute above your chosen confidence threshold.

On Apple WebKit with WebGPU, greedy generation measures about 45 tok/s in pure TypeScript and 55 tok/s with resident TypeGPU execution. Small projections stay on CPU because a WebGPU round trip costs more than their compute. See the numbers.