Loading…
Loading…
Written by Max Zeshut
Founder at Agentmelt · Last updated Sep 9, 2026
The property that a system can produce different outputs from the same input. It is the root cause of why AI agents are hard to make reliable and why ordinary software-testing intuitions fail on them. A traditional function is deterministic — same input, same output, forever — so you can test it once and trust it. An agent is non-deterministic: the language model samples from a probability distribution over next tokens (even at temperature zero, floating-point and infrastructure effects leave residual variance), the context it sees shifts as memory and retrieval change, and the world it acts on differs every run. You can *reduce* variance (low temperature, tight prompts, structured outputs, pinned model versions) but not eliminate it, because the flexibility that lets an agent handle novel tasks is the same flexibility that makes its behavior vary — determinism and adaptability are the same dial. The practical consequence: an agent is never 'verified once and trusted forever'; it must be measured continuously, and any single passing run is one sample from a distribution, not a guarantee.
Asked twice to 'reconcile this invoice against the PO,' an agent produces two valid but different plans — one checks totals first, the other line items; one calls the tax API, the other reasons about tax itself. Both may be correct, but they are not identical, which is why a `expect(result).toBe(x)` assertion either rejects good behavior or gets loosened until it tests nothing. The fix is statistical evaluation over many runs, not exact-match tests.
See it as a workflow
Document & Proposal Generation WorkflowTrigger, steps, n8n nodes, guardrails and an importable template — plus what it costs to have it built.
Or skip the build
Workflows from $197/month, custom agents from $2,000.