Multi-agent architecture for measuring the energy consumption of LLMs based on prompting techniques and context length

Problem

Every prompt engineering guide optimizes the same variable: answer quality. Add examples, ask the model to think step by step, give it more context. None of them report the other side of the ledger. Chain-of-thought produces longer answers and, with them, longer inference, and inference is where LLMs spend most of their energy at scale.

The question this project set out to answer is narrow enough to be measurable: how much extra energy does a prompting technique cost, and does the quality it buys justify it? Answering that honestly means measuring energy and quality on the same runs, across models that behave nothing alike, from a 3B model on a local GPU to a frontier API endpoint whose internals are invisible.

This was my MSc thesis in Data Science at the Universitat Oberta de Catalunya (tutor: Josep-Anton Mir Tutusaus), defended in June 2026.

Architecture

The system is a multi-agent harness orchestrated with LangGraph, split into three phases. That split carries the whole design: LLM non-determinism is confined to a pre-processing phase that runs once and writes its output to disk, so the experiment loop replays identical prompts across every repetition.

The experimental grid: three tasks (XSum summarization, HumanEval code generation, GSM8K arithmetic reasoning) × 5 instances × 12 cells (3 techniques × 2 context lengths × 2 syntactic formats) × 5 models × 3 repetitions = 2,700 evaluated inferences.

Metering had to be split by model type, because no single tool sees both. Local models run through Ollama as an external process, so CodeCarbon runs in tracking_mode='machine' around the call. Remote models are opaque, so EcoLogits estimates from token counts and published model characteristics. The two are not interchangeable and are never compared directly: every statistical model is fitted separately for API and local.

The judge scores each answer against the rubric for its instance, blind to which technique produced it, with structured output via tool_use.

Decisions & trade-offs

• The Router is an LLM at pre-processing time and a dictionary lookup at runtime. Generating prompt variants on the fly would have been a more impressive agent demo, but then the three repetitions of each cell would have received three slightly different prompts and the variance would have been unattributable. The runtime graph is therefore less agentic than the architecture diagram suggests, which is deliberate.
• Rubrics are semi-dynamic: a fixed structure per task with contextual slots filled by an LLM. Fully generated rubrics would score every instance on a different scale; fully static ones would score a summary without knowing which facts mattered. That means more moving parts than a single global rubric, in exchange for scores that are comparable across cells.
• The judge comes from a model family absent from the experiment. Anthropic judges OpenAI, Google and local open-weight models, so no model can prefer its own output, and 13.3% of the runs were re-scored by a stronger judge to validate the cheap one. It worked, and it was expensive: the meta-system ended up burning more energy than everything it was measuring.
• No thermal cooldowns between runs; GPU temperature is a covariate instead. Waiting for the GPU to return to baseline between inferences would have added roughly 13 hours to the run. Temperature had to be modelled rather than controlled, and the model priced it at +6.9% energy per additional degree, which turned an operational shortcut into a result.

Metrics

All figures come from the consolidated run (f3fb12ab-8bab-…, 2,700 executions over ~11 hours) and the statistical models in the thesis, chapter 4.

Metric	Value
Evaluated inferences	2,700 · 100% judged · 0 unrecovered failures (182 retried)
Energy: zero-shot vs CoT (API)	zero-shot draws 33.5% of CoT
Energy: few-shot vs CoT (API)	few-shot draws 31.4% of CoT
Energy: zero-shot vs CoT (local)	zero-shot draws 50%, few-shot 64.6%
Effect size, zero-shot vs CoT	Cohen’s d = −0.485 (small to medium), p < 1e−63
Effect size, zero-shot vs few-shot	Cohen’s d = −0.071 (negligible)
Quality (judge, 0 to 100)	zero-shot 49.0 · few-shot 52.4 · CoT 53.1
XML vs plain text, long context	−29.8% energy
Model spread	gpt-5-mini ≈ 26% of gemini-2.5-flash for the same cell
GPU temperature	+6.9% energy per additional degree
Inter-judge agreement (n = 359)	Spearman ρ = 0.948
Judge vs objective metric	ρ = 0.868 (GSM8K) · 0.600 (HumanEval) · 0.467 (XSum)
Total energy accounted	826.47 Wh · 143.16 g CO₂
Meta-system share of that total	59.4%

Lessons learned

Few-shot is the honest default. It matches chain-of-thought on quality for API models, where the difference is not statistically significant, while costing what zero-shot costs. Chain-of-thought buys its quality with a real, measurable energy premium, and on local models that premium doubles consumption. The common advice to “just add step-by-step reasoning” is not free, and now there is a number attached to it.

Measuring cost more energy than the thing being measured. The judge and its validation pass accounted for 59.4% of total consumption. That is an uncomfortable result to publish in a Green AI thesis, and reporting it is more useful than hiding it: any LLM-as-a-judge pipeline needs its own efficiency budget, and the validation judge in particular deserves a cheaper strategy.

Validating the judge is most of the work. Three independent layers (inter-judge agreement, correlation with objective metrics, and a verbosity-bias check) all passed except one: quadratic-weighted kappa for gsm8k.conciseness came in at 0.187, well under the 0.4 threshold. One criterion out of fifteen failed, and pretending otherwise would have invalidated the rest. Conciseness on short arithmetic answers turns out to be something two judges simply disagree about.

What I would do differently: five instances per task is thin for generalizing at the instance level, and ten to fifteen would have been better, scaling the cost proportionally. I would also add TOON as a third syntactic format. If XML already cuts 29.8% off long contexts, a token-oriented notation is the obvious next lever.

Links

• Source code, artifacts and data: github.com/lepablito/TFM_UOC_PabloMarcosParra
• Full thesis (67 pp., PDF, written in Spanish): Multi-agent architecture for measuring the energy impact of prompting techniques and context length on heterogeneous LLMs