Google: Gemma 4 26B A4B

google/gemma-4-26b-a4b-it · google · context 262 144 · in $0.070/1M · out $0.340/1M

Global Index

710

95% CI [542–879] · index v0.2.0

Per-domain scores

Domain	Score (95% CI)		Accuracy (IRT)	Consistency	Calibration	Contam. Δ	p50	$/1k
knowledge	710 [541–878]		0.516	1.00	1.00	0.000	619ms	$0.012

Every answer, every test

Full transparency: the model's latest graded answer on every instantiated item of the current index version. Answer keys never leave the server; anchor-item prompts are withheld to protect the longitudinal subset.

knowledge 30/30 correct

correctknowledge.fr.factbank-v2conf 100% · 669ms · $0.000 · 17 tok

question

Name the chemical element with symbol W.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Tungsten

correctknowledge.fr.factbank-v2conf 100% · 619ms · $0.000 · 24 tok

question

Identify the writer of the novel "Snow Country".

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Yasunari Kawabata

correctknowledge.fr.factbank-v2conf 100% · 1.1s · $0.000 · 15 tok

question

What is the capital of Kazakhstan?

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Astana

correctknowledge.fr.factbank-v2conf 100% · 1.1s · $0.000 · 24 tok

question

Name the writer of the novel "Snow Country".

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Yasunari Kawabata

correctknowledge.fr.factbank-v2conf 100% · 560ms · $0.000 · 24 tok

question

Name the author of "Snow Country".

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Yasunari Kawabata

correctknowledge.fr.factbank-v2conf 100% · 4.6s · $0.000 · 14 tok

question

Name the chemical element with symbol Sn.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Tin

correctknowledge.fr.factbank-v2conf 100% · 131ms · $0.000 · 24 tok

question

Name the writer of the novel "Snow Country".

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Yasunari Kawabata

correctknowledge.fr.factbank-v2conf 100% · 1.0s · $0.000 · 17 tok

question

Name the element whose symbol is W.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Tungsten

correctknowledge.fr.factbank-v2conf 100% · 1.8s · $0.000 · 15 tok

question

What is the capital of Brazil?

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Brasília

correctknowledge.fr.factbank-v2conf 100% · 232ms · $0.000 · 15 tok

question

Name the Brazilian capital city.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Brasília

correctknowledge.fr.factbank-v2conf 100% · 4.5s · $0.000 · 20 tok

question

Name the Burmese capital city.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Naypyidaw

correctknowledge.fr.factbank-v2conf 100% · 108ms · $0.000 · 16 tok

question

Name the element whose symbol is Sb.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Antimony

correctknowledge.fr.factbank-v2conf 100% · 135ms · $0.000 · 18 tok

question

Name the author of "One Hundred Years of Solitude".

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Gabriel García Márquez

correctknowledge.fr.factbank-v2conf 100% · 1.3s · $0.000 · 18 tok

question

Identify the author of "One Hundred Years of Solitude".

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Gabriel García Márquez

correctknowledge.fr.factbank-v2conf 100% · 120ms · $0.000 · 16 tok

question

Name the chemical element with symbol Sb.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Antimony

correctknowledge.fr.factbank-v2conf 100% · 160ms · $0.000 · 15 tok

question

Name the capital of Australia.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Canberra

correctknowledge.fr.factbank-v2conf 100% · 1.5s · $0.000 · 17 tok

question

What is the element whose symbol is W?

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Tungsten

correctknowledge.fr.factbank-v2conf 100% · 673ms · $0.000 · 15 tok

question

Name the Kazakh capital city.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Astana

correctknowledge.fr.factbank-v2conf 100% · 119ms · $0.000 · 20 tok

question

Identify the Burmese capital city.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Naypyidaw

correctknowledge.fr.factbank-v2conf 100% · 99ms · $0.000 · 18 tok

question

What is the writer of the novel "One Hundred Years of Solitude"?

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Gabriel García Márquez

correctknowledge.fr.factbank-v2conf 100% · 120ms · $0.000 · 15 tok

question

Name the Brazilian capital city.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Brasília

correctknowledge.fr.factbank-v2conf 100% · 3.8s · $0.000 · 15 tok

question

Name the capital of Brazil.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Brasília

correctknowledge.fr.factbank-v2conf 100% · 257ms · $0.000 · 18 tok

question

What is the author of "One Hundred Years of Solitude"?

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Gabriel García Márquez

correctknowledge.fr.factbank-v2conf 100% · 140ms · $0.000 · 14 tok

question

Name the element whose symbol is Pb.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Lead

correctknowledge.fr.factbank-v2conf 100% · 181ms · $0.000 · 20 tok

question

Identify the Burmese capital city.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Naypyidaw

correctknowledge.fr.factbank-v2anchorconf 100% · 3.3s · $0.000 · 17 tok

model answer: Tungsten

correctknowledge.fr.factbank-v2conf 100% · 662ms · $0.000 · 15 tok

question

Name the capital of Australia.

Answer with the name only — no explanation.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: Canberra

correctknowledge.fr.factbank-v2anchorconf 100% · 361ms · $0.000 · 14 tok

model answer: Mercury

correctknowledge.fr.factbank-v2anchorconf 100% · 1.0s · $0.000 · 16 tok

model answer: Antimony

correctknowledge.fr.factbank-v2anchorconf 100% · 468ms · $0.000 · 14 tok

model answer: Lead

multilingual 30/30 correct

correctmultilingual.wordnum-v1conf 100% · 114ms · $0.000 · 164 tok

question

A number is written in French: « cinq cent trente-quatre ». Another is written in Spanish: « novecientos cincuenta y cuatro ». Compute (French number) + (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: 1488

correctmultilingual.wordnum-v1conf 100% · 5.2s · $0.000 · 184 tok

question

A number is written in French: « deux cent quatre-vingt-huit ». Another is written in Spanish: « seiscientos cuarenta y ocho ». Compute (French number) − (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: -360

correctmultilingual.numword-v2conf 100% · 140ms · $0.000 · 25 tok

question

Compute 343 + 107, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: cuatrocientos cincuenta

correctmultilingual.wordnum-v1conf 100% · 114ms · $0.000 · 163 tok

question

A number is written in French: « trois cent cinquante-neuf ». Another is written in Spanish: « doscientos setenta y uno ». Compute (French number) + (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: 630

correctmultilingual.numword-v2conf 100% · 117ms · $0.000 · 30 tok

question

Compute 494 + 189, then write the result out in French number words (lowercase). Answer with the French words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: six cent quatre-vingt-trois

correctmultilingual.wordnum-v1conf 100% · 1.1s · $0.000 · 176 tok

question

A number is written in French: « six cent quatre-vingt-un ». Another is written in Spanish: « ciento treinta y cuatro ». Compute (French number) + (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: 815

correctmultilingual.numword-v2conf 100% · 937ms · $0.000 · 24 tok

question

Compute 343 + 411, then write the result out in French number words (lowercase). Answer with the French words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: sept cent cinquante-quatre

correctmultilingual.numword-v2conf 100% · 5.1s · $0.000 · 24 tok

question

Compute 339 + 302, then write the result out in French number words (lowercase). Answer with the French words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: six cent quarante et un

correctmultilingual.wordnum-v1conf 100% · 554ms · $0.000 · 167 tok

question

A number is written in French: « sept cent soixante-douze ». Another is written in Spanish: « ciento veinte ». Compute (French number) − (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: 652

correctmultilingual.numword-v2conf 100% · 259ms · $0.000 · 28 tok

question

Compute 154 + 115, then write the result out in French number words (lowercase). Answer with the French words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: deux cent soixante-neuf

correctmultilingual.wordnum-v1conf 100% · 825ms · $0.000 · 214 tok

question

A number is written in French: « six cent huit ». Another is written in Spanish: « novecientos cincuenta y cinco ». Compute (French number) − (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: -347

correctmultilingual.numword-v2conf 100% · 743ms · $0.000 · 29 tok

question

Compute 257 + 119, then write the result out in French number words (lowercase). Answer with the French words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: trois cent soixante-seize

correctmultilingual.wordnum-v1conf 100% · 1.1s · $0.000 · 152 tok

question

A number is written in French: « cinq cent douze ». Another is written in Spanish: « quinientos setenta y seis ». Compute (French number) − (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: -64

correctmultilingual.wordnum-v1conf 100% · 1.2s · $0.000 · 156 tok

question

A number is written in French: « six cent quarante-neuf ». Another is written in Spanish: « trescientos veintinueve ». Compute (French number) + (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: 978

correctmultilingual.numword-v2conf 100% · 602ms · $0.000 · 26 tok

question

Compute 416 + 408, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: ochocientos veinticuatro

correctmultilingual.numword-v2conf 100% · 240ms · $0.000 · 31 tok

question

Compute 443 + 118, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: quinientos sesenta y uno

correctmultilingual.wordnum-v1conf 100% · 123ms · $0.000 · 164 tok

question

A number is written in French: « six cent soixante-treize ». Another is written in Spanish: « doscientos treinta y tres ». Compute (French number) + (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: 906

correctmultilingual.wordnum-v1conf 100% · 175ms · $0.000 · 184 tok

question

A number is written in French: « cinq cent cinq ». Another is written in Spanish: « trescientos tres ». Compute (French number) + (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: 808

correctmultilingual.numword-v2conf 100% · 348ms · $0.000 · 28 tok

question

Compute 328 + 424, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: setecientos cincuenta y dos

correctmultilingual.wordnum-v1conf 100% · 220ms · $0.000 · 181 tok

question

A number is written in French: « cent trente-trois ». Another is written in Spanish: « seiscientos setenta y siete ». Compute (French number) − (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: -544

correctmultilingual.numword-v2conf 100% · 1.1s · $0.000 · 22 tok

question

Compute 58 + 89, then write the result out in French number words (lowercase). Answer with the French words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: cent quarante-sept

correctmultilingual.wordnum-v1conf 100% · 126ms · $0.000 · 167 tok

question

A number is written in French: « huit cent soixante-six ». Another is written in Spanish: « doscientos setenta y cinco ». Compute (French number) − (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: 591

correctmultilingual.numword-v2conf 100% · 333ms · $0.000 · 29 tok

question

Compute 174 + 399, then write the result out in French number words (lowercase). Answer with the French words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: cinq cent soixante-treize

correctmultilingual.wordnum-v1conf 100% · 1.0s · $0.000 · 179 tok

question

A number is written in French: « quatre cent quatre-vingt-dix ». Another is written in Spanish: « cuatrocientos noventa y ocho ». Compute (French number) − (Spanish number). Answer with digits only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: -8

correctmultilingual.wordnum-v1anchorconf 100% · 1.5s · $0.000 · 192 tok

model answer: 762

correctmultilingual.numword-v2conf 100% · 1.1s · $0.000 · 24 tok

question

Compute 192 + 64, then write the result out in French number words (lowercase). Answer with the French words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: deux cent cinquante-six

correctmultilingual.wordnum-v1anchorconf 100% · 5.9s · $0.000 · 157 tok

model answer: 150

correctmultilingual.numword-v2conf 100% · 1.4s · $0.000 · 24 tok

question

Compute 267 + 351, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only.

End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>

model answer: seiscientos dieciocho

correctmultilingual.numword-v2anchorconf 100% · 2.0s · $0.000 · 20 tok

model answer: seiscientos ocho

correctmultilingual.numword-v2anchorconf 100% · 443ms · $0.000 · 35 tok

model answer: huit cent soixante-dix-neuf

Run history

2026-08-05v0.2.0index_fit710