| ground truth | LLM average | LLM std. dev. | DeepSeek-V4-Pro | Out tokens | Time | Kimi K2.6 | Out tokens | Time | MiniMax M2.7 | Out tokens | Time | Qwen 3.6 Plus | Out tokens | Time | GLM 5.1 | Out tokens | Time | GPT-5.5 | Out tokens | Time | GPT-5 nano | Out tokens | Time | Claude Opus 4.6 | Out tokens | Time | Claude Haiku 4.5 | Out tokens | Time | Gemini 3.1 Pro Preview | Out tokens | Time | Gemini 3.1 Flash-Lite Preview | Out tokens | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ENEL - SPA 2022 | 4.84 | 4.01 | 1.75 | 4.21 | 536 | n/a | 4.2 | 7633 | n/a | 4.2 | 2710 | n/a | 4.35 | 6400 | n/a | 4.2 | 505 | n/a | 3.93 | 267 | n/a | 2.6 | 18 | n/a | 4.4 | 1103 | n/a | 4.21 | 388 | n/a | 4.47 | 16479 | n/a | 3.3787 | 623 | n/a |
| ENI S.P.A. 2022 | 0.95 | 0.92 | 1.94 | 0.51 | 985 | n/a | 0.88 | 9687 | n/a | 0.9 | 8913 | n/a | 0.48 | 6129 | n/a | 0.68 | 659 | n/a | 0.82 | 460 | n/a | 2.6 | 18 | n/a | 0.51 | 1341 | n/a | 1.28 | 537 | n/a | 0.48 | 14072 | n/a | 0.9984 | 605 | n/a |
| A2A ENERGIA S.P.A. 2022 | 3.47 | 3.07 | 2.84 | 2.86 | 388 | n/a | 2.86 | 4549 | n/a | 1.65 | 3063 | n/a | 2.86 | 1534 | n/a | 2.86 | 426 | n/a | 2.86 | 173 | n/a | 5.3 | 18 | n/a | 2.86 | 753 | n/a | 3.93 | 551 | n/a | 2.86 | 5789 | n/a | 2.86 | 404 | n/a |
| "TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022 | 4.35 | 4.17 | 1.78 | 4.2 | 489 | 5.772 | 4.06 | 8246 | 71.24 | 4.06 | 2515 | 13.76 | 4.33 | 3346 | 33.163 | 4.18 | 511 | 22.724 | 4.33 | 337 | 6.636 | 2.8 | 18 | 1.333 | 4.3 | 1222 | 22.473 | 5.27 | 496 | 3.464 | 4.18 | 4177 | 35.189 | 4.18 | 628 | 3.492 |
| REDEIA CORPORACION SA 2022 | 3.21 | 3.55 | 1.99 | 3.13 | 853 | n/a | 4.28 | 6115 | n/a | 3.78 | 3703 | n/a | 3.78 | 3380 | n/a | 3.6 | 493 | n/a | 3.6 | 266 | n/a | 2.01 | 18 | n/a | 3.6 | 1229 | n/a | 4.31 | 529 | n/a | 3.19 | 10130 | n/a | 3.7846 | 549 | n/a |
| EDP, S.A. 2022 | -8.22 | 3.60 | 2.75 | 3.34 | 1591 | n/a | 3.35 | 13156 | n/a | 4.8 | 4366 | n/a | 3.94 | 5321 | n/a | 5.28 | 534 | n/a | 3.63 | 398 | n/a | 2.8 | 18 | n/a | 3.35 | 1173 | n/a | 2.09 | 505 | n/a | 3.73 | 17879 | n/a | 3.34 | 615 | n/a |
| ENGIE 2022 | 3.25 | 2.68 | 1.87 | 1.71 | 1007 | n/a | 2.6 | 11881 | n/a | 2.65 | 8476 | n/a | 2.51 | 5591 | n/a | 3.96 | 589 | n/a | 2.63 | 263 | n/a | 2.1 | 18 | n/a | 2.6 | 1573 | n/a | 3.4 | 720 | n/a | 2.6349 | 11037 | n/a | 2.635 | 495 | n/a |
| IBERDROLA SA 2022 | 3.91 | 3.87 | 2.97 | 4.44 | 773 | n/a | 3.41 | 7291 | n/a | 4.67 | 6837 | n/a | 3.91 | 5269 | n/a | 4.17 | 702 | n/a | 3.73 | 231 | n/a | 1.63 | 18 | n/a | 3.6 | 1937 | n/a | 4.89 | 399 | n/a | 3.25 | 13782 | n/a | 4.89 | 580 | n/a |
| NATURGY ENERGY GROUP SA 2022 | 2.67 | 2.44 | 0.43 | 2.48 | 371 | n/a | 2.49 | 2072 | n/a | 2.4 | 2736 | n/a | 2.49 | 912 | n/a | 2.18 | 383 | n/a | 2.48 | 230 | n/a | 2.6 | 18 | n/a | 2.5 | 909 | n/a | 2.48 | 514 | n/a | 2.55 | 9429 | n/a | 2.18 | 556 | n/a |
| TenneT Holding B.V. 2022 | 83.85 | 45.41 | 64.49 | 47.11 | 486 | n/a | 51.2 | 12279 | n/a | 56.94 | 6630 | n/a | 56.9 | 3554 | n/a | 56.93 | 457 | n/a | 54.1 | 320 | n/a | -1.0 | 18 | n/a | 58.2 | 1422 | n/a | 11.2 | 580 | n/a | 56.68 | 28128 | n/a | 51.24 | 502 | n/a |
| VEOLIA ENVIRONNEMENT 2022 | 3.92 | 3.69 | 1.93 | 4.64 | 579 | n/a | 3.3 | 5527 | n/a | 3.99 | 8912 | n/a | 4.22 | 4205 | n/a | 3.95 | 752 | n/a | 3.63 | 346 | n/a | 2.6 | 18 | n/a | 4.3 | 1128 | n/a | 3.16 | 518 | n/a | 3.7 | 15071 | n/a | 3.0812 | 674 | n/a |
| ØRSTED A/S 2022 | 2.5 | 1.87 | 1.16 | 2.34 | 729 | n/a | 1.73 | 1979 | n/a | 1.73 | 2341 | n/a | 1.73 | 3666 | n/a | 1.47 | 1065 | n/a | 2.04 | 255 | n/a | 2.6 | 18 | n/a | 2.0 | 1068 | n/a | 1.32 | 635 | n/a | 1.86 | 10016 | n/a | 1.73 | 558 | n/a |
| ELECTRICITE DE FRANCE 2022 | 4.06 | -22.81 | 62.92 | -78.17 | 756 | n/a | -26.46 | 16510 | n/a | -14.83 | 4422 | n/a | -26.46 | 2881 | n/a | -21.9 | 646 | n/a | -26.5 | 255 | n/a | -1.27 | 18 | n/a | -12.9 | 1360 | n/a | -12.81 | 653 | n/a | -14.83 | 18360 | n/a | -14.83 | 557 | n/a |
| ERG S.P.A. 2022 | not available | 4.02 | 2.02 | 3.97 | 528 | n/a | 3.82 | 10935 | n/a | 5.79 | 7738 | n/a | 3.97 | 1995 | n/a | 4.15 | 568 | n/a | 3.82 | 246 | n/a | 3.2 | 18 | n/a | 3.8 | 1044 | n/a | 3.67 | 534 | n/a | 4.01 | 8640 | n/a | 3.97 | 549 | n/a |
| Acciona SA 2022 | not available | 2.72 | 0.48 | 2.66 | 490 | n/a | 2.8 | 13728 | n/a | 2.68 | 9573 | n/a | 2.87 | 7238 | n/a | 2.66 | 894 | n/a | 2.66 | 335 | n/a | 2.6 | 18 | n/a | 2.7 | 1468 | n/a | 3.09 | 487 | n/a | 2.54 | 10197 | n/a | 2.6565 | 672 | n/a |
| Fortum Oyj 2022 | 1.84 | 2.46 | 3.77 | 2.1 | 622 | n/a | 1.59 | 12756 | n/a | 1.59 | 4323 | n/a | 1.59 | 3765 | n/a | 2.1 | 518 | n/a | 1.59 | 234 | n/a | 4.2 | 18 | n/a | 2.1 | 964 | n/a | 3.95 | 474 | n/a | 1.63 | 10628 | n/a | 4.67 | 583 | n/a |
| VERBUND AG 2022 | 1.6 | 1.17 | 1.13 | 1.12 | 383 | n/a | 1.12 | 1874 | n/a | 0.94 | 2360 | n/a | 1.12 | 812 | n/a | 1.12 | 405 | n/a | 1.12 | 215 | n/a | 2.2 | 18 | n/a | 1.12 | 839 | n/a | 0.78 | 347 | n/a | 1.12 | 5768 | n/a | 1.12147 | 677 | n/a |
| ITALGAS S.P.A. 2022 | 5.58 | 5.27 | 1.25 | 5.44 | 490 | n/a | 5.43 | 3862 | n/a | 5.43 | 6820 | n/a | 5.44 | 1358 | n/a | 5.44 | 612 | n/a | 5.44 | 302 | n/a | 4.2 | 18 | n/a | 5.43 | 689 | n/a | 4.85 | 528 | n/a | 5.44 | 8753 | n/a | 5.44 | 612 | n/a |
| ACEA S.P.A. 2022 | not available | 3.78 | 1.35 | 3.73 | 379 | n/a | 4.12 | 9902 | n/a | 3.66 | 4465 | n/a | 3.89 | 7203 | n/a | 3.89 | 551 | n/a | 3.66 | 278 | n/a | 2.6 | 18 | n/a | 4.12 | 1200 | n/a | 4.12 | 534 | n/a | 3.8934 | 23632 | n/a | 3.8934 | 543 | n/a |
| REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022 | 4.37 | 4.33 | 2.23 | 4.01 | 521 | n/a | 4.03 | 926 | n/a | 4.03 | 4758 | n/a | 4.03 | 3909 | n/a | 4.03 | 468 | n/a | 4.03 | 271 | n/a | 6.2 | 18 | n/a | 4.03 | 676 | n/a | 5.14 | 443 | n/a | 4.03 | 6911 | n/a | 4.03 | 536 | n/a |
| HERA S.P.A. 2022 | 4.23 | 3.81 | 2.39 | 3.73 | 438 | n/a | 4.42 | 8080 | n/a | 3.9 | 11494 | n/a | 3.73 | 3259 | n/a | 4.34 | 553 | n/a | 3.73 | 268 | n/a | 1.76 | 18 | n/a | 4.42 | 1029 | n/a | 3.73 | 530 | n/a | 4.45 | 10102 | n/a | 3.66 | 591 | n/a |
| VINCI 2022 | 1.89 | 1.98 | 0.97 | 1.93 | 592 | n/a | 1.86 | 5017 | n/a | 1.73 | 5137 | n/a | 1.79 | 4263 | n/a | 1.72 | 512 | n/a | 1.87 | 305 | n/a | 2.8 | 18 | n/a | 1.9 | 915 | n/a | 2.15 | 385 | n/a | 1.86 | 8585 | n/a | 2.135 | 726 | n/a |
| BOUYGUES 2022 | 3.26 | 1.70 | 2.17 | 1.58 | 361 | n/a | 1.64 | 9299 | n/a | 1.52 | 11963 | n/a | 1.6 | 4220 | n/a | 1.58 | 441 | n/a | 1.6 | 233 | n/a | 3.1 | 18 | n/a | 2.21 | 1004 | n/a | 1.61 | 707 | n/a | 2.1 | 12535 | n/a | 0.21 | 465 | n/a |
| FERROVIAL SA 2022 | not available | 8.01 | 4.90 | 7.21 | 465 | n/a | 9.2 | 8502 | n/a | 9.4 | 5743 | n/a | 7.82 | 7698 | n/a | 9.11 | 702 | n/a | 9.3 | 318 | n/a | 4.2 | 18 | n/a | 7.6 | 1167 | n/a | 9.03 | 716 | n/a | 6.88 | 16442 | n/a | 8.4 | 653 | n/a |
| Failures | Mean Absolute Error (ratio) | Mean Bias Error (ratio) | Root Mean Squared Error (ratio) | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 1.253 | -1.064 | 4.546 | nan | nan | nan | 0.75 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.603 | -0.424 | 1.723 | nan | nan | nan | 1.15 | 0.05 | 0.02 |
| MiniMax M2.7 | 0 | 0.483 | -0.286 | 1.124 | nan | nan | nan | 0.29 | 0.01 | 0.00 |
| Qwen 3.6 Plus | 0 | 0.609 | -0.435 | 1.727 | nan | nan | nan | 0.56 | 0.02 | 0.01 |
| GLM 5.1 | 0 | 0.559 | -0.336 | 1.491 | nan | nan | nan | 0.64 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.585 | -0.428 | 1.724 | nan | nan | nan | 2.21 | 0.09 | 0.03 |
| GPT-5 nano | 0 | 0.595 | 0.025 | 0.758 | nan | nan | nan | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.416 | -0.234 | 1.003 | nan | nan | nan | 2.95 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 0.560 | -0.156 | 1.052 | nan | nan | nan | 0.52 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 0.444 | -0.293 | 1.107 | nan | nan | nan | 4.61 | 0.19 | 0.07 |
| Gemini 3.1 Flash-Lite Preview | 0 | 0.574 | -0.219 | 1.174 | nan | nan | nan | 0.15 | 0.01 | 0.00 |
| ground truth | LLM average | LLM std. dev. | DeepSeek-V4-Pro | Out tokens | Time | Kimi K2.6 | Out tokens | Time | MiniMax M2.7 | Out tokens | Time | Qwen 3.6 Plus | Out tokens | Time | GLM 5.1 | Out tokens | Time | GPT-5.5 | Out tokens | Time | GPT-5 nano | Out tokens | Time | Claude Opus 4.6 | Out tokens | Time | Claude Haiku 4.5 | Out tokens | Time | Gemini 3.1 Pro Preview | Out tokens | Time | Gemini 3.1 Flash-Lite Preview | Out tokens | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ENEL - SPA 2022 | 0.1567 | 0.15 | 0.19 | 0.238 | 423 | n/a | 0.161 | 21536 | n/a | 0.1 | 26045 | n/a | 0.17 | 7249 | n/a | 0.2378 | 556 | n/a | 0.0601 | 321 | n/a | 0.174 | 18 | n/a | 0.16 | 1009 | n/a | 0.11 | 512 | n/a | 0.13 | 16138 | n/a | 0.07483 | 574 | n/a |
| ENI S.P.A. 2022 | 0.6593 | 1.17 | 1.84 | 0.745 | 426 | n/a | 1.1 | 16319 | n/a | 2.49 | 4423 | n/a | 0.97 | 7328 | n/a | 1.33 | 543 | n/a | 1.62 | 387 | n/a | 0.27 | 18 | n/a | 1.04 | 1068 | n/a | 0.79 | 649 | n/a | 1.57 | 10989 | n/a | 0.9653 | 595 | n/a |
| A2A ENERGIA S.P.A. 2022 | 0.2317 | 0.28 | 0.20 | 0.3436 | 374 | n/a | 0.3 | 17079 | n/a | 0.3 | 6173 | n/a | 0.28 | 3899 | n/a | 0.283 | 912 | n/a | 0.295 | 439 | n/a | 0.1 | 18 | n/a | 0.25 | 906 | n/a | 0.31 | 592 | n/a | 0.272 | 12268 | n/a | 0.2927 | 408 | n/a |
| "TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022 | 0.1724 | 0.22 | 0.12 | 0.192 | 436 | 8.591 | 0.241 | 8584 | 71.945 | 0.28 | 7064 | 36.91 | 0.178 | 5379 | 53.136 | 0.239 | 523 | 22.244 | 25.4% | 354 | 6.483 | 0.18 | 18 | 1.107 | 0.19 | 841 | 16.002 | 0.27 | 590 | 5.372 | 0.18 | 5671 | 47.758 | 0.232702 | 631 | 2.461 |
| REDEIA CORPORACION SA 2022 | 0.2063 | 0.20 | 0.15 | 0.15 | 4117 | n/a | 0.2 | 9259 | n/a | 0.2428 | 7301 | n/a | 0.187 | 6678 | n/a | 0.2 | 665 | n/a | 18.3% | 259 | n/a | 0.1 | 18 | n/a | 0.263 | 1123 | n/a | 0.25 | 497 | n/a | 0.2127 | 19334 | n/a | 0.19 | 465 | n/a |
| EDP, S.A. 2022 | -0.1579 | 0.21 | 0.16 | 0.176 | 1037 | n/a | 0.278 | 7226 | n/a | 0.28 | 15120 | n/a | 0.13 | 3143 | n/a | 0.132 | 750 | n/a | 17.64% | 395 | n/a | 0.221 | 18 | n/a | 0.21 | 1015 | n/a | 0.21 | 459 | n/a | 0.21 | 14637 | n/a | 0.2498 | 590 | n/a |
| ENGIE 2022 | 0.2483 | 0.40 | 0.35 | 0.4962 | 686 | n/a | 0.32 | 24921 | n/a | 0.34 | 4230 | n/a | 0.5 | 6624 | n/a | 0.2417 | 945 | n/a | 49.6% | 214 | n/a | 0.22 | 18 | n/a | 0.44 | 926 | n/a | 0.33 | 460 | n/a | 0.4962 | 3947 | n/a | 0.49618 | 670 | n/a |
| IBERDROLA SA 2022 | 0.1965 | 0.19 | 0.12 | 0.155 | 929 | n/a | 0.21 | 13183 | n/a | 0.186 | 5377 | n/a | 0.19 | 5610 | n/a | 0.14 | 442 | n/a | 25.7% | 450 | n/a | 0.18 | 18 | n/a | 0.22 | 1278 | n/a | 0.16 | 499 | n/a | 0.254 | 8799 | n/a | 0.1614 | 602 | n/a |
| NATURGY ENERGY GROUP SA 2022 | 0.2847 | 0.32 | 0.33 | 0.3145 | 415 | n/a | 0.38 | 16216 | n/a | 0.35 | 6031 | n/a | 0.272 | 4050 | n/a | 0.44 | 629 | n/a | 36.7% | 299 | n/a | 0.21 | 18 | n/a | 0.42 | 1090 | n/a | 0.078 | 621 | n/a | 0.3665 | 10995 | n/a | 0.3444 | 575 | n/a |
| TenneT Holding B.V. 2022 | -0.0105 | 0.14 | 0.46 | 0.02977 | 444 | n/a | 0.096 | 20853 | n/a | 0.09 | 8476 | n/a | 0.025 | 6110 | n/a | 0.128 | 698 | n/a | 4.3% | 278 | n/a | 0.22 | 18 | n/a | 0.14 | 1237 | n/a | 0.54 | 538 | n/a | 0.106 | 26918 | n/a | 0.0828 | 680 | n/a |
| VEOLIA ENVIRONNEMENT 2022 | 0.1972 | 0.23 | 0.15 | 0.1767 | 876 | n/a | 0.22 | 8909 | n/a | 0.22 | 7680 | n/a | 0.214 | 4577 | n/a | 0.16 | 771 | n/a | 18.4% | 451 | n/a | 0.22 | 18 | n/a | 0.24 | 1167 | n/a | 0.26 | 628 | n/a | 0.279 | 13350 | n/a | 0.3298 | 688 | n/a |
| ØRSTED A/S 2022 | 0.3124 | 0.32 | 0.56 | 0.242 | 359 | n/a | 0.61 | 13112 | n/a | 0.4 | 11051 | n/a | 0.49 | 2857 | n/a | 0.5791 | 512 | n/a | 15.3% | 364 | n/a | 0.18 | 18 | n/a | 0.18 | 1125 | n/a | 0.11 | 551 | n/a | 0.28 | 21566 | n/a | 0.2531 | 540 | n/a |
| ELECTRICITE DE FRANCE 2022 | 0.1884 | -0.09 | 0.30 | -0.056 | 933 | n/a | 0.0066 | 22299 | n/a | -0.07 | 19990 | n/a | -0.039 | 4745 | n/a | -0.03 | 746 | n/a | -20.2% | 339 | n/a | -0.25 | 18 | n/a | 0.014 | 1143 | n/a | -0.04 | 446 | n/a | -0.054155 | 23789 | n/a | -0.224155 | 772 | n/a |
| ERG S.P.A. 2022 | not available | 0.26 | 0.19 | 0.187 | 8050 | n/a | 0.311 | 10993 | n/a | 0.23 | 22942 | n/a | 0.2 | 7594 | n/a | 0.31 | 1775 | n/a | 0.368 | 565 | n/a | 0.24 | 18 | n/a | 0.252 | 1003 | n/a | 0.2515 | 485 | n/a | 0.323 | 10662 | n/a | 0.19092 | 656 | n/a |
| Acciona SA 2022 | not available | 0.27 | 0.16 | 0.323 | 554 | n/a | 0.326 | 11291 | n/a | 0.3229 | 13895 | n/a | 0.277 | 6551 | n/a | 0.2446 | 944 | n/a | 0.272 | 272 | n/a | 0.15 | 18 | n/a | 0.27 | 985 | n/a | 0.26 | 597 | n/a | 0.288 | 5811 | n/a | 0.2717 | 768 | n/a |
| Fortum Oyj 2022 | 0.4556 | 0.43 | 0.60 | 0.537 | 368 | n/a | 0.6 | 12427 | n/a | 0.241 | 9997 | n/a | 0.41 | 2705 | n/a | 0.59 | 603 | n/a | 60.1% | 238 | n/a | 0.18 | 18 | n/a | 0.6 | 845 | n/a | 0.22 | 1016 | n/a | 0.6 | 6730 | n/a | 0.178 | 762 | n/a |
| VERBUND AG 2022 | 0.5174 | 0.75 | 1.42 | 0.71 | 376 | n/a | 0.71 | 19364 | n/a | 0.47 | 20456 | n/a | 0.7 | 3315 | n/a | 2.02 | 593 | n/a | 60.5% | 448 | n/a | 0.25 | 18 | n/a | 0.71 | 820 | n/a | 0.57 | 548 | n/a | 0.7098 | 5654 | n/a | 0.7847 | 643 | n/a |
| ITALGAS S.P.A. 2022 | 0.1456 | 0.15 | 0.07 | 0.1531 | 487 | n/a | 0.147 | 15140 | n/a | 0.09 | 2713 | n/a | 0.15 | 3020 | n/a | 0.15 | 677 | n/a | 0.146 | 408 | n/a | 0.18 | 18 | n/a | 0.146 | 695 | n/a | 0.18 | 506 | n/a | 0.1452 | 12136 | n/a | 0.15416 | 774 | n/a |
| ACEA S.P.A. 2022 | not available | 0.22 | 0.11 | 0.2314 | 475 | n/a | 0.222 | 3732 | n/a | 0.2 | 8544 | n/a | 0.21 | 6024 | n/a | 0.2215 | 613 | n/a | 0.222 | 196 | n/a | 0.29 | 18 | n/a | 0.19 | 924 | n/a | 0.2 | 436 | n/a | 0.2393 | 21475 | n/a | 0.15197 | 523 | n/a |
| REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022 | 0.1686 | 0.19 | 0.14 | 0.178 | 422 | n/a | 0.18 | 4203 | n/a | 0.18 | 5870 | n/a | 0.19 | 5279 | n/a | 0.1835 | 990 | n/a | 0.312 | 268 | n/a | 0.15 | 18 | n/a | 0.183 | 880 | n/a | 0.14 | 419 | n/a | 0.1834 | 3898 | n/a | 0.17623 | 591 | n/a |
| HERA S.P.A. 2022 | 0.1733 | 0.23 | 0.17 | 0.219 | 439 | n/a | 0.269 | 11916 | n/a | 0.27 | 2382 | n/a | 0.27 | 3447 | n/a | 0.178 | 457 | n/a | 0.207 | 322 | n/a | 0.18 | 18 | n/a | 0.175 | 1031 | n/a | 0.35 | 562 | n/a | 0.2267 | 10590 | n/a | 0.22695 | 641 | n/a |
| VINCI 2022 | 0.4221 | 0.45 | 0.30 | 0.4548 | 621 | n/a | 0.59 | 14822 | n/a | 0.48 | 6232 | n/a | 0.43 | 4612 | n/a | 0.43 | 701 | n/a | 54.4% | 297 | n/a | 0.22 | 18 | n/a | 0.48 | 1048 | n/a | 0.36 | 555 | n/a | 0.46 | 17380 | n/a | 0.4692 | 626 | n/a |
| BOUYGUES 2022 | 0.2446 | 0.79 | 3.39 | 0.551 | 235 | n/a | 0.54 | 14576 | n/a | 0.4 | 9666 | n/a | 0.55 | 2116 | n/a | 0.53 | 677 | n/a | 55.1% | 219 | n/a | 0.23 | 18 | n/a | 0.447 | 845 | n/a | 0.4 | 618 | n/a | 0.45 | 8097 | n/a | 4.005 | 593 | n/a |
| FERROVIAL SA 2022 | not available | 0.14 | 0.16 | 0.081 | 1816 | n/a | 0.15 | 9668 | n/a | 0.2 | 11651 | n/a | 0.1 | 5315 | n/a | 0.09 | 1096 | n/a | 0.137 | 383 | n/a | 0.25 | 18 | n/a | 0.14 | 949 | n/a | 0.15 | 423 | n/a | 0.092 | 13723 | n/a | 0.1409 | 627 | n/a |
| Failures | Mean Absolute Error (ratio) | Mean Bias Error (ratio) | Root Mean Squared Error (ratio) | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.633 | 0.422 | 1.105 | nan | nan | nan | 0.79 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.999 | 0.899 | 2.402 | nan | nan | nan | 1.70 | 0.07 | 0.02 |
| MiniMax M2.7 | 0 | 1.067 | 0.794 | 2.355 | nan | nan | nan | 0.41 | 0.02 | 0.01 |
| Qwen 3.6 Plus | 0 | 0.574 | 0.426 | 0.997 | nan | nan | nan | 0.62 | 0.03 | 0.01 |
| GLM 5.1 | 0 | 1.240 | 1.070 | 3.092 | nan | nan | nan | 0.66 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.873 | 0.535 | 1.441 | nan | nan | nan | 2.25 | 0.09 | 0.03 |
| GPT-5 nano | 0 | 1.578 | 0.912 | 4.976 | nan | nan | nan | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 1.120 | 0.985 | 3.272 | nan | nan | nan | 2.86 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 3.131 | 2.740 | 11.746 | nan | nan | nan | 0.52 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 1.031 | 0.875 | 2.594 | nan | nan | nan | 4.70 | 0.20 | 0.08 |
| Gemini 3.1 Flash-Lite Preview | 0 | 1.730 | 1.354 | 4.061 | nan | nan | nan | 0.15 | 0.01 | 0.00 |
| Failures | Mean Absolute Error | Mean Bias Error | Root Mean Squared Error | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.43 | 0.17 | 0.88 | nan | nan | nan | 0.73 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.57 | -0.22 | 1.00 | nan | nan | nan | 0.52 | 0.02 | 0.01 |
| MiniMax M2.7 | 0 | 0.26 | 0.09 | 0.72 | nan | nan | nan | 0.23 | 0.01 | 0.00 |
| Qwen 3.6 Plus | 0 | 0.17 | -0.09 | 0.51 | nan | nan | nan | 0.37 | 0.02 | 0.00 |
| GLM 5.1 | 0 | 0.52 | -0.09 | 0.98 | nan | nan | nan | 0.64 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.39 | 0.22 | 0.86 | nan | nan | nan | 2.27 | 0.09 | 0.03 |
| GPT-5 nano | 1 | 0.59 | 0.05 | 0.88 | nan | nan | nan | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.35 | -0.26 | 0.78 | nan | nan | nan | 2.77 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 0.35 | 0.17 | 0.78 | nan | nan | nan | 0.55 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 0.52 | 0.26 | 0.98 | nan | nan | nan | 1.97 | 0.08 | 0.03 |
| Gemini 3.1 Flash-Lite Preview | 0 | 0.35 | -0.26 | 0.78 | nan | nan | nan | 0.15 | 0.01 | 0.00 |
| Failures | Mean Absolute Error | Mean Bias Error | Root Mean Squared Error | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.92 | 0.33 | 1.15 | nan | nan | nan | 0.74 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.79 | 0.29 | 1.06 | nan | nan | nan | 0.47 | 0.02 | 0.01 |
| MiniMax M2.7 | 0 | 0.88 | 0.38 | 1.14 | nan | nan | nan | 0.22 | 0.01 | 0.00 |
| Qwen 3.6 Plus | 0 | 0.96 | 0.88 | 1.24 | nan | nan | nan | 0.34 | 0.01 | 0.00 |
| GLM 5.1 | 0 | 0.92 | 0.92 | 1.22 | nan | nan | nan | 0.63 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.83 | 0.25 | 1.04 | nan | nan | nan | 2.31 | 0.10 | 0.03 |
| GPT-5 nano | 2 | 0.77 | -0.05 | 0.98 | nan | nan | nan | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.92 | 0.92 | 1.22 | nan | nan | nan | 2.78 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 0.88 | 0.29 | 1.17 | nan | nan | nan | 0.57 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 1.04 | 0.88 | 1.34 | nan | nan | nan | 1.49 | 0.06 | 0.01 |
| Gemini 3.1 Flash-Lite Preview | 0 | 0.96 | 0.88 | 1.24 | nan | nan | nan | 0.15 | 0.01 | 0.00 |
| ground truth | LLM average | LLM std. dev. | DeepSeek-V4-Pro | Out tokens | Time | Kimi K2.6 | Out tokens | Time | MiniMax M2.7 | Out tokens | Time | Qwen 3.6 Plus | Out tokens | Time | GLM 5.1 | Out tokens | Time | GPT-5.5 | Out tokens | Time | GPT-5 nano | Out tokens | Time | Claude Opus 4.6 | Out tokens | Time | Claude Haiku 4.5 | Out tokens | Time | Gemini 3.1 Pro Preview | Out tokens | Time | Gemini 3.1 Flash-Lite Preview | Out tokens | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ENEL - SPA 2022 | 50% | 0.23 | 0.24 | 25% | 512 | 7.718 | 25% | 1004 | 10.332 | 25% | 12302 | 74.518 | 25% | 1814 | 20.656 | 25% | 599 | 28.511 | 25% | 366 | 8.157 | 25% | 513 | 6.347 | 25% | 784 | 21.711 | 25% | 975 | 10.911 | 0% | 2561 | 27.254 | 25% | 473 | 2.742 |
| ENI S.P.A. 2022 | 25% | 0.20 | 0.32 | 25% | 424 | 8.63 | 25% | 1134 | 13.146 | 25% | 7029 | 49.736 | 25% | 2607 | 28.606 | 25% | 638 | 27.221 | 25% | 410 | 10.028 | 0% | 378 | 4.346 | 25% | 770 | 20.064 | 25% | 962 | 10.818 | 0% | 3098 | 32.257 | 25% | 451 | 2.927 |
| A2A ENERGIA S.P.A. 2022 | 25% | 0.20 | 0.32 | 25% | 383 | 5.013 | 25% | 980 | 9.517 | 25% | 5180 | 26.046 | 25% | 1080 | 10.257 | 25% | 555 | 23.998 | 25% | 454 | 8.658 | 0% | 188 | 3.667 | 25% | 730 | 19.847 | 25% | 1044 | 10.756 | 0% | 2040 | 22.27 | 25% | 509 | 2.799 |
| "TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022 | 25% | 0.23 | 0.24 | 25% | 688 | 8.516 | 25% | 917 | 10.698 | 25% | 7658 | 41.438 | 25% | 1956 | 19.836 | 25% | 590 | 22.297 | 25% | 349 | 7.624 | 25% | 350 | 5.026 | 25% | 729 | 20.559 | 25% | 1117 | 13.323 | 0% | 3629 | 39.348 | 25% | 354 | 2.373 |
| REDEIA CORPORACION SA 2022 | 50% | 0.18 | 0.37 | 25% | 303 | 8.461 | 25% | 1199 | 14.462 | 0% | 6028 | 37.927 | 25% | 1213 | 13.371 | 25% | 640 | 26.281 | 25% | 443 | 8.58 | 25% | 388 | 4.146 | 25% | 687 | 18.58 | 25% | 974 | 10.732 | 0% | 1596 | 19.459 | 0% | 507 | 3.14 |
| EDP, S.A. 2022 | 50% | 0.18 | 0.37 | 25% | 431 | 6.312 | 25% | 1299 | 16.882 | 25% | 1918 | 17.247 | 25% | 1303 | 14.106 | 0% | 540 | 21.751 | 25% | 342 | 7.962 | 0% | 97 | 2.19 | 25% | 723 | 21.258 | 25% | 1217 | 14.1 | 0% | 1494 | 15.999 | 25% | 354 | 2.303 |
| ENGIE 2022 | 50% | 0.25 | 0.35 | 25% | 439 | 5.698 | 25% | 956 | 11.571 | 25% | 1976 | 12.829 | 25% | 1806 | 17.89 | 25% | 569 | 22.94 | 25% | 369 | 7.795 | 50% | 277 | 3.915 | 25% | 680 | 19.154 | 25% | 1020 | 11.832 | 0% | 3713 | 39.391 | 25% | 522 | 2.936 |
| IBERDROLA SA 2022 | 75% | 0.23 | 0.43 | 25% | 486 | 6.847 | 25% | 1032 | 16.05 | 25% | 4826 | 28.872 | 0% | 1971 | 19.322 | 25% | 726 | 43.443 | 25% | 399 | 8.596 | 25% | 292 | 3.825 | 25% | 719 | 19.926 | 50% | 1097 | 11.896 | 0% | 2541 | 26.652 | 25% | 420 | 2.456 |
| NATURGY ENERGY GROUP SA 2022 | 25% | 0.20 | 0.32 | 25% | 481 | 7.555 | 25% | 1067 | 14.17 | 25% | 2719 | 17.881 | 25% | 1289 | 13.513 | 25% | 509 | 17.814 | 25% | 456 | 9.951 | 0% | 267 | 3.299 | 25% | 718 | 18.146 | 25% | 1011 | 11.964 | 0% | 2706 | 26.221 | 25% | 455 | 2.395 |
| TenneT Holding B.V. 2022 | 50% | 0.25 | 0.35 | 25% | 571 | 8.896 | 50% | 902 | 8.909 | 25% | 3331 | 18.915 | 25% | 1687 | 18.728 | 25% | 647 | 21.759 | 25% | 373 | 9.362 | 25% | 176 | 3.019 | 25% | 761 | 21.168 | 25% | 1469 | 15.5 | 0% | 9981 | 89.001 | 25% | 506 | 2.725 |
| VEOLIA ENVIRONNEMENT 2022 | 50% | 0.25 | 0.35 | 25% | 1013 | 12.575 | 25% | 1053 | 16.658 | 25% | 4094 | 30.555 | 25% | 1135 | 12.741 | 25% | 654 | 23.096 | 25% | 358 | 8.883 | 50% | 558 | 6.414 | 25% | 695 | 21.543 | 25% | 1468 | 16.112 | 0% | 1081 | 12.11 | 25% | 465 | 2.852 |
| ØRSTED A/S 2022 | 50% | 0.23 | 0.24 | 25% | 404 | 6.202 | 25% | 1052 | 14.36 | 25% | 3767 | 24.179 | 25% | 1766 | 17.554 | 25% | 495 | 19.271 | 25% | 339 | 7.258 | 25% | 188 | 3.787 | 25% | 838 | 21.606 | 25% | 896 | 9.278 | 0% | 3391 | 36.36 | 25% | 324 | 3.742 |
| ELECTRICITE DE FRANCE 2022 | 75% | 0.28 | 0.43 | 25% | 587 | 10.377 | 25% | 903 | 12.912 | 25% | 5285 | 29.952 | 25% | 1331 | 15.792 | 50% | 583 | 31.329 | 25% | 260 | 5.909 | FAIL | 153 | 2.846 | 25% | 899 | 24.668 | 25% | 1033 | 12.33 | 0% | 1575 | 16.624 | 50% | 385 | 2.406 |
| ERG S.P.A. 2022 | 0% | 0.27 | 0.43 | 25% | 422 | 6.574 | 50% | 913 | 9.15 | 25% | 3642 | 28.47 | 25% | 1245 | 14.608 | 25% | 652 | 37.162 | 25% | 422 | 8.836 | 50% | 76 | 1.772 | 25% | 781 | 21.972 | 25% | 866 | 9.68 | 0% | 2227 | 25.053 | 25% | 432 | 2.495 |
| Acciona SA 2022 | 0% | 0.25 | 0.50 | 25% | 365 | 6.316 | 50% | 963 | 11.719 | 25% | 3075 | 26.865 | 25% | 1453 | 14.957 | 25% | 515 | 27.694 | 25% | 380 | 9.116 | 0% | 146 | 2.182 | 25% | 803 | 21.814 | 25% | 967 | 10.425 | 0% | 2734 | 29.328 | 50% | 423 | 3.537 |
| Fortum Oyj 2022 | 0% | 0.23 | 0.43 | 0% | 997 | 15.11 | 25% | 857 | 8.429 | 25% | 2228 | 11.626 | 25% | 1464 | 16.301 | 25% | 549 | 36.541 | 25% | 389 | 9.411 | 50% | 231 | 3.714 | 25% | 740 | 20.401 | 25% | 1081 | 11.965 | 0% | 1811 | 18.75 | 25% | 330 | 2.204 |
| VERBUND AG 2022 | 0% | 0.18 | 0.37 | 25% | 626 | 9.025 | 25% | 779 | 10.252 | 25% | 5531 | 32.531 | 0% | 1424 | 14.868 | 25% | 557 | 18.478 | 25% | 418 | 8.58 | 25% | 245 | 3.761 | 25% | 652 | 18.547 | 25% | 1129 | 12.638 | 0% | 1611 | 17.181 | 0% | 469 | 2.559 |
| ITALGAS S.P.A. 2022 | 0% | 0.25 | 0.35 | 25% | 496 | 8.911 | 50% | 919 | 11.956 | 25% | 7096 | 55.344 | 25% | 1807 | 20.045 | 25% | 835 | 29.132 | 25% | 380 | 8.342 | FAIL | 198 | 2.911 | 25% | 711 | 18.315 | 25% | 930 | 11.106 | 0% | 2176 | 20.447 | 25% | 446 | 2.531 |
| ACEA S.P.A. 2022 | 25% | 0.23 | 0.43 | 25% | 455 | 7.627 | 50% | 936 | 10.071 | 25% | 2929 | 19.229 | 25% | 1235 | 13.525 | 25% | 543 | 16.227 | 25% | 394 | 9.319 | 0% | 66 | 1.672 | 25% | 749 | 21.853 | 25% | 1085 | 11.966 | 0% | 1691 | 18.806 | 25% | 468 | 2.754 |
| REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022 | 25% | 0.23 | 0.24 | 25% | 393 | 7.19 | 25% | 848 | 10.108 | 25% | 4420 | 26.399 | 25% | 2057 | 19.811 | 25% | 558 | 17.726 | 25% | 364 | 8.184 | FAIL | 198 | 2.726 | 25% | 601 | 18.181 | 25% | 1161 | 12.514 | 0% | 1554 | 17.845 | 25% | 346 | 2.388 |
| HERA S.P.A. 2022 | 25% | 0.20 | 0.32 | 25% | 818 | 13.38 | 25% | 1007 | 13.524 | 25% | 7027 | 39.046 | 25% | 1292 | 13.895 | 25% | 593 | 17.625 | 25% | 351 | 7.316 | 0% | 201 | 2.615 | 25% | 693 | 20.741 | 25% | 1129 | 12.013 | 0% | 2660 | 26.02 | 25% | 460 | 2.6 |
| VINCI 2022 | 0% | 0.23 | 0.43 | 25% | 484 | 9.079 | 25% | 988 | 13.835 | 25% | 5546 | 34.24 | 25% | 1176 | 13.066 | 25% | 603 | 24.642 | 25% | 417 | 9.698 | 50% | 16 | 1.218 | 25% | 725 | 21.597 | 0% | 914 | 10.932 | 0% | 1905 | 20.896 | 25% | 452 | 3.449 |
| BOUYGUES 2022 | 0% | 0.18 | 0.37 | 25% | 816 | 12.728 | 25% | 1044 | 12.138 | 25% | 6081 | 36.692 | 0% | 1207 | 13.932 | 25% | 512 | 18.513 | 25% | 288 | 6.593 | 0% | 117 | 2.166 | 25% | 703 | 20.629 | 25% | 995 | 10.666 | 0% | 2132 | 27.249 | 25% | 511 | 2.641 |
| FERROVIAL SA 2022 | 0% | 0.23 | 0.24 | 25% | 439 | 7.275 | 25% | 884 | 10.188 | 25% | 3611 | 27.563 | 25% | 1094 | 12.885 | 25% | 524 | 30.61 | 25% | 334 | 7.952 | FAIL | 164 | 2.435 | 25% | 636 | 18.757 | 25% | 1146 | 13.55 | 0% | 1668 | 18.762 | 25% | 451 | 3.356 |
| Failures | Mean Absolute Error | Mean Bias Error | Root Mean Squared Error | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.188 | -0.042 | 0.239 | 206.02 | 8.58 | 2.58 | 0.77 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.229 | 0.021 | 0.289 | 291.04 | 12.13 | 2.47 | 0.49 | 0.02 | 0.01 |
| MiniMax M2.7 | 0 | 0.208 | -0.042 | 0.260 | 748.10 | 31.17 | 14.12 | 0.26 | 0.01 | 0.00 |
| Qwen 3.6 Plus | 0 | 0.188 | -0.062 | 0.260 | 390.26 | 16.26 | 3.88 | 0.39 | 0.02 | 0.00 |
| GLM 5.1 | 0 | 0.198 | -0.031 | 0.245 | 604.06 | 25.17 | 7.00 | 0.66 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.198 | -0.031 | 0.245 | 202.11 | 8.42 | 1.02 | 2.34 | 0.10 | 0.03 |
| GPT-5 nano | 4 | 0.250 | -0.075 | 0.306 | 80.00 | 3.33 | 1.32 | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.198 | -0.031 | 0.245 | 491.04 | 20.46 | 1.59 | 2.78 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 0.177 | -0.031 | 0.222 | 287.01 | 11.96 | 1.66 | 0.60 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 0.281 | -0.281 | 0.372 | 643.28 | 26.80 | 15.14 | 1.82 | 0.08 | 0.02 |
| Gemini 3.1 Flash-Lite Preview | 0 | 0.198 | -0.031 | 0.255 | 66.31 | 2.76 | 0.42 | 0.15 | 0.01 | 0.00 |