| ground truth | LLM average | LLM std. dev. | DeepSeek-V4-Pro | Out tokens | Time | Kimi K2.6 | Out tokens | Time | MiniMax M2.7 | Out tokens | Time | Qwen 3.6 Plus | Out tokens | Time | GLM 5.1 | Out tokens | Time | GPT-5.5 | Out tokens | Time | GPT-5 nano | Out tokens | Time | Claude Opus 4.6 | Out tokens | Time | Claude Haiku 4.5 | Out tokens | Time | Gemini 3.1 Pro Preview | Out tokens | Time | Gemini 3.1 Flash-Lite Preview | Out tokens | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ENEL - SPA 2022 | 4.84 | 4.14 | 0.46 | 4.2 | 445 | 35.742 | 4.2 | 7579 | 76.677 | 4.2 | 3478 | 19.618 | 4.06 | 5715 | 66.266 | 4.2 | 476 | 11.014 | 4.2 | 357 | 8.017 | 2.9 | 18 | 1.235 | 4.4 | 1053 | 21.907 | 4.21 | 628 | 59.104 | 4.8 | 11185 | 88.043 | 4.2 | 671 | 2.58 |
| ENI S.P.A. 2022 | 0.95 | 0.86 | 0.44 | 0.452 | 447 | 35.558 | 0.82 | 9572 | 186.059 | 0.73 | 9952 | 65.979 | 0.82 | 3529 | 42.704 | 0.68 | 476 | 17.743 | 0.82 | 363 | 5.363 | 2.1 | 18 | 1.382 | 0.5 | 2014 | 43.177 | 0.88 | 611 | 4.942 | 1.0 | 15306 | 120.836 | 0.63 | 840 | 3.254 |
| A2A ENERGIA S.P.A. 2022 | 3.47 | 3.22 | 0.94 | 2.86 | 294 | 24.323 | 2.86 | 821 | 15.299 | 2.86 | 2318 | 14.845 | 3.03 | 3612 | 58.832 | 2.86 | 375 | 12.058 | 2.86 | 177 | 3.477 | 6.0 | 18 | 1.015 | 2.86 | 1094 | 25.353 | 2.86 | 421 | 3.485 | 3.5003322259136214 | 18112 | 145.78 | 2.86 | 392 | 1.679 |
| "TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022 | 4.35 | 4.21 | 0.72 | 4.06 | 438 | 13.593 | 4.18 | 6236 | 106.456 | 4.06 | 8282 | 109.503 | 4.33 | 2684 | 45.844 | 4.16 | 565 | 16.255 | 4.18 | 342 | 5.374 | 2.8 | 18 | 1.598 | 4.3 | 1077 | 21.386 | 5.92 | 482 | 3.559 | 4.45 | 6114 | 48.154 | 3.86 | 630 | 2.484 |
| REDEIA CORPORACION SA 2022 | 3.21 | 3.56 | 0.56 | 3.63 | 495 | 12.269 | 4.28 | 5904 | 51.827 | 4.28 | 4916 | 25.769 | 3.1 | 4193 | 45.594 | 3.79 | 519 | 10.624 | 3.79 | 350 | 6.947 | 2.4 | 18 | 2.211 | 3.2 | 2031 | 36.065 | 3.81 | 463 | 3.665 | 3.1 | 12256 | 93.498 | 3.7924 | 538 | 2.205 |
| EDP, S.A. 2022 | -8.22 | 3.74 | 0.71 | 5.319 | 1953 | 75.117 | 3.35 | 6278 | 76.501 | 3.35 | 12591 | 100.042 | 4.12 | 6363 | 89.356 | 2.83 | 673 | 18.769 | 3.64 | 327 | 5.263 | ~2.1x | 20 | 1.754 | 3.4 | 1496 | 33.4 | 3.65 | 749 | 6.566 | 4.42 | 16705 | 130.028 | 3.34 | 556 | 2.493 |
| ENGIE 2022 | 3.25 | 2.70 | 0.57 | 2.04 | 270 | 12.457 | 2.6 | 8924 | 183.528 | 2.64 | 22247 | 112.952 | 2.35 | 4488 | 63.895 | 2.35 | 664 | 24.216 | 2.64 | 269 | 4.463 | 2.6 | 18 | 1.611 | 2.6 | 1345 | 27.327 | 4.3 | 468 | 3.655 | 2.7 | 12304 | 111.003 | 2.86 | 556 | 2.051 |
| IBERDROLA SA 2022 | 3.91 | 4.04 | 1.06 | 3.408 | 390 | 13.838 | 4.44 | 8029 | 173.226 | 4.53 | 10996 | 69.908 | 3.88 | 5660 | 73.157 | 4.89 | 963 | 41.131 | 4.89 | 246 | 5.048 | 1.4 | 18 | 1.398 | 3.4 | 1582 | 33.817 | 4.89 | 566 | 4.395 | 3.8 | 21484 | 155.205 | 4.89 | 482 | 2.3 |
| NATURGY ENERGY GROUP SA 2022 | 2.67 | 2.57 | 0.37 | 2.48 | 272 | 32.114 | 2.48 | 3531 | 37.684 | 2.49 | 2464 | 17.042 | 2.48 | 1732 | 19.342 | 2.18 | 378 | 11.816 | 2.48 | 241 | 5.094 | 2.6 | 18 | 1.603 | 2.5 | 631 | 14.626 | 3.64 | 466 | 3.967 | 2.49 | 11815 | 97.897 | 2.486 | 747 | 2.958 |
| TenneT Holding B.V. 2022 | 83.85 | 37.08 | 25.50 | 56.9 | 515 | 37.551 | 54.07 | 8770 | 92.393 | 54.1 | 16489 | 83.902 | 2.01 | 7139 | 120.177 | 54.04 | 558 | 15.092 | 54.1 | 250 | 4.827 | -2.24 | 18 | 1.08 | 9.5 | 8104 | 162.736 | 11.8 | 461 | 3.57 | 56.68 | 13120 | 108.404 | 56.93 | 658 | 2.563 |
| VEOLIA ENVIRONNEMENT 2022 | 3.92 | 3.61 | 0.34 | 3.18 | 424 | 20.121 | 3.6 | 13393 | 151.236 | 3.58 | 8437 | 48.97 | 3.96 | 7089 | 85.908 | 3.9 | 792 | 25.925 | 3.63 | 386 | 5.554 | 4.1 | 18 | 1.336 | 3.8 | 1208 | 30.079 | 3.06 | 526 | 4.838 | 3.63 | 21824 | 166.08 | 3.22 | 697 | 2.679 |
| ØRSTED A/S 2022 | 2.5 | 1.61 | 0.33 | 1.73 | 301 | 10.347 | 1.73 | 877 | 16.631 | 0.94 | 5221 | 21.044 | 2.04 | 3358 | 38.134 | 1.47 | 754 | 19.163 | 1.73 | 240 | 4.547 | 1.14 | 18 | 1.361 | 2.0 | 1080 | 22.525 | 1.47 | 482 | 4.45 | 1.74 | 12107 | 95.487 | 1.7272 | 551 | 2.427 |
| ELECTRICITE DE FRANCE 2022 | 4.06 | -17.36 | 17.94 | -26.46 | 301 | 21.531 | -26.46 | 8927 | 152.658 | -24.75 | 8230 | 51.158 | -30.75 | 4494 | 64.026 | 26.47 | 837 | 22.465 | -26.47 | 299 | 5.45 | 6.4 | 18 | 1.027 | -12.1 | 1027 | 24.213 | -28.17 | 477 | 4.865 | -22.2 | 14364 | 112.402 | -26.46 | 517 | 2.143 |
| ERG S.P.A. 2022 | not available | 4.07 | 0.82 | 3.97 | 434 | 11.064 | 3.82 | 7422 | 100.893 | 3.53 | 4607 | 24.435 | 3.82 | 2963 | 36.768 | 3.65 | 718 | 25.56 | 3.8 | 273 | 4.969 | 6.2 | 18 | 1.073 | 3.8 | 945 | 20.65 | 4.42 | 528 | 4.26 | 3.11 | 12114 | 101.46 | 4.63 | 679 | 2.686 |
| Acciona SA 2022 | not available | 2.69 | 0.21 | 2.68 | 455 | 13.221 | 2.66 | 6723 | 93.78 | 2.66 | 12030 | 69.027 | 2.66 | 6880 | 75.438 | 2.18 | 778 | 16.592 | 2.66 | 280 | 4.936 | 2.9 | 18 | 1.032 | 2.64 | 937 | 20.855 | 2.96 | 614 | 4.355 | 2.93 | 23395 | 189.459 | 2.66 | 631 | 2.464 |
| Fortum Oyj 2022 | 1.84 | 2.61 | 2.41 | 1.74 | 364 | 20.636 | 1.59 | 14877 | 201.658 | 1.59 | 5612 | 40.019 | 1.59 | 2504 | 39.496 | 1.587 | 661 | 17.303 | 1.59 | 214 | 4.202 | 9.3 | 18 | 1.035 | 1.6 | 753 | 18.6 | 3.53 | 541 | 4.926 | 0.67 | 7467 | 66.667 | 3.95 | 618 | 2.592 |
| VERBUND AG 2022 | 1.6 | 1.17 | 0.32 | 1.213 | 574 | 19.677 | 1.12 | 1400 | 13.56 | 1.12 | 3004 | 18.066 | 1.12 | 1679 | 28.856 | 0.94 | 371 | 11.037 | 1.12 | 291 | 4.895 | 2.1 | 18 | 1.635 | 1.12 | 508 | 10.217 | 0.94 | 445 | 3.198 | 1.12 | 6428 | 51.348 | 0.942 | 470 | 1.909 |
| ITALGAS S.P.A. 2022 | 5.58 | 5.24 | 0.48 | 5.44 | 557 | 17.391 | 5.44 | 6832 | 83.734 | 5.43 | 8106 | 43.409 | 5.44 | 2642 | 51.565 | 5.44 | 546 | 15.69 | 5.42 | 368 | 6.063 | 3.9 | 18 | 1.88 | 5.4 | 903 | 17.215 | 4.85 | 425 | 3.532 | 5.5 | 6661 | 50.668 | 5.37 | 748 | 2.608 |
| ACEA S.P.A. 2022 | not available | 3.79 | 0.36 | 3.66 | 248 | 15.619 | 4.12 | 9579 | 132.071 | 3.66 | 2507 | 12.949 | 3.99 | 6092 | 92.102 | 3.892 | 789 | 16.697 | 3.73 | 376 | 6.202 | 2.9 | 18 | 1.134 | 4.12 | 1184 | 22.853 | 3.79 | 578 | 4.656 | 4.2178 | 12286 | 98.696 | 3.58 | 549 | 2.349 |
| REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022 | 4.37 | 4.32 | 0.71 | 4.04 | 738 | 15.216 | 4.03 | 4437 | 83.181 | 3.96 | 8409 | 46.971 | 4.03 | 1358 | 19.664 | 4.03 | 449 | 10.754 | 4.03 | 300 | 4.171 | 6.2 | 18 | 1.798 | 4.0 | 471 | 9.037 | 5.14 | 519 | 3.75 | 4.03 | 8028 | 65.473 | 4.0266 | 585 | 2.18 |
| HERA S.P.A. 2022 | 4.23 | 3.77 | 0.59 | 3.66 | 561 | 34.648 | 4.42 | 10520 | 140.652 | 3.66 | 5509 | 28.951 | 4.42 | 4694 | 72.691 | 3.73 | 652 | 18.435 | 3.73 | 350 | 5.767 | 2.3 | 18 | 1.309 | 4.42 | 774 | 16.076 | 3.73 | 558 | 4.104 | 3.7 | 17930 | 140.591 | 3.7255 | 653 | 2.59 |
| VINCI 2022 | 1.89 | 1.84 | 0.09 | 1.86 | 678 | 14.193 | 1.93 | 8409 | 107.956 | 1.84 | 6564 | 38.414 | 1.93 | 5839 | 85.906 | 1.93 | 504 | 17.608 | 1.86 | 388 | 5.922 | 1.8 | 18 | 1.182 | 1.86 | 643 | 14.307 | 1.72 | 462 | 3.762 | 1.86 | 8087 | 59.759 | 1.649 | 718 | 2.641 |
| BOUYGUES 2022 | 3.26 | 1.75 | 1.00 | 1.74 | 402 | 15.257 | 1.6 | 12444 | 147.537 | 1.6 | 3831 | 23.491 | 2.43 | 5719 | 88.801 | 1.58 | 766 | 22.625 | 1.61 | 168 | 3.671 | 3.8 | 18 | 1.073 | 2.2 | 1200 | 26.114 | 0.21 | 438 | 3.929 | 2.25 | 14165 | 109.021 | 0.21 | 578 | 2.363 |
| FERROVIAL SA 2022 | not available | 6.92 | 2.79 | 6.67 | 792 | 29.754 | 9.21 | 8373 | 106.363 | 7.5 | 9758 | 55.204 | 9.21 | 4863 | 80.187 | 2.9 | 644 | 21.915 | 9.3 | 402 | 6.681 | 2.6 | 18 | 1.266 | 7.6 | 2042 | 43.064 | 2.9 | 462 | 4.148 | 9.21 | 9079 | 70.436 | 9.0346 | 520 | 2.359 |
| Failures | Mean Absolute Error (ratio) | Mean Bias Error (ratio) | Root Mean Squared Error (ratio) | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.630 | -0.452 | 1.736 | 551.24 | 22.97 | 14.18 | 0.74 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.600 | -0.406 | 1.722 | 2531.56 | 105.48 | 54.92 | 1.11 | 0.05 | 0.02 |
| MiniMax M2.7 | 0 | 0.607 | -0.418 | 1.636 | 1141.67 | 47.57 | 30.37 | 0.34 | 0.01 | 0.01 |
| Qwen 3.6 Plus | 0 | 0.646 | -0.488 | 1.963 | 1484.71 | 61.86 | 25.64 | 0.59 | 0.02 | 0.01 |
| GLM 5.1 | 0 | 0.524 | 0.208 | 1.292 | 440.49 | 18.35 | 6.75 | 0.64 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.603 | -0.415 | 1.724 | 126.90 | 5.29 | 1.03 | 2.22 | 0.09 | 0.03 |
| GPT-5 nano | 1 | 0.619 | 0.172 | 1.064 | 33.03 | 1.38 | 0.32 | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.430 | -0.284 | 0.980 | 715.60 | 29.82 | 29.72 | 3.12 | 0.13 | 0.05 |
| Claude Haiku 4.5 | 0 | 0.776 | -0.374 | 1.851 | 155.68 | 6.49 | 11.23 | 0.52 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 0.529 | -0.367 | 1.501 | 2476.39 | 103.18 | 37.98 | 4.80 | 0.20 | 0.06 |
| Gemini 3.1 Flash-Lite Preview | 0 | 0.698 | -0.400 | 1.752 | 58.56 | 2.44 | 0.33 | 0.15 | 0.01 | 0.00 |
| ground truth | LLM average | LLM std. dev. | DeepSeek-V4-Pro | Out tokens | Time | Kimi K2.6 | Out tokens | Time | MiniMax M2.7 | Out tokens | Time | Qwen 3.6 Plus | Out tokens | Time | GLM 5.1 | Out tokens | Time | GPT-5.5 | Out tokens | Time | GPT-5 nano | Out tokens | Time | Claude Opus 4.6 | Out tokens | Time | Claude Haiku 4.5 | Out tokens | Time | Gemini 3.1 Pro Preview | Out tokens | Time | Gemini 3.1 Flash-Lite Preview | Out tokens | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ENEL - SPA 2022 | 0.1567 | 0.15 | 0.05 | 0.183 | 1922 | 79.045 | 0.257 | 9722 | 116.381 | 0.12 | 15938 | 100.371 | 0.15 | 5349 | 75.179 | 13.23% | 673 | 18.923 | 9.1% | 353 | 6.309 | 0.2 | 18 | 2.376 | 0.15 | 1172 | 28.153 | 0.13 | 514 | 4.744 | 0.174 | 8849 | 72.155 | 0.07483 | 716 | 3.081 |
| ENI S.P.A. 2022 | 0.6593 | 1.06 | 0.36 | 1.502 | 794 | 58.721 | 1.4 | 26755 | 306.378 | 1.2 | 10083 | 72.323 | 1.03 | 6868 | 87.373 | 1.221 | 780 | 22.271 | 85.5% | 501 | 7.991 | 0.28 | 18 | 1.858 | 1.2 | 1182 | 25.18 | 1.34 | 548 | 5.327 | 0.63 | 13430 | 109.565 | 0.9653 | 552 | 2.725 |
| A2A ENERGIA S.P.A. 2022 | 0.2317 | 0.29 | 0.18 | 0.246 | 388 | 18.667 | 0.27 | 18841 | 225.778 | 0.81 | 25216 | 163.478 | 0.23 | 6952 | 88.2 | 0.31 | 631 | 26.157 | 10.1% | 535 | 8.58 | 0.121 | 18 | 1.147 | 0.28 | 1054 | 20.732 | 0.3 | 530 | 4.227 | 0.27 | 23124 | 206.084 | 0.2855 | 685 | 2.915 |
| "TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022 | 0.1724 | 0.19 | 0.02 | 0.1915 | 606 | 22.181 | 0.187 | 14872 | 319.514 | 0.19 | 22843 | 101.469 | 0.22 | 4205 | 50.2 | 0.178 | 896 | 30.041 | 18.6% | 402 | 5.557 | 0.23 | 18 | 2.515 | 0.194 | 1105 | 23.382 | 0.149 | 559 | 5.151 | 0.17 | 12097 | 89.229 | 0.18595 | 778 | 2.725 |
| REDEIA CORPORACION SA 2022 | 0.2063 | 1.59 | 4.58 | 0.173 | 8091 | 168.105 | 0.173 | 22208 | 254.76 | 0.246 | 28578 | 182.68 | 0.19 | 5621 | 71.369 | 0.23 | 701 | 15.611 | 15.4 | 346 | 6.617 | 0.24 | 18 | 1.718 | 0.22 | 1145 | 24.123 | 0.265 | 502 | 4.304 | 0.174 | 24091 | 185.013 | 0.1537 | 496 | 2.251 |
| EDP, S.A. 2022 | -0.1579 | 0.19 | 0.06 | 0.1857 | 826 | 24.463 | 0.134 | 30580 | 356.565 | 0.25 | 19344 | 150.723 | 0.12 | 6770 | 94.519 | 0.3 | 1436 | 33.077 | 19.4% | 404 | 7.413 | 0.15 | 18 | 1.963 | 0.14 | 1171 | 26.825 | 0.28 | 642 | 5.917 | 0.124 | 13851 | 115.019 | 0.20918 | 628 | 2.815 |
| ENGIE 2022 | 0.2483 | 0.33 | 0.13 | 0.4356 | 481 | 21.956 | 0.46 | 20740 | 269.434 | 0.376 | 16361 | 89.714 | 0.48 | 6978 | 85.018 | 0.11 | 816 | 20.464 | 43.6% | 323 | 5.11 | 0.11 | 18 | 1.121 | 0.32 | 1069 | 24.339 | 0.31 | 465 | 4.846 | 0.26 | 20353 | 178.157 | 0.3751 | 626 | 2.59 |
| IBERDROLA SA 2022 | 0.1965 | 0.19 | 0.04 | 0.166 | 667 | 37.789 | 0.19 | 17572 | 236.581 | 0.191 | 14938 | 83.76 | 0.18 | 5187 | 65.309 | 0.12 | 851 | 30.358 | 22.4% | 550 | 8.261 | 0.235 | 18 | 2.022 | 0.253 | 1042 | 23.643 | 0.18 | 559 | 5.12 | 0.204 | 9867 | 67.513 | 0.17899 | 627 | 2.681 |
| NATURGY ENERGY GROUP SA 2022 | 0.2847 | 3.62 | 10.97 | 0.2985 | 1438 | 43.44 | 0.31 | 22415 | 255.093 | 0.345 | 12094 | 62.325 | 0.27 | 5547 | 75.7 | 0.398 | 730 | 19.164 | 36.7 | 374 | 6.207 | 0.182 | 18 | 1.315 | 0.37 | 850 | 20.087 | 0.24 | 745 | 6.993 | 0.3665 | 10531 | 85.752 | 0.3665 | 577 | 2.126 |
| TenneT Holding B.V. 2022 | -0.0105 | 0.14 | 0.16 | -0.013 | 672 | 29.562 | 0.179 | 26526 | 284.116 | 0.1 | 19959 | 137.784 | 0.024 | 7268 | 91.079 | -0.0039 | 1332 | 53.775 | 17.7% | 482 | 7.496 | 0.2 | 18 | 1.197 | 0.07 | 1140 | 26.939 | 0.54 | 871 | 7.578 | 0.076 | 28760 | 225.117 | 0.2269 | 815 | 3.265 |
| VEOLIA ENVIRONNEMENT 2022 | 0.1972 | 0.21 | 0.04 | 0.23 | 1310 | 27.936 | 0.245 | 13835 | 184.139 | 0.239 | 24175 | 432.212 | 0.23 | 4790 | 59.882 | 21.16% | 821 | 33.347 | 17.2% | 535 | 8.213 | 0.14 | 18 | 1.445 | 0.163 | 1189 | 26.648 | 0.18 | 645 | 5.708 | 0.21 | 21541 | 162.603 | 0.2792 | 612 | 3.023 |
| ØRSTED A/S 2022 | 0.3124 | 3.88 | 11.75 | 0.593 | 697 | 34.453 | 0.52 | 21554 | 444.245 | 0.51 | 17398 | 87.339 | 43.4% | 8433 | 91.499 | -8.4% | 708 | 26.7 | 33.0% | 493 | 8.153 | 0.2 | 18 | 1.852 | 0.24 | 1197 | 26.146 | 0.29 | 501 | 4.793 | 39.3 | 25603 | 194.703 | 0.3635 | 709 | 2.849 |
| ELECTRICITE DE FRANCE 2022 | 0.1884 | 0.04 | 0.15 | 0.00664 | 785 | 27.719 | 0.48 | 21365 | 235.839 | 0.0048 | 17103 | 86.905 | 0.0048 | 5794 | 63.625 | -0.0 | 733 | 32.012 | 3.45% | 397 | 6.947 | 0.08 | 18 | 1.707 | 0.011 | 1085 | 27.122 | -0.028 | 692 | 6.032 | -0.0467 | 24694 | 194.691 | -0.0561 | 736 | 3.084 |
| ERG S.P.A. 2022 | not available | 0.22 | 0.06 | 0.153 | 973 | 19.233 | 0.25 | 17683 | 238.292 | 0.213 | 19667 | 296.727 | 0.14 | 9497 | 96.315 | 18.56% | 766 | 22.505 | 0.34 | 509 | 7.853 | 0.23 | 18 | 1.885 | 0.2 | 974 | 20.465 | 0.288 | 594 | 5.345 | 0.204 | 25891 | 184.727 | 0.2322 | 836 | 2.93 |
| Acciona SA 2022 | not available | 0.35 | 0.23 | 0.3428 | 816 | 24.992 | 0.302 | 17368 | 203.301 | 0.372 | 9578 | 45.78 | 0.31 | 6129 | 66.722 | 1.03 | 588 | 20.199 | 30.9% | 350 | 6.119 | 0.14 | 18 | 2.246 | 0.27 | 965 | 20.185 | 0.19 | 1025 | 8.859 | 0.288 | 13364 | 106.25 | 0.299 | 625 | 2.707 |
| Fortum Oyj 2022 | 0.4556 | 0.40 | 0.18 | 0.535 | 903 | 30.743 | 0.55 | 18016 | 224.631 | 0.55 | 12637 | 67.015 | 0.29 | 8290 | 87.45 | 0.344 | 1078 | 43.41 | 60.1% | 286 | 5.457 | 0.2 | 18 | 1.133 | 0.44 | 1060 | 24.389 | 0.12 | 629 | 5.695 | 0.6 | 8096 | 67.234 | 0.2079 | 640 | 2.505 |
| VERBUND AG 2022 | 0.5174 | 14.05 | 29.78 | 0.786 | 655 | 21.938 | 0.71 | 15351 | 191.836 | 0.845 | 12235 | 63.209 | 0.85 | 5789 | 72.427 | 77.44 | 1161 | 45.001 | 70.98 | 336 | 6.148 | 0.24 | 18 | 1.803 | 0.78 | 1003 | 22.769 | 0.57 | 619 | 5.681 | 0.78 | 19532 | 166.76 | 0.5698 | 565 | 2.417 |
| ITALGAS S.P.A. 2022 | 0.1456 | 1.46 | 4.33 | 0.1746 | 1031 | 27.762 | 0.153 | 14888 | 262.946 | 0.1773 | 16836 | 87.866 | 0.15 | 5733 | 70.607 | 0.18 | 774 | 33.263 | 14.5 | 333 | 6.882 | 0.11 | 18 | 1.24 | 0.146 | 977 | 21.284 | 0.17 | 624 | 5.394 | 0.1452 | 13791 | 101.864 | 0.1452 | 623 | 2.449 |
| ACEA S.P.A. 2022 | not available | 0.22 | 0.03 | 0.2185 | 780 | 24.531 | 0.222 | 19947 | 207.894 | 0.2222 | 11734 | 94.159 | 0.21 | 3624 | 40.174 | 0.202545 | 562 | 12.085 | 21.8% | 395 | 7.202 | 0.28 | 18 | 1.479 | 0.16 | 1175 | 22.978 | 0.225 | 486 | 5.152 | 0.216 | 27471 | 217.017 | 0.2025 | 589 | 2.406 |
| REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022 | 0.1686 | 0.21 | 0.08 | 0.183 | 705 | 13.97 | 0.182 | 16758 | 204.728 | 0.37 | 20421 | 100.323 | 0.18 | 5394 | 78.067 | 0.18 | 977 | 29.321 | 12.6% | 398 | 6.861 | 0.18 | 18 | 1.509 | 0.145 | 1105 | 24.86 | 0.31 | 610 | 5.699 | 0.19 | 18603 | 152.666 | 0.31156 | 618 | 2.717 |
| HERA S.P.A. 2022 | 0.1733 | 0.26 | 0.11 | 0.2192 | 479 | 20.386 | 0.235 | 13449 | 228.861 | 0.27 | 6324 | 46.987 | 0.24 | 7661 | 89.497 | 0.25 | 639 | 26.179 | 23.2% | 372 | 5.627 | 0.2 | 18 | 0.85 | 0.17 | 903 | 20.767 | 0.57 | 542 | 4.567 | 0.211 | 11487 | 101.168 | 0.2443 | 745 | 4.196 |
| VINCI 2022 | 0.4221 | 0.43 | 0.08 | 0.428 | 438 | 13.3 | 0.442 | 22592 | 259.666 | 0.54 | 25476 | 125.175 | 0.43 | 6658 | 74.411 | 0.43 | 789 | 29.029 | 42.9% | 504 | 8.957 | 0.24 | 18 | 1.155 | 0.47 | 1063 | 24.851 | 0.375 | 569 | 4.687 | 0.479 | 23527 | 201.545 | 0.469 | 664 | 2.752 |
| BOUYGUES 2022 | 0.2446 | 0.41 | 0.08 | 0.4099 | 2106 | 48.729 | 0.447 | 23366 | 319.924 | 0.38 | 37802 | 188.949 | 0.51 | 8184 | 98.664 | 0.4329 | 1077 | 32.782 | 48.2% | 150 | 4.049 | 0.29 | 18 | 1.797 | 0.36 | 1255 | 29.067 | 0.54 | 719 | 7.045 | 0.33 | 10693 | 82.658 | 0.3188 | 444 | 2.045 |
| FERROVIAL SA 2022 | not available | 0.14 | 0.05 | 0.1 | 1040 | 42.424 | 0.092 | 21911 | 275.016 | 0.1725 | 15552 | 113.924 | 0.1 | 7405 | 86.658 | 0.2 | 800 | 31.862 | 0.198 | 426 | 7.15 | 0.1 | 18 | 1.402 | 0.13 | 925 | 20.512 | 0.19 | 543 | 5.157 | 0.088 | 17870 | 142.789 | 0.1409 | 473 | 2.204 |
| Failures | Mean Absolute Error (ratio) | Mean Bias Error (ratio) | Root Mean Squared Error (ratio) | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.456 | 0.304 | 0.698 | 882.04 | 36.75 | 31.72 | 0.80 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 1.372 | 1.353 | 4.096 | 6105.92 | 254.41 | 65.20 | 2.26 | 0.09 | 0.02 |
| MiniMax M2.7 | 0 | 1.158 | 1.035 | 2.537 | 2981.20 | 124.22 | 85.94 | 0.64 | 0.03 | 0.01 |
| Qwen 3.6 Plus | 0 | 0.560 | 0.400 | 0.953 | 1859.94 | 77.50 | 14.89 | 0.73 | 0.03 | 0.01 |
| GLM 5.1 | 0 | 7.958 | 7.596 | 33.254 | 687.54 | 28.65 | 9.52 | 0.67 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 23.078 | 22.860 | 50.188 | 165.16 | 6.88 | 1.22 | 2.30 | 0.10 | 0.03 |
| GPT-5 nano | 0 | 1.417 | 0.920 | 4.519 | 38.74 | 1.61 | 0.43 | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.716 | 0.558 | 1.800 | 575.45 | 23.98 | 2.72 | 2.91 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 3.218 | 2.948 | 11.763 | 134.02 | 5.58 | 1.09 | 0.53 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 6.933 | 6.786 | 27.972 | 3410.28 | 142.09 | 52.63 | 6.17 | 0.26 | 0.08 |
| Gemini 3.1 Flash-Lite Preview | 0 | 1.579 | 1.307 | 5.103 | 65.46 | 2.73 | 0.45 | 0.15 | 0.01 | 0.00 |
| ground truth | LLM average | LLM std. dev. | DeepSeek-V4-Pro | Out tokens | Time | Kimi K2.6 | Out tokens | Time | MiniMax M2.7 | Out tokens | Time | Qwen 3.6 Plus | Out tokens | Time | GLM 5.1 | Out tokens | Time | GPT-5.5 | Out tokens | Time | GPT-5 nano | Out tokens | Time | Claude Opus 4.6 | Out tokens | Time | Claude Haiku 4.5 | Out tokens | Time | Gemini 3.1 Pro Preview | Out tokens | Time | Gemini 3.1 Flash-Lite Preview | Out tokens | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ENEL - SPA 2022 | Deteriorating | 0.00 | 0.45 | Stable | 521 | 14.804 | Stable | 12249 | 126.102 | Stable | 8721 | 161.763 | Stable | 2094 | 29.904 | Deteriorating | 984 | 38.85 | Stable | 456 | 6.431 | Stable | 16 | 1.377 | Stable | 1158 | 19.267 | Improving | 1276 | 9.095 | Stable | 4339 | 34.253 | Stable | 909 | 3.093 |
| ENI S.P.A. 2022 | Improving | 0.91 | 0.30 | Improving | 883 | 27.571 | Improving | 2339 | 22.518 | Improving | 10338 | 216.765 | Improving | 1568 | 34.005 | Improving | 930 | 36.951 | Improving | 537 | 7.541 | Stable | 16 | 1.008 | Improving | 936 | 14.542 | Improving | 893 | 6.289 | Improving | 5928 | 39.727 | Improving | 799 | 3.168 |
| A2A ENERGIA S.P.A. 2022 | Stable | 0.09 | 0.30 | Stable | 438 | 12.892 | Stable | 840 | 17.09 | Stable | 7733 | 84.769 | Stable | 1371 | 20.025 | Stable | 749 | 18.721 | Stable | 293 | 4.727 | Improving | 906 | 6.017 | Stable | 757 | 13.61 | Stable | 876 | 5.995 | Stable | 3228 | 26.322 | Stable | 710 | 2.987 |
| "TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022 | Improving | 0.91 | 0.30 | Improving | 504 | 16.928 | Improving | 806 | 32.324 | Improving | 3293 | 82.47 | Improving | 1955 | 31.162 | Improving | 733 | 17.37 | Improving | 473 | 6.924 | Stable | 16 | 1.362 | Improving | 943 | 16.004 | Improving | 863 | 5.809 | Improving | 4028 | 29.774 | Improving | 747 | 2.673 |
| REDEIA CORPORACION SA 2022 | Stable | 0.55 | 0.52 | Improving | 609 | 12.946 | Improving | 10917 | 151.575 | Stable | 3277 | 155.858 | Stable | 2329 | 25.659 | Improving | 863 | 30.498 | Stable | 431 | 6.221 | Stable | 16 | 1.322 | Improving | 1134 | 19.752 | Improving | 905 | 6.118 | Improving | 7389 | 54.474 | Stable | 785 | 3.419 |
| EDP, S.A. 2022 | Improving | 0.82 | 0.40 | Improving | 591 | 22.612 | Improving | 1781 | 18.916 | Improving | 5431 | 65.566 | Stable | 3363 | 42.385 | Improving | 785 | 17.609 | Improving | 415 | 8.736 | Stable | 16 | 0.938 | Improving | 1007 | 18.487 | Improving | 868 | 6.453 | Improving | 8811 | 62.924 | Improving | 707 | 3.453 |
| ENGIE 2022 | Deteriorating | 0.09 | 0.70 | Stable | 346 | 17.597 | Stable | 13229 | 211.462 | Improving | 7608 | 83.913 | Stable | 3594 | 55.306 | Deteriorating | 862 | 24.119 | Deteriorating | 488 | 7.227 | Stable | 16 | 1.017 | Stable | 853 | 15.822 | Improving | 659 | 4.768 | Improving | 4680 | 37.187 | Stable | 596 | 2.114 |
| IBERDROLA SA 2022 | Deteriorating | -0.73 | 0.47 | Deteriorating | 519 | 16.846 | Deteriorating | 1323 | 33.03 | Deteriorating | 4637 | 44.923 | Deteriorating | 2465 | 34.98 | Deteriorating | 608 | 21.776 | Deteriorating | 334 | 5.657 | Stable | 16 | 0.816 | Stable | 1114 | 18.11 | Deteriorating | 653 | 5.06 | Stable | 4937 | 36.139 | Deteriorating | 701 | 2.785 |
| NATURGY ENERGY GROUP SA 2022 | Improving | 0.91 | 0.30 | Improving | 676 | 17.914 | Improving | 1277 | 13.355 | Improving | 5950 | 47.073 | Improving | 1740 | 19.01 | Improving | 704 | 24.692 | Improving | 353 | 5.548 | Stable | 16 | 1.367 | Improving | 743 | 12.724 | Improving | 822 | 6.308 | Improving | 3085 | 22.762 | Improving | 728 | 2.395 |
| TenneT Holding B.V. 2022 | NA | -0.91 | 0.30 | Deteriorating | 304 | 7.837 | Deteriorating | 3153 | 96.306 | Deteriorating | 7823 | 194.746 | Deteriorating | 1809 | 22.94 | Deteriorating | 526 | 13.475 | Deteriorating | 327 | 4.943 | Stable | 16 | 0.915 | Deteriorating | 891 | 16.278 | Deteriorating | 788 | 5.508 | Deteriorating | 3768 | 26.625 | Deteriorating | 691 | 2.526 |
| VEOLIA ENVIRONNEMENT 2022 | Deteriorating | -0.91 | 0.30 | Deteriorating | 514 | 18.427 | Deteriorating | 1084 | 45.51 | Deteriorating | 5549 | 60.613 | Deteriorating | 3216 | 47.165 | Deteriorating | 838 | 16.803 | Deteriorating | 636 | 8.337 | Stable | 16 | 0.948 | Deteriorating | 1149 | 19.91 | Deteriorating | 876 | 6.827 | Deteriorating | 5337 | 38.052 | Deteriorating | 1121 | 6.087 |
| ØRSTED A/S 2022 | Improving | 0.64 | 0.50 | Stable | 517 | 13.685 | Improving | 4354 | 43.87 | Improving | 6658 | 100.524 | Improving | 2268 | 45.172 | Improving | 682 | 20.366 | Improving | 442 | 6.498 | Stable | 16 | 1.015 | Stable | 1070 | 21.869 | Improving | 879 | 6.266 | Improving | 5724 | 41.672 | Stable | 596 | 2.524 |
| ELECTRICITE DE FRANCE 2022 | Improving | -0.91 | 0.30 | Deteriorating | 462 | 16.895 | Deteriorating | 2621 | 37.603 | Deteriorating | 6432 | 181.751 | Deteriorating | 1538 | 31.501 | Deteriorating | 582 | 17.191 | Deteriorating | 361 | 6.328 | Stable | 16 | 0.928 | Deteriorating | 708 | 13.489 | Deteriorating | 844 | 6.565 | Deteriorating | 2802 | 24.494 | Deteriorating | 598 | 2.52 |
| ERG S.P.A. 2022 | Improving | 0.91 | 0.30 | Improving | 824 | 30.26 | Improving | 1780 | 55.744 | Improving | 3616 | 51.221 | Improving | 1409 | 17.355 | Improving | 838 | 18.793 | Improving | 680 | 9.364 | Stable | 16 | 0.901 | Improving | 1121 | 19.941 | Improving | 989 | 6.967 | Improving | 3994 | 27.151 | Improving | 963 | 3.262 |
| Acciona SA 2022 | Improving | 0.55 | 0.52 | Improving | 619 | 15.733 | Improving | 9739 | 117.753 | Improving | 7945 | 104.693 | Improving | 2624 | 50.391 | Improving | 812 | 24.987 | Stable | 443 | 6.101 | Stable | 16 | 0.89 | Stable | 1253 | 22.365 | Stable | 835 | 5.783 | Improving | 6560 | 48.951 | Stable | 892 | 3.66 |
| Fortum Oyj 2022 | Improving | 0.45 | 0.82 | Improving | 384 | 26.546 | Improving | 10563 | 197.906 | Stable | 5357 | 50.056 | Improving | 1477 | 15.428 | Deteriorating | 632 | 19.992 | Stable | 324 | 5.606 | Deteriorating | 621 | 4.815 | Improving | 876 | 16.43 | Improving | 712 | 4.943 | Improving | 3350 | 27.812 | Improving | 731 | 2.9 |
| VERBUND AG 2022 | Improving | 0.82 | 0.40 | Improving | 370 | 11.979 | Improving | 1039 | 12.639 | Stable | 5953 | 44.37 | Improving | 1825 | 31.016 | Improving | 1034 | 37.561 | Improving | 359 | 5.052 | Stable | 16 | 0.979 | Improving | 570 | 9.388 | Improving | 737 | 5.733 | Improving | 2785 | 21.974 | Improving | 711 | 2.423 |
| ITALGAS S.P.A. 2022 | Deteriorating | -0.91 | 0.30 | Deteriorating | 637 | 12.689 | Deteriorating | 661 | 14.623 | Deteriorating | 2820 | 48.821 | Deteriorating | 1367 | 26.881 | Deteriorating | 950 | 20.317 | Deteriorating | 467 | 6.882 | Stable | 16 | 0.94 | Deteriorating | 893 | 18.198 | Deteriorating | 880 | 6.064 | Deteriorating | 3507 | 27.135 | Deteriorating | 886 | 2.873 |
| ACEA S.P.A. 2022 | Deteriorating | 0.18 | 0.40 | Stable | 425 | 25.143 | Stable | 6504 | 91.2 | Stable | 11979 | 207.234 | Stable | 2801 | 45.116 | Improving | 806 | 22.885 | Stable | 341 | 5.566 | Stable | 16 | 1.786 | Stable | 649 | 10.76 | Improving | 899 | 6.956 | Stable | 6527 | 50.619 | Stable | 663 | 2.405 |
| REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022 | Improving | 0.91 | 0.30 | Improving | 514 | 14.687 | Improving | 1225 | 12.511 | Improving | 5700 | 106.426 | Improving | 1790 | 30.608 | Improving | 909 | 26.317 | Improving | 423 | 5.937 | Stable | 16 | 0.855 | Improving | 713 | 10.921 | Improving | 704 | 4.686 | Improving | 3492 | 26.825 | Improving | 567 | 2.234 |
| HERA S.P.A. 2022 | Deteriorating | -0.91 | 0.30 | Deteriorating | 836 | 25.315 | Deteriorating | 7943 | 108.022 | Deteriorating | 5305 | 158.094 | Deteriorating | 1927 | 37.389 | Deteriorating | 570 | 13.861 | Deteriorating | 462 | 6.452 | Stable | 16 | 0.871 | Deteriorating | 895 | 16.079 | Deteriorating | 882 | 5.588 | Deteriorating | 3093 | 21.377 | Deteriorating | 673 | 2.754 |
| VINCI 2022 | Improving | 0.82 | 0.40 | Improving | 688 | 12.695 | Improving | 8580 | 107.676 | Stable | 15438 | 116.275 | Improving | 1836 | 35.964 | Improving | 722 | 27.617 | Improving | 489 | 7.065 | Stable | 16 | 1.038 | Improving | 785 | 13.242 | Improving | 825 | 5.767 | Improving | 3485 | 26.797 | Improving | 616 | 2.845 |
| BOUYGUES 2022 | Deteriorating | -1.00 | 0.00 | Deteriorating | 372 | 13.096 | Deteriorating | 3410 | 38.761 | Deteriorating | 11267 | 122.359 | Deteriorating | 1337 | 23.417 | Deteriorating | 461 | 21.902 | Deteriorating | 304 | 4.826 | Deteriorating | 700 | 4.721 | Deteriorating | 896 | 17.326 | Deteriorating | 735 | 5.942 | Deteriorating | 2466 | 18.063 | Deteriorating | 562 | 2.4 |
| FERROVIAL SA 2022 | Deteriorating | -0.91 | 0.30 | Deteriorating | 701 | 35.554 | Deteriorating | 9413 | 107.181 | Deteriorating | 7330 | 220.25 | Deteriorating | 3813 | 65.345 | Deteriorating | 703 | 15.667 | Deteriorating | 540 | 8.865 | Stable | 16 | 0.985 | Deteriorating | 983 | 18.668 | Deteriorating | 663 | 5.046 | Deteriorating | 6481 | 47.203 | Deteriorating | 720 | 2.952 |
| Failures | Mean Absolute Error | Mean Bias Error | Root Mean Squared Error | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.30 | 0.04 | 0.63 | 440.65 | 18.36 | 6.81 | 0.75 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.26 | 0.09 | 0.59 | 1713.68 | 71.40 | 59.21 | 0.86 | 0.04 | 0.02 |
| MiniMax M2.7 | 0 | 0.39 | -0.04 | 0.75 | 2710.53 | 112.94 | 59.83 | 0.32 | 0.01 | 0.00 |
| Qwen 3.6 Plus | 0 | 0.26 | 0.00 | 0.59 | 818.12 | 34.09 | 12.62 | 0.43 | 0.02 | 0.00 |
| GLM 5.1 | 0 | 0.30 | -0.04 | 0.75 | 548.32 | 22.85 | 7.15 | 0.66 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.26 | -0.09 | 0.59 | 156.83 | 6.53 | 1.30 | 2.33 | 0.10 | 0.03 |
| GPT-5 nano | 0 | 0.96 | -0.17 | 1.02 | 37.81 | 1.58 | 1.43 | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.39 | 0.04 | 0.69 | 393.18 | 16.38 | 3.50 | 2.83 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 0.43 | 0.17 | 0.88 | 144.54 | 6.02 | 0.92 | 0.56 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 0.35 | 0.17 | 0.72 | 818.31 | 34.10 | 11.78 | 2.37 | 0.10 | 0.03 |
| Gemini 3.1 Flash-Lite Preview | 0 | 0.30 | -0.04 | 0.63 | 70.45 | 2.94 | 0.78 | 0.16 | 0.01 | 0.00 |
| Failures | Mean Absolute Error | Mean Bias Error | Root Mean Squared Error | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.96 | 0.46 | 1.21 | 463.14 | 19.30 | 9.34 | 0.74 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.79 | 0.29 | 0.98 | 336.20 | 14.01 | 5.09 | 0.47 | 0.02 | 0.01 |
| MiniMax M2.7 | 0 | 0.88 | 0.29 | 1.14 | 1643.93 | 68.50 | 30.44 | 0.22 | 0.01 | 0.00 |
| Qwen 3.6 Plus | 0 | 0.92 | 0.83 | 1.22 | 409.45 | 17.06 | 5.36 | 0.34 | 0.01 | 0.00 |
| GLM 5.1 | 0 | 0.92 | 0.92 | 1.22 | 414.37 | 17.27 | 3.62 | 0.63 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.83 | 0.33 | 1.04 | 252.37 | 10.52 | 1.34 | 2.32 | 0.10 | 0.03 |
| GPT-5 nano | 1 | 0.78 | 0.09 | 1.02 | 83.74 | 3.49 | 1.22 | 0.02 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.92 | 0.83 | 1.22 | 537.09 | 22.38 | 2.41 | 2.79 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 0.92 | 0.33 | 1.12 | 232.96 | 9.71 | 1.27 | 0.57 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 0.96 | 0.79 | 1.27 | 340.36 | 14.18 | 3.09 | 1.48 | 0.06 | 0.01 |
| Gemini 3.1 Flash-Lite Preview | 0 | 0.92 | 0.75 | 1.26 | 58.27 | 2.43 | 0.28 | 0.15 | 0.01 | 0.00 |
| ground truth | LLM average | LLM std. dev. | DeepSeek-V4-Pro | Out tokens | Time | Kimi K2.6 | Out tokens | Time | MiniMax M2.7 | Out tokens | Time | Qwen 3.6 Plus | Out tokens | Time | GLM 5.1 | Out tokens | Time | GPT-5.5 | Out tokens | Time | GPT-5 nano | Out tokens | Time | Claude Opus 4.6 | Out tokens | Time | Claude Haiku 4.5 | Out tokens | Time | Gemini 3.1 Pro Preview | Out tokens | Time | Gemini 3.1 Flash-Lite Preview | Out tokens | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ENEL - SPA 2022 | 50% | 0.55 | 0.33 | 100% | 639 | 19.263 | 50% | 1123 | 16.232 | 100% | 6357 | 86.146 | 25% | 1370 | 34.894 | 25% | 564 | 16.77 | 25% | 334 | 7.29 | 25% | 725 | 5.89 | 75% | 970 | 27.234 | 50% | 1078 | 11.701 | 100% | 2012 | 17.593 | 25% | 618 | 3.466 |
| ENI S.P.A. 2022 | 25% | 0.64 | 0.34 | 50% | 635 | 29.834 | 50% | 2588 | 28.25 | 25% | 7106 | 170.717 | 100% | 990 | 23.45 | 100% | 693 | 18.292 | 75% | 550 | 11.435 | 100% | 893 | 7.415 | 50% | 917 | 24.884 | 0% | 900 | 9.716 | 50% | 9787 | 69.779 | 100% | 644 | 2.731 |
| A2A ENERGIA S.P.A. 2022 | 25% | 0.55 | 0.37 | 50% | 615 | 18.666 | 50% | 1171 | 24.254 | 50% | 3282 | 94.762 | 0% | 1824 | 27.805 | 100% | 588 | 19.509 | 75% | 479 | 12.182 | 0% | 161 | 1.934 | 50% | 719 | 18.455 | 25% | 858 | 9.382 | 100% | 3418 | 32.052 | 100% | 520 | 3.012 |
| "TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022 | 25% | 0.55 | 0.25 | 100% | 520 | 12.224 | 50% | 1403 | 16.649 | 50% | 6834 | 65.115 | 50% | 3003 | 42.965 | 50% | 466 | 10.239 | 50% | 393 | 8.804 | 25% | 831 | 6.664 | 50% | 884 | 21.461 | 50% | 1045 | 11.166 | 100% | 4203 | 36.748 | 25% | 589 | 2.629 |
| REDEIA CORPORACION SA 2022 | 50% | 0.66 | 0.32 | 75% | 595 | 30.832 | 50% | 1514 | 20.298 | 100% | 2299 | 22.529 | 50% | 1492 | 33.511 | 100% | 857 | 26.463 | 50% | 591 | 12.865 | 0% | 588 | 5.479 | 50% | 863 | 20.936 | 50% | 897 | 10.246 | 100% | 2344 | 18.963 | 100% | 594 | 2.595 |
| EDP, S.A. 2022 | 50% | 0.68 | 0.32 | 75% | 593 | 17.706 | 50% | 1530 | 50.282 | 50% | 3366 | 34.582 | 100% | 1371 | 18.935 | 50% | 581 | 12.435 | 75% | 421 | 10.962 | 0% | 631 | 4.934 | 100% | 998 | 27.009 | 50% | 1218 | 13.197 | 100% | 2045 | 18.658 | 100% | 478 | 2.798 |
| ENGIE 2022 | 50% | 0.70 | 0.29 | 75% | 469 | 22.095 | 50% | 1270 | 13.163 | 75% | 4409 | 63.825 | 100% | 1081 | 18.863 | 100% | 652 | 18.956 | 50% | 448 | 10.882 | 100% | 393 | 3.723 | 75% | 946 | 26.088 | 25% | 1201 | 13.038 | 100% | 3413 | 31.439 | 25% | 649 | 2.699 |
| IBERDROLA SA 2022 | 75% | 0.75 | 0.27 | 100% | 473 | 20.651 | 75% | 1286 | 15.963 | 100% | 20022 | 187.526 | 50% | 1459 | 18.913 | 100% | 520 | 14.307 | 50% | 344 | 10.073 | 25% | 16 | 1.037 | 75% | 965 | 24.938 | 50% | 1097 | 11.578 | 100% | 2837 | 22.82 | 100% | 563 | 2.677 |
| NATURGY ENERGY GROUP SA 2022 | 25% | 0.66 | 0.39 | 50% | 477 | 18.605 | 100% | 966 | 15.98 | 100% | 2541 | 34.688 | 100% | 1643 | 41.726 | 100% | 537 | 20.409 | 75% | 424 | 10.288 | 0% | 509 | 4.337 | 50% | 914 | 23.325 | 50% | 1029 | 10.537 | 100% | 4646 | 37.31 | 0% | 713 | 2.766 |
| TenneT Holding B.V. 2022 | 50% | 0.57 | 0.20 | 75% | 527 | 18.972 | 100% | 1143 | 11.165 | 50% | 6428 | 47.989 | 50% | 3324 | 39.18 | 50% | 441 | 12.398 | 50% | 408 | 9.198 | 50% | 709 | 5.094 | 75% | 899 | 30.674 | 25% | 1115 | 12.452 | 50% | 4070 | 28.831 | 50% | 639 | 2.676 |
| VEOLIA ENVIRONNEMENT 2022 | 50% | 0.57 | 0.25 | 50% | 542 | 15.862 | 50% | 2756 | 35.287 | 50% | 4547 | 25.613 | 50% | 2454 | 31.214 | 50% | 678 | 16.089 | 75% | 340 | 9.199 | 25% | 361 | 3.631 | 25% | 1042 | 24.832 | 50% | 903 | 10.55 | 100% | 4985 | 38.83 | 100% | 542 | 2.924 |
| ØRSTED A/S 2022 | 50% | 0.82 | 0.16 | 100% | 480 | 15.157 | 100% | 3851 | 69.518 | 100% | 6819 | 129.954 | 75% | 3305 | 62.045 | 75% | 594 | 13.047 | 75% | 332 | 7.929 | 100% | 768 | 8.614 | 75% | 956 | 24.336 | 50% | 1233 | 12.757 | 75% | 3311 | 27.034 | 75% | 671 | 2.801 |
| ELECTRICITE DE FRANCE 2022 | 75% | 0.77 | 0.36 | 100% | 501 | 26.483 | 50% | 1886 | 39.039 | 100% | 6105 | 171.561 | 100% | 1207 | 18.626 | 100% | 776 | 24.612 | 75% | 370 | 10.491 | 25% | 526 | 4.668 | 100% | 841 | 23.415 | 0% | 1069 | 12.178 | 100% | 2806 | 28.354 | 100% | 549 | 2.7 |
| ERG S.P.A. 2022 | 0% | 0.59 | 0.39 | 50% | 398 | 11.713 | 50% | 1488 | 22.432 | 100% | 6023 | 34.364 | 100% | 2855 | 26.609 | 0% | 811 | 22.446 | 75% | 395 | 11.149 | 0% | 268 | 2.711 | 50% | 943 | 25.657 | 25% | 969 | 11.476 | 100% | 3544 | 31.817 | 100% | 618 | 2.904 |
| Acciona SA 2022 | 0% | 0.66 | 0.39 | 100% | 507 | 9.51 | 50% | 1135 | 12.572 | 0% | 4317 | 99.14 | 100% | 2482 | 41.624 | 100% | 574 | 16.211 | 75% | 436 | 12.657 | 0% | 207 | 3.06 | 50% | 813 | 20.911 | 50% | 979 | 9.946 | 100% | 1878 | 17.654 | 100% | 658 | 2.972 |
| Fortum Oyj 2022 | 0% | 0.43 | 0.36 | 50% | 585 | 13.301 | 50% | 1449 | 29.197 | 0% | 14947 | 218.634 | 100% | 1528 | 36.975 | 0% | 771 | 26.45 | 50% | 338 | 7.96 | 0% | 686 | 5.999 | 50% | 826 | 22.896 | 25% | 988 | 11.574 | 100% | 3538 | 33.55 | 50% | 619 | 2.907 |
| VERBUND AG 2022 | 0% | 0.55 | 0.33 | 50% | 488 | 15.056 | 50% | 1171 | 21.185 | 100% | 3671 | 62.844 | 100% | 1164 | 15.946 | 100% | 603 | 13.566 | 50% | 418 | 9.363 | 25% | 492 | 3.936 | 50% | 861 | 22.176 | 50% | 1336 | 13.674 | 0% | 1536 | 15.452 | 25% | 513 | 2.524 |
| ITALGAS S.P.A. 2022 | 0% | 0.80 | 0.25 | 50% | 643 | 18.404 | 75% | 1489 | 26.587 | 100% | 8871 | 58.49 | 100% | 2200 | 32.012 | 100% | 618 | 15.868 | 50% | 378 | 9.832 | 100% | 727 | 6.008 | 50% | 963 | 24.248 | 50% | 1038 | 11.453 | 100% | 1221 | 12.926 | 100% | 552 | 2.569 |
| ACEA S.P.A. 2022 | 25% | 0.59 | 0.30 | 50% | 668 | 20.22 | 50% | 872 | 8.741 | 100% | 6498 | 47.269 | 100% | 1067 | 14.277 | 50% | 743 | 20.692 | 75% | 320 | 8.013 | 25% | 843 | 6.935 | 50% | 691 | 19.405 | 25% | 1581 | 17.737 | 100% | 3295 | 27.638 | 25% | 544 | 2.52 |
| REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022 | 25% | 0.70 | 0.40 | 0% | 540 | 12.812 | 100% | 1058 | 13.329 | 100% | 19800 | 151.713 | 100% | 1867 | 17.308 | 100% | 623 | 15.87 | 75% | 519 | 12.595 | 0% | 959 | 7.138 | 50% | 811 | 22.268 | 50% | 1119 | 12.039 | 100% | 1508 | 15.446 | 100% | 493 | 2.363 |
| HERA S.P.A. 2022 | 25% | 0.61 | 0.38 | 50% | 597 | 18.419 | 50% | 1854 | 19.927 | 100% | 4289 | 96.521 | 100% | 1781 | 24.856 | 50% | 548 | 11.904 | 75% | 387 | 10.882 | 0% | 618 | 4.898 | 50% | 767 | 21.006 | 0% | 1038 | 10.637 | 100% | 3576 | 29.195 | 100% | 530 | 2.396 |
| VINCI 2022 | 0% | 0.59 | 0.39 | 100% | 518 | 20.299 | 50% | 1282 | 13.785 | 0% | 9470 | 171.524 | 100% | 1050 | 13.85 | 50% | 643 | 17.962 | 75% | 431 | 10.38 | 0% | 688 | 8.814 | 50% | 928 | 23.562 | 25% | 851 | 9.305 | 100% | 2278 | 23.679 | 100% | 651 | 2.721 |
| BOUYGUES 2022 | 0% | 0.75 | 0.32 | 100% | 546 | 16.902 | 75% | 1845 | 24.257 | 100% | 6463 | 80.014 | 0% | 1996 | 33.846 | 50% | 625 | 15.073 | 75% | 309 | 7.876 | 100% | 523 | 4.594 | 75% | 934 | 25.833 | 50% | 1052 | 11.127 | 100% | 2157 | 21.184 | 100% | 540 | 2.435 |
| FERROVIAL SA 2022 | 0% | 0.61 | 0.26 | 50% | 584 | 19.13 | 100% | 1812 | 16.987 | 100% | 4014 | 79.038 | 50% | 1890 | 32.895 | 50% | 666 | 25.155 | 50% | 406 | 10.931 | 25% | 16 | 1.401 | 50% | 960 | 24.959 | 50% | 1056 | 11.305 | 100% | 3075 | 27.801 | 50% | 635 | 3.139 |
| Failures | Mean Absolute Error | Mean Bias Error | Root Mean Squared Error | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 0.427 | 0.406 | 0.503 | 442.12 | 18.42 | 5.23 | 0.77 | 0.03 | 0.01 |
| Kimi K2.6 | 0 | 0.375 | 0.354 | 0.473 | 565.08 | 23.54 | 13.77 | 0.55 | 0.02 | 0.01 |
| MiniMax M2.7 | 0 | 0.448 | 0.448 | 0.589 | 2234.56 | 93.11 | 57.48 | 0.32 | 0.01 | 0.01 |
| Qwen 3.6 Plus | 0 | 0.531 | 0.469 | 0.643 | 702.33 | 29.26 | 11.62 | 0.41 | 0.02 | 0.00 |
| GLM 5.1 | 0 | 0.427 | 0.406 | 0.538 | 424.72 | 17.70 | 4.73 | 0.66 | 0.03 | 0.01 |
| GPT-5.5 | 0 | 0.396 | 0.354 | 0.462 | 243.24 | 10.13 | 1.63 | 2.36 | 0.10 | 0.03 |
| GPT-5 nano | 0 | 0.323 | 0.031 | 0.436 | 118.91 | 4.95 | 2.08 | 0.03 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 0.333 | 0.312 | 0.375 | 570.51 | 23.77 | 2.73 | 2.88 | 0.12 | 0.03 |
| Claude Haiku 4.5 | 0 | 0.250 | 0.083 | 0.323 | 278.77 | 11.62 | 1.76 | 0.60 | 0.02 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 0.625 | 0.625 | 0.703 | 664.75 | 27.70 | 11.65 | 2.01 | 0.08 | 0.03 |
| Gemini 3.1 Flash-Lite Preview | 0 | 0.510 | 0.448 | 0.610 | 65.92 | 2.75 | 0.25 | 0.16 | 0.01 | 0.00 |
| Ground truth priorities | Ground truth order | LLM dist average | LLM dist std. dev. | DeepSeek-V4-Pro | Distance | Out tokens | Time | Kimi K2.6 | Distance | Out tokens | Time | MiniMax M2.7 | Distance | Out tokens | Time | Qwen 3.6 Plus | Distance | Out tokens | Time | GLM 5.1 | Distance | Out tokens | Time | GPT-5.5 | Distance | Out tokens | Time | GPT-5 nano | Distance | Out tokens | Time | Claude Opus 4.6 | Distance | Out tokens | Time | Claude Haiku 4.5 | Distance | Out tokens | Time | Gemini 3.1 Pro Preview | Distance | Out tokens | Time | Gemini 3.1 Flash-Lite Preview | Distance | Out tokens | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ELECTRIC - "TERNA - - IBERDROL 2022 | 11, 8, 3 | C,B,A | 0.73 | 0.65 | C,B,A | 0 | 691 | 21.587 | C,A,B | 1 | 1174 | 22.838 | C,B,A | 0 | 6977 | 71.368 | B,C,A | 1 | 1118 | 54.67 | B,C,A | 1 | 617 | 20.005 | C,A,B | 1 | 568 | 13.526 | C,A,B | 1 | 17 | 3.836 | C,A,B | 1 | 1089 | 30.264 | C,B,A | 0 | 975 | 11.953 | A,C,B | 2 | 3149 | 30.46 | C,B,A | 0 | 372 | 2.563 |
| ENI S.P. - TenneT H - REDEIA C 2022 | 12, 7, 4 | C,B,A | 2.09 | 0.83 | A,B,C | 3 | 670 | 45.462 | A,C,B | 2 | 989 | 18.762 | A,C,B | 2 | 2448 | 89.993 | A,B,C | 3 | 1169 | 105.613 | B,C,A | 1 | 564 | 19.804 | B,C,A | 1 | 784 | 15.411 | A,B,C | 3 | 17 | 2.374 | B,C,A | 1 | 1104 | 36.058 | A,C,B | 2 | 732 | 8.498 | A,B,C | 3 | 4227 | 40.118 | A,C,B | 2 | 520 | 3.447 |
| VEOLIA E - ELECTRIC - REDEIA C 2022 | 5, 11, 4 | C,A,B | 1.27 | 1.27 | C,A,B | 0 | 457 | 16.123 | A,C,B | 1 | 863 | 17.957 | C,A,B | 0 | 6622 | 99.041 | C,A,B | 0 | 1253 | 28.912 | B,A,C | 3 | 581 | 11.212 | B,A,C | 3 | 496 | 13.779 | A,B,C | 2 | 17 | 2.185 | B,A,C | 3 | 1134 | 32.934 | A,C,B | 1 | 1414 | 14.535 | A,C,B | 1 | 5137 | 57.352 | C,A,B | 0 | 425 | 3.106 |
| "TERNA - - A2A ENER - REDEIA C 2022 | 8, 9, 4 | C,A,B | 1.09 | 0.70 | C,A,B | 0 | 775 | 26.161 | A,C,B | 1 | 892 | 14.103 | A,C,B | 1 | 6102 | 74.77 | C,A,B | 0 | 2837 | 51.867 | A,C,B | 1 | 473 | 19.374 | B,C,A | 2 | 461 | 12.417 | A,B,C | 2 | 18 | 2.07 | B,C,A | 2 | 944 | 26.304 | A,C,B | 1 | 967 | 11.918 | A,C,B | 1 | 7494 | 66.231 | A,C,B | 1 | 436 | 3.191 |
| A2A ENER - ENEL - S - "TERNA - 2022 | 9, 1, 8 | B,C,A | 0.36 | 0.50 | C,B,A | 1 | 531 | 23.337 | C,B,A | 1 | 1501 | 24.007 | B,C,A | 0 | 3463 | 47.329 | B,C,A | 0 | 1004 | 48.729 | B,C,A | 0 | 602 | 19.038 | B,C,A | 0 | 528 | 11.101 | B,A,C | 1 | 17 | 2.227 | C,B,A | 1 | 1170 | 31.79 | B,C,A | 0 | 768 | 9.744 | B,C,A | 0 | 2140 | 23.342 | B,C,A | 0 | 403 | 2.997 |
| EDP, S.A - ELECTRIC - VEOLIA E 2022 | 2, 11, 5 | A,C,B | 1.36 | 0.92 | C,A,B | 1 | 898 | 25.123 | C,A,B | 1 | 1198 | 24.16 | C,A,B | 1 | 3992 | 69.172 | C,A,B | 1 | 1825 | 258.216 | C,A,B | 1 | 599 | 13.533 | B,C,A | 3 | 431 | 13.231 | C,B,A | 2 | 17 | 2.323 | C,A,B | 1 | 1143 | 32.567 | A,C,B | 0 | 943 | 10.533 | B,C,A | 3 | 3605 | 36.645 | C,A,B | 1 | 415 | 3.428 |
| EDP, S.A - A2A ENER - TenneT H 2022 | 2, 9, 7 | A,C,B | 0.82 | 0.40 | C,A,B | 1 | 627 | 18.758 | A,B,C | 1 | 893 | 14.346 | A,B,C | 1 | 4844 | 62.812 | C,A,B | 1 | 1094 | 123.56 | A,C,B | 0 | 639 | 19.406 | C,A,B | 1 | 574 | 10.97 | C,A,B | 1 | 17 | 1.792 | A,C,B | 0 | 1054 | 29.401 | A,B,C | 1 | 1182 | 13.29 | C,A,B | 1 | 2656 | 29.07 | C,A,B | 1 | 488 | 3.101 |
| ØRSTED A - ENEL - S - A2A ENER 2022 | 6, 1, 9 | B,A,C | 0.73 | 0.47 | A,B,C | 1 | 355 | 15.113 | A,B,C | 1 | 999 | 20.205 | A,B,C | 1 | 10040 | 106.747 | A,B,C | 1 | 990 | 59.772 | B,A,C | 0 | 488 | 13.562 | B,A,C | 0 | 545 | 11.219 | B,A,C | 0 | 17 | 2.898 | A,B,C | 1 | 932 | 23.561 | A,B,C | 1 | 915 | 11.072 | A,B,C | 1 | 3211 | 33.735 | A,B,C | 1 | 467 | 3.38 |
| REDEIA C - ELECTRIC - "TERNA - 2022 | 4, 11, 8 | A,C,B | 1.45 | 1.13 | B,C,A | 3 | 429 | 18.748 | C,A,B | 1 | 1172 | 21.571 | C,A,B | 1 | 2306 | 117.46 | A,C,B | 0 | 1332 | 181.231 | B,C,A | 3 | 560 | 17.991 | C,A,B | 1 | 493 | 12.643 | B,A,C | 2 | 17 | 2.055 | C,A,B | 1 | 1168 | 32.927 | A,C,B | 0 | 1050 | 11.437 | B,C,A | 3 | 3019 | 30.976 | C,A,B | 1 | 414 | 2.763 |
| ENEL - S - ELECTRIC - VEOLIA E 2022 | 1, 11, 5 | A,C,B | 1.27 | 0.90 | B,A,C | 2 | 1066 | 30.014 | A,C,B | 0 | 1428 | 22.518 | A,C,B | 0 | 4653 | 136.139 | B,A,C | 2 | 1100 | 46.405 | C,B,A | 2 | 724 | 20.748 | B,A,C | 2 | 631 | 11.963 | C,B,A | 2 | 17 | 2.507 | C,A,B | 1 | 1019 | 28.498 | A,C,B | 0 | 836 | 9.599 | B,A,C | 2 | 2993 | 27.689 | C,A,B | 1 | 441 | 3.393 |
| TenneT H - ENI S.P. - ENEL - S 2022 | 7, 12, 1 | C,A,B | 0.91 | 0.83 | A,C,B | 1 | 676 | 28.69 | C,B,A | 1 | 993 | 22.559 | C,A,B | 0 | 7906 | 146.289 | A,B,C | 2 | 1361 | 92.966 | C,A,B | 0 | 607 | 22.778 | C,A,B | 0 | 497 | 12.563 | B,C,A | 2 | 516 | 7.161 | A,C,B | 1 | 1077 | 29.202 | A,C,B | 1 | 799 | 9.915 | C,A,B | 0 | 6575 | 60.725 | B,C,A | 2 | 459 | 3.552 |
| TenneT H - REDEIA C - ENGIE 2022 | 7, 4, 10 | B,A,C | 1.55 | 0.93 | A,C,B | 2 | 377 | 22.778 | A,C,B | 2 | 1161 | 52.416 | B,C,A | 1 | 4530 | 185.779 | B,A,C | 0 | 1076 | 46.068 | A,C,B | 2 | 656 | 16.594 | A,C,B | 2 | 397 | 8.901 | A,B,C | 1 | 17 | 1.74 | A,C,B | 2 | 1053 | 35.391 | B,A,C | 0 | 1154 | 14.707 | C,A,B | 3 | 2955 | 27.373 | A,C,B | 2 | 532 | 2.949 |
| Failures | Average Distance | Time (total) | Time (average) | Time (std. dev.) | Cost (total) | Cost (average) | Cost (std. dev.) | |
|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Pro | 0 | 1.25 | 291.89 | 24.32 | 8.13 | 1.17 | 0.10 | 0.02 |
| Kimi K2.6 | 0 | 1.08 | 275.44 | 22.95 | 9.89 | 0.68 | 0.06 | 0.01 |
| MiniMax M2.7 | 0 | 0.67 | 1206.90 | 100.57 | 40.25 | 0.27 | 0.02 | 0.00 |
| Qwen 3.6 Plus | 0 | 0.92 | 1098.01 | 91.50 | 67.99 | 0.49 | 0.04 | 0.01 |
| GLM 5.1 | 0 | 1.17 | 214.04 | 17.84 | 3.44 | 0.97 | 0.08 | 0.01 |
| GPT-5.5 | 0 | 1.33 | 147.72 | 12.31 | 1.66 | 3.46 | 0.29 | 0.05 |
| GPT-5 nano | 0 | 1.58 | 33.17 | 2.76 | 1.49 | 0.03 | 0.00 | 0.00 |
| Claude Opus 4.6 | 0 | 1.25 | 368.90 | 30.74 | 3.63 | 4.02 | 0.34 | 0.06 |
| Claude Haiku 4.5 | 0 | 0.58 | 137.20 | 11.43 | 1.96 | 0.80 | 0.07 | 0.01 |
| Gemini 3.1 Pro Preview | 0 | 1.67 | 463.72 | 38.64 | 14.55 | 2.27 | 0.19 | 0.03 |
| Gemini 3.1 Flash-Lite Preview | 0 | 1.00 | 37.87 | 3.16 | 0.30 | 0.22 | 0.02 | 0.00 |