q1a: S&P Net Debt / EBITDA (x)

ground truthLLM averageLLM std. dev.DeepSeek-V4-ProOut tokensTimeKimi K2.6Out tokensTimeMiniMax M2.7Out tokensTimeQwen 3.6 PlusOut tokensTimeGLM 5.1Out tokensTimeGPT-5.5Out tokensTimeGPT-5 nanoOut tokensTimeClaude Opus 4.6Out tokensTimeClaude Haiku 4.5Out tokensTimeGemini 3.1 Pro PreviewOut tokensTimeGemini 3.1 Flash-Lite PreviewOut tokensTime
ENEL - SPA 20224.844.011.754.21536n/a4.27633n/a4.22710n/a4.356400n/a4.2505n/a3.93267n/a2.618n/a4.41103n/a4.21388n/a4.4716479n/a3.3787623n/a
ENI S.P.A. 20220.950.921.940.51985n/a0.889687n/a0.98913n/a0.486129n/a0.68659n/a0.82460n/a2.618n/a0.511341n/a1.28537n/a0.4814072n/a0.9984605n/a
A2A ENERGIA S.P.A. 20223.473.072.842.86388n/a2.864549n/a1.653063n/a2.861534n/a2.86426n/a2.86173n/a5.318n/a2.86753n/a3.93551n/a2.865789n/a2.86404n/a
"TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 20224.354.171.784.24895.7724.06824671.244.06251513.764.33334633.1634.1851122.7244.333376.6362.8181.3334.3122222.4735.274963.4644.18417735.1894.186283.492
REDEIA CORPORACION SA 20223.213.551.993.13853n/a4.286115n/a3.783703n/a3.783380n/a3.6493n/a3.6266n/a2.0118n/a3.61229n/a4.31529n/a3.1910130n/a3.7846549n/a
EDP, S.A. 2022-8.223.602.753.341591n/a3.3513156n/a4.84366n/a3.945321n/a5.28534n/a3.63398n/a2.818n/a3.351173n/a2.09505n/a3.7317879n/a3.34615n/a
ENGIE 20223.252.681.871.711007n/a2.611881n/a2.658476n/a2.515591n/a3.96589n/a2.63263n/a2.118n/a2.61573n/a3.4720n/a2.634911037n/a2.635495n/a
IBERDROLA SA 20223.913.872.974.44773n/a3.417291n/a4.676837n/a3.915269n/a4.17702n/a3.73231n/a1.6318n/a3.61937n/a4.89399n/a3.2513782n/a4.89580n/a
NATURGY ENERGY GROUP SA 20222.672.440.432.48371n/a2.492072n/a2.42736n/a2.49912n/a2.18383n/a2.48230n/a2.618n/a2.5909n/a2.48514n/a2.559429n/a2.18556n/a
TenneT Holding B.V. 202283.8545.4164.4947.11486n/a51.212279n/a56.946630n/a56.93554n/a56.93457n/a54.1320n/a-1.018n/a58.21422n/a11.2580n/a56.6828128n/a51.24502n/a
VEOLIA ENVIRONNEMENT 20223.923.691.934.64579n/a3.35527n/a3.998912n/a4.224205n/a3.95752n/a3.63346n/a2.618n/a4.31128n/a3.16518n/a3.715071n/a3.0812674n/a
ØRSTED A/S 20222.51.871.162.34729n/a1.731979n/a1.732341n/a1.733666n/a1.471065n/a2.04255n/a2.618n/a2.01068n/a1.32635n/a1.8610016n/a1.73558n/a
ELECTRICITE DE FRANCE 20224.06-22.8162.92-78.17756n/a-26.4616510n/a-14.834422n/a-26.462881n/a-21.9646n/a-26.5255n/a-1.2718n/a-12.91360n/a-12.81653n/a-14.8318360n/a-14.83557n/a
ERG S.P.A. 2022not available4.022.023.97528n/a3.8210935n/a5.797738n/a3.971995n/a4.15568n/a3.82246n/a3.218n/a3.81044n/a3.67534n/a4.018640n/a3.97549n/a
Acciona SA 2022not available2.720.482.66490n/a2.813728n/a2.689573n/a2.877238n/a2.66894n/a2.66335n/a2.618n/a2.71468n/a3.09487n/a2.5410197n/a2.6565672n/a
Fortum Oyj 20221.842.463.772.1622n/a1.5912756n/a1.594323n/a1.593765n/a2.1518n/a1.59234n/a4.218n/a2.1964n/a3.95474n/a1.6310628n/a4.67583n/a
VERBUND AG 20221.61.171.131.12383n/a1.121874n/a0.942360n/a1.12812n/a1.12405n/a1.12215n/a2.218n/a1.12839n/a0.78347n/a1.125768n/a1.12147677n/a
ITALGAS S.P.A. 20225.585.271.255.44490n/a5.433862n/a5.436820n/a5.441358n/a5.44612n/a5.44302n/a4.218n/a5.43689n/a4.85528n/a5.448753n/a5.44612n/a
ACEA S.P.A. 2022not available3.781.353.73379n/a4.129902n/a3.664465n/a3.897203n/a3.89551n/a3.66278n/a2.618n/a4.121200n/a4.12534n/a3.893423632n/a3.8934543n/a
REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 20224.374.332.234.01521n/a4.03926n/a4.034758n/a4.033909n/a4.03468n/a4.03271n/a6.218n/a4.03676n/a5.14443n/a4.036911n/a4.03536n/a
HERA S.P.A. 20224.233.812.393.73438n/a4.428080n/a3.911494n/a3.733259n/a4.34553n/a3.73268n/a1.7618n/a4.421029n/a3.73530n/a4.4510102n/a3.66591n/a
VINCI 20221.891.980.971.93592n/a1.865017n/a1.735137n/a1.794263n/a1.72512n/a1.87305n/a2.818n/a1.9915n/a2.15385n/a1.868585n/a2.135726n/a
BOUYGUES 20223.261.702.171.58361n/a1.649299n/a1.5211963n/a1.64220n/a1.58441n/a1.6233n/a3.118n/a2.211004n/a1.61707n/a2.112535n/a0.21465n/a
FERROVIAL SA 2022not available8.014.907.21465n/a9.28502n/a9.45743n/a7.827698n/a9.11702n/a9.3318n/a4.218n/a7.61167n/a9.03716n/a6.8816442n/a8.4653n/a
FailuresMean Absolute Error (ratio)Mean Bias Error (ratio)Root Mean Squared Error (ratio)Time (total)Time (average)Time (std. dev.)Cost (total)Cost (average)Cost (std. dev.)
DeepSeek-V4-Pro01.253-1.0644.546nannannan0.750.030.01
Kimi K2.600.603-0.4241.723nannannan1.150.050.02
MiniMax M2.700.483-0.2861.124nannannan0.290.010.00
Qwen 3.6 Plus00.609-0.4351.727nannannan0.560.020.01
GLM 5.100.559-0.3361.491nannannan0.640.030.01
GPT-5.500.585-0.4281.724nannannan2.210.090.03
GPT-5 nano00.5950.0250.758nannannan0.020.000.00
Claude Opus 4.600.416-0.2341.003nannannan2.950.120.03
Claude Haiku 4.500.560-0.1561.052nannannan0.520.020.01
Gemini 3.1 Pro Preview00.444-0.2931.107nannannan4.610.190.07
Gemini 3.1 Flash-Lite Preview00.574-0.2191.174nannannan0.150.010.00

q1b: S&P FFO / Net Debt (x)

ground truthLLM averageLLM std. dev.DeepSeek-V4-ProOut tokensTimeKimi K2.6Out tokensTimeMiniMax M2.7Out tokensTimeQwen 3.6 PlusOut tokensTimeGLM 5.1Out tokensTimeGPT-5.5Out tokensTimeGPT-5 nanoOut tokensTimeClaude Opus 4.6Out tokensTimeClaude Haiku 4.5Out tokensTimeGemini 3.1 Pro PreviewOut tokensTimeGemini 3.1 Flash-Lite PreviewOut tokensTime
ENEL - SPA 20220.15670.150.190.238423n/a0.16121536n/a0.126045n/a0.177249n/a0.2378556n/a0.0601321n/a0.17418n/a0.161009n/a0.11512n/a0.1316138n/a0.07483574n/a
ENI S.P.A. 20220.65931.171.840.745426n/a1.116319n/a2.494423n/a0.977328n/a1.33543n/a1.62387n/a0.2718n/a1.041068n/a0.79649n/a1.5710989n/a0.9653595n/a
A2A ENERGIA S.P.A. 20220.23170.280.200.3436374n/a0.317079n/a0.36173n/a0.283899n/a0.283912n/a0.295439n/a0.118n/a0.25906n/a0.31592n/a0.27212268n/a0.2927408n/a
"TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 20220.17240.220.120.1924368.5910.241858471.9450.28706436.910.178537953.1360.23952322.24425.4%3546.4830.18181.1070.1984116.0020.275905.3720.18567147.7580.2327026312.461
REDEIA CORPORACION SA 20220.20630.200.150.154117n/a0.29259n/a0.24287301n/a0.1876678n/a0.2665n/a18.3%259n/a0.118n/a0.2631123n/a0.25497n/a0.212719334n/a0.19465n/a
EDP, S.A. 2022-0.15790.210.160.1761037n/a0.2787226n/a0.2815120n/a0.133143n/a0.132750n/a17.64%395n/a0.22118n/a0.211015n/a0.21459n/a0.2114637n/a0.2498590n/a
ENGIE 20220.24830.400.350.4962686n/a0.3224921n/a0.344230n/a0.56624n/a0.2417945n/a49.6%214n/a0.2218n/a0.44926n/a0.33460n/a0.49623947n/a0.49618670n/a
IBERDROLA SA 20220.19650.190.120.155929n/a0.2113183n/a0.1865377n/a0.195610n/a0.14442n/a25.7%450n/a0.1818n/a0.221278n/a0.16499n/a0.2548799n/a0.1614602n/a
NATURGY ENERGY GROUP SA 20220.28470.320.330.3145415n/a0.3816216n/a0.356031n/a0.2724050n/a0.44629n/a36.7%299n/a0.2118n/a0.421090n/a0.078621n/a0.366510995n/a0.3444575n/a
TenneT Holding B.V. 2022-0.01050.140.460.02977444n/a0.09620853n/a0.098476n/a0.0256110n/a0.128698n/a4.3%278n/a0.2218n/a0.141237n/a0.54538n/a0.10626918n/a0.0828680n/a
VEOLIA ENVIRONNEMENT 20220.19720.230.150.1767876n/a0.228909n/a0.227680n/a0.2144577n/a0.16771n/a18.4%451n/a0.2218n/a0.241167n/a0.26628n/a0.27913350n/a0.3298688n/a
ØRSTED A/S 20220.31240.320.560.242359n/a0.6113112n/a0.411051n/a0.492857n/a0.5791512n/a15.3%364n/a0.1818n/a0.181125n/a0.11551n/a0.2821566n/a0.2531540n/a
ELECTRICITE DE FRANCE 20220.1884-0.090.30-0.056933n/a0.006622299n/a-0.0719990n/a-0.0394745n/a-0.03746n/a-20.2%339n/a-0.2518n/a0.0141143n/a-0.04446n/a-0.05415523789n/a-0.224155772n/a
ERG S.P.A. 2022not available0.260.190.1878050n/a0.31110993n/a0.2322942n/a0.27594n/a0.311775n/a0.368565n/a0.2418n/a0.2521003n/a0.2515485n/a0.32310662n/a0.19092656n/a
Acciona SA 2022not available0.270.160.323554n/a0.32611291n/a0.322913895n/a0.2776551n/a0.2446944n/a0.272272n/a0.1518n/a0.27985n/a0.26597n/a0.2885811n/a0.2717768n/a
Fortum Oyj 20220.45560.430.600.537368n/a0.612427n/a0.2419997n/a0.412705n/a0.59603n/a60.1%238n/a0.1818n/a0.6845n/a0.221016n/a0.66730n/a0.178762n/a
VERBUND AG 20220.51740.751.420.71376n/a0.7119364n/a0.4720456n/a0.73315n/a2.02593n/a60.5%448n/a0.2518n/a0.71820n/a0.57548n/a0.70985654n/a0.7847643n/a
ITALGAS S.P.A. 20220.14560.150.070.1531487n/a0.14715140n/a0.092713n/a0.153020n/a0.15677n/a0.146408n/a0.1818n/a0.146695n/a0.18506n/a0.145212136n/a0.15416774n/a
ACEA S.P.A. 2022not available0.220.110.2314475n/a0.2223732n/a0.28544n/a0.216024n/a0.2215613n/a0.222196n/a0.2918n/a0.19924n/a0.2436n/a0.239321475n/a0.15197523n/a
REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 20220.16860.190.140.178422n/a0.184203n/a0.185870n/a0.195279n/a0.1835990n/a0.312268n/a0.1518n/a0.183880n/a0.14419n/a0.18343898n/a0.17623591n/a
HERA S.P.A. 20220.17330.230.170.219439n/a0.26911916n/a0.272382n/a0.273447n/a0.178457n/a0.207322n/a0.1818n/a0.1751031n/a0.35562n/a0.226710590n/a0.22695641n/a
VINCI 20220.42210.450.300.4548621n/a0.5914822n/a0.486232n/a0.434612n/a0.43701n/a54.4%297n/a0.2218n/a0.481048n/a0.36555n/a0.4617380n/a0.4692626n/a
BOUYGUES 20220.24460.793.390.551235n/a0.5414576n/a0.49666n/a0.552116n/a0.53677n/a55.1%219n/a0.2318n/a0.447845n/a0.4618n/a0.458097n/a4.005593n/a
FERROVIAL SA 2022not available0.140.160.0811816n/a0.159668n/a0.211651n/a0.15315n/a0.091096n/a0.137383n/a0.2518n/a0.14949n/a0.15423n/a0.09213723n/a0.1409627n/a
FailuresMean Absolute Error (ratio)Mean Bias Error (ratio)Root Mean Squared Error (ratio)Time (total)Time (average)Time (std. dev.)Cost (total)Cost (average)Cost (std. dev.)
DeepSeek-V4-Pro00.6330.4221.105nannannan0.790.030.01
Kimi K2.600.9990.8992.402nannannan1.700.070.02
MiniMax M2.701.0670.7942.355nannannan0.410.020.01
Qwen 3.6 Plus00.5740.4260.997nannannan0.620.030.01
GLM 5.101.2401.0703.092nannannan0.660.030.01
GPT-5.500.8730.5351.441nannannan2.250.090.03
GPT-5 nano01.5780.9124.976nannannan0.020.000.00
Claude Opus 4.601.1200.9853.272nannannan2.860.120.03
Claude Haiku 4.503.1312.74011.746nannannan0.520.020.01
Gemini 3.1 Pro Preview01.0310.8752.594nannannan4.700.200.08
Gemini 3.1 Flash-Lite Preview01.7301.3544.061nannannan0.150.010.00

q1c: Leverage Trend (Moody's Adjusted)

ground truthLLM averageLLM std. dev.DeepSeek-V4-ProOut tokensTimeKimi K2.6Out tokensTimeMiniMax M2.7Out tokensTimeQwen 3.6 PlusOut tokensTimeGLM 5.1Out tokensTimeGPT-5.5Out tokensTimeGPT-5 nanoOut tokensTimeClaude Opus 4.6Out tokensTimeClaude Haiku 4.5Out tokensTimeGemini 3.1 Pro PreviewOut tokensTimeGemini 3.1 Flash-Lite PreviewOut tokensTime
ENEL - SPA 2022Deteriorating-0.910.95Deteriorating320n/aDeteriorating951n/aDeteriorating2561n/aDeteriorating1062n/aDeteriorating399n/aDeteriorating387n/aStable238n/aDeteriorating769n/aDeteriorating577n/aDeteriorating2396n/aDeteriorating617n/a
ENI S.P.A. 2022Improving1.000.00Improving547n/aImproving919n/aImproving3877n/aImproving1056n/aImproving530n/aImproving506n/aImproving178n/aImproving752n/aImproving704n/aImproving2047n/aImproving638n/a
A2A ENERGIA S.P.A. 2022Stable-0.552.17Deteriorating217n/aDeteriorating2006n/aStable5601n/aDeteriorating1249n/aDeteriorating655n/aStable344n/aImproving281n/aDeteriorating846n/aStable807n/aDeteriorating3362n/aDeteriorating371n/a
"TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022Improving1.000.00Improving4158.361Improving13409.564Improving353717.623Improving150115.817Improving53234.35Improving4796.799Improving1562.392Improving94117.53Improving8327.82Improving317428.269Improving4912.384
REDEIA CORPORACION SA 2022Stable0.821.28Improving324n/aImproving1245n/aStable3186n/aImproving1309n/aImproving794n/aImproving307n/aStable196n/aImproving826n/aImproving677n/aImproving1989n/aImproving432n/a
EDP, S.A. 2022Improving0.642.56Improving414n/aDeteriorating1836n/aImproving4412n/aImproving1555n/aImproving717n/aImproving338n/aImproving271n/aImproving963n/aImproving839n/aImproving2604n/aDeteriorating507n/a
ENGIE 2022Deteriorating-0.362.92Deteriorating248n/aDeteriorating1169n/aDeteriorating1499n/aDeteriorating1463n/aImproving560n/aImproving324n/aStable210n/aDeteriorating964n/aDeteriorating688n/aImproving2196n/aDeteriorating450n/a
IBERDROLA SA 2022Deteriorating-0.910.95Deteriorating366n/aDeteriorating1201n/aDeteriorating5735n/aDeteriorating1125n/aDeteriorating415n/aDeteriorating373n/aStable264n/aDeteriorating990n/aDeteriorating756n/aDeteriorating8196n/aDeteriorating502n/a
NATURGY ENERGY GROUP SA 2022Improving0.910.95Improving420n/aImproving1434n/aImproving1884n/aImproving1141n/aImproving517n/aImproving360n/aStable16n/aImproving604n/aImproving939n/aImproving2161n/aImproving334n/a
TenneT Holding B.V. 2022NA-1.000.00Deteriorating293n/aDeteriorating1055n/aDeteriorating3940n/aDeteriorating1852n/aDeteriorating727n/aDeteriorating388n/aDeteriorating477n/aDeteriorating806n/aDeteriorating834n/aDeteriorating2597n/aDeteriorating318n/a
VEOLIA ENVIRONNEMENT 2022Deteriorating-0.093.30Improving551n/aImproving1746n/aImproving5686n/aDeteriorating1666n/aDeteriorating551n/aImproving640n/aDeteriorating152n/aDeteriorating1081n/aDeteriorating744n/aImproving6127n/aDeteriorating382n/a
ØRSTED A/S 2022Improving0.362.92Improving285n/aDeteriorating1431n/aImproving1873n/aImproving1694n/aDeteriorating548n/aImproving297n/aStable167n/aDeteriorating822n/aImproving745n/aImproving3414n/aImproving516n/a
ELECTRICITE DE FRANCE 2022Improving-1.000.00Deteriorating356n/aDeteriorating814n/aDeteriorating2476n/aDeteriorating927n/aDeteriorating410n/aDeteriorating286n/aDeteriorating213n/aDeteriorating676n/aDeteriorating755n/aDeteriorating1630n/aDeteriorating356n/a
ERG S.P.A. 2022Improving1.000.00Improving255n/aImproving1205n/aImproving3175n/aImproving952n/aImproving414n/aImproving353n/aImproving366n/aImproving750n/aImproving825n/aImproving2157n/aImproving377n/a
Acciona SA 2022Improving0.821.91Improving399n/aImproving1387n/aImproving4327n/aImproving1677n/aImproving610n/aImproving372n/aImproving777n/aImproving857n/aImproving994n/aImproving3738n/aDeteriorating557n/a
Fortum Oyj 2022Improving0.732.04Improving305n/aImproving2193n/aImproving4910n/aImproving1380n/aImproving494n/aImproving244n/aStable16n/aDeteriorating823n/aImproving576n/aImproving3609n/aImproving468n/a
VERBUND AG 2022Improving0.642.56Improving234n/aDeteriorating844n/aImproving2808n/aImproving1365n/aDeteriorating571n/aImproving328n/aImproving187n/aImproving668n/aImproving681n/aImproving2108n/aImproving398n/a
ITALGAS S.P.A. 2022Deteriorating-0.092.98Improving349n/aDeteriorating1416n/aImproving2083n/aDeteriorating1585n/aImproving395n/aDeteriorating432n/aStable16n/aDeteriorating1069n/aStable890n/aImproving5871n/aDeteriorating529n/a
ACEA S.P.A. 2022Deteriorating-0.642.56Deteriorating351n/aDeteriorating1655n/aDeteriorating4400n/aDeteriorating1853n/aDeteriorating822n/aDeteriorating358n/aImproving580n/aDeteriorating799n/aImproving718n/aDeteriorating3770n/aDeteriorating329n/a
REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022Improving0.910.95Improving310n/aImproving1988n/aImproving2654n/aImproving966n/aImproving546n/aImproving385n/aStable16n/aImproving648n/aImproving584n/aImproving1671n/aImproving426n/a
HERA S.P.A. 2022Deteriorating-1.000.00Deteriorating337n/aDeteriorating1444n/aDeteriorating3579n/aDeteriorating1074n/aDeteriorating1028n/aDeteriorating381n/aDeteriorating329n/aDeteriorating945n/aDeteriorating995n/aDeteriorating2551n/aDeteriorating454n/a
VINCI 2022Improving1.000.00Improving376n/aImproving1031n/aImproving4696n/aImproving1552n/aImproving522n/aImproving335n/aFAIL161n/aImproving707n/aImproving666n/aImproving2805n/aImproving599n/a
BOUYGUES 2022Deteriorating-1.000.00Deteriorating397n/aDeteriorating768n/aDeteriorating1699n/aDeteriorating806n/aDeteriorating761n/aDeteriorating372n/aDeteriorating193n/aDeteriorating870n/aDeteriorating557n/aDeteriorating2282n/aDeteriorating352n/a
FERROVIAL SA 2022Deteriorating-0.183.10Improving505n/aStable1950n/aDeteriorating11788n/aDeteriorating1770n/aDeteriorating614n/aImproving306n/aDeteriorating322n/aDeteriorating957n/aImproving988n/aImproving3912n/aDeteriorating411n/a
FailuresMean Absolute ErrorMean Bias ErrorRoot Mean Squared ErrorTime (total)Time (average)Time (std. dev.)Cost (total)Cost (average)Cost (std. dev.)
DeepSeek-V4-Pro00.430.170.88nannannan0.730.030.01
Kimi K2.600.57-0.221.00nannannan0.520.020.01
MiniMax M2.700.260.090.72nannannan0.230.010.00
Qwen 3.6 Plus00.17-0.090.51nannannan0.370.020.00
GLM 5.100.52-0.090.98nannannan0.640.030.01
GPT-5.500.390.220.86nannannan2.270.090.03
GPT-5 nano10.590.050.88nannannan0.020.000.00
Claude Opus 4.600.35-0.260.78nannannan2.770.120.03
Claude Haiku 4.500.350.170.78nannannan0.550.020.01
Gemini 3.1 Pro Preview00.520.260.98nannannan1.970.080.03
Gemini 3.1 Flash-Lite Preview00.35-0.260.78nannannan0.150.010.00

q2: Suitability

ground truthLLM averageLLM std. dev.DeepSeek-V4-ProOut tokensTimeKimi K2.6Out tokensTimeMiniMax M2.7Out tokensTimeQwen 3.6 PlusOut tokensTimeGLM 5.1Out tokensTimeGPT-5.5Out tokensTimeGPT-5 nanoOut tokensTimeClaude Opus 4.6Out tokensTimeClaude Haiku 4.5Out tokensTimeGemini 3.1 Pro PreviewOut tokensTimeGemini 3.1 Flash-Lite PreviewOut tokensTime
ENEL - SPA 2022Strongly Suitable1.451.65Marginally Suitable771n/aMarginally Suitable878n/aMarginally Suitable4916n/aStrongly Suitable766n/aStrongly Suitable395n/aMarginally Suitable373n/aMarginally Suitable709n/aStrongly Suitable870n/aMarginally Suitable858n/aStrongly Suitable1492n/aStrongly Suitable441n/a
ENI S.P.A. 2022Marginally Suitable1.910.95Strongly Suitable623n/aStrongly Suitable635n/aStrongly Suitable1245n/aStrongly Suitable977n/aStrongly Suitable422n/aStrongly Suitable380n/aMarginally Suitable316n/aStrongly Suitable688n/aStrongly Suitable971n/aStrongly Suitable1507n/aStrongly Suitable355n/a
A2A ENERGIA S.P.A. 2022Marginally Suitable1.451.65Marginally Suitable562n/aMarginally Suitable738n/aMarginally Suitable3283n/aStrongly Suitable894n/aStrongly Suitable546n/aMarginally Suitable473n/aMarginally Suitable378n/aStrongly Suitable868n/aMarginally Suitable855n/aStrongly Suitable1222n/aStrongly Suitable426n/a
"TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 2022Marginally Suitable2.000.00Strongly Suitable3306.551Strongly Suitable7387.068Strongly Suitable13997.356Strongly Suitable6988.091Strongly Suitable48515.173Strongly Suitable4038.047Strongly Suitable2173.137Strongly Suitable67517.286Strongly Suitable114112.52Strongly Suitable135813.201Strongly Suitable2381.798
REDEIA CORPORACION SA 2022Strongly Suitable1.821.28Strongly Suitable603n/aStrongly Suitable605n/aStrongly Suitable3189n/aStrongly Suitable815n/aStrongly Suitable487n/aMarginally Suitable480n/aMarginally Suitable520n/aStrongly Suitable863n/aStrongly Suitable688n/aStrongly Suitable1294n/aStrongly Suitable413n/a
EDP, S.A. 2022Strongly Suitable1.451.65Marginally Suitable544n/aMarginally Suitable888n/aMarginally Suitable2945n/aStrongly Suitable1034n/aStrongly Suitable546n/aMarginally Suitable461n/aMarginally Suitable428n/aStrongly Suitable1069n/aMarginally Suitable834n/aStrongly Suitable1189n/aStrongly Suitable453n/a
ENGIE 2022Strongly Suitable1.452.17Marginally Suitable329n/aMarginally Suitable965n/aStrongly Suitable4520n/aStrongly Suitable749n/aStrongly Suitable444n/aMarginally Suitable483n/aMarginally Suitable488n/aStrongly Suitable773n/aNot Suitable1139n/aStrongly Suitable845n/aStrongly Suitable365n/a
IBERDROLA SA 2022Strongly Suitable1.910.95Strongly Suitable374n/aStrongly Suitable936n/aStrongly Suitable2485n/aStrongly Suitable735n/aStrongly Suitable531n/aStrongly Suitable494n/aMarginally Suitable641n/aStrongly Suitable861n/aStrongly Suitable735n/aStrongly Suitable1436n/aStrongly Suitable333n/a
NATURGY ENERGY GROUP SA 2022Marginally Suitable1.641.60Strongly Suitable637n/aMarginally Suitable972n/aStrongly Suitable2731n/aStrongly Suitable951n/aStrongly Suitable434n/aMarginally Suitable530n/aMarginally Suitable400n/aStrongly Suitable961n/aMarginally Suitable804n/aStrongly Suitable1763n/aStrongly Suitable368n/a
TenneT Holding B.V. 2022Strongly Suitable1.451.65Marginally Suitable719n/aMarginally Suitable746n/aMarginally Suitable6787n/aStrongly Suitable1177n/aStrongly Suitable465n/aMarginally Suitable399n/aMarginally Suitable375n/aStrongly Suitable807n/aMarginally Suitable1307n/aStrongly Suitable956n/aStrongly Suitable375n/a
VEOLIA ENVIRONNEMENT 2022Strongly Suitable1.641.60Marginally Suitable507n/aStrongly Suitable700n/aMarginally Suitable5964n/aStrongly Suitable875n/aStrongly Suitable548n/aMarginally Suitable409n/aMarginally Suitable399n/aStrongly Suitable835n/aStrongly Suitable1184n/aStrongly Suitable1513n/aStrongly Suitable372n/a
ØRSTED A/S 2022Strongly Suitable2.000.00Strongly Suitable524n/aStrongly Suitable834n/aStrongly Suitable3570n/aStrongly Suitable713n/aStrongly Suitable429n/aStrongly Suitable465n/aStrongly Suitable229n/aStrongly Suitable713n/aStrongly Suitable974n/aStrongly Suitable1009n/aStrongly Suitable426n/a
ELECTRICITE DE FRANCE 2022Strongly Suitable0.642.56Not Suitable377n/aNot Suitable780n/aNot Suitable1994n/aMarginally Suitable1087n/aStrongly Suitable557n/aMarginally Suitable326n/aNot Suitable270n/aStrongly Suitable928n/aNot Suitable670n/aNot Suitable2099n/aMarginally Suitable536n/a
ERG S.P.A. 2022Not Suitable1.551.65Strongly Suitable732n/aMarginally Suitable960n/aMarginally Suitable3617n/aStrongly Suitable944n/aStrongly Suitable529n/aMarginally Suitable450n/aMarginally Suitable410n/aStrongly Suitable788n/aMarginally Suitable815n/aStrongly Suitable1992n/aStrongly Suitable402n/a
Acciona SA 2022Not Suitable1.910.95Strongly Suitable326n/aStrongly Suitable893n/aStrongly Suitable3847n/aStrongly Suitable905n/aStrongly Suitable556n/aStrongly Suitable337n/aMarginally Suitable378n/aStrongly Suitable845n/aStrongly Suitable890n/aStrongly Suitable2317n/aStrongly Suitable414n/a
Fortum Oyj 2022Not Suitable1.091.71Not Suitable639n/aMarginally Suitable841n/aMarginally Suitable6166n/aStrongly Suitable1525n/aMarginally Suitable587n/aMarginally Suitable370n/aMarginally Suitable175n/aMarginally Suitable915n/aMarginally Suitable942n/aStrongly Suitable1701n/aMarginally Suitable441n/a
VERBUND AG 2022Not Suitable1.910.95Strongly Suitable227n/aStrongly Suitable831n/aStrongly Suitable4539n/aStrongly Suitable824n/aStrongly Suitable422n/aStrongly Suitable403n/aMarginally Suitable503n/aStrongly Suitable886n/aStrongly Suitable786n/aStrongly Suitable1770n/aStrongly Suitable413n/a
ITALGAS S.P.A. 2022Not Suitable1.451.65Marginally Suitable509n/aMarginally Suitable865n/aMarginally Suitable846n/aStrongly Suitable802n/aStrongly Suitable568n/aMarginally Suitable376n/aMarginally Suitable487n/aStrongly Suitable772n/aMarginally Suitable929n/aStrongly Suitable1617n/aStrongly Suitable393n/a
ACEA S.P.A. 2022Marginally Suitable1.551.65Strongly Suitable591n/aMarginally Suitable855n/aMarginally Suitable4521n/aStrongly Suitable1097n/aStrongly Suitable611n/aMarginally Suitable375n/aMarginally Suitable371n/aStrongly Suitable841n/aMarginally Suitable1036n/aStrongly Suitable1170n/aStrongly Suitable380n/a
REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 2022Marginally Suitable1.451.65Marginally Suitable393n/aMarginally Suitable898n/aMarginally Suitable2967n/aStrongly Suitable964n/aStrongly Suitable584n/aMarginally Suitable430n/aMarginally Suitable358n/aStrongly Suitable808n/aMarginally Suitable1052n/aStrongly Suitable1639n/aStrongly Suitable393n/a
HERA S.P.A. 2022Marginally Suitable1.501.58Marginally Suitable659n/aMarginally Suitable1161n/aMarginally Suitable3362n/aStrongly Suitable1620n/aStrongly Suitable440n/aMarginally Suitable477n/aFAIL12n/aStrongly Suitable905n/aMarginally Suitable1029n/aStrongly Suitable1733n/aStrongly Suitable409n/a
VINCI 2022Not Suitable2.000.00Strongly Suitable281n/aStrongly Suitable827n/aStrongly Suitable2535n/aStrongly Suitable673n/aStrongly Suitable439n/aStrongly Suitable404n/aFAIL135n/aStrongly Suitable858n/aStrongly Suitable690n/aStrongly Suitable1595n/aStrongly Suitable409n/a
BOUYGUES 2022Not Suitable1.641.60Marginally Suitable431n/aMarginally Suitable899n/aStrongly Suitable2370n/aStrongly Suitable713n/aStrongly Suitable413n/aMarginally Suitable360n/aStrongly Suitable435n/aStrongly Suitable864n/aMarginally Suitable887n/aStrongly Suitable1453n/aStrongly Suitable450n/a
FERROVIAL SA 2022Not Suitable1.362.13Marginally Suitable422n/aMarginally Suitable700n/aMarginally Suitable5114n/aMarginally Suitable1236n/aStrongly Suitable600n/aMarginally Suitable493n/aInsufficient Data262n/aStrongly Suitable690n/aStrongly Suitable949n/aStrongly Suitable1600n/aStrongly Suitable443n/a
FailuresMean Absolute ErrorMean Bias ErrorRoot Mean Squared ErrorTime (total)Time (average)Time (std. dev.)Cost (total)Cost (average)Cost (std. dev.)
DeepSeek-V4-Pro00.920.331.15nannannan0.740.030.01
Kimi K2.600.790.291.06nannannan0.470.020.01
MiniMax M2.700.880.381.14nannannan0.220.010.00
Qwen 3.6 Plus00.960.881.24nannannan0.340.010.00
GLM 5.100.920.921.22nannannan0.630.030.01
GPT-5.500.830.251.04nannannan2.310.100.03
GPT-5 nano20.77-0.050.98nannannan0.020.000.00
Claude Opus 4.600.920.921.22nannannan2.780.120.03
Claude Haiku 4.500.880.291.17nannannan0.570.020.01
Gemini 3.1 Pro Preview01.040.881.34nannannan1.490.060.01
Gemini 3.1 Flash-Lite Preview00.960.881.24nannannan0.150.010.00

q3: Size

ground truthLLM averageLLM std. dev.DeepSeek-V4-ProOut tokensTimeKimi K2.6Out tokensTimeMiniMax M2.7Out tokensTimeQwen 3.6 PlusOut tokensTimeGLM 5.1Out tokensTimeGPT-5.5Out tokensTimeGPT-5 nanoOut tokensTimeClaude Opus 4.6Out tokensTimeClaude Haiku 4.5Out tokensTimeGemini 3.1 Pro PreviewOut tokensTimeGemini 3.1 Flash-Lite PreviewOut tokensTime
ENEL - SPA 202250%0.230.2425%5127.71825%100410.33225%1230274.51825%181420.65625%59928.51125%3668.15725%5136.34725%78421.71125%97510.9110%256127.25425%4732.742
ENI S.P.A. 202225%0.200.3225%4248.6325%113413.14625%702949.73625%260728.60625%63827.22125%41010.0280%3784.34625%77020.06425%96210.8180%309832.25725%4512.927
A2A ENERGIA S.P.A. 202225%0.200.3225%3835.01325%9809.51725%518026.04625%108010.25725%55523.99825%4548.6580%1883.66725%73019.84725%104410.7560%204022.2725%5092.799
"TERNA - RETE ELETTRICA NAZIONALE SOCIETA' PER AZIONI" (IN FORMA ABBREVIATA "TERNA S.P.A.") 202225%0.230.2425%6888.51625%91710.69825%765841.43825%195619.83625%59022.29725%3497.62425%3505.02625%72920.55925%111713.3230%362939.34825%3542.373
REDEIA CORPORACION SA 202250%0.180.3725%3038.46125%119914.4620%602837.92725%121313.37125%64026.28125%4438.5825%3884.14625%68718.5825%97410.7320%159619.4590%5073.14
EDP, S.A. 202250%0.180.3725%4316.31225%129916.88225%191817.24725%130314.1060%54021.75125%3427.9620%972.1925%72321.25825%121714.10%149415.99925%3542.303
ENGIE 202250%0.250.3525%4395.69825%95611.57125%197612.82925%180617.8925%56922.9425%3697.79550%2773.91525%68019.15425%102011.8320%371339.39125%5222.936
IBERDROLA SA 202275%0.230.4325%4866.84725%103216.0525%482628.8720%197119.32225%72643.44325%3998.59625%2923.82525%71919.92650%109711.8960%254126.65225%4202.456
NATURGY ENERGY GROUP SA 202225%0.200.3225%4817.55525%106714.1725%271917.88125%128913.51325%50917.81425%4569.9510%2673.29925%71818.14625%101111.9640%270626.22125%4552.395
TenneT Holding B.V. 202250%0.250.3525%5718.89650%9028.90925%333118.91525%168718.72825%64721.75925%3739.36225%1763.01925%76121.16825%146915.50%998189.00125%5062.725
VEOLIA ENVIRONNEMENT 202250%0.250.3525%101312.57525%105316.65825%409430.55525%113512.74125%65423.09625%3588.88350%5586.41425%69521.54325%146816.1120%108112.1125%4652.852
ØRSTED A/S 202250%0.230.2425%4046.20225%105214.3625%376724.17925%176617.55425%49519.27125%3397.25825%1883.78725%83821.60625%8969.2780%339136.3625%3243.742
ELECTRICITE DE FRANCE 202275%0.280.4325%58710.37725%90312.91225%528529.95225%133115.79250%58331.32925%2605.909FAIL1532.84625%89924.66825%103312.330%157516.62450%3852.406
ERG S.P.A. 20220%0.270.4325%4226.57450%9139.1525%364228.4725%124514.60825%65237.16225%4228.83650%761.77225%78121.97225%8669.680%222725.05325%4322.495
Acciona SA 20220%0.250.5025%3656.31650%96311.71925%307526.86525%145314.95725%51527.69425%3809.1160%1462.18225%80321.81425%96710.4250%273429.32850%4233.537
Fortum Oyj 20220%0.230.430%99715.1125%8578.42925%222811.62625%146416.30125%54936.54125%3899.41150%2313.71425%74020.40125%108111.9650%181118.7525%3302.204
VERBUND AG 20220%0.180.3725%6269.02525%77910.25225%553132.5310%142414.86825%55718.47825%4188.5825%2453.76125%65218.54725%112912.6380%161117.1810%4692.559
ITALGAS S.P.A. 20220%0.250.3525%4968.91150%91911.95625%709655.34425%180720.04525%83529.13225%3808.342FAIL1982.91125%71118.31525%93011.1060%217620.44725%4462.531
ACEA S.P.A. 202225%0.230.4325%4557.62750%93610.07125%292919.22925%123513.52525%54316.22725%3949.3190%661.67225%74921.85325%108511.9660%169118.80625%4682.754
REN - REDES ENERGÉTICAS NACIONAIS, SGPS, S.A. 202225%0.230.2425%3937.1925%84810.10825%442026.39925%205719.81125%55817.72625%3648.184FAIL1982.72625%60118.18125%116112.5140%155417.84525%3462.388
HERA S.P.A. 202225%0.200.3225%81813.3825%100713.52425%702739.04625%129213.89525%59317.62525%3517.3160%2012.61525%69320.74125%112912.0130%266026.0225%4602.6
VINCI 20220%0.230.4325%4849.07925%98813.83525%554634.2425%117613.06625%60324.64225%4179.69850%161.21825%72521.5970%91410.9320%190520.89625%4523.449
BOUYGUES 20220%0.180.3725%81612.72825%104412.13825%608136.6920%120713.93225%51218.51325%2886.5930%1172.16625%70320.62925%99510.6660%213227.24925%5112.641
FERROVIAL SA 20220%0.230.2425%4397.27525%88410.18825%361127.56325%109412.88525%52430.6125%3347.952FAIL1642.43525%63618.75725%114613.550%166818.76225%4513.356
FailuresMean Absolute ErrorMean Bias ErrorRoot Mean Squared ErrorTime (total)Time (average)Time (std. dev.)Cost (total)Cost (average)Cost (std. dev.)
DeepSeek-V4-Pro00.188-0.0420.239206.028.582.580.770.030.01
Kimi K2.600.2290.0210.289291.0412.132.470.490.020.01
MiniMax M2.700.208-0.0420.260748.1031.1714.120.260.010.00
Qwen 3.6 Plus00.188-0.0620.260390.2616.263.880.390.020.00
GLM 5.100.198-0.0310.245604.0625.177.000.660.030.01
GPT-5.500.198-0.0310.245202.118.421.022.340.100.03
GPT-5 nano40.250-0.0750.30680.003.331.320.020.000.00
Claude Opus 4.600.198-0.0310.245491.0420.461.592.780.120.03
Claude Haiku 4.500.177-0.0310.222287.0111.961.660.600.020.01
Gemini 3.1 Pro Preview00.281-0.2810.372643.2826.8015.141.820.080.02
Gemini 3.1 Flash-Lite Preview00.198-0.0310.25566.312.760.420.150.010.00