TY - JOUR
T1 - An Improved and Explainable Electricity Price Forecasting Model via SHAP-based Error Compensation Approach
AU - Heistrene, Leena
AU - Belikov, Juri
AU - Baimel, Dmitry
AU - Katzir, Liran
AU - Machlev, Ram
AU - Levy, Kfir
AU - Mannor, Shie
AU - Levron, Yoash
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2024
Y1 - 2024
N2 - Forecasting errors in power markets, even as small as 1%, can have significant financial implications. However, even high-performance artificial intelligence (AI) based electricity price forecasting (EPF) models have instances when their prediction error is much higher than those shown by mean performance metrics. To date, explainable AI has been used to enhance the model transparency and trustworthiness of AI-based EPF models. However, this paper demonstrates that insights from explainable AI (XAI) techniques can be expanded beyond its primary task of explanatory visualizations. This work presents a XAI-based error compensation approach to improve model performance and identify irregular predictions. The first phase of the proposed approach involves error quantification through a Shapley additive explanations (SHAP) based corrector model that fine-Tunes the base predictor's forecasts. Using this corrector model's SHAP explanations, the proposed approach distinguishes high-Accuracy predictions from lower ones in the second stage. Additionally, these explanations are more simplified than the base model, making them easier for non-expert users like bidding agents. Performance enhancement and insightful user-centric explanations are crucial for real-world scenarios like price spikes during network congestion, high renewable penetration, and fluctuating fuel costs. Case studies discussed here show the efficacy of the proposed approach independent of model architecture, feature combination, or behavioral patterns of electricity prices in different markets.
AB - Forecasting errors in power markets, even as small as 1%, can have significant financial implications. However, even high-performance artificial intelligence (AI) based electricity price forecasting (EPF) models have instances when their prediction error is much higher than those shown by mean performance metrics. To date, explainable AI has been used to enhance the model transparency and trustworthiness of AI-based EPF models. However, this paper demonstrates that insights from explainable AI (XAI) techniques can be expanded beyond its primary task of explanatory visualizations. This work presents a XAI-based error compensation approach to improve model performance and identify irregular predictions. The first phase of the proposed approach involves error quantification through a Shapley additive explanations (SHAP) based corrector model that fine-Tunes the base predictor's forecasts. Using this corrector model's SHAP explanations, the proposed approach distinguishes high-Accuracy predictions from lower ones in the second stage. Additionally, these explanations are more simplified than the base model, making them easier for non-expert users like bidding agents. Performance enhancement and insightful user-centric explanations are crucial for real-world scenarios like price spikes during network congestion, high renewable penetration, and fluctuating fuel costs. Case studies discussed here show the efficacy of the proposed approach independent of model architecture, feature combination, or behavioral patterns of electricity prices in different markets.
KW - CNN
KW - Electricity price forecasting
KW - Explainable artificial intelligence
KW - LSTM
KW - Price spike
KW - SHAP
KW - XAI
KW - XGBoost
UR - http://www.scopus.com/inward/record.url?scp=85203505505&partnerID=8YFLogxK
U2 - 10.1109/TAI.2024.3455313
DO - 10.1109/TAI.2024.3455313
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
AN - SCOPUS:85203505505
JO - IEEE Transactions on Artificial Intelligence
JF - IEEE Transactions on Artificial Intelligence
ER -