Predictions in Energy Economics – Which Error Metrics Are Suitable?
Accurate predictions play an increasingly important role in the energy sector, as the volatility of renewable energy presents challenges to the energy system. To avoid bottlenecks and network overload, the flexibilities within the energy system need to be effectively utilized. This requires not only accurate predictions for energy production, demand, and prices but also for the flexibilities themselves. It is crucial to predict how long an electric vehicle will remain connected to utilize its flexibility optimally or when the heat from a heat pump will be needed to use it at the right time. In this regard, one can use supervised machine learning methods (among other applications in the energy industry).
Evaluating the accuracy of predictions is essential in this context. Specific metrics are used to describe the deviations between forecasts, and actual events and to estimate future deviations. Different metrics are suitable for various accuracy assessments. In the following document, we provide an overview of some metrics, their advantages and disadvantages, and their specific application areas. There are two types of tasks that a prediction can fulfill: regression and classification. Regression involves predicting continuous values, while classification involves predicting group memberships. In a subsequent article, we will discuss the selection of the appropriate method for time series forecasting.
Summary
Selecting the most appropriate metrics for evaluating predictions in energy economics requires a thoughtful analysis of the prediction task, evaluation criteria, scale considerations, interpretability, and the impact of features. This overview of metrics presents the most commonly utilized ones, highlighting their strengths and weaknesses. It serves as a valuable starting point to help select the most appropriate metric for specific prediction needs.
More Information
- Grundlagen künstlicher Intelligenz und Machine Learning in der Energiewirtschaft
- Anwendungsfälle von Supervised Machine Learning in der Energiewirtschaft
- Supervised Machine Learning
Literature
[1] Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media.
[2] Brockwell, P. J., & Davis, R. A. (2016). Introduction to Time Series and Forecasting. Springer.
[3] Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: Principles and Practice (2nd ed.). OTexts.
[4] R. Hyndman (2006). Another look at forecast accuracy metrics for intermittent demand. Foresight: The International Journal of Applied Forecasting, vol. 4, pp. 43–46.
[5] Makridakis, S. (1993). Accuracy measures: theoretical and practical concerns. International Journal of Forecasting, 9(4), 527-529.
[6] Huber, P.J. (1964). Robust Estimation of a Location Parameter. The Annals of Mathematical Statistics, 35(1), 73-101.
[7] Koenker, R., & Bassett, G. (1978). Regression quantiles. Econometrica, 46(1), 33-50.
[8] Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4), 427-437.
[9] Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
[10] Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1-3.
