-
1
-
2
Collect and
pre-process dataWe download accurate generation figures from the appropriate sources, which is done automatically and in real time to ensure only the most up-to-date information is included in the forecasts. The TSO data is then processed with a range of statistical methods to eliminate outliers and further improve the quality of the data set.
-
3
Obtain weather data
We work directly with weather companies to process, format and store the relevant weather data.
-
4
Aggregate data
In order for our models to work, we have to reduce the weather variables to a single value.
For wind, we:
- optimise the generator set on a forecast-by-forecast basis
- perform a capacity weighted average for all the generators in the optimal subset
This results in a wind speed that is representative of the country.
For solar, we:
- follow the same process, except include a proportion of a country’s rooftop solar within the forecast
- back calculate the data to best fit the generation data
- perform a capacity weighted average for solar farms and rooftop generators
This results in a solar irradiance that is representative of the country.
-
5
Apply power generation model
We select the best model based on a variety of error metrics, using a similar approach for the majority of forecasts.
For wind, we use:
- a regression-based statistical fitting on a training data set
- an optimal amount of recent data to forecast on a case-by-case basis
- the aggregated wind speed and our fitting data to calculate the power forecast
For solar, we use:
- a daily generation profile, fitted through cumulative solar irradiance data, and a training data set
- a training data set pulled from the repository of actual generation
- an optimal amount of recent data to forecast on a case-by-case basis
- the aggregated solar irradiance and our fitting data to calculate the power forecast
In operational forecasts, wind forecasts are sent in 1, 3 or 6 hour packets, and solar forecasts are sent in 24 hour packets. These packets are sent within 10 seconds of the data being received.
A similar process applies to ensemble forecasts, except the packets are sent within 20 seconds of the data being received. This is because we run between 30-50 different forecasts to calculate the mean, minimum and maximum values.
-
6
Automate delivery
We have our own supercomputing cluster, as well as a full redundancy system as backup, which currently operates at >99% availability. All files are delivered as soon as they are ready via SFTP – typically within seconds.
-
7
Assure quality
We benchmark our forecasts against TSO and other relevant sources. Our models are continually evaluated for quality and speed, so if a model goes off track, corrective action can be taken immediately.
-
8
Think to the future
We are always researching new ways to improve our offering, including advanced data filtering, better outlier detection, ensemble calibration, statistical fitting, machine learning, and analogue forecasting.