Identify generator locations

We regularly purchase detailed generator location databases, and these are manually checked and verified.

Collect and
pre-process data

We download accurate generation figures from the appropriate sources, which is done automatically and in real time to ensure only the most up-to-date information is included in the forecasts. The TSO data is then processed with a range of statistical methods to eliminate outliers and further improve the quality of the data set.

Obtain weather data

We work directly with weather companies to process, format and store the relevant weather data.

Aggregate data

In order for our models to work, we have to reduce the weather variables to a single value.

For wind, we:

• optimise the generator set on a forecast-by-forecast basis
• perform a capacity weighted average for all the generators in the optimal subset

This results in a wind speed that is representative of the country.

For solar, we:

• follow the same process, except include a proportion of a country’s rooftop solar within the forecast
• back calculate the data to best fit the generation data
• perform a capacity weighted average for solar farms and rooftop generators

This results in a solar irradiance that is representative of the country.

Apply power generation model

We select the best model based on a variety of error metrics, using a similar approach for the majority of forecasts.

For wind, we use:

• a regression-based statistical fitting on a training data set
• an optimal amount of recent data to forecast on a case-by-case basis
• the aggregated wind speed and our fitting data to calculate the power forecast

For solar, we use:

• a daily generation profile, fitted through cumulative solar irradiance data, and a training data set
• a training data set pulled from the repository of actual generation
• an optimal amount of recent data to forecast on a case-by-case basis
• the aggregated solar irradiance and our fitting data to calculate the power forecast

In operational forecasts, wind forecasts are sent in 1, 3 or 6 hour packets, and solar forecasts are sent in 24 hour packets. These packets are sent within 10 seconds of the data being received.

A similar process applies to ensemble forecasts, except the packets are sent within 20 seconds of the data being received. This is because we run between 30-50 different forecasts to calculate the mean, minimum and maximum values.

Automate delivery

We have our own supercomputing cluster, as well as a full redundancy system as backup, which currently operates at >99% availability. All files are delivered as soon as they are ready via SFTP – typically within seconds.

Assure quality

We benchmark our forecasts against TSO and other relevant sources. Our models are continually evaluated for quality and speed, so if a model goes off track, corrective action can be taken immediately.

Think to the future

We are always researching new ways to improve our offering, including advanced data filtering, better outlier detection, ensemble calibration, statistical fitting, machine learning, and analogue forecasting.