Back to OverviewSuccess Story: Active power losses forecasting for Swissgrid
Imagine: you have three days to set up an automated intra-day (and day-ahead) forecasting to compete against five other companies, with one goal: to beat your own Swissgrid’s forecasts.
The performance was to be measured for three months, basically starting at the outset of Covid-19 in Europe. Since March till May of 2020, TIM’s forecasts were delivered several times a day. The result: TIM beat the benchmark and other players’ forecasts.
How did TIM manage to stay ahead? And what is the value it can deliver with its approach? Read it in this inspiring success story.
What is TSO?
TSO (Transmission System Operator), and its impact on our everyday life is critical. The challenge for TSOs is their responsibility to keep electricity transmission grid operational 24/7. The fundamental principle of avoiding any disruptions is to keep the grid balanced – the energy generated and fed into the grid must be equal to the energy consumed at any time.
Considering the number of parties connected to the grid and energy transited to and from other countries, and factors influencing consumption as well as generation especially with the rise of renewable sources of energy, power losses on the grid, and other factors is a complicated task.
In Switzerland, the electricity transmission grid is owned by Swissgrid. The grid comprises of 141 substations, is more than 6,700 km in length, transporting electrical energy at a voltage of 380 and 220 kilovolts and is connected to neighbouring countries grids via 41 cross-border lines.
Swissgrid, as well as balancing and monitoring the grid, carries out regular maintenance, upgrades, and expansions to ensure that the grid is always available.
Grid balancing
To keep the electricity grid in a balanced state, the amount of energy generated must be in balance with the energy consumed (sometimes, consumption is called “load”).
Due to the physical nature of materials and components used in all assets and infrastructure, technical power losses need to be considered. Power can be lost due to various reasons; active power losses are lost due to physical conditions, other losses, for example, represent cases of un-metered supply, misuse of supply etc. And so, we can express these principles in a simplified equation like this.
Energy generated = Energy consumed + Active power losses + Other losses
Knowing in advance, and as precisely as possible how much energy will be generated, consumed, and lost in minutes, hours, days, and months ahead can make a significant difference in power grid operations.
TSOs receive forecasts of expected energy generation and consumption ahead of time, as parties connected to the transmission grid, so-called Balancing Responsible Groups (BRG), provide their forecasts in advance; nevertheless, the deviations against the reality that occurs daily must be managed.
If there is a deviation against expected values, corrective actions must be taken as quickly as possible. For example, if the load would suddenly increase, reserves must be activated ASAP. There are reserve mechanisms in place that can quickly ramp up production of energy when needed; however, their capacities are limited. Hence, it may be required to procure energy on intra-day energy markets with expected delivery in the upcoming minutes – hours. Typically, with shorter time comes a higher price.
Active power losses
For physical reasons, transport of electrical energy via electricity transmission grid leads to losses; thus, less power can be withdrawn than is fed into the grid. Losses are generated, for example, through resistance in lines or transformers, and are released as heat. Factors such as load, outside temperature and switching states in the grid have an impact on power loss. The longer a transport line and the lower the voltage, the higher the active power loss.
Swissgrid is responsible for procuring active energy on the electricity market to compensate for transmission losses on the extra-high-voltage grid. Compensation of an active power loss belongs to so-called “ancillary” services.
To ensure the availability of ancillary services, Swissgrid relies on assistance from pre-qualified service providers. Energy for offsetting active power losses is also purchased on the power exchange.
Capacities for active power losses are allocated via tenders in monthly, quarterly, and yearly periods.
1% -7%
These active power losses are proportional to the transported energy and amount to between 1% and 7% of the transmitted energy.
110 MW
The average active power losses in the Swiss transmission grid amount to approx. 110 MW.
Swissgrid’s challenge
For Swissgrid, and all TSO’s in general, the challenge is to foresee (predict) the magnitude of losses in advance to aid clarity into the grid balancing process.
For example, predictions for longer horizons – weeks or months ahead – are necessary to be appropriately calculated as they serve as input for tenders to procure energy for compensation.
Forecasts for short and very short horizons – days, hours, or quarter hours ahead – help with everyday grid operations and are critical to grid balancing. If forecasts are right, eventually they can contribute to decreases in the volume of energy that needs to be dispatched by a reserve mechanism and can be procured with short notice on intra-day energy markets.
Tangent Works, among other companies, was invited to participate in benchmarking a pilot to forecast operational power losses for three months commencing March 2020 – basically the start of Covid-19 in Europe. Using Tangent Works RTInstantML technology, Tangent Works were confident that their engine TIM (Tangent Information Modeler) was capable of coping with such an unprecedented and challenging situation.
Forecasting scenarios
At Swissgrid, active power losses were predicted for intra and day ahead prediction horizons with data sampled on an hourly basis as follows.
Data
A historical dataset with values dating back two years was shared with Tangent Works before the pilot. The dataset contained more than 30 predictors and was sampled on an hourly basis.
During the pilot, Swissgrid provided Tangent Works with data for future forecasting situations with historical values dating back a few days. It contained updated actual values for target variable (until the most recent timestamp) as well as predictors and predicted values for predictors. However, this availability was not uniformed. There were several groups from an availability perspective; some contained values for all points in the prediction horizon, while others were incomplete and were only shared a few hours in advance.
Target variable parameters
Mean: 116
Standard deviation: 3
Disruptions in data
During the process, values for specific predictors were not available in the provided dataset due to technical issues, which resulted in the shape of data appearing different than in any previous occasions. In such situation, other standard machine learning approaches require development of a new model.
Data anomalies
Throughout the project, some of the predictors diverged to extreme values that had not been seen before. Clarification by Swissgrid revealed that it is an error in data, and would need to be treated accordingly, i.e., be instead switched off for a while. TIM can, besides forecasting, also detect anomalies in data, and so, such a situation could be handled by adding an anomaly detection step before the forecasting step.
Deployment timeframe
The period between Tangent Works participation, and the beginning of the first forecast was limited to a few days. Time is required to prepare the interfaces for data exchange, orchestration, pipelines for data transformation, and retention had to be ready, notwithstanding the machine learning models development, packaging, deployment etc. How could this mission be accomplished in such a short space of time?
Covid-19
Covid-19 impacted Europe in March 2020, bringing profound implications in every aspect of our lives and launched an avalanche of challenges for businesses, which lead to structural changes in data.
In the world of machine learning and predictive forecasting, models are built on historical values, learning what to expect in certain conditions (perceived via data). However, during March 2020, and subsequent months, were like no other, especially from a data perspective in terms of training a dataset.
This huge structural change meant a significant challenge to any machine learning solution.
How did TIM overcome these challenges?
To recap the situation:
- Short deploy time available.
- Structural changes in data caused by pandemic starting to spread.
- Unpredictable conditions with data were expected to occur (and they did).
Those challenges set the stage for TIM to shine as TIM is a unique, machine learning solution that was designed to withstand and cope with such challenges.
TIM delivers automatic model building capability that builds models in a fraction of time on demand, regardless of the sudden change in data, prepared to be consumed right away. Due to the high degree of automation that reduces the efforts generally associated with ML operations, TIM is a solution that hyper automates ML operations process, which we refer to as – MLDevOps.
MLDevOps
DevOps discipline transformed the way how applications are developed (dev), deployed and run (ops). With the arrival of ML, it is not an application that is being deployed and used; it is a model as a product of ML phase. Nevertheless, to make it operational, in Dev part, it is still needed to build required functionality thus enable the use of models. So, we arrive at MLDevOps acronym.
ML – The model development and building process is fully automated thanks to Tangent Works (RT)InstantML technology. TIM does not build just one model, to achieve the very best accuracy possible, TIM builds a whole range of models – so-called ModelZOO; for example, each hour of prediction horizon a different model is built to calculate a prediction. TIM derives the best possible features from the given data, and it spares the user the hassle from the hyperparameter optimisation phase.
For every prediction we delivered to Swissgrid, RTInstantML technology developed the most up-to-date models from the latest data. This approach allowed us to gain the best accuracy possible. We could rely on this method because it took just a couple of minutes to train the models and calculate a prediction.
Dev – The development of code that would enable to use models in a production environment is not required. Traditionally, it is necessary to take efforts to develop, build, test, release and deploy an application to serve models, e.g., on a container with a dedicated API endpoint. By design, TIM has an API endpoint already prepared, that means zero effort for us.
Ops – Forecasting in production with TIM is simple. All that was required was to orchestrate API calls to TIM Engine with the data and wait for the models to be built in the background and consume returned predictions.
What was the outcome?
After three months of forecasting, Swissgrid shared evaluation report. It compared their in-house forecasts against forecasts calculated by TIM, or to be specific, errors of their forecasts against errors of TIM’s forecasts.
The results proved that TIM could deliver more accuracy than that of their own forecast for intra-day forecasts and was on par for day-ahead forecasts. TIM delivered the most accurate predictions for intra-day forecasts among the competing companies.
The chart below shows the evolution of cumulative error over time.
Sum of errors per month comparison can be seen in the chart below.
Business Value
The core mission of electricity grid operators is to keep the grid balanced and knowing how much power will be lost at certain hours in the future, affords operational benefit, as failing in doing so has dire financial implications.
Transmission grid operator recovers cost associated with procuring power balancing active power losses via tariffs that are set and published upfront. The tariff for individual ancillary services active power losses covers the costs of compensating active power losses, which are incurred and charged to respective parties.
Energy compensating active power losses is procured via tenders’ month, quarter, and year ahead and delivered solely from Swiss parties. This helps Swissgrid to be prepared to a certain level while still expecting deviations. The situation changes on a minute basis; thus, one month seems to be a very distant future. It is impossible to predict values correctly, on an upfront hourly basis, and it is necessary to act when imbalance on the grid is to occur.
For example, if a grid operator would have predicted more significant losses, more power would be planned for compensating the grid than necessary, thus creating a challenge on balancing.
The gaps flow into the balancing process that considers other gaps between forecast and real values. Depending on result power reserve mechanism is dispatched or procurement of energy on spot markets is needed. Isolating the net impact of better active power loss forecast can be a challenging task; nevertheless, consider the amount – typically prices for energy with short notice delivery on spot markets are very high.
Speed and cost of implementation
Aside from the benefits linked to Swissgrid’s core business process, we can demonstrate benefits related to IT and forecasting operations too. And not only in terms of financials.
Taking the traditional approach of developing hand-built models would require the capacity of Data Scientist(s) who would create deep learning models or other methods, this approach would be necessary during the implementation phase as well as during operations.
TIM brings automated model building capability, so there is no need to invest in model development and re-building at all. Once the data are ready, it is possible to start forecasting immediately. Both methods require having the right data to build the models, but the time needed using TIM is reduced significantly.
From a time perspective, it can take a Data Scientist several weeks to develop (machine learning) models. Typically, such an investment would be the responsibility of the TSO organisation. However. should they choose to outsource this task, they need to consider the fee to hire a data scientist with an average daily rate.
Once a model is developed, it needs to be packaged and deployed on the infrastructure so it can be consumed and connected to the forecasting pipeline to digest input data and be repeatedly used. A typical approach would be to rely on the release of models in a container with an API endpoint. Thus, the capacity of DevOps and/or software engineer needs to be considered. The TSO generally accepts the responsibility of this process; however, if they choose to outsource it (and manage it), they need to consider the significant additional expenses.
TIM is designed with a no-code approach to deployment, so when it comes to using ML models, engineers would only need to take care of sending requests to and processing response from TIM.
From an IT security policy perspective, having ML solution deployed on-premises (i.e. infrastructure inside the company network) could be a requirement. TIM was designed to run, besides in the cloud, on-premises as well. The effort needed by an engineer to make it work is not expected to take more than one working day.
Operations
In production, it is crucial to monitor the performance of the deployed solution.
Questions such as – are all the data that my models require to perform available, has something changed with my data – need to be asked and addressed. They are real-life challenges that can occur that would require building new models. These scenarios translate to increased costs due to lost opportunities that Data Scientists and engineers could be working on to resolve.
TIM builds new models from data automatically with every new forecast, without compromising on accuracy, this has many benefits; for instance, models continue to learn from the latest actual values, even with missing data.
TIM’s capabilities
Below is a list of additional use cases where TIM can offer business value:
- Forecasting energy generation from renewable (and very volatile) sources of energy such as solar, wind, hydro affected by weather conditions.
- Forecasting consumption, or imbalance.
- Change focus on different levels of aggregation – predictions on a national level could be in some scenarios insufficient and having focus on a specific region/area could deliver another benefit.
- What about long-term forecasting, weeks, or months ahead? With TIM it is possible to predict both longer and shorter prediction horizons.
- What if data for some scenario would be sampled on a 15-min or 5-min basis, or even lower? TIM supports all sampling rates, from milliseconds, and can cope equally well with irregularly sampled data.
By adopting TIM, you can unlock the endless potential of time series forecasting for any case you need in your business process in an automated fashion, delivering stable performance and enjoying a significant ROI.
DSOs
DSOs (Distribution System Operators). DSOs play a vital role in delivering electricity as they operate distribution grids which are connected to the transmission grid. Distribution grids connect the smaller electricity generators and most consumers (e.g., households and commercial consumers) to the electricity system and operate on lower voltages than transmission grids (e.g., 110 kV).
About Swissgrid
Swissgrid was founded in 2006 in response to the gradual liberalisation of the Swiss electricity market. Since 2008, the Electricity Supply Act has stipulated that the national grid company must own the transmission grid.
As the national grid company, Swissgrid has been responsible for the operation, safety, and expansion of the 6,700 km-long extra-high-voltage grid since 2009. Swissgrid took over the grid in 2013, which represented an important milestone on the path towards electricity market liberalisation.
Swissgrid employs around 500 people from 20 countries.
As a member of the European Network of Transmission System Operators for Electricity (ENTSO-E), Swissgrid also supports the coordination and grid utilisation activities in the European exchange of electricity.
