Deployment of artificial intelligence and supercomputers will help monitor cloud behaviour to deal with sudden changes to solar generation
UNEARTH PANELS
Discovering the exact location of the distributed solar panels can help grid operators manage the grid and better predict dips in generation JUST A MINUTE
Forecasting software is being developed to help predict cloud movements and the effect this may have on the solar panels below minutes before it happens, allowing for greater flexibility KEY QUOTE
The more confidence we have in our forecasts, the less we’ll have to cover for uncertainty by keeping traditional, more controllable fossil fuel plants ticking over
“There is really no such thing as bad weather, only different kinds of good weather,” said the English writer and philosopher John Ruskin. Operators of electricity grids may not necessarily agree with this sentiment, but new forecasting technologies will help maximise the use of all available wind and sun resources while keeping the grid in balance. Forecasting how much wind and sun will be available has always been of huge interest to renewable energy operators who need to be able to predict revenue. But forecasting is also important to grid operators as levels of variable renewable energy on power networks rise. Currently, grids keep reserve power capacity on standby to respond to unexpected changes in supply and demand so that peaks and troughs in generation do not unbalance the electricity system frequency. These are often provided by expensive gas peaker plants, resulting in additional and unnecessary carbon emissions. By using better weather forecasting technologies, grid operators can replace this backup capacity with more modern and cleaner sources such as battery storage systems or renewable energy generation capacity to keep emissions and costs down. In the UK, the government estimated that unleashing the full potential of smart systems and flexibility in the nation’s energy sector could reduce the costs of managing the system by up to £10 billion a year by 2050. The UK’s National Grid Electricity System Operator (ESO) is targetting a zero-carbon electricity system by 2025. Grid operators need to maximise the use of every kilowatt of renewable energy available in order to decarbonise the grid cost-effectively. More accurate forecasts for weather-dependent renewable energy generation will be vital if it is to achieve this goal. “The more confidence we have in our forecasts, the less we’ll have to cover for uncertainty by keeping traditional, more controllable fossil fuel plants ticking over,” says Carolina Tortora, head of innovation strategy and digital transformation at National Grid ESO. Grid operators are now more aware of the importance of forecasting compared to the 1990s when they “didn’t really care” if wind and solar plants actually produced power or not, because the proportion of electricity coming from such sources was so low, says John Zack at US-based certification company UL, which also specialises in resource measurements. 
HELLO SUNSHINE The penetration of wind and solar generation on national electricity systems is increasing worldwide—it reached a 43% share in the UK in 2020. As a result, research and development into weather forecasting is taking off to meet the demand for increased accuracy and more granular detail. National Grid ESO has been investigating the use of data science and machine learning to develop more accurate forecasting methods. Through this research, it has already improved the accuracy of its predictions for how much solar energy will be generated by 33% through a collaboration with the Alan Turing Institute in 2019. Prior to this work, the ESO forecast solar energy generation using a calculation involving installed solar capacity and solar irradiance. The team of data scientists instead took historic data and around 80 input variables, including temperature and much more granular solar irradiation data. They then used a model that put the data through hundreds of different mathematical pathways to arrive at generation figures, the average of which was taken as the prediction for solar generation. The ESO combined this approach with several other machine learning techniques to calculate its final forecast. It is now further enhancing its solar forecasts with Open Climate Fix, a non-profit start-up co-founded by former DeepMind researcher Jack Kelly. The aim is to develop a first-of-its-kind solar “nowcasting” service for the ESO’s national control room. Nowcasting uses a machine learning model to forecast the near future just minutes or hours ahead rather than days. It has historically been used to predict rainfall but the team will train a machine learning model to read satellite images and understand how and where clouds are moving in relation to solar arrays below. IN THE CLOUD Solar generation is particularly tricky to forecast accurately due to the unpredictable nature of clouds, which can create short-term swings in output, explains Kelly. “Met Office supercomputers are state-of-the-art if you want to know about wind speed tomorrow, but not quite so good at predicting the amount of sunlight over the next few hours,” he says. Models currently in use typically take an hour or two to run, during which time they are blind to any changes taking place, he explains. This is not important for weather features that do not change rapidly such as temperature, but clouds can change a lot over two hours. A machine learning system will instead take just a few minutes to run, he says. This means National Grid ESO can predict a drop in solar generation on the grid and secure sufficient flexibility to maintain stability. THE HUNT FOR SOLAR Another factor affecting the accuracy of solar generation is uncertainty over the location of many solar panels since most are connected to regional networks. A separate National Grid ESO project with the University of Sheffield is mapping Britain’s solar panels installed across thousands of homes, schools, factories and fields to feed into the ESO’s models. Kelly is aiming to complete the research and build a pilot operational forecasting service for the ESO’s control room by January 2023. He believes there is scope for plenty more projects on solar forecasting, particularly by bringing in more data, such as that from the photovoltaic (PV) systems themselves. It should then be possible to calculate how much sunlight will penetrate a particular cloud by using data from PV panels to calibrate the satellite imagery. “We could use that calculation to label a particular cloud as letting through 200W/m2, and then predict how solar power changes as the cloud moves over other panels,” Kelly predicts. 
Under control National Grid ESO wants to better understand weather patterns in order to maximise the UK’s renewables resources
FUTURE FORECASTS The UK’s Met Office has also been heavily involved in the research and development of new forecasting products and services. Some involve several partners and are EU-funded, while others are collaborations with energy companies and the ESO, explains Joana Mendes from the Met Office, the UK’s national weather service. They are also looking at different timescales, from day-ahead, or next season, or even several decades into the future, depending on the users’ needs, she adds. The Met Office’s supercomputer is currently the most powerful in the world dedicated to weather and climate projections. Next year, it will start operating an even more powerful computer to speed up its capability to produce forecasts based on Numerical Weather Prediction, which uses mathematical models of the atmosphere and oceans. “The Met Office model assimilates billions of observations from all over the world, to drive the physical equations of atmospheric dynamics. These are highly non-linear, which requires a lot of computational power. Calculations are done frequently, hourly or more,” Mendes says. The technology will allow the Met Office to process higher volumes of data, but also to store more data—in the order of thousands of petabytes, she adds. One petabyte is equal to one million gigabytes. “We’ll be able to increase the resolution of models, both spatially and in terms of time. It will also allow us to look at smaller-scale weather, which is particularly relevant for very localised phenomena including wind and solar radiation. This will hopefully lead to more accurate forecasts for renewable energy,” she adds. FIGHT CLIMATE CHANGE The EU has also become very interested in how forecasting can help tackle climate change in recent years. There are now some 30 projects funded through Europe, with investment totalling around €100 million, according to professor Alberto Troccoli, co-founder of the World Energy Meteorological Council. “This is unique, in other parts of the world this kind of research is more patchy, they’re watching to see what comes out of our research,” Troccoli says. Troccoli leads the EU-funded SECLI-FIRM project, a 42-month project aiming to demonstrate how the use of improved climate forecasts, up to several months ahead, can aid decision-making processes and outcomes in both the energy and water sectors. The project will promote research advances in the most effective seasonal forecasts for specific applications and encourage industry to take them up in plans and decisions. Specific cases are being studied, such as the impact of high and low wind speeds on energy generation in Spain and Italy, and how heatwaves can impact the grid. “With this forecast, you can look two to three months ahead and see how different elements evolve under different scenarios. It’s an additional tool to allow energy companies to adapt so they can reassess how they produce electricity and in so doing, the prices and constraints, so that production and use can be more efficient,” Troccoli explains. Optimism is high that forecasting technologies will evolve rapidly over the next few years. Until recently, the renewable energy sector has been lacking sufficiently large datasets on which to train machine forecasting models, but this will change as the length of time wind and solar plants have been in operation increases, notes Zack from UL. “It will be evolutionary, not revolutionary—as we get more data, the methods we now have will perform better,” he says. 
OPEN-SOURCE SOFTWARE Kelly is hoping that the work of Open Climate Fix will have a big impact by acting as a bridge between researchers who are keen to solve problems but do not necessarily know what industry needs and industries which may lack the capacity for speculative research. The idea is for Kelly’s team to take ideas from recent academic papers, research them internally and adapt them to solar forecasting issues. Once it has a proven solution, it will allow forecasting companies around the world to integrate its codes into existing products. “Everything we do will be completely open-source—it’s a mechanism by which we can pull together thousands of brains to solve a problem,” he says. Kelly wants to see forecasting companies share the machine learning models, a trend that has proved very effective in other machine learning fields such as natural language processing, he says. “The rate of progress there has been amazing in recent years and that’s partly because that community is really good at sharing papers, code and the learnt parameters that define the models. It feels like that hasn’t happened at all in forecasting yet, but there’s good reason to believe that just by bringing across that cultural shift there’ll be a rapid change in forecasting ability,” he says. •
TEXT
Catherine Early