Electricity markets in Europe and elsewhere are in dire need of overhaul although there is little clarity on what to aim for
MARKET HAVOC
High costs and erratic government intervention highlighted the need for European electricity market reform in 2022
MANY MODELS
Europe is not alone in seeking market reform—regulators around the world are pondering new models, with little consensus
KEY QUOTE
The EU approach is interesting—and it will be interesting to see how well it works in practice
Early March 2023 saw a buzz ripple through energy circles as the European Commission prepared to release new proposals on electricity market design. Mention of power purchase agreements, contracts for difference and subsidy limitations in a leaked version of the proposal was “music to my ears”, renewables finance heavyweight Jérôme Guillet wrote on his Substack blog. Lars Stephan of US-based battery firm Fluence praised the draft for having, “The strongest legislative language of the European Commission yet on the need for flexibility and the important role of energy storage for the integration of renewables.” Meanwhile Kristian Ruby of the European electric utility industry body Eurelectric was more cautious over the proposed market reforms. Some of the measures discussed in the run-up to the proposal could be “poison for investor confidence”, he said. Having endured sky-high power bills for much of 2022, few European consumers would doubt that the continent’s electricity markets are due for reform. However, few outside of the power sector could imagine the complexity of the system that delivers electricity to their businesses and homes—and the intricacies of how that energy is priced. Modern electricity provision depends on two interlinked systems. One is the infrastructure that ensures electricity supply always matches demand. The other is a financial market that prices every kilowatt-hour based on the availability of energy at a given point in time. Both systems can be thrown off kilter, affecting the other in the process.
GRID CHALLENGES
Imbalances on the physical network can lead to blackouts, while those on the financial side can result in unexpectedly high prices. Grid operators try to avoid such imbalances, but doing so is becoming more difficult as generation moves from dispatchable thermal plants to variable wind and solar power. Hence the need for a rethink, which in Europe was further emphasised by events in 2022. It would have been a tough year even without Russia’s invasion of Ukraine, analysis by energy insight firm EnAppSys shows. Drought depleted Nordic hydro reserves, forcing Norway to limit its renewable generation and import more energy from its neighbours than usual. The drought also sapped the rivers used for nuclear plant cooling. France’s significant nuclear fleet, already affected by scheduled outages, was forced to scale back operations further with output dropping even lower than it had when the lockdowns induced by the Covid-19 pandemic slashed demand. The issues “caused the nation’s day-ahead prices to rise to record highs,” EnAppSys says, “resulting in a switch in the position of the French market from being one of the largest exporters of power in Europe to being a net importer.”
GAS PRICES
It was against this backdrop that Europe also faced the loss of Russian gas supply, vital for winter electricity production. Prices on the Dutch Title Transfer Facility gas market hit a record of €308.18 per megawatt- hour in August 2022, EnAppSys says. The price of gas is significant because wholesale electricity markets in Europe operate on a marginal pricing model. In this model, different forms of generation enter the system based on how much they cost, with the lowest-cost technologies—typically solar and wind— being given priority. Provided they cover all that is needed, electricity prices stay low. However, if demand goes up then the wholesale price rises to accommodate more expensive forms of generation. When gas hit record levels, then electricity prices in areas burning gas also shot up.
PRICE CAPS
This happened in many European nations, forcing lawmakers to implement arbitrary price caps that Eurelectric’s Ruby slammed in a social media post. “Politically agreed revenue caps are not only different per technology,” Ruby said. “They are also different per country, creating serious distortions of the internal market.” Coming on top of inflation and “other market interventions”, the caps were denting investor confidence, he said, with 2022 European wind turbine orders down 47% on 2021 levels. “A disruptive reform would cloud the horizon further and lead to a multi- year slump in investments,” Ruby wrote. “If the internal electricity market is permanently damaged, that could be the beginning of the end for the entire internal market.” Ruby’s hyperbole may have been coloured by the need to protect the interests of Eurelectric’s utility members, which have plenty to lose in a shakeup of electricity market rules. However, his downbeat view of market conditions was appropriate, with the situation of 2022 highlighting a range of challenges. Facing the loss of Russian gas, the European Commission rushed out the RePowerEU legislative package to speed up renewables permitting and foster investment in clean energy. Yet at a time when Europe should have been racing towards a low-cost, low-carbon economy, governments were unable to fill renewable energy auction quotas and the EU failed to greenlight a single offshore wind farm in the whole of 2022. These problems are not all the result of electricity market design, but the extraordinary events of 2022 highlighted tensions that are growing as European grids move to a model dominated by intermittent distributed generation. Such tensions are not unique to Europe but will ultimately emerge in all established electricity markets as they go through an energy transition.
TWO VISIONS
A central challenge for such markets is how to go from managing a few generation assets that can be turned on and off more or less at will, to dealing with, and rewarding, thousands of market participants delivering intermittent supplies. To complicate matters further, the energy transition is blurring the distinction between buyers and sellers. Electricity markets increasingly need to recognise and reward consumers who, for example, take part in demand response programmes or sell excess generation from distributed energy resources (DERs) such as solar panels. How markets might do this was summarised by American researchers Lorenzo Kristov, Paul De Martini and Jeffrey Taft in a 2016 paper titled “A Tale of Two Visions: Designing a Decentralized Transactive Electric System”. The trio contrasted two views of market control. “One vision is based on a centralised, whole-system optimisation performed by the transmission system operator (TSO), which may also operate wholesale spot markets,” they said. “Under this model, the TSO needs detailed information and visibility into all levels of the system, from the balancing authority area down through the distribution system to the meters on end-use customers and distribution-connected devices.” The other vision, “Involves a decentralised, layered- decomposition optimisation structure, for which optimisation at any given layer of the system only requires visibility to the interface points with the next layers above and below. The choice of which vision to aim for in any jurisdiction will have major implications for specifying the complementary roles and responsibilities of the distribution system operator (DSO) and the TSO.” The centralised option is a logical extension of traditional command-and-control grid models. The alternative layered optimisation model, meanwhile, “Represents a substantial break from today’s models of DER participation in the future grid and the wholesale market,” the researchers said. “Instead of numerous DERs and DER aggregations bidding directly into the wholesale market and being scheduled and dispatched… the DSO would aggregate all DERs within each local distribution area.”
EVOLVING IDEAS
It was unclear at the time the report was written, in 2016, which of these models might be best for low-carbon energy systems, with Kristov, De Martini and Taft noting that either could accommodate applications such as peer-to-peer energy trading. Seven years on, the jury is still out—but some aspects of the discussion have evolved. For instance, one potential stumbling block for a centralised model in previous years was the sheer amount of data that a TSO would need to process if it were trying to keep track of thousands or millions of assets, from solar panels to heat pumps. Nowadays that might not be such a challenge. Electric utilities are already increasing the amount of data they capture and process, says Ben Hertz-Shargel of analyst firm Wood Mackenzie. “It’s not about market redesign or transacted energy markets. It could be for integrated resource planning or visibility into what kind of dark loads [unregistered sources of electricity consumption] they have,” he says. Whatever the motive, the fact is that grid actors have been strengthening their data collection and processing capabilities. At the same time, computing technology has improved. Thanks to what is known as edge computing, a growing share of data processing can be carried out at the device level, without involving a central entity. Central data processors, meanwhile, can store and process vast amounts of information in the cloud. With these advances, managing billions of transactions in a DER-based energy system might not be so hard. In parallel with these developments, the case for layered optimisation has been strengthened by progress in DER aggregation and peer-to-peer networks.
CENTRALISED CONCERNS
These have shown how electricity distribution and trading could be carried out efficiently at the DSO level. As things stand, “TSOs having full control in principle allows for economies of scale,” says Bram Claeys of the Regulatory Assistance Project, a non-governmental body dedicated to accelerating the energy transition. Centralised models could also ease regulation to enable consumer protection, proper service quality and competition in wholesale and retail markets, he says. However, the way TSOs are allowed to recover costs “has proven to lead to a bias towards infrastructure projects as well as a preference for large-scale traditional power plants to manage the grid,” Claeys notes. “Often cheaper alternative resources on the demand side get overlooked or pushed to the back too quickly. Separating ownership from operation would improve the transmission network operation’s transparency,” he says. Alongside the debate around centralised or decentralised market design, Hertz-Shargel of Wood Mackenzie says there is also a growing conversation over how DER asset owners should interact with markets. One school of thought believes the participants should have direct access to markets, including regional and national incentives and service streams, while another says they should go through a utility. The latter assumes utilities are best positioned to account for the value being provided by that customer. “[They] can be a single point of monetisation, rather than [customers] trying to stitch together different programmes,” Hertz-Shargel says. Going straight to markets, “There’s a danger in everyone responding to the system level but wreaking havoc at the local utility level,” he adds.
MARKET DEVELOPMENTS
As it stands, some countries have started to test future market designs to deal with a rising share of renewable DERs. In Australia, there has been considerable debate over the form the electricity market should take. The country has two unconnected electric grids: one for Western Australia and the other, called the National Electricity Market, covering New South Wales, the Australian Capital Territory, Queensland, South Australia, Victoria and Tasmania. Variable renewables contributed 23% of the country’s overall electricity generation in 2021 but the level varies widely by state, from 6% in Northern Territory and 15% in Western Australia to more than 65% in South Australia. The Energy Security Board, which is tasked with shaping the country’s future energy market, has tended to support a centralised grid model where control lies with the national TSO, the Australian Energy Market Operator (AEMO). However, a project to test a centralised market and data platform for DERs has cast doubt over the approach. An interim AEMO report on Victoria’s Project Energy Demand and Generation Exchange found that a decentralised data hub model “has the potential to deliver greater benefits” in terms of scalability, interoperability, security and accessibility. “While Project EDGE is led by AEMO and they really want the solution to be centralised, the evaluation so far is suggesting that a decentralised approach is better,” says Jill Cainey of Melbourne-based consultancy Erne Energy. Paul De Martini, one of the authors of the Tale of Two Visions paper, told an Australian energy forum in February 2023 that grid operators in the United States were facing similar puzzles. “We have a number of states with 2030 targets for decarbonisation that won’t be met unless you start to tap into consumer energy resources,” he said. “There’s a lot of discussions… looking to create new tariffs and structures.”
BOLD DECISIONS
Centralised and decentralised market operations each have their benefits
VANGUARD NATIONS
The state that is furthest along this journey is Hawaii, where each of its six main islands has an independent grid. It is targeting 100% renewable energy by 2045. So far, moves to adapt to rising renewables—which accounted for 19% of electricity generation in 2021— have been managed by the grid operator without the need for a decentralised approach. Whether this would work in larger electricity markets is open to question, Cainey says. “Maybe in smaller markets, the problem is less complex,” she says. “If you have lots of distributed energy but there’s not too much then you can manage it centrally.” Another country that is embracing market reform is the UK. Observers have lauded the country’s progressive moves on electricity market modernisation, including the introduction of features such as a capacity market for reserves that might be needed in the event of a shortfall in renewables. The extent to which the UK system supports decentralisation is evident from the presence of companies such as UrbanChain, which operates a renewables-only peer-to-peer electricity exchange. UrbanChain allows energy consumers to buy electricity directly from producers, without having to go through a wholesale market. It is like Airbnb or Uber, but for energy, says UrbanChain’s Somayeh Taheri. This model removes inefficiencies and sidesteps the problem of marginal pricing in wholesale markets, she says. The concept is proving popular with local authorities, housing associations and manufacturing companies.
EUROPEAN FOCUS
It could also prove seductive for European lawmakers after the price rises of 2022. “Extreme market conditions have revealed the limitations in Europe’s marginal price-based power markets. Some generators made huge margins as consumer bills hit the roof,” said Simon Flowers of Wood Mackenzie in February 2023. “Governments have been forced to subsidise bills temporarily to deal with an affordability crisis, and regulators have intervened with the aim of reforming wholesale markets,” he said. “The challenge is to alight on a means of price setting that’s fair and still incentivises the industry to invest in new generating capacity and supporting technologies,” he added. The complexity of electricity market dynamics means there is little chance of an easy answer. More likely, there will be different answers for different markets, with different levels of centralisation and market control. Ultimately, says Hertz-Shargel of Wood Mackenzie, “It’s hard to see anything as decentralised, truly, even in a highly transactive energy market that operates at a distribution level.” There will always need to be a clearing mechanism, he says. “Even though the individual actors have the freedom to do what they want in response to a price signal. It’s simply a question of how much control does the end customer have, versus the utility or an aggregator, to actively manage customer resources,” Hertz-Shargel adds.
CUSTOMER PARTICIPATION
One unknown in future electricity markets is the extent to which DER and demand response asset owners will want to manage their participation in the market. Based on the effort involved, and the limited return, it is likely that new participants in tomorrow’s electricity markets will be happy to let third parties deal with their energy trading matters. “Managing demand charges, managing transmission charges—an end customer generally can’t do it on their own,” Hertz-Shargel says. “You see residential and commercial customers gaining comfort from being actively managed.” The European Commission’s electricity market design proposal from March 2023 runs to 50 pages, showing that lawmakers have tried to cover most of the angles. Some of the proposals are relatively straightforward, such as further fostering power purchase agreements and contract for difference arrangements that have helped speed up European renewable energy deployment in recent years. Another logical proposal is to encourage trading over shorter intervals than is currently the case, although a suggestion of 30-minute interconnector trades lags the five-minute periods already in use in Australia. More exciting is the focus on flexibility that drew praise from Lars Stephan at Fluence.
FLEXIBLE RESOURCES
Almost all future grid models foresee the need for flexible resources such as energy storage and demand response programmes, but few markets reward these at present. “Smart meters and digitisation are seen as facilitating the participation of customers in flexibility markets,” says Cainey, adding that mobilising flexible resources has been difficult in Australia and the UK. Perhaps the most innovative proposal is for local energy sharing of up to 100 megawatts of capacity, which spells support for peer-to-peer trading. “The EU approach is interesting—and it will be interesting to see how well it works in practice,” says Cainey. The trade body WindEurope welcomed the proposal on its publication in mid-March and cautioned that the need for change should not imply it was good to tinker. “Responsibility for getting this market design right now lies with EU member states in the Council and with the European Parliament,” it said. “They must keep the balance of the Commission’s proposal and send a clear signal to renewables investors that Europe will become an attractive place for investments again.”
TEXT Jason Deign ILLUSTRATIONS Bernardo França