Siddartha Ramakanth, Assistant Professor, CES, ASCI; Victor Vanya, Managing Director, EMA Solutions; Professor Rajkiran Bilolikar, Director, CES, ASCI
The Indian power market is making strides towards next-level transition, which is likely to be completed in the next three to six years. The focus of this transition is to have competitive market mechanisms and institutions centred around digitalisation and IT to bring in scale, speed, transparency and standardisation.
Leading the world against climate change, India is focusing on the mass deployment of clean energy. Solar and wind are the dominating clean energy sources, which come with complexities such as uncontrollable generation and intermittency. With climate change impacting the major weather patterns, solar and wind generation estimation is more complicated. The recent coal crisis triggered by the prolonged monsoon is another effect of climate change. Given the aggressive clean energy transition of the Indian power system, bolstered by the recent commitment at COP26 for achieving net zero carbon emissions by 2070, the power market design has to evolve rapidly to accommodate the transition.
As such, the policy and regulatory environment is driven towards competitive market mechanisms to take up next-generation reforms, by reducing the dependence on long-term power purchase agreements (PPAs), accommodating the uncertainty of renewable energy generation, upgrading load despatch operations to handle high renewable energy penetration and scale, introducing new spot markets like the real-time market (RTM), green term-ahead market (G-TAM), green day-ahead market (G-DAM), and market-based deviation settlement mechanism (DSM), and moving towards general network access (GNA) for transmission allocation and pricing, in addition to market-based economic despatch (MBED), long-duration power exchange-based short-term open access (STOA) contracts, derivatives, market-based ancillary mechanisms and capacity contracts.
Conventionally, the wholesale power market is dominated by a few large buyers and sellers with capacities of less than 100 MW and has limited participation from small commercial and industrial players through open access. Transactions in the market are largely long-term PPA based (90 per cent), while the rest are done through the short-term mode of collective (DAM/RTM), bilateral and DSM segments, which are more competitive and transparent. High renewable energy penetration brought new dynamics like technology cost changes, intermittency and despatchability, geographical and temporal diversity, installation time, smaller capacity, ancillary requirements, and weather dependency. With a conventional human-centric approach, scheduling operations and trading practices are posing as bottlenecks due to the growing scale, speed and interdependence of markets and transactions in the short-term market, reducing dependence on long-term-centric transactions. The adoption of big data analytics and automation has become a prerequisite. Discoms and utilities are increasingly finding it impractical to navigate the changing market by continuing with the traditional operations and trading methods due to the human limitations of scale, speed and reliability.
Big data analytics has gained a prominent role in all sectors in this digital age. With the power sector operating within the 15-minute time block resolution for energy accounting and sometimes 30-second resolution, colossal data is generated on a real-time basis from millions of supply-demand grid connection points through RTUs and ABT/SEM meters and trading and scheduling activities. With the spot and STOA markets gaining a pivotal role in the new evolving market design and the likely launch of electricity derivatives, analytics has become essential for decision-making for trading and planning activities. Analytics is classified into descriptive, predictive and prescriptive analytics, and each has relevance in the power market, especially in planning, scheduling and trading activities.
The power market is a national market, with hundreds of players. The price volume data points are generated on a 15-minute time block basis. DSM is now linked to DAM prices, while STOA bilateral and banking transactions are benchmarked primarily to DAM prices. With the scheduling of long-term plants taking place on a day-ahead basis in sync with DAM and gate closure for revisions linked to RTM gate closure, the PLF of long-term PPA-based plants is now in the realm of spot market liquidity and prices. Investors have started to factor in spot market prices in deciding their generation investments. There is a strong need to understand the past and present influencers of the national market. This understanding must be complemented by descriptive analytics-based tools. This analytics uses data aggregation and data mining to understand and uncover trends quantitatively, thereby gaining valuable insights on the parameters influencing market prices, and taking well-informed and strategic decisions in planning and trading activities.
Descriptive analytics also plays a crucial role in the portfolio and energy management activities of discoms, given the scale of supply demand-side interactions, scheduling, regulatory compliances, market mechanisms. Real-time SCADA and meter data makes it complex to understand and navigate the market-centric operations to make the right decisions and serve the load optimally. A unified descriptive analytics-based platform will help understand and assess the techno-commercial aspects of supply and do a post-facto analysis for corrective actions. Real-time big-data analytics tools help manage a unified analysis of all the interlinked techno-commercial parameters of discoms on a near-real-time basis, eliminating lag and aiding both operations and management in effectively understanding their respective portfolio holistically and taking timely informed decisions. Regulators can also use such descriptive analytics platforms to monitor and assess the performance of regulated entities in this area on a near-real-time basis. The lack of such descriptive analytics platforms with discoms and generators that primarily rely on the experience-based decisions of operations and trading personnel leads to poor trading and planning decisions. Regulators are handicapped with regard to independently monitoring the performance of discoms in trading and planning efficacy and optimality, that too promptly. With scalable best-in-class cloud-based technology tools for data analytics and business intelligence becoming widely available in recent years, even for small-scale applications, power market players can adopt these technology enablers for moving to a big-data-based quantitative trading and planning paradigm.
Predictive analytics is another key evolving focus area for the market transition, given the increasing share of intermittent renewable generation, as well as spot/short-term market-based transactions, which derivatives will aid in the near future. Predictive analytics builds over descriptive analytics, whereby data is deployed for statistical modelling or machine learning (ML) techniques to identify the likelihood of future outcomes based on historical data. ML techniques are increasingly deployed for solar and wind generation forecasting, numerical weather predictions, short-term DAM and RTM price forecasting and short-term demand forecasting. Statistical approaches are prevalent for mid- and long-range DAM and RTM price forecasting and demand forecasting. Numerous predictive analytics approaches are evolving, with varying degrees of success in the Indian power market, which has witnessed considerable improvement on all fronts.
Prescriptive analytics provides actional decision-making inputs with result-based insights. It draws heavily from descriptive and predictive analytics and uses statistical approaches for providing scenario-based outcome analysis. The two key areas of power markets are scheduling/despatch optimisation and derivative contract valuation/market risk assessments. Globally, linear programming-based optimisation tools are used by load serving entities/discoms and system operators to optimise schedules/despatch supply for the least cost and trade optimisation.
Given the increased spot market dependence of utilities and the complexity of gate closure and RTM, system operations and scheduling teams will be left with limited time to process and generate schedules that are market-linked DSM mechanisms techno-commercially compliant, as well as optimised for RTM and DSM. As the NLDC has shown the way in adopting optimisation tools for SCED scheduling and the proposed MBED mechanism, discoms and LDCs should move towards deploying prescriptive analytics tools for scheduling/despatch optimisation and automated trading decision support systems aimed at the RTM.
Automation is another critical area neglected in power market-related areas, where significant strides were made at the NLDC/RLDC end, power exchanges and, to some extent, at the level of traders. With near-real-time trading and scheduling coupled with a considerable amount of data, diversity of suppliers and market mechanisms, collation of various data points and readings for analysis to make near-real-time decisions, coupled with the need to analyse the extraneous market ecosystem (as the power market is mainly national level), there is a need to tap 30-100 data sources for a discom, that too with little or no API support from the ecosystem players. Hence, for effective decision-making, there is a need to do away with manual-based data collation, compilation and communication activities.
With the growing complexity of markets and regulatory compliances, there is a need for automation using IT tools for process automation and communications. Cloud-based automation tools, including robotic process automation, are available even for small-scale applications, which will ease the processing time for the operations teams, and thereby provide scheduling and operations teams with error-free information and analysis for decision-making, and thus enhance the productivity and decision-making capability of the teams multifold.