With the smart transformation of the energy industry, the volume, variety and velocity of data has been expanding. As utility companies become larger and more diverse, the data that has to be managed is also getting increasingly complex. Smart meters and smart grids are becoming commonplace in augmenting data for enhanced customer management systems. However, devices like smart meters receive and transmit measurements multiple times a day, sometimes multiple times per minute. This makes the handling of big data a massive challenge for utilities with limited infrastructure. It is critical to leverage these vast sets of data for understanding customer needs and requirements in a better manner and use them to maximise operational efficiency in energy generation and distribution. Thus, the ability to access, analyse and manage vast volumes of data as the information architecture evolves rapidly is vital for the successful operation of utility companies.
Managing big data
The rate at which a utility receives information has increased manyfold with the emergence of smart technologies. Data arrives from smart meters, smart grids, outage management systems, asset tracking systems for consumers, electric grids, power generation, supervisory control and data acquisition systems, alternative energy sources, social media, weather monitoring systems, and wholesale markets. High speed data in large volumes from the internet of things creates big data. Along with handling such large amounts of data, sifting out the relevant bits from a large pool is another tedious task.
Traditionally, utilities have been employing last-generation business intelligence reporting solutions to deal with data. This mainly involves descriptive analyses, and, in most cases, does not help in finding the important information hidden in vast pools of big data. Thus, real-time and predictive analysis tools are required to undertake the complex processes of getting useful data insights. Utilities are now looking to become more proactive in taking up predictive data analysis to adjust their decision-making strategies and adopt smart technologies. By analysing historical data patterns, predictive analytics helps them handle intermittent loads from renewable sources, rapidly changing weather patterns and other grid conditions.
Business analysts use various big data analysis (BDA) solutions, which help utilities optimise their operations in a variety of ways. The ability to analyse and act upon real-time or near-real-time data helps utilities create a competitive advantage in the marketplace, manage resources in a sustainable manner, and promote appropriate service delivery.
Key application areas
BDA helps improve customer insights by understanding their wants, needs and expectations. It also helps in analysing the unstructured data found on social media and in emails with the aim of reducing risks of churn. BDA solutions enable utilities to leverage data from smart grids to improve customer service by providing better visibility and more detailed information regarding individual customer usage patterns. This helps in the development of more customer-centric offerings, such as proactive demand- response programmes and new pricing regimes to manage the rising demand and supply of power for a particular point of time in a day/month/year. This helps in identifying problems and irregularities in customer accounts.
BDA further helps improve the operational efficiency of utilities. Predictive analytics can minimise outages and improve power distribution reliability by anticipating demand and taking the required steps before an outage occurs. Analytics can help identify trends and forecast demand, in addition to identifying theft and fraud using smart meters and developing a power supply strategy for improving network reliability and providing optimal load scheduling. It can also use situational intelligence to identify and pre-empt maintenance issues and asset failures and perform condition-based maintenance; combine smart meter data with other grid data points to highlight a complete transformer load flow for project planning; and sense and assess outages by analysing meter data delivered through advanced metering infrastructure. Predictive customer modelling and customer sentiment analysis from sources like social media can provide insights into customer behaviour. Sensors on equipment and vehicles can pinpoint problems in a quicker manner and enable better logistical support.
BDA solutions can also be used for revenue protection and loss prevention. For instance, they can be used to determine unusual customer usage patterns, such as why a meter signature suddenly appears differently, or for matching meter data consistency with billing data. Usage patterns can also be leveraged to formulate demand-based pricing after taking into account peak usages on the grid for improved revenue and better demand-supply management.
Meter data acquisition and demand response can be carried out through BDA solutions as well. While meter acquisition rates are typically set to four-hour intervals, Hadoop-based architectures can handle much higher data rates. A meter can be sampled every 5 or 15 minutes instead of every four hours to better analyse demand and manage the grid more efficiently. This helps streamline supply (power generation) to meet demand in an effective manner.
BDA also makes it possible to improve financial forecasting. Detailed histories can help determine, measure and track the elasticity of demand and carry out electricity tariff analyses while improving asset optimisation and planning by determining operational efficiency.
The management of risks and threats can also be enhanced through BDA. Utility companies face an unusually high number and variety of risks, including those with serious health, safety and environmental implications. While these cannot be eradicated, an increased amount of varied data can be used to help manage and mitigate them. Detecting unauthorised or illegal access or usage can save companies from the consequences of possibly malicious activities.
Conclusion
Utility companies find themselves at a pivotal juncture, with more data available to them than at any time in the past, facilitating the gathering of more information. They can take advantage of this to invent better business processes and ensure efficiencies by evolving their information architecture in an impactful manner. However, the use of these techniques has been very limited, which limits the ability of utilities to optimise operational efficiency. The key reason is the lack of awareness about the potential of data, which confines them to traditional methods of analysis. In other cases, even if utilities recognise the potential of BDA, financial constraints hold them back. They need to have a clear understanding of the long-term value of the data available to them if they are to adopt strategic investment initiatives.