Empowering the Energy Sector: Utilizing CKAN and PortalJS to Build a Transparency and Innovation-Centric Data Portal for Scottish Southern Electricity Networks

Key facts

Brief summary of the project

Datopian partnered with Scottish Southern Electricity Networks (SSEN), a leading Distribution Network Operator serving over 3.8 million homes and businesses across central southern England and northern Scotland. Our collaborative project centered on building a Data Portal to advance SSEN's strategic goals of innovation, partnership, and a sustainable energy future.

Problem

SSEN faced the challenge of efficiently managing and sharing large volumes of complex data with multiple stakeholders in an increasingly data-driven industry. The lack of a unified data-sharing platform not only complicated efficient data management but also constrained stakeholder engagement, creating a bottleneck for innovation and strategic partnerships.

Need

Recognizing the essential role of data transparency and accessibility, SSEN identified a critical need for a robust Data Portal. This platform would not only democratize access to key energy network datasets but also serve a diverse array of stakeholders, from local governments to energy suppliers and advocates for net-zero carbon emissions. This need coherently aligns with SSEN's overarching objectives, including the promotion of a sustainable energy future.

Solution

We deployed a robust, user-centric Data Portal utilizing CKAN and PortalJS technologies. The platform is equipped with specialized geospatial data capabilities, a customizable and visually engaging user interface, and efficient data management features. It facilitates streamlined data dissemination, allowing both the public and various stakeholders to effortlessly access critical data, thereby elevating SSEN's commitment to transparency and stakeholder engagement.

Main technologies & tools used

Context

Scottish Southern Electricity Networks (SSEN) is a leading Distribution Network Operator (DNO), powering over 3.8 million homes and businesses across central southern England and northern Scotland. As a key player in the energy sector, SSEN has a multifaceted role: from infrastructure maintenance and distribution to renewable energy integration, customer service and support, regulation, and emergency response. The company is at the forefront of fostering sustainable energy practices, driving innovation, and enabling a greener, interconnected energy landscape.

The situation

Current Landscape and Identified Need

In today's data-driven landscape, increasingly influenced by data analytics and digital transformation, SSEN has laid out a strategic framework for open data within its Digital Action Plan for 2023. Given the complex ecosystem in which it operates, which involves multiple stakeholders from various sectors, SSEN recognizes the value of data transparency and accessibility in driving informed decision-making and sustainable practices. Their action plan articulates a strategic approach to responding to Open Data requests from customers and stakeholders. It outlines an efficient methodology aimed at enhancing data accessibility and transparency, thereby empowering more proactive utilization of data to accelerate the transition to net-zero emissions.

Objectives and Features of the Data Portal

SSEN aims to develop a centralized Data Portal with the functionality to serve both public and specific users. Initially, the portal will provide unrestricted access to key energy network datasets, thereby democratizing information. Over the long term, SSEN intends to integrate optional identification features, allowing users to create personalized profiles for an enhanced, customized experience.

Targeting a diverse audience—ranging from local governments and supply chain partners to energy suppliers and proponents of net-zero carbon emissions—the portal aims to become the go-to resource for data-driven insights in the energy sector.

Strategic Goals: Innovation and Partnership

By implementing the Data Portal based on data sharing principles, SSEN seeks to accomplish several strategic goals. Firstly, it aims to drive innovation within the industry by making valuable data accessible for analysis and development. Secondly, it aims to foster partnerships across different segments of the energy sector by providing a common ground for data sharing and collaboration. Finally, SSEN is committed to supporting the transition towards a sustainable energy future, which aligns with broader environmental goals.

The criteria

User Accessibility and Experience

• The portal should be built on a CKAN-based framework to ensure a user-friendly experience.
• The front-end platform must support public users. Management actions can be done via the CKAN backend interface.

Data Types and Formats

• Geospatial Data Support: Essential for providing location-based data, the portal must handle geospatial data formats like GeoJSON and Shapefiles.
• Single-Point Access: The portal should serve as a unified access point for all datasets or data assets published by SSEN, thereby increasing transparency and accountability. []

Data Catalog

• Comprehensive Catalog: The portal needs to offer a thorough catalog of data that includes details about network assets, their location, utilization, and performance metrics.

Insights

• Beyond conventional datasets, the portal should offer "Insights," a category of special data assets accompanied by associated visualizations or Power BI reports. These Insights should be crafted to be shared widely and may originate either from a published dataset or table or stand alone without the underpinning data, when aggregated or summarized data is used to support data classification and enable access to data consumers.

Standard Metadata

• Metadata Standard: The portal should have Dublin Core (also known as the Dublin Core Metadata Element Set or DCMES) and DCAT metadata standards to ensure consistent and standardized metadata descriptions for all data assets within the portal.
• Seamless Portal Interoperability: The portal should support interoperability with various external applications and systems following the Dublin Core and DCAT metadata specifications.

Privacy and Sensitivity

• Some insights may be built without displaying the underlying raw data to protect sensitive information.

This encapsulates SSEN’s emphasis on treating data as a valuable asset, rather than just raw datasets.

The solution

Overview

In alignment with its objective to offer a user-friendly, centralized source of data, SSEN has selected CKAN for their Data Portal. Leveraging PortalJS and CKAN we've created an intuitive, scalable, and dynamic platform that enhances the data accessibility and management experience for both users and administrators.

Geospatial Data

Special attention has been given to the integration of geospatial data assets, including GeoJSON and Shapefiles. This focused approach avoids confusing users with irrelevant maps or broad reports, aligning instead with the expectation for precise spatial data—crucial for understanding SSEN's network assets, usage patterns, and performance metrics.

User-Centric Frontend Design

Our design ethos prioritizes user engagement and navigation. SSEN opted for an aesthetically pleasing front end, featuring a sophisticated color palette aligning with the SSEN brand, ensuring customer familiarity. This design not only ensures visual appeal but also enhances user navigation and engagement on the portal.

Customization, Adaptability, and Extensibility

The front end is highly adaptable, empowering SSEN to customize the platform according to evolving requirements. With the flexibility offered by PortalJS, SSEN has the agility to continuously update features, ensuring it remains a dynamic and responsive part of their data management ecosystem.

Metadata standards

The indispensability of standardized metadata in data portals cannot be overstated. It serves as a universal language for describing and organizing data assets. Metadata standards such as “Dublin Core”, and “DCAT” ensure that data descriptions are clear and understandable for everyone, preventing confusion and enabling seamless integration with various external applications and systems.

Some of the metadata required for the SSEN project was not readily available in CKAN out of the box. Therefore, custom metadata was incorporated to fulfill the requirements. Implementing a CKAN DCAT extension enables access to DCAT standard metadata via a dedicated URL. However, the SSEN project also requires Dublin Core metadata, so the extension has been customized to support it. As a result, the portal now seamlessly supports both Dublin Core and DCAT metadata, facilitating smooth data integration across various applications, in alignment with the Icebreaker One guidance on Dublin Core and DCAT crosswalk.

User-Focused Features

Homepage

The homepage is streamlined for user-friendly navigation, providing quick and visible access to essential categories like ‘Data’, ‘Showcases’, ‘Collections’, and special features, such as ‘Insights’. There’s also a ‘Quick Links’ section which provides immediate access to high-interest areas such as ‘Most Visited Data Assets’, ‘Recently Added Data Assets’, ‘Geospatial Data Assets’, etc. These quick links are strategically positioned for optimal visibility, ensuring that users can instantly navigate to the most relevant and up-to-date information.

Highlights Section

The portal includes a strategically placed 'Highlights' section, showcasing the latest reports related to SSEN’s Digital Strategy, Digital Action Plan, and other most recent and viewed data assets.

Search Page

Users can effortlessly search for data assets through a straightforward interface, complete with efficient filters for sorting and locating the desired datasets.

Showcases Page

A specialized 'Showcases' section exists to feature data assets that have external applications. Whether linked to existing data assets or standing alone, this section is instrumental for sharing reports, dashboards, panels, articles, and other data-centric materials.

Collections Page

The collections page provides a categorized overview of the data assets, designed to enhance both user experience and data asset management. Utilizing clear straightforward labeling, assets are displayed in an aesthetically pleasing grid of vibrant folders, each designated to group data by specific subject matter:

Administrative Capabilities

The solution runs on a CKAN 2.10-based admin instance, empowering administrators with an extensive toolkit for the effective management of user roles, data assets, collections, and showcases, aligning with SSEN's high standards for data governance and operational efficiency.

The outcome

The upcoming launch of SSEN's open data portal marks a significant milestone in the company's sustained efforts to advance data accessibility and transparency. Leveraging the power of CKAN for backend infrastructure and integrating PortalJS with NextJS on the front end, we’ve created a robust, scalable, and user-centric data portal that aligns perfectly with SSEN's strategic objectives.

Data Dissemination and Public Engagement

The new data portal is designed to set a higher standard for information sharing, establishing a new benchmark for transparency and accountability. Through its user-friendly interface, powered by PortalJS, the platform aims to enhance the user experience by simplifying data discovery and access. This user-friendly approach is crucial for engaging a broad audience, including the public, stakeholders, and various interest groups, thereby reinforcing SSEN's position as an industry leader committed to data transparency and stakeholder engagement.

Optimized Data Management

The portal's backend, powered by CKAN, brings unparalleled efficiency to data management tasks. It streamlines the storage, organization, and retrieval of data, effectively reducing administrative overhead while enhancing data accuracy and reliability. The seamless integration of CKAN and PortalJS equips the portal with the flexibility to adapt and scale according to SSEN's evolving needs, further underlining its value to the organization.

The combined strengths of CKAN and PortalJS equip the portal with the flexibility to adapt and scale according to SSEN's evolving needs.

What’s next?

To enhance user engagement and continue to be a leader in energy innovation, the SSEN team is gearing up to introduce smart metering statistics into the portal by February 2024. This represents a key milestone in SSEN's commitment to a digitally transformative user experience, setting the stage for a future where data analytics serve as the linchpin of superior service delivery. By democratizing access to smart metering data, SSEN aims to improve energy supply reliability and enable more informed decision-making regarding energy consumption and conservation.

This strategic update serves dual purposes. On one hand, it advances SSEN's mission to lead in customer-centric energy solutions. On the other, it aligns cohesively with the UK's ambitious net-zero emissions goals, propelling us all toward a more sustainable future. The strategic use of smart meter data will facilitate the integration of renewable energy resources, minimize carbon footprints, and solidify SSEN's role in shaping a more responsible energy ecosystem.

The forthcoming inclusion of smart metering statistics is an emblematic step towards fulfilling SSEN's vision for a data-centric, reliable, and environmentally conscientious energy landscape. SSEN eagerly anticipates the forthcoming enhancements and remains dedicated to advancing toward a brighter, more sustainable energy future.