Birmingham's Open Data Revamp: Civic Transparency through CKAN and PortalJS

Key facts

Brief summary of the project

In collaboration with Datopian, Birmingham City Council, which manages the Birmingham City Observatory, undertook a comprehensive revamp of their open data portal. The updated platform aims to serve as the city's authoritative 'single source of truth' for public data, facilitating enhanced civic transparency, and collaboration.

Problem

The council's previous open data platform, Data Factory, launched in 2014, had become outdated and static, failing to meet the evolving demands of stakeholders and the general populace for reliable and accessible data.

Need

Given Birmingham's complex ecosystem of public, private, and academic stakeholders, and its role as a burgeoning service-based economy, there was a compelling need for a modern, reliable open data platform. The platform needed to catalyze innovation, support transparent governance, and engage the community effectively.

Solution

Datopian implemented a comprehensive solution that included upgrading the CKAN instance from version 2.6 to 2.9 and migrating it to Datopian’s secure cloud infrastructure. A sleek, user-centric front-end was developed using the PortalJS framework. The project also featured the integration of custom data charts on the homepage, SEO automation covering dynamic and static pages, and web analytics capabilities through the open-source Umami tool. These enhancements have revitalized the open data portal, turning it into a state-of-the-art resource for both the public and various institutional stakeholders.

Main technologies & tools used

Context

In a strategic and ambitious step toward enhancing civic transparency and fostering collaboration, Birmingham City Council, the administrative entity overseeing the Birmingham City Observatory, aspired to revitalize its existing open data portal. The objective was to reposition the platform as the city's authoritative 'single source of truth'. This pivotal initiative aims to facilitate both collaboration and innovation by making a wealth of reliable data available for public download and re-use.

Birmingham, located 160 km from London, is the UK's second-largest city with a diverse metropolitan population of 4.3 million. As a cultural and tourism hotspot and host of the 2022 Commonwealth Games, it ranks as the 4th most-visited city in the UK. Historically an epicenter of the Industrial Revolution, Birmingham's modern economy is predominantly service-based, boasting a metropolitan GDP of $121.1 billion. As a cultural, financial, and educational hub, Birmingham is home to a complex ecosystem of stakeholders, including five universities, making it a fertile ground for pioneering open data initiatives.

The situation

Before our collaboration began, Birmingham City Council was already leveraging the capabilities of CKAN for its Data Factory website, designed to facilitate public access to open data resources. Launched in 2014, Data Factory had progressively become stagnant, failing to meet the dynamic needs of contemporary data users. Recognizing an opportunity to further enhance this existing open data infrastructure, Birmingham City Observatory engaged with Datopian to explore advanced CKAN functionalities. After a series of in-depth consultations, we collectively charted a roadmap to optimize both the utility and reach of Birmingham’s open data services. The plan included upgrading their existing instance to the latest version of CKAN and implementing a brand new decoupled custom front end with Datopian's PortalJS framework. This collaborative approach paved the way for transformative advancements in Birmingham's open data capabilities, setting a new standard for public data accessibility and user engagement.

You can read more about the Birmingham City Observatory and its mission here.

The criteria

The dual-focused objective of this project was crystal clear. First, to elevate Birmingham City's open data initiatives by launching a custom-designed open data portal, developed with the latest technologies and best practices. Second, to upgrade the CKAN platform to its latest version, thereby equipping the Birmingham City Observatory team with new and innovative tools for data management and distribution. These strategic upgrades aimed to enhance the Observatory's capability for data dissemination, bringing improved levels of data transparency to a diverse audience - ranging from local citizens and journalists to academic researchers and industry professionals. Additionally, the enhancements would optimize the team's utilization of CKAN capabilities, enabling them to make more effective and strategic use of the features and functionalities available within the platform.

Project Requirements:

• Web Analytics Integration: Equip the Birmingham City Observatory team with analytics tools to gain actionable insights into user engagement and traffic patterns on the open data portal
• SEO Enhancement: Implement advanced SEO strategies to increase the discoverability and reach of Birmingham City's new open data portal, thereby making a larger societal impact
• Data Visualizations: Introduce custom, real-time data charts to offer immediate insights, leveraging the rich datasets available on the portal
• Content Management: Enable the streamlined addition of timely news articles to keep users updated on developments
• User Analytics: Incorporate analytics features that demonstrate how external applications are effectively utilizing the open data provided.
• UI/UX Design: Develop a modern, clean, and user-friendly interface to encourage user navigation and engagement.

The solution

Database Migration and CKAN upgrade

Our first step in this project was to deploy a new and updated CKAN instance in Datopian’s cloud, elevating from version 2.6 to the more robust 2.9. This move included seamless data migration from the legacy CKAN 2.6 environment, ensuring all valuable datasets from the previous version were securely transferred to the new platform. With Datopian's Cloud Services, we guarantee high availability, stringent security, and streamlined technical support, thereby enhancing the operational efficiency of the CKAN instances

Front-end Development

Leveraging best practices in UI/UX design, our team strategically aligned the new open data portal's front end with Birmingham's pre-existing digital branding. By adhering to the previously established visual language, style, and feel of the portal, we aimed to foster an immediate sense of familiarity and ease of navigation for citizens and stakeholders alike. As a result, the portal becomes a more approachable destination for those seeking transparent civic data, effectively widening its user base and impact.

Header and news section on the homepage

Data Visualizations

The portal's homepage incorporates a curated Insights section, showcasing vital city statistics, such as housing distribution across various wards, presented through interactive data charts.

A few of the statistics provided on the homepage

Insights and News

Taking advantage of CKAN's established ecosystem, the Birmingham City Observatory open data portal was enhanced through the integration of specialized 'News' (pages extension) and 'Insight' (showcases extension) pages, both native to the CKAN ecosystem.

The 'Insights' section is particularly geared towards showcasing how the data within the portal is being used by external applications. This section highlights third-party applications that are leveraging the portal's data, providing a valuable resource for data analysts looking for creative applications of the data. Additionally, it serves to inform the general public about various third-party applications, such as PowerBI dashboards, that are making effective use of the data available on the portal.

Insights page

Community Engagement

The Birmingham City Observatory open data portal includes a specialized 'Publishers' section (‘Add your data’), showcasing organizations that are currently active contributors to the platform. This section also functions as an open invitation for other organizations to participate, further enhancing the community-driven aspect of the portal. Elevating community involvement in data sharing and transparency is a core objective of this project, reinforcing the portal's role as a collaborative platform for civic engagement and data-driven decision-making.

Some of the publishers ealy on present in the portal

Web analytics and SEO Optimization

After experiencing increased web traffic due to optimized SEO strategies across all site pages, our web analytics solutions offer administrators deeper insights into their audience. The portal is equipped with Umami, a privacy-respecting open-source alternative to Google Analytics that's hosted on the Datopian Cloud. This tool allows for the collection of 100% anonymous data, ensuring full GDPR compliance. With this setup, administrators can conveniently monitor real-time and historical website traffic, gaining valuable insights into user demographics, user behavior, engagement patterns, etc.

Section of the analytics dashboard displaying basic Information on website usage

The outcome

Upon reaching the milestone of Version 1.0.0, the Birmingham City Observatory celebrated the official launch of the open data portal on November 21, 2022. As of the most recent evaluation, the portal has achieved significant metrics, including:

• The incorporation of more than 200 datasets spanning 17 distinct subject areas
• The participation of six data publishers
• The presentation of eight insights derived from data utilization
• The attraction of over 6,000 unique site visits

What's next?

At present, the Birmingham City Observatory team is actively pursuing community expansion strategies, welcoming new data publishers to the open data platform, and developing specialized training modules to facilitate effective platform usage. Concurrently, collaborative dialogues with the Datopian team are underway to identify and implement value-added features, such as headless CMS integration and a streamlined approval process for resource submissions.

Datopian takes great pride in contributing to this groundbreaking technological initiative, driven by the Birmingham City Council's noble vision and mission. We are committed to fostering a long-term, mutually beneficial relationship with the city to further enhance data transparency and civic engagement.