Open Energy Data
Open Energy Data / How SSEN Tackled Massive Energy Data Challenges with PortalJS and CKAN
Learn how Scottish & Southern Electricity Networks partnered with us at Datopian to create a user-first open data portal that handles millions of data points daily, enhances energy management, and supports sustainable goals.
- 3.8M endpoints seamlessly managed
- 1TB+ of data processed every day
- 10K+ datasets published
The Problem
Wrangling endless energy data was draining resources
- Massive Datasets – SSEN needed to manage and query terabytes of energy data daily, including data from 3.8 million endpoints.
- Operational Bottlenecks – Outdated tools struggled with complex data queries, hampering decision-making.
- Security Risks – Sensitive data lacked robust access controls, raising concerns for both administrators and users.
The Solution
A data portal built for efficiency, security, and scale
- PortalJS Frontend: A fast, responsive interface designed to make data exploration intuitive.
- CKAN Backend: A data powerhouse to organize, store, and share energy datasets with precision.
- ETL Workflow: Prefect and BigQuery enable efficient data ingestion and analysis, turning massive data loads into actionable insights.
- Enhanced Security: Extensions like ckanext-noanonaccess protect sensitive data while allowing seamless administrative controls.
The Impact
Making energy data work smarter, not harder
- SSEN’s new portal can process and query over 1TB of data daily without breaking a sweat.
- User satisfaction skyrocketed with faster queries and better access to critical insights.
- The portal supports sustainability initiatives, paving the way for net-zero goals.
- Lightning-Fast Frontend – PortalJS delivers a sleek, intuitive interface that’s built for usability.
- Reliable Data Backbone – CKAN ensures large datasets are stored, organized, and shared efficiently.
- Powerful Add-Ons – Extensions like ckanext-dcat and ckanext-scheming boost interoperability and customization.
- Scalable Infrastructure – BigQuery and Prefect make processing and analyzing huge datasets (over 1TB of data) effortless.
I would like to take this opportunity to extend my heartfelt gratitude for the outstanding support and collaboration your team has offered over the past year. The agility, technical prowess, and steadfast dedication you have demonstrated are truly commendable. Your efforts have been integral to our successes, and I am deeply appreciative. I am hopeful that there will be opportunities for me to engage with future projects and contribute. Thank you once again for your exemplary contribution and for the positive spirit you bring to our work.
Shailesh Kumar, Analytics and Data Architect - SSEN
SSEN Open Data Portal
Designed for flexibility and scalability, PortalJS allows the frontend to evolve independently from the backend, ensuring seamless updates and a user-centric experience.
Explore
| Feature | Before PortalJS | With PortalJS + CKAN |
|---|---|---|
| Data Handling | Slow and unreliable queries | Fast, seamless data processing |
| User Experience | Basic, clunky interface | Modern, responsive design |
| Security | Weak access controls | Advanced security with restricted access |
| Scalability | Struggled with large data | Easily handles 1TB+ daily datasets |
| Customization | Minimal flexibility | Fully customizable to meet SSEN’s needs |
Let’s discuss how PortalJS can meet your needs.
- Institutions worldwide
- 100+
- Saved per year
- $5,000+
- Client satisfaction rate
- 95%