Custom Automated Reporting Platform for Metropolis
Sam Fairweather
Company Overview
Metropolis is a marketing consultancy for the property industry.
Metropolis are a trusted partner to developers, agents, designers, builders, lenders, owners and managers, with a track record of driving commercial success.
Whether it’s building your brand, generating leads or leading innovation and digital transformation, Metropolis have the breadth of service delivery and experience to achieve exceptional results.
The Problem
Metropolis manages monthly reporting for over 50 clients where data is retrieved from a diverse range of applications and files.
Their reliance on third-party tools like Improvado & Supermetrics for data extraction and manual manipulation in Google Sheets became increasingly problematic. The key issues they faced included:
High Costs: Supermetrics and Improvado came with significant subscription fees that scaled poorly as data needs increased.
Manual Overhead: Extracting and cleaning data required hours of manual work, leading to inefficiencies and human errors.
Scalability Issues: As the number of clients grew, their existing reporting workflows couldn’t handle the increased data volume and complexity.
Limited Customization: Off-the-shelf tools offered limited flexibility, restricting the agency’s ability to tailor reporting to client needs.
API Connectors: API's were only available that were commonly used in the marketplace eg Google Analytics, Google Ads, Facebook Ads etc. . Metropolis required data to be extracted from API's that weren't offered on these platforms including realestate.com.au and domain.com.au.
Complexity in Reporting: Metropolis needed a flexible yet standardized reporting system that would work across different client needs.
Data Fragmentation: Data was scattered across different APIs, excel spreadsheets and google sheets.
Complexity in Reporting: Metropolis needed a flexible yet standardized reporting system that would work across different client needs.
It was clear Metropolis required a cost effective, automated and scalable solution to meet their business and client's reporting needs.
The Brief
Metropolis engaged The Data Lab to design & develop a custom fully automated reporting platform to extract data from multiple data sources including API's (REA, Domain, Shopify, Facebook, LinkedIn, TikTok, Google Adwords, Google Analytics), Google Sheets & load into their own data warehouse.
The data would then need to be transformed, blended and aggregated into a number of tabular database views that could used to build out their client reports.
Summary Of Objectives
Automated Data Extraction: Automate the extraction of data from data sources to save time and ensure consistent data access.
Centralized Data Warehouse: Create a data warehouse that would consolidate data from various sources, making it easier to manage and analyse.
Business Intelligence Reporting: Develop intuitive, automated BI reports that would help Metropolis deliver real-time, data-driven insights to their clients.
The Solution
The Data Lab implemented a modern, cost-effective data warehouse using Google Cloud Platform (GCP).
Key Steps in the Transformation:
(1) Setting Up a Google Cloud Data Warehouse
Migrated all client performance data from platforms like REA, Domain, Facebook Ads, Google Ads, and Shopify into a centralized BigQuery data warehouse.
Automated data ingestion pipelines using Google Cloud Functions and Cloud Scheduler for day -1 reporting.
The data warehouse allowed for faster querying, reliable storage, and easy scalability as Metropolis data needs grow.
The architecture also ensured data from different platforms could be cross-referenced and blended. For example, campaign performance from Facebook Ads could be compared with sales data from Shopify to assess return on investment (ROI).
(3) Data Transformation with SQL and Python
Replaced manual processes with SQL and Python to automate data transformation workflows.
Established standardized data models to ensure consistency and reduce errors in reporting.
(3) Automating Client Reporting
Connected BigQuery to Looker Studio (formerly Google Data Studio) for automatically up to date, interactive dashboards tailored to each client’s KPIs.
Enabled clients to access up-to-date insights without waiting for manual updates.
(4) Cost Optimization
Eliminated Improvado & Supermetrics subscription costs by building custom API connectors for marketing platforms.
Took advantage of BigQuery’s 1TB free data per month.
A key milestone in the project was The Data Lab were able to build a custom reporting platform that wouldn't incur any future monthly subscription fees from Google. Metropolis only had to pay an upfront development fee.
Over the next 3 years we will be saving Metropolis over 70% compared with using third party software tools as well as saving approximately 400 man hours on manual reporting processes per year (refer to cost savings chart below).
In the near future we are looking at building an AI feature that will automatically analyse client data and automatically distribute customers informative marketing insights on a weekly basis.
(4) Web Application
A custom web application was developed that allowed Metropolis to set up and maintain data connections for each client, automating the generation of reports based on incoming data (refer to screenshots of web application below).
Each client had their own secure login to access a personalized portal, ensuring they could only view reports relevant to them.
The platform included a monitoring dashboard that displayed the status of data loads in real time, enabling users to quickly identify and resolve any issues. This solution significantly improved efficiency, reducing manual effort while ensuring timely and accurate reporting.
Workflow
Impact and Benefits
Highly efficient and automated reporting system, drastically reducing the time and effort needed for monthly campaign performance reporting.
Manual errors, which had previously caused data inconsistencies, were virtually eliminated due to the automated ETL process. Data was now accurate, standardized, and ready for analysis as soon as it entered the data warehouse.
The data warehouse built was designed for scalability. As Metropolis onboarded new clients, the warehouse could easily scale to handle increased data volume without performance degradation.
Reporting platform strengthened client relationships and paved the way to being able to provide other forms of customised reporting.
We established a centralized and secure data infrastructure which improved overall efficiency and guaranteed precise and confidential reporting for each client.
By having data loaded into their very own data warehouse Metropolis now have the means & flexibility to blend & transform all of their data to gain deeper insights for their clients.
Dynamic reporting templates were developed, which were a key element in enhancing the reporting experience. This innovative addition allowed for personalized and insightful reports, contributing to an overall improved client-agency interaction.
The Data Lab also developed a reporting portal which provided easy access to dashboard links for each client, fostering a seamless and interactive experience for the marketing agency.
We achieved substantial cost savings by steering away from traditional methods and automating the reporting process.
Workflow
Conclusion
Metropolis Agency’s transition to Google Cloud transformed their business intelligence capabilities, enabling them to scale efficiently while cutting costs. By automating their data pipelines and leveraging a modern data stack, they now deliver faster, more accurate insights to their clients, positioning themselves as a leader in data-driven marketing. The cost and time savings allowed the agency to reinvest in client acquisition and strategic growth.
If your business faces similar challenges, consider exploring Google Cloud and custom data solutions to future-proof your reporting infrastructure.
Learn how The Data Lab can help you drive growth. Contact us now.
What Our Clients Say
Working with The Data Lab, we built a custom automated reporting platform and data warehouse that has drastically reduced the time, effort, and cost involved in generating monthly performance reports for our clients. What used to take days is now delivered in hours, giving us faster insights and allowing us to focus on strategic decisions. The solution has streamlined our processes and improved our overall efficiency, making a huge impact on our business operations.
Like this project
Posted May 4, 2025
Developed a custom automated reporting platform for Metropolis, enhancing efficiency and reducing costs.
Likes
0
Views
6
Timeline
Oct 5, 2023 - Oct 5, 2024
Clients
Metropolis
