Priceya - Price Comparison Website

Data Scraper

Fullstack Engineer

Web Developer

Next.js

Python

Selenium

Project Overview:

The Priceya is a comprehensive platform designed to help users find the best deals on electronics by comparing prices from various top Australian eCommerce sites. Built using a combination of modern web development and data scraping technologies, this project offers a robust solution for price comparison across multiple categories of electronics.

Home Page in the initial Stage of development

Technologies Used:

Frontend: Next JS

Backend: Flask

Database: PostgreSQL

Scraping Tools: Python, Selenium, BeautifulSoup

Data Sources: TheGoodGuys, Betta, JB-HiFi, eBay, Amazon, and other major Australian eCommerce sites

Key Features:

Data Scraping:

Utilised Python with Selenium and BeautifulSoup to scrape data from leading Australian eCommerce sites.

Gathered comprehensive information on various electronics, including prices, specifications, and availability.

Diagram to explain how scraping get data from websites and then store them structurally.

Price Comparison:

Compiled and processed the scraped data to create detailed price comparison lists across multiple electronics categories using a name mapping algorithm since every product is listed with a different name on different website.

Enabled users to quickly and easily compare prices from different retailers to find the best deals.

Product Page of Priceya Price Comparison

UI/UX:

While the project did not have a dedicated UI/UX designer, the user interface was influenced by Pricerunner.com (formerly pricerunner.co.uk).

Focused on a clean and intuitive layout to ensure ease of use for visitors.

Product Category Page of Priceya

Product Specification Comparison Page

Development Process:

Planning and Requirement Gathering:

Defined project goals and identified key eCommerce sites for data scraping.

Outlined the scope and functionality of the price comparison features.

Data Scraping Implementation:

Developed web scraping scripts using Python, Selenium, and BeautifulSoup to extract data from targeted eCommerce sites.

Ensured data accuracy and handled challenges related to dynamic content and anti-scraping measures.

Backend Development:

Built the backend using Flask to handle data processing and serve the scraped information.

Designed and implemented a PostgreSQL database to store and manage the collected data.

Frontend Development:

Developed the frontend using Next JS, creating a responsive and user-friendly interface.

Integrated the backend APIs to display real-time price comparisons to users.

Testing and Optimization:

Conducted thorough testing to ensure the reliability and performance of the data scraping and comparison functionalities.

Optimised the scraping scripts and backend processes for efficiency.

Deployment:

Deployed the application, ensuring it was scalable and secure.

Monitored the platform to maintain data accuracy and performance.

Outcome:

The Price Comparison Website successfully launched, providing users with a valuable tool for finding the best deals on electronics. The platform's ability to scrape and compare prices from multiple major retailers in real-time helped users make informed purchasing decisions, ultimately enhancing their shopping experience. The platform shut down due to lack of funding and competition.

This project showcased the effective use of web scraping and modern web development technologies to deliver a practical and user-friendly solution for price comparison.