Data Scraping for an E-commerce Website
Asif Farhan Khan
Asif is one of the best Python developers I have worked with till date. He went above and beyond to deliver the work and accommodated all my requests. You will not go wrong with Asif, he is a capable developer and a thorough professional.
Client's Objective
I would like to get the data for products from a Website based on a subcategory that I input
- The script will get the response of subcategory URL
- Iterate through the products from the subcategory page and visits each product page
- Retrieve the following details after visiting each product page
- Product Price
- Product Name in Arabic & English
- Product Description in Arabic & English
- ISBN Number
- Image URL
Solution Provided
The Client had requested a script that could be altered in a way so that a URL is configured and the product details from that URL are scraped and saved as a CSV
- Delivered a script that takes in command line promp so that the URL wouldn't have to be configured in code
- multi process was implemented allowing to scrape large data in very less time'
- Packaged documentations for debugging, maintaining and support for installation along with the scraper bot.
- As the website had Arabic texts, implemented proper encoding to parse data and avoid any issues
Data Retrieval Process
The scraper run commands is to be included with the subcategory name. The program automatically builds the complete URL based on the subcategory and requests the response of the products page, After which each product page is visited to retrieve the requested result.
Output
The final data is parsed, cleaned and checked for any encoding issues and then saved as a CSV to be used for further Analysis
