Web scraper | ETL development

Max Bohomolov

(0)

Starting at

$

50

/hr

About this service

Summary

I develop highly optimized web scrapers for targeted websites, primarily using Python without relying on browser automation services. This approach delivers fast, efficient solutions with flexible configuration options, but requires in-depth site analysis.
For particularly challenging sites, I may employ automation services like Playwright, but only when absolutely necessary.
I have extensive experience extracting data from websites of any complexity. This includes standard HTML parsing, using JS, data from maps, charts, online tables, and more.

Process

Preliminary Requirements: You provide the website link, target data fields, and desired delivery method. Initial Analysis: I conduct a preliminary analysis to identify potential challenges and provide an initial estimate. Sample data: I prepare a sample dataset using the first test version of the code for your approval. Main development: Based on your feedback, I refine the sample and proceed with the main development of the web scraper. Final Validation: I perform thorough testing and validation of the collected data. Code Finalization: I complete the code according to the chosen delivery method.

FAQs

  • Are proxies used?

    The code is developed with proxy support. Proxies must be provided by the client or can be rented at an additional cost. I recommend Apify for proxy services.

  • How does the delivery method affect the project cost?

    I work on an hourly basis. The total project time (and thus cost) varies depending on the chosen delivery method and project complexity.

What's included

  • One-time data collection, only results

    This option is ideal if you need targeted data once or prefer to request updates directly (for an additional fee). Data will be provided in your preferred format: CSV, JSON, XLSX, or others as required.

  • Source code

    Suitable for long-term projects that may need ongoing support. You'll receive full access to the code, along with clear instructions on how to run it.

  • Source code as a package.

    Perfect for developers or when the web scraper needs to be imported as an external module. Includes optimized code, comprehensive documentation, and proper package structure.

  • Other

    Additional services available: Dockerized application; Deployment to your infrastructure; Deployment to my infrastructure with exclusive support (requires payment of server rental fees).

Example projects

Development of Apify Actors for E-Commerce

Comprehensive Data Collection and Analysis of UK Real Estate

Skills and tools

Data Scraper
Data Analyst
Data Engineer
BeautifulSoup
Playwright
Python
SQL

Industries

Data Mining
Data Storage
Data Management

Work with me

More services

Developing Actor for Apify

STARTING AT

$50/hour

hourly rate