Python script for extracting and processing website sitemaps. by Vatche ThorossianPython script for extracting and processing website sitemaps. by Vatche Thorossian

Python script for extracting and processing website sitemaps.

Vatche Thorossian

Vatche Thorossian

This project includes a Python script for extracting and processing sitemaps of websites. The script is capable of identifying and downloading website sitemaps from the robots.txt file of a website, and is compatible with various sitemap formats such as XML and GZ. Additionally, it has the capability to process sitemaps recursively and store the data in a database.
Like this project

Posted May 9, 2024

This project includes a Python script for extracting and processing sitemaps of websites