Python script for extracting and processing website sitemaps.
This project includes a Python script for extracting and processing sitemaps of websites. The script is capable of identifying and downloading website sitemaps from the robots.txt file of a website, and is compatible with various sitemap formats such as XML and GZ. Additionally, it has the capability to process sitemaps recursively and store the data in a database.
0 likes · 13 views · May 9, 2024
This project includes a Python script for extracting and processing sitemaps of websites



