Building Supplies Pricing Data

Leveraging a dataset of over 10,000 building supply names, I employed artificial intelligence to standardize the data, subsequently harnessing the Bing API to generate search results for these products. I then programmatically extracted pricing data from various websites, enriching the dataset with detailed metadata autonomously. By employing quantile analysis to refine precision, this multi-threaded endeavor achieved the capability to analyze hundreds of websites every second. The aggregated results were systematically stored in a Cosmos No-SQL database, showcasing the project's robust and efficient data handling capacity.