Client
Electronic Retail Group
Our client needed Retail price monitoring competition data on a monthly basis. They were expecting to reach about 250 million SKUs in a year, and wanted to use this data for their pricing strategy and retail price monitoring.
The data fields they wished to extract were product part number, quantity, price associated with quantity, stock levels and lead times. For them, easy access to data was critical and that’s why they decided to partner with PromptCloud, a managed service provider, instead of continuing with their internal web scraping setup.
They are specifically needed clean and ready-to-use data, which can be directly uploaded to their database, so they could run the retail price monitoring activities for their business.
Client Requirements: Client provided us with the list of competitor sites and the data points they needed to crawl and extract. The crawl frequency was set to monthly, meaning fresh data sets would be extracted on a monthly basis.
Custom Crawler Set Up:PromptCloud team then set up the crawlers for the project, within two days. This was a use case of our site crawl offering, since the source websites in the list had different structuring and design.
Access Data:Client needed the extracted data in CSV format and to be uploaded on their Dropbox account. We set up the crawler in an automated manner, to deliver the data in the required frequency
Data Template: A data template was created based on the schema provided by the client, and data structuring was done as per the template.
Data Delivery: Once the crawler setup was complete, we started delivering data to the client in their preferred format and delivery method.
Minimum Maintenance: There was no need for any further manual intervention from the client’s side, as we setup automated monitoring for the source sites in order to detect source site changes.
The first data delivery consisted about 200K records.
Benefits to the client:
- 100% API availability and continuous data feeds
- Zero data processing efforts at client’s end
- Scalable infrastructure reduced client’s costs
- Client’s analysts only focused on querying final datasets and running analyses
- Clean and ready-to-use data
- Client could focus on their core business functions with the easy access to data
