Need some work done? Post a Project Today
You are asked to create a python spidering program that downloads every product information from a popular website, returning a set of information about each product (cars) (make, model, year, color, description, preice, URL pictures, etc)
1) The spider will work from a list of dealers URLs that contains the inventory of each dealer.
2) The spider has to access each product URL and save the information in an array, XML file or SQL;
3) It has to work fast or in parallel cause I need to update the information everyday for more than 100,000 products;