I want to automatically extract certain content from a certain site (will disclose website later) to auto-update my online eshop website content with new or updated content from the above source only.
One key criteria should be that all existing and new movies/documentaries/ etc... should be downloadable (must have the download link active) when automatically extracting to update my site.
Some key data fields for extracting will be;
- all headers of the site including various categories. This can be simplified.
- description and screen shots of all movies/docs/animations, etc..
- an active download link for each downloadable movie or any such item from the site. The link needs to be updated automatically to avoid being expired.
I have noticed the above link usually expires for each downloadable item. Am not sure what would be the easiest way to do this but I would want all similar links for each downloadable item to be automatically refreshed every 30 minutes or so. I am open to suggestions if any. I would like to include Movies, Animations, etc as different downloadable categories under an e-store am building - [login to view URL]
A dedicated movie e-shop with downloadable items can be set also if need. Let me know if the above can be done? Iam also seeking assistance from other sources so do please let me know soon
21 freelancers are bidding on average $207 for this job
I have prior experience of developing web scrapping programs. It will work as per your requirements. Please share your best discussion time to meo further.
This is my first freelancing project but not my first web crawling script using python, to avoid the link being expired the script can run every 30 minutes to retireve the data and update them.