I need a web crawler which will
take an input field provided and crawl up specific sites, every hour and grab the latest news links and video links posted on those sites for the input fields.
This site will keep growing, ie. the inputs will keep increasing.
The crawler needs to be robust enough to keep taking in additional inputs and searching the specific sites for those given inputs.
The bidder who can demonstrate experience in this type of design will be given the highest priority.