Create webspiders / crawler to compare prices across multiple sites;
Should be implemented using configuration files*, separate modules for easy maintenance, change control (for example if the tag-tag has changed), map the locations of these pages with relevant information (eg price);
The user of the crawler will pass the name of a product, eg 32GB iPad 2, and the crawler will:
1. navigate through the search engines for each site
2. find the parameter passed
3. find a hash / unique identification code of the product
4. navigate to another site
5. if the products are identical to compare prices
# Knowledge required:
1. Experience in developing systems for processing and
data extraction using Object Oriented Perl, more specifically, developing webspiders / crawler.
2. Fluency in the use of parsers HTML / XML.
3. Domain in the use of Regular Expressions.
4. Knowledge of SQL.
# Specific knowledge:
# Sample of module file:
*[url removed, login to view]
--->> Available in an accessible via http
11 freelancers are bidding on average $999 for this job
Hello, 10+ years exp full time freelancer here, can deliver a quality & professional work in the timeframe posted. Please contact me via PM for any question, Thanks.
Have 10 years experience with perl. Experience parsing HTML and XML and creating high-performance web crawlers/spiders with LWP. Experience with MySQL, PostgreSQL databases.