Web scrapping and web automation bot or customization of irobotsoft

I needed a Data extraction bot. A good part of the functionalities I need are already in the open source software irobotsoft. A customization of this, more appropriate script or one build from scratch will be welcomed. The bot should be able to do the following:

1. Extract all applicable fields and entries given a list of search keywords

2. Extract all entries plus sub entries given a list of links

3. save content in a local MySQL database and CSV

4. Remove duplicates from the final results

5. Sort out items missing with extraction using keywords search vs extracting with respect to given links


Fields to be scraped can be selected by auto-record and can be adjusted manually to improve accuracy. The fields to be scrapped will differ per website. During the selection of the field the user will indicated to which column in the dataset the fields belong. The keywords to be used for the search will be provided as a list.

A good part of what I need exist in irobotsoft or other extraction tools. I need someone who knows the tools or can build one to customize it for my use. Please let me know if you have any questions, suggestions or need further information.



Skills: MySQL, PHP, Software Architecture, User Interface / IA, Windows Desktop

See more: web search remove, web scrapping c#, the final results, respect-search and selection, windows automation, script automation, data scrapping software, bot needed for web, automation bot, scrapping software, mysql sort field, remove duplicates mysql, database remove duplicates, list auto bot, script php extract data mysql, build bot php, extracting data csv, web bot needed, php mysql remove duplicates script, bot auto web bot, auto bot list, search bot database, source auto bot, data scrapped, mysql data extraction

About the Employer:
( 21 reviews ) rotterdam, Netherlands

Project ID: #4670903

5 freelancers are bidding on average $377 for this job


I can help in your project, please check PMB and our ratings/reviews to get idea of our experience. Please let me know if you have any queries.

$257 USD in 5 days
(240 Reviews)

Hi, we are a team of programmers and designers with expertise and experience in PHP, Mysql, jQuery and Ajax. We have coded and designed websites for various types of businesses very successfully. We have developed many More

$144 USD in 3 days
(54 Reviews)

Hey I can start work on this rightaway.

$155 USD in 5 days
(25 Reviews)

Your budget is too small for your requirements. Please check PMB for my explanation.

$1250 USD in 45 days
(6 Reviews)

I have experience with your project requirements , check pm.

$144 USD in 3 days
(4 Reviews)

Hi, Expert solutions here. Please see PMB.

$70 USD in 3 days
(6 Reviews)

Hello , We have gone through with your description and yes the same we already have done, So it would be like a well known project for us and we definitely do this for you. To see what we have done please check your PM More

$155 USD in 3 days
(0 Reviews)

Cover Letter We have completed 14 projects successfully in ODESK,and we are new to Freelancer. All the projects are related to the data mining projects with reference to that I am applying for the above positio More

$155 USD in 3 days
(0 Reviews)