Datascrape and Search Program

Closed

Description

Project requires: Datascrape pre-defined URLs, custom search of stored pages, google translate results

1) For a predefined list of URLs (obtain from database, by reference to specific fields in each record);

2) Scrape and store the entire website (text only) ("Level 0 Copy") - avoid duplication/recursion of pages generated from URL-local content databases - time and depth limits to be admin editable and flagged if reached;

3) Search stored Level 0 Copy of each page for certain keywords, keep only those pages for which there is a positive result on search ("Level 1 Copy");

4) Search remaining Level 1 Copy pages for a separate set of keywords, keep only those pages for which there is a positive result ("Level 2 Copy");

Up to 5 iterations of search and keep only positive results - the structure is of multiple filters, applied sequentially, to produce a small number of remaining pages that match all keyword sets;

5) Final step if keyword searches conducted in non-english languages (defined at beginning of process), send resulting pages (after all 5 filters) to google translate to english;

6) Output - Excel spreadsheet report on search meta-data and results (template will be provided); and pdf of all remaining pages at end of process with translated copy if applicable.

This is stage 1 of a larger project and must be completed quickly. Clean coding required due to long-term nature of project. Immediate follow on to next stage.

You must advise on any external processing (e.g. cloud) requirements. All testing and development to be on your own account. Final large scale test only to be conducted on my accounts.

Your bid must have at least a price estimate, subject to confirmation following discussion and provision of more detailed description of sources and process.

Skills: Data Mining, Database Programming, PHP, Software Architecture, Web Search

See more: datascrape, testing iterations, template for testing software, software testing pdf, software development price list, sets in data structure, search data structure, price structure template, php cloud development, external software testing, all subject website development, list of coding languages, search google translate, testing small scale project, search match, Google Custom Search, google cloud project, search keywords list, google cloud architecture, copy url search results google, scrape data list urls, project data structure program, php excel database report, large scale php, google search copy url

Project ID: #5929261

16 freelancers are bidding on average $988 for this job

FreeSkyTeam

Dear Sir, We are FreeSkyTeam from Hanoi, Vietnam. To meet your requirement, we propose following solutions: 1- We develop a JVM based application using Scala programming language. 2- The app will read list of UR More

$1263 AUD in 50 days
(5 Reviews)
7.2
mhmhz

Hi I can provide a desktop application in C# that will fulfilled the requirements. Ready to provide demo if you are interested. Thanks

$1157 AUD in 8 days
(123 Reviews)
6.7
Dhruvika111

Greetings! Proper Communication and a small samples before starting a work is the most important key to complete the project successfully , since its the most important parameter it has to be easy and clear. For thi More

$526 AUD in 5 days
(108 Reviews)
6.7
SICS

Hello, We have some couple of questions..Could you please respond back,. so that we can clarify our queries. Can we have a discussion? We are an ISO 9001-2008 certified company.We are very good in mobile app More

$1546 AUD in 20 days
(32 Reviews)
6.4
uumairkhalid

Hi.. Expert Web Scraper & Data Minor here. I have done too many similar project in past. Having best scraping tools and experience i assure you 100% accurate and good quality work. I have too too scraping experience. More

$526 AUD in 10 days
(58 Reviews)
6.0
fhasanbd

Hi, I would like to work on this project. I have done a lot of similar project of this. So willing to response to discuss before awarding this project. Hopefully you will me chance to work on this project. Thanks.

$631 AUD in 10 days
(83 Reviews)
6.0
diamond247

Hello Sir, We are a well built set up with excellent skilled operator with lot of experience in this segment/skill,have complete more than 200 similar job in freelancer in last 4 years, i have gone through your proj More

$526 AUD in 10 days
(62 Reviews)
6.0
vsolcorp

Hello, Sir I am from vSol CORP. We are 19 people team, worked on more than 500 completed projects. We provide round the clock service. I have checked your project requirement. We are professional web researcher. Pl More

$526 AUD in 10 days
(41 Reviews)
6.0
phpdudes

Hi Ashton, I have updated my bid to reflect the new info provided. This will be a very complex scraping process and will involve a lot of trial and error. Personally i think 95% accuracy is not doable , perhaps a 80 More

$3000 AUD in 40 days
(51 Reviews)
5.9
angwangri

Hi sir, I am a web expert in dahuasoft online support. I have just read your job description and been excited. Because all your requirements are suitable to me. Here is sample for web scraping. http://www.craigsl More

$527 AUD in 10 days
(21 Reviews)
5.8
theclientisking

Hi, I am an experienced software and Web developer, and I am highly expert at Web scraping. I understand that this is the first stage of a larger project, and I will keep this in mind during development. First thing More

$560 AUD in 14 days
(61 Reviews)
5.3
ghazalpasha

I have done 50+ scraping projects with 5 star feedback and should be able to do this one fairly quickly. Please specify the following: - Rough number of URLs you will need to scrap - Rough number of Level 5 documen More

$526 AUD in 10 days
(20 Reviews)
5.0
prashushinde9

Hi, I (Myself a senior PHP, ASP.NET programmer using C# and VB) have a team having experience of more than 10 years in developing web and mobile applications. I have a team of expert programmers who write robust and bu More

$789 AUD in 20 days
(12 Reviews)
4.9
binarycodersvw

Hi.I have read the description of your project and I am interested. I already have a 1000$ internet automating and scraping robot software which I can use in your software. I can even make the software solve 5000 chapc More

$1357 AUD in 17 days
(10 Reviews)
4.4
dabing1205

Good at data scraping and processing.##########################################################################################################################################

$1052 AUD in 10 days
(15 Reviews)
4.3
Time2win

Hello, We have excellent team of programmers and designers to work on your project efficiently and complete job in time. We have read your deepest requirement at our best and will surely give better results. thanks

$833 AUD in 20 days
(11 Reviews)
4.2
phamtech211

My team is really interested in this project as it is exactly within our scope of expertise: We are WEB application design & development experts. Please kindly visit our website phamtech.net to learn more about us and More

$1888 AUD in 30 days
(7 Reviews)
3.9
talhamunir2002

Hello AshtonRS we can do any type of scarping,hacking and cracking of software contact me......................................................................................................................

$789 AUD in 10 days
(1 Review)
3.0
saminatinny

I would like to work on your this project. I'm new in freelancer.com but not know in this sector. I have more than 4 years experience in this sector. So willing to response to discuss about the project. Thanks and More

$611 AUD in 10 days
(1 Review)
0.0
bhadresh22

Hello Sir, Greetings of the day!!! It'll be pleasure to work with you. Allow me to do this task for you. Thanks Bhadresh Skype- bd_orbit

$700 AUD in 20 days
(2 Reviews)
0.0