Closed

Need To Process a List with 30 000 000 Domains

Hi,

I'm looking for an expert in crawling/data processing/data mining to process a list with 30 000 00 domains.

What I need is someone to crawl these 30 000 000 domains ( + up to 100 pages on each domain so this will make up to 3 000 000 000 urls which need to be crawled) and to collect all external backlinks found on them.

This will create a very large log file ( maybe around 50 gigs) and i want the expert to help me upload that log on a server and to be able to search inside the log file.

If you have more question regarding this project, please contact me.

Skills: Data Entry, Data Mining, Data Processing, Excel, Web Scraping

See more: i want to need data, write a letter to some who want to help u continue your education and write the school you like, need to write a poem help, data entry, need drupal 6 to drupal 7 migration help, i want to write a article help, i want atranslator hindi to sanskrit and please help to ned a translaor, i d like to visit volgograd i need an interpreter, to need in spanish, want to give advertisement to need an accountant, process to get project from freelancer, process to design good web pages, how can you get imail adress write a rich people need help and how can are send my imail adresss to a rich for help money or fix, uk debt management companies outsourcing their process to india, process to get a project through freelancer, outsource your assembly process to china, need various it help around the office hourly mostly tech projects also project management for outsourcing web stuff, need various it help around the office - hourly - mostly tech projects also project management for outsourcing web stuff, i want to need net web developer mumbai, i want to need a graphics design job in dubai

About the Employer:
( 14 reviews ) Pernik, Bulgaria

Project ID: #12685180

42 freelancers are bidding on average $637 for this job

$600 USD in 10 days
(247 Reviews)
7.9
$789 USD in 20 days
(310 Reviews)
7.9
mirniyazuddin92

Dear Sir/Ma'am, I am a Web research, Data Entry & Webs Scrapping expert. I checked and understood your requirements. I can handle this job very well to your appreciation. I can find and extract the information More

$750 USD in 7 days
(304 Reviews)
7.6
Kalpanasekhar

Hello There, I have questions regarding this project. Would you be able to answer them before. My name is Kal. Looking forward to hear from you soon. Thanks. Kalpanasekhar

$526 USD in 10 days
(161 Reviews)
7.1
khushh07

Hi, I understand the project requirements and willing to start as soon as possible. Kindly share the details and I will deliver with high quality. Looking forward to work with you. Thanks & Regards Khushi India

$526 USD in 5 days
(274 Reviews)
7.2
Harun1986

Dear Sir, I will provide you Current data from (source website list for 30 000 000 domains). Also, I can scrap after login from current data Scraping source website, I Will follow (name, details (Email, phone, website More

$250 USD in 5 days
(207 Reviews)
6.7
asifdwan

Hi there! I am specialist on scraping data from any kind of websites including frequently blocking sites. Also an expert on all of data entry & research jobs. I’m ready to start it right away. I look forward to hea More

$3333 USD in 30 days
(195 Reviews)
6.6
rangacbe

Hi, Thanks for the opportunity to bid on this project. I have done such data gathering and research in the past for various clients so that I will provide good service. Regards, Jai

$555 USD in 60 days
(272 Reviews)
6.7
devi222

Hi Sir, Iam expert in Web scrapping, Research, Websearch & Data entry. I will Java code to scrape the Huge data can be scrapped in Minutes, you can see in my previous project i have scrapped 80,00,000 Urls from a Web More

$750 USD in 10 days
(147 Reviews)
6.5
mmadi

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. Our late More

$525 USD in 12 days
(6 Reviews)
5.9
dynamitewebsol

Hello, I read your requirements and understand very-well and ready to start work from now. Let's discuss more in chat, waiting for your reply. Thank you

$750 USD in 20 days
(32 Reviews)
5.9
catstevens2009

Hi there! Have done this type of work before and am a big data/scraping expert ! Please feel free to message me with any additional information or with questions if any ! Thanks! -Steve

$750 USD in 5 days
(86 Reviews)
5.6
abdussamad

Self-motivated professional, certified data entry specialist, a proficient web researcher and transcriber, blogger, web designer, Wordpress and various social sites marketer, knowledgeable in SEO, and technical / help More

$400 USD in 20 days
(40 Reviews)
5.8
shafaqat11

Dear Hiring Manager, I have gone through your job posting and become very much interested to work with you. I am an expert in these fields. I have already completed several projects like this. For evidence you can s More

$277 USD in 10 days
(107 Reviews)
5.7
Anasra

Ok ! I will help you to scrap the urls and will deliver you fast results. Moreover please check my reviews. Thank you! Amir Ishfaq

$250 USD in 1 day
(56 Reviews)
5.3
shahiddar

Hello, Iam shahid from kashmir. Over the last 7 years, I have worked for several clients. Joined Freelancer with over 7 years of experience in , Data entry, Linkedin Lead generation , Google Research Expert,Web scrap More

$250 USD in 0 days
(17 Reviews)
5.5
sheikDev

Hello, Im interested to work on this. I gone through the job description. Im looking forward to get more details about the log file and format of the logfile. Thanks, Sheik Mubarak S

$500 USD in 15 days
(21 Reviews)
5.1
utkarshkatiyar19

A proposal has not yet been provided

$1000 USD in 10 days
(14 Reviews)
4.4
crawlermonkey

Hello Friend, I have gone through your requirement. I am EXPERT in building scraping tools so I can SURELY work on your project. You will love to check my work at: [login to view URL] Skype id: [login to view URL] More

$2222 USD in 10 days
(6 Reviews)
4.5
AndreiMotpan

Dear Freelancer, Working in a multinational research company, I gathered an impressive number of project in this area which you can indeed confirm on my profile. I have experience working with many platforms incl More

$250 USD in 10 days
(12 Reviews)
4.2