Need To Process a List with 30 000 000 Domains


I'm looking for an expert in crawling/data processing/data mining to process a list with 30 000 00 domains.

What I need is someone to crawl these 30 000 000 domains ( + up to 100 pages on each domain so this will make up to 3 000 000 000 urls which need to be crawled) and to collect all external backlinks found on them.

This will create a very large log file ( maybe around 50 gigs) and i want the expert to help me upload that log on a server and to be able to search inside the log file.

If you have more question regarding this project, please contact me.

Skills: Data Entry, Data Mining, Data Processing, Excel, Web Scraping

See more: i want to need data, write a letter to some who want to help u continue your education and write the school you like, need to write a poem help, need drupal 6 to drupal 7 migration help, i want to write a article help, i want atranslator hindi to sanskrit and please help to ned a translaor, i d like to visit volgograd i need an interpreter, to need in spanish, want to give advertisement to need an accountant, process to get project from freelancer, process to design good web pages, how can you get imail adress write a rich people need help and how can are send my imail adresss to a rich for help money or fix, uk debt management companies outsourcing their process to india, process to get a project through freelancer, outsource your assembly process to china, need various it help around the office hourly mostly tech projects also project management for outsourcing web stuff, need various it help around the office - hourly - mostly tech projects also project management for outsourcing web stuff, i want to need net web developer mumbai, i want to need a graphics design job in dubai, i want help to write a conference paper, i am going to need a consultant with expertise in managing surveys using tablet computers, hi there i’d like to be considered for your writing position and i’m glad to answer any questions you may have thank you for you, d designer to hire, d character to 3d, companies that need that want marketers to be paid in facebook and twitter

About the Employer:
( 14 reviews ) Pernik, Bulgaria

Project ID: #12685180

42 freelancers are bidding on average $637 for this job

$600 USD in 10 days
(247 Reviews)
$789 USD in 20 days
(310 Reviews)

Dear Sir/Ma'am, I am a Web research, Data Entry & Webs Scrapping expert. I checked and understood your requirements. I can handle this job very well to your appreciation. I can find and extract the information More

$750 USD in 7 days
(304 Reviews)

Hello There, I have questions regarding this project. Would you be able to answer them before. My name is Kal. Looking forward to hear from you soon. Thanks. Kalpanasekhar

$526 USD in 10 days
(161 Reviews)

Hi, I understand the project requirements and willing to start as soon as possible. Kindly share the details and I will deliver with high quality. Looking forward to work with you. Thanks & Regards Khushi India

$526 USD in 5 days
(274 Reviews)

Dear Sir, I will provide you Current data from (source website list for 30 000 000 domains). Also, I can scrap after login from current data Scraping source website, I Will follow (name, details (Email, phone, website More

$250 USD in 5 days
(207 Reviews)

Hi there! I am specialist on scraping data from any kind of websites including frequently blocking sites. Also an expert on all of data entry & research jobs. I’m ready to start it right away. I look forward to hea More

$3333 USD in 30 days
(195 Reviews)

Hi, Thanks for the opportunity to bid on this project. I have done such data gathering and research in the past for various clients so that I will provide good service. Regards, Jai

$555 USD in 60 days
(272 Reviews)

Hi Sir, Iam expert in Web scrapping, Research, Websearch & Data entry. I will Java code to scrape the Huge data can be scrapped in Minutes, you can see in my previous project i have scrapped 80,00,000 Urls from a Web More

$750 USD in 10 days
(147 Reviews)

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. Our late More

$525 USD in 12 days
(6 Reviews)

Hello, I read your requirements and understand very-well and ready to start work from now. Let's discuss more in chat, waiting for your reply. Thank you

$750 USD in 20 days
(32 Reviews)

Hi there! Have done this type of work before and am a big data/scraping expert ! Please feel free to message me with any additional information or with questions if any ! Thanks! -Steve

$750 USD in 5 days
(86 Reviews)

Self-motivated professional, certified data entry specialist, a proficient web researcher and transcriber, blogger, web designer, Wordpress and various social sites marketer, knowledgeable in SEO, and technical / help More

$400 USD in 20 days
(40 Reviews)

Dear Hiring Manager, I have gone through your job posting and become very much interested to work with you. I am an expert in these fields. I have already completed several projects like this. For evidence you can s More

$277 USD in 10 days
(107 Reviews)

Ok ! I will help you to scrap the urls and will deliver you fast results. Moreover please check my reviews. Thank you! Amir Ishfaq

$250 USD in 1 day
(56 Reviews)

Hello, Iam shahid from kashmir. Over the last 7 years, I have worked for several clients. Joined Freelancer with over 7 years of experience in , Data entry, Linkedin Lead generation , Google Research Expert,Web scrap More

$250 USD in 0 days
(17 Reviews)

Hello, Im interested to work on this. I gone through the job description. Im looking forward to get more details about the log file and format of the logfile. Thanks, Sheik Mubarak S

$500 USD in 15 days
(21 Reviews)

A proposal has not yet been provided

$1000 USD in 10 days
(14 Reviews)

Hello Friend, I have gone through your requirement. I am EXPERT in building scraping tools so I can SURELY work on your project. You will love to check my work at: [url removed, login to view] Skype id: [url removed, login to view] More

$2222 USD in 10 days
(6 Reviews)

Dear Freelancer, Working in a multinational research company, I gathered an impressive number of project in this area which you can indeed confirm on my profile. I have experience working with many platforms incl More

$250 USD in 10 days
(12 Reviews)