Closed

Fix and Review php scraping file

Hi,

I have a scraping file that scrapes data from avaialble timeslots on a website, and saves the result into a database.

1) For some reason, the first timeslots scraped are done in 2/3 seconds per timeslot, whereas the last timeslots scraped take up to 15 seconds per timeslot. This makes not much sense and I believe there is a fix for that.

2) Second, the scraper is adding the timeslots into the target website, creating a cart with a huge amount. I believe this is not very good either and may eplain, maybe, the first issue. I believe we should be able to fix that as well.

3) Last, the scraper runs every hour via cron. It deletes the previous database info and adds the new results. The problem is that when it deletes the info, the scraper takes up to 30/40 minutes (probably less with fixing 1st issue) to scrap the data, so during this time I am loosing data on the database. I would need to save in the database only when scraper has finished its job or any other alternative solution.

Thanks.

Skills: PHP, Web Scraping

See more: rating review php ratemyprofessor, fix wordpress php problems, translate php language file software, php file_get_contents scraping, webpage scraping with php, web scraping php curl, scrape content from website php, php price scraper, news scraping php, web scraping php example, php script for data scraping, php upload file generate thumbnail, php check file newer update, php video file, ignore php html file, php upload file pieces, php check file contains variable, php email file, tiny mce php write file, php mysql file upload utility

About the Employer:
( 43 reviews ) Paris, France

Project ID: #18770177

17 freelancers are bidding on average $70 for this job

coolguyinus2004

First this is not a $60 or $100 JOB as the JOB may take more price from you depends on the code of web scraping code it is. So let me know from you. i need to check code and review and work on it , so i can plan things More

$111 USD in 10 days
(343 Reviews)
7.6
alexgospodarets

Hi! I have 11+ years of web developing experience. Main skills: PHP, MYSQL, WORDPRESS, MONGODB, JAVASCRIPT, ELASTICSEARCH, JQUERY, AJAX, HTML, CSS, XML, JSON, CODEIGNITER, KOHANA, LARAVEL, SYMFONY, PHALCON I can pr More

$60 USD in 1 day
(82 Reviews)
6.9
phpprogramming

hi there, I have read your requirements and its clear to me. I am ready to fix it now. I am a professional developer having experienced in PHP, MySQL, Javascript, Bootstrap, Web Scraping and others let me know More

$55 USD in 1 day
(216 Reviews)
6.9
ferozstk

Hello, After reading your project details I believe I'm suitable for this project. As I'm expert on this with more than 7 years experience. Please feel free to contact me. I am looking forward to hear from yo More

$60 USD in 2 days
(134 Reviews)
6.6
masterlancer999

Hello I've just read your job posting and I'm very interested. I have a wealth of experience in web scraping using php And I think your current problems are not big. You can check my ability in my profile. Pleas More

$61 USD in 10 days
(45 Reviews)
6.9
schoudhary1553

Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. More

$160 USD in 4 days
(55 Reviews)
6.3
bhaveshjnariya

Hi, I am PREFERRED Freelancer and expert in Web scraping using PHP and simple html dom. I can FIX this in couple of hours. Let's do it

$100 USD in 1 day
(84 Reviews)
6.3
mcinteersoftware

I currently work remotely part time for Amerinfo as a datamining / web scraping developer. My software scrapes mines and classifies between 30k and 1.2m records per day for the company. I've built many scrapers for fre More

$61 USD in 10 days
(13 Reviews)
5.3
aswebsmart

Hi, I have read through your job descriptions, and I understand all of the requirements. I am applying because I believe that I have the required skill set to complete your project. I have been working as a PHP deve More

$55 USD in 1 day
(28 Reviews)
4.6
ganeshrasekar

I can do it .............................................................................................................................................................................................................. More

$61 USD in 10 days
(12 Reviews)
4.4
digital4design

Hi! Can you please share details info for further discussion? I would like to apprise you that I am a Web developer with around 8+ years of experience in PHP and it's a framework. I have developed several appl More

$61 USD in 10 days
(3 Reviews)
4.4
jayantbyad

“HI THERE, I’D LOVE TO HELP OUT WITH YOUR PROJECT”. I’VE DONE A LOT OF PROJECTS LIKE THIS :) Hello, Hope you are doing good in FRANCE :). I HAVE GONE THROUGH YOUR PROJECT DESCRIPTION AND GOT TO KNOW THAT YOU WANT More

$60 USD in 10 days
(20 Reviews)
4.2
danleyb2

Hello I am php developer and very good finding and fixing bugs in code, i would love to help you fix the issues you've listed, send me a message

$61 USD in 10 days
(2 Reviews)
2.5
Intranetman

HI, I AM A PHP PROGRAMMER AND WEB DEVELOPER. I HAVE A LOT OF EXPERIENCE IN PHP CODING AND DIFFERENT KINDS OF SCRIPTING LANGUAGES. I HAVE DEVELOPED MANY DYNAMIC WEBSITES USING PHP. I CAN MAKE SURE WHAT YOU NEED. PLE More

$61 USD in 2 days
(5 Reviews)
2.2
nilanjanslg

* Well come to Our Website: [login to view URL] Hello there, I'd be glad to assist "Fix and Review php scraping file" [login to view URL] last time slots scraped take up to 15 seconds per time slot. we believe that we wil More

$50 USD in 3 days
(4 Reviews)
1.2
AngelaZhang94

Dear Sir, I'm a web scrapping master and have great experience with Excel, C++ and python. I also have good experience with php and mysql. I want to know which framework your program based on. Please feel free to h More

$60 USD in 3 days
(2 Reviews)
1.3
artirajora

Hello We are team of smart developers working in IT industry from past 7-years have developed 200+ simple and complex Mobile phone apps for Android & iPhone, Hybrid app and many Websites in RubyOnRails,WordPress More

$60 USD in 10 days
(2 Reviews)
0.0