Closed

Data Scraping

The task is to

- create a puppeteer script to go to a specific page,

- scrape the information and download files,

- and upload the data and files to our server.

The script

- is preferably in php using puphpeteer, but this is no strict requirement

- should avoid rate limiting

- can use the SDK Apify if it helps, this is no requirement, but should be able to run locally

Your work will be accepted if

- it can run without problems for at least 1000 pages

## The script

You will obtain API key and URLS to be able to submit the results to the server.

getNextTask() from our server

returns **kbonumber**: 0525.901.534

1) Use the number to go to the following page:

[[login to view URL]]([login to view URL])**kbonumber_without_points**

for example:

[[login to view URL]]([login to view URL])

2) Update the data

do a POST action with the following name and parameters:

updateData(K**bo_number**, Naam, Adres, NIS, Rechtsvorm, Rechtstoestand, NACE)

In this example:

updateData( "0525.901.534", "SOCIALCOM", "Sinter-Goedeleplein 5 BE 1000 Brussel", "21004 Brussel", "Naamloze Vennootschap", "Opschorting (gerechtelijke reorganisatie) Sinds 15/01/2020", "73200 - Markt- en opinieonderzoekbureaus");

3) upload the 3 newest PDF and XBRL files (if available, there may be more, there may be less)

for each line, do:

uploadFiles(K**bo_number, L**inenumber**,** Datum_van_afsluiting_van_het_boekjaar, Aard_van_het_document, Taal, NeerleggingsDatum, Referte, Files)

in this example:

POST

uploadFile("0525.901.534", 1, "31/12/2019", "Jaarrekening", "F", "24/07/2020", "2020-3400369")

with **files** containing the 2 types of files

4) confirm succesful completion

setTaskCompleted(**kbonumber)**

for example

setTaskCompleted("0525.901.534")

## Deadline

Please set a deadline for your own work.

Skills: PHP, Web Scraping, Software Architecture, MySQL, Python

See more: data scraping linux, perl data scraping, data scraping software pfpc, data scraping software, data scraping python, web scraping, data scraping uipath, is data scraping legal, data scraping tutorial, data scraping tools, how to do data scraping, data scraping website using, data scraping vbulletin, data scraping php tool url, php data scraping tool, data scraping php, data scraping etfconnect, data scraping process, data scraping software reviews, data scraping ajax

About the Employer:
( 2 reviews ) Waregem, Belgium

Project ID: #27374444

24 freelancers are bidding on average €142 for this job

(672 Reviews)
7.9
schoudhary1553

Hi, Greetings! ✅checked your project details: Data Scraping ✅Completed Time: In project deadline We have worked on 600 + Projects. I have 6 + years of the experience in same kind of projects. If you are looking More

€220 EUR in 4 days
(148 Reviews)
7.3
(96 Reviews)
6.6
masud0012

Dear Koen V., What's going on? https://www.freelancer.com/projects/Python/Build-Web-Scraper-Python/reviews Waiting for your kind reply. Respectfully, Masud Web Designer & Developer

€30 EUR in 2 days
(57 Reviews)
5.6
AHMEDAMER232

✔️ Hi, There! I'm very confident of your project. I can complete your project in 1 day. 1. My scraping works and experience I have 9 years of experience in web scraping and scraped so many many websites(around 2,000 More

€200 EUR in 1 day
(90 Reviews)
5.8
Pashakolp31

✨✔⭐⭐Hello⭐⭐✔✨ My name is ✨Pavel✨ and strong experience in Web site scraping use Python selenium. If you hire me in the project, I will do my best. Best Regards

€150 EUR in 1 day
(19 Reviews)
5.3
RusselExpert

deadline is 5 days including weekend. Greetings. Thanks for your post. I understood your goal. I am scraping expert who has built 100+ scrapers using php/python/c# so I can deliver the perfect result in high quality pe More

€140 EUR in 7 days
(20 Reviews)
5.3
pavelmishin2019

Dear client As an expert in web scrapping, I am very glad to see this job posting. I have experienced in web scrapping with Python, PHP Curl, Node Pupeteer for the past 6+ years of period. Please check some of my past More

€140 EUR in 7 days
(8 Reviews)
4.2
shynar88

Hi. Nice to meet you. I have good skills with data scraping. [login to view URL] was built by me and it base the scrapping data from the [login to view URL] I have good experiences with scra More

€200 EUR in 7 days
(9 Reviews)
4.4
juneadkhan

Hello Sir!. I am a web scrping expert, I think I'm a great fit for this project because I have an interest in your project and can deliver on time, according to your specifications Thanks

€140 EUR in 7 days
(10 Reviews)
3.9
bantenunix

Hello. I read your requirement in detail. I am a scraping expert and have good experience wfor 5 years with scraping. I have done many scraping projects with php and node.js. I am sure I can provide satisfactory result More

€200 EUR in 7 days
(4 Reviews)
3.1
nemanja1029

Hello. Koen [login to view URL] [login to view URL] [login to view URL] I have made website for front end and back end for cost estimator and multi vendor. I have many experience about More

€140 EUR in 7 days
(5 Reviews)
2.6
yurijklimov98

Hello Potential Client. I am professional software developer with 7 years experiences of Web Scraping, MySQL, Excel, Data Mining and Python. I have developed many scraping tools including for amazon.com. I can start wo More

€50 EUR in 1 day
(2 Reviews)
2.8
guerreroalvaro10

Greetings to Koen. Nice to see you. Hope you are doing well from COVID-19. I have various experience in web scraping, especially using Scrapy, Selenium, Beautiful Soup and Splash. I have a very good understanding of ho More

€140 EUR in 5 days
(1 Review)
2.4
vadimdev8

Hello, sir. How are you today? I have built hundreds of web scraping tools using Python and NodeJS puppeteer. I can provide you fully functional and high-performance scrapping tool with my programming skills. Please co More

€140 EUR in 2 days
(1 Review)
2.5
oxygenweb

Hey Dear, Hope you are doing well in this pandemic! I’ve gone through your project requirements in deep. But I’ve some questions and want to discuss this opportunity in detail. I’ve 8+ years of experience in PHP d More

€140 EUR in 7 days
(6 Reviews)
2.4
DressyFiddle

Hi there, I can create this scraper for you but I have a few questions about it first. Contact me for more information. Best regards.

€155 EUR in 7 days
(1 Review)
2.0
Umerfarooq92912

Hi! Greeting, I am interested in your project. I will show you how I can work diligently and with expertise. I guarantee you to submit the work within the timeline and as per your expectations. I understand your proj More

€250 EUR in 7 days
(3 Reviews)
2.1
Ahsan01AHMAD

Hello, Dear Prospective Client I have checked your project's description & I think I am the perfect one for your job. I'm ready to START now with a sample for the accuracy and quality check. I will make sure to deliv More

€30 EUR in 2 days
(2 Reviews)
1.2
OthmenBraham

Hi, i'm an expert in highly responsive website with optimale web technologies, please check my feedback then you will know. i will work this project with c # desktop application with buttons, progress bar, multithreadi More

€140 EUR in 7 days
(0 Reviews)
0.0