Build a web scraper to retrieve specific website information on a regular basis


For this project several bits of information must be gathered from a website and saved into a MySQL database. The information type is always the same but the parameters are different.

In most cases the website has APIs for retrieving the data and this should be the preferred way of getting the data.

Please see attached PDF for the exact data required.


- Project needs to be created in Java.

- Application needs to use the Selenium Framework and Chrome/Firefox in either window mode or headless mode.

- All calls must be parameterised and should work for all Leagues, Teams, Players, Matches.

- APIs from the target website must be used only, no scraping of HTML. In some cases the data is embedded in the code using JSON, it is acceptable to retrieve the data from Javascript variables or using Javascript calls if APIs are not available.

- API endpoints seem to be available only after visiting the main pages, so some protection exists, possibly based on cookies. So this needs to be circumvented.

- Data will be saved in a MySQL database with tables per each dataset. Eg. Players, Teams, Leagues, Matches.

- IDs must be used and preferred as opposed to Strings. IDs from the target website must be used as opposed to locally created auto-increment IDs.

- For each ID stored in the database an associated table must exist with the String value. Eg. Player ID – Player Name.

- So tables need to be normalised.

Skills: Java, Javascript, MySQL, Web Scraping, Website Testing

See more: best web admin php manage website, build content management system newspaper website using dreamwaever, web develop booking calendar website, python 3 web scraping, web scraping using python tutorial, html scraping, web scraping python beautifulsoup, web scraping tutorial, web scraping with python pdf, web scraping with python amazon, python web scraping library, create web scraper, build web site badoo, extracting information website using regular expression php, javascript rollover embedded html wont work web, web scraper regular expression sample, build basic web scraper java, build dotnet web scraper, php code build web based scraper, regular expressions url web scraper

About the Employer:
( 0 reviews ) London, United Kingdom

Project ID: #18194755

Awarded to:


Hello Sir, We are expert in web scrapping using java and Python. we will write automation script which you I will setup at you local system or server. or you can run it manually on daily/weekly basis. please ping me More

£80 GBP in 20 days
(27 Reviews)

9 freelancers are bidding on average £89 for this job


Good Day Sir! I read your post. But I have something to ask you. Your idea is amazing and it will change the world! I am a magic talented developer in your skill. If you wanna be the success, hire me I am looking More

£50 GBP in 10 days
(563 Reviews)

Dear Client. I have fully read the job description and I am interested in your project. I am Li G from China. I would like to help you with my skill and experience. As you can see my profile, I am a talented we More

£80 GBP in 10 days
(26 Reviews)

Hi There, This is S Mills From ByteSols LLC located at CA. I have gone through your requirement of wordpress website designing and I can assure you that I can help you out with a custom website, template developm More

£250 GBP in 10 days
(10 Reviews)

Hi there, I see you're looking for an automation engineer, with excellent skills in selenium and java, to help you with your website scraper project. Do you already have the MYSQL database created? Have you already More

£77 GBP in 15 days
(19 Reviews)

Dear, employer. My name is Xungxiao, I am an experienced web developer, and web scraping expert. I am a new freelancer here, but I have good experiences in web scraping using PHP, Python, Java and so on. I read yo More

£50 GBP in 1 day
(14 Reviews)

Hi sir I am an expert in web scrapping with high technology. I am good at java, Javascript, MySQL, Web Scraping, Website Testing. I think you can get a good result with this project. please chat in detail. Thank y More

£50 GBP in 10 days
(9 Reviews)

Hi Sir, I am Lavish Karankar having vast experience in Automation of jobs using Python and JAVA language. I have gone through your provided description, I would like to help you on this as I have done this task in my More

£88 GBP in 10 days
(3 Reviews)

I have 6 years experience in automation testing. please get in touch

£72 GBP in 2 days
(0 Reviews)