Urgent Experienced Python Selenium Developer for Web Scraping Project

In Progress Posted 2 months ago Paid on delivery
In Progress Paid on delivery

I am looking for a highly skilled and experienced Python developer with expertise in Selenium web scraping to assist with optimizing and enhancing our current web scraping project. This project involves collecting data from 3 specific websites for personal use, with a focus on improving speed and efficiency. The ideal candidate will have a strong background in web scraping, a deep understanding of Python and Selenium, and the ability to implement cost-effective solutions for running web scraping tasks.

Responsibilities:

Review and optimize existing Python Selenium web scraping scripts to improve speed and efficiency.

Implement headless browsing, and optimize browser settings to disable unnecessary elements such as images and JavaScript for faster page loads.

Introduce efficient data handling and storage mechanisms to manage the data collected from web scraping activities.

Explore and implement concurrency and parallel processing techniques to enable simultaneous scraping of multiple pages.

Advise on and implement caching strategies to reduce redundant downloads and speed up the scraping process.

Evaluate and recommend free or cost-effective cloud solutions for deploying the web scraping scripts, considering the potential for scaling and reducing latency.

Ensure the robustness of the web scraping solution, including error handling and avoiding IP bans or CAPTCHAs by target websites.

Document the code and provide guidance on running and maintaining the web scraping setup.

Requirements:

Proven experience with Python and Selenium for web scraping projects.

Strong understanding of web technologies (HTML, CSS, JavaScript) and how to interact with them using Selenium.

Experience with headless browsers and optimizing Selenium configurations for performance.

Knowledge of concurrency and parallel processing in Python (e.g., threading, asyncio).

Familiarity with caching strategies and their implementation in web scraping contexts.

Experience with cloud platforms (AWS, Google Cloud, etc.) and understanding of their free tiers and how to leverage them for web scraping tasks.

Ability to work independently, with minimal supervision, and deliver high-quality code.

Excellent problem-solving skills and attention to detail.

Strong communication skills, with fluency in English.

Nice to Have:

Experience with Docker and Selenium Grid for parallel processing.

Knowledge of other web scraping frameworks or libraries in Python such as Scrapy.

Python Web Scraping JavaScript Software Architecture Django

Project ID: #37795990

About the project

19 proposals Remote project Active 2 mos ago

19 freelancers are bidding on average ₹7398 for this job

vinzin

As a seasoned Python developer with strong experience in web scraping, specifically using Selenium, I am confident in my ability to magnify the speed and efficiency of your web scraping project. My five-year immersion More

₹6000 INR in 4 days
(34 Reviews)
5.1
priyanshu8090

I have Decent knowledge of Python and can help you with your projects. Can we have a chat to discuss it further in detail?

₹2700 INR in 7 days
(35 Reviews)
4.5
malkesh3m

⭐ Hi, My availability is immediate. I read your project post on Python/Web Scraping Developer. We are experienced full-stack Python developers with skill sets in - Python, Django, Flask, FastAPI, Jupyter Notebook, Sel More

₹10200 INR in 2 days
(25 Reviews)
4.3
artazim

As an experienced Full Stack Python and JavaScript developer with a demonstrated aptitude for full life-cycle software architecture, I would be a valuable asset to your web scraping project. Having previously utilized More

₹12000 INR in 7 days
(5 Reviews)
3.6
akkiwadhwa1995

Hey Hope this message finds you in the best of tech-savvy spirits! As a web scraping specialist and data scientist, I bring together a diverse set of technologies and tools to extract valuable data from the web efficie More

₹7000 INR in 7 days
(9 Reviews)
3.8
larsenm2020

As a seasoned Python developer with a comprehensive understanding of Selenium and web scraping, I am certain that I am the right fit for your project. My skills in headless browsing, optimizing browser settings, and im More

₹2000 INR in 7 days
(9 Reviews)
2.8
odhiamborober

As an experienced Python developer specializing in web scraping with Selenium, I am well-suited to optimize and enhance your current project. With a proven track record of improving speed and efficiency in web scraping More

₹12500 INR in 7 days
(4 Reviews)
2.5
fcalus

Hi, I can help you achieve your goal. I have experience with web scraping (bs4) and automation (selenium). I have scraped a lot of dynamic sites and handled different captchas. Looking forward to hearing from you. Be More

₹7000 INR in 4 days
(0 Reviews)
0.0
dipshikhatiwari2

I am thrilled to submit my proposal for the role of optimizing and enhancing your current web scraping project. With extensive experience in Python development and expertise in Selenium web scraping, I am confident in More

₹7000 INR in 7 days
(0 Reviews)
0.0
techindia999

Hi Greetings! We are a team of software developers. We have developers working on almost all the technologies. We will assign you a project manager who will assist you during the entire process of project completion. More

₹12500 INR in 5 days
(0 Reviews)
0.0
kipkogeir

Hi Thank you for inviting me to apply for this job. I am a very hard worker who is in serious need of a job. I’ve been going through some difficult times and I am willing to do anything to get this position. I am will More

₹7000 INR in 7 days
(0 Reviews)
0.0
kmhasancpa

I read your project details, I have 13 years experience on your project. I can do it very easily Already i completed some projects like this. Just a Close your eyes and Trust me. I am sure, you will be happy

₹1670 INR in 3 days
(0 Reviews)
0.0
Subanii

As the project demands a seasoned professional, I strongly believe my 4.5 years of industry experience and my expertise in Python would be an ideal fit for this project. Not only have I specialized in cloud technologie More

₹7000 INR in 7 days
(0 Reviews)
0.0
Kirtig15

Hi I am kirtee Goyal a data engineer. I have 4 years of experience in web and mobile app scraping. I use php and python for scraping. I scraped more than 100 sites till now and handled US based client. Please share me More

₹7000 INR in 7 days
(2 Reviews)
0.0