Python Web Scraper for Data Collection (Eastern Europe or Latin America)

Closed Posted 1 month ago Paid on delivery
Closed Paid on delivery

*******

LOCATION

*******

You must live in Eastern Europe or Latin America.

*******

OVERVIEW

*******

I will conduct searches on Google like "Plumbers Minneapolis, Minnesota" or "Carpenters Charlotte, North Carolina."

Then I will run the script(s) you created for me.

The script(s) you create will save the Google search results in a .csv file. That is trivial and straightforward. The script(s) will also open the websites associated with the search results. That is also trivial and straightforward. The main challenge lies in locating and filling out contact forms on these websites with my first name, last name, email address, phone number, etc.

Most search results will include a link to the company’s website. For example, when I search on Google for… Jake The Plumber Minneapolis, MN … I see a link to Jake The Plumber’s website.

Your script(s) will not submit the forms; rather, I will review each form and then manually submit each myself. In other words, I will click on, for example, the “Send” or “Submit” button.

If I were to loop your script(s) X times, then your script(s) would open X tabs in Google Chrome. Each tab would contain a website for a particular business.

For example, if I were to loop your script(s) 89 times, then your script(s) would open 89 tabs in Google Chrome. Each tab would contain a website for a particular business.

Most such websites I have visited take about 100 MB of RAM. I have 16 GB of RAM. If I were to free up 12 GB of RAM, then I could open [takes out pencil and paper, calculates furiously for a few minutes] 120 websites.

That limitation would not bother me because I plan to scrape not more than a hundred pages each day. In other words, I do not plan to scrape hundreds of thousands, nor tens of thousands, nor even thousands of websites each day. I plan to use your scripts on a relatively small scale of not more than one hundred websites each day.

*******

TESTING

*******

You must test your script on Linux Mint 21.3 Cinnamon Edition to ensure that it runs properly on Python 3.10.12, on the current stable version of Google Chrome, on Linux Mint 21.3 Cinnamon Edition. I do not care if your script runs on any other operating systems or any other browsers.

If you typically use Microsoft Windows, please consider setting up a virtual machine to install Linux Mint 21.3 Cinnamon Edition so you can test your code properly.

In particular, if I were to send you a bug report, I would not like to receive a reply from you like, “The script works 100% perfectly for me on Google Chrome on Windows 11. Because I am a 100% perfect engineer, who always ships code that is 100% bug-free, I am 100% certain that the code is 100% cross-platform compatible; therefore, I am 100% certain that it is your fault that the script is not running properly.”

Of course I am being facetious. No engineer I've ever worked with has explicitly made such ludicrous statements. Nonetheless, I want you to understand the frustration I have had dealing with incompetent, self-professed software engineers who have shipped me buggy code which they failed to test on Linux, and then blamed me.

*******

SPECIFICATIONS

*******

I will send you complete specifications for this project.

*******

A MAIN SCRIPT THAT EXECUTES SUBORDINATE SCRIPTS

*******

To make it easier to modify (in particular, to debug) you will provide me with a main script named “Main script for scraping and filling in a contact [login to view URL]”

that executes subordinate scripts. For example, “Main script for scraping and filling in a contact [login to view URL]” might contain…

import subprocess

# Common base directory

base_dir = "~/abc/xyz/"

# Run script 1

[login to view URL](["python3", base_dir + "1 of 3 script for scraping and filling in a contact [login to view URL]"])

# Run script 2

[login to view URL](["python3", base_dir + "2 of 3 script for scraping and filling in a contact [login to view URL]"])

# Run script 3

[login to view URL](["python3", base_dir + "3 of 3 script for scraping and filling in a contact [login to view URL]"])

In this example, almost all of your code would have been in…

"1 of 3 script for scraping and filling in a contact [login to view URL]", and

"2 of 3 script for scraping and filling in a contact [login to view URL]", and

"3 of 3 script for scraping and filling in a contact [login to view URL]"

*******

THE NUMBER OF ITERATIONS AND THE LENGTH OF THE PAUSES

*******

I don’t want a Google algorithm to ban me for web scraping. Therefore, you will give me a script named, “Trigger main script for scraping and filling in a contact [login to view URL]” which will include something like the following…

import subprocess, random, time

# Define the Python script file to be executed

script_file = "Main script for scraping and filling in a contact [login to view URL]"

# Define the number of times to loop

number_of_iterations = 71

# Define the range for random interval (between X and Y seconds)

min_interval = 10 # Minimum interval in seconds

max_interval = 120 # Maximum interval in seconds

# Loop to execute the script multiple times with a random delay

for _ in range(number_of_iterations):

# Generate a random delay between min_interval and max_interval seconds

random_delay = [login to view URL](min_interval, max_interval)

# Execute the script

[login to view URL](["python", script_file])

# Pause execution for the random delay

[login to view URL](random_delay)

Obviously, this will enable me to manually change the number of iterations, and the length of the pause between each iteration.

You don’t have to use the precise code above; but your script(s) must contain the functionality above.

*******

QUESTION

*******

Which do you intend to use: Playwright or Selenium?

*******

FINAL NOTE

*******

Please do not begin working on this project for me unless I create a Milestone Payment for you on Freelancer.

Python Web Scraping

Project ID: #38010873

About the project

19 proposals Remote project Active 1 mo ago

19 freelancers are bidding on average $94 for this job

chirgeo

Hello! As a skilled web crawler expert with experience in creating custom web scraping tools, I am excited to bid on your project for a Python Web Scraper for Data Collection. I live in [LOCATION] and am knowledgeable More

$500 USD in 3 days
(155 Reviews)
8.1
MashoodurRehman1

I am a highly experienced Python Web Scraping developer, having expertise in creating scripts to collect Google search results and automate the process of filling out contact forms on websites. I ensure the script(s) w More

$100 USD in 2 days
(178 Reviews)
7.8
gm341473

As an experienced Python developer with an expertise in web scraping, I believe I'm a perfect fit for your data collection project. I have a strong track record of delivering high-quality results reliably and consisten More

$63 USD in 72 days
(19 Reviews)
5.1
anushka03367

Hello I can do this. Please share the details of the task so that I can check and confirm accordingly

$163 USD in 7 days
(7 Reviews)
4.8
aznairtariq03

With my proficiency in Python and extensive experience in web scraping and data extraction, I am confident that I am the best fit for your project. My portfolio showcases a wide range of successful data collection proj More

$100 USD in 1 day
(4 Reviews)
3.7
stheven19

With my years of experience as a proficient data analyst, focused on web scraping and data manipulation, I am confident I can provide you with the high-quality Python web scraper you need. I fully grasp the challenge o More

$63 USD in 2 days
(2 Reviews)
2.0
meregyn

❤️Hi there❤️ I am a highly skilled professional with a proven track record of delivering successful outcomes. With a deep understanding of your requirements, I am confident in my ability to exceed your expectations. I More

$63 USD in 7 days
(1 Review)
2.0
Fahadwahid1

Hello Your job description has piqued my interest. With a rich background in web development, I offer senior-level expertise with over 4 years of hands-on experience. My skill set includes: Laravel, NodeJs, Python, Rea More

$25 USD in 3 days
(0 Reviews)
0.0
chornyiartur0451

❤️❤️Hi Eric A.,❤️❤️ I am Artur, a seasoned Web Developer with nearly 6 years of experience in Python and Web Scraping. After reviewing your project details, I am confident in my ability to design and develop an Eye-C More

$70 USD in 3 days
(0 Reviews)
0.0
Adrian10001

Although my name is Adrian, most people refer to me as a "coding wizard" with over six years of experience under my belt. The bulk of my work has been in web/cloud development working with a variety of sectors, includi More

$100 USD in 2 days
(0 Reviews)
0.0
FaizanCorporate

As an experienced Python developer with a strong background in web scraping, I believe I am the perfect fit for your project. I have spent over 8 years working in the field of Software Development, specializing in Pyth More

$25 USD in 7 days
(0 Reviews)
0.0
DavidMiramag

? Hello ? As an experienced Python developer with a strong background in web scraping and automation, I am confident in my ability to deliver scripts that meet your requirements. I have successfully completed similar More

$80 USD in 7 days
(0 Reviews)
0.0
fahadahmed50000

As a seasoned web developer with extensive knowledge in Python and web scraping, I am confident that I've got the skills and expertise you need to complete this project. With over a decade of experience in the industry More

$63 USD in 7 days
(0 Reviews)
1.4
matt463

Hi, Your task is very similar to one of my previous tasks and I would like to talk more about the project via chat. As an Scraping Expert with extended experience in several scraping task and scraping tools like Scrapy More

$63 USD in 7 days
(0 Reviews)
0.0
workerproff

With over 8 years’ experience as a dynamic full stack developer, I have a comprehensive understanding of the skills, experience and dedication needed for your web scraping project. I’ve worked on numerous data scraping More

$63 USD in 7 days
(0 Reviews)
0.0
rajko2

Dear sir. How are you? I can complete this Scrapping project for a few days and start working right now. I’m a Python development expert with 7 years rich experiences. In 7 years, I've built many web scrapping objects More

$63 USD in 4 days
(0 Reviews)
0.0
denys0

Hello. Note: 25$/hour. I'm a Python developer. I can develop the various libraries like Selenium and so on. I have rich experiences about web scrapping. I'd like to work with you. Thanks for considering me.

$25 USD in 1 day
(0 Reviews)
0.0