Closed

Write a scraping script using python / selenium / bs4

I collect stock market ticker symbols (and their associated company names) for private research. I prefer to gather these from the stock exchanges around the world. Some exchanges make it very easy to gather this information by providing downloadable spreadsheets or CSV files, others present the data in compact HTML tables, while still others spread this information out over a fairly large number of pages that require considerable navigation to view.

I am looking for a person with extensive experience with Python (2.7), Beautiful Soup and Selenium. The task is to write script to automate the process of navigating sites (using selenium) and extracting and assembling [ticker symbol, company name] tuples from the HTML using BS4, and ultimately dumping the collected data to a flat file (preferably JSON).

Here is an example of a national stock exchange site for which such a script needs to be written: [login to view URL]

The interface consists of three levels. The top level allows a user to select the starting letter of a company name [A-Z,0-9].

Each of these hyperlinked starting characters leads to the second level of the interface that displays a table of companies. Each row in this table displays a company's name, but not its ticker.

To see the ticker, the user must click on the name hyperlink to be taken to a "Company detail" page (the third level of the interface). This detail page shows one or more alphanumeric ticker symbols (called the "Trading Code") along with other useful company information to be parsed.

I need this script to run on a headless Chrome browser running on Linux (Ubuntu).

This 3-level tree must be traversed by using Selenium since some of the hrefs fire JS events.

If this is something you have extensive experience with, please apply! Make sure you have an excellent command of the three requisite technologies before applying.

I am looking for one or more people to whom I can assign many such automated data harvesting tasks of varying degrees of complexity. Please base your quote on the particular example described above.

Skills: Python, Web Scraping, Javascript, Data Mining, Selenium Webdriver

See more: read csv file using python script, screen scraping craigs list using python, write pdf files using perl script, code selenium test development using python, write screen scraping script, web scraping using python, write script telit device using python script, python scraping script, script convert text csv using python, web scraping script using selenium python, data scraping using python, how to write data in excel using java selenium webdriver, how to write installation script in python, install software using python script, how to write multiple sheets in excel using python, edit a file using python script, read and write in excel using python, web scraping example using python

About the Employer:
( 1 review ) Olympia, United States

Project ID: #22247611

42 freelancers are bidding on average $160 for this job

zekovicm

Hi there,I am Python Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it wit More

$175 USD in 3 days
(112 Reviews)
7.3
gangabass

I have huge Python and web scraping experience (check my reviews) that's why I'm sure that you'll be impressed with my work. I can provide you a Python Selenium based (but there is no need to use it for above website) More

$200 USD in 2 days
(421 Reviews)
7.4
lkhelladi

Hello, I'd be glad to develop a web scraping tool that retrieves stock exchange data as per your requirements. The tool will be developped in Python using the powerful libraries of BS4 and selnium headless chrome dr More

$95 USD in 2 days
(157 Reviews)
7.2
etuannv

Hi there, I am interested in your project. I would approach your project by making a program to collect the data automatically. The program will be written in Python with Selenium or Scrapy. It could be run on any oper More

$85 USD in 3 days
(93 Reviews)
6.7
mananraja

hey, I have checked the website you mentioned. I can make a PYTHON script to extract the data you mentioned in your description. Would like to talk more details with you in chat. Thanks!

$70 USD in 1 day
(167 Reviews)
6.4
zeke

I have done many similar projects related to web scraping information from different websites. Very interested to work on this project. I am absolutely confident I can finish this work on time and on budget to highest More

$100 USD in 5 days
(67 Reviews)
6.4
logos106

Hi! Call me Wang! I am here to work for you! It's just me whom you are looking for! I can complete your project perfectly ASAP. Skills that you require are owned by me so you don't worry about that. The thing that I wo More

$140 USD in 7 days
(31 Reviews)
5.6
marcnealer

Hi I'm a python Coder with experience of Selenium, Requests, and BS4. I have have done a number of scraping projects in the past, and am very familiar with html and Javascript frameworks, thus can decode what is need More

$250 USD in 5 days
(10 Reviews)
5.8
topexpert713

Hi, Nice to meet you! I have read your requirements carefully and I am very interested in your project. I am confident of this project as I'm a professional Python expert with over 5 years of experience. It seems to b More

$140 USD in 7 days
(42 Reviews)
6.0
khoadeveloper

Hi, Dear client. I am a web scraping/automation/bot/chrome extension expert with 6 years of experience in NodeJS, Puppeteer, Chromy, Selenium, Chrome-extension, Chrome-headless, Firefox-headless, SilmerJS, PhantomJS, More

$250 USD in 1 day
(18 Reviews)
5.6
asifkhan268

Hi there, Thank You for reviewing my job proposal. As per your job description I am glad to let you know that I can create a python selenium script to scrap required data from the above mentioned website URL. I h More

$55 USD in 1 day
(34 Reviews)
5.4
kunitsynartem

Hi! I have looked your website and I can assure you I can extract all info you need from [login to view URL] without using Selenium. This may be simple Python + requests + BeautifulSoup script or Python + Scrapy script. I can ex More

$140 USD in 1 day
(35 Reviews)
5.5
milossimicms

Hi! I've done a lot of scraping projects using precisely python 2.x (and 3.x), Selenium and bs4 (but have also worked with Scrapy and plain requests to extract the data from the sites). For example, I scraped the dat More

$134 USD in 4 days
(17 Reviews)
4.9
ChanakyaNaag

Hello there! I understand your requirements of scraping stocks data and willing to work on it. I have been working on python, selenium and beautiful soup for almost 3 years now. I have worked on complex stock data s More

$150 USD in 4 days
(42 Reviews)
5.1
Kane96419

Dear Thanks for your posting. My name is Ze S. I have read your proposal carefully and understood your needs clearly. I am a member of a team for developing cloud solution and have 10+ years experience to develop lot' More

$200 USD in 7 days
(17 Reviews)
4.5
oswaldoodavidal

Hello, I have experience in web scraping with Python. I can use Selenium, Scrapy, BeautifulSoup and Requests to make the best web scrapers! This is my quote for the example that you state on your description. I hope More

$210 USD in 7 days
(5 Reviews)
4.4
alexixiano63

hi how are you? I checked your description carefully I am expert in scraping. Please contact me and let 's discuss more in detail. Thank you.

$140 USD in 7 days
(4 Reviews)
4.6
jinridong19713

Hello, sir Thank you for your job posting. I read your project description and I am an expert with your required skills. I have read your post. I am an expert who have many experiences in Web scraping with python selen More

$200 USD in 7 days
(18 Reviews)
4.9
yashashahj1397

Hi,I saw your post regarding Automation. I have done many projects based on Automation and scraping I am IT, engineer. And my primary skills are Python(Selenium, Beutifulsoup, scrappy, pandas),UIpath. I have made many More

$140 USD in 7 days
(11 Reviews)
4.3
AntonKozel

Hello, dear, Thanks for posting your job. Please check my review. I have many years of work experience in php, python, scrapy, Selenium Webdriver, BeautifulSoup, Scrapy, [login to view URL], Database(Mysql, Mssql...), Data Minin More

$140 USD in 3 days
(8 Reviews)
4.1