Moving a python web parsing function to Scrapy
£20-250 GBP
Paid on delivery
Dear freelancers,
I have a simple python program that can parse and extract data from one website and store the data into mongoDB and send an email to notify.
The source code is quite simple (core functions are just about 150 lines of code) and is based on python requests.
It only extracts certain data from fixed pages/links and will not follow, so it is a not a web-crawler. I have attached two of the core functions, as you can see, they are very simple.
And it is single threaded. It works quite well and stable but is a bit slow. As I need to monitor more pages, so I am planning to move it to Scrapy to speed up the program.
You don’t need to change the functions of web parsing, database or sending email. The focus is just to use the existing functions in Scrapy to improve the speed and performance.
If you need more information, please feel free to contact me. Thank you for your attention.
Project ID: #18744199
About the project
23 freelancers are bidding on average £151 for this job
Hi If you care about performance speed. I can implement a "multi-threading" desktop application in C# which do the exact as python. Can work on a demo if you like. Thanks
Hello, Dear I have checked your project description and am ready for discussing with you about project for now. I have experienced in developing Python, Web Scraping. I will work very hard and best for you for 5 ye More
i am scrapy expert............................................................................................................................
Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. More
Hi there, I have read your project description and i'm confident i can do this project for you perfectly.I still have a few questions. please leave a message on my chat so we can discuss the budget and deadline of the More
Hello How are you My name is Xu i have full time and I can start to work immediately Please contact me and do let us discuss about your project Thanks for your posting
can build your functionality with Scrapy to crawl across the pages. can do your job. can integrate with existing script These are my skills related to web scraping and crawling Have done scraping in CasperJS Phantom More
Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO More
Hello! I've just seen you python code and project description. These 2 functions are just part of the workwflow of your one-threaded app. By my opinion it is not the best choice to simply import them into Scrapy, becau More
i have more than 3 years experience in python development and worked with more than 20 applications . i have checked your job description and interested in this job . feel free to contact me for any consulting solution More
Hi I'm a python developer with 5 years of experience working on scrapping projects . I can translate the requests script to scrapy and get it up and running . can you please send me the script you want to convert ? More
Server Backend, Machine Learning, Data Science applications. - Python - Django, Flask, Asyncio, Celery - Machine Learning and Big data applications - SciPy (Numpy, Pandas, scikit-learn, Jupyter) - Node.js, puppeteer - More
Hi, Hope you are doing good today! I am a software developer and have experience with Threading and Data Structure and algorithms. I can help you to make your script faster. Please share your code details s More
Hello How are you? I have read your job description. I am good on python, django web scraping , would like to check your source code, and as possible as I can help you to speed up I am ready to start work, Looking More
Hello, I read Job description very carefully and I'm very interested in the job. I have 2 years experience in web Scrapy using Scrapy framework, Beautifulsoup, selenium, Phantomjs. I will set up your existing project More
I have a good experience of Python. I am working on Python from last 2.5 Years and have awesome grief on Python Modules.
