[login to view URL] should fetch multiple urls from mysql db(urls are manually product matched and updated in product_urls table) and crawl one by one based on competitor and product id and the parsed data should be inserted in competitors_price table.(done using native query but should be implemented using django models)

on completion scrapy should notify success/failure to backend server using post rest api.

scrapyd must be implemented over scrapy

django rest should call the scrapyd passing competitorID and productID as parameters

Skills: Django, Elasticsearch, NoSQL Couch & Mongo, Python, Web Scraping

See more: web scraping visual basic, web scraping flash sites, web scraping jobs, google base web scraping, data mining web scraping, web scraping projects, web scraping practice projects, web scraping experience, web scraping password protected, php web scraping expert needed, nutch web scraping, web scraping program, free web scraping software, web scraping source, web scraping using excel vba, creating web scraping bot, web scraping autotrader, kayak web scraping script, web scraping download file

About the Employer:
( 0 reviews ) Chennai, India

Project ID: #19942942

7 freelancers are bidding on average ₹9833 for this job


can build this framework in scrapy Django and scrapyd. can build API to trigger job for scraping. These are my skills related to web scraping and crawling Have done scraping in CasperJS Phantomjs, python. Have don More

₹22222 INR in 5 days
(41 Reviews)

Hi, Harshar. Why did you delete the job? I am bidding again for your work. Is it yours? I am ready to get started now. Do not forget to create milestone. Without no freelancer can start to work. Thanks.

₹10000 INR in 3 days
(12 Reviews)

Dear , ** I am software developer. I'm very interested in your project. I worked too many jobs before and have confidence in web scrapping. I have ever scrapped Gumtree, yelp, bet365, 1xbet, etc... let's discuss more More

₹12500 INR in 7 days
(3 Reviews)

Hi I can help you to do your interesting work. And here is my skill: ◆Web Development: Websites Back and Frontend◆ .NET Framework 2.0/3.0/3.5/4.0/4.5 ASP.NET Webform/MVC/WPF/WCF IIS 6.0/7.0/8.0, Windows2008/ More

₹6000 INR in 3 days
(1 Review)

Hi, this is NIKHIL I am a Team Leader of experienced web & mobile developers with a Bachelor of software from the Pacific National University(PNU) and more than 8+ years of experience in Developing. We have a solid ex More

₹7777 INR in 3 days
(0 Reviews)

"Hi !! Nice to E-meet you We provide services in the field of web-development, web-designing, web-hosting development. My service offering INTERNET Business solutions, E-Commerce Management solutions, Portals & Direct More

₹3333 INR in 3 days
(0 Reviews)

hi I have good profile in upwork please visit and then decide to give me project or not [login to view URL]~012e1921383e5680cf?viewMode=1 Thanks

₹7000 INR in 7 days
(0 Reviews)