Please read and approve this project feature scope:
A client wants a webcrawler capable of scraping / scanning websites to look for email addresses and save them in a DB (MySQL). It should be possible to set filters and selections like country and keywords to scan for (these options should be available in an admin section for them to update and add later). Also, an option should be to type in a URL in the admin backend and then the crawler would scrape that URL for email addresses.
1. Domain lists (csv files) provided by employer will be imported on a daily basis into DB.
2. User sets keywords (Add and edit keywords via admin panel password protected) to be used in filtering the websites to scan. Set country filter via same admin panel.
Filters to be configured by user:
- 1. Keywords filter from keywords list (ON/OFF)
- 2. Country filter (all countries or choose 1 country)
- 3. Domain filter (all domains or choose 1 domain)
3. Webscraper searches through the domains (from the list) using keywords and finds any email addresses from the website pages. Saves the emails in db and shows the results in 1 live in the admin backend with DATE, FILTERS, NUMBER OF EMAILS in csv so they are easy to download.
4. It's important that the results page is made simple and easy to use. A simple list in the admin backend showing DATE, FILTERS, NUMBER OF EMAILS from the search and a download link for the CSV file from the search.
23 freelancers are bidding on average $179 for this job
Hi, i am expert in data crawling and web scraping and i have done a lot of similar projects. Please contact me so we can discuss the project and take it to next level. Regards Umar
Dear Sir, we made lots of crawler before also we have your requirement same work we did before also one time give us chance to work for you we will have long term business relations Thanks
Hello, I already have created many crawler, I have recently created crawler for phone number can create desktop application for you. ping me so we can discuss and I can show case my work.
Hello, I can do the job, please send me a message so we can discuss in detail, i have 7+ years of experience for the same, i have scraped data from 40+ websites. Thanks!