We’re looking to develop a script that will work off a dedicated server, it’s aimed at scraping domains and checking whether they are able to be registered or not (Expired) The tool uses Google to scrape data from search results – the script searches Google for a particular search term, then opens all links in the search results and scraps them for url’s.
The script then bulk checks all the scraped domains to check if they are able to be registered or not, the available domain list is then evaluated on its quality. Using API’s from SEOmoz, Ahrefs, Majestic SEO etc – we will be pulling in domain metrics – which will show if the domain is a good option or not.
This script will be used by hundreds of people and needs to be robust with a powerful engine capable of handling thousands of requests simultaneously. We will host this script on whatever server is necessary and we will need to devise a solution to bypass google blacklisting – socks/proxies etc.
You need to have good experience in using API’s and handling domain metrics etc – this products needs to be really polished, we will provide a User interface and you’ll need to integrate it (html/css)
We want this integrated into a WP plugin first.
Please tell us about your experience and please – no copy/paste bids, you’ll be ignored!
13 freelancers are bidding on average $610 for this job
Hi i have a lot of experience scraping google. One solution I can offer you is to use Amazon AWS ec2 server instances as proxies(great for google) although it can be expensive(based on load).