Scraping Expert for XML Sitemap Generator
Budget $250-750 USD
I would like to build an XML sitemap generator in either Python or other language that is preferred by the developer for this type of project.
Here are the features I would like to have for the generator:
- Pattern-matching to check for parallel pages in other languages/locale
- Configurable patterns based on the site URLs
- Ability to configure to crawl only specific directories
- Ability to disallow from crawling parameters
- Cloud-based GUI to add URL and start crawl (I can host on my VPS)
- Menu to export crawl results, configure crawl URLs, disallow parameters, etc. (See Screaming Frog)
- Crawl large sites up to 1,000,000 pages
- Check http status codes and ensure all 200 status
- Only add https URLs to sitemap and if http, check https for a page to add
- Check box for adding video, images, etc. to sitemap either combined or separate sitemaps
- Build sitemap index for large sites and then link off to each associated locale sitemap
- Option to divide sitemap based on URL patterns (Have a certain directory or part of the site in each sitemap)
- No more than 50,000 URLs per sitemap
15 freelancers are bidding on average $484 for this job
We can take your work for fulltime, as we can provide you with the dedicated react native developer for your project. We are ready to start as soon as we discuss the details. We can provide you with the quality remote More
Hi. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spiders I wor More
Greetings, I am an experienced professional scrapper and have done similar projects in the past. Same can be verified from my profile. Let me allow to assist you with your requirements. Thanks
Hi sir, I am expert in scrapping data and made scrappers for Amazon, eBay, Adidas, and Nike. I made successful scrappers which are running 24/7 and downloading data for my clients. I need some more information from yo More
Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO More
Hi, I am 10 years experienced scraping and developing expert. I have done hundreds of similar projects. Accuracy is my main goal! Hire me and you will be satisfied for 100%!
Hi Employer, I can also extract any data from any [login to view URL] I have scraped the data from these websites & stored scraped data into CSVs and excels as per my clients requirement & I have made desktop applicatio More
Hi, This is Vipin.I have 6+ years of experience in customized software,web scrapping, web and mobile app development and has expertise in Android, iOS, Python, Magento, PHP, HTML, Java, Angular and Ionic framework. More
Dear Prospect Hiring Manager. Thank you for giving me a chance to bid on your project. i am a serious bidder here and i have already worked on a similar project before and can deliver as u have mentioned "I can do th More
Hello Project manager I have more than 7 years of experience in Android, Iphone, Mobile Applications,Python , PHP, Ionic ,React native ,Angular JS, WORDPRESS, , HTML5,CSS, Ajax, jquery, JavaScript, Mysql, Graphic Des More
