Closed

Create me a [login to view URL] file to delay crawlers from pinterest and optimise my server

Pinterest crawlers have brought my dedicated server to its knees with crawling it

I don't want to block the crawlers because i will be penalised i want someone to

make a [login to view URL] file with a crawl delay witch will let my traffic threw as well as pinterest been able to crawl. So that means you will have to copy the [login to view URL] file in to the cpanel root directory of every cpanel and then i want you to copy it to every addon domain folder. This can be accomplished in bulk instead of doing it manually

Skills: MySQL, PHP, Web Hosting

See more: excel macro create txt file, create txt file html javascript, create txt file perl, create txt file cell excel macro, crawlers plugin wordpress robots txt, create write txt file windows application asp net, excel create txt file, java create binary tree txt file, create linked list given txt file java, robots.txt file, download robots.txt file, how to create robots.txt file for seo, google couldn t crawl your site because we were unable to access your site's robots txt file, how to create a txt file in android, how to create a txt file, robots.txt crawl-delay, has a data discrepancy further analysis blocked by a robots txt file, how to create robots.txt in wordpress, best robots txt file for opencart, vba how to create a txt file

About the Employer:
( 17 reviews ) Leeds, United Kingdom

Project ID: #19970521

8 freelancers are bidding on average £16 for this job

vyasrohan

Hello I will add Crawl delay for Pinterest to your [login to view URL] file. Then I will upload it to your main and addon domains. We just need to add this line to them: User-agent: Pinterest *Crawl-delay: 50 Let me know an More

£15 GBP in 1 day
(231 Reviews)
7.8
muhamed3uda

Hello, got it work now , code added and fixed Hello, got it work now , code added and fixed Hello, got it work now , code added and fixed

£20 GBP in 1 day
(90 Reviews)
5.7
yongjin818

Dear As an expert in web scraping, I developed 1000+ spiders using python, scrapy, php,nodejs,c# and proxies. And I scraped data and download images, files for CEOs and output result as csv, json, xml(yellow, ebay, w More

£15 GBP in 7 days
(29 Reviews)
5.5
Zaiyan

Hello, I can do the work and can start working now. Please message me for further discussion. Best Regards, Zaiyan

£10 GBP in 1 day
(37 Reviews)
5.3
bloowebs

Hello, I AM AVAILABLE RIGHT NOW. If you like we can discuss this project. I can do this task and create custom robot file. A few websites I created recently: [login to view URL] https://www.greentechpk. More

£15 GBP in 1 day
(34 Reviews)
4.7
shivani011

Hi, I can build your web site in , WORDPRESS I have more than 10 years of experience in custom web development, project management and planning. In these years I have got a vast experience in gaining the client satisf More

£15 GBP in 7 days
(6 Reviews)
2.7
£23 GBP in 1 day
(7 Reviews)
2.7
sangkall

Hi client! I am going to do your project. I have been make these projects for 2 years. so I would like you to give me this task. Thank you for your invitation.

£18 GBP in 1 day
(0 Reviews)
0.0