Cancelled

Web Robot - Media Type Indexer

I need a web robot to gather links from web pages and store the information in a database.

Links to HTML Documents and RSS Feeds will be followed and data will be collected about the new document.

The following from each page will be collected:

- Document Type

- Domain Name

- Url String (Relative to the domain)

- Query String

- Date / Time visited

- All outgoing links from that page, indicating the Media Type of the subsequent page (includes type and version where applicable).

Document Types are:

- HTML Document, RSS Feed, Image, Video, email Address, File download, etc..

Must follow 'courteous' robot ettiquette:

- Adheres to [url removed, login to view] inclusions / exclusions

- Browses pages at a comfortable pace - does not overload a single site with multiple hits at once.

- Tracking - knows when a page was last visited, only re-visits a site at a given interval

- Identifies itself properly (configurable agent name)

Technologies:

- Must be built using C#.NET.

- Runs as a service.

- Must use MySQL database.

- May use / refine existing Open Source Software - developer to provide reference to source location and licenses.

Deliverables:

- Working application

- All source code.

Skills: .NET, Windows Desktop

See more: web service developer, web pages html code, web code html, web code developer, type web developer, source code technologies, software developer download, rss indexer, robot name, pace technologies, open source robot, need web application developer, html site technologies, file download service, download mysql developer, developer application web, database technologies, type indexer, new net technologies, open source software download, download web developer, web software developer, web domain, web agent, type pages

About the Employer:
( 7 reviews ) Ajax, Canada

Project ID: #196947

1 freelancer is bidding on average $110 for this job

del3

This will be my first freelance project.so i wont require much of a payment.

$110 USD in 10 days
(0 Reviews)
0.0