I need to build a web spider on [url removed, login to view] 2005, the whole application is already done and working but is using a free ActiveX from <[url removed, login to view]> . The problem is that this component is too slow, that's why I want to create my own version. The new version don't need to do exactly the same, I only need to extract all the links, and I have the code to extract the TITLE and META tags
The 2nd part of the project is to replace the old ActiveX with the new .net component with the abilitie of using multithreading for the crawler so it can spider maybe 2 or 3 urls at one time.
Is very important a good knowledge of thread programming, the program show a simple listbox where the user add urls then use checkboxes of the listbox control to select one or many urls to spider, that's why I need threads, then appear a button to cancel the crawler, all the page found are saving to a access DB, all these is already done, your part is to create the spider and the threads based from the user selection on the listbox
I have more than the half of the spider already done, anyway please post if you are interested, and a estimate amount for this.
1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done.
2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables):
a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment.
b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request.
3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement).
.net framework 2.0
[url removed, login to view]