I need someone that can make a custom bot. I use Windows and can run Perl, .Net, etc. I do not want it to be web based as that has proved to be slower but if you insist, I'll give it a try.
Here's how it could work.
In order to extract the data, you first have to have an active session with the website. That means I need a window (or a portion of one that I can visibility see) that stays open that I log-in. (I'll keep this window open and occasionally click around to keep the session active.)
I need the ability to load a list of URL's to extract data. I am OK with either the text being extracted or the code. Which ever is faster. It needs to be saved in CSV format. The HTML code is at the bottom and runs from line 156-228 on the page. Each page is the same.
Notice in the code that each field has a field name (ie: Name, Address, etc). I want to keep these in place so that it will look like "Name: Chris Jones". That way, when I sort it it will help better organize the data in case things get shifted around.
Each field name will needs its own column in CSV. Name, Address, City, State, Zip, etc.
This will need to support multiple threads since it needs to be fast.
I do not need to download any images or other files. I just need the text from each page.
Each URL is in this format: [url removed, login to view]
PS - Due to the nature of the data, I cannot provide a link for you to test. I realize this impacts the job but I cannot give out the data.
SAMPLE CODE ATTACHED -- (Runs from line 156-228)
Looking to make some money?
- Set your budget and the time frame
- Outline your proposal
- Get paid for your work
Bids on this Project
I am an individual freelancer, not a company. I can convert your manual tasks into a program using Perl/PHP/Python scripting or Java, C# program. Below are few example that goes with me. ** Writing crawler, spider, scrapper to parse/mine data from any website. For example parsing yellowpages, ecomerce sites, blogs, forums, video sites, etc. ** Making a bot easily(with proxy support) for websites to comment, follow, friend request, forum posting, etc. Example sites Facebook, Twitter, Youtube, Pinterest. ** Making email parser using POP3/IMAP or directly from yahoo, gmail, hotmail, etc sites. ** API integration from any service providers like amazon, ebay, youtube, google, or any ecommerce sites for product. ** Creating downloader/grabber for any website like keepvid or clipnabber.
I am a software analyst and developer in Java, PHP, C/C++. Great experience with web development and databases.
I believe in Agile methodology and practices the same for past 4+ years and adhere to its ten principles. I always love meeting deadlines or deliver earlier, welcome change in requirements, even in late development. Makes sure to deliver working software frequently (in few weeks) with close, daily cooperation with customers. Always love to talk one-to-one with customer about the project and working software is considered as the principal measure of progress. Using tools to track distraction, I am able to maintain a constant pace & sustainable development. I update myself in latest technologies for excellence and believe in simplicity. With 12+ years of experience as developer, I am able to provide best solution, in minimum time possible. My area of expertise are Python, Django, Flask, Perl, Dancer, PHP, CodeIgniter, MySQL, PayPal, IPN, JQuery, Angular, D3, MySQL, PostgreSQL, SQLite, Oracle, MSSQL & many more
New Delhi, India
I'm a professional in LAMP(Perl+PHP) and also windows GUI applications too. 1. Developed many websites in perl and php both. 2. Developed GUI apps in perl, dot net. 3. Crawled lots of websites using perl. 4. Hosted many sites, maintained servers also.
We are a team of technocrats with expertise in : - Perl - PHP - Python - SQLite - MySQL - PostgreSQL -JQuery -HTML5 -AmCharts -Highcharts -Payment Gateway Integrations