Purpose & Overview:
- A simple coding project to create a program to scrape a public council website which is freely available.
- I am looking to use your code / program or visual web ripper/autohotkey/jit bit macro recorder project file ONLY so that I may to search upwards to 18,000 search entries similar to the provided sample list.
- I have no/limited Python or any coding knowledge but am keen to learn it if its the most effective way to scrape data websites.
- Requires a person skilled in any data mining / web cawing / Python language techniques to complete this task with a view that raw results will be exported to an Excel Spreadsheet.
- The project is similar to my other one open right now but all I need is the code / program file to run it myself: [url removed, login to view]
A project file that is capable of searching the noted website below using a list of search items I have (sample provided). Project file can by in Visual Web Ripper, Autohotkey, JitBit Macro Recorder, python based or any platform you recommend.
Website / Search Flow from Manual Process:
The following steps outline how a user would manually search a property lot and plan on the website; I want to automate this process.
Step 1. [url removed, login to view]
Step 2. Click the "I Agree" button under the T&C which takes you to this webpage: [url removed, login to view]
Step 3. Click the "Property Enquiry" link to take you this webpage:
[url removed, login to view]
Step 4. Repeatedly search the required list of (1) Lot Numbers and (2) Plan Numbers (a sample excel spreadsheet is provided) under the Real Property Description (RPD) heading at the bottom of the page.
- Then click the "Search" url which takes you to the Properties Found page under the same URL (being [url removed, login to view])
Step 5. Click on each of the Details Button (can range from 1 - potentially 100, but usually around 2-3 results) on each of the Found Properties to take you to this URL for example:
[url removed, login to view]
- Please note this URL's last key number (being "&Key= 869608") changes based on each property searched. However, I cannot find any correlation between the property address and the key displayed in the URL.
I assume the website service monitor's IP addresses who getting all the URLs will probably result in your IP being temporarily banned.
Step 6. Copy and/or export the contents of the Properties Detail page fields including:
- Details (Property Number, Lot/DP, Land Number(s), Description, Ward)
7. Repeat Step 4 - 6 until every item on the search list has been completed.
Conclusions & Required Responses:
- Please respond with what technique you will provide to complete the task, the estimated time you will require to complete the task and an estimate of cost.
- Please contact me should you have any further questions or suggestions regarding price or scope.
10 freelancers are bidding on average $51 for this job
Hi.. Expert web scraper/Data Minor here. Interested in your project. I assure you 100% accurate and good quality work. Ready to start. have a look at PM. Regards