This project will be awarded to PigtailXL user. One of our employees approached him for a project called "Build a semi-automated scrapper with user input" - [url removed, login to view] . Our employee did the research and did it on our behalf. Now we can go ahead and do the project.
Original project post:
I have a client and they want to be able to create a new lead entry to Dynamics 365 by scrapping the content from [url removed, login to view] using the "Entity Name" from [url removed, login to view] (please be aware that these 2 websites are only accessible from the US or using a VPN connection).
The scrapping process will work like this:
1. user goes to [url removed, login to view], selects manually "Corporate and LLC", "Corporate" or "LLC"
2. user inputs the "SEARCH METHOD" and the "SEARCH" fields
3. the scrapper uses the company name from the bottom of "Corporation/LLC Name" and gets to the "CORPORATION FILE DETAIL REPORT"
4. the scrapper copies the "Entity Name" from the right site of the table (let's name this "copied field"
5. the scrapper goes to [url removed, login to view]
6. the user validates the Google reCaptcha
7. the scrapper searches in the "Employer" tab for state "Illinois", current date and "copied field" from the previous website, then click below "Search", then click on the search result
8. the scrapper copies the text that is below each of the following: "Insurance Coverage Provider", "Policy Number" and "Coverage Date"
9. the scrapper then uses Microsoft Dynamics 365 Web API to create a new lead inside a Dynamics 365 account that is supplied
I also attached images for all 9 steps.