2. circumvent anti-scraping techniques
3. Scalable - since every user who logs in the front end will create an instance of the scraper
Frontend controls -> Scraper Middleware -> Airline site
- Thus the middleware should scrape/parse the necessary data and send it to the front end in real time.
Flight data to be provided - airport code, domestic/international, # passengers, date of departure, date of arrival
The use case for the scraper middleware is:
1. User interacts with front end to search for a flight
2. Front end triggers scraper middleware to run (a headless browser/selenium/etc).
3. Scraper middleware will then wait for the results to load and pass back the data to the front end.
4. User in front end selects flights to book and triggers scraper to load these info to the airline site.
5. Airline site provides confirmation and PDF tickets.
6. Scraper middleware passes back these data to the front end.
Before you submit a bid, please send me a proposed system architecture/framework for the scraper. Please include services/database tools etc.
12 freelancers are bidding on average $732 for this job
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi