Real time scraper / browser automation middleware
- Status Closed
- Budget $250 - $750 USD
- Total Bids 12
2. circumvent anti-scraping techniques
3. Scalable - since every user who logs in the front end will create an instance of the scraper
Frontend controls -> Scraper Middleware -> Airline site
- Thus the middleware should scrape/parse the necessary data and send it to the front end in real time.
Flight data to be provided - airport code, domestic/international, # passengers, date of departure, date of arrival
The use case for the scraper middleware is:
1. User interacts with front end to search for a flight
2. Front end triggers scraper middleware to run (a headless browser/selenium/etc).
3. Scraper middleware will then wait for the results to load and pass back the data to the front end.
4. User in front end selects flights to book and triggers scraper to load these info to the airline site.
5. Airline site provides confirmation and PDF tickets.
6. Scraper middleware passes back these data to the front end.
Before you submit a bid, please send me a proposed system architecture/framework for the scraper. Please include services/database tools etc.Get free quotes for a project like this
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online