A simple as possible program is needed that extracts specific info from a URL search engine through Java written code and stores it into a simple database in MySQL. I have some broad requirements that are posted below to give an idea of what exactly needs to be done as well as some given code to use in java. I also have some links I can send you privately on examples of what I need. I must stress I DO NOT need a super amazing program written, I'd prefer the project to be completed under the simplest possible terms, I can discuss with you exactly what I mean by this via private messages. Also not ALL the requirements need to be 100% COMPLETELY met, but this can vary. Again, we can discuss this further in detail via private message if you are interested of course. Feel free to ask me any specific questions you have regarding this project.
- Should have the means to take a URL, retrieve the webpage, separate
the displayed text and images in separate files and store the pathways/file names in a database
associated with the URL and/or keywords and associate a timestamp for that information for further
- Demonstrate that it can interact with some website that has a search
engine that when supplied with keywords (provided by your system) will return a number of URLS
related to the search. You are to extract these URLS and obtain all of their
webpages and store the relevant information to the database.
- Integrate a distance function that when querying the database for URLS associated
with the keywords based on distance from a given location to the location associated with the webpages. For any webpage where location info is not available, assign a random zip code to the webpage in order to perform the function. (THIS REQUIREMENT IS VERY BROAD AND CAN VARY ON HOW OR IF YOU INTEGRATE IT, we can talk specifically more about this privately)
- Provide a user-manual (screenshots with descriptions what to do perhaps). Provide a README that explains how to compile and run your program from command line. (AGAIN this requirement can vary)
- Comment major parts of code, comments classes/methods/variables.
13 freelancers are bidding on average $186 for this job
Hello, I have created scrappers for Amzon.com, google.com, alibaba.com and may more sites using java. I can show my work if u like to see. I use java with selenium. I really can help you.Thanks
I'm a certified Java expert and would love a whack at this project. This looks right up my alley and I don't think I would have any issues completing it. Read PM for further details. Cheers, Aaron