URL information extraction

  • Status Completed
  • Budget $30 - $250 USD
  • Total Bids 14

Project Description

A simple as possible program is needed that extracts specific info from a URL search engine through Java written code and stores it into a simple database in MySQL. I have some broad requirements that are posted below to give an idea of what exactly needs to be done as well as some given code to use in java. I also have some links I can send you privately on examples of what I need. I must stress I DO NOT need a super amazing program written, I'd prefer the project to be completed under the simplest possible terms, I can discuss with you exactly what I mean by this via private messages. Also not ALL the requirements need to be 100% COMPLETELY met, but this can vary. Again, we can discuss this further in detail via private message if you are interested of course. Feel free to ask me any specific questions you have regarding this project.

General Requirements:

- Should have the means to take a URL, retrieve the webpage, separate

the displayed text and images in separate files and store the pathways/file names in a database

associated with the URL and/or keywords and associate a timestamp for that information for further

possible query.

- Demonstrate that it can interact with some website that has a search

engine that when supplied with keywords (provided by your system) will return a number of URLS

related to the search. You are to extract these URLS and obtain all of their

webpages and store the relevant information to the database.

- Integrate a distance function that when querying the database for URLS associated

with the keywords based on distance from a given location to the location associated with the webpages. For any webpage where location info is not available, assign a random zip code to the webpage in order to perform the function. (THIS REQUIREMENT IS VERY BROAD AND CAN VARY ON HOW OR IF YOU INTEGRATE IT, we can talk specifically more about this privately)

- Provide a user-manual (screenshots with descriptions what to do perhaps). Provide a README that explains how to compile and run your program from command line. (AGAIN this requirement can vary)

- Comment major parts of code, comments classes/methods/variables.

Get free quotes for a project like this
Completed by:
Skills Required

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online