Fortran clustering for making groups if related terms.

Avg Bid (USD)
Project Budget (USD)
$250 - $750

Project Description:
Write custom clustering code for making groups if related terms. Note, this is the same as this project:
but for several reasons, I prefer it written in Fortran. Please bid on this if you have strong fortran experience, and also with clustering algorithms.

The source information will be a set of N-dimensional vectors, where N is a set of words that often appear in the same paragraphs as other words. The input are topics generated from a proprietary corpus using latent Dirichlet allocation (LDA). We currently have a dozen vectors (each vector is a topic from LDA), and N ~= 300. We have a simple file format delimitated with newlines, "|" and ";".

Code should be in Fortran. You will probably use Group Average Agglomerative Clusterer. We used python NLTK as a proof of concept, and we had preliminary success. You can see our simple python. There will be additional weighting information, as we have additional data about the weights of some of the other N words between eachother. The algorithm is intended to have the degree of clustering depend on the initial similarity of the clusters.

There will be 5, tightly related tasks:

1) Write compiled code for merging our source vectors.
The result will be analogous to our python NLTK sample.

2) Add weighting information we provide. (We have weighting scores for some of the N terms, which will cause any cluster they are in to be more or less important.). Specifically, we have 100 themes. Example themes are "sports" and "food". We know that the word "apple" has a high weight for the "food" theme, and a low score for the "sports" theme. Therefore a cluster containing [apple, THEME:sports] would be weighted lower than a cluster containing [apple, THEME:food].

3) Adjust similarities for a subset M of N terms, so they are less likely to be combined. For example, if M = [orange, apple], then two sets [orange, banana] and [pear, apple] would be considered more distant. (not the subset M is the same as the THEMES in #2). Not all M have different relationships. Some are negative or positive. e.g., food:sports = -1; but computer:science = 0.8. We will provide a list.

4) Add information from an additional set of W vectors. These vectors are sets of terms extracted from Wikipedia. For example, a vector in W would be all the outgoing links from a wikipedia article, with higher weights depending on their closeness to the start of the wikipedia article.

5) Filter to omit stopwords (will be provided), irrelevant parts of speech (tbd), duplicates (i.e., no word should be in >1 final cluster), and low-probability groups (eliminated).

The output will be a list of potentially related terms.

Skills required:
Algorithm, Fortran, Mathematics, Natural Language, Software Architecture
About the employer:
Public Clarification Board
Bids are hidden by the project creator. Log in as the employer to view bids or to bid on this project.
You will not be able to bid on this project if you are not qualified in one of the job categories. To see your qualifications click here.

Hire renardpaul
$ 1600
in 21 days
Hire tulebaev
$ 400
in 10 days
$ 1030
in 30 days
Hire mathworker
$ 400
in 7 days
$ 1546
in 14 days
$ 444
in 15 days