Please see below for the details. Can you please get back to me with:
-if this is the sort of thing you can solve effectively
-an estimate of how long this could take
-any other feedback you need from me
Here are the details:
Several users/voters on our website vote on polls. We want the users to each have a score ranging from 0 to 100 (with 2 decimals. Is this range sufficient if there will be 100,000 users?)
The goal is to create one Scoring calculation based on three different variables (ranked in order of importance):
1) # of correct votes (answers)
2) # of votes
3) More recent votes get more weight-age than older votes.
If the users have the same score, the one with more # of votes should rank higher.
We want to avoid users with small number of votes ranking very high. Not fair if one user that has voted only twice and answered both questions correctly ranks higher than a user that has voted 20 times and has been 90% correct. The one with the more # of votes should rank higher.
We also want to avoid users score being penalized (or weighted down) from wrongfully answering questions that took place a long time ago. The user that has evolved into answering questionnaires correctly more often should not be penalized from having a bad score from older votes when he/she started out as an amateur. We don’t want the past holding down onto the future.
Some questions involve the users inputting a number as an estimate (forecast) before the actual event takes place.
We want to calculate how accurate the users Estimates are compared to the actual numbers (once released). This will be a score ranging from -100 to +100 where 0 is the perfect score (estimate = actual).
Things to keep in mind:
The size of the estimates will vary dramatically depending on the event they are voting on. For example, one estimate for an event can be 2.1 whereas another estimate can be 1641. Because of the big range of numbers, this will vary the percentage accuracy of the estimate compared to the actual number. A 2.1 number may vary much more dramatically in percentage terms compared to a number of 1641. Therefore we want a method that compares apples with apples across all our datasets.