es.nlp.uned.weps.evaluation
Class SystemScorer

java.lang.Object
  extended by es.nlp.uned.weps.evaluation.SystemScorer

public class SystemScorer
extends java.lang.Object

The Class MultiSystemScorer evaluates a set of systems and outputs the final ranking.


Constructor Summary
SystemScorer(java.io.File keysDir, java.io.File systemsAnswersDir, java.io.File outputDir, java.util.ArrayList<Measure> measures, java.util.HashSet<java.lang.String> baselines, boolean overwriteEvaluations)
          The Constructor for scoring multiple clustering problems and their corresponding solutions.
 
Method Summary
 TeamEvaluation evaluateBaseline(java.lang.String baselineType)
          Generates a baseline answer and evaluates it with the key.
 TeamEvaluation evaluateTeam(java.lang.String team_id, java.io.File teamAnswersDir)
          Evaluate team using all the clustering problems.
 double[] getAveragedScores(java.io.File evalFile)
          Gets the averaged evaluation for a particular team.
 java.util.HashMap<java.lang.String,TeamEvaluation> getBaselineEvaluations(java.util.HashSet<java.lang.String> baselines, java.io.File outputDir, boolean overwriteEvaluations)
           
 java.lang.String[] getColumnLabels(java.io.File evalFile)
           
 java.util.HashMap<java.lang.String,TeamEvaluation> getEvaluations(java.io.File systemsAnswersDir, java.io.File outputDir, boolean overwriteEvaluations)
           
static void main(java.lang.String[] args)
          The main method.
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

SystemScorer

public SystemScorer(java.io.File keysDir,
                    java.io.File systemsAnswersDir,
                    java.io.File outputDir,
                    java.util.ArrayList<Measure> measures,
                    java.util.HashSet<java.lang.String> baselines,
                    boolean overwriteEvaluations)
             throws java.io.IOException
The Constructor for scoring multiple clustering problems and their corresponding solutions.

Parameters:
keys_path - the directory that contains all the key clustering files (following the WePS 2007 format)
Throws:
java.io.IOException
Method Detail

evaluateBaseline

public TeamEvaluation evaluateBaseline(java.lang.String baselineType)
Generates a baseline answer and evaluates it with the key.

Parameters:
baseline_id - the baseline_id
Returns:
the team evaluation

evaluateTeam

public TeamEvaluation evaluateTeam(java.lang.String team_id,
                                   java.io.File teamAnswersDir)
Evaluate team using all the clustering problems.

Parameters:
answers_path - the answers_path
team_id - the team_id
Returns:
the team evaluation

getEvaluations

public java.util.HashMap<java.lang.String,TeamEvaluation> getEvaluations(java.io.File systemsAnswersDir,
                                                                         java.io.File outputDir,
                                                                         boolean overwriteEvaluations)
                                                                  throws java.io.IOException
Throws:
java.io.IOException

getBaselineEvaluations

public java.util.HashMap<java.lang.String,TeamEvaluation> getBaselineEvaluations(java.util.HashSet<java.lang.String> baselines,
                                                                                 java.io.File outputDir,
                                                                                 boolean overwriteEvaluations)
                                                                          throws java.io.IOException
Throws:
java.io.IOException

getAveragedScores

public double[] getAveragedScores(java.io.File evalFile)
                           throws java.io.IOException
Gets the averaged evaluation for a particular team.

Parameters:
eval_path - the eval_path
Returns:
the averaged evaluation
Throws:
java.io.IOException - the IO exception

getColumnLabels

public java.lang.String[] getColumnLabels(java.io.File evalFile)
                                   throws java.io.IOException
Throws:
java.io.IOException

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception
The main method.

Parameters:
args - the args
Throws:
java.lang.Exception - the exception