edu.cmu.minorthird.classify.experiments
Class Tester

java.lang.Object
  edu.cmu.minorthird.classify.experiments.Tester

public class Tester
extends java.lang.Object

Test a classifier, in a number of ways.

Author:

William Cohen

Constructor Summary

Tester()

Method Summary

static double	errorRate(Classifier c, Dataset d) Return the error rate of a classifier on a dataset.
static Evaluation	evaluate(ClassifierLearner learner, Dataset trainData, Dataset testData) Do a train and test experiment
static Evaluation	evaluate(ClassifierLearner learner, Dataset d, Splitter<Example> splitter) Do some sort of hold-out experiment, as determined by the splitter
static Evaluation	evaluate(SemiSupervisedClassifierLearner learner, SemiSupervisedDataset trainData, SemiSupervisedDataset testData) Do a train and test experiment
static Evaluation	evaluate(SemiSupervisedClassifierLearner learner, SemiSupervisedDataset d, Splitter<Example> splitter) Do some sort of hold-out experiment, as determined by the splitter
static Evaluation	evaluate(SequenceClassifierLearner learner, SequenceDataset trainData, SequenceDataset testData) Do a train and test experiment
static Evaluation	evaluate(SequenceClassifierLearner learner, SequenceDataset d, Splitter<Example[]> splitter) Do some sort of hold-out experiment, as determined by the splitter
static Evaluation	evaluate(StackedBatchClassifierLearner learner, RealRelationalDataset d, Splitter<Example> splitter, java.lang.String stacked) Do some sort of hold-out experiment, as determined by the splitter
static double	logLoss(BinaryClassifier c, Dataset d) Return the average log loss on a dataset.
static double	logLoss(BinaryClassifier c, Example e) Return the log loss on an example with known true class.
static MultiEvaluation	multiEvaluate(ClassifierLearner learner, MultiDataset d, Splitter<MultiExample> splitter) Do some sort of hold-out experiment, as determined by the splitter
static MultiEvaluation	multiEvaluate(ClassifierLearner learner, MultiDataset d, Splitter<MultiExample> splitter, boolean cross) Do some sort of hold-out experiment, as determined by the splitter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tester

public Tester()

Method Detail

evaluate

public static Evaluation evaluate(StackedBatchClassifierLearner learner,
                                  RealRelationalDataset d,
                                  Splitter<Example> splitter,
                                  java.lang.String stacked)

Do some sort of hold-out experiment, as determined by the splitter

evaluate

public static Evaluation evaluate(ClassifierLearner learner,
                                  Dataset d,
                                  Splitter<Example> splitter)

Do some sort of hold-out experiment, as determined by the splitter

multiEvaluate

public static MultiEvaluation multiEvaluate(ClassifierLearner learner,
                                            MultiDataset d,
                                            Splitter<MultiExample> splitter)

Do some sort of hold-out experiment, as determined by the splitter

multiEvaluate

public static MultiEvaluation multiEvaluate(ClassifierLearner learner,
                                            MultiDataset d,
                                            Splitter<MultiExample> splitter,
                                            boolean cross)

Do some sort of hold-out experiment, as determined by the splitter

evaluate

public static Evaluation evaluate(SequenceClassifierLearner learner,
                                  SequenceDataset d,
                                  Splitter<Example[]> splitter)

Do some sort of hold-out experiment, as determined by the splitter

evaluate

public static Evaluation evaluate(SemiSupervisedClassifierLearner learner,
                                  SemiSupervisedDataset d,
                                  Splitter<Example> splitter)

Do some sort of hold-out experiment, as determined by the splitter

evaluate

public static Evaluation evaluate(ClassifierLearner learner,
                                  Dataset trainData,
                                  Dataset testData)

Do a train and test experiment

evaluate

public static Evaluation evaluate(SequenceClassifierLearner learner,
                                  SequenceDataset trainData,
                                  SequenceDataset testData)

Do a train and test experiment

evaluate

public static Evaluation evaluate(SemiSupervisedClassifierLearner learner,
                                  SemiSupervisedDataset trainData,
                                  SemiSupervisedDataset testData)

Do a train and test experiment

logLoss

public static double logLoss(BinaryClassifier c,
                             Example e)

Return the log loss on an example with known true class.

logLoss

public static double logLoss(BinaryClassifier c,
                             Dataset d)

Return the average log loss on a dataset.

errorRate

public static double errorRate(Classifier c,
                               Dataset d)

Return the error rate of a classifier on a dataset.