edu.cmu.minorthird.text
Class CompoundTokenizer

java.lang.Object
  edu.cmu.minorthird.text.CompoundTokenizer

All Implemented Interfaces:

Tokenizer

Direct Known Subclasses:

FilterTokenizer, SpanTypeTokenizer

public abstract class CompoundTokenizer
extends java.lang.Object
implements Tokenizer

Field Summary

protected Tokenizer

parentTokenizer

Constructor Summary

CompoundTokenizer()

Method Summary

Tokenizer	getParent()
abstract TextToken[]	splitIntoTokens(Document document) Tokenize a document.
abstract java.lang.String[]	splitIntoTokens(java.lang.String string) Tokenize a string.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

parentTokenizer

protected Tokenizer parentTokenizer

Constructor Detail

CompoundTokenizer

public CompoundTokenizer()

Method Detail

getParent

public Tokenizer getParent()

splitIntoTokens

public abstract java.lang.String[] splitIntoTokens(java.lang.String string)

Description copied from interface: Tokenizer

Tokenize a string.

Specified by:

splitIntoTokens in interface Tokenizer

splitIntoTokens

public abstract TextToken[] splitIntoTokens(Document document)

Description copied from interface: Tokenizer

Tokenize a document.

Specified by:

splitIntoTokens in interface Tokenizer