edu.cmu.minorthird.text
Class SplitTokenizer

java.lang.Object
  edu.cmu.minorthird.text.SplitTokenizer

All Implemented Interfaces:

Tokenizer

public class SplitTokenizer
extends java.lang.Object
implements Tokenizer

Constructor Summary

SplitTokenizer(java.lang.String s)

Method Summary

java.lang.String	getSplitString()
TextToken[]	splitIntoTokens(Document document) Tokenize a document
java.lang.String[]	splitIntoTokens(java.lang.String string) Tokenize a string.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SplitTokenizer

public SplitTokenizer(java.lang.String s)

Method Detail

getSplitString

public java.lang.String getSplitString()

splitIntoTokens

public java.lang.String[] splitIntoTokens(java.lang.String string)

Description copied from interface: Tokenizer

Tokenize a string.

Specified by:

splitIntoTokens in interface Tokenizer

splitIntoTokens

public TextToken[] splitIntoTokens(Document document)

Tokenize a document

Specified by:

splitIntoTokens in interface Tokenizer