Package io.opsit.explang.autosuggest

Class Tokenization

java.lang.Object

io.opsit.explang.autosuggest.Tokenization

public class Tokenization
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
String	token
Integer	tokenIndex
int	tokenPos
List<LanguageToken>	tokens

Constructor Summary

Constructors

Constructor	Description
Tokenization()	Construct empty structure.
Tokenization(List<LanguageToken> tokens, Integer tokenIndex, String token, int tokenPos)	Construct Tokenization structure for parsed tokens.

Method Summary

Modifier and Type	Method	Description
String	toString()	Build string representation fit for debug output.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

token

public String token

tokenPos

public int tokenPos

tokens

public List<LanguageToken> tokens

tokenIndex

public Integer tokenIndex

Constructor Detail

Tokenization

public Tokenization()

Construct empty structure.

Tokenization

public Tokenization(List<LanguageToken> tokens,
                    Integer tokenIndex,
                    String token,
                    int tokenPos)

Construct Tokenization structure for parsed tokens.

Method Detail

toString

public String toString()

Build string representation fit for debug output.

Overrides:

toString in class Object