lucene::analysis::WhitespaceTokenizer Class Reference
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. More...
#include <Analyzers.h>
Inheritance diagram for lucene::analysis::WhitespaceTokenizer:
Public Member Functions | |
| WhitespaceTokenizer (lucene::util::Reader *in) | |
| Construct a new WhitespaceTokenizer. | |
| virtual | ~WhitespaceTokenizer () |
Protected Member Functions | |
| bool | isTokenChar (const TCHAR c) const |
| Collects only characters which do not satisfy _istspace. | |
Detailed Description
A WhitespaceTokenizer is a tokenizer that divides text at whitespace.Adjacent sequences of non-Whitespace characters form tokens.
Constructor & Destructor Documentation
| lucene::analysis::WhitespaceTokenizer::WhitespaceTokenizer | ( | lucene::util::Reader * | in | ) |
Construct a new WhitespaceTokenizer.
| virtual lucene::analysis::WhitespaceTokenizer::~WhitespaceTokenizer | ( | ) | [virtual] |
Member Function Documentation
| bool lucene::analysis::WhitespaceTokenizer::isTokenChar | ( | const TCHAR | c | ) | const [protected, virtual] |
Collects only characters which do not satisfy _istspace.
Implements lucene::analysis::CharTokenizer.
The documentation for this class was generated from the following file: