Uses of Class
org.apache.lucene.analysis.TokenStream
Packages that use TokenStream
Package
Description
Text analysis.
Analyzer for Arabic.
Analyzer for Bulgarian.
Analyzer for Bengali Language.
Provides various convenience classes for creating boosts on Tokens.
Analyzer for Brazilian Portuguese.
Analyzer for Catalan.
Analyzer for Chinese, Japanese, and Korean, which indexes bigrams.
Analyzer for Sorani Kurdish.
Fast, general-purpose grammar-based tokenizers.
Analyzer for Simplified Chinese, which indexes words.
Construct n-grams for frequently occurring terms and phrases.
A filter that decomposes compound words you find in many Germanic languages into the word parts.
Basic, general-purpose analysis components.
A general-purpose Analyzer that can be created with a builder-style API.
Analyzer for Czech.
Analyzer for Danish.
Analyzer for German.
Analyzer for Greek.
Fast, general-purpose URLs and email addresses tokenizers.
Analyzer for English.
Analyzer for Spanish.
Analyzer for Estonian.
Analyzer for Basque.
Analyzer for Persian.
Analyzer for Finnish.
Analyzer for French.
Analyzer for Irish.
Analyzer for Galician.
Analyzer for Hindi.
Analyzer for Hungarian.
A Java implementation of Hunspell stemming and
spell-checking algorithms (
Hunspell
), and a stemming
TokenFilter (HunspellStemFilter
) based on it.Analyzer for Armenian.
Analysis components based on ICU
Tokenizer that breaks text into words with the Unicode Text Segmentation algorithm.
Analyzer for Indonesian.
Analyzer for Indian languages.
Analyzer for Italian.
Analyzer for Japanese.
Analyzer for Korean.
Analyzer for Lithuanian.
Analyzer for Latvian.
MinHash filtering (for LSH).
Miscellaneous Tokenstreams.
Analyzer for Nepali.
Character n-gram tokenizers and filters.
Analyzer for Dutch.
Analyzer for Norwegian.
Analysis components for path-like strings such as filenames.
Set of components for pattern-based (regex) analysis.
Provides various convenience classes for creating payloads on Tokens.
Analysis components for phonetic search.
Analyzer for Polish.
Analyzer for Portuguese.
Filter to reverse token text.
Analyzer for Romanian.
Analyzer for Russian.
Word n-gram filters.
Analyzer for Serbian.
Fast, general-purpose grammar-based tokenizer
StandardTokenizer
implements the Word Break rules from the
Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29.Stempel: Algorithmic Stemmer
Analyzer for Swedish.
Analysis components for Synonyms.
Analysis components for Synonyms using Word2Vec model.
Analyzer for Tamil.
Analyzer for Telugu Language.
Analyzer for Thai.
Analyzer for Turkish.
Utility functions for text analysis.
Tokenizer that is aware of Wikipedia syntax.
Uses already seen data (the indexed documents) to classify new documents.
The logical representation of a
Document
for indexing and
searching.Taxonomy index implementation using on top of a Directory.
Code to maintain and access indices.
High-performance single-document main memory Apache Lucene fulltext search index.
Misc extensions of the Document/Field API.
Monitoring framework
Intervals queries
This package contains a flexible graph-based proximity query, TermAutomatonQuery, and geospatial
queries.
Highlighting search terms.
Analyzer based autosuggest.
Support for document suggestion
The UnifiedHighlighter -- a flexible highlighter that can get offsets from postings, term
vectors, or analysis.
Some utility classes.
Utility classes for working with token streams as graphs.
-
Uses of TokenStream in org.apache.lucene.analysis
Subclasses of TokenStream in org.apache.lucene.analysisModifier and TypeClassDescriptionprivate static final class
private static class
Token Stream that outputs tokens from a topo sorted graph.final class
This class can be used if the token attributes of a TokenStream are intended to be consumed more than once.class
Abstract base class for TokenFilters that may remove tokens.class
An abstract TokenFilter that exposes its input stream as a graphclass
Normalizes token text to lower case.class
Removes stop words from a token stream.class
A TokenFilter is a TokenStream whose input is another TokenStream.class
A Tokenizer is a TokenStream whose input is a Reader.Fields in org.apache.lucene.analysis declared as TokenStreamModifier and TypeFieldDescriptionprotected final TokenStream
TokenFilter.input
The source of tokens for this filter.protected final TokenStream
Analyzer.TokenStreamComponents.sink
Sink tokenstream, such as the outer tokenfilter decorating the chain.Methods in org.apache.lucene.analysis that return TokenStreamModifier and TypeMethodDescriptionabstract TokenStream
TokenFilterFactory.create
(TokenStream input) Transform the specified input TokenStreamAnalyzer.TokenStreamComponents.getTokenStream()
Returns the sinkTokenStream
protected TokenStream
Analyzer.normalize
(String fieldName, TokenStream in) Wrap the givenTokenStream
in order to apply normalization filters.protected final TokenStream
AnalyzerWrapper.normalize
(String fieldName, TokenStream in) TokenFilterFactory.normalize
(TokenStream input) Normalize the specified input TokenStream While the default implementation returns input unchanged, filters that should be applied at normalization time can delegate tocreate
method.final TokenStream
Analyzer.tokenStream
(String fieldName, Reader reader) Returns a TokenStream suitable forfieldName
, tokenizing the contents ofreader
.final TokenStream
Analyzer.tokenStream
(String fieldName, String text) Returns a TokenStream suitable forfieldName
, tokenizing the contents oftext
.static TokenStream
AutomatonToTokenStream.toTokenStream
(Automaton automaton) converts an automaton into a TokenStream.TokenFilter.unwrap()
protected TokenStream
AnalyzerWrapper.wrapTokenStreamForNormalization
(String fieldName, TokenStream in) Wraps / alters the given TokenStream for normalization purposes, taken from the wrapped Analyzer, to form new components.protected final TokenStream
DelegatingAnalyzerWrapper.wrapTokenStreamForNormalization
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis with parameters of type TokenStreamModifier and TypeMethodDescriptionabstract TokenStream
TokenFilterFactory.create
(TokenStream input) Transform the specified input TokenStreamprotected TokenStream
Analyzer.normalize
(String fieldName, TokenStream in) Wrap the givenTokenStream
in order to apply normalization filters.protected final TokenStream
AnalyzerWrapper.normalize
(String fieldName, TokenStream in) TokenFilterFactory.normalize
(TokenStream input) Normalize the specified input TokenStream While the default implementation returns input unchanged, filters that should be applied at normalization time can delegate tocreate
method.TokenStreamToAutomaton.toAutomaton
(TokenStream in) Pulls the graph (includingPositionLengthAttribute
) from the providedTokenStream
, and creates the corresponding automaton where arcs are bytes (or Unicode code points if unicodeArcs = true) from each term.protected TokenStream
AnalyzerWrapper.wrapTokenStreamForNormalization
(String fieldName, TokenStream in) Wraps / alters the given TokenStream for normalization purposes, taken from the wrapped Analyzer, to form new components.protected final TokenStream
DelegatingAnalyzerWrapper.wrapTokenStreamForNormalization
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis with parameters of type TokenStreamModifierConstructorDescriptionCachingTokenFilter
(TokenStream input) Create a new CachingTokenFilter aroundinput
.Create a newFilteringTokenFilter
.GraphTokenFilter
(TokenStream input) Create a new GraphTokenFilterCreate a new LowerCaseFilter, that normalizes token text to lower case.StopFilter
(TokenStream in, CharArraySet stopWords) Constructs a filter which removes words from the input TokenStream that are named in the Set.protected
TokenFilter
(TokenStream input) Construct a token stream filtering the given input.TokenStreamComponents
(Consumer<Reader> source, TokenStream result) Creates a newAnalyzer.TokenStreamComponents
instance.TokenStreamComponents
(Tokenizer tokenizer, TokenStream result) Creates a newAnalyzer.TokenStreamComponents
instance -
Uses of TokenStream in org.apache.lucene.analysis.ar
Subclasses of TokenStream in org.apache.lucene.analysis.arModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesArabicNormalizer
to normalize the orthography.final class
ATokenFilter
that appliesArabicStemmer
to stem Arabic words..Methods in org.apache.lucene.analysis.ar that return TokenStreamModifier and TypeMethodDescriptionArabicNormalizationFilterFactory.create
(TokenStream input) protected TokenStream
ArabicAnalyzer.normalize
(String fieldName, TokenStream in) ArabicNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.ar with parameters of type TokenStreamModifier and TypeMethodDescriptionArabicNormalizationFilterFactory.create
(TokenStream input) ArabicStemFilterFactory.create
(TokenStream input) protected TokenStream
ArabicAnalyzer.normalize
(String fieldName, TokenStream in) ArabicNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.ar with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.bg
Subclasses of TokenStream in org.apache.lucene.analysis.bgModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesBulgarianStemmer
to stem Bulgarian words.Methods in org.apache.lucene.analysis.bg that return TokenStreamModifier and TypeMethodDescriptionBulgarianStemFilterFactory.create
(TokenStream input) protected TokenStream
BulgarianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.bg with parameters of type TokenStreamModifier and TypeMethodDescriptionBulgarianStemFilterFactory.create
(TokenStream input) protected TokenStream
BulgarianAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.bg with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.bn
Subclasses of TokenStream in org.apache.lucene.analysis.bnModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesBengaliNormalizer
to normalize the orthography.final class
ATokenFilter
that appliesBengaliStemmer
to stem Bengali words.Methods in org.apache.lucene.analysis.bn that return TokenStreamModifier and TypeMethodDescriptionBengaliNormalizationFilterFactory.create
(TokenStream input) BengaliStemFilterFactory.create
(TokenStream input) protected TokenStream
BengaliAnalyzer.normalize
(String fieldName, TokenStream in) BengaliNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.bn with parameters of type TokenStreamModifier and TypeMethodDescriptionBengaliNormalizationFilterFactory.create
(TokenStream input) BengaliStemFilterFactory.create
(TokenStream input) protected TokenStream
BengaliAnalyzer.normalize
(String fieldName, TokenStream in) BengaliNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.bn with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.boost
Subclasses of TokenStream in org.apache.lucene.analysis.boostModifier and TypeClassDescriptionfinal class
Characters before the delimiter are the "token", those after are the boost.Methods in org.apache.lucene.analysis.boost with parameters of type TokenStreamConstructors in org.apache.lucene.analysis.boost with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.br
Subclasses of TokenStream in org.apache.lucene.analysis.brMethods in org.apache.lucene.analysis.br that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
BrazilianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.br with parameters of type TokenStreamModifier and TypeMethodDescriptionBrazilianStemFilterFactory.create
(TokenStream in) protected TokenStream
BrazilianAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.br with parameters of type TokenStreamModifierConstructorDescriptionCreates a new BrazilianStemFilter -
Uses of TokenStream in org.apache.lucene.analysis.ca
Methods in org.apache.lucene.analysis.ca that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
CatalanAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.ca with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
CatalanAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.cjk
Subclasses of TokenStream in org.apache.lucene.analysis.cjkModifier and TypeClassDescriptionfinal class
Forms bigrams of CJK terms that are generated from StandardTokenizer or ICUTokenizer.final class
ATokenFilter
that normalizes CJK width differences: Folds fullwidth ASCII variants into the equivalent basic latin Folds halfwidth Katakana variants into the equivalent kanaMethods in org.apache.lucene.analysis.cjk that return TokenStreamModifier and TypeMethodDescriptionCJKBigramFilterFactory.create
(TokenStream input) CJKWidthFilterFactory.create
(TokenStream input) protected TokenStream
CJKAnalyzer.normalize
(String fieldName, TokenStream in) CJKWidthFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.cjk with parameters of type TokenStreamModifier and TypeMethodDescriptionCJKBigramFilterFactory.create
(TokenStream input) CJKWidthFilterFactory.create
(TokenStream input) protected TokenStream
CJKAnalyzer.normalize
(String fieldName, TokenStream in) CJKWidthFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.cjk with parameters of type TokenStreamModifierConstructorDescriptionCJKBigramFilter
(TokenStream in, int flags) CJKBigramFilter
(TokenStream in, int flags, boolean outputUnigrams) Create a new CJKBigramFilter, specifying which writing systems should be bigrammed, and whether or not unigrams should also be output.CJKWidthFilter
(TokenStream input) -
Uses of TokenStream in org.apache.lucene.analysis.ckb
Subclasses of TokenStream in org.apache.lucene.analysis.ckbModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesSoraniNormalizer
to normalize the orthography.final class
ATokenFilter
that appliesSoraniStemmer
to stem Sorani words.Methods in org.apache.lucene.analysis.ckb that return TokenStreamModifier and TypeMethodDescriptionSoraniNormalizationFilterFactory.create
(TokenStream input) protected TokenStream
SoraniAnalyzer.normalize
(String fieldName, TokenStream in) SoraniNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.ckb with parameters of type TokenStreamModifier and TypeMethodDescriptionSoraniNormalizationFilterFactory.create
(TokenStream input) SoraniStemFilterFactory.create
(TokenStream input) protected TokenStream
SoraniAnalyzer.normalize
(String fieldName, TokenStream in) SoraniNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.ckb with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.classic
Subclasses of TokenStream in org.apache.lucene.analysis.classicModifier and TypeClassDescriptionclass
Normalizes tokens extracted withClassicTokenizer
.final class
A grammar-based tokenizer constructed with JFlexMethods in org.apache.lucene.analysis.classic that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
ClassicAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.classic with parameters of type TokenStreamModifier and TypeMethodDescriptionClassicFilterFactory.create
(TokenStream input) protected TokenStream
ClassicAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.classic with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.cn.smart
Subclasses of TokenStream in org.apache.lucene.analysis.cn.smartModifier and TypeClassDescriptionclass
Tokenizer for Chinese or mixed Chinese-English text.Methods in org.apache.lucene.analysis.cn.smart that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
SmartChineseAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.cn.smart with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
SmartChineseAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.commongrams
Subclasses of TokenStream in org.apache.lucene.analysis.commongramsModifier and TypeClassDescriptionfinal class
Construct bigrams for frequently occurring terms while indexing.final class
Wrap a CommonGramsFilter optimizing phrase queries by only returning single words when they are not a member of a bigram.Methods in org.apache.lucene.analysis.commongrams with parameters of type TokenStreamModifier and TypeMethodDescriptionCommonGramsFilterFactory.create
(TokenStream input) CommonGramsQueryFilterFactory.create
(TokenStream input) Create a CommonGramsFilter and wrap it with a CommonGramsQueryFilterConstructors in org.apache.lucene.analysis.commongrams with parameters of type TokenStreamModifierConstructorDescriptionCommonGramsFilter
(TokenStream input, CharArraySet commonWords) Construct a token stream filtering the given input using a Set of common words to create bigrams. -
Uses of TokenStream in org.apache.lucene.analysis.compound
Subclasses of TokenStream in org.apache.lucene.analysis.compoundModifier and TypeClassDescriptionclass
Base class for decomposition token filters.class
ATokenFilter
that decomposes compound words found in many Germanic languages.class
ATokenFilter
that decomposes compound words found in many Germanic languages.Methods in org.apache.lucene.analysis.compound that return TokenStreamModifier and TypeMethodDescriptionDictionaryCompoundWordTokenFilterFactory.create
(TokenStream input) Methods in org.apache.lucene.analysis.compound with parameters of type TokenStreamModifier and TypeMethodDescriptionDictionaryCompoundWordTokenFilterFactory.create
(TokenStream input) HyphenationCompoundWordTokenFilterFactory.create
(TokenStream input) Constructors in org.apache.lucene.analysis.compound with parameters of type TokenStreamModifierConstructorDescriptionprotected
CompoundWordTokenFilterBase
(TokenStream input, CharArraySet dictionary) protected
CompoundWordTokenFilterBase
(TokenStream input, CharArraySet dictionary, boolean onlyLongestMatch) protected
CompoundWordTokenFilterBase
(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch) DictionaryCompoundWordTokenFilter
(TokenStream input, CharArraySet dictionary) Creates a newDictionaryCompoundWordTokenFilter
DictionaryCompoundWordTokenFilter
(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch) Creates a newDictionaryCompoundWordTokenFilter
HyphenationCompoundWordTokenFilter
(TokenStream input, HyphenationTree hyphenator) Create a HyphenationCompoundWordTokenFilter with no dictionary.HyphenationCompoundWordTokenFilter
(TokenStream input, HyphenationTree hyphenator, int minWordSize, int minSubwordSize, int maxSubwordSize) Create a HyphenationCompoundWordTokenFilter with no dictionary.HyphenationCompoundWordTokenFilter
(TokenStream input, HyphenationTree hyphenator, CharArraySet dictionary) Creates a newHyphenationCompoundWordTokenFilter
instance.HyphenationCompoundWordTokenFilter
(TokenStream input, HyphenationTree hyphenator, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch) Creates a newHyphenationCompoundWordTokenFilter
instance.HyphenationCompoundWordTokenFilter
(TokenStream input, HyphenationTree hyphenator, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch, boolean noSubMatches, boolean noOverlappingMatches) Creates a newHyphenationCompoundWordTokenFilter
instance. -
Uses of TokenStream in org.apache.lucene.analysis.core
Subclasses of TokenStream in org.apache.lucene.analysis.coreModifier and TypeClassDescriptionfinal class
Folds all Unicode digits in[:General_Category=Decimal_Number:]
to Basic Latin digits (0-9
).final class
Converts an incoming graph token stream, such as one fromSynonymGraphFilter
, into a flat form so that all nodes form a single linear chain with no side paths.final class
Emits the entire input as a single token.class
A LetterTokenizer is a tokenizer that divides text at non-letters.final class
Normalizes token text to lower case.final class
Removes stop words from a token stream.final class
Removes tokens whose types appear in a set of blocked types from a token stream.final class
A UnicodeWhitespaceTokenizer is a tokenizer that divides text at whitespace.final class
Normalizes token text to UPPER CASE.final class
A tokenizer that divides text at whitespace characters as defined byCharacter.isWhitespace(int)
.Methods in org.apache.lucene.analysis.core that return TokenStreamModifier and TypeMethodDescriptionDecimalDigitFilterFactory.create
(TokenStream input) FlattenGraphFilterFactory.create
(TokenStream input) LowerCaseFilterFactory.create
(TokenStream input) StopFilterFactory.create
(TokenStream input) TypeTokenFilterFactory.create
(TokenStream input) UpperCaseFilterFactory.create
(TokenStream input) DecimalDigitFilterFactory.normalize
(TokenStream input) LowerCaseFilterFactory.normalize
(TokenStream input) protected TokenStream
SimpleAnalyzer.normalize
(String fieldName, TokenStream in) protected TokenStream
StopAnalyzer.normalize
(String fieldName, TokenStream in) UpperCaseFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.core with parameters of type TokenStreamModifier and TypeMethodDescriptionDecimalDigitFilterFactory.create
(TokenStream input) FlattenGraphFilterFactory.create
(TokenStream input) LowerCaseFilterFactory.create
(TokenStream input) StopFilterFactory.create
(TokenStream input) TypeTokenFilterFactory.create
(TokenStream input) UpperCaseFilterFactory.create
(TokenStream input) DecimalDigitFilterFactory.normalize
(TokenStream input) LowerCaseFilterFactory.normalize
(TokenStream input) protected TokenStream
SimpleAnalyzer.normalize
(String fieldName, TokenStream in) protected TokenStream
StopAnalyzer.normalize
(String fieldName, TokenStream in) UpperCaseFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.core with parameters of type TokenStreamModifierConstructorDescriptionDecimalDigitFilter
(TokenStream input) Creates a new DecimalDigitFilter overinput
Create a new LowerCaseFilter, that normalizes token text to lower case.StopFilter
(TokenStream in, CharArraySet stopWords) Constructs a filter which removes words from the input TokenStream that are named in the Set.TypeTokenFilter
(TokenStream input, Set<String> stopTypes) Create a newTypeTokenFilter
that filters tokens out (useWhiteList=false).TypeTokenFilter
(TokenStream input, Set<String> stopTypes, boolean useWhiteList) Create a newTypeTokenFilter
.Create a new UpperCaseFilter, that normalizes token text to upper case. -
Uses of TokenStream in org.apache.lucene.analysis.custom
Methods in org.apache.lucene.analysis.custom that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
CustomAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.custom with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
CustomAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.cz
Subclasses of TokenStream in org.apache.lucene.analysis.czModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesCzechStemmer
to stem Czech words.Methods in org.apache.lucene.analysis.cz that return TokenStreamModifier and TypeMethodDescriptionCzechStemFilterFactory.create
(TokenStream input) protected TokenStream
CzechAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.cz with parameters of type TokenStreamModifier and TypeMethodDescriptionCzechStemFilterFactory.create
(TokenStream input) protected TokenStream
CzechAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.cz with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.da
Methods in org.apache.lucene.analysis.da that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
DanishAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.da with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
DanishAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.de
Subclasses of TokenStream in org.apache.lucene.analysis.deModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesGermanLightStemmer
to stem German words.final class
ATokenFilter
that appliesGermanMinimalStemmer
to stem German words.final class
Normalizes German characters according to the heuristics of the German2 snowball algorithm.final class
ATokenFilter
that stems German words.Methods in org.apache.lucene.analysis.de that return TokenStreamModifier and TypeMethodDescriptionGermanLightStemFilterFactory.create
(TokenStream input) GermanMinimalStemFilterFactory.create
(TokenStream input) GermanNormalizationFilterFactory.create
(TokenStream input) protected TokenStream
GermanAnalyzer.normalize
(String fieldName, TokenStream in) GermanNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.de with parameters of type TokenStreamModifier and TypeMethodDescriptionGermanLightStemFilterFactory.create
(TokenStream input) GermanMinimalStemFilterFactory.create
(TokenStream input) GermanNormalizationFilterFactory.create
(TokenStream input) GermanStemFilterFactory.create
(TokenStream in) protected TokenStream
GermanAnalyzer.normalize
(String fieldName, TokenStream in) GermanNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.de with parameters of type TokenStreamModifierConstructorDescriptionGermanLightStemFilter
(TokenStream input) Creates aGermanStemFilter
instance -
Uses of TokenStream in org.apache.lucene.analysis.el
Subclasses of TokenStream in org.apache.lucene.analysis.elModifier and TypeClassDescriptionfinal class
Normalizes token text to lower case, removes some Greek diacritics, and standardizes final sigma to sigma.final class
ATokenFilter
that appliesGreekStemmer
to stem Greek words.Methods in org.apache.lucene.analysis.el that return TokenStreamModifier and TypeMethodDescriptionGreekLowerCaseFilterFactory.create
(TokenStream in) GreekStemFilterFactory.create
(TokenStream input) protected TokenStream
GreekAnalyzer.normalize
(String fieldName, TokenStream in) GreekLowerCaseFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.el with parameters of type TokenStreamModifier and TypeMethodDescriptionGreekLowerCaseFilterFactory.create
(TokenStream in) GreekStemFilterFactory.create
(TokenStream input) protected TokenStream
GreekAnalyzer.normalize
(String fieldName, TokenStream in) GreekLowerCaseFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.el with parameters of type TokenStreamModifierConstructorDescriptionCreate a GreekLowerCaseFilter that normalizes Greek token text.GreekStemFilter
(TokenStream input) -
Uses of TokenStream in org.apache.lucene.analysis.email
Subclasses of TokenStream in org.apache.lucene.analysis.emailModifier and TypeClassDescriptionfinal class
This class implements Word Break rules from the Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29 URLs and email addresses are also tokenized according to the relevant RFCs.Methods in org.apache.lucene.analysis.email that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
UAX29URLEmailAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.email with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
UAX29URLEmailAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.en
Subclasses of TokenStream in org.apache.lucene.analysis.enModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesEnglishMinimalStemmer
to stem English words.final class
TokenFilter that removes possessives (trailing 's) from words.final class
A high-performance kstem filter for english.final class
Transforms the token stream as per the Porter stemming algorithm.Methods in org.apache.lucene.analysis.en that return TokenStreamModifier and TypeMethodDescriptionEnglishMinimalStemFilterFactory.create
(TokenStream input) EnglishPossessiveFilterFactory.create
(TokenStream input) protected TokenStream
EnglishAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.en with parameters of type TokenStreamModifier and TypeMethodDescriptionEnglishMinimalStemFilterFactory.create
(TokenStream input) EnglishPossessiveFilterFactory.create
(TokenStream input) KStemFilterFactory.create
(TokenStream input) PorterStemFilterFactory.create
(TokenStream input) protected TokenStream
EnglishAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.en with parameters of type TokenStreamModifierConstructorDescription -
Uses of TokenStream in org.apache.lucene.analysis.es
Subclasses of TokenStream in org.apache.lucene.analysis.esModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesSpanishLightStemmer
to stem Spanish words.final class
Deprecated.final class
ATokenFilter
that appliesSpanishPluralStemmer
to stem Spanish words.Methods in org.apache.lucene.analysis.es that return TokenStreamModifier and TypeMethodDescriptionSpanishLightStemFilterFactory.create
(TokenStream input) SpanishMinimalStemFilterFactory.create
(TokenStream input) Deprecated.SpanishPluralStemFilterFactory.create
(TokenStream input) protected TokenStream
SpanishAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.es with parameters of type TokenStreamModifier and TypeMethodDescriptionSpanishLightStemFilterFactory.create
(TokenStream input) SpanishMinimalStemFilterFactory.create
(TokenStream input) Deprecated.SpanishPluralStemFilterFactory.create
(TokenStream input) protected TokenStream
SpanishAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.es with parameters of type TokenStreamModifierConstructorDescriptionDeprecated. -
Uses of TokenStream in org.apache.lucene.analysis.et
Methods in org.apache.lucene.analysis.et that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
EstonianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.et with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
EstonianAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.eu
Methods in org.apache.lucene.analysis.eu that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
BasqueAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.eu with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
BasqueAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.fa
Subclasses of TokenStream in org.apache.lucene.analysis.faModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesPersianNormalizer
to normalize the orthography.final class
ATokenFilter
that appliesPersianStemmer
to stem Persian words.Methods in org.apache.lucene.analysis.fa that return TokenStreamModifier and TypeMethodDescriptionPersianNormalizationFilterFactory.create
(TokenStream input) protected TokenStream
PersianAnalyzer.normalize
(String fieldName, TokenStream in) PersianNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.fa with parameters of type TokenStreamModifier and TypeMethodDescriptionPersianNormalizationFilterFactory.create
(TokenStream input) PersianStemFilterFactory.create
(TokenStream input) protected TokenStream
PersianAnalyzer.normalize
(String fieldName, TokenStream in) PersianNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.fa with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.fi
Subclasses of TokenStream in org.apache.lucene.analysis.fiModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesFinnishLightStemmer
to stem Finnish words.Methods in org.apache.lucene.analysis.fi that return TokenStreamModifier and TypeMethodDescriptionFinnishLightStemFilterFactory.create
(TokenStream input) protected TokenStream
FinnishAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.fi with parameters of type TokenStreamModifier and TypeMethodDescriptionFinnishLightStemFilterFactory.create
(TokenStream input) protected TokenStream
FinnishAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.fi with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.fr
Subclasses of TokenStream in org.apache.lucene.analysis.frModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesFrenchLightStemmer
to stem French words.final class
ATokenFilter
that appliesFrenchMinimalStemmer
to stem French words.Methods in org.apache.lucene.analysis.fr that return TokenStreamModifier and TypeMethodDescriptionFrenchLightStemFilterFactory.create
(TokenStream input) FrenchMinimalStemFilterFactory.create
(TokenStream input) protected TokenStream
FrenchAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.fr with parameters of type TokenStreamModifier and TypeMethodDescriptionFrenchLightStemFilterFactory.create
(TokenStream input) FrenchMinimalStemFilterFactory.create
(TokenStream input) protected TokenStream
FrenchAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.fr with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.ga
Subclasses of TokenStream in org.apache.lucene.analysis.gaModifier and TypeClassDescriptionfinal class
Normalises token text to lower case, handling t-prothesis and n-eclipsis (i.e., that 'nAthair' should become 'n-athair')Methods in org.apache.lucene.analysis.ga that return TokenStreamModifier and TypeMethodDescriptionIrishLowerCaseFilterFactory.create
(TokenStream input) protected TokenStream
IrishAnalyzer.normalize
(String fieldName, TokenStream in) IrishLowerCaseFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.ga with parameters of type TokenStreamModifier and TypeMethodDescriptionIrishLowerCaseFilterFactory.create
(TokenStream input) protected TokenStream
IrishAnalyzer.normalize
(String fieldName, TokenStream in) IrishLowerCaseFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.ga with parameters of type TokenStreamModifierConstructorDescriptionCreate an IrishLowerCaseFilter that normalises Irish token text. -
Uses of TokenStream in org.apache.lucene.analysis.gl
Subclasses of TokenStream in org.apache.lucene.analysis.glModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesGalicianMinimalStemmer
to stem Galician words.final class
ATokenFilter
that appliesGalicianStemmer
to stem Galician words.Methods in org.apache.lucene.analysis.gl that return TokenStreamModifier and TypeMethodDescriptionGalicianMinimalStemFilterFactory.create
(TokenStream input) GalicianStemFilterFactory.create
(TokenStream input) protected TokenStream
GalicianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.gl with parameters of type TokenStreamModifier and TypeMethodDescriptionGalicianMinimalStemFilterFactory.create
(TokenStream input) GalicianStemFilterFactory.create
(TokenStream input) protected TokenStream
GalicianAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.gl with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.hi
Subclasses of TokenStream in org.apache.lucene.analysis.hiModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesHindiNormalizer
to normalize the orthography.final class
ATokenFilter
that appliesHindiStemmer
to stem Hindi words.Methods in org.apache.lucene.analysis.hi that return TokenStreamModifier and TypeMethodDescriptionHindiNormalizationFilterFactory.create
(TokenStream input) HindiStemFilterFactory.create
(TokenStream input) protected TokenStream
HindiAnalyzer.normalize
(String fieldName, TokenStream in) HindiNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.hi with parameters of type TokenStreamModifier and TypeMethodDescriptionHindiNormalizationFilterFactory.create
(TokenStream input) HindiStemFilterFactory.create
(TokenStream input) protected TokenStream
HindiAnalyzer.normalize
(String fieldName, TokenStream in) HindiNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.hi with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.hu
Subclasses of TokenStream in org.apache.lucene.analysis.huModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesHungarianLightStemmer
to stem Hungarian words.Methods in org.apache.lucene.analysis.hu that return TokenStreamModifier and TypeMethodDescriptionHungarianLightStemFilterFactory.create
(TokenStream input) protected TokenStream
HungarianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.hu with parameters of type TokenStreamModifier and TypeMethodDescriptionHungarianLightStemFilterFactory.create
(TokenStream input) protected TokenStream
HungarianAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.hu with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.hunspell
Subclasses of TokenStream in org.apache.lucene.analysis.hunspellModifier and TypeClassDescriptionfinal class
TokenFilter that uses hunspell affix rules and words to stem tokens.Methods in org.apache.lucene.analysis.hunspell that return TokenStreamMethods in org.apache.lucene.analysis.hunspell with parameters of type TokenStreamConstructors in org.apache.lucene.analysis.hunspell with parameters of type TokenStreamModifierConstructorDescriptionHunspellStemFilter
(TokenStream input, Dictionary dictionary) Create aHunspellStemFilter
outputting all possible stems.HunspellStemFilter
(TokenStream input, Dictionary dictionary, boolean dedup) Create aHunspellStemFilter
outputting all possible stems.HunspellStemFilter
(TokenStream input, Dictionary dictionary, boolean dedup, boolean longestOnly) Creates a new HunspellStemFilter that will stem tokens from the given TokenStream using affix rules in the provided Dictionary -
Uses of TokenStream in org.apache.lucene.analysis.hy
Methods in org.apache.lucene.analysis.hy that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
ArmenianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.hy with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
ArmenianAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.icu
Subclasses of TokenStream in org.apache.lucene.analysis.icuModifier and TypeClassDescriptionfinal class
A TokenFilter that applies search term folding to Unicode text, applying foldings from UTR#30 Character Foldings.class
Normalize token text with ICU'sNormalizer2
final class
ATokenFilter
that transforms text with ICU.Methods in org.apache.lucene.analysis.icu that return TokenStreamModifier and TypeMethodDescriptionICUFoldingFilterFactory.create
(TokenStream input) ICUNormalizer2FilterFactory.create
(TokenStream input) ICUTransformFilterFactory.create
(TokenStream input) ICUFoldingFilterFactory.normalize
(TokenStream input) ICUNormalizer2FilterFactory.normalize
(TokenStream input) ICUTransformFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.icu with parameters of type TokenStreamModifier and TypeMethodDescriptionICUFoldingFilterFactory.create
(TokenStream input) ICUNormalizer2FilterFactory.create
(TokenStream input) ICUTransformFilterFactory.create
(TokenStream input) ICUFoldingFilterFactory.normalize
(TokenStream input) ICUNormalizer2FilterFactory.normalize
(TokenStream input) ICUTransformFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.icu with parameters of type TokenStreamModifierConstructorDescriptionICUFoldingFilter
(TokenStream input) Create a new ICUFoldingFilter on the specified inputICUFoldingFilter
(TokenStream input, com.ibm.icu.text.Normalizer2 normalizer) Create a new ICUFoldingFilter on the specified input with the specified normalizerICUNormalizer2Filter
(TokenStream input) Create a new Normalizer2Filter that combines NFKC normalization, Case Folding, and removes Default Ignorables (NFKC_Casefold)ICUNormalizer2Filter
(TokenStream input, com.ibm.icu.text.Normalizer2 normalizer) Create a new Normalizer2Filter with the specified Normalizer2ICUTransformFilter
(TokenStream input, com.ibm.icu.text.Transliterator transform) Create a new ICUTransformFilter that transforms text on the given stream. -
Uses of TokenStream in org.apache.lucene.analysis.icu.segmentation
Subclasses of TokenStream in org.apache.lucene.analysis.icu.segmentationModifier and TypeClassDescriptionfinal class
Breaks text into words according to UAX #29: Unicode Text Segmentation (http://www.unicode.org/reports/tr29/) -
Uses of TokenStream in org.apache.lucene.analysis.id
Subclasses of TokenStream in org.apache.lucene.analysis.idModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesIndonesianStemmer
to stem Indonesian words.Methods in org.apache.lucene.analysis.id that return TokenStreamModifier and TypeMethodDescriptionIndonesianStemFilterFactory.create
(TokenStream input) protected TokenStream
IndonesianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.id with parameters of type TokenStreamModifier and TypeMethodDescriptionIndonesianStemFilterFactory.create
(TokenStream input) protected TokenStream
IndonesianAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.id with parameters of type TokenStreamModifierConstructorDescriptionIndonesianStemFilter
(TokenStream input) IndonesianStemFilter
(TokenStream input, boolean stemDerivational) Create a new IndonesianStemFilter. -
Uses of TokenStream in org.apache.lucene.analysis.in
Subclasses of TokenStream in org.apache.lucene.analysis.inModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesIndicNormalizer
to normalize text in Indian Languages.Methods in org.apache.lucene.analysis.in that return TokenStreamModifier and TypeMethodDescriptionIndicNormalizationFilterFactory.create
(TokenStream input) IndicNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.in with parameters of type TokenStreamModifier and TypeMethodDescriptionIndicNormalizationFilterFactory.create
(TokenStream input) IndicNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.in with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.it
Subclasses of TokenStream in org.apache.lucene.analysis.itModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesItalianLightStemmer
to stem Italian words.Methods in org.apache.lucene.analysis.it that return TokenStreamModifier and TypeMethodDescriptionItalianLightStemFilterFactory.create
(TokenStream input) protected TokenStream
ItalianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.it with parameters of type TokenStreamModifier and TypeMethodDescriptionItalianLightStemFilterFactory.create
(TokenStream input) protected TokenStream
ItalianAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.it with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.ja
Subclasses of TokenStream in org.apache.lucene.analysis.jaModifier and TypeClassDescriptionfinal class
Replaces term text with theBaseFormAttribute
.final class
ATokenFilter
that adds Japanese romanized tokens to the term attribute.final class
ATokenFilter
that normalizes small letters (捨て仮名) in hiragana into normal letters.final class
ATokenFilter
that normalizes common katakana spelling variations ending in a long sound character by removing this character (U+30FC).final class
ATokenFilter
that normalizes small letters (捨て仮名) in katakana into normal letters.class
ATokenFilter
that normalizes Japanese numbers (kansūji) to regular Arabic decimal numbers in half-width characters.final class
Removes tokens that match a set of part-of-speech tags.final class
ATokenFilter
that replaces the term attribute with the reading of a token in either katakana or romaji form.final class
Tokenizer for Japanese that uses morphological analysis.Methods in org.apache.lucene.analysis.ja that return TokenStreamModifier and TypeMethodDescriptionJapaneseBaseFormFilterFactory.create
(TokenStream input) JapaneseCompletionFilterFactory.create
(TokenStream input) JapaneseHiraganaUppercaseFilterFactory.create
(TokenStream input) JapaneseKatakanaStemFilterFactory.create
(TokenStream input) JapaneseKatakanaUppercaseFilterFactory.create
(TokenStream input) JapaneseNumberFilterFactory.create
(TokenStream input) JapanesePartOfSpeechStopFilterFactory.create
(TokenStream stream) JapaneseReadingFormFilterFactory.create
(TokenStream input) protected TokenStream
JapaneseAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.ja with parameters of type TokenStreamModifier and TypeMethodDescriptionJapaneseBaseFormFilterFactory.create
(TokenStream input) JapaneseCompletionFilterFactory.create
(TokenStream input) JapaneseHiraganaUppercaseFilterFactory.create
(TokenStream input) JapaneseKatakanaStemFilterFactory.create
(TokenStream input) JapaneseKatakanaUppercaseFilterFactory.create
(TokenStream input) JapaneseNumberFilterFactory.create
(TokenStream input) JapanesePartOfSpeechStopFilterFactory.create
(TokenStream stream) JapaneseReadingFormFilterFactory.create
(TokenStream input) protected TokenStream
JapaneseAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.ja with parameters of type TokenStreamModifierConstructorDescriptionCreates a newJapaneseCompletionFilter
with default configurationsCreates a newJapaneseCompletionFilter
JapaneseKatakanaStemFilter
(TokenStream input, int minimumLength) JapaneseNumberFilter
(TokenStream input) JapanesePartOfSpeechStopFilter
(TokenStream input, Set<String> stopTags) Create a newJapanesePartOfSpeechStopFilter
.JapaneseReadingFormFilter
(TokenStream input, boolean useRomaji) -
Uses of TokenStream in org.apache.lucene.analysis.ko
Subclasses of TokenStream in org.apache.lucene.analysis.koModifier and TypeClassDescriptionclass
ATokenFilter
that normalizes Korean numbers to regular Arabic decimal numbers in half-width characters.final class
Removes tokens that match a set of part-of-speech tags.final class
Replaces term text with theReadingAttribute
which is the Hangul transcription of Hanja characters.final class
Tokenizer for Korean that uses morphological analysis.Methods in org.apache.lucene.analysis.ko that return TokenStreamModifier and TypeMethodDescriptionKoreanNumberFilterFactory.create
(TokenStream input) KoreanPartOfSpeechStopFilterFactory.create
(TokenStream stream) KoreanReadingFormFilterFactory.create
(TokenStream input) protected TokenStream
KoreanAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.ko with parameters of type TokenStreamModifier and TypeMethodDescriptionKoreanNumberFilterFactory.create
(TokenStream input) KoreanPartOfSpeechStopFilterFactory.create
(TokenStream stream) KoreanReadingFormFilterFactory.create
(TokenStream input) protected TokenStream
KoreanAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.ko with parameters of type TokenStreamModifierConstructorDescriptionKoreanNumberFilter
(TokenStream input) Create a newKoreanPartOfSpeechStopFilter
with the default list of stop tagsKoreanPartOfSpeechStopFilter.DEFAULT_STOP_TAGS
.KoreanPartOfSpeechStopFilter
(TokenStream input, Set<POS.Tag> stopTags) Create a newKoreanPartOfSpeechStopFilter
. -
Uses of TokenStream in org.apache.lucene.analysis.lt
Methods in org.apache.lucene.analysis.lt that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
LithuanianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.lt with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
LithuanianAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.lv
Subclasses of TokenStream in org.apache.lucene.analysis.lvModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesLatvianStemmer
to stem Latvian words.Methods in org.apache.lucene.analysis.lv that return TokenStreamModifier and TypeMethodDescriptionLatvianStemFilterFactory.create
(TokenStream input) protected TokenStream
LatvianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.lv with parameters of type TokenStreamModifier and TypeMethodDescriptionLatvianStemFilterFactory.create
(TokenStream input) protected TokenStream
LatvianAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.lv with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.minhash
Subclasses of TokenStream in org.apache.lucene.analysis.minhashModifier and TypeClassDescriptionclass
Generate min hash tokens from an incoming stream of tokens.Methods in org.apache.lucene.analysis.minhash that return TokenStreamMethods in org.apache.lucene.analysis.minhash with parameters of type TokenStreamConstructors in org.apache.lucene.analysis.minhash with parameters of type TokenStreamModifierConstructorDescriptionMinHashFilter
(TokenStream input, int hashCount, int bucketCount, int hashSetSize, boolean withRotation) create a MinHash filter -
Uses of TokenStream in org.apache.lucene.analysis.miscellaneous
Subclasses of TokenStream in org.apache.lucene.analysis.miscellaneousModifier and TypeClassDescriptionfinal class
This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.final class
A filter to apply normal capitalization rules to Tokens.final class
Removes words that are too long or too short from the stream.final class
Concatenates/Joins every incoming token with a separator into one output token for every path through the token stream (which is a graph).final class
A TokenStream that takes an array of input TokenStreams as sources, and concatenates them together.class
Allows skipping TokenFilters based on the current set of attributes.private final class
class
Filters all tokens that cannot be parsed to a date, using the providedDateFormat
.final class
Characters before the delimiter are the "token", the textual integer after is the term frequency.final class
Allows Tokens with a given combination of flags to be dropped.final class
An always exhausted token stream.class
Filter outputs a single token which is a concatenation of the sorted and de-duplicated set of input tokens.final class
Deprecated.Fix the token filters that create broken offsets in the first place.final class
When the plain text is extracted from documents, we will often have many words hyphenated and broken into two lines.final class
A TokenFilter that only keeps tokens with text contained in the required words.class
Marks terms as keywords via theKeywordAttribute
.final class
This TokenFilter emits each incoming token twice once as keyword and once non-keyword, in other words once withKeywordAttribute.setKeyword(boolean)
set totrue
and once set tofalse
.final class
Removes words that are too long or too short from the stream.final class
This TokenFilter limits the number of tokens while indexing.final class
Lets all tokens pass through until it sees one with a start offset <= a configured limit, which won't pass and ends the stream.final class
This TokenFilter limits its emitted tokens to those with positions that are not greater than the configured limit.final class
Marks terms as keywords via theKeywordAttribute
.class
A ConditionalTokenFilter that only applies its wrapped filters to tokens that are not contained in a protected set.final class
A TokenFilter which filters out Tokens at the same position and Term text as the previous token in the stream.final class
This filter folds Scandinavian characters åÅäæÄÆ->a and öÖøØ->o.final class
This filter normalize use of the interchangeable Scandinavian characters æÆäÄöÖøØ and folded variants (aa, ao, ae, oe and oo) by transforming them to åÅæÆøØ.final class
Marks terms as keywords via theKeywordAttribute
.final class
Provides the ability to override anyKeywordAttribute
aware stemmer with custom dictionary-based stemming.final class
Trims leading and trailing whitespace from Tokens in the stream.final class
A token filter for truncating the terms into a specific length.final class
Adds theTypeAttribute.type()
as a synonym, i.e.final class
Deprecated.UseWordDelimiterGraphFilter
instead: it produces a correct token graph so that e.g.final class
Splits words into subwords and performs optional transformations on subword groups, producing a correct token graph so that e.g.Fields in org.apache.lucene.analysis.miscellaneous declared as TokenStreamModifier and TypeFieldDescriptionprivate final TokenStream
ConditionalTokenFilter.delegate
private final TokenStream
ConcatenateGraphFilter.inputTokenStream
private final TokenStream[]
ConcatenatingTokenStream.sources
Methods in org.apache.lucene.analysis.miscellaneous that return TokenStreamModifier and TypeMethodDescriptionASCIIFoldingFilterFactory.create
(TokenStream input) ConcatenateGraphFilterFactory.create
(TokenStream input) ConditionalTokenFilterFactory.create
(TokenStream input) DateRecognizerFilterFactory.create
(TokenStream input) DropIfFlaggedFilterFactory.create
(TokenStream input) FingerprintFilterFactory.create
(TokenStream input) FixBrokenOffsetsFilterFactory.create
(TokenStream input) Deprecated.KeepWordFilterFactory.create
(TokenStream input) KeywordMarkerFilterFactory.create
(TokenStream input) KeywordRepeatFilterFactory.create
(TokenStream input) LimitTokenCountFilterFactory.create
(TokenStream input) LimitTokenOffsetFilterFactory.create
(TokenStream input) LimitTokenPositionFilterFactory.create
(TokenStream input) ScandinavianFoldingFilterFactory.create
(TokenStream input) StemmerOverrideFilterFactory.create
(TokenStream input) TrimFilterFactory.create
(TokenStream input) TruncateTokenFilterFactory.create
(TokenStream input) TypeAsSynonymFilterFactory.create
(TokenStream input) ASCIIFoldingFilterFactory.normalize
(TokenStream input) ScandinavianFoldingFilterFactory.normalize
(TokenStream input) ScandinavianNormalizationFilterFactory.normalize
(TokenStream input) TrimFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.miscellaneous with parameters of type TokenStreamModifier and TypeMethodDescriptionprivate static AttributeSource
ConcatenatingTokenStream.combineSources
(TokenStream... sources) ASCIIFoldingFilterFactory.create
(TokenStream input) CapitalizationFilterFactory.create
(TokenStream input) CodepointCountFilterFactory.create
(TokenStream input) ConcatenateGraphFilterFactory.create
(TokenStream input) ConditionalTokenFilterFactory.create
(TokenStream input) protected abstract ConditionalTokenFilter
ConditionalTokenFilterFactory.create
(TokenStream input, Function<TokenStream, TokenStream> inner) Modify the incomingTokenStream
with aConditionalTokenFilter
DateRecognizerFilterFactory.create
(TokenStream input) DelimitedTermFrequencyTokenFilterFactory.create
(TokenStream input) DropIfFlaggedFilterFactory.create
(TokenStream input) FingerprintFilterFactory.create
(TokenStream input) FixBrokenOffsetsFilterFactory.create
(TokenStream input) Deprecated.HyphenatedWordsFilterFactory.create
(TokenStream input) KeepWordFilterFactory.create
(TokenStream input) KeywordMarkerFilterFactory.create
(TokenStream input) KeywordRepeatFilterFactory.create
(TokenStream input) LengthFilterFactory.create
(TokenStream input) LimitTokenCountFilterFactory.create
(TokenStream input) LimitTokenOffsetFilterFactory.create
(TokenStream input) LimitTokenPositionFilterFactory.create
(TokenStream input) protected ConditionalTokenFilter
ProtectedTermFilterFactory.create
(TokenStream input, Function<TokenStream, TokenStream> inner) RemoveDuplicatesTokenFilterFactory.create
(TokenStream input) ScandinavianFoldingFilterFactory.create
(TokenStream input) ScandinavianNormalizationFilterFactory.create
(TokenStream input) StemmerOverrideFilterFactory.create
(TokenStream input) TrimFilterFactory.create
(TokenStream input) TruncateTokenFilterFactory.create
(TokenStream input) TypeAsSynonymFilterFactory.create
(TokenStream input) WordDelimiterFilterFactory.create
(TokenStream input) Deprecated.WordDelimiterGraphFilterFactory.create
(TokenStream input) ASCIIFoldingFilterFactory.normalize
(TokenStream input) ScandinavianFoldingFilterFactory.normalize
(TokenStream input) ScandinavianNormalizationFilterFactory.normalize
(TokenStream input) TrimFilterFactory.normalize
(TokenStream input) Method parameters in org.apache.lucene.analysis.miscellaneous with type arguments of type TokenStreamModifier and TypeMethodDescriptionprotected abstract ConditionalTokenFilter
ConditionalTokenFilterFactory.create
(TokenStream input, Function<TokenStream, TokenStream> inner) Modify the incomingTokenStream
with aConditionalTokenFilter
protected abstract ConditionalTokenFilter
ConditionalTokenFilterFactory.create
(TokenStream input, Function<TokenStream, TokenStream> inner) Modify the incomingTokenStream
with aConditionalTokenFilter
protected ConditionalTokenFilter
ProtectedTermFilterFactory.create
(TokenStream input, Function<TokenStream, TokenStream> inner) protected ConditionalTokenFilter
ProtectedTermFilterFactory.create
(TokenStream input, Function<TokenStream, TokenStream> inner) Constructors in org.apache.lucene.analysis.miscellaneous with parameters of type TokenStreamModifierConstructorDescriptionASCIIFoldingFilter
(TokenStream input) ASCIIFoldingFilter
(TokenStream input, boolean preserveOriginal) Create a newASCIIFoldingFilter
.Creates a CapitalizationFilter with the default parameters.CapitalizationFilter
(TokenStream in, boolean onlyFirstWord, CharArraySet keep, boolean forceFirstLetter, Collection<char[]> okPrefix, int minWordLength, int maxWordCount, int maxTokenLength) Creates a CapitalizationFilter with the specified parameters.CodepointCountFilter
(TokenStream in, int min, int max) Create a newCodepointCountFilter
.ConcatenateGraphFilter
(TokenStream inputTokenStream) Creates a token stream to convertinput
to a token stream of accepted strings by its token stream graph.ConcatenateGraphFilter
(TokenStream inputTokenStream, boolean preserveSep, boolean preservePositionIncrements, int maxGraphExpansions) ConcatenateGraphFilter
(TokenStream inputTokenStream, Character tokenSeparator, boolean preservePositionIncrements, int maxGraphExpansions) Creates a token stream to convertinput
to a token stream of accepted strings by its token stream graph.ConcatenatingTokenStream
(TokenStream... sources) Create a new ConcatenatingTokenStream from a set of inputsprotected
ConditionalTokenFilter
(TokenStream input, Function<TokenStream, TokenStream> inputFactory) Create a new ConditionalTokenFilterDateRecognizerFilter
(TokenStream input) DateRecognizerFilter
(TokenStream input, DateFormat dateFormat) DelimitedTermFrequencyTokenFilter
(TokenStream input, char delimiter) DropIfFlaggedFilter
(TokenStream input, int dropFlags) Construct a token stream filtering the given input.FingerprintFilter
(TokenStream input) Create a new FingerprintFilter with default settingsFingerprintFilter
(TokenStream input, int maxOutputTokenSize, char separator) Create a new FingerprintFilter with control over all settingsDeprecated.Creates a new HyphenatedWordsFilterKeepWordFilter
(TokenStream in, CharArraySet words) Create a newKeepWordFilter
.protected
Creates a newKeywordMarkerFilter
KeywordRepeatFilter
(TokenStream input) Construct a token stream filtering the given input.LengthFilter
(TokenStream in, int min, int max) Create a newLengthFilter
.LimitTokenCountFilter
(TokenStream in, int maxTokenCount) Build a filter that only accepts tokens up to a maximum number.LimitTokenCountFilter
(TokenStream in, int maxTokenCount, boolean consumeAllTokens) Build an filter that limits the maximum number of tokens per field.LimitTokenOffsetFilter
(TokenStream input, int maxStartOffset) Lets all tokens pass through until it sees one with a start offset <=maxStartOffset
which won't pass and ends the stream.LimitTokenOffsetFilter
(TokenStream input, int maxStartOffset, boolean consumeAllTokens) LimitTokenPositionFilter
(TokenStream in, int maxTokenPosition) Build a filter that only accepts tokens up to and including the given maximum position.LimitTokenPositionFilter
(TokenStream in, int maxTokenPosition, boolean consumeAllTokens) Build a filter that limits the maximum position of tokens to emit.PatternKeywordMarkerFilter
(TokenStream in, Pattern pattern) Create a newPatternKeywordMarkerFilter
, that marks the current token as a keyword if the tokens term buffer matches the providedPattern
via theKeywordAttribute
.ProtectedTermFilter
(CharArraySet protectedTerms, TokenStream input, Function<TokenStream, TokenStream> inputFactory) Creates a new ProtectedTermFilterCreates a new RemoveDuplicatesTokenFilterSetKeywordMarkerFilter
(TokenStream in, CharArraySet keywordSet) Create a new KeywordSetMarkerFilter, that marks the current token as a keyword if the tokens term buffer is contained in the given set via theKeywordAttribute
.StemmerOverrideFilter
(TokenStream input, StemmerOverrideFilter.StemmerOverrideMap stemmerOverrideMap) Create a new StemmerOverrideFilter, performing dictionary-based stemming with the provideddictionary
.Create a newTrimFilter
.TruncateTokenFilter
(TokenStream input, int length) TypeAsSynonymFilter
(TokenStream input) TypeAsSynonymFilter
(TokenStream input, String prefix) TypeAsSynonymFilter
(TokenStream input, String prefix, Set<String> ignore, int synFlagsMask) WordDelimiterFilter
(TokenStream in, byte[] charTypeTable, int configurationFlags, CharArraySet protWords) Deprecated.Creates a new WordDelimiterFilterWordDelimiterFilter
(TokenStream in, int configurationFlags, CharArraySet protWords) Deprecated.Creates a new WordDelimiterFilter usingWordDelimiterIterator.DEFAULT_WORD_DELIM_TABLE
as its charTypeTableWordDelimiterGraphFilter
(TokenStream in, boolean adjustInternalOffsets, byte[] charTypeTable, int configurationFlags, CharArraySet protWords) Creates a new WordDelimiterGraphFilterWordDelimiterGraphFilter
(TokenStream in, int configurationFlags, CharArraySet protWords) Creates a new WordDelimiterGraphFilter usingWordDelimiterIterator.DEFAULT_WORD_DELIM_TABLE
as its charTypeTableConstructor parameters in org.apache.lucene.analysis.miscellaneous with type arguments of type TokenStreamModifierConstructorDescriptionprotected
ConditionalTokenFilter
(TokenStream input, Function<TokenStream, TokenStream> inputFactory) Create a new ConditionalTokenFilterprotected
ConditionalTokenFilter
(TokenStream input, Function<TokenStream, TokenStream> inputFactory) Create a new ConditionalTokenFilterProtectedTermFilter
(CharArraySet protectedTerms, TokenStream input, Function<TokenStream, TokenStream> inputFactory) Creates a new ProtectedTermFilterProtectedTermFilter
(CharArraySet protectedTerms, TokenStream input, Function<TokenStream, TokenStream> inputFactory) Creates a new ProtectedTermFilter -
Uses of TokenStream in org.apache.lucene.analysis.ne
Methods in org.apache.lucene.analysis.ne that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
NepaliAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.ne with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
NepaliAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.ngram
Subclasses of TokenStream in org.apache.lucene.analysis.ngramModifier and TypeClassDescriptionfinal class
Tokenizes the given token into n-grams of given size(s).class
Tokenizes the input from an edge into n-grams of given size(s).final class
Tokenizes the input into n-grams of the given size(s).class
Tokenizes the input into n-grams of the given size(s).Methods in org.apache.lucene.analysis.ngram with parameters of type TokenStreamModifier and TypeMethodDescriptionEdgeNGramFilterFactory.create
(TokenStream input) NGramFilterFactory.create
(TokenStream input) Constructors in org.apache.lucene.analysis.ngram with parameters of type TokenStreamModifierConstructorDescriptionEdgeNGramTokenFilter
(TokenStream input, int gramSize) Creates an EdgeNGramTokenFilter that produces edge n-grams of the given size.EdgeNGramTokenFilter
(TokenStream input, int minGram, int maxGram, boolean preserveOriginal) Creates an EdgeNGramTokenFilter that, for a given input term, produces all edge n-grams with lengths >= minGram and <= maxGram.NGramTokenFilter
(TokenStream input, int gramSize) Creates an NGramTokenFilter that produces n-grams of the indicated size.NGramTokenFilter
(TokenStream input, int minGram, int maxGram, boolean preserveOriginal) Creates an NGramTokenFilter that, for a given input term, produces all contained n-grams with lengths >= minGram and <= maxGram. -
Uses of TokenStream in org.apache.lucene.analysis.nl
Methods in org.apache.lucene.analysis.nl that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
DutchAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.nl with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
DutchAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.no
Subclasses of TokenStream in org.apache.lucene.analysis.noModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesNorwegianLightStemmer
to stem Norwegian words.final class
ATokenFilter
that appliesNorwegianMinimalStemmer
to stem Norwegian words.final class
This filter normalize use of the interchangeable Scandinavian characters æÆäÄöÖøØ and folded variants (ae, oe, aa) by transforming them to åÅæÆøØ.Methods in org.apache.lucene.analysis.no that return TokenStreamModifier and TypeMethodDescriptionNorwegianLightStemFilterFactory.create
(TokenStream input) NorwegianMinimalStemFilterFactory.create
(TokenStream input) protected TokenStream
NorwegianAnalyzer.normalize
(String fieldName, TokenStream in) NorwegianNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.no with parameters of type TokenStreamModifier and TypeMethodDescriptionNorwegianLightStemFilterFactory.create
(TokenStream input) NorwegianMinimalStemFilterFactory.create
(TokenStream input) NorwegianNormalizationFilterFactory.create
(TokenStream input) protected TokenStream
NorwegianAnalyzer.normalize
(String fieldName, TokenStream in) NorwegianNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.no with parameters of type TokenStreamModifierConstructorDescriptionNorwegianLightStemFilter
(TokenStream input, int flags) Creates a new NorwegianLightStemFilterNorwegianMinimalStemFilter
(TokenStream input, int flags) Creates a new NorwegianLightStemFilter -
Uses of TokenStream in org.apache.lucene.analysis.path
Subclasses of TokenStream in org.apache.lucene.analysis.pathModifier and TypeClassDescriptionclass
Tokenizer for path-like hierarchies.class
Tokenizer for domain-like hierarchies. -
Uses of TokenStream in org.apache.lucene.analysis.pattern
Subclasses of TokenStream in org.apache.lucene.analysis.patternModifier and TypeClassDescriptionfinal class
CaptureGroup uses Java regexes to emit multiple tokens - one for each capture group in one or more patterns.final class
A TokenFilter which applies a Pattern to each token in the stream, replacing match occurrences with the specified replacement string.final class
This tokenizer uses regex pattern matching to construct distinct tokens for the input stream.class
Set a type attribute to a parameterized value when tokens are matched by any of a several regex patterns.final class
final class
Methods in org.apache.lucene.analysis.pattern that return TokenStreamMethods in org.apache.lucene.analysis.pattern with parameters of type TokenStreamModifier and TypeMethodDescriptionPatternCaptureGroupFilterFactory.create
(TokenStream input) PatternReplaceFilterFactory.create
(TokenStream input) PatternTypingFilterFactory.create
(TokenStream input) Constructors in org.apache.lucene.analysis.pattern with parameters of type TokenStreamModifierConstructorDescriptionPatternCaptureGroupTokenFilter
(TokenStream input, boolean preserveOriginal, Pattern... patterns) PatternReplaceFilter
(TokenStream in, Pattern p, String replacement, boolean all) Constructs an instance to replace either the first, or all occurrencesPatternTypingFilter
(TokenStream input, PatternTypingFilter.PatternTypingRule... replacementAndFlagByPattern) -
Uses of TokenStream in org.apache.lucene.analysis.payloads
Subclasses of TokenStream in org.apache.lucene.analysis.payloadsModifier and TypeClassDescriptionfinal class
Characters before the delimiter are the "token", those after are the payload.class
Assigns a payload to a token based on theTypeAttribute
class
Adds theOffsetAttribute.startOffset()
andOffsetAttribute.endOffset()
First 4 bytes are the startclass
Makes theTypeAttribute
a payload.Methods in org.apache.lucene.analysis.payloads with parameters of type TokenStreamModifier and TypeMethodDescriptionDelimitedPayloadTokenFilterFactory.create
(TokenStream input) NumericPayloadTokenFilterFactory.create
(TokenStream input) TokenOffsetPayloadTokenFilterFactory.create
(TokenStream input) TypeAsPayloadTokenFilterFactory.create
(TokenStream input) Constructors in org.apache.lucene.analysis.payloads with parameters of type TokenStreamModifierConstructorDescriptionDelimitedPayloadTokenFilter
(TokenStream input, char delimiter, PayloadEncoder encoder) NumericPayloadTokenFilter
(TokenStream input, float payload, String typeMatch) -
Uses of TokenStream in org.apache.lucene.analysis.phonetic
Subclasses of TokenStream in org.apache.lucene.analysis.phoneticModifier and TypeClassDescriptionfinal class
TokenFilter for Beider-Morse phonetic encoding.final class
Create tokens for phonetic matches based on Daitch–Mokotoff Soundex.final class
Filter for DoubleMetaphone (supporting secondary codes)final class
Create tokens for phonetic matches.Methods in org.apache.lucene.analysis.phonetic that return TokenStreamMethods in org.apache.lucene.analysis.phonetic with parameters of type TokenStreamModifier and TypeMethodDescriptionBeiderMorseFilterFactory.create
(TokenStream input) DaitchMokotoffSoundexFilterFactory.create
(TokenStream input) DoubleMetaphoneFilterFactory.create
(TokenStream input) PhoneticFilterFactory.create
(TokenStream input) Constructors in org.apache.lucene.analysis.phonetic with parameters of type TokenStreamModifierConstructorDescriptionBeiderMorseFilter
(TokenStream input, org.apache.commons.codec.language.bm.PhoneticEngine engine) BeiderMorseFilter
(TokenStream input, org.apache.commons.codec.language.bm.PhoneticEngine engine, org.apache.commons.codec.language.bm.Languages.LanguageSet languages) Create a new BeiderMorseFilterDaitchMokotoffSoundexFilter
(TokenStream in, boolean inject) Creates a DaitchMokotoffSoundexFilter by either adding encoded forms as synonyms (inject=true
) or replacing them.DoubleMetaphoneFilter
(TokenStream input, int maxCodeLength, boolean inject) Creates a DoubleMetaphoneFilter with the specified maximum code length, and either adding encoded forms as synonyms (inject=true
) or replacing them.PhoneticFilter
(TokenStream in, org.apache.commons.codec.Encoder encoder, boolean inject) Creates a PhoneticFilter with the specified encoder, and either adding encoded forms as synonyms (inject=true
) or replacing them. -
Uses of TokenStream in org.apache.lucene.analysis.pl
Methods in org.apache.lucene.analysis.pl that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
PolishAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.pl with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
PolishAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.pt
Subclasses of TokenStream in org.apache.lucene.analysis.ptModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesPortugueseLightStemmer
to stem Portuguese words.final class
ATokenFilter
that appliesPortugueseMinimalStemmer
to stem Portuguese words.final class
ATokenFilter
that appliesPortugueseStemmer
to stem Portuguese words.Methods in org.apache.lucene.analysis.pt that return TokenStreamModifier and TypeMethodDescriptionPortugueseLightStemFilterFactory.create
(TokenStream input) PortugueseMinimalStemFilterFactory.create
(TokenStream input) PortugueseStemFilterFactory.create
(TokenStream input) protected TokenStream
PortugueseAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.pt with parameters of type TokenStreamModifier and TypeMethodDescriptionPortugueseLightStemFilterFactory.create
(TokenStream input) PortugueseMinimalStemFilterFactory.create
(TokenStream input) PortugueseStemFilterFactory.create
(TokenStream input) protected TokenStream
PortugueseAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.pt with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.reverse
Subclasses of TokenStream in org.apache.lucene.analysis.reverseModifier and TypeClassDescriptionfinal class
Reverse token string, for example "country" => "yrtnuoc".Methods in org.apache.lucene.analysis.reverse with parameters of type TokenStreamConstructors in org.apache.lucene.analysis.reverse with parameters of type TokenStreamModifierConstructorDescriptionCreate a new ReverseStringFilter that reverses all tokens in the suppliedTokenStream
.ReverseStringFilter
(TokenStream in, char marker) Create a new ReverseStringFilter that reverses and marks all tokens in the suppliedTokenStream
. -
Uses of TokenStream in org.apache.lucene.analysis.ro
Methods in org.apache.lucene.analysis.ro that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
RomanianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.ro with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
RomanianAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.ru
Subclasses of TokenStream in org.apache.lucene.analysis.ruModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesRussianLightStemmer
to stem Russian words.Methods in org.apache.lucene.analysis.ru that return TokenStreamModifier and TypeMethodDescriptionRussianLightStemFilterFactory.create
(TokenStream input) protected TokenStream
RussianAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.ru with parameters of type TokenStreamModifier and TypeMethodDescriptionRussianLightStemFilterFactory.create
(TokenStream input) protected TokenStream
RussianAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.ru with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.shingle
Subclasses of TokenStream in org.apache.lucene.analysis.shingleModifier and TypeClassDescriptionfinal class
A FixedShingleFilter constructs shingles (token n-grams) from a token stream.final class
A ShingleFilter constructs shingles (token n-grams) from a token stream.Methods in org.apache.lucene.analysis.shingle that return TokenStreamMethods in org.apache.lucene.analysis.shingle with parameters of type TokenStreamModifier and TypeMethodDescriptionFixedShingleFilterFactory.create
(TokenStream input) ShingleFilterFactory.create
(TokenStream input) Constructors in org.apache.lucene.analysis.shingle with parameters of type TokenStreamModifierConstructorDescriptionFixedShingleFilter
(TokenStream input, int shingleSize) Creates a FixedShingleFilter over an input token streamFixedShingleFilter
(TokenStream input, int shingleSize, String tokenSeparator, String fillerToken) Creates a FixedShingleFilter over an input token streamShingleFilter
(TokenStream input) Construct a ShingleFilter with default shingle size: 2.ShingleFilter
(TokenStream input, int maxShingleSize) Constructs a ShingleFilter with the specified shingle size from theTokenStream
input
ShingleFilter
(TokenStream input, int minShingleSize, int maxShingleSize) Constructs a ShingleFilter with the specified shingle size from theTokenStream
input
ShingleFilter
(TokenStream input, String tokenType) Construct a ShingleFilter with the specified token type for shingle tokens and the default shingle size: 2 -
Uses of TokenStream in org.apache.lucene.analysis.sinks
Subclasses of TokenStream in org.apache.lucene.analysis.sinksModifier and TypeClassDescriptionfinal class
This TokenFilter provides the ability to set aside attribute states that have already been analyzed.static final class
TokenStream output from a tee.Methods in org.apache.lucene.analysis.sinks that return TokenStreamModifier and TypeMethodDescriptionTeeSinkTokenFilter.newSinkTokenStream()
Returns a newTeeSinkTokenFilter.SinkTokenStream
that receives all tokens consumed by this stream.Constructors in org.apache.lucene.analysis.sinks with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.snowball
Subclasses of TokenStream in org.apache.lucene.analysis.snowballModifier and TypeClassDescriptionfinal class
A filter that stems words using a Snowball-generated stemmer.Methods in org.apache.lucene.analysis.snowball with parameters of type TokenStreamConstructors in org.apache.lucene.analysis.snowball with parameters of type TokenStreamModifierConstructorDescriptionSnowballFilter
(TokenStream in, String name) Construct the named stemming filter.SnowballFilter
(TokenStream input, SnowballStemmer stemmer) -
Uses of TokenStream in org.apache.lucene.analysis.sr
Subclasses of TokenStream in org.apache.lucene.analysis.srModifier and TypeClassDescriptionfinal class
Normalizes Serbian Cyrillic and Latin characters to "bald" Latin.final class
Normalizes Serbian Cyrillic to Latin.Methods in org.apache.lucene.analysis.sr that return TokenStreamModifier and TypeMethodDescriptionSerbianNormalizationFilterFactory.create
(TokenStream input) protected TokenStream
SerbianAnalyzer.normalize
(String fieldName, TokenStream in) SerbianNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.sr with parameters of type TokenStreamModifier and TypeMethodDescriptionSerbianNormalizationFilterFactory.create
(TokenStream input) protected TokenStream
SerbianAnalyzer.normalize
(String fieldName, TokenStream in) SerbianNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.sr with parameters of type TokenStreamModifierConstructorDescription -
Uses of TokenStream in org.apache.lucene.analysis.standard
Subclasses of TokenStream in org.apache.lucene.analysis.standardModifier and TypeClassDescriptionfinal class
A grammar-based tokenizer constructed with JFlex.Methods in org.apache.lucene.analysis.standard that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
StandardAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.standard with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
StandardAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.stempel
Subclasses of TokenStream in org.apache.lucene.analysis.stempelModifier and TypeClassDescriptionfinal class
Transforms the token stream as per the stemming algorithm.Methods in org.apache.lucene.analysis.stempel that return TokenStreamMethods in org.apache.lucene.analysis.stempel with parameters of type TokenStreamConstructors in org.apache.lucene.analysis.stempel with parameters of type TokenStreamModifierConstructorDescriptionStempelFilter
(TokenStream in, StempelStemmer stemmer) Create filter using the supplied stemming table.StempelFilter
(TokenStream in, StempelStemmer stemmer, int minLength) Create filter using the supplied stemming table. -
Uses of TokenStream in org.apache.lucene.analysis.sv
Subclasses of TokenStream in org.apache.lucene.analysis.svModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesSwedishLightStemmer
to stem Swedish words.final class
ATokenFilter
that appliesSwedishMinimalStemmer
to stem Swedish words.Methods in org.apache.lucene.analysis.sv that return TokenStreamModifier and TypeMethodDescriptionSwedishLightStemFilterFactory.create
(TokenStream input) SwedishMinimalStemFilterFactory.create
(TokenStream input) protected TokenStream
SwedishAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.sv with parameters of type TokenStreamModifier and TypeMethodDescriptionSwedishLightStemFilterFactory.create
(TokenStream input) SwedishMinimalStemFilterFactory.create
(TokenStream input) protected TokenStream
SwedishAnalyzer.normalize
(String fieldName, TokenStream in) Constructors in org.apache.lucene.analysis.sv with parameters of type TokenStreamModifierConstructorDescription -
Uses of TokenStream in org.apache.lucene.analysis.synonym
Subclasses of TokenStream in org.apache.lucene.analysis.synonymModifier and TypeClassDescriptionfinal class
Deprecated.UseSynonymGraphFilter
instead, but be sure to also useFlattenGraphFilter
at index time (not at search time) as well.final class
Applies single- or multi-token synonyms from aSynonymMap
to an incomingTokenStream
, producing a fully correct graph output.Methods in org.apache.lucene.analysis.synonym that return TokenStreamModifier and TypeMethodDescriptionSynonymFilterFactory.create
(TokenStream input) Deprecated.SynonymGraphFilterFactory.create
(TokenStream input) Methods in org.apache.lucene.analysis.synonym with parameters of type TokenStreamModifier and TypeMethodDescriptionSynonymFilterFactory.create
(TokenStream input) Deprecated.SynonymGraphFilterFactory.create
(TokenStream input) Constructors in org.apache.lucene.analysis.synonym with parameters of type TokenStreamModifierConstructorDescriptionSynonymFilter
(TokenStream input, SynonymMap synonyms, boolean ignoreCase) Deprecated.SynonymGraphFilter
(TokenStream input, SynonymMap synonyms, boolean ignoreCase) Apply previously built synonyms to incoming tokens. -
Uses of TokenStream in org.apache.lucene.analysis.synonym.word2vec
Subclasses of TokenStream in org.apache.lucene.analysis.synonym.word2vecModifier and TypeClassDescriptionfinal class
Applies single-token synonyms from a Word2Vec trained network to an incomingTokenStream
.Methods in org.apache.lucene.analysis.synonym.word2vec that return TokenStreamMethods in org.apache.lucene.analysis.synonym.word2vec with parameters of type TokenStreamConstructors in org.apache.lucene.analysis.synonym.word2vec with parameters of type TokenStreamModifierConstructorDescriptionWord2VecSynonymFilter
(TokenStream input, Word2VecSynonymProvider synonymProvider, int maxSynonymsPerTerm, float minAcceptedSimilarity) Apply previously built synonymProvider to incoming tokens. -
Uses of TokenStream in org.apache.lucene.analysis.ta
Methods in org.apache.lucene.analysis.ta that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
TamilAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.ta with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
TamilAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.te
Subclasses of TokenStream in org.apache.lucene.analysis.teModifier and TypeClassDescriptionfinal class
ATokenFilter
that appliesTeluguNormalizer
to normalize the orthography.final class
ATokenFilter
that appliesTeluguStemmer
to stem Telugu words.Methods in org.apache.lucene.analysis.te that return TokenStreamModifier and TypeMethodDescriptionTeluguNormalizationFilterFactory.create
(TokenStream input) TeluguStemFilterFactory.create
(TokenStream input) protected TokenStream
TeluguAnalyzer.normalize
(String fieldName, TokenStream in) TeluguNormalizationFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.te with parameters of type TokenStreamModifier and TypeMethodDescriptionTeluguNormalizationFilterFactory.create
(TokenStream input) TeluguStemFilterFactory.create
(TokenStream input) protected TokenStream
TeluguAnalyzer.normalize
(String fieldName, TokenStream in) TeluguNormalizationFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.te with parameters of type TokenStream -
Uses of TokenStream in org.apache.lucene.analysis.th
Subclasses of TokenStream in org.apache.lucene.analysis.thMethods in org.apache.lucene.analysis.th that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
ThaiAnalyzer.normalize
(String fieldName, TokenStream in) Methods in org.apache.lucene.analysis.th with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
ThaiAnalyzer.normalize
(String fieldName, TokenStream in) -
Uses of TokenStream in org.apache.lucene.analysis.tr
Subclasses of TokenStream in org.apache.lucene.analysis.trModifier and TypeClassDescriptionfinal class
Strips all characters after an apostrophe (including the apostrophe itself).final class
Normalizes Turkish token text to lower case.Methods in org.apache.lucene.analysis.tr that return TokenStreamModifier and TypeMethodDescriptionApostropheFilterFactory.create
(TokenStream input) TurkishLowerCaseFilterFactory.create
(TokenStream input) protected TokenStream
TurkishAnalyzer.normalize
(String fieldName, TokenStream in) TurkishLowerCaseFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.tr with parameters of type TokenStreamModifier and TypeMethodDescriptionApostropheFilterFactory.create
(TokenStream input) TurkishLowerCaseFilterFactory.create
(TokenStream input) protected TokenStream
TurkishAnalyzer.normalize
(String fieldName, TokenStream in) TurkishLowerCaseFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.tr with parameters of type TokenStreamModifierConstructorDescriptionCreate a new TurkishLowerCaseFilter, that normalizes Turkish token text to lower case. -
Uses of TokenStream in org.apache.lucene.analysis.util
Subclasses of TokenStream in org.apache.lucene.analysis.utilModifier and TypeClassDescriptionclass
An abstract base class for simple, character-oriented tokenizers.final class
Removes elisions from aTokenStream
.class
Breaks text into sentences with aBreakIterator
and allows subclasses to decompose these sentences into words.Methods in org.apache.lucene.analysis.util that return TokenStreamModifier and TypeMethodDescriptionElisionFilterFactory.create
(TokenStream input) ElisionFilterFactory.normalize
(TokenStream input) Methods in org.apache.lucene.analysis.util with parameters of type TokenStreamModifier and TypeMethodDescriptionElisionFilterFactory.create
(TokenStream input) ElisionFilterFactory.normalize
(TokenStream input) Constructors in org.apache.lucene.analysis.util with parameters of type TokenStreamModifierConstructorDescriptionElisionFilter
(TokenStream input, CharArraySet articles) Constructs an elision filter with a Set of stop words -
Uses of TokenStream in org.apache.lucene.analysis.wikipedia
Subclasses of TokenStream in org.apache.lucene.analysis.wikipediaModifier and TypeClassDescriptionfinal class
Extension of StandardTokenizer that is aware of Wikipedia syntax. -
Uses of TokenStream in org.apache.lucene.classification.document
Methods in org.apache.lucene.classification.document with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected String[]
SimpleNaiveBayesDocumentClassifier.getTokenArray
(TokenStream tokenizedText) Returns a token array from theTokenStream
in input -
Uses of TokenStream in org.apache.lucene.document
Subclasses of TokenStream in org.apache.lucene.documentModifier and TypeClassDescriptionprivate static final class
private static final class
private static final class
Fields in org.apache.lucene.document declared as TokenStreamModifier and TypeFieldDescriptionprotected TokenStream
Field.tokenStream
Pre-analyzed tokenStream for indexed fields; this is separate from fieldsData because you are allowed to have both; eg maybe field has a String value but you customize how it's tokenizedMethods in org.apache.lucene.document that return TokenStreamModifier and TypeMethodDescriptionFeatureField.tokenStream
(Analyzer analyzer, TokenStream reuse) Field.tokenStream
(Analyzer analyzer, TokenStream reuse) ShapeDocValuesField.tokenStream
(Analyzer analyzer, TokenStream reuse) TokenStreams are not yet supportedField.tokenStreamValue()
The TokenStream for this field to be used when indexing, or null.Methods in org.apache.lucene.document with parameters of type TokenStreamModifier and TypeMethodDescriptionvoid
Field.setTokenStream
(TokenStream tokenStream) Expert: sets the token stream to be used for indexing and causes isIndexed() and isTokenized() to return true.FeatureField.tokenStream
(Analyzer analyzer, TokenStream reuse) Field.tokenStream
(Analyzer analyzer, TokenStream reuse) ShapeDocValuesField.tokenStream
(Analyzer analyzer, TokenStream reuse) TokenStreams are not yet supportedConstructors in org.apache.lucene.document with parameters of type TokenStreamModifierConstructorDescriptionField
(String name, TokenStream tokenStream, IndexableFieldType type) Create field with TokenStream value.TextField
(String name, TokenStream stream) Creates a new un-stored TextField with TokenStream value. -
Uses of TokenStream in org.apache.lucene.facet.taxonomy.directory
Subclasses of TokenStream in org.apache.lucene.facet.taxonomy.directoryModifier and TypeClassDescriptionprivate static class
-
Uses of TokenStream in org.apache.lucene.index
Fields in org.apache.lucene.index declared as TokenStreamModifier and TypeFieldDescription(package private) TokenStream
IndexingChain.PerField.tokenStream
Methods in org.apache.lucene.index that return TokenStreamModifier and TypeMethodDescriptionIndexableField.tokenStream
(Analyzer analyzer, TokenStream reuse) Creates the TokenStream used for indexing this field.IndexingChain.ReservedField.tokenStream
(Analyzer analyzer, TokenStream reuse) Methods in org.apache.lucene.index with parameters of type TokenStreamModifier and TypeMethodDescriptionIndexableField.tokenStream
(Analyzer analyzer, TokenStream reuse) Creates the TokenStream used for indexing this field.IndexingChain.ReservedField.tokenStream
(Analyzer analyzer, TokenStream reuse) -
Uses of TokenStream in org.apache.lucene.index.memory
Methods in org.apache.lucene.index.memory that return TokenStreamModifier and TypeMethodDescription<T> TokenStream
MemoryIndex.keywordTokenStream
(Collection<T> keywords) Convenience method; Creates and returns a token stream that generates a token for each keyword in the given collection, "as is", without any transforming text analysis.Methods in org.apache.lucene.index.memory with parameters of type TokenStreamModifier and TypeMethodDescriptionvoid
MemoryIndex.addField
(String fieldName, TokenStream stream) Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, LuceneField
.void
MemoryIndex.addField
(String fieldName, TokenStream stream, int positionIncrementGap) Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, LuceneField
.void
MemoryIndex.addField
(String fieldName, TokenStream tokenStream, int positionIncrementGap, int offsetGap) Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, LuceneField
.private void
MemoryIndex.storeTerms
(MemoryIndex.Info info, TokenStream tokenStream, int positionIncrementGap, int offsetGap) -
Uses of TokenStream in org.apache.lucene.misc.document
Methods in org.apache.lucene.misc.document that return TokenStreamModifier and TypeMethodDescriptionLazyDocument.LazyField.tokenStream
(Analyzer analyzer, TokenStream reuse) Methods in org.apache.lucene.misc.document with parameters of type TokenStreamModifier and TypeMethodDescriptionLazyDocument.LazyField.tokenStream
(Analyzer analyzer, TokenStream reuse) -
Uses of TokenStream in org.apache.lucene.monitor
Subclasses of TokenStream in org.apache.lucene.monitorModifier and TypeClassDescription(package private) final class
(package private) class
A TokenStream created from aTermsEnum
Methods in org.apache.lucene.monitor that return TokenStreamModifier and TypeMethodDescriptiondefault TokenStream
CustomQueryHandler.wrapTermStream
(String field, TokenStream in) Adds additional processing to theTokenStream
over a document's terms indexRegexpQueryHandler.wrapTermStream
(String field, TokenStream ts) Methods in org.apache.lucene.monitor with parameters of type TokenStreamModifier and TypeMethodDescriptiondefault TokenStream
CustomQueryHandler.wrapTermStream
(String field, TokenStream in) Adds additional processing to theTokenStream
over a document's terms indexRegexpQueryHandler.wrapTermStream
(String field, TokenStream ts) Constructors in org.apache.lucene.monitor with parameters of type TokenStreamModifierConstructorDescriptionSuffixingNGramTokenFilter
(TokenStream input, String suffix, String wildcardToken, int maxTokenLength) Creates SuffixingNGramTokenFilter. -
Uses of TokenStream in org.apache.lucene.queries.intervals
Methods in org.apache.lucene.queries.intervals with parameters of type TokenStreamModifier and TypeMethodDescriptionstatic IntervalsSource
Intervals.analyzedText
(TokenStream tokenStream, int maxGaps, boolean ordered) Returns intervals that correspond to tokens from the providedTokenStream
.private static List
<IntervalsSource> IntervalBuilder.analyzeGraph
(TokenStream source) private static IntervalsSource
IntervalBuilder.analyzeSynonyms
(TokenStream ts, int maxGaps, boolean ordered) private static IntervalsSource
IntervalBuilder.analyzeTerm
(TokenStream ts) private static List
<IntervalsSource> IntervalBuilder.analyzeTerms
(TokenStream ts) -
Uses of TokenStream in org.apache.lucene.sandbox.search
Methods in org.apache.lucene.sandbox.search with parameters of type TokenStreamModifier and TypeMethodDescriptionTokenStreamToTermAutomatonQuery.toQuery
(String field, TokenStream in) Pulls the graph (includingPositionLengthAttribute
) from the providedTokenStream
, and creates the corresponding automaton where arcs are bytes (or Unicode code points if unicodeArcs = true) from each term. -
Uses of TokenStream in org.apache.lucene.search.highlight
Subclasses of TokenStream in org.apache.lucene.search.highlightModifier and TypeClassDescription(package private) final class
This is a simplified version of org.apache.lucene.analysis.miscellaneous.LimitTokenOffsetFilter to prevent a dependency on analysis-common.jar.final class
This TokenFilter limits the number of tokens while indexing by adding up the current offset.final class
TokenStream created from a term vector field.Fields in org.apache.lucene.search.highlight declared as TokenStreamMethods in org.apache.lucene.search.highlight that return TokenStreamModifier and TypeMethodDescriptionstatic TokenStream
TokenSources.getAnyTokenStream
(IndexReader reader, int docId, String field, Analyzer analyzer) Deprecated.static TokenStream
TokenSources.getAnyTokenStream
(IndexReader reader, int docId, String field, Document document, Analyzer analyzer) Deprecated.static TokenStream
TokenSources.getTermVectorTokenStreamOrNull
(String field, Fields tvFields, int maxStartOffset) Get a token stream by un-inverting the term vector.static TokenStream
TokenSources.getTokenStream
(String field, String contents, Analyzer analyzer) Deprecated.static TokenStream
TokenSources.getTokenStream
(String field, Fields tvFields, String text, Analyzer analyzer, int maxStartOffset) Get a token stream from either un-inverting a term vector if possible, or by analyzing the text.static TokenStream
TokenSources.getTokenStream
(Document doc, String field, Analyzer analyzer) Deprecated.static TokenStream
TokenSources.getTokenStream
(IndexReader reader, int docId, String field, Analyzer analyzer) Deprecated.static TokenStream
TokenSources.getTokenStream
(Terms tpv) Deprecated.static TokenStream
TokenSources.getTokenStream
(Terms vector, boolean tokenPositionsGuaranteedContiguous) Deprecated.WeightedSpanTermExtractor.getTokenStream()
Returns the tokenStream which may have been wrapped in a CachingTokenFilter.static TokenStream
TokenSources.getTokenStreamWithOffsets
(IndexReader reader, int docId, String field) Deprecated.QueryScorer.init
(TokenStream tokenStream) QueryTermScorer.init
(TokenStream tokenStream) Scorer.init
(TokenStream tokenStream) Called to init the Scorer with aTokenStream
.private TokenStream
QueryScorer.initExtractor
(TokenStream tokenStream) Methods in org.apache.lucene.search.highlight with parameters of type TokenStreamModifier and TypeMethodDescriptionfinal String
Highlighter.getBestFragment
(TokenStream tokenStream, String text) Highlights chosen terms in a text, extracting the most relevant section.final String[]
Highlighter.getBestFragments
(TokenStream tokenStream, String text, int maxNumFragments) Highlights chosen terms in a text, extracting the most relevant sections.final String
Highlighter.getBestFragments
(TokenStream tokenStream, String text, int maxNumFragments, String separator) Highlights terms in the text , extracting the most relevant sections and concatenating the chosen fragments with a separator (typically "...").final TextFragment[]
Highlighter.getBestTextFragments
(TokenStream tokenStream, String text, boolean mergeContiguousFragments, int maxNumFragments) Low level api to get the most relevant (formatted) sections of the document.WeightedSpanTermExtractor.getWeightedSpanTerms
(Query query, float boost, TokenStream tokenStream) Creates a Map ofWeightedSpanTerms
from the givenQuery
andTokenStream
.WeightedSpanTermExtractor.getWeightedSpanTerms
(Query query, float boost, TokenStream tokenStream, String fieldName) Creates a Map ofWeightedSpanTerms
from the givenQuery
andTokenStream
.WeightedSpanTermExtractor.getWeightedSpanTermsWithScores
(Query query, float boost, TokenStream tokenStream, String fieldName, IndexReader reader) Creates a Map ofWeightedSpanTerms
from the givenQuery
andTokenStream
.QueryScorer.init
(TokenStream tokenStream) QueryTermScorer.init
(TokenStream tokenStream) Scorer.init
(TokenStream tokenStream) Called to init the Scorer with aTokenStream
.private TokenStream
QueryScorer.initExtractor
(TokenStream tokenStream) void
Fragmenter.start
(String originalText, TokenStream tokenStream) Initializes the Fragmenter.void
NullFragmenter.start
(String s, TokenStream tokenStream) void
SimpleFragmenter.start
(String originalText, TokenStream stream) void
SimpleSpanFragmenter.start
(String originalText, TokenStream tokenStream) Constructors in org.apache.lucene.search.highlight with parameters of type TokenStreamModifierConstructorDescription(package private)
LimitTokenOffsetFilter
(TokenStream input, int maxStartOffset) OffsetLimitTokenFilter
(TokenStream input, int offsetLimit) TokenGroup
(TokenStream tokenStream) -
Uses of TokenStream in org.apache.lucene.search.suggest.analyzing
Subclasses of TokenStream in org.apache.lucene.search.suggest.analyzingModifier and TypeClassDescriptionfinal class
LikeStopFilter
except it will not remove the last token if that token was not followed by some token separator.Methods in org.apache.lucene.search.suggest.analyzing that return TokenStreamMethods in org.apache.lucene.search.suggest.analyzing with parameters of type TokenStreamConstructors in org.apache.lucene.search.suggest.analyzing with parameters of type TokenStreamModifierConstructorDescriptionSuggestStopFilter
(TokenStream input, CharArraySet stopWords) Sole constructor. -
Uses of TokenStream in org.apache.lucene.search.suggest.document
Subclasses of TokenStream in org.apache.lucene.search.suggest.documentModifier and TypeClassDescriptionfinal class
AConcatenateGraphFilter
but we can set the payload and provide access to config options.private static final class
TheContextSuggestField.PrefixTokenFilter
wraps aTokenStream
and adds a set prefixes ahead.Fields in org.apache.lucene.search.suggest.document declared as TokenStreamModifier and TypeFieldDescription(package private) final TokenStream
CompletionTokenStream.inputTokenStream
Methods in org.apache.lucene.search.suggest.document that return TokenStreamMethods in org.apache.lucene.search.suggest.document with parameters of type TokenStreamModifier and TypeMethodDescriptionSuggestField.tokenStream
(Analyzer analyzer, TokenStream reuse) protected CompletionTokenStream
ContextSuggestField.wrapTokenStream
(TokenStream stream) protected CompletionTokenStream
SuggestField.wrapTokenStream
(TokenStream stream) Wraps astream
with a CompletionTokenStream.Constructors in org.apache.lucene.search.suggest.document with parameters of type TokenStreamModifierConstructorDescription(package private)
CompletionTokenStream
(TokenStream inputTokenStream) (package private)
CompletionTokenStream
(TokenStream inputTokenStream, boolean preserveSep, boolean preservePositionIncrements, int maxGraphExpansions) PrefixTokenFilter
(TokenStream input, char separator, Iterable<CharSequence> prefixes) Create a newContextSuggestField.PrefixTokenFilter
-
Uses of TokenStream in org.apache.lucene.search.uhighlight
Subclasses of TokenStream in org.apache.lucene.search.uhighlightModifier and TypeClassDescriptionprivate static final class
Wraps anAnalyzer
and string text that represents multiple values delimited by a specified character.Fields in org.apache.lucene.search.uhighlight declared as TokenStreamModifier and TypeFieldDescription(package private) TokenStream
TokenStreamOffsetStrategy.TokenStreamOffsetsEnum.stream
Methods in org.apache.lucene.search.uhighlight that return TokenStreamModifier and TypeMethodDescriptionprotected TokenStream
AnalysisOffsetStrategy.tokenStream
(String content) Methods in org.apache.lucene.search.uhighlight with parameters of type TokenStreamModifier and TypeMethodDescriptionprivate static FilteringTokenFilter
MemoryIndexOffsetStrategy.newKeepWordFilter
(TokenStream tokenStream, CharArrayMatcher matcher) Constructors in org.apache.lucene.search.uhighlight with parameters of type TokenStreamModifierConstructorDescriptionprivate
MultiValueTokenStream
(TokenStream subTokenStream, String fieldName, Analyzer indexAnalyzer, String content, char splitChar, int splitCharIdx) (package private)
TokenStreamOffsetsEnum
(TokenStream ts, CharArrayMatcher[] matchers) -
Uses of TokenStream in org.apache.lucene.util
Methods in org.apache.lucene.util with parameters of type TokenStreamModifier and TypeMethodDescriptionprotected Query
QueryBuilder.analyzeBoolean
(String field, TokenStream stream) Creates simple boolean query from the cached tokenstream contentsprotected Query
QueryBuilder.analyzeGraphBoolean
(String field, TokenStream source, BooleanClause.Occur operator) Creates a boolean query from a graph token stream.protected Query
QueryBuilder.analyzeGraphPhrase
(TokenStream source, String field, int phraseSlop) Creates graph phrase query from the tokenstream contentsprotected Query
QueryBuilder.analyzeMultiBoolean
(String field, TokenStream stream, BooleanClause.Occur operator) Creates complex boolean query from the cached tokenstream contentsprotected Query
QueryBuilder.analyzeMultiPhrase
(String field, TokenStream stream, int slop) Creates complex phrase query from the cached tokenstream contentsprotected Query
QueryBuilder.analyzePhrase
(String field, TokenStream stream, int slop) Creates simple phrase query from the cached tokenstream contentsprotected Query
QueryBuilder.analyzeTerm
(String field, TokenStream stream) Creates simple term query from the cached tokenstream contentsprotected Query
QueryBuilder.createFieldQuery
(TokenStream source, BooleanClause.Occur operator, String field, boolean quoted, int phraseSlop) Creates a query from a token stream. -
Uses of TokenStream in org.apache.lucene.util.graph
Subclasses of TokenStream in org.apache.lucene.util.graphModifier and TypeClassDescriptionprivate class
Methods in org.apache.lucene.util.graph that return types with arguments of type TokenStreamModifier and TypeMethodDescriptionGraphTokenStreamFiniteStrings.getFiniteStrings()
Get all finite strings from the automaton.GraphTokenStreamFiniteStrings.getFiniteStrings
(int startState, int endState) Get all finite strings that start atstartState
and end atendState
.Methods in org.apache.lucene.util.graph with parameters of type TokenStreamModifier and TypeMethodDescriptionprivate Automaton
GraphTokenStreamFiniteStrings.build
(TokenStream in) Build an automaton from the providedTokenStream
.Constructors in org.apache.lucene.util.graph with parameters of type TokenStream
SpanishPluralStemFilter
instead.