[[analysis-synonym-graph-tokenfilter]]
=== Synonym graph token filter
++++
<titleabbrev>Synonym graph</titleabbrev>
++++

The `synonym_graph` token filter allows to easily handle synonyms,
including multi-word synonyms correctly during the analysis process.

In order to properly handle multi-word synonyms this token filter
creates a <<token-graphs,graph token stream>> during processing. For more
information on this topic and its various complexities, please read the
http://blog.mikemccandless.com/2012/04/lucenes-tokenstreams-are-actually.html[Lucene's TokenStreams are actually graphs] blog post.

["NOTE",id="synonym-graph-index-note"]
===============================
This token filter is designed to be used as part of a search analyzer
only. If you want to apply synonyms during indexing please use the
standard <<analysis-synonym-tokenfilter,synonym token filter>>.
===============================

Synonyms are configured using a configuration file.
Here is an example:

[source,console]
--------------------------------------------------
PUT /test_index
{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "search_synonyms": {
            "tokenizer": "whitespace",
            "filter": [ "graph_synonyms" ]
          }
        },
        "filter": {
          "graph_synonyms": {
            "type": "synonym_graph",
            "synonyms_path": "analysis/synonym.txt"
          }
        }
      }
    }
  }
}
--------------------------------------------------

The above configures a `search_synonyms` filter, with a path of
`analysis/synonym.txt` (relative to the `config` location). The
`search_synonyms` analyzer is then configured with the filter.

Additional settings are:

* `expand` (defaults to `true`).
* `lenient` (defaults to `false`). If `true` ignores exceptions while parsing the synonym configuration. It is important
to note that only those synonym rules which cannot get parsed are ignored. For instance consider the following request:

[source,console]
--------------------------------------------------
PUT /test_index
{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "synonym": {
            "tokenizer": "standard",
            "filter": [ "my_stop", "synonym_graph" ]
          }
        },
        "filter": {
          "my_stop": {
            "type": "stop",
            "stopwords": [ "bar" ]
          },
          "synonym_graph": {
            "type": "synonym_graph",
            "lenient": true,
            "synonyms": [ "foo, bar => baz" ]
          }
        }
      }
    }
  }
}
--------------------------------------------------

With the above request the word `bar` gets skipped but a mapping `foo => baz` is still added. However, if the mapping
being added was `foo, baz => bar` nothing would get added to the synonym list. This is because the target word for the
mapping is itself eliminated because it was a stop word. Similarly, if the mapping was "bar, foo, baz" and `expand` was
set to `false` no mapping would get added as when `expand=false` the target mapping is the first word. However, if
`expand=true` then the mappings added would be equivalent to `foo, baz => foo, baz` i.e, all mappings other than the
stop word.

[discrete]
[[synonym-graph-tokenizer-ignore_case-deprecated]]
==== `tokenizer` and `ignore_case` are deprecated

The `tokenizer` parameter controls the tokenizers that will be used to
tokenize the synonym, this parameter is for backwards compatibility for indices that created before 6.0..
The `ignore_case` parameter works with `tokenizer` parameter only.

Two synonym formats are supported: Solr, WordNet.

[discrete]
==== Solr synonyms

The following is a sample format of the file:

[source,synonyms]
--------------------------------------------------
include::{es-test-dir}/cluster/config/analysis/synonym.txt[]
--------------------------------------------------

You can also define synonyms for the filter directly in the
configuration file (note use of `synonyms` instead of `synonyms_path`):

[source,console]
--------------------------------------------------
PUT /test_index
{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "synonym": {
            "type": "synonym_graph",
            "synonyms": [
              "lol, laughing out loud",
              "universe, cosmos"
            ]
          }
        }
      }
    }
  }
}
--------------------------------------------------

However, it is recommended to define large synonyms set in a file using
`synonyms_path`, because specifying them inline increases cluster size unnecessarily.

[discrete]
==== WordNet synonyms

Synonyms based on https://wordnet.princeton.edu/[WordNet] format can be
declared using `format`:

[source,console]
--------------------------------------------------
PUT /test_index
{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "synonym": {
            "type": "synonym_graph",
            "format": "wordnet",
            "synonyms": [
              "s(100000001,1,'abstain',v,1,0).",
              "s(100000001,2,'refrain',v,1,0).",
              "s(100000001,3,'desist',v,1,0)."
            ]
          }
        }
      }
    }
  }
}
--------------------------------------------------

Using `synonyms_path` to define WordNet synonyms in a file is supported
as well.

[discrete]
==== Parsing synonym files

Elasticsearch will use the token filters preceding the synonym filter
in a tokenizer chain to parse the entries in a synonym file. So, for example, if a
synonym filter is placed after a stemmer, then the stemmer will also be applied
to the synonym entries. Because entries in the synonym map cannot have stacked
positions, some token filters may cause issues here. Token filters that produce
multiple versions of a token may choose which version of the token to emit when
parsing synonyms, e.g. `asciifolding` will only produce the folded version of the
token. Others, e.g. `multiplexer`, `word_delimiter_graph` or `ngram` will throw an
error.

If you need to build analyzers that include both multi-token filters and synonym
filters, consider using the <<analysis-multiplexer-tokenfilter,multiplexer>> filter,
with the multi-token filters in one branch and the synonym filter in the other.

WARNING: The synonym rules should not contain words that are removed by
a filter that appears after in the chain (a `stop` filter for instance).
Removing a term from a synonym rule breaks the matching at query time.

