Custom Analyzer versus Ingest Pipeline

Elastic Stack Elasticsearch
1

Hello,

I'm having some trouble playing around with a custom analyzer. I created an index with a custom analyzer to map IT to Italy, and reindexed to the new index. The search features work, I can search "Italy" and it finds the documents that contain "IT". However, I want to know if the data can/should be manipulated after the reindex to reflect the change. All my documents still show "IT". I know I can do this with an ingest pipeline, but I'm wondering why it doesn't work with the analyzers.

PUT kibana_sample_data_flights6
{
  "settings": {
    "analysis": {
      "char_filter": {
        "IT_filter": {
          "type": "mapping",
          "mappings": "IT => Italy"
          
        }
      },
      "analyzer": {
        "content_anaylzer": {
          "tokenizer": "keyword",
          "char_filter": ["IT_filter"]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "DestCountry": {
        "type": "text",
        "analyzer": "content_anaylzer"
      }
    }
  }
}
POST _reindex
{
  "source": {
    "index": "kibana_sample_data_flights"
  },
  "dest": {
    "index": "kibana_sample_data_flights6"
  }
}
2

Hi @gigabyte87 : Welcome to the community!
When you work with custom analyzers, they would never change/touch the "_source" form of the data. If you do a simple search : index/_search, you get all your documents under "_source" context. That is your raw data. Analyzers would not change that. However if you do a specific search to search only "Italy" it would fetch you what you are looking for.

In order to change the _source, you might have to go for ingest pipelines or scripting.
Hope that answers your question.

3

Thank you! That makes sense now. I knew I could do it with the ingest pipeline but wasn't sure about analyzers. I just needed some confirmation.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.