How to apply the right stopwords langage depending on the field of a document?

Elastic Stack Elasticsearch
1

Let's say I have an index of News with many documents of different langages. Users have one search bar to look for the title they desire among the different langages. I don't know which langage the user is typing so I would like to remove stopwords depending on the document langage.

This is how I tried to define my index:

{
    name: "news",
    mappings: {
      dynamic: false,
      properties: {
        id: { type: "keyword" },
        title: { type: "text" },
        content: { type: "text" },
        lang: { type: "keyword" },
        created_at: { type: "date" },
        updated_at: { type: "date" },
      },
    },
    settings: {
      analysis: {
        analyzer: {
          default: {
            type: "custom",
            tokenizer: "standard",
            filter: ["my_custom_stop_words_filter"],
          },
        },
        filter: {
          my_custom_stop_words_filter: {
            type: "stop",
            stopwords: "_english_",
            script: {
              source: "lang.getText() === 'english'",
            },
          },
        },
      },
    }

I want to be able to filter out the right stopwords for every documents depending on the field "lang" of each document. How can I achieve this ?

2

Hi @minikali

If the user is searching through the browser, you know the language through the browser itself. Is this not possible for you?

Today I do this, I mapped a field for each language and use the specific analyzer for each language.
Would be like this:

   "title": {
         "type": "text",
         "fields": {
           "pt-br": {
             "type": "text",
             "analyzer": "brazilian"
           },
           "fr": {
             "type": "text",
             "analyzer": "french"
           },
           "en": {
             "type": "text",
             "analyzer": "english"
           },
       }

When I do a search, I know the language because I search the browser and set up the query so that the search is on the field that represents the browser's language.

                      "multi_match":{
                         "query":"test",
                         "fields":[
                            "description.pt-br",
                            "title.pt-br"
                         ]

This works fine with me.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.