I'm confused: Keyword Mapping and Multi-fields

[tomx1]

I'm trying to switch our logging data to ECS. So I've adapted our logstash config to use some new field names and created a static mapping in the destination index. But the data is not indexed as keyword as I would expect,... in this example I'll show you the 'url.full' field (URL Fields | Elastic Common Schema (ECS) Reference [8.11] | Elastic).

My mapping for that field looks like that:

Request:

GET logstash-operations-2021.07.09/_mapping/field/url.full

Response:

{
  "logstash-operations-2021.07.09" : {
    "mappings" : {
      "url.full" : {
        "full_name" : "url.full",
        "mapping" : {
          "full" : {
            "type" : "keyword",
            "fields" : {
              "text" : {
                "type" : "text"
              }
            }
          }
        }
      }
    }
  }
}

So I think thats consistent with what the ECS recommends:
url.full > Keyword
url.full.text > Text (as Multi Field)

But due some reason, data that is stored into the url.full field is not analyzed as Keyword:

Request:

GET logstash-operations-2021.07.09/_analyze
{
"field": "url.full",
"text": "https://example.com/api/v1/suggest?term=monitor_qrsew"
}
Response:

{
  "tokens" : [
    {
      "token" : "https://example.com/api/v1/suggest?term=monitor_qrsew",
      "start_offset" : 0,
      "end_offset" : 53,
      "type" : "word",
      "position" : 0
    }
  ]
}

If I do the same request with target 'url.full.text' I see that the data is parsed as a keyword,... so exactly the opposite of what I would expect. As the data in 'url.full' is stored as text and the data in 'url.full.text' is stored as keyword.

{
  "tokens" : [
    {
      "token" : "https",
      "start_offset" : 0,
      "end_offset" : 5,
      "type" : "<ALPHANUM>",
      "position" : 0
    },
    {
      "token" : "example.com",
      "start_offset" : 8,
      "end_offset" : 19,
      "type" : "<ALPHANUM>",
      "position" : 1
    },
    {
      "token" : "api",
      "start_offset" : 20,
      "end_offset" : 23,
      "type" : "<ALPHANUM>",
      "position" : 2
    },
    {
      "token" : "v1",
      "start_offset" : 24,
      "end_offset" : 26,
      "type" : "<ALPHANUM>",
      "position" : 3
    },
    {
      "token" : "suggest",
      "start_offset" : 27,
      "end_offset" : 34,
      "type" : "<ALPHANUM>",
      "position" : 4
    },
    {
      "token" : "term",
      "start_offset" : 35,
      "end_offset" : 39,
      "type" : "<ALPHANUM>",
      "position" : 5
    },
    {
      "token" : "monitor_qrsew",
      "start_offset" : 40,
      "end_offset" : 53,
      "type" : "<ALPHANUM>",
      "position" : 6
    }
  ]
}

What am I missing here?

PS: Thats how the mapping config looks like in Kibana:

[dadoonet (David Pilato)]

@tomx1

But due some reason, data that is stored into the url.full field is not analyzed as Keyword:

It is analyzed as a keyword data type. Which means no analysis happening on that field.

@tomx1

If I do the same request with target 'url.full.text' I see that the data is parsed as a keyword,..

No. It's analyzed as a text. Meaning that the analysis process is happening on that field.

So everything is good here.

[tomx1]

Thanks David, seems like I have confused here some basic things...

May I ask you a second question: The "url.*" field does not show up in Kibana search results (under discover). I see the field if I click on the JSON tab (for a single document) and I can do a search for it (like: url.full:*example.com*). But due some reason I can't see it here:

[dadoonet (David Pilato)]

I don't know. May be you need to refresh the mapping in Kibana? Under Kibana management, update the index pattern?