Kibana data tables visualizations seperate fields on punctuation. How can I combine them?

[Ambrose]

I'm sorting through some security logs that include windows paths as a data table visualization in Kibana and notice that the table splits up the Path field based on the non alphabet characters in the path. This of course throws off my data table significantly.

The actual fields, when combined, should look like this:

But this what it looks like in data table visualization. See "Path:Descending" for the offending field.

How can I combine these fields together so that the Path field divides and displays data properly in my data table visualization?

Thank you very much.

[jbudz (Jon Budzenski)]

Path and Process are analyzed fields which means elasticsearch will tokenize the field into these pieces, https://www.elastic.co/guide/en/elasticsearch/guide/current/analysis-intro.html has more info. If you have a not_analyzed version of this field available you'll want to use that, otherwise the fix for this is to modify your elasticsearch mappings to make a not_analyzed version.

[Ambrose]

Thank you for the reply. You are absolutely correct, I need to turn all the string fields I have in my data to "not_analyzed" rather than "string" so they show up in my visualizations correctly. The problem is I am having a difficult time figuring out exactly the way to do that.

Some of the help documents that I have been searching through (stack exchange, elastic discussion boards, and elastic documentation) point out that I may have a "not_analyzed" version of this field available to me, but despite my searches for this I have been unable to find these. When I attempt to point Kibana to my indexes it shows all the fields as "analyzed", as shown in the screen shot below. If there are "not_analyzed" versions of these fields available, I'm not sure where to find them.

So then continuing to research, I attempted to use the REST API to update the field I want to index as "not_analyzed". I discovered I could enter the following commmand:

curl -XGET '192.168.1.1:9200/ports/_mapping/ports/field/Path/?pretty'

to get these results:

{"ports":{"mappings":{"ports":{"Path":{"full_name":"Path","mapping":{"Path":{"type":"string"}}}}}}}

Awesome, I am getting closer. Now I just have to add one more field to it. So I entered this:

curl -XPUT '192.168.1.1:9200/ports/_mapping/ports/field/Path/' '{"ports":{"mappings":{"ports":{"Path":{"full_name":"Path","mapping":{"Path":{"type":"string","index":"not_analyzed"}}}}}}}'

which, in theory, should just add the "index":"not_analyzed" field into the mapping for the Ports index, Path field, but instead I received this error:

No handler found for uri [/ports/_mapping/ports/field/Path/] and method [PUT]curl: (3) [globbing] nested brace in column 10

Apparently I am doing something wrong here but I am unsure what. Is there any more specific guidance you can recommend for getting this Path field to index as "not_analyzed"?

Thank you in advance.

[JuanCarniglia (Juan Ignacio Carniglia)]

I think the URL has to be "server:port/index/type/_mapping" also you would probably have to remove the index before changing the mapping.

[Ambrose]

So I figured it out.

As was pointed out, it was necessary to change the fields that I wanted to visualize as "not_analyzed" in the mappings. I figured out how to do that by following this guide:

Update mapping API | Elasticsearch Guide [8.11] | Elastic

While this had the effect of changing all of the fields in my index to "not_analyzed" it unfortunately made it such that Kibana could no longer see the data anymore. After Pointing to that index, Kibana knew that there was data there but couldn't see it in either the visualize or the discover tab.

After almost giving up on this entire ELK project in general, I found this topic:

Confused about how to use .raw fields and not analyze string fields

I apologize if this is too newbie a question but I am struggling with setting up a terms visualization in Kibana 4 because the string field with the terms is analysed. I am using logstash with the elasticsearch output to populate my index. In my searching around the web, my understanding is that the default logstash template for ES creates a multi-field for each string field where one of the fields is .raw. So somewhere in my index is the ability to use a not_analyzed version of my field. I thi…

Which points out that the .raw fields exist as long as the index starts with "logstash-". I was renaming my index something else that does not have "logstash-" in it and it wasn't matching properly with some default template I have somewhere.

Once I renamed all my indexes to start with "logstash-", Kibana started seeing the .raw fields and I was able to do my visualizations and discoveries correctly.