Kibana5 data table and message term

Elastic Stack Kibana
1

Hi all,

I'm using kibana 5 and I want to use a data table visualization but it seems that I cannot add the message as a term to it. Anyone know why is that?

Thanks.

2

What is the mapping for the message field?

3

Hi,

I'm using logstash 6 template by default:

{
  "template" : "indexname-*",
  "version" : 60001,
  "settings" : {
    "number_of_shards": "1",
    "number_of_replicas": "0",
    "index.refresh_interval" : "5s"
  },
  "mappings" : {
    "_default_" : {
      "dynamic_templates" : [ {
        "message_field" : {
          "path_match" : "message",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "text",
            "norms" : false
          }
        }
      }, {
        "string_fields" : {
          "match" : "*",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "text", "norms" : false,
            "fields" : {
              "keyword" : { "type": "keyword", "ignore_above": 256 }
            }
          }
        }
      } ],
      "properties" : {
        "@timestamp": { "type": "date"},
        "@version": { "type": "keyword"},
        "geoip"  : {
          "dynamic": true,
          "properties" : {
            "ip": { "type": "ip" },
            "location" : { "type" : "geo_point" },
            "latitude" : { "type" : "half_float" },
            "longitude" : { "type" : "half_float" }
          }
        }
      }
    }
  },
  "aliases": { "alias_name": {} }
}'
4

If that is the case the message field is mapped as a string, which means you can not aggregate on it. Using the message field in a data table does however sound unusual. What is it you are looking to achieve?

5

Our users want to display this information on the dashboard, how can I achieve this? What type of mapping should I use for the field? and more important would that be too cumbersome for the performance of the database/dashboard?
Thanks.

6

If you store your the search you want to display based on as a saved search in the Discover app, you can import this into a dashboard (select saved search instead of visualisation when adding) and show the raw data that way.

7

Yes, I know that but last time we tried that on other index/dashboard we had lot of problems of performance even crashes. If we change the data type of the field message would be worse for the performance?
Thanks

8

How long are your messages? What sample size have you got set in Kibana (advanced settings)?

9

It's quite variable but some of the bigger ones are longer than 16K characters...

10

Have you tried reducing the sample size to see if that helps with the performance problems and crashes?

11

Ok, I will try. I suppose that making such long fields as aggregatable is not a good idea, right?
Thanks.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.