Discover tab shows data but table vizualization shows empty

[katara]

Hi,
Heres the data in Discover tab

And when the same is included in the Data table vizualization, the message is blank.

I tried using top hit as well, all shows blank.
Please help me fix this.

Thanks!

[matw (Matthias Wilhelm)]

Hi Karara

There are some warning icons displayed, could you hover the one at log message with the mouse and tell me what kind of warning is displayed? Think your problem might have to do with this.

Thx & Best,
Matthias

[katara]

@matw, Thank you. I am not using those fields with a warning,
these fields are rather encoded with base64. which could be the reason.

I tried saving a discover table with appropriate columns, which works well, but a data table seems to ignore all columns with large character counts.

Katara.

[matw (Matthias Wilhelm)]

could you paste an example document here and the mapping in Elasticsearch you're using? many thx!
Matthias

[katara]

This data eventually comes from a log shipper,
and the below is my logstash configuration:

input {
        tcp {
                port => 8443
                codec => json_lines { charset => CP1252 }
                }
}
filter
{
ruby {
    code => "
        event.set('logmessage', Base64.encode64(event.get('[Message]')))
        event.set('device', Base64.encode64(event.get('[Hostname]')))
        event.set('sev', Base64.encode64(event.get('[Severity]')))
        event.set('tagpath', Base64.encode64(event.get('[tag_filepath]')))
        event.set('appname', Base64.encode64(event.get('[ApplicationName]')))
    "
}
mutate {
        gsub => [
          # replace all forward slashes with underscore
          "Hostname", " ", "_",
          "Severity", " ", "",
          "tag_filepath", " ", "_",
          "ApplicationName", " ", "_",
          "Message", " ", "_"
        ]
      }
}
output {
elasticsearch {
    hosts => ["10.1.1.1:9200"]
    user => "elastic"
    password => "*******"
    index => "nxlogapps"
}
stdout { codec => rubydebug }
}

[matw (Matthias Wilhelm)]

Visualization usually use the keyword type of Elasticsearch for such cases, could you check the ignore_above settings of the indices mappings? Could be a reason why longer messages are ignored https://www.elastic.co/guide/en/elasticsearch/reference/current/keyword.html