Decode json in _source into fields


I'm using Filebeat and Logstash to analyze logs from Wazuh, the logs are json files where each line is a json string, I can read the log correctly and push it into Elasticsearch, through Logstash, and it correctly populate the fields, but in the discover tab on Kibana, the _source field shows the raw json.

Is there any way to decode the _source raw json and show the fields in the discover tab?

This is what I'm getting now:

This is the way I want the _source field to behave:

My filebeat configuration is:

  - input_type: log
     - "/var/ossec/logs/alerts/alerts.json"
    document_type: wazuh-ossec
    codec: json_lines
    json.message_key: log
    json.keys_under_root: true
    json.overwrite_keys: true

   hosts: [""]

My logstash pipeline for this index is:

input {
	beats {
        host    => ""
        port    => 5000
        codec   => "json_lines"
        type    => "wazuh-ossec"
filter {
	if [type] == "wazuh-ossec" {
	        geoip {
                source => "srcip"
                target => "GeoLocation"
                fields => ["city_name", "continent_code", "country_code2", "country_name", "region_name", "location"]
	        date {
                match => ["timestamp", "ISO8601"]
                target => "@timestamp"
	        mutate {
                remove_field => [ "timestamp", "beat", "fields", "input_type", "tags", "count", "@version", "log", "offset"]
output {
	if [type] == "wazuh-ossec" {
	        elasticsearch {
                hosts                   => ["localhost:9200"]
                index                   => "wazuh-alerts-%{+YYYY.MM.dd.HH}"
                codec                   => "json_lines"
                document_type           => "wazuh-ossec"
                template                => "/etc/logstash/wazuh-elastic5-template.json"
                template_name           => "wazuh"
                template_overwrite      => true

What am I missing?


With the help from the guys from the Wazuh mailing list I was able to solve the problem.

The solution was to explicit declare the _source field as the value on the metaFields variable in the advanced settings on Kibana.

I don't know if this is the normal behavior, if I missed something or if it is a bug, but explicit declaring the values on the metaFields variable solves the problem.

Just posting the solution before the topic closes.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.