How to create new visualisation component from fields dissected from log in logstash?

Elastic Stack Kibana
,
1

Hi, I am currently into Elasticsearch.
I created a filter inside logstash.conf to extract several fields from log message.

File path:
/usr/share/logstash/pipeline/logstash.conf:

Configuration:

input { 
	beats {
	    port => 5044
  	}
} 
filter {
    if [container][name] =~ "docker-elk"  or [container][name] =~ "docker-jenkins" or [container][name] =~ "rabbitmq" {
        drop { }
    }
    dissect {
        mapping => { "message" => "%{?ts} %{log_level} %{component_name} %{guid} %{msg}" }
    }
}
output { 
	elasticsearch { 
		hosts => "elasticsearch:9200"
		manage_template => false
    	index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
	} 
	
    stdout { codec => rubydebug }
}

The fields are available on the discover page.
image

However , I cannot search those fields on "Dashboard" page when trying to create a new visualisation diagram from those fields.

image
image1456×682 84.7 KB

What should I do to have the fields available when I want to create a new visualisation diagram?

2

Hi @ckong welcome to the community.

Quick question what version are you on?

2nd I noticed that your visualization time picker says last 15 minutes, Is the data from the last 15 minutes? If not it would be outside of the visualization time frame and therefore you wouldn't see any fields or data? Make sure your time picker includes the time of your data.

Also it looks like they're using a default mapping meaning you did not define a mapping So your fields we'll have the default mapping which will create both a text and keyword field.

3

The version of my elk stack is 7.10.2.

I haved tried different time period and I still can't get those fields.

image
image1919×618 77.5 KB

How can I create a mapping for those fields?

4

After I refresh "Refresh field list", the fields are available.

image
image1421×752 72.7 KB

image
image1066×583 56.9 KB

Thanks for your support @stephenb