How to create new visualisation component from fields dissected from log in logstash?

[ckong]

Hi, I am currently into Elasticsearch.
I created a filter inside logstash.conf to extract several fields from log message.

File path:
/usr/share/logstash/pipeline/logstash.conf:

Configuration:

input { 
	beats {
	    port => 5044
  	}
} 
filter {
    if [container][name] =~ "docker-elk"  or [container][name] =~ "docker-jenkins" or [container][name] =~ "rabbitmq" {
        drop { }
    }
    dissect {
        mapping => { "message" => "%{?ts} %{log_level} %{component_name} %{guid} %{msg}" }
    }
}
output { 
	elasticsearch { 
		hosts => "elasticsearch:9200"
		manage_template => false
    	index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
	} 
	
    stdout { codec => rubydebug }
}

The fields are available on the discover page.

However , I cannot search those fields on "Dashboard" page when trying to create a new visualisation diagram from those fields.

What should I do to have the fields available when I want to create a new visualisation diagram?

[stephenb (Stephen Brown)]

Hi @ckong welcome to the community.

Quick question what version are you on?

2nd I noticed that your visualization time picker says last 15 minutes, Is the data from the last 15 minutes? If not it would be outside of the visualization time frame and therefore you wouldn't see any fields or data? Make sure your time picker includes the time of your data.

Also it looks like they're using a default mapping meaning you did not define a mapping So your fields we'll have the default mapping which will create both a text and keyword field.

[ckong]

The version of my elk stack is 7.10.2.

I haved tried different time period and I still can't get those fields.

How can I create a mapping for those fields?

[ckong]

After I refresh "Refresh field list", the fields are available.

Thanks for your support @stephenb