How to create new visualisation component from fields dissected from log in logstash?

Hi, I am currently into Elasticsearch.
I created a filter inside logstash.conf to extract several fields from log message.

File path:
/usr/share/logstash/pipeline/logstash.conf:

Configuration:

input { 
	beats {
	    port => 5044
  	}
} 
filter {
    if [container][name] =~ "docker-elk"  or [container][name] =~ "docker-jenkins" or [container][name] =~ "rabbitmq" {
        drop { }
    }
    dissect {
        mapping => { "message" => "%{?ts} %{log_level} %{component_name} %{guid} %{msg}" }
    }
}
output { 
	elasticsearch { 
		hosts => "elasticsearch:9200"
		manage_template => false
    	index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
	} 
	
    stdout { codec => rubydebug }
}

The fields are available on the discover page.
image

However , I cannot search those fields on "Dashboard" page when trying to create a new visualisation diagram from those fields.

What should I do to have the fields available when I want to create a new visualisation diagram?

Hi @ckong welcome to the community.

Quick question what version are you on?

2nd I noticed that your visualization time picker says last 15 minutes, Is the data from the last 15 minutes? If not it would be outside of the visualization time frame and therefore you wouldn't see any fields or data? Make sure your time picker includes the time of your data.

Also it looks like they're using a default mapping meaning you did not define a mapping So your fields we'll have the default mapping which will create both a text and keyword field.

2 Likes

The version of my elk stack is 7.10.2.

I haved tried different time period and I still can't get those fields.

How can I create a mapping for those fields?

After I refresh "Refresh field list", the fields are available.


Thanks for your support @stephenb

2 Likes

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.