Kibana dashboard with "message" filter as a user input

[melkamar (Martin Melka)]

Hi! I am in the process of setting up Filebeat-based monitoring of our Kubernetes workloads' logs. For starters, I want to have a simple page where devs and PMs can go to to view the logs ingested from our deployments.

I need to be able to select what log datapoints/messages to show based on three Kubernetes fields - two custom labels and the Kubernetes namespace. No problem so far, there are the Controls widgets for that.

But then in addition I want there to be an input text field where the user can type anything and this will be used as a search query on the message index field. This is already possible by using the Search bar on top of the page and KQL, but that requires the user to type in the whole query:

message: *<query to search for>*

I want to make this easier and have an input box which would be "linked" to the message field. This is not possible to do with the Controls widget because that only allows for a dropdown and range select from a discrete set of values. On top of that, for some reason the message field cannot even be selected in as the field to filter on.

See the screenshot below for what I mean -

What is the best/recommended way to do this?

Thanks

[stephenb (Stephen Brown)]

First that message field is a text field not a keyword that is why it does not work in the dropdown control only keywords are supported.

There is an interesting way to do this .... it may not be what you want but here is a solution.

In the Index or Index Template you set the default query fields to see here

index.query.default_field

(string or array of strings) Wildcard ( * ) patterns matching one or more fields. The following query types search these matching fields by default:

• More like this
• Multi-match
• Query string
• Simple query string

"index.query.default_field" : ["message"]

That wili then have the effect that in the KQL bar you can put in a KQL search without a field name the KQL search will be performed on the default query field (in this case the message field) and so it will do a full text search on the message field

You don't need to put in the *s either just a word or token you are searching for looks like this...

Note using wildcards is not a great idea (seperate topic) the message field will be tokenized and so any token can be searched on without wildcards ... not that you can't but but if you are looking for fetch in the message field you don't need *fetch*

[melkamar (Martin Melka)]

I see, I wonder if there is a way in Kibana/Elastic to do some additional processing/indexing on top of the text message so that there would be a keyword-type field in the data for me to filter on? Asking where to start looking, if this is something in the responsibility of filebeat before it's sent away, or elastic after it's been received?

The default_field indeed helped, it's a suitable workaround for my case.

Thanks for the reply!