Logs UI - Selectable message fields?

[ceekay]

I'd like to use the Logs UI for a bit more than plain syslog (e.g., Apache all the logs) however I can't get it to display anything other than a plain message field. This is not ideal when displaying non-syslog logs for more than one host as it's impossible to tell the source of an entry in the Logs UI if it doesn't contain the hostname. Even if you do identify an interesting log entry, you can't do anything with it as there's no interaction in the UI, so you need to go and find it again in the Discover tab.

I've tried playing with the setting xpack.infra.sources.default.fields.message: ['message', '@message'] by adding extra fields but it doesn't have any effect whatsoever on the interface. In fact, this setting appears to be hard-coded, as even completely replacing the default fields does nothing to the interface (unless I'm misunderstanding its purpose).

So, is it possible to customise the output at all? Ideally I'd like to be able to set up a table of sorts, e.g., [timestamp], [hostname], [severity], [message] or variations on this depending on the log type.
Perhaps a setting like xpack.infra.sources.default.fields.columns would allow people to format their own output by providing a list of event fields?

Also, are there any plans to add interactivity to this interface, (i.e., clickable log entries to create filters, etc.) At the moment it just seems to be a big combined tail of everything.

[Kerry (Kerry Gallagher)]

Hi @ceekay,

We're aware the Logs UI message formatting is not quite optimal yet.

As it stands we use a heuristic to attempt to extract the log message from the fields of a few select and known Filebeat modules, if this fails it falls back to the message and @message attribute of the document. There is a meta issue for that here.

For now you could copy the desired fields of the log event to the message attribute during ingestion, to display the information you desire.

We are working on making all of this configurable via the UI. There is a meta issue here for tracking a lot of the groundwork that needs to happen first, before we can add that configurability to the UI: [Infra UI] Source Settings UI · Issue #26539 · elastic/kibana · GitHub. We can't provide any ETAs, it would be best to track the issues.

Perhaps a setting like xpack.infra.sources.default.fields.columns would allow people to format their own output by providing a list of event fields?

I don't think we have this planned on our roadmap, but I'll log the feedback, and thank you for the suggestion.

Also, are there any plans to add interactivity to this interface, (i.e., clickable log entries to create filters, etc.) At the moment it just seems to be a big combined tail of everything.

Yep - we are looking to improve the Logs UI, we have flyout functionality to show the details of log events in review now: [Log UI] Flyout for Log Events by simianhacker · Pull Request #28885 · elastic/kibana · GitHub, and more functionality planned.

[ceekay]

Hi @Kerry,

Thanks for the update. Regarding columns, it looks like it's being discussed (somewhat) at https://github.com/elastic/kibana/issues/29415 and as everything else looks to be on the roadmap I'll just sit tight for now.

Cheers.