Logs UI beta, how it works !?

Elastic Observability Logs
1

Hello,

I finally upgraded to ES 6.5 and was pleasantly surprised by the innovations. Really liked the feature Logs UI (watching logs in real time), but unfortunately I did not find a description of her work. Documentation describes step by step installation from Filebeat to Kibana, but I want to use this feature on my existing indices. Is I understand, they use specific fields from defined indices patterns. (like filebeat-*). Is this parameters configurable?

Best Regards.

2

Hi @dimuskin,

thanks for trying out version 6.5 of the Elastic Stack! :elasticheart:

You're right that the documentation for the Logs UI leaves a lot to be desired.

It is currently designed to work with filebeat out-of-the box. That said, there is some flexibility if you're willing to change the Kibana configuration file (there will be a UI for that soon as well).

  • The index pattern used to read log events can be changed via the xpack.infra.sources.default.logAlias setting, which can contain any index pattern supported by Elasticsearch.
  • The timestamp and sorting tiebreaker fields can be changed via the xpack.infra.sources.default.fields.timestamp and xpack.infra.sources.default.fields.tiebreaker, respectively.
  • The logic to read the message from the individual documents looks at several fields specific to filebeat modules first, but then falls back to the message and @message fields.

That means no matter what the ingestion pipeline is, as long as it is possible to formulate an index pattern and structure the documents therein such that they contain timestamp and message/@message fields, the Log UI should pick them up, e.g.:

xpack.infra:
  sources:
    default:
      logAlias: 'logstash-*'
      fields:
        timestamp: 'my_timestamp_field'
        tiebreaker: 'my_line_number_field'

We would be very interested in hearing whether this worked for you and what other improvements you would wish for.

3

Thanks for the quick response,

yes, in my case I got the expected result, but I agree with you, component need more convenient management (like managment from UI)

Thanks.

4

Hello could you please clarify where the file xpack.infra.sources.default.logAlias is?

Thanks
G

5

it's a not filename, it's param in elasticsearch.yml config file

Best Regards,

6

That's how my file now looks like:

bootstrap.memory_lock: false
cluster.name: elasticsearch-cluster-orchestrator
http.port: 9200
network.host: LAPTOP-1PHLDDMB
node.data: true
node.ingest: true
node.master: true
node.max_local_storage_nodes: 1
node.name: elasticsearch-node
path.data: C:\ProgramData\Elastic\Elasticsearch\data
path.logs: C:\ProgramData\Elastic\Elasticsearch\logs
transport.tcp.port: 9300
xpack.license.self_generated.type: basic
xpack.security.enabled: false
xpack.infra:
  sources:
default:
  logAlias: 'logstash-*'
  fields:
    timestamp: 'my_timestamp_field'
    tiebreaker: 'my_line_number_field'
7

From: https://github.com/elastic/kibana/issues/26769

"The index pattern can be adjusted via the Kibana configuration file (usually kibana.yml )"

8

As @phr0gz correctly points out, these settings have to go into the Kibana config file, not the Elasticsearch config file.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.