How to allow elastisearch for proper indexing of logs from logstash/filebeat

Elastic Stack Elasticsearch
1

Hi!

My intention is to store logs in elastisearch to be able search the logs via severity, thread, request id, etc.
My logs have already form of JSON:

{
  'severity': 'INFO',
  'requestId': '24fd34',
  'message': 'Saving a person',
  ....
}

Now my question is how to do this correctly using a) logstash and b) filebeat.

a) Logstash

I understand that to inform elasticsearch that in future I would like to search my log via above attributes I need to use "codec => json" in elasticsearch plugin. If I am wrong in this point please correct me.

b) Filebeat

The question is if for filebeat sth similar is possible? Can I just read from json logs, directly save them to elastisearch and search via severity/requestId/etc afterwards?

Because with following setup:

output.elasticsearch:
      # Array of hosts to connect to.
      hosts: ["url:9243"]

      # Optional protocol and basic auth credentials.
      protocol: "https"
      username: "elastic"
      password: "password"
      index: "filebeat"

the log line is treated as a whole piece. And I am able to search it only via timestamp.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.