Several log formatting with ansible loop

Elastic Stack Logstash
1

Hi,

I have an ansible dictionary where I define all my logfiles to parse and what it the format of the message:

logstash.conf

grok {
  match => [
"message", '%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:dockerhost} %{DATA:conteneur}(?:\[%{POSINT:pid}\])( time="%{TIMESTAMP_ISO8601:logtime}")?( level=%{LOGLEVEL:severity})?( msg=)?%{GREEDYDATA:message}'
  ],
"message", '%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:dockerhost} %{DATA:conteneur}'
  ],
}

Don't focus on the syntax, because I still don't work on differents formats that message will have, it is just an example.

I get it from logstash.conf.j2

  grok {
    match => [
    {% for log in filebeat_dict.filebeat.logs | default([]) %}
      "message" => "{{ log.format }}",
    {% endfor %}
    ]
  }

Here is my ansible dictionary:

logs:
      - path: "/var/log/docker"
        app_id: "docker"
        format: '%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:dockerhost} %{DATA:conteneur}(?:\[%{POSINT:pid}\])?:?:( time="%{TIMESTAMP_ISO8601:logtime}")?( level=%{LOGLEVEL:severity})?( msg=)?%{GREEDYDATA:message}'
      - path: "/data/nexus/data/log"
        app_id: "nexus"
        format: '%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:dockerhost} %{DATA:conteneur}'

I have 2 questions for that:

  • Is it the correct approach ?
  • For the last line, I will always have a last comma that make logstash fail. How should I get rid of the last comma ???

Thanks for your help!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.