Log UI failed to format message from

[pjanzen (Paul Janzen)]

Hi,

I was looking in to Logs UI and I have setup filebeat to send the logs over to ES. However all I see in the Logs UI is an error message saying

Failed to format message from /opt/logstash/logs/logstash-plain.log

Does anyone have any pointers on what to check?

Thanks,
Paul.

[skh (Sonja Krause Harder)]

Hi @pjanzen,

do your documents have a message field? If not, you can configure the Logs UI to use a different field as described here: https://github.com/elastic/kibana/pull/26579/files#diff-3d8e25bfa60ef76c1ce7b5e7a68232f2R9

[pjanzen (Paul Janzen)]

I have the below documents in my filebeat index. I have adjusted the kibana.yml file to reflect the message field but I still get the same error..

xpack.infra.sources.default.fields.message: ["message","logstash.log.message"]

The json looks like this.

{
  "_index": "filebeat-6.5.3-2019.01.07",
  "_type": "doc",
  "_id": "_BuPKGgBV6zRSi0mlcqO",
  "_version": 1,
  "_score": null,
  "_source": {
    "offset": 497073858,
    "prospector": {
      "type": "log"
    },
    "read_timestamp": "2019-01-07T13:45:24.132Z",
    "source": "/opt/logstash/logs/logstash-plain.log",
    "fileset": {
      "module": "logstash",
      "name": "log"
    },
    "tags": [
      "logstash",
      "tb-clog-ls1"
    ],
    "input": {
      "type": "log"
    },
    "logstash": {
      "log": {
        "level": "WARN",
        "module": "org.logstash.dissect.Dissector",
        "message": "Dissector mapping, pattern not found {\"field\"=>\"message\", \"pattern\"=>\"lmtp(%{email}): %{}: msgid=<%{msgid}>: %{action} %{} %{} %{} '%{location}'\", \"event\"=>{\"severity\"=>6, \"host\"=>\"172.25.11.174\", \"severity_label\"=>\"Informational\", \"priority\"=>174, \"logsource\"=>\"host.iss.local\", \"message\"=>\"lmtp(xxxx): mJhlMulXM1yBPAAARICP/Q: sieve: msgid=? <XZ8L9MV488_5c3357e86799e_1ee073f82f32bcf542684ba_sprut@zendesk.com>: stored mail into mailbox 'INBOX'\", \"program\"=>\"dovecot\", \"@timestamp\"=>2019-01-07T13:45:13.000Z, \"source_affiliate\"=>\"nlmail\", \"timestamp\"=>\"Jan  7 14:45:13\", \"@version\"=>\"1\", \"tags\"=>[\"_dissectfailure\"], \"facility\"=>21, \"facility_label\"=>\"local5\"}}"
      }
    },
    "@timestamp": "2019-01-07T14:45:20,630",
    "host": {
      "os": {
        "codename": "xenial",
        "family": "debian",
        "version": "16.04.5 LTS (Xenial Xerus)",
        "platform": "ubuntu"
      },
      "containerized": false,
      "name": "tb-clog-ls1",
      "id": "b7b98b16da1e4f89b37eb536c57ef6dd",
      "architecture": "x86_64"
    },
    "beat": {
      "hostname": "tb-clog-ls1",
      "name": "tb-clog-ls1",
      "version": "6.5.3"
    }
  },

[skh (Sonja Krause Harder)]

Thanks for trying that out.

Unfortunately you seem to have run into an issue that's tracked here: https://github.com/elastic/kibana/issues/26759

cheers,
Sonja

[pjanzen (Paul Janzen)]

Thank you for the pointer, I'll see if I really want Logs UI and create a workaround