Question regarding question 4 of the Observability practice exam

[EvanKara]

Hi All,

I am attempting question 4 of the practice exam, and am stuck on how to use the dissect processor to escape the "\t" character:

The Input sample pipeline document is as follows:

{
"_source": {
"message": "2020-10-22T20:28:26.267397Z\t 12 Query\tset session transaction read write"
}
}

The Dissect string pattern I am using is as follows:

%{@timestamp}\t %{process.thread.id} %{event.action}\t%{message}

However, I keep getting the following error when testing it using the sample document:

{
  "docs": [
    {
      "error": {
        "root_cause": [
          {
            "type": "find_match",
            "reason": "Unable to find match for dissect pattern: %{@timestamp}\\t %{process.thread.id} %{event.action}\\t%{message} against source: 2020-10-22T20:28:26.267397Z\t 12 Query\tset session transaction read write"
          }
        ],
        "type": "find_match",
        "reason": "Unable to find match for dissect pattern: %{@timestamp}\\t %{process.thread.id} %{event.action}\\t%{message} against source: 2020-10-22T20:28:26.267397Z\t 12 Query\tset session transaction read write"
      }
    }
  ]
}

Any ideas on how to escape the "\t" character would be appreciated.

[dinyes (Will Dinyes)]

I'm not entirely sure what the issue might be, the "answer" (or at least, an answer) is:

"dissect": {
      "field": "message",
      "pattern": "%{@timestamp}\t%{->} %{process.thread.id} %{event.action}\t%{message}"
}

Which does seem to mostly match what you've written.

You could try using the Load JSON button, and pasting this complete solution code in:

{
"processors": [
  {
    "dissect": {
      "field": "message",
      "pattern": "%{@timestamp}\t%{->} %{process.thread.id} %{event.action}\t%{message}"
    }
  },
  {
    "lowercase": {
      "field": "event.action"
    }
  },
  {
    "set": {
      "field": "event.category",
      "value": "database"
    }
  }
]
}

[EvanKara]

Hi Will,

Thanks a lot J

Also, with regards to entering a tab character in the ingest dissect processor pattern, there was an issue depending on what version of ELK that was being used that I have found:

· For v7.9.1: Need to type "\t"

· For v7.10.1: Need to copy and paste the actual tab space character (i.e." ", with no spaces) from an editor like notepad or notepad++, and not from vi or nano in the linux terminal

Will we have access to a graphical editor similar to notepad or notepad++ during the exam? We require the tab character from an editor similar to these, as apparently the vi or nano editor provided by the terminal does not provide an adequate tab character needed for the ingest pipeline processor dissect in kibana. Does the On-screen keyboard in the exam desktop contain this tab space character that can be copied onto the dissect pattern?

Regards,

Evangelo Karagiannis

0431 826 759

[Rich_Raposa]

FWIW, none of this conversation about tab characters is an issue on the exam.

[dinyes (Will Dinyes)]

And when the head of Certification says it's not an issue, it's not an issue.

Thanks for looking into the bug here, though! Clearly we picked a less than ideal log format for the practice exam!

[Stuart_Duncan]

Further question on this one (and understood its not relevant for the exam!) - the Dissect processor works fine for the sample log source given in the Practice Exam Question:

  {
    "_source": {
      "message": "2020-10-22T20:28:26.267397Z\t   12 Query\tset session transaction read write"
    }
  }

However, the first part of the exam question gives another example log line, that does NOT work with the defined dissect:

Read lines from the /var/lib/mysql/mysql.log file, which have the format:

                2020-10-22T20:29:05.942482Z\t 12 Query\tSET autocommit=1

E.g. when I test with:

{
    "_source": {
      "message": "2020-10-22T20:29:05.942482Z\t 12 Query\tSET autocommit=1"
    }
  } 

I get:

{
      "error": {
        "root_cause": [
          {
            "type": "find_match",
            "reason": "Unable to find match for dissect pattern: %{@timestamp}\t%{->} %{process.thread.id} %{event.action}\t%{message} against source: 2020-10-22T20:29:05.942482Z\t 12 Query\tSET autocommit=1"
          }
        ],
        "type": "find_match",
        "reason": "Unable to find match for dissect pattern: %{@timestamp}\t%{->} %{process.thread.id} %{event.action}\t%{message} against source: 2020-10-22T20:29:05.942482Z\t 12 Query\tSET autocommit=1"
      }
    }

I guess this is because there is only a single space between the '12' and the timestamp in this log, vs multiple spaces in the other line? But I'd have thought that %{->} would have catered for this.

Is this a bug, or would it be worth updating the practice exam question to have consistent whitespace in both example logs?

This is tested in Strigo on 7.9.1.

[EvanKara]

Hi Stuart,

What version of Kibana are you using? I have found that bugs vary depending on what Kibana version you are using.

[Stuart_Duncan]

Hi Evan, I am testing against 7.9.1.

[Stuart_Duncan]

Just tested in a different environment against 7.11.0 and it works correctly there, so looks like this was indeed a bug in 7.9.1.