Dissector mapping, pattern not found error

[Shubha]

Hi All,

I am having the ELK version 7.10 on ec2 instances.
facing the below error in logstash-plain.log. Can anyone help me on this.

[2021-01-22T10:29:00,423][WARN ][org.logstash.dissect.Dissector][main][1cbe1dfc036a21234e56451763ca81e1416abb2a094feb5ed5d1dbefd461c2e5] Dissector mapping, pattern not found {"field"=>"message", "pattern"=>"%{WORD:event_type} %{NUMBER:time_stamp} %{DATA:REQUEST_ID} %{DATA:ORGANIZATION_ID} %{DATA:USER_ID} %{NUMBER:RUN_TIME} %{NUMBER:CPU_TIME} %{DATA:URI} %{DATA:SESSION_KEY} %{DATA:LOGIN_KEY} %{DATA:REQUEST_STATUS} %{NUMBER:DB_TOTAL_TIME} %{DATA:JOB_ID} %{DATA:ENTRY_POINT} %{TIMESTAMP_ISO8601:TIMESTAMP_DERIVED} %{DATA:USERID_DERIVED} %{IP:CLIENT_IP} %{DATA:URI_ID_DERIVED} %{}", "event"=>{"tags"=>["_dissectfailure"], "@timestamp"=>2021-01-22T10:28:59.904Z, "host"=>"ip-10-218-2-13.eu-west-1.compute.internal", "@version"=>"1", "message"=>"\"QueuedExecution\",\"20210122081531.598\",\"4_lzkE_0xNkvgvX2-pPuc-\",\"00Df2000001Ggxq\",\"005f20000097sks\",\"571\",\"329\",\"SerialBatchApexRangeChunkHandler\",\"\",\"\",\"\",\"164746508\",\"7074O0000BC6XjC\",\"TPM_RefreshProductAndCustomerBatch\",\"2021-01-22T08:15:31.598Z\",\"005f20000097sksAAA\",\"\",\"\"", "path"=>"/opt/sfs_sandbox_logs_hourly_AMS/QueuedExecution/sfdc-hourly-2021-01-22T10:13:21.csv"}} 

Regards,
Shubha

[leandrojmp (Leandro Pereira)]

This error happens because your message does not match your dissect mapping.

From your error log, your dissect mapping is, which is already wrong, this seems like from a grok filter, in dissect you do not use the regex patterns like WORD, NUMBER, DATA etc.

%{WORD:event_type} %{NUMBER:time_stamp} %{DATA:REQUEST_ID} %{DATA:ORGANIZATION_ID} %{DATA:USER_ID} %{NUMBER:RUN_TIME} %{NUMBER:CPU_TIME} %{DATA:URI} %{DATA:SESSION_KEY} %{DATA:LOGIN_KEY} %{DATA:REQUEST_STATUS} %{NUMBER:DB_TOTAL_TIME} %{DATA:JOB_ID} %{DATA:ENTRY_POINT} %{TIMESTAMP_ISO8601:TIMESTAMP_DERIVED} %{DATA:USERID_DERIVED} %{IP:CLIENT_IP} %{DATA:URI_ID_DERIVED} %{}

But your message is:

QueuedExecution,20210122081531.598,4_lzkE_0xNkvgvX2-pPuc-,00Df2000001Ggxq,005f20000097sks,571,329,SerialBatchApexRangeChunkHandler,,,,164746508,7074O0000BC6XjC,TPM_RefreshProductAndCustomerBatch,2021-01-22T08:15:31.598Z,005f20000097sksAAA,,

This does not match, you need to change your dissect mapping and add the commas in your message.

Considering the two first fields.

QueuedExecution,20210122081531.598

Your dissect mapping should be:

%{event_type},%{time_stamp}