Dissect Processor not working correctly in elastic search

Elastic Stack Elasticsearch
1

Hi Team,

I am trying to parse logs using dissect processor in elastic search . However I dont get the appropriate output. When I run the log through logstash using dissect filter, it works fine.

`

PUT _ingest/pipeline/my_pipeline_id_saket
{
"description" : "describe pipeline",
"processors" : [
{
"dissect": {
"field": "message",
"pattern": "%{log_TraceId}, %{log_SpanId}, %{log_UserId}, [%{ip},"
}
}
]

}

POST test/_doc/3?pipeline=my_pipeline_id_saket
{
"message" : "b5721cae-573f-4da5-90dd-2f1c7783b21d, b5721cae-573f-4da5-90dd-2f1c7783b21d, , [0:0:0:0:0:0:0:1,"

}

GET test/_doc/3

`

This gives the output :-

"log_TraceId" : "b5721cae-573f-4da5-90dd-2f1c7783b21d", "log_UserId" : "", "ip" : "[0:0:0:0:0:0:0:1", "log_SpanId" : "b5721cae-573f-4da5-90dd-2f1c7783b21d"
However the output in logstash (correct & expected):-
{ "ip": "0:0:0:0:0:0:0:1", "log_SpanId": "b5721cae-573f-4da5-90dd-2f1c7783b21d", "log_TraceId": "b5721cae-573f-4da5-90dd-2f1c7783b21d", "log_UserId": "" }

The Ip address does not get parsed correctly and has a '[' added to it which is a delimiter.

Thanks
Saket

2

If I recall correctly I think the dissect pattern need to end with a match clause, so I would recommend adding and empty one at the end and see if that makes a difference.

3

Thanks for your reply. Sorry I could not understand " adding and empty one at the end and see if that makes a difference.". Could you please give an example ?

4

Was on my mobile so could not provide an example. Meant something like this:

"pattern": "%{log_TraceId}, %{log_SpanId}, %{log_UserId}, [%{ip},%{}"
5

THanks for your reply. I just did try that unfortunately I still get a '[' in my ip

6

If the same config in Logstash produces a different result it sounds like a bug.

7

Yes it does..!! Thanks for your reply.

8

Hi,

Humble Request, if anyone from Elastic Team can look into this ?

Thanks
Saket

9

If it's a bug, could you open an issue in elasticsearch GitHub repository?

You should link to this thread from the issue.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.