LogStash::Json::ParserError: Unexpected character ('(' (code 40))

Elastic Stack Logstash
1

I'm getting following error

Error: :exception=>#<LogStash::Json::ParserError: Unexpected character ('(' (code 40)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')

my conf file looks like

input {
        sqs {
                QUEUE_DETAILS
                codec => "plain"
        }
       
}
filter {
        mutate {
                gsub => [
                        "message", "\[", "(",
                        "message", "\]", ")"
                        ]
                }
        json {
                source => "message"
                }
}
output {
        stdout{codec=> "rubydebug" }
        }

my log is

{"output":"04:05:22.832345727: Error File below a monitored directory opened for writing (user=test) k8s.ns=iam-ce k8s.pod=redisearch-replicas-0 container=test k8s.ns=iam-ce k8s.pod=rtest container=test","priority":"Error","rule":"Write below monitored dir","time":"2022-07-12T04:05:22.832345727Z","output_fields":{"cluster":"eks","clustername":"ptest","container.id":"test","container.image.repository":null,"csp":"test","evt.time":1657598722832345727,"fd.name":"/usr/lib/test","k8s.ns.name":"test","k8s.pod.name":"redisearch-replicas-0","proc.aname[2]":null,"proc.cmdline":"test 1","proc.pcmdline":"busybox /health/test.sh 1","proc.pname":"busybox","region":"us-west-2","tenant_id":"123456789012","user.loginuid":-1,"user.name":null,"version":"1.0"}}
2

Do you have anything else in the message? I can't replicate your error using your pipeline and your message as a sample message.

Please share your logstash output for the message that is failing.

This is the output I got, no errors in the json parsing.

{
           "output" => "04:05:22.832345727: Error File below a monitored directory opened for writing (user=test) k8s.ns=iam-ce k8s.pod=redisearch-replicas-0 container=test k8s.ns=iam-ce k8s.pod=rtest container=test",
             "time" => "2022-07-12T04:05:22.832345727Z",
             "host" => "elk-lab",
       "@timestamp" => 2022-07-12T16:26:37.741Z,
         "@version" => "1",
          "message" => "{\"output\":\"04:05:22.832345727: Error File below a monitored directory opened for writing (user=test) k8s.ns=iam-ce k8s.pod=redisearch-replicas-0 container=test k8s.ns=iam-ce k8s.pod=rtest container=test\",\"priority\":\"Error\",\"rule\":\"Write below monitored dir\",\"time\":\"2022-07-12T04:05:22.832345727Z\",\"output_fields\":{\"cluster\":\"eks\",\"clustername\":\"ptest\",\"container.id\":\"test\",\"container.image.repository\":null,\"csp\":\"test\",\"evt.time\":1657598722832345727,\"fd.name\":\"/usr/lib/test\",\"k8s.ns.name\":\"test\",\"k8s.pod.name\":\"redisearch-replicas-0\",\"proc.aname(2)\":null,\"proc.cmdline\":\"test 1\",\"proc.pcmdline\":\"busybox /health/test.sh 1\",\"proc.pname\":\"busybox\",\"region\":\"us-west-2\",\"tenant_id\":\"123456789012\",\"user.loginuid\":-1,\"user.name\":null,\"version\":\"1.0\"}}",
             "rule" => "Write below monitored dir",
         "sequence" => 0,
         "priority" => "Error",
    "output_fields" => {
                            "region" => "us-west-2",
                         "user.name" => nil,
                           "version" => "1.0",
                        "proc.pname" => "busybox",
                     "proc.pcmdline" => "busybox /health/test.sh 1",
                           "cluster" => "eks",
        "container.image.repository" => nil,
                         "tenant_id" => "123456789012",
                           "fd.name" => "/usr/lib/test",
                          "evt.time" => 1657598722832345727,
                      "k8s.pod.name" => "redisearch-replicas-0",
                      "proc.cmdline" => "test 1",
                       "k8s.ns.name" => "test",
                     "user.loginuid" => -1,
                               "csp" => "test",
                       "clustername" => "ptest",
                      "container.id" => "test",
                     "proc.aname(2)" => nil
    }
}
3

just sent you a message with the failed output

4

Please, share in this public topic so more people could see it and maybe offer help.

5 
[WARN ] 2022-07-12 16:21:44.302 [[main]>worker0] json - Error parsing json {:source=>"message", :raw=>"{\"output\":\"03:56:51.478076657: Notice Unexpected connection to K8s API Server from container (command=flb-pipeline -c /fluent-bit/etc/fluent-bit.conf k8s.ns=one-eye-system k8s.pod=one-eye-fluentbit-pxss4 container=test image=\\u003cNA\\u003e:\\u003cNA\\u003e connection=10.10.1.00:12345-\\u003e000.20.0.1:443)\",\"priority\":\"Notice\",\"rule\":\"Contact K8S API Server From Container\",\"time\":\"2022-07-12T03:56:51.478076657Z\",\"output_fields\":{\"cluster\":\"eks\",\"clustername\":\"eks-dev-1\",\"container.id\":\"71a4c10c6395\",\"container.image.repository\":null,\"container.image.tag\":null,\"csp\":\"aws\",\"evt.time\":1657598211478076657,\"fd.name\":\"00.10.0.00:12345-\\u003e000.20.0.1:443\",\"k8s.ns.name\":\"one-eye-system\",\"k8s.pod.name\":\"one-eye-fluentbit-pxss4\",\"proc.cmdline\":\"flb-pipeline -c /fluent-bit/etc/fluent-bit.conf\",\"region\":\"us-east-2\",\"tenant_id\":\"123456789012\",\"version\":\"0.0.1\"},\"source\":\"syscall\",\"tags\":(\"container\",\"k8s\",\"mitre_discovery\",\"network\")}", :exception=>#<LogStash::Json::ParserError: Unexpected character ('(' (code 40)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])"{"output":"03:56:51.478076657: Notice Unexpected connection to K8s API Server from container (command=flb-pipeline -c /fluent-bit/etc/fluent-bit.conf k8s.ns=one-eye-system k8s.pod=one-eye-fluentbit-pxss4 container=test image=\u003cNA\u003e:\u003cNA\u003e connection=00.00.1.41:123456-\u003e000.20.0.1:000)","priority":"Notice","rule":"Contact K8S API Server From Container","time":"2022-07-12T03:56:51.478076657Z","output_fields":{"cluster":"eks","clustername":"eks-dev-1","container.id":"71a4"[truncated 442 bytes]; line: 1, column: 896]>}
6

The issue is with your mutate gsub.

mutate {
    gsub => [
        "message", "\[", "(",
        "message", "\]", ")"
    ]
}

This will change every [ into ( and every ] into ).

It solves the issue with parsing fields with names like proc.aname[2], which will not be parsed if not changed, but will break for fields like tags: [ "tag1", "tag2", "tag3"], that will become tags: ("tag1", "tag2", "tag3"), which is not valid in JSON and is your issue.

Change your gsub to only replace cases where you have [some-digit].

This one should do the trick:

mutate {
     gsub => ["message","\[(\d+)\]", "(\1)"]
}
3 Likes
7

is it always going to replace with "(\1) in the field?

8

\1 is a back-reference to the capture group in the regexp "\[(\d+)\]". The capture group is everything inside the parentheses. So it will convert name[1] to name(1), or name[23] to name(23).

2 Likes
9

Thanks for clearing it. :slight_smile:

10

That worked. Thank you :slight_smile:

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.