### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 4, 2021, 6:08pm UTC](https://discuss.elastic.co/t/logstash-error/288444/1 "2021-11-04T18:08:08Z")
Hi Team,
I'm getting exception=\>#\ "abc"
secret_access_key => "def"
queue => "queue_name"
region => "us-east-1"
codec => "line"
}
}
```
How i can avoid that error?
---
### Author:  [@zx8086](https://discuss.elastic.co/u/zx8086)
#### Post date: [November 4, 2021, 6:18pm UTC](https://discuss.elastic.co/t/logstash-error/288444/2 "2021-11-04T18:18:01Z")
Can you share the full error from the logstash error log ?
It seems you are using a wrong field or parameter.
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 4, 2021, 6:23pm UTC](https://discuss.elastic.co/t/logstash-error/288444/3 "2021-11-04T18:23:55Z")
Please see the error
```auto
[2021-11-04T18:19:55,109][ERROR][logstash.codecs.json]
[main][9e4478322092ac46867421da34e83caeaf3c3c469d1a01e83f6d915678e5a9a2]
JSON parse error, original data now in message field
{:message=>"Invalid FieldReference: `proc.aname[2]`",
:exception=>LogStash::Json::ParserError,
:data=>"{\"output\":\"18:19:55.042663479: some message (user=root user_loginuid=-1 command=httpd --loglevel info run ^syscall.ReadSensitiveFileUntrusted$ --sleep 6s parent=httpd file=/etc/shadow parent=httpd gparent=containerd-abc container_id=123 image=abc/event-generator) k8s.ns=abc-logstash k8s.pod=abc-def-ghi container=123 k8s.ns=abc-logstash k8s.pod=abc-def-ghi container=123\",\"priority\":\"Warning\",\"rule\":\"Read sensitive file trusted after startup\",\"time\":\"2021-11-04T18:19:55.042663479Z\",\"output_fields\":{\"clustername\":\"eks-logstash-test\",\"container.id\":\"123\",\"container.image.repository\":\"abc/event-generator\",\"cloud\":\"aws\",\"evt.time\":1636049995042663479,\"fd.name\":\"/etc/shadow\",\"k8s.ns.name\":\"abc-logstash\",\"k8s.pod.name\":\"abc-def-ghi\",\"proc.aname[2]\":\"containerd-shim\",\"proc.cmdline\":\"httpd --loglevel info run ^syscall.ReadSensitiveFileUntrusted$ --sleep 6s\",\"proc.pname\":\"httpd\",\"user.loginuid\":-1,\"user.name\":\"root\",\"version\":\"abc-def\"}}"}
```
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [November 4, 2021, 6:26pm UTC](https://discuss.elastic.co/t/logstash-error/288444/4 "2021-11-04T18:26:54Z")
Can you share your full pipeline? You just shared your input, your error seems to be coming from a `json` filter in your pipeline.
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 4, 2021, 6:28pm UTC](https://discuss.elastic.co/t/logstash-error/288444/5 "2021-11-04T18:28:21Z")
This is how i defined my conf file
```auto
input {
sqs {
access_key_id => "abc"
secret_access_key => "def"
queue => "queue_name"
region => "us-east-1"
codec => "line"
}
}
filter {
json {
source => "message"
}
mutate {
remove_field => ["@timestamp", "host", "@version"]
}
}
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 4, 2021, 6:55pm UTC](https://discuss.elastic.co/t/logstash-error/288444/6 "2021-11-04T18:55:06Z")
It is a known [issue](https://github.com/logstash-plugins/logstash-filter-json/issues/38).
You could try something like
```
mutate { gsub => ["message", "\[\d+\]", "" ] }
```
to adjust the field name before trying to parse it.
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 4, 2021, 8:01pm UTC](https://discuss.elastic.co/t/logstash-error/288444/7 "2021-11-04T20:01:48Z")
Do i need to keep json filter or i can remove that?
Will the below pipleine work?
```auto
filter {
json {
source => "message"
}
mutate {
remove_field => ["@timestamp", "host", "@version"]
}
mutate { gsub => ["message", "\[\d+\]", "" ] }
}
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 4, 2021, 8:12pm UTC](https://discuss.elastic.co/t/logstash-error/288444/8 "2021-11-04T20:12:08Z")
You still need the json filter, but the mutate must come first
```
mutate { gsub => ["message", "\[\d+\]", "" ] }
json {
source => "message"
remove_field => ["@timestamp", "host", "@version"]
}
```
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 4, 2021, 8:27pm UTC](https://discuss.elastic.co/t/logstash-error/288444/9 "2021-11-04T20:27:48Z")
> [@Badger](#):
>
> ```auto
> mutate { gsub => ["message", "\[\d+\]", "" ] }
> json {
> source => "message"
> remove_field => ["@timestamp", "host", "@version"]
> }
>
> ```
i tried that but still getting the same error
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 4, 2021, 9:09pm UTC](https://discuss.elastic.co/t/logstash-error/288444/10 "2021-11-04T21:09:57Z")
> [@John\_snow](#):
>
> `[ERROR][logstash.codecs.json]`
You cannot use a json codec if the JSON keys contains things like [2]. You will need to use a json filter instead.
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 4, 2021, 9:17pm UTC](https://discuss.elastic.co/t/logstash-error/288444/11 "2021-11-04T21:17:06Z")
> [@John\_snow](#):
>
> ERROR][logstash.codecs.json]
`ERROR][logstash.codecs.json]`
Sorry i did'n get that ?
Codecs which i'm using in input?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 4, 2021, 9:19pm UTC](https://discuss.elastic.co/t/logstash-error/288444/12 "2021-11-04T21:19:55Z")
The error message that you posted came from a codec. That means the error has already occurred in the input, before the message is sent to the pipeline and the mutate can fix it before the json _filter_ parses it.
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 4, 2021, 9:31pm UTC](https://discuss.elastic.co/t/logstash-error/288444/13 "2021-11-04T21:31:44Z")
> [@Badger](#):
>
> `mutate { gsub => ["message", "\[\d+\]", "" ] }`
making sense now.
What does d mean here in gsub
```auto
mutate { gsub => ["message", "\[\d+\]", "" ] }
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 4, 2021, 10:29pm UTC](https://discuss.elastic.co/t/logstash-error/288444/14 "2021-11-04T22:29:30Z")
The regexp is
\[ -- a literal square bracket (not the start of a character group)
\d+ -- \d is a digit, + means one or more
\] -- a literal square bracket
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 5, 2021, 4:33pm UTC](https://discuss.elastic.co/t/logstash-error/288444/15 "2021-11-05T16:33:37Z")
Thanks @Badger for responding. That was very helpful
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 15, 2021, 6:09pm UTC](https://discuss.elastic.co/t/logstash-error/288444/16 "2021-11-15T18:09:26Z")
```auto
mutate { gsub => ["message", "\[\d+\]", "" ] }
```
how do i keep the data which is is inside the ?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 15, 2021, 6:26pm UTC](https://discuss.elastic.co/t/logstash-error/288444/17 "2021-11-15T18:26:30Z")
If you want to keep the number and lose the square brackets you could try a capture group
```
mutate { gsub => ["message", "\[(\d+)\]", "\1" ] }
```
or a character group
```
mutate { gsub => ["message", "[\[\]]", "" ] }
```
---
### Author:  [@John\_snow](https://discuss.elastic.co/u/John_snow)
#### Post date: [November 15, 2021, 6:45pm UTC](https://discuss.elastic.co/t/logstash-error/288444/18 "2021-11-15T18:45:43Z")
Can i keep both square brackets and number/character inside brackets. ?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 15, 2021, 7:30pm UTC](https://discuss.elastic.co/t/logstash-error/288444/19 "2021-11-15T19:30:37Z")
You will need to substitute the [and] before parsing the JSON, then substitute them back in afterwards. That said, I very much doubt that a json filter is the only place where field names that look like array references cause problems.
```
mutate {
gsub => [
"message", "\[", "LeftSquareBracket",
"message", "\]", "RightSquareBracket"
]
}
json { ... }
ruby {
code => '
# Untested and has no error checking or recovery...
event.to_hash.each { |k, v|
if k.match(/LeftSquareBracket|RightSquareBracket/)
newK = k.gsub(/LeftSquareBracket/, "[").gsub(/RightSquareBracket/, "]")
event.set(newK, v)
end
}
'
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 13, 2021, 7:30pm UTC](https://discuss.elastic.co/t/logstash-error/288444/20 "2021-12-13T19:30:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.