Invalid FieldReference: `[]`

[Roberto_B]

Hi all,

i receive from logstash error this kind of error:
:exception=>#<RuntimeError: Invalid FieldReference: []

this pipeline transform a csv , in the line i've [ ] value a couple of field.
for exaple the line can be:
field1,field2,field3,field4
value1,value2,[ ] ,"value4"

How can I handle this kind of value in the configuration?
Tried some controls but all fails i'm going crazy.

KR

Roberto

[Badger]

The csv filter will always call event.set if you have told it to store that column, and event.set will always try to dereference the [ ]. You could try using mutate+gsub to remove it before calling the csv filter.

[Roberto_B]

Hi Badger,

let's assume that I trim [ ] then i have a nil value ?
Which is the best metod to control the nil values?

I wrote this :

filter {
mutate {
gsub => [
"field1", "[\[\]]", ""
]
}

}

filter{
	ruby {
  code => "
	if event.get('field1').nil?
		event.remove('[field1]')
	end
  "
	}
}

[Roberto_B]

The final solution , as mentioned by Badger remove all [ ] from the original message before the CSV parsing (substituting with a space) if you only trim it will not work:

filter {
       mutate {
                    gsub => ["message", "[\[\]]", " "]
                    }
 }