Using conditional to find nil values

Elastic Stack Logstash
1

I'm writing a config file for logstash to read in a CSV file.

Some of the fields are empty some of the time but when they are not empty they have to have data types assigned (integer, bool, etc...), and when I run log stash I get an exception:

"exception =>#<NoMethodError: undefined method 'strip' for nil:NilClass Did you mean? String>
(I assume this is related to the empty fields I'm trying to read in)

I've tried using if statements to check if I need to set a data type :

if [field] != null {CSV{set data type}}
if [field] != nil {CSV{set data type}}
if [field] != "" {CSV{set data type}}

Is there a way to build a config file for Logstash that looks to see if the field has a value and if it does then change the data type of the field? Is the exception I'm seeing related to all this?

I'm very, very new to Logstash and file ingestion work in general and would appreciate any help I could get.

Here's what my config file looks like (roughly):

input {
file {
path => [ "/Elastic/logstash-7.3.1/ICN_BIT.csv" ]
start_position => "beginning"
sincedb_path => "/Elastic/logstash-7.3.1/icn_bit_sincedb.txt"
}
}
filter {
csv {
autodetect_column_names => true
skip_header => true
skip_empty_rows => true
columns => [all my column names]
convert => {
"timestamp" => "date_time"
"hwserial" => "integer"
"swpart" => "float"
"ospart" => "float"
}
}
if [field] != "" {
csv {
convert => {
"field => "data type"
}
}
}
date {
match => [ "timestamp", "yyyyMMdd'T'HHmmss" ]
}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "index_test"
}
}

2

The only time I can see a csv filter producing that error is if you do a boolean conversion on a nil valued field. If you move that conversion to a mutate filter then you do not get an error (but it will be left nil-valued).

3

Thanks Badger,
Using mutate along with a couple of other tweaks seemed to do the trick.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.