NoMethodError: undefined method `[]' for nil:NilClass

Elastic Stack Logstash
1

Hi,

I try to use logstash.filters.aggregate plugin.

I get an aggregate exception.

Aggregate exception occurred
{:error=>#<NoMethodError: undefined method `' for nil:NilClass>, :code=>"\n\t\t\tmap['user']['scopes'] << {'id' => 'asd', 'name' => 'asd'}\n\t\t\tevent.cancel()\n\t\t", :map=>{}, :event_data=>{"@timestamp"=>2019-05-12T14:28:00.803Z, "user.scopes.id"=>nil, "user.scopes.name"=>"xxx", "@version"=>"1", "id"=>1000002, "type"=>"yyy", "user"=>{"tel2"=>nil, "address"=>nil, "relatedbapuser"=>nil, "university"=>"xxx", "titleId"=>xxx, "departmentId"=>xxx, "title"=>"xxx", "isdeleted"=>"false", "countryId"=>xxx, "relatedid"=>nil}}}

logstash.conf

aggregate {
	task_id => '%{id}'
	code => "
		map['user']['scopes'] << {'id' => 'asd', 'name' => 'asd'}
		event.cancel()
	"
	push_previous_map_as_event => true
	timeout => 3
}

Do you have any suggestion about this exception?

2

Try adding

    map['user'] ||= {}
    map['user']['scopes'] ||= []
3

Thanks but not work :frowning:

In the code block changes not effected to output. But remove fields in remove_field part works.

input {
	jdbc {
        jdbc_driver_library => "...\sqljdbc42.jar"
        jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
        jdbc_connection_string => "..."
        jdbc_user => "..."
        jdbc_password => "..."
        jdbc_validate_connection => true
        clean_run => true
        schedule => "..."
        type => "a_type"
        statement => "..."
    }
}
filter {
	if [type] == "a_type"{
		aggregate {
			task_id => '%{user.id}'
			code => "
				map['user'] ||= {}
				map['user']['id'] = event.get('user.id')
				map['user']['scopes'] ||= []
				map['user']['scopes'] << {'id' => event.get('user.scopes.id'), 'name' => event.get('user.scopes.name')}
			"
			remove_field => [ "user.id", "user.scopes.id", "user.scopes.name" ]
		}
    }
}
output {
	if [type] == "a_type"{
        elasticsearch {
			hosts => ["localhost:9200"]
			index => "xxx"
			document_type => "yyy"
			document_id => "%{id}"
        }
    }
}
4

In your sample data you do not have a [user.id] field, just [id]

5

Yes, you are right. I change some piece of config. But is it matter id or user.id field exist?

Some fields return from sql are below:
id, user.id, user.name, user.surname, user.email, user.scopes.id, user.scopes.name

6

You are right, that might prevent it working (if the task_id field does not exist) but it would not result in that error. I would expect that error if you commented out

map['user'] ||= {}

I suggest running "--log.level debug --config.debug --config.test_and_exit" and see if the configuration is the one you expect (that you do not, for example, have both my.conf and my.conf.bck getting picked up).