Logstash not updating record and not giving any error

Elastic Stack Logstash
1

Logstash not updating documents. I am not receiving any error . Here is my code.

    input{
jdbc {
jdbc_connection_string => "-------" 
jdbc_user => "------" 
jdbc_password => "-----" 
jdbc_validate_connection => true 
jdbc_driver_library => "/mysql-connector-java-5.1.36-bin.jar" 
jdbc_driver_class => "com.mysql.jdbc.Driver" 

charset=>"UTF-8"
jdbc_default_timezone => "UTC"
schedule => "* * * * * *"
parameters => { "object_id" => 265636 }
statement => "------------" 

use_column_value => true 
tracking_column => "updated_timestamp" 
tracking_column_type => "timestamp"

last_run_metadata_path => "logstash_post_sync_jdbc_last_run" 
}}filter {
aggregate {
	task_id => "%{box_id}"
	code => "
	map['box_posts'] ||= []
	map['box_posts'] << {
		'id' => event.get('id'),
		'post_box_id' => event.get('post_box_id'),
		'post_type_id' => event.get('post_type_id'),
		'created_at' => event.get('created_at'),
		'user_id' => event.get('user_id'),				
		'score' => event.get('score'),
		'text_content' => event.get('text_content'),
		'short_code' => event.get('short_code')
	}
	event.cancel()"
	push_previous_map_as_event => true
		
}  }output {
stdout { codec => rubydebug }
elasticsearch {
    "document_id" => "%{box_id}"
    "hosts" => "http://127.0.0.1:9200"
    "action" =>"update"
    "index" =>  "test"
    "document_type" => "_doc"
    "routing" => 1
    "doc_as_upsert" => true
}}'

Please help!

2

That would happen if all the events had the same value for box_id. Verify that this changes by reviewing the output from stdout.

1 Like
3

I am only updating one single nested filed of a document.

4

I have fixed it. The document_id should be present in query, filters, also output. if we remove the field in filter/aggregate or override it then it would not work.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.