# Logstash Indexing Error - Aggregate plugin: For task\_id pattern '%{id}', there are more than one filter **URL:** \ **Category:** Logstash\ **Created:** [April 9, 2018, 7:35pm UTC](https://discuss.elastic.co/t/logstash-indexing-error-aggregate-plugin-for-task-id-pattern-id-there-are-more-than-one-filter/127376 "2018-04-09T19:35:59Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![halogeek](https://avatars.discourse-cdn.com/v4/letter/h/c37758/32.png) [@halogeek](https://discuss.elastic.co/u/halogeek)\ **Post date:** [April 9, 2018, 7:35pm UTC](https://discuss.elastic.co/t/logstash-indexing-error-aggregate-plugin-for-task-id-pattern-id-there-are-more-than-one-filter/127376/1 "2018-04-09T19:35:59Z")
Am using Elasticsearch 5.5.0 and logstash 5.5.0 on Linux - AWS ec2-instance. Note: Have already posted the same exact question on Stack Overflow [here](https://stackoverflow.com/questions/49701823/logstash-indexing-error-aggregate-plugin-for-task-id-pattern-id-there-a). Have a logstash\_etl.conf file which resides in /etc/logstash/conf.d: ``` input { jdbc { jdbc_connection_string => "jdbc:mysql://localhost:3306/mydatabase" jdbc_user => "root" jdbc_password => "" jdbc_driver_library => "/etc/logstash/mysql-connector/mysql-connector-java-5.1.21.jar" jdbc_driver_class => "com.mysql.jdbc.driver" schedule => "*/5 * * * *" statement => "select * from customers" use_column_value => false clean_run => true } } filter { if ([api_key]) { aggregate { task_id => "%{id}" push_map_as_event_on_timeout => false #timeout_task_id_field => "[@metadata][index_id]" #timeout => 60 #inactivity_timeout => 30 code => "sample code" timeout_code => "sample code" } } } # sudo /usr/share/logstash/bin/logstash-plugin install logstash-output-exec output { if ([purge_task] == "yes") { exec { command => "curl -XPOST '127.0.0.1:9200/_all/_delete_by_query?conflicts=proceed' -H 'Content-Type: application/json' -d' { \"query\": { \"range\" : { \"@timestamp\" : { \"lte\" : \"now-3h\" } } } } '" } } else { stdout { codec => json_lines} elasticsearch { "hosts" => "127.0.0.1:9200" "index" => "myindex_%{api_key}" "document_type" => "%{[@metadata][index_type]}" "document_id" => "%{[@metadata][index_id]}" "doc_as_upsert" => true "action" => "update" "retry_on_conflict" => 7 } } } ``` When I restart logstash like this: ``` sudo initctl restart logstash ``` Inside /var/log/logstash/logstash-plain.log - everything works an actual indexing into Elasticsearch is occuring! However if I add another SQL input into this config file: ``` input { jdbc { jdbc_connection_string => "jdbc:mysql://localhost:3306/mydatabase" jdbc_user => "root" jdbc_password => "" jdbc_driver_library => "/etc/logstash/mysql-connector/mysql-connector-java-5.1.21.jar" jdbc_driver_class => "com.mysql.jdbc.driver" schedule => "*/5 * * * *" statement => "select * from orders" use_column_value => false clean_run => true } } ``` The indexing stops because of an error inside the config file! Inside /var/log/logstash/logstash-plain.log: ``` [2018-04-06T21:33:54,123][ERROR][logstash.agent] Pipeline aborted due to error {:exception=>#, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-aggregate-2.6.1/lib/logstash/filters/aggregate.rb:486:in `register'", "org/jruby/ext/thread/Mutex.java:149:in `synchronize'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-aggregate-2.6.1/lib/logstash/filters/aggregate.rb:480:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:281:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:292:in `register_plugins'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:292:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:302:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:226:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:398:in `start_pipeline'"]} [2018-04-06T21:33:54,146][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600} [2018-04-06T21:33:57,131][WARN][logstash.agent] stopping pipeline {:id=>"main"} ``` Am really new to logstash and Elasticsearch... What does this mean? Would appreciate if someone could tell me why by just by adding one new input causes this tool to crash?! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 9, 2018, 8:23pm UTC](https://discuss.elastic.co/t/logstash-indexing-error-aggregate-plugin-for-task-id-pattern-id-there-are-more-than-one-filter/127376/2 "2018-04-09T20:23:31Z")
How are you telling logstash where to look for the pipeline configuration? Are you telling it to use all files in /etc/logstash/conf.d? If so, do you have more than one file in there? They will get concatenated, so if you have (for example) logstash\_etl.conf and logstash\_etl.conf.backup you actually have two aggregate filters. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 7, 2018, 8:23pm UTC](https://discuss.elastic.co/t/logstash-indexing-error-aggregate-plugin-for-task-id-pattern-id-there-are-more-than-one-filter/127376/3 "2018-05-07T20:23:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.