I seem to getting my input and output swapped:
[ops-elk-1]$ sudo ./logstash-plugin install --version='3.0.2' logstash-input-kafka
Validating logstash-input-kafka-3.0.2
Installing logstash-input-kafka
Installation successful
[ops-elk-1 10.0.196.98]$ ls
00_clamav.conf 00_heartbeat.conf 50_env.conf
00_consul.conf 00_kafka_broker_input.conf 50_host_role.conf
00_datadog.conf 00_rabbitmq_broker_input.conf 99_elasticsearch_output.conf
ops-elk-1 10.0.196.98]$ grep kafka *
00_kafka_broker_input.conf: kafka {
00_kafka_broker_input.conf: zk_connect => "localhost:2181/kafka"
[ops-elk-1]$ cat 00_kafka_broker_input.conf
# This file is managed by ansible, don't make changes here - they will be overwritten.
input {
kafka {
zk_connect => "localhost:2181/kafka"
topic_id => "logstash"
group_id => "ops-elk-logstash"
reset_beginning => "true"
auto_offset_reset => "smallest"
consumer_threads => 3
codec => json
}
}
[ops-elk-1]$ sudo service logstash configtest
Unknown setting 'zk_connect' for kafka {:level=>:error}
Unknown setting 'topic_id' for kafka {:level=>:error}
Unknown setting 'reset_beginning' for kafka {:level=>:error}
The given configuration is invalid. Reason: Something is wrong with your configuration. {:level=>:fatal}
If I change 'input' to 'output' then the config passes.
I uninstalled all kafka plugins, then just installed input. Same problem. Could they have been pushed to the repository backward? Am I just seeing things backwards?