Hi
I have installed logstash 7.1.1 on centos 7, I have installed the kineseis plugin and codec plugin. the aws credentials and the perm to access the stream are all good what i see in the logs is and shows its connected to my stream
Jun 12 05:38:43 logstash-server logstash: OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Jun 12 05:39:23 logstash-server logstash: Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
Jun 12 05:39:24 logstash-server logstash: [2019-06-12T05:39:24,759][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.1.1"}
Jun 12 05:39:37 logstash-server logstash: [2019-06-12T05:39:37,105][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, :thread=>"#<Thread:0x550e226f run>"}
Jun 12 05:39:37 logstash-server logstash: [2019-06-12T05:39:37,643][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
Jun 12 05:39:37 logstash-server logstash: [2019-06-12T05:39:37,818][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
Jun 12 05:39:37 logstash-server logstash: Jun 12, 2019 5:39:37 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker createClient
Jun 12 05:39:37 logstash-server logstash: WARNING: Received configuration for region as us-west-2.
Jun 12 05:39:38 logstash-server logstash: Jun 12, 2019 5:39:38 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker createClient
Jun 12 05:39:38 logstash-server logstash: WARNING: Received configuration for region as us-west-2.
Jun 12 05:39:38 logstash-server logstash: Jun 12, 2019 5:39:38 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker createClient
Jun 12 05:39:38 logstash-server logstash: WARNING: Received configuration for region as us-west-2.
Jun 12 05:39:39 logstash-server logstash: [2019-06-12T05:39:39,474][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
As a test i have posted data to my stream using json and also verified pulling it using a python program and all works, however with this configuration
input {
kinesis {
kinesis_stream_name => "estest"
application_name => "logstash"
profile => "xxxx"
initial_position_in_stream => "TRIM_HORIZON"
region => "xxxx"
}
}
output {
file {
path => "/tmp/estest"
}
}
With the above config i dont see anything happening, it doesnt seem to pull the messages.
Plz advise any bug in the plugin or 7.1.1 logstash?