We are trying to use logstash-input-kinesis to ingest data from AWS Kinesis into Elasticsearch. Here is our logstash pipeline:
input {
kinesis{
kinesis_stream_name=>"xxx"
application_name=>"yyy"
codec=> json
region=> "us-west-2"
metrics=> cloudwatch
type => "kinesisinput"
}
...
output {
if [type]=="kinesisinput" {
elasticsearch {
action => "index"
hosts => ["zzz"]
index => "events"
document_type => "kinesis"
}
}
}
Messages from the "xxx" stream in Kinesis get ingested successfully into Elasticsearch, except that the partitionkey associated with each message is missing. I'd like to know how to retrieve the partitionkey from the stream. Thanks.