I am using Macbook and I am trying to take input from Amazon S3. I have a bucket in my S3 and it has a textfile.
I am using the following config:
input
{
s3 {
access_key_id => "Access_Key"
bucket => "timselktest1"
region => "us-west-2"
secret_access_key => "Secret_Access_Key"
prefix => "/"
type => "s3"
sincedb_path => "/dev/null"
}
}
filter
{
csv
{
separator => ","
columns => ["ID KEY","Client ID","Account ID","Record Source","Created On Date","Modified On Date","Form Type","Tax Status", "Error Field","Error Field Value","Error Message"]
autogenerate_column_names => false
convert => {"ID KEY" => "integer"}
}
date
{
timezone => "UTC"
match => ["Created On Date", "dd-MMM-YY HH.mm.ss.SSSSSS a"]
}
date
{
timezone => "UTC"
match => ["Created On Date", "dd-MMM-YY HH.mm.ss.SSSSSS a"]
target => "Created On Date"
}
date
{
timezone => "UTC"
match => ["Modified On Date", "dd-MMM-YY HH.mm.ss.SSSSSS a"]
target => "Modified On Date"
}
}
output
{
elasticsearch
{
action => "index"
hosts => "localhost"
index => "w8ss3"
workers => 1
}
stdout
{
codec => rubydebug
}
}
when i run logstash, i get the following:
Pipeline main has started:
after that, nothing gets printed out on the screen. If I check in elastic search also, no index is made.
Kindly help me.