seems to be related to the date conversion, but couldn't figure out why it complains about token 0
My Config
# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.
input {
file {
path => "~/logstash-6.5.3/input/test.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["TRANSACTION_DATE", "CHECKOUT_DATE", "CHECKOUT_TIMESTAMP", "PAYMENT_TIMESTAMP", "PROGRAM_ID", "PROGRAM_NAME", "COMPANY NAME", "ID",
"CAMPAIGN_NAME", "CAMPAIGN_ID", "TOOL_NAME", "LEAF_CATEGORY_ID", "VERTICAL_ID", "SITE_ID", "BUYER_ID",
"BUYER_FREQUENCY", "SELLER_ID", "ITEM_ID", "PURCHASE_LATENCY_IN_MINUTES", "TRAFFIC_SOURCE", "QUANTITY", "GMB", "REVENUE", "PUBLISHER_EARNING", "BUYER_COUNTRY_ID","SELLER_COUNTRY_ID", "LEAF_CATEGORY_NAME", "VERTICAL_NAME","BUYER_COUNTRY","SELLER_COUNTRY"]
}
mutate {convert => ["QUANTITY", "integer"]}
mutate {convert => ["PURCHASE_LATENCY_IN_MINUTES", "float"]}
mutate {convert => ["EARNING", "float"]}
mutate {convert => ["GMB", "float"]}
date {
match => [ "TRANSACTION_DATE", "yyyy-MM-dd"]
match => [ "CHECKOUT_DATE", "yyyy-MM-dd"]
match => [ "CHECKOUT_TIMESTAMP", "yyyy-MM-dd HH:mm:ss"]
match => [ "PAYMENT_TIMESTAMP", "yyyy-MM-dd HH:mm:ss"]
timezone => "MST"
locale => "en"
}
}
output {
stdout {}
}
My input
2018-11-19,2018-11-19,2018-11-19 01:11:46,2018-11-19 01:11:46,1,US,MyPoints,123,sample+,444,sample,514,17,2,444,sample,444,111,31.222,desktop,1,3.80,0.78,0.123,1,2,"Jewellery & Watches:Vintage & Antique Jewellery:Costume:Victorian, Edwardian 1837-1910:Other Victorian Costume Jewels","Jewelry, Gems, Watches",Canada,United States
my error
Date filter with format=yyyy-MM-dd, locale=en, timezone=MST built as org.logstash.filters.parser.JodaParser
[2018-12-12T12:03:19,897][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalArgumentException", :message=>"Illegal pattern component: O", :backtrace=>["org.joda.time.format.DateTimeFormat.parsePatternTo(org/joda/time/format/DateTimeFormat.java:566)", "org.joda.time.format.DateTimeFormat.createFormatterForPattern(org/joda/time/format/DateTimeFormat.java:687)"