Unable to read the txt file through Logstash

[sukumar.koganti (Suresh Koganti)]

Thanks Badger,
but i put the same as you suggested but no luck.

filter {
csv {
separator => "|"
columns => ["Time","Application","Message"]
autodetect_column_names => true
autogenerate_column_names => true
}
date { match => ["Time", "YYYY-MM-DD HH:mm:ss"] }
#target => "Time"
mutate {
rename => ["Time", "Date"]
}
}

output:

[2020-10-07T16:10:14,868][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2020-10-07T16:10:14,897][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/Users/skkoganti/Documents/Elastic/elastic_download/logstash-7.9.1/logstash-7.9.1/config/edi_development.conf"], :thread=>"#<Thread:0x18a36f92 run>"}

[llcarr_work (Linda)]

when you use rename you need to use the => characters -
rename => { "Time" => "Date" }

[Badger]

@sukumar.koganti

"YYYY-MM-DD HH:mm:ss"

Case matters. That should be "YYYY-MM-dd HH:mm:ss". dd is day of the month. DD is day of the year.

[sukumar.koganti (Suresh Koganti)]

Yeah thanks Badger, it's working now.

how can i read only weekly wise files from the path based on date. which means i have files in the below path.

\pneus.touchette\documents\EDI\Logs\Production

how can i read .txt files based on latest date.

[sukumar.koganti (Suresh Koganti)]

How to read only October files and tomorrow how my logstash will pic latest .txt file from the path.

\pneus.touchette\documents\EDI\Logs\Production

EDI Log-2020-10-08
EDI Log-2020-10-07
EDI Log-2020-10-06
EDI Log-2020-10-05
EDI Log-2020-10-04
EDI Log-2020-10-03
EDI Log-2020-10-02
EDI Log-2020-10-01

input

{

file

{

path => ["//pneus.touchette/documents/EDI/Logs/Production/EDI Log-2020-10*"]

start_position => "beginning"

sincedb_path => "NUL"

#ignore_olderedit => 8000

}

}