Logstash Time Column Issue

Elastic Stack Elasticsearch
1

I am trying to get logstash to ingest the time column correctly into elasticsearch but it is not working. I am stuck and need help.

in the csv I am trying to ingest the time looks like this:

2024-09-01 12:10:40.309

My Logstash Conf file looks like this in relation to the time column

    mutate {
		gsub => ["time", "IST", "Asia/Kolkata"]
		}
	date {
		match => [ "time", "yyyy-mm-dd'T'hh:mm:ss'Z'"]
        target => "time"
		}

When I view the data in Elasticsearch the time column is showing up as text and not date. Can someone please help figure this out. I don't understand what is happening.

One time I was able to get it to ingest like this, however out of 52,000 documents it only ingested 900. I have 8 CSVs I need to ingest all with the same time format.

Sep 3, 2024 @ 06:05:55.000

Can I adjust the time column from text to date after it has been indexed?

2

Hello,

Please share the output you have in Elasticsearch, it is not clear how your document looks like and what is the Logstash output.

Also, you are using this gsub.

gsub => ["time", "IST", "Asia/Kolkata"]

This would replace the value IST for Asia/Kolkata in the field time, but the value you shared does not have IST in it.

2024-09-01 12:10:40.309

What are you trying to achieve with this filter?

Your date filter also does not match the time string you shared.

Please share a sample of your csv, not just one field, and your full logstash pipeline.

3

Thanks for this. I realized by analyzing the CSV data set that the issues are with the data and not the ingest.