# Issue with date field in CSV plugin

**URL:** <https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047>\
**Category:** Logstash\
**Created:** [March 23, 2021, 6:33am UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047 "2021-03-23T06:33:44Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\
**Post date:** [March 23, 2021, 6:33am UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/1 "2021-03-23T06:33:44Z")

</div>

Hi All,

I'm trying to ingest data into elasticsearch using csv plugin, data is going smooth for one or two days then all of a sudden there is a wired entry which will get updated with some random date and year.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/7/976ce830e78b5c998b6f141dad10e2fc0160c618.png)

I'm not sure from where its picking up the date as 01-01-5541 , i have searched for the data in the csv file but there is no such entry with this date.

Below is my logstash config file

```
input {
  file {
    path => "/etc/logstash/http_poller/solarwinds/15mins_perfromance/conf.d/data/finaldata.csv"
    start_position => "beginning"
   sincedb_path => "/dev/null"
  }
}
filter {
  csv {
      separator => ","
      columns => ["data.results.Availability","data.results.DateTime","data.results.AvgResponseTime","data.results.Caption","data.results.MaxResponseTime","data.results.MinResponseTimedata.results.MinResponseTime","data.results.NodeID","data.results.PercentDown","data.results.PercentLoss"]
  }
}
output {
  elasticsearch {
    hosts => ["1.3.5.2:9200"]
    user => "user"
    password => " *****"
    index => "perfchk"
  }
#stdout { codec => rubydebug }
}

```

Any suggestions please.

Thanks  
Gauti

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 23, 2021, 7:15am UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/2 "2021-03-23T07:15:01Z")

</div>

Hard to say without seeing the input and the mapping.  
You probably need to use a date filter as well but that's just a guess.

Anyway I moved your question to #elastic-stack:logstash

---

<div class="post-metadata">

**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\
**Post date:** [March 23, 2021, 1:25pm UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/3 "2021-03-23T13:25:54Z")

</div>

@dadoonet Here is the output of stdout, looks like a dateparse error, not sure how to avoid this, any advice please

```
"data.results.vendor" => "Windows",
"@timestamp" => 2021-03-23T12:29:03.529Z,
"data.results.PercentDown" => 0,
"data.results.Node_Category_Type" => "Database",
"message" => "2021-03-23T12:14:21.1162574,1641,0.0,0.0,0.0,0,0.0,100.0,2021-03-23T12:14:21.1162574,PRD01,sep-db-prd01,1.1.6.2,Windows,Boulder,United States,Database,Windows 2008 Server,Symantec Endpoint Encryption",
"data.results.Application_Name" => "Symantec Endpoint Encryption",
"data.results.OS" => "Windows 2008 Server",
"host" => "ip-1-3-5-3.global.internal",
"@version" => "1",
"path" => "/etc/logstash/http_poller/perfromance/conf.d/data/finaldata1.csv",
"data.results.MaxResponseTime" => 0,
"data.results.AvgResponseTime" => "0.0",
"data.results.NodeID" => "11",
"data.results.PercentLoss" => 0,
"data.results.country" => "United States",
"data.results.Availability" => 100,
"data.results.ObservationTimestamp" => "2021-03-23T12:14:21.1162574",
"data.results.city" => "Boulder",
"data.results.ipaddress" => "1.1.6.2",
"data.results.DateTime" => "2021-03-23T12:14:21.1162574"
}
{
"data.results.dns" => "appv6.global.com",
"data.results.MinResponseTime" => 1,
"data.results.ObservationFrequency" => nil,
"data.results.Caption" => "appv6.global.com",
"tags" => [
        [0] "_dateparsefailure"
    ],
}

```

here is my mapping for date field

```
date { match => ["data.results.DateTime", "yyyy-MM-dd HH:mm:ss.SSS"] }
date { match => ["data.results.ObservationTimestamp", "yyyy--MM-dd HH:mm:ss.SSS"] 

```

Any advice please

Thanks  
Gauti

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 23, 2021, 1:40pm UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/4 "2021-03-23T13:40:48Z")

</div>

To parse `2021-03-23T12:14:21.1162574` , I think you probably need `yyyy-MM-dd'T'HH:mm:ss.SSS`. See the documentation at [Date filter plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html)

> For non-formatting syntax, you’ll need to put single-quote characters around the value. For example, if you were parsing ISO8601 time, "2015-01-01T01:12:23" that little "T" isn’t a valid time format, and you want to say "literally, a T", your format would be this: "yyyy-MM-dd’T’HH:mm:ss"

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 23, 2021, 1:44pm UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/5 "2021-03-23T13:44:57Z")

</div>

Think just `ISO8601` may work also for this. You can check if it's in that format [here](https://www.regextester.com/97766).

```auto
      date {
        match => ["data.results.DateTime", "ISO8601"]
      }

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 23, 2021, 4:06pm UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/6 "2021-03-23T16:06:46Z")

</div>

> [@Gauti](#):
>
> I'm not sure from where its picking up the date as 01-01-5541 , i have searched for the data in the csv file but there is no such entry with this date.

The [message] field in your screenshot has 5541 in the second column, which is "data.results.DateTime". Since that field has type date elasticsearch has to decide how to interpret 5541 as a date. It chooses January 1st, 5541.

---

<div class="post-metadata">

**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\
**Post date:** [March 24, 2021, 4:24am UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/7 "2021-03-24T04:24:56Z")

</div>

Putting ISO8601 had worked out...Thanks

Usually it'll run for one day and then this weird year will pop up all of a sudden, have started ingesting the data, will check and update back here if i stuck again with the issue.

Thanks  
Gauti

---

<div class="post-metadata">

**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\
**Post date:** [March 24, 2021, 4:26am UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/8 "2021-03-24T04:26:41Z")

</div>

@Badger You are rite, i observed that and that was my main issue, and i'm not able to figure out from where it is getting that information.  
As i'm ingesting data using CSV, those type of values are not there in the CSV at all.

Thanks  
Gauti

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 21, 2021, 4:26am UTC](https://discuss.elastic.co/t/issue-with-date-field-in-csv-plugin/268047/9 "2021-04-21T04:26:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.