# Overwrite @timestamp filed by date and time filed in the CSV file

**URL:** <https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142>\
**Category:** Logstash\
**Created:** [August 1, 2022, 4:04pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142 "2022-08-01T16:04:08Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Faiz\_Shamri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faiz_shamri/32/50097_2.png) [@Faiz\_Shamri](https://discuss.elastic.co/u/Faiz_Shamri)\
**Post date:** [August 1, 2022, 4:04pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/1 "2022-08-01T16:04:08Z")

</div>

My logstash conf. file is like the below:

```auto
input {
  file {
    path => "/somepath/balance_20220731.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
      separator => ";"
      skip_header => "true"
      columns => ["date","time","ActorId", "ClosingBalance"]
  }
  date {
        add_field => { "ndate" => "%{date} %{time}" }
        match => ["ndate", "yyyy-MM-dd HH:mm:ss"]
        target => "@timestamp"
        remove_field => "ndate"
  }
  mutate {
      convert => {
          ClosingBalance => "integer"
      }
          remove_field => ["message","path","host","@version"]
  }
}
output {

stdout {}

}

```

m csv file format is like:

```auto
Date;time;ActorId;ClosingBalance
2022-07-31;00:00:00;somename;91003700

```

I am trying to overwrite @timestamp field value with date and time field values that come from the csv file. but i am getting document timestamp in the @timestamp field.

following is my parrsed output:

```auto
{
              "time" => "00:00:00",
        "@timestamp" => 2022-08-01T16:01:56.718590Z,
             "event" => {
        "original" => "2022-07-31;00:00:00;Api-ASANApp;68590000"
    },
              "date" => "2022-07-31",
           "ActorId" => "Api-ASANApp",
               "log" => {
        "file" => {
            "path" => "/somepath/balance_20220731.csv"
        }
    },
    "ClosingBalance" => 68590000
}

```

any help plz

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 1, 2022, 4:10pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/2 "2022-08-01T16:10:39Z")

</div>

> [@Faiz\_Shamri](#):
>
> ```auto
> date {
> add_field => { "ndate" => "%{date} %{time}" }
> match => ["ndate", "yyyy-MM-dd HH:mm:ss"]
> target => "@timestamp"
> remove_field => "ndate"
> }
> 
> ```

This is wrong, the `add_field` will only be executed if the `date` filter is successful, which will not happen because the `ndate` file does not exist.

You should first create the `ndate` field using a mutate filter and then use the date filter.

Try this:

```auto
  mutate {
        add_field => { "ndate" => "%{date} %{time}" }
  }
  date {
        match => ["ndate", "yyyy-MM-dd HH:mm:ss"]
        target => "@timestamp"
        remove_field => "ndate"
  }

```

---

<div class="post-metadata">

**Author:** ![Faiz\_Shamri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faiz_shamri/32/50097_2.png) [@Faiz\_Shamri](https://discuss.elastic.co/u/Faiz_Shamri)\
**Post date:** [August 1, 2022, 4:22pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/3 "2022-08-01T16:22:10Z")

</div>

> [@leandrojmp](#):
>
> ```auto
> mutate {
> add_field => { "ndate" => "%{date} %{time}" }
> }
> 
> ```

Thanks @leandrojmp, I changed my conf file as suguested:

```auto
input {
  file {
    path => "/somepath/balance_20220731.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
      separator => ";"
      skip_header => "true"
      columns => ["date","time","ActorId", "ClosingBalance"]
      convert => {
          "ClosingBalance" => "integer"
        }
  }
  mutate {
        add_field => { "ndate" => "%{date} %{time}" }
  }
  date {
        match => ["ndate", "yyyy-MM-dd HH:mm:ss"]
        target => "@timestamp"
        #remove_field => "ndate"
  }
  mutate {
      convert => {
          ClosingBalance => "integer"
      }
          remove_field => ["message","path","host","@version"]
  }
}
output {

stdout {}

```

but @timestamp field still remain same:

```auto

               "log" => {
        "file" => {
            "path" => "/somepath/balance_20220731.csv"
        }
    },
             "event" => {
        "original" => "2022-07-31;00:00:00;SuperWifi;19990000"
    },
             "ndate" => "2022-07-31 00:00:00",
           "ActorId" => "SuperWifi",
              "time" => "00:00:00",
    "ClosingBalance" => 19990000,
              "date" => "2022-07-31",
        "@timestamp" => 2022-07-30T19:30:00Z
}

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 1, 2022, 4:54pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/4 "2022-08-01T16:54:24Z")

</div>

> [@Faiz\_Shamri](#):
>
> `"@timestamp" => 2022-07-30T19:30:00Z`

It does not seem that it is stays the same since it is not using the current time, it looks like that it got the `2022-07-31 00:00:00` time and shifted it by `4:30` hours, this would happen if you are in the timezone of `UTC+04:30` and your Logstash is configured to use this timezone.

In which timezone are you in? And what is the timezone of your date time?

All dates in logstash and elasticsearch will always be in UTC, if your log time is not in UTC you need to explicitly configure the timezone in the date filter.

It would be something like this example:

```auto
  date {
        match => ["ndate", "yyyy-MM-dd HH:mm:ss"]
        target => "@timestamp"
        timezone => "+0430"
        remove_field => "ndate"
  }

```

In this case I'm telling Logstash that the value of the `ndate` field is a date and time from a timezone with `UTC+04:30`.

---

<div class="post-metadata">

**Author:** ![Faiz\_Shamri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faiz_shamri/32/50097_2.png) [@Faiz\_Shamri](https://discuss.elastic.co/u/Faiz_Shamri)\
**Post date:** [August 1, 2022, 5:05pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/5 "2022-08-01T17:05:43Z")

</div>

> [@leandrojmp](#):
>
> ```auto
> date {
> match => ["ndate", "yyyy-MM-dd HH:mm:ss"]
> target => "@timestamp"
> timezone => "+0430"
> remove_field => "ndate"
> }
> 
> ```

Hello @leandrojmp,  
Yes i am in zone `UTC+04:30` which is the zone for the `date` and `time` fields in my csv file.  
I have explicitly configure the timezone in the date filter but still the @timestamp date is not as it is there in ndate field. plz see the below output:

```auto
{
             "event" => {
        "original" => "2022-07-31;00:00:00;Geopol;5335822"
    },
             "ndate" => "2022-07-31 00:00:00",
    },
           "ActorId" => "Geopol",
              "date" => "2022-07-31",
              "time" => "00:00:00",
        "@timestamp" => 2022-07-30T19:30:00Z,
    "ClosingBalance" => 5335822
}

```

---

<div class="post-metadata">

**Author:** ![Faiz\_Shamri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faiz_shamri/32/50097_2.png) [@Faiz\_Shamri](https://discuss.elastic.co/u/Faiz_Shamri)\
**Post date:** [August 1, 2022, 5:14pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/6 "2022-08-01T17:14:09Z")

</div>

Thanks a lot @leandrojmp, i fixed it by adding ` timezone => "UTC"` in the filter date ..

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 1, 2022, 5:46pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/7 "2022-08-01T17:46:57Z")

</div>

If the `date` and `time` of your files are in UTC + 04:30, you should not use the timezone as UTC, it needs to be `timezone => "+0430"`.

The date and times in Elasticsearch are stored in UTC, but Kibana will convert it into the Timezone of the Browser, which normally is your timezone.

If the time `2022-07-31 00:00:00` is in `UTC+04:30`, then the UTC time is `2022-07-30 19:30:00`.

---

<div class="post-metadata">

**Author:** ![Faiz\_Shamri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faiz_shamri/32/50097_2.png) [@Faiz\_Shamri](https://discuss.elastic.co/u/Faiz_Shamri)\
**Post date:** [August 2, 2022, 2:42pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/8 "2022-08-02T14:42:43Z")

</div>

Hello @leandrojmp ,  
can you plz help me here:  
[https://discuss.elastic.co/t/extract-time-from-txt-file-and-date-from-file-name-and-add-them-to-timestamp-field/311196](https://discuss.elastic.co/t/extract-time-from-txt-file-and-date-from-file-name-and-add-them-to-timestamp-field/311196)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 30, 2022, 2:43pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-filed-by-date-and-time-filed-in-the-csv-file/311142/9 "2022-08-30T14:43:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.