# Getting timestamp inside timestamp
**URL:** https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511
**Category:** Logstash
**Created:** [October 12, 2021, 1:56pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511 "2021-10-12T13:56:02Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@Prasang-Biyani](https://discuss.elastic.co/u/Prasang-Biyani)
#### Post date: [October 12, 2021, 1:56pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/1 "2021-10-12T13:56:02Z")
Hi Everyone,
I am new to the Logstash, so please forgive me for asking silly question.
I am trying to convert timestamp from the log file into @timestamp.
Here is my logstash conf file.
```auto
input {
udp {
port => 5959
}
}
filter {
csv {
separator => "|"
quote_char => "\x00"
columns => ["timestamp", "probablecause", "alarmseverity", "servicename", "servername"]
}
mutate {
#remove_field => ["@version", "host", "path"]
convert => { "timestamp" => "string" }
}
date {
match => ["timestamp", "yyyy-MM-dd HH:mm:ss", "ISO8601"]
#yyyy-MM-dd HH:mm:ss
#target => "@timestamp"
#timezone => "IST"
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "indexfortesting"
codec => rubydebug
}
}
```
This is my actual log file content.
> 2021-10-12 19:22:11|connection-establishment-error|cleared|ER|SAPC\_PL-4|
> 2021-10-12 19:23:12|connection-establishment-error|major|ER|SAPC\_PL-4|
Here's my data looks like on the Dashboard.
Please suggest me how to use timestamp from the log file.
Thanks in advance.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 12, 2021, 2:18pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/2 "2021-10-12T14:18:46Z")
I'm not sure how your output `timestamp` field could get the value you showed, can you change your output to `stdout` and share what is the output of your pipeline?
Also, you are using `quote_char` in your csv input, but there is nothing like that the sample you shared, do you really have the NULL character around your csv values?
Considering that your parse is working, and you have the following value in the `timestamp` field.
```auto
timestamp => 2021-10-12 19:22:11
```
Your `date` filter is ok to parse this value into the `@timestamp` field, if you are not in the UTC timezone you will need to specify that, but you cannot use the `IST` named timezone as this is ambiguous, you will need to pass the time difference in numbers.
For example:
```auto
date {
match => ["timestamp", "yyyy-MM-dd HH:mm:ss"]
timezone => "+0530"
}
```
---
### Author:  [@Prasang-Biyani](https://discuss.elastic.co/u/Prasang-Biyani)
#### Post date: [October 12, 2021, 2:33pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/3 "2021-10-12T14:33:44Z")
@leandrojmp
The reason being for using `quote_char` is that I am sending data via Python using Log Handler, so it has no double quotes around it. Current quote\_char worked for all the fields except timestamp.
I took your suggestion and added timezone.
```auto
date {
match => ["timestamp", "yyyy-MM-dd HH:mm:ss"]
target => "@timestamp"
timezone => "+0530"
}
```
Here's the output on the terminal
I am getting `_dateparsefailure`, not sure why
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 12, 2021, 2:42pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/4 "2021-10-12T14:42:23Z")
Please, do not share images, it can be very hard to see and some people can not even be able to see it.
Can you change your output to `stdout { codec => rubydebug }` and share the output of one event using the preformatted text?
For what I was able to see in the image your share, your message is not being parsed, your `timestamp` field is not right, it has an entire `json` document in it.
---
### Author:  [@Prasang-Biyani](https://discuss.elastic.co/u/Prasang-Biyani)
#### Post date: [October 12, 2021, 2:48pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/5 "2021-10-12T14:48:16Z")
@leandrojmp
Here's the output:
```auto
{
"@version" => "1",
"alarmseverity" => "WA",
"type" => "5G Logs",
"probablecause" => "minor",
"@timestamp" => 2021-10-12T14:47:14.642Z,
"servicename" => "SAPC_PL-5",
"timestamp" => "{\"@timestamp\": \"2021-10-12T14:47:14.642Z\", \"@version\": \"1\", \"message\": \"system-resources-overload",
"tags" => [
[0] "_dateparsefailure"
],
"message" => "{\"@timestamp\": \"2021-10-12T14:47:14.642Z\", \"@version\": \"1\", \"message\": \"system-resources-overload|minor|WA|SAPC_PL-5|2021-10-13 11:24:05\", \"host\": \"DDC5-L-DFD82B3\", \"path\": \"C:\\\\Users\\\\PRASAN~1.BIY\\\\AppData\\\\Local\\\\Temp/ipykernel_25640/3145687460.py\", \"tags\": [], \"type\": \"logstash\", \"level\": \"INFO\", \"logger_name\": \"5G Logs\", \"stack_info\": null}",
"servername" => "2021-10-13 11:24:05\", \"host\": \"DDC5-L-DFD82B3\", \"path\": \"C:\\\\Users\\\\PRASAN~1.BIY\\\\AppData\\\\Local\\\\Temp/ipykernel_25640/3145687460.py\", \"tags\": [], \"type\": \"logstash\", \"level\": \"INFO\", \"logger_name\": \"5G Logs\", \"stack_info\": null}",
"host" => "127.0.0.1"
}
```
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 12, 2021, 2:52pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/6 "2021-10-12T14:52:42Z")
> [@Prasang-Biyani](#):
>
> `"timestamp" => "{\"@timestamp\": \"2021-10-12T14:47:14.642Z\", \"@version\": \"1\", \"message\": \"system-resources-overload"`
Your `csv` filter is not working, your original `message` field is a json, you will need to parse this field, so you will get a new `message` field with the format that your `csv` filter is expecting.
Try to add the following filter above your csv filter.
```auto
json {
source => "message"
}
```
---
### Author:  [@Mustafa\_NAJIB](https://discuss.elastic.co/u/Mustafa_NAJIB)
#### Post date: [October 12, 2021, 2:55pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/7 "2021-10-12T14:55:32Z")
i'm using Europe/Paris as Timezone (+2)
INPUT DATA :
2021-10-12 19:22:11|connection-establishment-error|cleared|ER|SAPC\_PL-4|
2021-10-12 19:23:12|connection-establishment-error|major|ER|SAPC\_PL-4|
Filter :
```auto
'filter {
grok {
match => {
"message" => [
"^%{TIMESTAMP_ISO8601:logdate}%{GREEDYDATA:[@metadata][specificlogcontent]}",
"%{GREEDYDATA:FAILPARSE}"
]
}
}
################### DATE ####################
# Restore @timestamp with its original value
if [origtimestamp] {
mutate {
convert => ["origtimestamp", "string"]
}
date {
target => "@timestamp"
match => ["origtimestamp", "ISO8601", "yyyy-MM-dd HH:mm:ss", "YYYY-MM-dd HH:mm:ss"]
#remove_field => ["origtimestamp"]
}
}
date {
match => ["logdate", "yyyy-MM-dd HH:mm:ss"]
timezone => "Europe/Paris"
target => "TIMESTAMP"
}
}
```
OUTPUT :
{
"@metadata": {
"specificlogcontent": "|connection-establishment-error|cleared|ER|SAPC\_PL-4|\r"
},
"@timestamp": "2021-10-12T14:46:47.173Z",
"@version": "1",
**"TIMESTAMP": "2021-10-12T17:22:11.000Z",**
"host": "localhost",
**"logdate": "2021-10-12 19:22:11",**
"message": "2021-10-12 19:22:11|connection-establishment-error|cleared|ER|SAPC\_PL-4|\r"
}
{
"@metadata": {
"specificlogcontent": "|connection-establishment-error|major|ER|SAPC\_PL-4|"
},
"@timestamp": "2021-10-12T14:46:47.175Z",
"@version": "1",
**"TIMESTAMP": "2021-10-12T17:23:12.000Z",**
"host": "localhost",
**"logdate": "2021-10-12 19:23:12",**
"message": "2021-10-12 19:23:12|connection-establishment-error|major|ER|SAPC\_PL-4|"
}
---
### Author:  [@Prasang-Biyani](https://discuss.elastic.co/u/Prasang-Biyani)
#### Post date: [October 12, 2021, 3:13pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/8 "2021-10-12T15:13:47Z")
@leandrojmp
I tried your approach. I am getting this data:
```auto
{
"@timestamp" => 2021-10-14T12:51:28.000Z,
"type" => "logstash",
"level" => "INFO",
"alarmseverity" => "cleared",
"servicename" => "ER",
"timestamp" => "2021-10-14 18:21:28",
"servername" => "SAPC_PL-3",
"message" => "system-resources-overload|cleared|ER|SAPC_PL-3|2021-10-14 18:21:28",
"host" => "DDC5-L-DFD82B3",
"path" => "C:\\Users\\PRASAN~1.BIY\\AppData\\Local\\Temp/ipykernel_25640/885417759.py",
"@version" => "1",
"probablecause" => "system-resources-overload",
"logger_name" => "5G Logs",
"tags" => [],
"stack_info" => nil
}
```
Here, I am getting two timestamps i.e., timestamp and @timestamp
Is this expected?
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 12, 2021, 4:00pm UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/9 "2021-10-12T16:00:55Z")
Yes, `@timestamp` is a field created by logstash when the event enters the filter block of your pipeline, the `timestamp` field is the name you gave when parsing your message with the `csv` filter.
Also, the `@timestamp` field will always be in UTC/GMT.
It seems that your date filter is working now as the difference between the two fields is `05:30`, which I think is the difference from your timezone to UTC, assuming that `IST` means India Standard Time.
---
### Author:  [@Prasang-Biyani](https://discuss.elastic.co/u/Prasang-Biyani)
#### Post date: [October 13, 2021, 2:30am UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/10 "2021-10-13T02:30:13Z")
Yes, `IST` is Indian Standard Time.
Thanks for the quick solution @leandrojmp, really appreciated 😃
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 13, 2021, 2:47am UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/11 "2021-10-13T02:47:03Z")
Just a tip, abbreviated timezones, like `IST`, are not always reliable as they can mean different times.
For example, `IST` can be the abbreviation of:
- India Standard Time (UTC + 05:30)
- Irish Standard Time (UTC + 01:00)
- Israel Standard Time (UTC + 02:00)
So, try to always use the exact time difference, like `+0530` or the name used by IANA in the [tz databases](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), in this case `Asia/Kolkata`.
I would say that it is better to use `Asia/Kolkata` instead of `+0530`, but this will give you the same result.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 13, 2021, 4:01am UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/12 "2021-10-13T04:01:01Z")
The filter is built on top of Joda, which maintains its own [set](https://www.joda.org/joda-time/timezones.html) of timezones. Basically IANA/Olson plus a few others. When a set at the bottom of the stack changes then each of the layers above need a release to incorporate it This can take a while.
The reality is that a lot of places have a ?ST or ?DT, and obviously the ?ST do not get adjusted twice a year to match ?DT, whereas ?DT may or may not be adjust to match the equivalent ?ST. It is a local convention. Do not use them.
---
### Author:  [@Prasang-Biyani](https://discuss.elastic.co/u/Prasang-Biyani)
#### Post date: [October 13, 2021, 5:28am UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/13 "2021-10-13T05:28:05Z")
Thanks for the tip @leandrojmp. I will take that into consideration 😊
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 10, 2021, 5:28am UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511/14 "2021-11-10T05:28:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.