# Time to timestamp issue

**URL:** <https://discuss.elastic.co/t/time-to-timestamp-issue/113455>\
**Category:** Logstash\
**Created:** [December 28, 2017, 2:33pm UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455 "2017-12-28T14:33:56Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![uvarisubash](https://avatars.discourse-cdn.com/v4/letter/u/c37758/32.png) [@uvarisubash](https://discuss.elastic.co/u/uvarisubash)\
**Post date:** [December 28, 2017, 2:33pm UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/1 "2017-12-28T14:33:56Z")

</div>

Hi All,

I use filter to extract the @time into a field of its own called "@time"  
message ==\> "@time": "2017-12-04T17:44:34"  
"@timestamp" =\> 2017-12-28T12:29:22.096Z,

## .conf file content

input {  
file {  
path =\> "/home/sdc/PycharmProjects/Kibana\_Pro/utility/MAYOPETMR01\_2017-12-04.gz.log"  
type =\> "log"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
}  
}

filter {  
date {  
match =\> ["@time","EEE MMM dd HH:mm:ss YYYY"]  
target =\> "@timestamp"  
}  
}

## output { elasticsearch { hosts =\> ["localhost:9200"] sniffing =\> true manage\_template =\> false index =\> "gesyslog\_test" document\_type =\> "log" } stdout { codec =\> rubydebug } }

## Output:

{  
"type" =\> "log",  
"message" =\> "{"@time": "2017-12-04T17:44:34", "@code": "2212884484", "text": "Exception Class: Unknown Severity: Unknown\nFunction: ", "@systemID": "MA"NSP SCP:RfHubCanHWO::RfBias 5462", "detail": {"view\_Level": "4", "seq\_Num": "0", "name": null, "format": "1", "h\_Name": "prtte1\_Seq": "4767637676"}, "@type": "log"}",  
"@version" =\> "1",  
"path" =\> "/home/sdc/PycharmProjects/Kibana\_Pro/utility/4444444\_2017-12-04.gz.log",  
"@timestamp" =\> 2017-12-28T12:29:22.096Z,  
"host" =\> "sdc-VirtualBox"  
}

It is not replacing it. Please help me

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 29, 2017, 11:14am UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/2 "2017-12-29T11:14:56Z")

</div>

The `@time` field obviously doesn't match the "EEE MMM dd HH:mm:ss YYYY" pattern you've given. Try "ISO8601" instead.

---

<div class="post-metadata">

**Author:** ![uvarisubash](https://avatars.discourse-cdn.com/v4/letter/u/c37758/32.png) [@uvarisubash](https://discuss.elastic.co/u/uvarisubash)\
**Post date:** [December 29, 2017, 11:55am UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/3 "2017-12-29T11:55:16Z")

</div>

Thanks for replying, , the same issue is occuring after changing to "ISO8601".

## **Log file test\_site.log**

{"@type": "Log", "@source": "yeryryryr", "@systemID": "666666", "detail": {"view\_Level": "4", "time\_Seq": "1512536473", "suit\_Name": null, "tag": "162", "host\_Name": "test", "format": "1", "seq\_Num": "1"}, "@time": "2017-12-06T05:01:13", "text": "Signal 15 was received, causing a system shutdown.", "@code": "501370064"}  
{"@type": "LOG", "@source": "Uyryryryry", "@systemID": "666666", "detail": {"view\_Level": "4", "time\_Seq": "1512536473", "suit\_Name": null, "tag": "130", "host\_Name": "test", "format": "1", "seq\_Num": "2"}, "@time": "2017-12-06T05:01:13", "text": "start script failed", "@code": "0"}  
{"@type": "LOG", "@source":" yyyy", "@systemID": "666666", "detail": {"view\_Level": "4", "time\_Seq": "1512536473", "suit\_Name": null, "tag": "225", "host\_Name": "test", "format": "1", "seq\_Num": "3"}, "@time": "2017-12-06T05:01:13", "text": "Exception Class: Unknown\nFunction: yryryyr", "@code": "200002379"}  
{"@type": "Log", "@source": "testts1", "@systemID": "666666", "detail": {"view\_Level": "4", "time\_Seq": "1512536473", "suit\_Name": null, "tag": "221", "host\_Name": "test", "format": "1", "seq\_Num": "4"}, "@time": "2017-12-06T05:01:13", "text": "Exception Class: Unknown\nFunction", "@code": "200002379"}  
sdc@sdc-VirtualBox:~/PycharmProjects/Kibana\_Pro/utility$

**.conf file content :**

sdc@sdc-VirtualBox:~/PycharmProjects/Kibana\_Pro/utility$ cat /home/sdc/PycharmProjects/Kibana\_Pro/utility/logstash.conf  
input {  
file {  
path =\> "/home/sdc/PycharmProjects/Kibana\_Pro/utility/test\_site.log"  
type =\> "log"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
}  
}

filter {  
date {  
match =\> ["@time","ISO8601"]  
#match =\> ["@time","yyyy-MM-dd HH:mm:ss"]  
target =\> "@timestamp"  
}  
}

output {  
elasticsearch {  
hosts =\> ["localhost:9200"]

# sniffing =\> true

# manage\_template =\> false

```
index => "gesyslog_test"
document_type => "log"

```

}  
stdout { codec =\> rubydebug }  
}  
sdc@sdc-VirtualBox:~/PycharmProjects/Kibana\_Pro/utility$

**Output message**

root@sdc-VirtualBox:~# /usr/share/logstash/bin/logstash --path.settings=/etc/logstash -f /home/sdc/PycharmProjects/Kibana\_Pro/utility/logstash.conf --path.data /usr/share/logstash/data  
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties  
{  
"type" =\> "log",  
"@version" =\> "1",  
"path" =\> "/home/sdc/PycharmProjects/Kibana\_Pro/utility/test\_site.log",  
"host" =\> "sdc-VirtualBox",  
"@timestamp" =\> 2017-12-29T11:42:23.968Z,  
"message" =\> .....................

Could you please suggest me what needs to be corrected?

thanks,  
Subash

---

<div class="post-metadata">

**Author:** ![uvarisubash](https://avatars.discourse-cdn.com/v4/letter/u/c37758/32.png) [@uvarisubash](https://discuss.elastic.co/u/uvarisubash)\
**Post date:** [December 30, 2017, 3:54am UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/5 "2017-12-30T03:54:37Z")

</div>

Someone can look at this issue?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 31, 2017, 1:00pm UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/6 "2017-12-31T13:00:36Z")

</div>

It doesn't look like you're parsing the JSON input in any way (hence, there is no `@time` field to parse). What does the full output from the stdout plugin look like?

---

<div class="post-metadata">

**Author:** ![uvarisubash](https://avatars.discourse-cdn.com/v4/letter/u/c37758/32.png) [@uvarisubash](https://discuss.elastic.co/u/uvarisubash)\
**Post date:** [January 3, 2018, 4:17am UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/7 "2018-01-03T04:17:26Z")

</div>

> [@magnusbaeck](#):
>
> there is no @time field to pars

**Full output file for a entry.**

{  
"type" =\> "log",  
"message" =\> "{"@time": "2017-12-04T17:44:34", "@code": "2212884484", "text": "Exception Class: Unknown Severity: Unknown\nFunction: ", "@systemID": "MA"NSP SCP:RfHubCanHWO::RfBias 5462", "detail": {"view\_Level": "4", "seq\_Num": "0", "name": null, "format": "1", "h\_Name": "prtte1\_Seq": "4767637676"}, "@type": "log"}",  
"@version" =\> "1",  
"path" =\> "/home/sdc/PycharmProjects/Kibana\_Pro/utility/4444444\_2017-12-04.gz.log",  
"@timestamp" =\> 2017-12-28T12:29:22.096Z,  
"host" =\> "sdc-VirtualBox"  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 3, 2018, 9:34am UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/8 "2018-01-03T09:34:46Z")

</div>

Right, no `@time` field. Your event only has `type`, `message`, `@version`, `path`, `@timestamp`, and `host` fields. Use a json or json\_lines codec in your file input, or process the `message` field with a json filter.

---

<div class="post-metadata">

**Author:** ![uvarisubash](https://avatars.discourse-cdn.com/v4/letter/u/c37758/32.png) [@uvarisubash](https://discuss.elastic.co/u/uvarisubash)\
**Post date:** [January 12, 2018, 12:54pm UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/9 "2018-01-12T12:54:09Z")

</div>

thanks Magnus.  
i have updated the same. Now i am getting below error in /var/log/logstash/logstash-plain.log. And logstash is struck. Please advise me.

2018-01-12T18:14:41,753][INFO][logstash.outputs.elasticsearch] retrying failed action with response code: 503 ({"type"=\>"unavailable\_shards\_exception", "reason"=\>"[gesyslog\_test][3] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[gesyslog\_test][3]] containing [2] requests]"})  
[2018-01-12T18:14:41,753][INFO][logstash.outputs.elasticsearch] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=\>4}

---

<div class="post-metadata">

**Author:** ![Krunal\_kalaria](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/krunal_kalaria/32/23862_2.png) [@Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)\
**Post date:** [January 12, 2018, 1:23pm UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/10 "2018-01-12T13:23:04Z")

</div>

hi @uvarisubash try this i dont know in your case its working or not i hope its work!

date  
{  
match =\> ["@time", "UNIX\_MS"]  
target =\> "Time"  
}

---

<div class="post-metadata">

**Author:** ![uvarisubash](https://avatars.discourse-cdn.com/v4/letter/u/c37758/32.png) [@uvarisubash](https://discuss.elastic.co/u/uvarisubash)\
**Post date:** [January 12, 2018, 1:46pm UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/11 "2018-01-12T13:46:59Z")

</div>

Thanks Krunal..

Now i am getting below error in /var/log/logstash/logstash-plain.log.

===============================================================================  
[2018-01-12T19:13:43,359][INFO][logstash.outputs.elasticsearch] retrying failed action with response code: 503 ({"type"=\>"unavailable\_shards\_exception", "reason"=\>"[gesyslog\_test][4] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[gesyslog\_test][4]] containing [29] requests]"})  
[2018-01-12T19:13:43,359][INFO][logstash.outputs.elasticsearch] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=\>29}  
[2018-01-12T19:13:43,401][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=\>500, :url=\>"[http://localhost:9200/\_xpack/monitoring/\_bulk?system\_id=logstash&system\_api\_version=2&interval=1s](http://localhost:9200/_xpack/monitoring/_bulk?system_id=logstash&system_api_version=2&interval=1s)"}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 18, 2018, 9:54pm UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/12 "2018-01-18T21:54:58Z")

</div>

Your ES cluster is in bad health. Look in the ES logs to find out more.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 15, 2018, 9:55pm UTC](https://discuss.elastic.co/t/time-to-timestamp-issue/113455/13 "2018-02-15T21:55:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.