# EPOC time issue **URL:** \ **Category:** Logstash\ **Created:** [November 7, 2017, 3:53pm UTC](https://discuss.elastic.co/t/epoc-time-issue/106715 "2017-11-07T15:53:55Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![guruprasadh4](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Post date:** [November 7, 2017, 3:53pm UTC](https://discuss.elastic.co/t/epoc-time-issue/106715/1 "2017-11-07T15:53:55Z")
**Sample Data -- (Data is in CSV file) and provided samples.** 25200,"1171007000040000","nzapap176:KUX","1171007000040000",40960,24618,16342,+000000000000000000000000000060.2,+000000000000000000000000000039.8,16384,52,16332,+000000000000000000000000000000.3,+000000000000000000000000000099.7,24576,24502,74,+000000000000000000000000000099.7,+000000000000000000000000000000.3,10986,0,4,29,0,0,15,118,29,6,4,1,112,27,29,27,11,44,138,28,11,8,-1,-1,-1,-000000000000000000000000000000.1,-000000000000000000000000000000.1,1379173,4893346,0,0,8,34,100,0,0,0,+000000000000000000000000000077.7,+000000000000000000000000000021.9,+000000000000000000000000000022.2,1,1,1,1,5387,19114,5461,+000000000000000000000000000063.4,1,+000000000000000000000000000014.4,1,+000000000000000000000000000022.8,1,+000000000000000000000000000005.0,1,+000000000000000000000000000090.0,1,-1,0,-1,0,-1,0,16384,1,52,1,16332,1,+000000000000000000000000000000.3,1,+000000000000000000000000000099.7,1,+000000000000000000000000000001.5,1,4096,1 **Logstash output :** "Total\_Real\_Mem\_MB" =\> "24576", "ADSS\_WHSC" =\> "1", "Net\_Memory\_Avail" =\> "-1", "Page\_In\_15Min" =\> "0", "SP\_WHSC" =\> "0", "path" =\> "/home/elkuser/KP/LogData/netcool-nzapap176/nzapap176\_mem1.csv", "Comp\_Mem\_Pct" =\> "+000000000000000000000000000089.5", "Page\_Out\_Reqs" =\> "0", "TDSS\_WHSC" =\> "1", "RMAVA\_WHSC" =\> "1", "Paging\_Space\_Write\_per\_Sec" =\> "0", "File\_Repl\_Mem\_Pct" =\> "+000000000000000000000000000010.5", "Page\_Ins" =\> "0", "Page\_Out\_60Min" =\> "17", "SYMEP\_WHSC" =\> "1", "Decay\_Rate" =\> "0", "Used\_Real\_Mem\_Pct" =\> "+000000000000000000000000000099.7", "Scanned\_Pages" =\> "-1", "MEMUS\_WHSC" =\> "1", "Page\_Scan" =\> "0", "Page\_Scan\_1Min" =\> "0", "UDSSP\_WHSC" =\> "1", "Repaging\_Rate" =\> "0", "Page\_Scan\_15Min" =\> "0", "Avail\_Disk\_Swap\_Space\_MB" =\> "16326", "System\_Name" =\> "nzapap176:KUX", "System\_Paging" =\> "-1", "Timestamp" =\> 39078-04-11T00:48:23.000Z, "Avail\_Virtual\_Storage\_MB" =\> "16359", "Used\_Disk\_Swap\_Space\_MB" =\> "58", "Avail\_Swap\_Space\_Pct" =\> "+000000000000000000000000000099.7", "SYSP\_WHSC" =\> "0", "Page\_Faults" =\> "2891", "Filesys\_Avail\_Mem\_Pct" =\> "+000000000000000000000000000010.4", "WRITETIME" =\> 39078-04-11T00:48:23.000Z, "Page\_Scan\_60Min" =\> "0", **Config File:** input { file { path =\> "/home/elkuser/KP/LogData/netcool-nzapap176/nzapap176\_mem1.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" # codec =\> multiline { # pattern =\> "^\A%{NUMBER},%{QUOTEDSTRING},%{QUOTEDSTRING}," # negate =\> "true" # what =\> "previous" # max\_lines =\> 1000 # } ``` } ``` } filter{ ``` csv{ ``` columns =\> ["TMZDIFF", "WRITETIME", "System\_Name", "Timestamp", "Total\_Virtual\_Storage\_MB", "Used\_Virtual\_Storage\_MB", "Avail\_Virtual\_Storage\_MB", "Virtual\_Storage\_Pct\_Used", "Virtual\_Storage\_Pct\_Avail", "Total\_Swap\_Space\_MB", "Used\_Swap\_Space\_MB", "Avail\_Swap\_Space\_MB", "Used\_Swap\_Space\_Pct", "Avail\_Swap\_Space\_Pct", "Total\_Real\_Mem\_MB", "Used\_Real\_Mem\_MB", "Avail\_Real\_Mem\_MB", "Used\_Real\_Mem\_Pct", "Avail\_Real\_Mem\_Pct", "Page\_Faults", "Page\_Reclaims", "Page\_Ins", "Page\_Outs", "Page\_In\_Reqs", "Page\_Out\_Reqs", "Page\_In\_KB\_S", "Page\_Out\_KB\_S", "Page\_In\_1Min", "Page\_In\_5Min", "Page\_In\_15Min", "Page\_In\_60Min", "Page\_Out\_1Min", "Page\_Out\_5Min", "Page\_Out\_15Min", "Page\_Out\_60Min", "Page\_Scan", "Page\_Scan\_KB", "Page\_Scan\_1Min", "Page\_Scan\_5Min", "Page\_Scan\_15Min", "Page\_Scan\_60Min", "ARC\_Size\_MB", "Net\_Memory\_Used", "Net\_Memory\_Avail", "Net\_Memory\_Used\_Pct", "Net\_Memory\_Avail\_Pct", "Non\_Comp\_Memory", "Comp\_Memory", "Decay\_Rate", "Repaging\_Rate", "Pages\_Read\_per\_Sec", "Pages\_Written\_per\_Sec", "Paging\_Space\_Free\_Pct", "Paging\_Space\_Used\_Pct", "Paging\_Space\_Read\_per\_Sec", "Paging\_Space\_Write\_per\_Sec", "Comp\_Mem\_Pct", "Non\_Comp\_Mem\_Pct", "Filesys\_Avail\_Mem\_Pct", "MEMAV\_WHSC", "RMAVA\_WHSC", "RMUSD\_WHSC", "MEMUS\_WHSC", "Non\_Comp\_Mem\_MB", "Comp\_Mem\_MB", "Filesys\_Avail\_Mem\_MB", "Process\_Mem\_Pct", "PRMEP\_WHSC", "System\_Mem\_Pct", "SYMEP\_WHSC", "File\_Repl\_Mem\_Pct", "FRMEP\_WHSC", "File\_Repl\_Min\_Mem\_Pct", "FRMIP\_WHSC", "File\_Repl\_Max\_Mem\_Pct", "FRMAP\_WHSC", "Unlocked\_Pageable\_Memory", "UPM\_WHSC", "Scanned\_Pages", "SP\_WHSC", "System\_Paging", "SYSP\_WHSC", "Total\_Disk\_Swap\_Space\_MB", "TDSS\_WHSC", "Used\_Disk\_Swap\_Space\_MB", "UDSS\_WHSC", "Avail\_Disk\_Swap\_Space\_MB", "ADSS\_WHSC", "Used\_Disk\_Swap\_Space\_Pct", "UDSSP\_WHSC", "Avail\_Disk\_Swap\_Space\_Pct", "ADSSP\_WHSC", "Real\_Mem\_on\_Disk\_Swap\_Ratio", "MR\_WHSC", "Page\_Size", "PSIZE\_WHSC"] convert =\> { "TMZDIFF" =\> "integer" "Total\_Virtual\_Storage\_MB" =\> "integer" "Used\_Virtual\_Storage\_MB" =\> "integer" } } mutate { convert =\> {"WRITETIME" =\> "integer"} } mutate { convert =\> {"Timestamp" =\> "integer"} } date{ match =\> ["WRITETIME", "UNIX\_MS"] target =\> "WRITETIME" } date{ match =\> ["Timestamp", "UNIX\_MS"] target =\> "Timestamp" } } output { elasticsearch { hosts =\> "localhost" index =\> "mem-index2" template\_overwrite =\> true } stdout { codec =\> rubydebug } Issues : 1. I don't see time stamp extracting. Below two Timestamp WRITETIME 2. Have also tried with out mutate, without converting to integer Thanks for the support in advance! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 7, 2017, 4:12pm UTC](https://discuss.elastic.co/t/epoc-time-issue/106715/2 "2017-11-07T16:12:27Z")
UNIX\_MS is milliseconds since epoch. Your timestamps are microseconds since epoch, so instead of being about 40 years past the epoch (1970) your timestamp is 40,000 years past the epoch: "Timestamp" =\> 39078-04-11T00:48:23000Z. Try ruby { code =\> "event.set('WRITETIME', event.get('WRITETIME') / 1000)" } and you will get "WRITETIME" =\> 2007-02-09T07:43:20.040Z ---
**Author:** ![guruprasadh4](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Post date:** [November 7, 2017, 5:17pm UTC](https://discuss.elastic.co/t/epoc-time-issue/106715/3 "2017-11-07T17:17:15Z")
Cool, it worked. Now other issue is not able to get this viewed in KIBANA as these are from 2007. So is there a way that we can see in KIBANA ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 8, 2017, 6:29am UTC](https://discuss.elastic.co/t/epoc-time-issue/106715/4 "2017-11-08T06:29:57Z")
> Now other issue is not able to get this viewed in KIBANA as these are from 2007. > So is there a way that we can see in KIBANA ? Kibana doesn't care if you have old logs. Are the indexes being created in the first place? ---
**Author:** ![guruprasadh4](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Post date:** [November 8, 2017, 3:06pm UTC](https://discuss.elastic.co/t/epoc-time-issue/106715/5 "2017-11-08T15:06:59Z")
hi, yup it did got created. Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 6, 2017, 3:07pm UTC](https://discuss.elastic.co/t/epoc-time-issue/106715/6 "2017-12-06T15:07:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.