# Parse UNIX timestamp as human readable date while moving the data into elasticsearch

**URL:** https://discuss.elastic.co/t/parse-unix-timestamp-as-human-readable-date-while-moving-the-data-into-elasticsearch/162317
**Category:** Logstash
**Created:** [December 28, 2018, 8:33am UTC](https://discuss.elastic.co/t/parse-unix-timestamp-as-human-readable-date-while-moving-the-data-into-elasticsearch/162317 "2018-12-28T08:33:57Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![amitnegi6190](https://avatars.discourse-cdn.com/v4/letter/a/b4bc9f/32.png) [@amitnegi6190](https://discuss.elastic.co/u/amitnegi6190)
#### Post date: [December 28, 2018, 8:33am UTC](https://discuss.elastic.co/t/parse-unix-timestamp-as-human-readable-date-while-moving-the-data-into-elasticsearch/162317/1 "2018-12-28T08:33:58Z")

</div>

Hi, I have a JSON file which has a field named **lastUpdated**. It has the UNIX timestamp value for the time when the specified record was last updated. I wish to parse this JSON file and feed it to elasticsearch where this UNIX timestamp becomes a human-readable date. And also, I wish to save this date in the same variable i.e. lastUpdated.

Sample JSON file: test.json

```
    {"name":"Jonathan","score":"9.9","address":"New Delhi","lastUpdated":"1545078074640", "firstUpdated":"1545078074640"}
    {"name":"Sam","score":"8.9","address":"New York","lastUpdated":"1545078074640", "firstUpdated":"1545078074640"}
    {"name":"Michelle","score":"9.0","address":"California","lastUpdated":"1545078074640", "firstUpdated":"1545078074640"}

```

My logstash configuration file: test.config

```
input{
file{
		path => "/Users/amsing/Study/data/test.json"
		codec => json
		sincedb_path => "/dev/null"
		start_position => "beginning"
	}
 }

filter{
json{
	source => "message"
}

date{
		match => ["lastUpdated", "UNIX_MS"]
}

   mutate{
	convert => { 
		"name" => "string"
		"score" => "float"
		"address" => "string"
		"lastupdated" => "date"
	}
   }
}

output{
elasticsearch{
	hosts => "localhost:9200"
	index => "test"
}
stdout { codec => rubydebug }
}

```

I load the data into elasticsearch with the command:

```
bin/logstash -f ../../data/test.config

```

And I get the following error:

> [2018-12-28T13:58:24,130][ERROR][logstash.pipeline] Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#\<LogStash::FilterDelegator:0x1c4c8d8 @metric\_events\_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @metric\_events\_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @metric\_events\_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration\_in\_millis value:0, @id="d392c2b542d17cc9e7e38b1343c08cc18d0de592a4047e69aa47bba14c089811", @klass=LogStash::Filters::Mutate, @metric\_events=#\<LogStash::Instrument::NamespacedMetric:0x4aeecaae\>, @filter=\<LogStash::Filters::Mutate convert=\>{"name"=\>"string", "score"=\>"float", "address"=\>"string", "lastupdated"=\>"date"}, id=\>"d392c2b542d17cc9e7e38b1343c08cc18d0de592a4047e69aa47bba14c089811", enable\_metric=\>true, periodic\_flush=\>false\>\>", :error=\>"translation missing: en.logstash.agent.configuration.invalid\_plugin\_register", :thread=\>"#\<Thread:0x3d44e76c run\>"}

> [2018-12-28T13:58:24,144][ERROR][logstash.pipeline] Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\<LogStash::ConfigurationError: translation missing: en.logstash.agent.configuration.invalid\_plugin\_register\>, :backtrace=\>["/Users/amsing/Study/logstash/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/logstash-filter-mutate-3.3.4/lib/logstash/filters/mutate.rb:219:in `block in register'", "org/jruby/RubyHash.java:1343:in `each'", "/Users/amsing/Study/logstash/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/logstash-filter-mutate-3.3.4/lib/logstash/filters/mutate.rb:217:in `register'", "/Users/amsing/Study/logstash/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:242:in `register\_plugin'", "/Users/amsing/Study/logstash/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:253:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/Users/amsing/Study/logstash/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:253:in `register_plugins'", "/Users/amsing/Study/logstash/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:595:in `maybe\_setup\_out\_plugins'", "/Users/amsing/Study/logstash/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:263:in `start_workers'", "/Users/amsing/Study/logstash/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:200:in `run'", "/Users/amsing/Study/logstash/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:160:in `block in start'"], :thread=\>"#\<Thread:0x3d44e76c run\>"}

Also, I wish to parse the firstUpdated field the same way as lastUpdated and have both the date fields in human-readable form when viewed in kibana. So what all changes are to be made in the test.config file to achieve this?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [December 29, 2018, 11:02am UTC](https://discuss.elastic.co/t/parse-unix-timestamp-as-human-readable-date-while-moving-the-data-into-elasticsearch/162317/2 "2018-12-29T11:02:53Z")

</div>

> [@amitnegi6190](#):
>
> "lastupdated" =\> "date"

You can only convert to types that exist in JSON (this only affects how the JSON documents being sent to Elasticsearch are formatted, not how Elasticsearch interprets and indexes these).

Your current date filter populates the @timestamp field. If you also want to store the other fields as dates I would probably do something like this (not tested):

```auto
date{
    match => ["lastUpdated", "UNIX_MS"]
    target => "lastUpdated"
}

date{
    match => ["firstUpdated", "UNIX_MS"]
    target => "firstUpdated"
}

```

---

<div class="post-metadata">

### Author: ![amitnegi6190](https://avatars.discourse-cdn.com/v4/letter/a/b4bc9f/32.png) [@amitnegi6190](https://discuss.elastic.co/u/amitnegi6190)
#### Post date: [January 7, 2019, 8:30am UTC](https://discuss.elastic.co/t/parse-unix-timestamp-as-human-readable-date-while-moving-the-data-into-elasticsearch/162317/3 "2019-01-07T08:30:19Z")

</div>

Thanks [Christian\_Dahlqvist](https://discuss.elastic.co/t/parse-unix-timestamp-as-human-readable-date-while-moving-the-data-into-elasticsearch/162317/2). It works 😀

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 4, 2019, 8:30am UTC](https://discuss.elastic.co/t/parse-unix-timestamp-as-human-readable-date-while-moving-the-data-into-elasticsearch/162317/4 "2019-02-04T08:30:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.