# Mapping date in milliseconds to basic\_date\_time

**URL:** https://discuss.elastic.co/t/mapping-date-in-milliseconds-to-basic-date-time/343160
**Category:** Kibana
**Created:** [September 15, 2023, 7:54pm UTC](https://discuss.elastic.co/t/mapping-date-in-milliseconds-to-basic-date-time/343160 "2023-09-15T19:54:14Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)
#### Post date: [September 15, 2023, 7:54pm UTC](https://discuss.elastic.co/t/mapping-date-in-milliseconds-to-basic-date-time/343160/1 "2023-09-15T19:54:15Z")

</div>

I followed the instructions here:

> **[format | Elasticsearch Guide \[7.17\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/mapping-date-format.html)**

I ran this command as instructed:

```auto
PUT /_index_template/itential_jobs_template
{
	"index_patterns": ["itential-jobs-*"],
	"template": {
		"mappings": {
			"properties": {
				"start_time": {
					"format": "basic_date_time",
					"type": "date"
				}
			}
		}
	}
}

```

The start\_time field is still being indexed into Elasticsearch as a number instead of as a timestamp.

```auto
"start_time": [
      1694806840000
    ]

```

Part of the problem is the source JSON had start\_time as a float

```auto
"metrics": {
		"start_time": 1.694726079118E+12,
		"user": {
			"$oid": "64654876fb0d9c4f8d6cccc5"
		},
		"progress": 0.23076923076923078
	}

```

I do have my Logstash pipeline config copying the start\_time out of the metrics JSON object before attempting to convert it to date.

---

<div class="post-metadata">

### Author: ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)
#### Post date: [September 15, 2023, 9:11pm UTC](https://discuss.elastic.co/t/mapping-date-in-milliseconds-to-basic-date-time/343160/2 "2023-09-15T21:11:27Z")

</div>

I modified the pipeline config because according to the docs the date filter looks for seconds instead of milliseconds if the "UNIX" literal is used.

Adding to my problems, the ruby filter does not like my code.

```auto
input {
  file {
    path => "/var/log/mongodb/errored-jobs.json"
    start_position => "beginning"
    codec => json
    #codec => multiline {
    # pattern => "^s"
    # what => "previous"
    #
    #sincedb_path => "/dev/null"
  }
}

filter {
  mutate {
    remove_field => ["tasks", "transitions", "variables"]
    rename => {"[metrics][start_time]" => "start_time"}
  }

  ruby {
    code => "event.set('start_time', event.get('start_time')/1000))"
  }

  mutate {
    rename => {"_id" => "jobId"}
  }

  date {
    match => ["start_time", "UNIX"]
    target => "start_time"
    timezone => "UTC"
  }
}

```

Error message in logstash-plain.log

```auto
[2023-09-15T21:45:38,999][ERROR][logstash.javapipeline][main] Pipeline error {:pipeline_id=>"main", :exception=>#<RuntimeError: unexpected error: (ruby filter code):2: syntax error, unexpected ')'
...event.get('start_time')/1000))

```

---

<div class="post-metadata">

### Author: ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)
#### Post date: [September 15, 2023, 11:19pm UTC](https://discuss.elastic.co/t/mapping-date-in-milliseconds-to-basic-date-time/343160/3 "2023-09-15T23:19:06Z")

</div>

I reversed the single and double quotes to fix the ruby filter.

I also added more fields to remove as Logstash/ES started complaining about them, and I don't think we need them. Now documents get indexed with start\_time as a timestamp instead of as a float.

```auto
input {
  file {
    path => "/var/log/mongodb/errored-jobs.json"
    start_position => "beginning"
    codec => json
    #codec => multiline {
    # pattern => "^s"
    # what => "previous"
    #
    #sincedb_path => "/dev/null"
  }
}

filter {
  mutate {
    remove_field => ["created", "last_updated", "tasks", "transitions", "variables", "watchers", "ancestors", "decorators"]
    rename => {"[metrics][start_time]" => "start_time"}
  }

  ruby {
    code => 'event.set("start_time", event.get("start_time") / 1000)'
  }

  mutate {
    rename => {"_id" => "jobId"}
  }

  date {
    match => ["start_time", "UNIX"]
    target => "start_time"
    timezone => "UTC"
  }
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 13, 2023, 11:19pm UTC](https://discuss.elastic.co/t/mapping-date-in-milliseconds-to-basic-date-time/343160/4 "2023-10-13T23:19:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.