Convert timestamp into date

[Bharath_Kumar_R]

I tried to extract anomaly results using the following request:

GET .ml-anomalies-.write-my_job_low_sum/_search?human
{
"size": 10000
}

One of the results I got was:

"_index" : ".ml-anomalies-custom-low_sum_results",
"_type" : "_doc",
"_id" : "my_job_low_sum_record_1560973500000_900_0_275145719_30",
"_score" : 8.38236,
"_source" : {
"job_id" : "my_job_low_sum",
"result_type" : "record",
"probability" : 6.795050328806717E-35,
"multi_bucket_impact" : -5.0,
"record_score" : 79.66493,
"initial_record_score" : 95.12116596413524,
"bucket_span" : 900,
"detector_index" : 0,
"is_interim" : false,
"timestamp" : 1560973500000,
...}
Is it possible to internally convert the timestamp into date time ?

[richcollier]

You can use script_fields to access and modify the values. For example, something like this:

GET .ml-anomalies-*/_search
{
    "query": {
            "bool": {
              "filter": [
                  { "range" : { "timestamp" : { "gte": "now-5y" }}},
                  { "term" :  { "job_id" : "farequote_responsetime" } },
                  { "term"  : { "result_type" : "record" }},
                  { "range"  : { "record_score" : { "gte": "75" }}}

              ]
            }
    },
    "script_fields": {
            "timestamp_iso8601": {
              "script": {
                  "lang": "painless",
                  "source": """doc["timestamp"]"""
                }
            },
            "record_score" : {
              "script": {
                  "lang": "painless",
                  "source": """Math.round(doc["record_score"].value)"""
              }
            },
            "airline" : {
              "script": {
                  "lang": "painless",
                  "source": """doc["partition_field_value"]"""
              }
            }
    }
}

The result would look something like:

{
  "took" : 12,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 0.0,
    "hits" : [
      {
        "_index" : ".ml-anomalies-shared",
        "_type" : "doc",
        "_id" : "farequote_responsetime_record_1486656000000_900_0_94347_3",
        "_score" : 0.0,
        "fields" : {
          "timestamp_iso8601" : [
            "2017-02-09T16:00:00.000Z"
          ],
          "airline" : [
            "AAL"
          ],
          "record_score" : [
            99
          ]
        }
      },
      {
        "_index" : ".ml-anomalies-shared",
        "_type" : "doc",
        "_id" : "farequote_responsetime_record_1486656900000_900_0_94347_3",
        "_score" : 0.0,
        "fields" : {
          "timestamp_iso8601" : [
            "2017-02-09T16:15:00.000Z"
          ],
          "airline" : [
            "AAL"
          ],
          "record_score" : [
            97
          ]
        }
      }
    ]
  }
}

[Bharath_Kumar_R]

will the date time be returned corresponding to my timezone?

[richcollier]

No, the fact that the timestamp has a Z at the end signifies that it is UTC. You can convert the timestamp into a local time using any Java-based method. For example:

            "timestamp_utc": {
              "script": {
                  "lang": "painless",
                  "source": """doc["timestamp"]"""
                }
            },
            "timestamp_local": {
              "script": {
                  "lang": "painless",
                  "source": """Instant.ofEpochMilli(doc["timestamp"].value.millis).atZone(ZoneOffset.ofHours(-5))"""
                }
            },

would yield:

          "timestamp_local" : [
            "2017-02-09T11:00:00.000-05:00"
          ],
          "timestamp_utc" : [
            "2017-02-09T16:00:00.000Z"
          ],