Imported Data in to Elastic Search with Epoch Nano Not Showing anything in Kibana

Elastic Stack Kibana
1

I have imported data in to Elastic Search via the bulk API. I have kept my date fields as EPOCH NANO within the data but have selected EPOCH MILLI as the date format. My understanding is JAVA TIME will just drop the last digits and convert it to EPOCH MILLI.

I imported by the bulk API and can see successes in the data

{"index":{"_index":"els","_type":"logs","_id":"3541c65dffdc4408","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true,"status":201}},{"index":{"_index":"els","_type":"logs","_id":"3541c65dffe04408","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true,"status":201}},{"index":{"_index":"els","_type":"logs","_id":"3541c66688954408","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true,"status":201}},{"index":{"_index":"els","_type":"logs","_id":"3541c67089434408","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true,"status":201}

I see data within the correct index (docs.count shows data)

I can curl Elasticsearch for the data and it returns successfully

curl -XGET "http://localhost:9200/els/logs/3541c67089434408"

{
  "_index": "els",
  "_type": "logs",
  "_id": "3541c67089434408",
  "_version": 1,
  "found": true,
  "_source": {
    "cacheRequest": {
      "headers": null,
      "keepaliveStatus": "noKeepalives"....................

However when I load up Kibana there is no data / no results found. Do I have to update my Kibana Mapping? Or does Kibana not support the Java Time?

2

Did you select the right field as the time one when setting up your index patterns?

3

so my index mapping is (well top part):

{
  "settings": {
    "index.mapping.ignore_malformed": true
  },
  "mappings": {
    "logs": {
      "properties": {
        "timestamp": {
          "type":   "date",
          "format": "epoch_millis"
        },
        "geoip.location": {
          "type": "geo_point"
      }
        }
      }
    }
  }'

I have also told Kibana to look @ timestamp This worked fine when I 'sed' EPOCH Nano to Milli in my script then bulk imported but was testing new Java Time function to remove this stage

4

elasticsearch does not support nanoseconds, you'll need to send the timestamps as milliseconds if you're going to use the epoch_millis date format

5

See https://github.com/elastic/elasticsearch/issues/10005

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.