Elasticsearch/client.go:408 Cannot index event

2022-02-14T15:46:03.114Z WARN [elasticsearch] elasticsearch/client.go:408 Cannot index event
 publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbaeb900, ext:63780449718, 
loc:(*time.Location)(0x5864aa0)}, Meta:null, Fields:{"agent":{"ephemeral_id":"039466b4-e76f-
4ec2-ac5d-d64aac166def","hostname":"metricbeat-elk-poc-metricbeat-metrics-5b7d5cbbc7-
rgztt","id":"61ca742f-27bf-4073-a2c7-40d70f7a7ce5","name":"metricbeat-elk-poc-metricbeat-
metrics-5b7d5cbbc7-rgztt","type":"metricbeat","version":"7.10.2"},"ecs":{"version":"1.6.0"},"event":
{"dataset":"prometheus.remote_write","module":"prometheus"},"host":{"name":"metricbeat-elk-
poc-metricbeat-metrics-5b7d5cbbc7-rgztt"},"metricset":{"name":"remote_write"},"prometheus":
{"labels":{"app":"gatekeeper","chart":"gatekeeper","control_plane":"audit-
controller","gatekeeper_sh_operation":"audit","gatekeeper_sh_system":"yes","heritage":"Helm","instance":"10.6.72.104:8888","job":"kubernetes-pods","namespace":"kube-
operations","pod":"gatekeeper-audit-64d4c866b4-
q44zf","pod_template_hash":"64d4c866b4","release":"opa-gatekeeper"},"metrics":
{"go_memstats_frees_total":133965610278.000000,"process_virtual_memory_max_bytes":18446744073709551616.000000}},"service":{"type":"prometheus"}}, Private:interface {}(nil), TimeSeries:true}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [prometheus.metrics.process_virtual_memory_max_bytes] of type [long] in document with id 'sZrp-H4BNkkcHW7KjvtZ'. Preview of field's value: '1.8446744073709552E19'","caused_by":{"type":"input_coercion_exception","reason":"Numeric value (1.8446744073709552e+19) out of range of long (-9223372036854775808 - 9223372036854775807)\n at [Source: (byte[])\"{\"create\":{\"_index\":\"prom-elk-poc-2022.02.14\"}}\n{\"@timestamp\":\"2022-02-14T15:35:18.196Z\",\"ecs\":{\"version\":\"1.6.0\"},\"prometheus\":{\"metrics\":{\"controller_runtime_reconcile_time_seconds_bucket\":38848},\"labels\":{\"app\":\"gatekeeper\",\"pod\":\"gatekeeper-audit-64d4c866b4-q44zf\",\"gatekeeper_sh_operation\":\"audit\",\"heritage\":\"Helm\",\"release\":\"opa-gatekeeper\",\"chart\":\"gatekeeper\",\"pod_template_hash\":\"64d4c866b4\",\"controller\":\"constraint-controller\",\"job\":\"kubernetes-pods\",\"control_plane\":\"audit-controller\",\"i\"[truncated 80320 bytes]; line: 1, column: 168]"}}

Getting this error when being pushed from Prometheus.

the index settings:

 "prom-elk-poc-2022.02.14" : {
    "settings" : {
      "index" : {
        "mapping" : {
          "nested_fields" : {
            "limit" : "10000"
          },
          "total_fields" : {
            "limit" : "10000"
          },
          "depth" : {
            "limit" : "500"
          }
        },
        "number_of_shards" : "5",
        "blocks" : {
          "read_only_allow_delete" : "false",
          "write" : "false"
        },
        "provided_name" : "prom-elk-poc-2022.02.14",
        "creation_date" : "1644837715451",
        "number_of_replicas" : "1",
        "uuid" : "599xUzfmTRKgyidapOBFiw",
        "version" : {
          "created" : "135227827"
        }
      }
    }
  }

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.