# Elasticsearch json logging @timestamp missing

**URL:** <https://discuss.elastic.co/t/elasticsearch-json-logging-timestamp-missing/310992>\
**Category:** Elasticsearch\
**Created:** [July 29, 2022, 2:01pm UTC](https://discuss.elastic.co/t/elasticsearch-json-logging-timestamp-missing/310992 "2022-07-29T14:01:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [July 29, 2022, 2:01pm UTC](https://discuss.elastic.co/t/elasticsearch-json-logging-timestamp-missing/310992/1 "2022-07-29T14:01:25Z")

</div>

Hello there,

I have a problem with elasticsearch logging and the filebeat module.

I collect the logs from elasticsearch with the filebeat module.  
For example the deprecation logs:

```auto
- module: elasticsearch
  deprecation:
    enabled: true
    var.paths:
      - /var/log/elasticsearch/*_deprecation.json

```

Logging path on the system: "/var/log/elasticsearch/elastic\_deprecation.json"

I have loaded all pipelines from filebeat:

- filebeat-7.17.5-elasticsearch-deprecation-pipeline
- filebeat-7.17.5-elasticsearch-deprecation-pipeline-json
- filebeat-7.17.5-elasticsearch-deprecation-pipeline-plaintext

Then I send it to our central logstashes and they will put it into elasticsearch:

logstash pipeline:

```auto
elasticsearch {
      hosts => ["https://elasticsearch-server:9220"] 
      index => "%{[@metadata][beat]}-int" 
      user => logstash_writer
      password => "<password>"
      pipeline => "%{[@metadata][pipeline]}"
      manage_template => false
      ssl => true
      manage_template => false
      ilm_enabled => false
      action => create
  }

```

It is a datastream and it works so far. But the elasticsearch logs do not log the timestamp correctly. The create a timestamp and put it into the field "timestamp" but the pipeline wants a "@timestamp" field to parse on.

I know I can just change the pipeline to this correctly. But in the next update process this would happen again and I dont want to do it manually everytime.

logstash logs this Warning:

```auto
{
  "level": "WARN",
  "loggerName": "logstash.outputs.elasticsearch",
  "timeMillis": 1659100307439,
  "thread": "[filebeat-int]>worker0",
  "logEvent": {
    "message": "Could not index event to Elasticsearch.",
    "status": 400,
    "action": [
      "create",
      {
        "_index": "filebeat-int",
        "pipeline": "filebeat-7.17.5-elasticsearch-deprecation-pipeline"
      },
      {
        "type": "deprecation.elasticsearch",
        "log": {
          "offset": 13609643,
          "file": {
            "path": "/var/log/elasticsearch/elastic_deprecation.json"
          }
        },
        "input": {
          "type": "log"
        },
        "service": {
          "environment": "int",
          "type": "elasticsearch"
        },
        "ecs": {
          "version": "1.12.0"
        },
        "host": {
          "name": "apm-int1"
        },
        "level": "DEPRECATION",
        "fileset": {
          "name": "deprecation"
        },
        "node.name": "apm-int1-2",
        "timestamp": "2021-11-11T07:55:22,480+01:00",
        "agent": {
          "type": "filebeat",
          "id": "21a8b519-2d08-41ac-81f5-e5fed36b42ff",
          "ephemeral_id": "297a785c-aefd-40d4-84dd-057352e9a1e7",
          "version": "7.17.5",
          "name": "apm-int1",
          "hostname": "host1"
        },
        "component": "o.e.d.c.s.Settings",
        "message": "[xpack.security.transport.ssl.keystore.password] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.",
        "cluster.name": "elastic",
        "@timestamp": "2022-07-29T13:11:45.052Z",
        "@version": "1",
        "event": {
          "module": "elasticsearch",
          "dataset": "elasticsearch.deprecation"
        }
      }
    ],
    "response": {
      "create": {
        "_index": ".ds-filebeat-int-2022.07.26-000017",
        "_type": "_doc",
        "_id": "F9sVSoIB2y2_gvQ4n4_c",
        "status": 400,
        "error": {
          "type": "mapper_parsing_exception",
          "reason": "failed to parse",
          "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "data stream timestamp field [@timestamp] is missing"
          }
        }
      }
    }
  }
}

```

The log line is:

```auto
{
  "type": "deprecation.elasticsearch",
  "timestamp": "2022-07-29T12:27:50,414+02:00",
  "level": "WARN",
  "component": "o.e.d.c.s.Settings",
  "cluster.name": "elastic-int",
  "node.name": "apm-int1-1",
  "message": "[xpack.security.transport.ssl.keystore.password] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.",
  "key": "xpack.security.transport.ssl.keystore.password",
  "category": "settings",
  "elasticsearch.elastic_product_origin": "beats",
  "cluster.uuid": "d_CHXTaaGdIpVYHbbrw",
  "node.id": "8gHrgv04Ccgc4cHynV5g"
}

```

As you can see the is only a "timestamp" field and no "@timestamp"

Can anybody give me a hint? Or does anyone know the solution?

Greetings  
Malte

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 26, 2022, 2:02pm UTC](https://discuss.elastic.co/t/elasticsearch-json-logging-timestamp-missing/310992/2 "2022-08-26T14:02:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.