# Create timestamp from nested JSON elements

**URL:** <https://discuss.elastic.co/t/create-timestamp-from-nested-json-elements/287992>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [October 29, 2021, 10:26am UTC](https://discuss.elastic.co/t/create-timestamp-from-nested-json-elements/287992 "2021-10-29T10:26:04Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![alt-glitch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alt-glitch/32/96509_2.png) [@alt-glitch](https://discuss.elastic.co/u/alt-glitch)\
**Post date:** [October 29, 2021, 10:26am UTC](https://discuss.elastic.co/t/create-timestamp-from-nested-json-elements/287992/1 "2021-10-29T10:26:04Z")

</div>

Hi all,  
I am indexing nested JSON files using Logstash and want to extract the timestamp from them. Here are a couple of examples.  
**Example 1**

```auto
{
  "ip": "127.0.0.1",
  "data": {
    "postgres": {
      "status": "success",
      "protocol": "postgres",
      "result": {
        "supported_versions": "FATAL: unsupported frontend protocol 0.0: server supports 1.0 to 3.0",
        "protocol_error": {
          "code": "0A000",
          "file": "postmaster.c",
          "line": "2071",
          "message": "unsupported frontend protocol 255.255: server supports 1.0 to 3.0",
          "routine": "ProcessStartupPacket",
          "severity": "FATAL",
          "severity_v": "FATAL"
        },
        "is_ssl": true
      },
      "timestamp": "2021-10-14T06:42:08Z"
    }
  }
}

```

**Example 2**

```auto
{
  "ip": "127.0.0.1",
  "data": {
    "ssh": {
      "status": "success",
      "protocol": "ssh",
      "result": {
        "server_id": {
          "raw": "SSH-2.0-OpenSSH_X.X",
          "version": "2.0",
          "software": "OpenSSH_X.X"
        },
        "algorithm_selection": {
          "dh_kex_algorithm": "curve25519-sha256@libssh.org",
          "host_key_algorithm": "ecdsa-sha2-nistp256",
          "client_to_server_alg_group": {
            "cipher": "aes128-ctr",
            "mac": "hmac-sha2-256",
            "compression": "none"
          },
          "server_to_client_alg_group": {
            "cipher": "aes128-ctr",
            "mac": "hmac-sha2-256",
            "compression": "none"
          }
        },
        "key_exchange": {
          "curve25519_sha256_params": {
            "server_public": "fnBsVPDLuMTsaQBUTii6/cBuG3+AUeDIFj4QlcdYqEM="
          }
        }
      },
      "timestamp": "2021-10-14T06:42:07Z"
    }
  }
}

```

As you can see, in both examples, the `timestamp` is inside nested JSON of the protocol.  
Now, I was successful in accessing the `timestamp` in the first example by the following configuration;

```auto
filter {
	json {
		source => "message"
	}
	date {
		match => ["[data][postgres][timestamp]", "ISO8601" ]
	}

	mutate {
		remove_field => ["message"]
	}
}

```

However, I am looking for a more viable solution. Is it possible to access the `timestamp` field by using indices instead of their names? Then, the timestamp would be `[data][0][timestamp]` (or something like this) and I wouldn't have to create an `if` statement for each protocol.

---

<div class="post-metadata">

**Author:** ![antoineco](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/antoineco/32/93214_2.png) [@antoineco](https://discuss.elastic.co/u/antoineco)\
**Post date:** [November 3, 2021, 11:44am UTC](https://discuss.elastic.co/t/create-timestamp-from-nested-json-elements/287992/2 "2021-11-03T11:44:02Z")

</div>

For posterity, OP was discussing this issue with me in a [Gitter chat](https://gitter.im/deviantony/docker-elk?at=618104de38377967f48aec1d). Here is a summary:

Due to the unpredictable key names in the incoming data, using a pre-made filter such as `date` might prove difficult without using one condition per expected "protocol".

The solution that came up was to use a `ruby` filter to traverse the data safely:

```ruby
data = event.get('[data]')
if data.is_a?(Hash)
    data.each do |key, value|
        if value.is_a?(Hash)
            ts = value.fetch('timestamp', nil)
            event.set('@timestamp', LogStash::Timestamp.parse_iso8601(ts)) if ts
        end
    end
end

```

This snippet can be either inlined inside the `code` attribute of the filter, **or** saved inside a file and referenced via the `path` attribute:

```ruby
filter {

    ruby {
        # Either inline ...
        code => "
        data = event.get('[data]')
        if data.is_a?(Hash)
            ...
        end
        "

        # ... or from file.
        path => "my_scripts/protocol_timestamp.rb"
    }

}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 1, 2021, 11:44am UTC](https://discuss.elastic.co/t/create-timestamp-from-nested-json-elements/287992/3 "2021-12-01T11:44:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.