How to retrieve fields content when quering runtime fields from Logstash

Elastic Stack Logstash
1

Hello
I am implementing Runtime fields.
I added my runtime fields in my Index mapping.
When searching the data from the Dev Console, i see by "fields" attribute, containing all the runtime fields I am expecting.

But, when I use the same query from Logstash to retrieve data from ES, without the _source and only using "fields", i do not see how to use the Runtime Fields values

My question is : How do I access the runtime fields in my Logstash filter section ?

Thanks

2

Hi, I did a test and was able to fetch the runtime field. Here is my setup for the test:

  1. index + mapping
❯ curl -X PUT "https://localhost:9200/test_runtime_index" -u user:pass -k -H 'Content-Type: application/json' -d'
{                                                                                                                                                                                             "mappings": {
    "properties": {
      "first_name": {
        "type": "keyword"
      },
      "last_name": {
        "type": "keyword"
      }
    },
    "runtime": {
      "full_name": {
        "type": "keyword",
        "script": {
          "source": "emit(doc[\"first_name\"].value + \" \" + doc[\"last_name\"].value)"
        }
      }
    }
  }
}'
  1. send a couple of documents:
curl -X POST "https://localhost:9200/test_runtime_index/_doc/1" -u user:pass -k -H 'Content-Type: application/json' -d'
{                                                                                                                                                                                             "first_name": "John",
  "last_name": "Doe"
}'
curl -X POST "https://localhost:9200/test_runtime_index/_doc/2" -u user:pass -k -H 'Content-Type: application/json' -d'
{                                                                                                                                                                                             "first_name": "Jane",
  "last_name": "Smith"
}'
  1. try to fetch full name in the filter section in logstash:
input {
  heartbeat { add_field => { "last_name" => "Doe" }}
}

filter {
  elasticsearch {
    hosts => ["https://localhost:9200"]
    user => "user"
    password => "pass"
    index => "test_runtime_index"
    query_template => "/private/tmp/logstash-8.17.0/template.json"
    docinfo_fields => {
      "[fields][full_name]" => "retrieved_full_name"
    }
    ssl => true
    ssl_verification_mode => none
  }                                                                                                                                                                                         }

output {  stdout { codec => rubydebug } }

And the query template:

{
  "query": {
    "term": {
      "last_name": "%{[last_name]}"
    }
  },
  "_source": false,
  "fields": ["full_name"]
}
  1. See the result in Logstash:
/private/tmp/logstash-8.17.0
❯ bin/logstash -f cfg --log.level=error
[...]
{
    "retrieved_full_name" => [
        [0] "John Doe"
    ],
              "last_name" => "Doe",
                "message" => "ok",
               "@version" => "1",
             "@timestamp" => 2025-01-23T09:56:49.113347Z,
                   "host" => {
        "name" => "arkham.local"
    }
}