# Convert message into eps data and create a visualization

**URL:** <https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903>\
**Category:** Elasticsearch\
**Tags:** ingest-pipeline\
**Created:** [July 20, 2023, 6:46pm UTC](https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903 "2023-07-20T18:46:42Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ryan\_Downey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ryan_downey/32/35987_2.png) [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\
**Post date:** [July 20, 2023, 6:46pm UTC](https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903/1 "2023-07-20T18:46:42Z")

</div>

Note - I do have a ticket in with support but I felt that maybe this would be useful to the community as well so I will update things as we move along.

_ **What were doing:** _  
Shipping from 9 DLC's running in Openshift with Filebeat to our Elastic deployment. The Filebeat is sending data to an ingest pipeline called eps-data that has the following setup. The ingest pipeline config is designed to utilize the grok and convert processors to pull the eps values out and then change the string to a float to be utilized for visualization.

_ **Sample Log** _  
"message": [  
"[0] dlc\_eps: [1689790390.673000000, {}, {"eps5sec"=\>5164.600000, "eps10sec"=\>"5088.60", "eps15sec"=\>"5153.60", "eps30sec"=\>"5497.33", "eps60sec"=\>"5301.27", "eps300sec"=\>"5365.53", "eps900sec"=\>"5365.53", "peak60sec"=\>"6490.20", "peak"=\>"9196.60", "throttles5sec"=\>"0.00", "throttles60sec"=\>"0", "throttlestotal"=\>"0", "threshold"=\>"1.0E8"}]"

_ **DLC Field** _  
"kubernetes.pod.name": [  
"dlc-9"  
]

_ **Processors** _  
[{ "grok": { "field": "message", "patterns": [ "(%{GREEDYDATA:drop\_start} \{\"eps5sec\"=\>%{NUMBER:5\_sec\_eps}, \"eps10sec\"=\>\"%{NUMBER:10\_sec\_eps}\", \"eps15sec\"=\>\"%{NUMBER:15\_sec\_eps}\", \"eps30sec\"=\>\"%{NUMBER:30\_sec\_eps}\", \"eps60sec\"=\>\"%{NUMBER:60\_sec\_eps}\", \"eps300sec\"=\>\"%{NUMBER:300\_sec\_eps}\", \"eps900sec\"=\>\"%{NUMBER:900\_sec\_eps}\", \"peak60sec\"=\>\"%{NUMBER:60\_sec\_peak}\", \"peak\"=\>\"%{NUMBER:peak\_sec}\", \"throttles5sec\"=\>\"%{NUMBER:5\_sec\_throttles}\", \"throttles60sec\"=\>\"%{NUMBER:60\_sec\_throttles}\", \"throttlestotal\"=\>\"%{NUMBER:total\_throttles}\", %{GREEDYDATA:drop\_end}\"}])" ] } }, { "convert": { "field": "5\_sec\_eps", "type": "integer" } }, { "convert": { "field": "10\_sec\_eps", "type": "float" } }, { "convert": { "field": "15\_sec\_eps", "type": "float" } }, { "convert": { "field": "30\_sec\_eps", "type": "float" } }, { "convert": { "field": "60\_sec\_eps", "type": "float" } }, { "convert": { "field": "300\_sec\_eps", "type": "float" } }, { "convert": { "field": "900\_sec\_eps", "type": "float" } } ]  
Failure Processors  
[{ "append": { "field": "error.message", "value": [ "{{{\_ingest.on\_failure\_message}}}"] } } ]

_ **What were trying to do** _  
Plot the 5, 10, 15, etc, EPS values for each DLC that we get data from as a visualization.

_ **Where were stuck** _  
The issue, I think, is that even though the values are being converted from the string to a float it might be stored as a keyword instead of a number. Or we may even have an issue where the fields are being stored as a nested array which makes it difficult to create the visualization.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 20, 2023, 8:28pm UTC](https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903/2 "2023-07-20T20:28:43Z")

</div>

> [@Ryan\_Downey](#):
>
> The issue, I think, is that even though the values are being converted from the string to a float it might be stored as a keyword instead of a number. Or we may even have an issue where the fields are being stored as a nested array which makes it difficult to create the visualization.

Yup , most likely ... can you share the mapping? Do you know how to do that?

You will need to create a template (aka mapping ) with the correct types

Can you share a sample document?

And btw when you say plot lines, viz are always aggregations in elastic, it will always be average, sum, max etc... in general kibana does not plot single values of course the average of a single value is the single value 😉

---

<div class="post-metadata">

**Author:** ![Ryan\_Downey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ryan_downey/32/35987_2.png) [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\
**Post date:** [July 21, 2023, 1:43pm UTC](https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903/3 "2023-07-21T13:43:44Z")

</div>

Any recommendations on how to post the mappings and a sample document without hitting this error? Remove the blank space seems like the way to go so I'm working on figuring that out.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/0/700a4bb72ade76ba995978ebabf5f8a007badac0.png)

Edit:  
An error occurred: Body is limited to 35000 characters; you entered  
Possible solutions: Split the responses into 2 responses or utilize something like Notepad++ and I figured it out, in Notepad I just used Edit --\> Blank Operations --\> Trim Trailing and Trim Leading Space may help get rid of the extra characters.

---

<div class="post-metadata">

**Author:** ![Ryan\_Downey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ryan_downey/32/35987_2.png) [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\
**Post date:** [July 21, 2023, 1:48pm UTC](https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903/4 "2023-07-21T13:48:26Z")

</div>

@stephenb I think I've got everything requested. Just had to scrub some fields this morning.  
Mapping:

```auto
#GET ocp-prod-its-prod-siem-infrastructure-dlc/_mapping
{
".dlc-infrastructure" : {
"mappings" : {
"_data_stream_timestamp" : {
"enabled" : true
},
"properties" : {
"10_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"15_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"300_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"30_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"5_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"5_sec_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_peak" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"900_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"@timestamp" : {
"type" : "date"
},
"agent" : {
"properties" : {
"ephemeral_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"container" : {
"properties" : {
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"image" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"runtime" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"drop_end" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"drop_start" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"ecs" : {
"properties" : {
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"error" : {
"properties" : {
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"host" : {
"properties" : {
"architecture" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"containerized" : {
"type" : "boolean"
},
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"ip" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"mac" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"os" : {
"properties" : {
"codename" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"family" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kernel" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"platform" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
},
"input" : {
"properties" : {
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"kubernetes" : {
"properties" : {
"container" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"labels" : {
"properties" : {
"app_kubernetes_io/instance" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"app_kubernetes_io/name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"controller-revision-hash" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"statefulset_kubernetes_io/pod-name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"namespace" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"namespace_labels" : {
"properties" : {
"kubernetes_io/metadata_name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"openshift-pipelines_tekton_dev/namespace-reconcile-version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"namespace_uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node" : {
"properties" : {
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"labels" : {
"properties" : {
"beta_kubernetes_io/arch" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"beta_kubernetes_io/os" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/arch" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/os" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node-role_kubernetes_io/worker" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"nodeID" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node_openshift_io/os_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"storage" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"twistlocknode" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"pod" : {
"properties" : {
"ip" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"statefulset" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
},
"log" : {
"properties" : {
"file" : {
"properties" : {
"path" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"offset" : {
"type" : "long"
}
}
},
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"peak_sec" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"stream" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"total_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
},
".dlc-infrastructure" : {
"mappings" : {
"_data_stream_timestamp" : {
"enabled" : true
},
"properties" : {
"10_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"15_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"300_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"30_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"5_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"5_sec_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_peak" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"900_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"@timestamp" : {
"type" : "date"
},
"agent" : {
"properties" : {
"ephemeral_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"container" : {
"properties" : {
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"image" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"runtime" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"drop_end" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"drop_start" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"ecs" : {
"properties" : {
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"error" : {
"properties" : {
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"host" : {
"properties" : {
"architecture" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"containerized" : {
"type" : "boolean"
},
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"ip" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"mac" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"os" : {
"properties" : {
"codename" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"family" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kernel" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"platform" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
},
"input" : {
"properties" : {
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"kubernetes" : {
"properties" : {
"container" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"labels" : {
"properties" : {
"app_kubernetes_io/instance" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"app_kubernetes_io/name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"controller-revision-hash" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"statefulset_kubernetes_io/pod-name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"namespace" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"namespace_labels" : {
"properties" : {
"kubernetes_io/metadata_name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"openshift-pipelines_tekton_dev/namespace-reconcile-version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"namespace_uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node" : {
"properties" : {
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"labels" : {
"properties" : {
"beta_kubernetes_io/arch" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"beta_kubernetes_io/os" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/arch" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/os" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node-role_kubernetes_io/worker" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"nodeID" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node_openshift_io/os_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"storage" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"twistlocknode" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"pod" : {
"properties" : {
"ip" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"statefulset" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
},
"log" : {
"properties" : {
"file" : {
"properties" : {
"path" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"offset" : {
"type" : "long"
}
}
},
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"peak_sec" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"stream" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"total_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
},
"shrink-fway-.dlc-infrastructure" : {
"mappings" : {
"_data_stream_timestamp" : {
"enabled" : true
},
"properties" : {
"10_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"15_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"300_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"30_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"5_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"5_sec_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_peak" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"60_sec_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"900_sec_eps" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"@timestamp" : {
"type" : "date"
},
"agent" : {
"properties" : {
"ephemeral_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"container" : {
"properties" : {
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"image" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"runtime" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"drop_end" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"drop_start" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"ecs" : {
"properties" : {
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"error" : {
"properties" : {
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"host" : {
"properties" : {
"architecture" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"containerized" : {
"type" : "boolean"
},
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"ip" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"mac" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"os" : {
"properties" : {
"codename" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"family" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kernel" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"platform" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
},
"input" : {
"properties" : {
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"kubernetes" : {
"properties" : {
"container" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"labels" : {
"properties" : {
"app_kubernetes_io/instance" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"app_kubernetes_io/name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"controller-revision-hash" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"statefulset_kubernetes_io/pod-name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"namespace" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"namespace_labels" : {
"properties" : {
"kubernetes_io/metadata_name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"openshift-pipelines_tekton_dev/namespace-reconcile-version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"namespace_uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node" : {
"properties" : {
"hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"labels" : {
"properties" : {
"beta_kubernetes_io/arch" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"beta_kubernetes_io/os" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/arch" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/hostname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"kubernetes_io/os" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node-role_kubernetes_io/worker" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"nodeID" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"node_openshift_io/os_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"storage" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"twistlocknode" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"pod" : {
"properties" : {
"ip" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"uid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"statefulset" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
},
"log" : {
"properties" : {
"file" : {
"properties" : {
"path" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"offset" : {
"type" : "long"
}
}
},
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"peak_sec" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"stream" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"total_throttles" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}

```

Example document:

```auto
{
"_index": ".dlc-infrastructure",
"_id": "yK-HeIkBTnfFfnI4ujSC",
"_version": 1,
"_score": 1,
"_ignored": [
"message.keyword"
],
"_source": {
"@timestamp": "2023-07-21T12:58:21.352Z",
"log": {
"offset": 6283114,
"file": {
"path": "/var/log/containers/infrastructure-dlc_dlc-1234.log"
}
},
"stream": "stdout",
"message": "[0] dlc_eps: [1689944058.764000000, {}, {\"eps5sec\"=>5488.200000, \"eps10sec\"=>\"4803.40\", \"eps15sec\"=>\"5836.73\", \"eps30sec\"=>\"5584.90\", \"eps60sec\"=>\"5677.70\", \"eps300sec\"=>\"5370.90\", \"eps900sec\"=>\"5370.90\", \"peak60sec\"=>\"7903.40\", \"peak\"=>\"8901.40\", \"throttles5sec\"=>\"0.00\", \"throttles60sec\"=>\"0\", \"throttlestotal\"=>\"0\", \"threshold\"=>\"1.0E8\"}]",
"input": {
"type": "container"
},
"kubernetes": {
"namespace_uid": "123456",
"namespace_labels": {
"kubernetes_io/metadata_name": "dlc-infrastructure",
"openshift-pipelines_tekton_dev/namespace-reconcile-version": "v1.6.4"
},
"pod": {
"uid": "123456",
"ip": "1.1.1.1",
"name": "dlc"
},
"namespace": "dlc-infrastructure",
"statefulset": {
"name": "dlc"
},
"labels": {
"app_kubernetes_io/instance": "dlc",
"app_kubernetes_io/name": "dlc",
"controller-revision-hash": "dlc-123456",
"statefulset_kubernetes_io/pod-name": "dlc"
},
"container": {
"name": "dlc"
},
"node": {
"labels": {
"node-role_kubernetes_io/worker": "",
"nodeID": "worker.gov",
"node_openshift_io/os_id": "rhcos",
"beta_kubernetes_io/arch": "amd64",
"beta_kubernetes_io/os": "linux",
"kubernetes_io/arch": "amd64",
"twistlocknode": "true",
"storage": "storagenode",
"kubernetes_io/os": "linux",
"kubernetes_io/hostname": "worker.gov"
},
"hostname": "worker.gov",
"name": "worker.gov",
"uid": "123456"
}
},
"host": {
"hostname": "worker.gov",
"architecture": "x86_64",
"name": "worker.gov",
"os": {
"family": "debian",
"name": "Ubuntu",
"kernel": "4.18.0-305.86.2.el8_4.x86_64",
"codename": "focal",
"type": "linux",
"platform": "ubuntu",
"version": "20.04.3 LTS (Focal Fossa)"
},
"containerized": true,
"ip": [
"1.1.1.1"
],
"mac": [
"00:00:00:00"
]
},
"container": {
"runtime": "cri-o",
"image": {
"name": "registry"
},
"id": "123456"
},
"ecs": {
"version": "1.12.0"
},
"agent": {
"id": "123456",
"name": "worker.gov",
"type": "filebeat",
"version": "7.17.0",
"hostname": "worker.gov",
"ephemeral_id": "123456"
}
},
"fields": {
"agent.version.keyword": [
"7.17.0"
],
"kubernetes.node.uid": [
"123456"
],
"kubernetes.namespace_uid.keyword": [
"123456"
],
"host.name.keyword": [
"worker.gov"
],
"kubernetes.namespace_uid": [
"123456"
],
"host.hostname": [
"worker.gov"
],
"host.mac": [
"00:00:00:00"
],
"kubernetes.node.labels.kubernetes_io/os": [
"linux"
],
"container.id": [
"123456"
],
"container.image.name": [
"registry"
],
"kubernetes.container.name.keyword": [
"dlc"
],
"host.os.version": [
"20.04.3 LTS (Focal Fossa)"
],
"kubernetes.node.labels.beta_kubernetes_io/os": [
"linux"
],
"kubernetes.pod.name.keyword": [
"dlc"
],
"kubernetes.namespace_labels.openshift-pipelines_tekton_dev/namespace-reconcile-version.keyword": [
"v1.6.4"
],
"agent.name": [
"worker.gov"
],
"kubernetes.labels.app_kubernetes_io/name": [
"dlc"
],
"kubernetes.statefulset.name.keyword": [
"dlc"
],
"host.os.type": [
"linux"
],
"kubernetes.node.labels.kubernetes_io/hostname.keyword": [
"worker.gov"
],
"agent.id.keyword": [
"123456"
],
"input.type": [
"container"
],
"kubernetes.node.uid.keyword": [
"123456"
],
"agent.hostname": [
"worker.gov"
],
"kubernetes.labels.statefulset_kubernetes_io/pod-name.keyword": [
"dlc"
],
"host.architecture": [
"x86_64"
],
"agent.id": [
"123456"
],
"host.containerized": [
true
],
"kubernetes.node.labels.twistlocknode": [
"true"
],
"input.type.keyword": [
"container"
],
"stream.keyword": [
"stdout"
],
"kubernetes.node.hostname": [
"worker.gov"
],
"kubernetes.node.labels.storage.keyword": [
"storagenode"
],
"kubernetes.node.hostname.keyword": [
"worker.gov"
],
"host.ip": [
"1.1.1.1"
],
"agent.type": [
"filebeat"
],
"host.os.kernel.keyword": [
"4.18.0-305.86.2.el8_4.x86_64"
],
"stream": [
"stdout"
],
"container.image.name.keyword": [
"registry"
],
"agent.type.keyword": [
"filebeat"
],
"kubernetes.pod.ip": [
"1.1.1.1"
],
"agent.ephemeral_id.keyword": [
"123456"
],
"kubernetes.node.labels.node_openshift_io/os_id.keyword": [
"rhcos"
],
"kubernetes.container.name": [
"dlc"
],
"agent.name.keyword": [
"worker.gov"
],
"kubernetes.node.labels.beta_kubernetes_io/arch.keyword": [
"amd64"
],
"host.os.codename": [
"focal"
],
"kubernetes.labels.app_kubernetes_io/instance.keyword": [
"dlc"
],
"kubernetes.node.labels.beta_kubernetes_io/arch": [
"amd64"
],
"@timestamp": [
"2023-07-21T12:58:21.352Z"
],
"kubernetes.pod.uid.keyword": [
"123456"
],
"container.runtime.keyword": [
"cri-o"
],
"host.os.platform": [
"ubuntu"
],
"log.file.path": [
"/var/log/containers/"
],
"agent.ephemeral_id": [
"123456"
],
"kubernetes.node.labels.kubernetes_io/arch": [
"amd64"
],
"host.architecture.keyword": [
"x86_64"
],
"kubernetes.namespace_labels.openshift-pipelines_tekton_dev/namespace-reconcile-version": [
"v1.6.4"
],
"agent.hostname.keyword": [
"worker.gov"
],
"kubernetes.node.labels.kubernetes_io/os.keyword": [
"linux"
],
"ecs.version.keyword": [
"1.12.0"
],
"host.ip.keyword": [
"1.1.1.1"
],
"kubernetes.node.labels.twistlocknode.keyword": [
"true"
],
"kubernetes.node.labels.node_openshift_io/os_id": [
"rhcos"
],
"kubernetes.namespace": [
"dlc-infrastructure"
],
"host.os.name": [
"Ubuntu"
],
"host.name": [
"worker.gov"
],
"kubernetes.labels.statefulset_kubernetes_io/pod-name": [
"dlc"
],
"kubernetes.node.labels.node-role_kubernetes_io/worker.keyword": [
""
],
"host.os.version.keyword": [
"20.04.3 LTS (Focal Fossa)"
],
"kubernetes.labels.app_kubernetes_io/instance": [
"dlc"
],
"kubernetes.node.labels.storage": [
"storagenode"
],
"log.offset": [
6283114
],
"container.runtime": [
"cri-o"
],
"ecs.version": [
"1.12.0"
],
"host.hostname.keyword": [
"worker.gov"
],
"agent.version": [
"7.17.0"
],
"kubernetes.namespace.keyword": [
"dlc-infrastructure"
],
"host.os.family": [
"debian"
],
"kubernetes.node.name": [
"worker.gov"
],
"kubernetes.node.name.keyword": [
"worker.gov"
],
"kubernetes.pod.uid": [
"123456"
],
"kubernetes.labels.controller-revision-hash.keyword": [
"dlc-123456"
],
"kubernetes.node.labels.nodeID.keyword": [
"worker.gov"
],
"host.os.kernel": [
"4.18.0-305.86.2.el8_4.x86_64"
],
"kubernetes.node.labels.kubernetes_io/arch.keyword": [
"amd64"
],
"host.os.name.keyword": [
"Ubuntu"
],
"kubernetes.pod.name": [
"dlc"
],
"kubernetes.node.labels.nodeID": [
"worker.gov"
],
"log.file.path.keyword": [
"/var/log/containers/.log"
],
"kubernetes.node.labels.node-role_kubernetes_io/worker": [
""
],
"host.os.codename.keyword": [
"focal"
],
"host.mac.keyword": [
"00:00:00:00",
],
"kubernetes.namespace_labels.kubernetes_io/metadata_name": [
"dlc-infrastructure"
],
"message": [
"[0] dlc_eps: [1689944058.764000000, {}, {\"eps5sec\"=>5488.200000, \"eps10sec\"=>\"4803.40\", \"eps15sec\"=>\"5836.73\", \"eps30sec\"=>\"5584.90\", \"eps60sec\"=>\"5677.70\", \"eps300sec\"=>\"5370.90\", \"eps900sec\"=>\"5370.90\", \"peak60sec\"=>\"7903.40\", \"peak\"=>\"8901.40\", \"throttles5sec\"=>\"0.00\", \"throttles60sec\"=>\"0\", \"throttlestotal\"=>\"0\", \"threshold\"=>\"1.0E8\"}]"
],
"kubernetes.node.labels.kubernetes_io/hostname": [
"worker.gov"
],
"host.os.family.keyword": [
"debian"
],
"kubernetes.statefulset.name": [
"dlc"
],
"host.os.type.keyword": [
"linux"
],
"host.os.platform.keyword": [
"ubuntu"
],
"kubernetes.labels.app_kubernetes_io/name.keyword": [
"dlc"
],
"kubernetes.namespace_labels.kubernetes_io/metadata_name.keyword": [
"dlc-infrastructure"
],
"kubernetes.labels.controller-revision-hash": [
"dlc-123456"
],
"container.id.keyword": [
"123456"
],
"kubernetes.node.labels.beta_kubernetes_io/os.keyword": [
"linux"
],
"kubernetes.pod.ip.keyword": [
"1.1.1.1"
]
},
"ignored_field_values": {
"message.keyword": [
"[0] dlc_eps: [1689944058.764000000, {}, {\"eps5sec\"=>5488.200000, \"eps10sec\"=>\"4803.40\", \"eps15sec\"=>\"5836.73\", \"eps30sec\"=>\"5584.90\", \"eps60sec\"=>\"5677.70\", \"eps300sec\"=>\"5370.90\", \"eps900sec\"=>\"5370.90\", \"peak60sec\"=>\"7903.40\", \"peak\"=>\"8901.40\", \"throttles5sec\"=>\"0.00\", \"throttles60sec\"=>\"0\", \"throttlestotal\"=>\"0\", \"threshold\"=>\"1.0E8\"}]"
]
}
}

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 21, 2023, 1:58pm UTC](https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903/5 "2023-07-21T13:58:26Z")

</div>

> [@Ryan\_Downey](#):
>
> ```auto
> "properties" : {
> "10_sec_eps" : {
> "type" : "text",
> "fields" : {
> "keyword" : {
> "type" : "keyword",
> "ignore_above" : 256
> }
> 
> ```

Yes that is the problem you need to create a mapping before you ingest (an index template) and set those to a float type... I do not see those in your sample document.

Elasticsearch should have guested the right type but I suspect the sample doce had those values in quotes so it interpreted them as strings

> **[Index templates | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html)**

> **[Numeric field types | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html)**

---

<div class="post-metadata">

**Author:** ![Ryan\_Downey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ryan_downey/32/35987_2.png) [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\
**Post date:** [July 21, 2023, 6:22pm UTC](https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903/6 "2023-07-21T18:22:45Z")

</div>

Looking much better! Thanks for the guidance.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/6/4630fa4ad06ca72e9cb61b9f1c69ca058b171305.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 18, 2023, 6:23pm UTC](https://discuss.elastic.co/t/convert-message-into-eps-data-and-create-a-visualization/338903/7 "2023-08-18T18:23:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.