# Latest **URL:** https://discuss.elastic.co/latest.md?page=669 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 670 --- ## [Unable to parse "message"](https://discuss.elastic.co/t/unable-to-parse-message/333635)
**Author:** [@bsauvage1](https://discuss.elastic.co/u/bsauvage1)\ **Replies:** 9\ **Last updated:** [May 23, 2023, 3:41am UTC](https://discuss.elastic.co/t/unable-to-parse-message/333635 "2023-05-23T03:41:55Z")
Hello. New user of logstash here so please bear with me! Sending over TCP from python using logstash\_async, I receive the item in logstash (see bottom of message). How can I parse the "message" into fields? I have tri… --- ## [Discover Top values "Calculated from 5,000 sample records."](https://discuss.elastic.co/t/discover-top-values-calculated-from-5-000-sample-records/334061)
**Author:** [@eorb7569](https://discuss.elastic.co/u/eorb7569)\ **Replies:** 1\ **Last updated:** [May 23, 2023, 2:06am UTC](https://discuss.elastic.co/t/discover-top-values-calculated-from-5-000-sample-records/334061 "2023-05-23T02:06:54Z")
Hello. I want to solve this problem. "Calculated from 5,000 sample records." I'd like to see Top values using all records instead of 5000. --- ## [Retrieve items sorted by mutual-terms match](https://discuss.elastic.co/t/retrieve-items-sorted-by-mutual-terms-match/333604)
**Author:** [@K\_K2](https://discuss.elastic.co/u/K_K2)\ **Replies:** 1\ **Last updated:** [May 23, 2023, 12:06am UTC](https://discuss.elastic.co/t/retrieve-items-sorted-by-mutual-terms-match/333604 "2023-05-23T00:06:08Z")
We have index mapping like this: { "mappings": { "\_source": { "includes": \[ "uid" \] }, "properties": { "follow": { "doc\_values": true, … --- ## [8.7.1: Stand Alone Kubernetes Deployment - filestream input with ID '' already exists](https://discuss.elastic.co/t/8-7-1-stand-alone-kubernetes-deployment-filestream-input-with-id-already-exists/333148)
**Author:** [@berg](https://discuss.elastic.co/u/berg)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 11:33pm UTC](https://discuss.elastic.co/t/8-7-1-stand-alone-kubernetes-deployment-filestream-input-with-id-already-exists/333148 "2023-05-22T23:33:29Z")
I've used the example K8s manifest to deploy Elastic Agent on our AWS EKS cluster. I followed the documentation in the EKS section to comment out modules that are unavailable in AWS EKS. Recently, I upgraded to 8.7.0 an… --- ## [Elasticsearch ECK on AWS EKS hosted on Fargate](https://discuss.elastic.co/t/elasticsearch-eck-on-aws-eks-hosted-on-fargate/332477)
**Author:** [@B\_Blank](https://discuss.elastic.co/u/B_Blank)\ **Replies:** 2\ **Last updated:** [May 22, 2023, 10:11pm UTC](https://discuss.elastic.co/t/elasticsearch-eck-on-aws-eks-hosted-on-fargate/332477 "2023-05-22T22:11:30Z")
I am curious to know if ECK is supported (even feasible) on AWS ECK hosted on Fargate. The ECK operator won't install when I try installing it on a Fargate hosted K8S cluster... but I can get it to install when the EKS … --- ## [Add day in Add fields](https://discuss.elastic.co/t/add-day-in-add-fields/333164)
**Author:** [@M.Naim](https://discuss.elastic.co/u/M.Naim)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 6:35pm UTC](https://discuss.elastic.co/t/add-day-in-add-fields/333164 "2023-05-22T18:35:03Z")
Hi All, I am trying to add a derived date field by adding days into exiting date fields using the below query. ZonedDateTime origValue = doc\['body.dates.dispenseDate'\].value; ZonedDateTime newValue = origValue.plusDays… --- ## [Exporting over 10K objects in Kibana](https://discuss.elastic.co/t/exporting-over-10k-objects-in-kibana/332845)
**Author:** [@Francisco\_Yanez](https://discuss.elastic.co/u/Francisco_Yanez)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 6:18pm UTC](https://discuss.elastic.co/t/exporting-over-10k-objects-in-kibana/332845 "2023-05-22T18:18:25Z")
Hello Everyone, I have to export from saved objects (Stack Management \>\> Saved Objects) over 10,000 objects. So when I tried to export them I get an error message, is there a way to export over 10K? Thanks --- ## [I want to remove nested elements from logs](https://discuss.elastic.co/t/i-want-to-remove-nested-elements-from-logs/333979)
**Author:** [@Ayushi\_bhardwaj](https://discuss.elastic.co/u/Ayushi_bhardwaj)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 5:49pm UTC](https://discuss.elastic.co/t/i-want-to-remove-nested-elements-from-logs/333979 "2023-05-22T17:49:11Z")
i want to remove nested elements from logs "x": { "test": { "rulesetname": "eq", "operation": { "name": "diagnosticresult", "version": "235" }, "device": "string… --- ## [Kubernetes custom pipeline processing](https://discuss.elastic.co/t/kubernetes-custom-pipeline-processing/333946)
**Author:** [@Vijayakumar\_Kannan](https://discuss.elastic.co/u/Vijayakumar_Kannan)\ **Replies:** 4\ **Last updated:** [May 22, 2023, 5:44pm UTC](https://discuss.elastic.co/t/kubernetes-custom-pipeline-processing/333946 "2023-05-22T17:44:50Z")
In pod following annotations mentioned but pipeline is not processing. annotations: co.elastic.logs/enabled: 'true' co.elastic.logs/fileset: syslog co.elastic.logs/module: system co… --- ## [Logstash and AWS Cloudtrail](https://discuss.elastic.co/t/logstash-and-aws-cloudtrail/333927)
**Author:** [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\ **Replies:** 3\ **Last updated:** [May 22, 2023, 5:33pm UTC](https://discuss.elastic.co/t/logstash-and-aws-cloudtrail/333927 "2023-05-22T17:33:57Z")
Help please. It appears that AWS cloudtrail puts multiple log records under one top level field, like this: "Records": \[ { "eventName": "AssumeRole", "requestParameters": { "durationSeconds": 1500, "role… --- ## [Slow ES ingestion using Dataflow template with partialUpdates](https://discuss.elastic.co/t/slow-es-ingestion-using-dataflow-template-with-partialupdates/334039)
**Author:** [@julius11](https://discuss.elastic.co/u/julius11)\ **Replies:** 0\ **Last updated:** [May 22, 2023, 5:14pm UTC](https://discuss.elastic.co/t/slow-es-ingestion-using-dataflow-template-with-partialupdates/334039 "2023-05-22T17:14:23Z")
We are using this template to ingest data once a day from BigQuery to Elastic Search. It creates a dataflow job using the following relevant parameters: "usePartialUpdate": "true", "batchSizeBytes": "5242880", … --- ## [Synonyms in kibana discover](https://discuss.elastic.co/t/synonyms-in-kibana-discover/333961)
**Author:** [@Bav\_Tech](https://discuss.elastic.co/u/Bav_Tech)\ **Replies:** 3\ **Last updated:** [May 22, 2023, 5:01pm UTC](https://discuss.elastic.co/t/synonyms-in-kibana-discover/333961 "2023-05-22T17:01:55Z")
i am able to create a synonym analyzer and use it to query and get result using the kibana dev tool. is it possible that i use that same custom synonym analyzer when searching for a text in kibana discover? or bet… --- ## [Unable to start Auditbeat on Proxmox Container](https://discuss.elastic.co/t/unable-to-start-auditbeat-on-proxmox-container/333886)
**Author:** [@tli](https://discuss.elastic.co/u/tli)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 4:36pm UTC](https://discuss.elastic.co/t/unable-to-start-auditbeat-on-proxmox-container/333886 "2023-05-22T16:36:15Z")
Hi, I tried to install auditbeat on Proxmox Container (Ubuntu) It failed with following msg written to the log 2023-05-19T15:03:04.117-0400 INFO instance/beat.go:309 Setup Beat: auditbeat; Version: 7.15.0 20… --- ## [Elastic Agent logs empty](https://discuss.elastic.co/t/elastic-agent-logs-empty/333681)
**Author:** [@liquidkite](https://discuss.elastic.co/u/liquidkite)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 4:09pm UTC](https://discuss.elastic.co/t/elastic-agent-logs-empty/333681 "2023-05-22T16:09:55Z")
Hello all, I've been working on installing integrations in elastic-agent. I removed some existing integrations and tried adding them again and the elastic-agent logs are not showing up in Fleet -\> Agents - \>logs. I get … --- ## [How to configure Elastic Agent Policy in Fleet to Handle Long Key Values in JSON Logging](https://discuss.elastic.co/t/how-to-configure-elastic-agent-policy-in-fleet-to-handle-long-key-values-in-json-logging/334033)
**Author:** [@abhidwi27](https://discuss.elastic.co/u/abhidwi27)\ **Replies:** 0\ **Last updated:** [May 22, 2023, 4:02pm UTC](https://discuss.elastic.co/t/how-to-configure-elastic-agent-policy-in-fleet-to-handle-long-key-values-in-json-logging/334033 "2023-05-22T16:02:07Z")
Hello, I have successfully set up an Elastic cluster by referring to the documentation provided by Elastic. I have configured the Elastic Stack version 8.7 in a self-managed manner. The following resources were particul… --- ## [How to configure Fleet Kubernetes Integration to Push Kubernetes Container Logs to Individual Data Streams Based on Container Name](https://discuss.elastic.co/t/how-to-configure-fleet-kubernetes-integration-to-push-kubernetes-container-logs-to-individual-data-streams-based-on-container-name/334032)
**Author:** [@abhidwi27](https://discuss.elastic.co/u/abhidwi27)\ **Replies:** 0\ **Last updated:** [May 22, 2023, 3:43pm UTC](https://discuss.elastic.co/t/how-to-configure-fleet-kubernetes-integration-to-push-kubernetes-container-logs-to-individual-data-streams-based-on-container-name/334032 "2023-05-22T15:43:41Z")
Hello, I have successfully set up an Elasticsearch license and started working on a proof-of-concept (POC). Following the documentation provided by Elastic, I have configured the Elastic Stack version 8.7 in a self-mana… --- ## [Change ML instance configuration](https://discuss.elastic.co/t/change-ml-instance-configuration/333773)
**Author:** [@Elijah\_Adeoye](https://discuss.elastic.co/u/Elijah_Adeoye)\ **Replies:** 6\ **Last updated:** [May 22, 2023, 3:18pm UTC](https://discuss.elastic.co/t/change-ml-instance-configuration/333773 "2023-05-22T15:18:43Z")
Not sure if this can be routed to a more appropriate space but how do we change the ML instance for Elastic Cloud deployments? For my eland experiment, I am currently assigned "aws.es.ml.c5d" (costly) but I'd like to cha… --- ## [I want to sort items by the array of numbers](https://discuss.elastic.co/t/i-want-to-sort-items-by-the-array-of-numbers/333981)
**Author:** [@CookiesPrompt](https://discuss.elastic.co/u/CookiesPrompt)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 3:07pm UTC](https://discuss.elastic.co/t/i-want-to-sort-items-by-the-array-of-numbers/333981 "2023-05-22T15:07:50Z")
Hello! I am using elasticsearch version 7.11 and I want to sort items by the array of numbers: lucky\_numbers.number\_list A have a lot of elements like below and arrays have a dynamic length, example: \_source { "ga… --- ## [POSTGRESQL 9.6 Y ELASTICSEARCH 8.7.0](https://discuss.elastic.co/t/postgresql-9-6-y-elasticsearch-8-7-0/333711)
**Author:** [@Giancarlo\_Huapaya\_Ra](https://discuss.elastic.co/u/Giancarlo_Huapaya_Ra)\ **Replies:** 2\ **Last updated:** [May 22, 2023, 1:54pm UTC](https://discuss.elastic.co/t/postgresql-9-6-y-elasticsearch-8-7-0/333711 "2023-05-22T13:54:15Z")
Hello everyone, I have a problem with the logs that I receive from postgresql version 9.6 to my elasticseach version 8.7.0. I have configured as instructed but I get the error shown in the image: I have another serv… --- ## [Elasticsearch is not working and gives " all shards not available" using the version 6.4.1](https://discuss.elastic.co/t/elasticsearch-is-not-working-and-gives-all-shards-not-available-using-the-version-6-4-1/334005)
**Author:** [@Shadi\_Almasri](https://discuss.elastic.co/u/Shadi_Almasri)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 1:51pm UTC](https://discuss.elastic.co/t/elasticsearch-is-not-working-and-gives-all-shards-not-available-using-the-version-6-4-1/334005 "2023-05-22T13:51:34Z")
elasticsearch is not working and gives " all shards not available" using the version 6.4.1 --- ## [How do I aggregate/rollup/transform an index that will allow me to see # of daily active users?](https://discuss.elastic.co/t/how-do-i-aggregate-rollup-transform-an-index-that-will-allow-me-to-see-of-daily-active-users/333873)
**Author:** [@pocketcolin](https://discuss.elastic.co/u/pocketcolin)\ **Replies:** 2\ **Last updated:** [May 22, 2023, 1:31pm UTC](https://discuss.elastic.co/t/how-do-i-aggregate-rollup-transform-an-index-that-will-allow-me-to-see-of-daily-active-users/333873 "2023-05-22T13:31:36Z")
My eventual goal is to graph number of daily active users in Kibana. However, to get there, I think I need an index that aggregates user activity per day. To give a little context - every user activity is logged as a sin… --- ## [Java or node.js APM agent installation without full internet access on host server](https://discuss.elastic.co/t/java-or-node-js-apm-agent-installation-without-full-internet-access-on-host-server/334001)
**Author:** [@Anand\_Hitachi](https://discuss.elastic.co/u/Anand_Hitachi)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 1:11pm UTC](https://discuss.elastic.co/t/java-or-node-js-apm-agent-installation-without-full-internet-access-on-host-server/334001 "2023-05-22T13:11:26Z")
The lab server on which the APM agent is to be installed doesn't have full internet access due to our organizations strict firewall policies. Any other way to install java or node.js APM agent. --- ## [Stack trace for async methods](https://discuss.elastic.co/t/stack-trace-for-async-methods/333846)
**Author:** [@AbhijithCV](https://discuss.elastic.co/u/AbhijithCV)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 1:07pm UTC](https://discuss.elastic.co/t/stack-trace-for-async-methods/333846 "2023-05-22T13:07:58Z")
Hi, I am trying to figure out how to trace async methods for a particular transaction. Currently I am not able to get the stack traces for end-to-end flow of the transaction. Could you please help me understand how th… --- ## [Don't Send Spans, Only Send Metadata on APM](https://discuss.elastic.co/t/dont-send-spans-only-send-metadata-on-apm/333548)
**Author:** [@sha\_games](https://discuss.elastic.co/u/sha_games)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 12:53pm UTC](https://discuss.elastic.co/t/dont-send-spans-only-send-metadata-on-apm/333548 "2023-05-22T12:53:34Z")
I need to decrease apm storage. To do it, I want to stop capturing of spans but I want to send metadata because metadata contains some important labels for me. Is it possible? --- ## [Issues regarding the installation of ElasticSearch on a remote server](https://discuss.elastic.co/t/issues-regarding-the-installation-of-elasticsearch-on-a-remote-server/333860)
**Author:** [@Gio\_27](https://discuss.elastic.co/u/Gio_27)\ **Replies:** 2\ **Last updated:** [May 22, 2023, 12:48pm UTC](https://discuss.elastic.co/t/issues-regarding-the-installation-of-elasticsearch-on-a-remote-server/333860 "2023-05-22T12:48:07Z")
Good morning everyone. First thing first: I am a newbie on topics like servers, unix systems and CLIs. I am working on a Logs monitoring & analysis tool project using the ELK stack. I need to install elasticsearch on … --- ## [Regarding not using data streams for logs](https://discuss.elastic.co/t/regarding-not-using-data-streams-for-logs/334013)
**Author:** [@Jay\_Timbadia](https://discuss.elastic.co/u/Jay_Timbadia)\ **Replies:** 1\ **Last updated:** [May 22, 2023, 12:26pm UTC](https://discuss.elastic.co/t/regarding-not-using-data-streams-for-logs/334013 "2023-05-22T12:26:40Z")
Hi, I am using latest version of Elastic Search. I am trying to Log my application logs to elasticsearch via logstash. It seems that its using data streams by default to create new indexes with weird index hidden name… --- ## [Event.ingested huge time difference](https://discuss.elastic.co/t/event-ingested-huge-time-difference/333975)
**Author:** [@AnkurYogi](https://discuss.elastic.co/u/AnkurYogi)\ **Replies:** 6\ **Last updated:** [May 22, 2023, 11:06am UTC](https://discuss.elastic.co/t/event-ingested-huge-time-difference/333975 "2023-05-22T11:06:23Z")
Hello All, While investigating on an event I noticed there was a huge difference between event.created and event.ingested which created a confusion on the real event time. Later digging in docs resulted event.ingested… --- ## [ILM on single-node?](https://discuss.elastic.co/t/ilm-on-single-node/333997)
**Author:** [@Skairik](https://discuss.elastic.co/u/Skairik)\ **Replies:** 6\ **Last updated:** [May 22, 2023, 10:44am UTC](https://discuss.elastic.co/t/ilm-on-single-node/333997 "2023-05-22T10:44:37Z")
Hello everyone, I am currently on a single-node infrastructure and I would like to know if the implementation of ILM was useful, I assumed that putting an ILM with the different phases would allow me to keep my data lon… --- ## [Kibana url template scripted field](https://discuss.elastic.co/t/kibana-url-template-scripted-field/329050)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 3\ **Last updated:** [May 22, 2023, 10:29am UTC](https://discuss.elastic.co/t/kibana-url-template-scripted-field/329050 "2023-05-22T10:29:53Z")
Hello All, I've created a scripted field and would like know how can i configure the url host and port dynamically through some external config for 1 specific index pattern? intention is not to come in kibana and do… --- ## [Equal field values show up as different](https://discuss.elastic.co/t/equal-field-values-show-up-as-different/333436)
**Author:** [@DarkKooky](https://discuss.elastic.co/u/DarkKooky)\ **Replies:** 2\ **Last updated:** [May 22, 2023, 9:42am UTC](https://discuss.elastic.co/t/equal-field-values-show-up-as-different/333436 "2023-05-22T09:42:03Z")
Why do these values show up as different although equal and how should it be fixed? If this could help: the stack consists of Elasticsearch, Kibana and Filebeat Filebeat's input comes from port 514 the logs are proces… [Previous page](https://discuss.elastic.co/latest.md?page=668) [Next page](https://discuss.elastic.co/latest.md?page=670)