# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1218 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1219 --- ## [Is it better to have one pipeline or multiple pipelines?](https://discuss.elastic.co/t/is-it-better-to-have-one-pipeline-or-multiple-pipelines/285493)
**Author:** [@riahc3](https://discuss.elastic.co/u/riahc3)\ **Replies:** 6\ **Last updated:** [September 30, 2021, 5:12pm UTC](https://discuss.elastic.co/t/is-it-better-to-have-one-pipeline-or-multiple-pipelines/285493 "2021-09-30T17:12:41Z")
Hello From the start, Ive implemented the Elastic Stack using Logstash as the reciever and sender of logs to Logstash. Ive always implemented it using various pipelines. Each pipeline is organized by a different config… --- ## [Define ordering in wildcard query](https://discuss.elastic.co/t/define-ordering-in-wildcard-query/285601)
**Author:** [@v\_kumar](https://discuss.elastic.co/u/v_kumar)\ **Replies:** 2\ **Last updated:** [September 30, 2021, 5:11pm UTC](https://discuss.elastic.co/t/define-ordering-in-wildcard-query/285601 "2021-09-30T17:11:44Z")
Hi, In one of our use-case we're using a wildcard query on a field("cmpy") with searchQuery(val). Let's say values on different documents for "cmpy" field is "val", "1 val", "a value". Ask is, is there a way to order th… --- ## [Sort by term position, ignoring term freqs](https://discuss.elastic.co/t/sort-by-term-position-ignoring-term-freqs/285611)
**Author:** [@maurienvy](https://discuss.elastic.co/u/maurienvy)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 3:36pm UTC](https://discuss.elastic.co/t/sort-by-term-position-ignoring-term-freqs/285611 "2021-09-30T15:36:16Z")
Hello everyone! I want to do a search that allows me to sort the results depending on wether the terms that I am looking for are closer to the beggining of the text field, and ignore term frequency altogether. So for ex… --- ## [Collecting syslog (514/udp) with Logstash in Docker swarm](https://discuss.elastic.co/t/collecting-syslog-514-udp-with-logstash-in-docker-swarm/285610)
**Author:** [@antoine.brun](https://discuss.elastic.co/u/antoine.brun)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 3:29pm UTC](https://discuss.elastic.co/t/collecting-syslog-514-udp-with-logstash-in-docker-swarm/285610 "2021-09-30T15:29:28Z")
Hello, we are trying to test Logstash to collect syslogs to replace our in-house syslog collector. This works fine when our application runs in Docker, the syslogs are sent to the nginx container that forwards them to … --- ## [ECK deployment fails on container runtime containerD](https://discuss.elastic.co/t/eck-deployment-fails-on-container-runtime-containerd/285488)
**Author:** [@keshara](https://discuss.elastic.co/u/keshara)\ **Replies:** 1\ **Last updated:** [September 30, 2021, 3:14pm UTC](https://discuss.elastic.co/t/eck-deployment-fails-on-container-runtime-containerd/285488 "2021-09-30T15:14:54Z")
Hi, I setup a Kubernetes cluster on vmware bare-metal environment. This Kubernetes cluster backed by container run-time =\> containerD(not Docker). # kubectl get nodes -o wide NAME STATUS ROLES … --- ## [How to modify a single value from timestamp](https://discuss.elastic.co/t/how-to-modify-a-single-value-from-timestamp/285567)
**Author:** [@Jose\_E](https://discuss.elastic.co/u/Jose_E)\ **Replies:** 3\ **Last updated:** [September 30, 2021, 3:08pm UTC](https://discuss.elastic.co/t/how-to-modify-a-single-value-from-timestamp/285567 "2021-09-30T15:08:13Z")
Hi all, I'm trying to find out a way to substitute a single value from @timestamp, from another one, for instance, the hour. So far I know you can get the timestamp separated values using %{+HH}, but I can't find a way… --- ## [Logstash misinterprets incoming data](https://discuss.elastic.co/t/logstash-misinterprets-incoming-data/285200)
**Author:** [@hellotty](https://discuss.elastic.co/u/hellotty)\ **Replies:** 4\ **Last updated:** [September 30, 2021, 2:45pm UTC](https://discuss.elastic.co/t/logstash-misinterprets-incoming-data/285200 "2021-09-30T14:45:55Z")
Hello. I am streaming mdaemon logs using filebeat and using multiline.pattern to concatenate lines into one event. Using logstash, I create fields "mail from", "mail to", "message id", etc. Is it possible to make grok … --- ## [TCP plugin tuning](https://discuss.elastic.co/t/tcp-plugin-tuning/285510)
**Author:** [@vaseemQA](https://discuss.elastic.co/u/vaseemQA)\ **Replies:** 5\ **Last updated:** [September 30, 2021, 2:40pm UTC](https://discuss.elastic.co/t/tcp-plugin-tuning/285510 "2021-09-30T14:40:16Z")
Hi Team, I'm Using TCP plugin inside input and opened port =\> 12345 and then I've started logstash which works a listener, on the other side I've generated a sample shell script which works as a client and opened a TCP … --- ## [Unexpected end of ZLIB input stream on parsing gzip files](https://discuss.elastic.co/t/unexpected-end-of-zlib-input-stream-on-parsing-gzip-files/285603)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 2:35pm UTC](https://discuss.elastic.co/t/unexpected-end-of-zlib-input-stream-on-parsing-gzip-files/285603 "2021-09-30T14:35:55Z")
hello , I have a few MBs of gzip files ,which logstash fails to parse for some reason . I have 3 configs set up in pipelines.yml and 1 succeeds first and the other 2 fail the same error . \[2021-09-30T14:29:47,055\]\[ERRO… --- ## [Is it possible to add a runtime field which is the percentage of two different columns in the same row?](https://discuss.elastic.co/t/is-it-possible-to-add-a-runtime-field-which-is-the-percentage-of-two-different-columns-in-the-same-row/285597)
**Author:** [@Amphagory](https://discuss.elastic.co/u/Amphagory)\ **Replies:** 1\ **Last updated:** [September 30, 2021, 2:01pm UTC](https://discuss.elastic.co/t/is-it-possible-to-add-a-runtime-field-which-is-the-percentage-of-two-different-columns-in-the-same-row/285597 "2021-09-30T14:01:46Z")
Hello, I see scripted fields has been deprecated in 7.13 and it is suggested that we use runtime fields. I was wondering if it is possible to add a runtime field which calculates the ratio of two different column value… --- ## [Authentication in elastic agent](https://discuss.elastic.co/t/authentication-in-elastic-agent/284979)
**Author:** [@Paurav\_Thakkar](https://discuss.elastic.co/u/Paurav_Thakkar)\ **Replies:** 2\ **Last updated:** [September 30, 2021, 1:17pm UTC](https://discuss.elastic.co/t/authentication-in-elastic-agent/284979 "2021-09-30T13:17:29Z")
Hi, I am trying to understand how the authentication is handled in elastic agent for Elasticsearch. just like we have multiple options to configure beats to send data to Elasticsearch by giving api\_key, id/pass or certi… --- ## [High CPU usage ElasticSearch Causing Search slow and timeout with Kibana](https://discuss.elastic.co/t/high-cpu-usage-elasticsearch-causing-search-slow-and-timeout-with-kibana/285221)
**Author:** [@mitesh.gangaramani](https://discuss.elastic.co/u/mitesh.gangaramani)\ **Replies:** 8\ **Last updated:** [September 30, 2021, 12:48pm UTC](https://discuss.elastic.co/t/high-cpu-usage-elasticsearch-causing-search-slow-and-timeout-with-kibana/285221 "2021-09-30T12:48:20Z")
Hello Guys, Hope you are all doing well! We are using Fluent-bit + Elastic + Kibana stack for logging our kubernetes containers. However, sometimes the CPU usage goes too high and search from kibana not working. Setup… --- ## [Override agent.name, or add custom fields to fleet managed agents?](https://discuss.elastic.co/t/override-agent-name-or-add-custom-fields-to-fleet-managed-agents/285073)
**Author:** [@buzzdeee](https://discuss.elastic.co/u/buzzdeee)\ **Replies:** 3\ **Last updated:** [September 30, 2021, 12:37pm UTC](https://discuss.elastic.co/t/override-agent-name-or-add-custom-fields-to-fleet-managed-agents/285073 "2021-09-30T12:37:43Z")
Hi, currently I deploy the \*beats with salt, and in the beats config file, I use to set the name: That name: field then in Elasticsearch/kibana shows up as the agent.name in the indices. This allows me easily to fin… --- ## [Logstash persistent queue max capacity](https://discuss.elastic.co/t/logstash-persistent-queue-max-capacity/285588)
**Author:** [@samia](https://discuss.elastic.co/u/samia)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 12:19pm UTC](https://discuss.elastic.co/t/logstash-persistent-queue-max-capacity/285588 "2021-09-30T12:19:27Z")
Hello, I would like to know if there's a maximum for the queue size of Logstash persistent queue ? I read that its default value is 1GB and it must not exceed the local disk size. I wander if it can handle volumes of … --- ## [Data/write/bulk tasks appearing on master nodes](https://discuss.elastic.co/t/data-write-bulk-tasks-appearing-on-master-nodes/285375)
**Author:** [@skijash](https://discuss.elastic.co/u/skijash)\ **Replies:** 2\ **Last updated:** [September 30, 2021, 11:58am UTC](https://discuss.elastic.co/t/data-write-bulk-tasks-appearing-on-master-nodes/285375 "2021-09-30T11:58:27Z")
I'm seeing some strange usage patters on dedicated master nodes in a 5.6.16 Elasticsearch cluster. Master nodes seem to be having excessive heap pressure during some workloads, causing them to crash with OOM errors. Whe… --- ## [Strings shown as Multifields](https://discuss.elastic.co/t/strings-shown-as-multifields/285580)
**Author:** [@KarlosA](https://discuss.elastic.co/u/KarlosA)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 11:14am UTC](https://discuss.elastic.co/t/strings-shown-as-multifields/285580 "2021-09-30T11:14:19Z")
Hello, Using Elasticsearch 7.8.1 the following mapping: "mappings": { "\_routing": { "required": true }, "properties": { "qualified\_host": { … --- ## [Extremely Large Documents: Querying and Dealing with](https://discuss.elastic.co/t/extremely-large-documents-querying-and-dealing-with/284393)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 16\ **Last updated:** [September 30, 2021, 11:19am UTC](https://discuss.elastic.co/t/extremely-large-documents-querying-and-dealing-with/284393 "2021-09-30T11:19:31Z")
We are in a situation where extremely large documents were indexed (to text fields) and our ElasticSearch instance has been going down/crazy recently with out of disk and out of memory. We've also run into HTTP response … --- ## [Metricbeat creates a Standalone Cluster](https://discuss.elastic.co/t/metricbeat-creates-a-standalone-cluster/285492)
**Author:** [@Kosodrom](https://discuss.elastic.co/u/Kosodrom)\ **Replies:** 1\ **Last updated:** [September 30, 2021, 11:15am UTC](https://discuss.elastic.co/t/metricbeat-creates-a-standalone-cluster/285492 "2021-09-30T11:15:00Z")
Hi folks, I have running 3 nodes elastic cluster and 2 nodes of logstash and I use metricbeat to monitor them and filebeat to collect logs. Everything is in 7.13.2 version Currently I am experiencing following issue wh… --- ## [ECS - Exported fields origin of rsa.foo.\*](https://discuss.elastic.co/t/ecs-exported-fields-origin-of-rsa-foo/285554)
**Author:** [@CamTheMan](https://discuss.elastic.co/u/CamTheMan)\ **Replies:** 1\ **Last updated:** [September 30, 2021, 10:56am UTC](https://discuss.elastic.co/t/ecs-exported-fields-origin-of-rsa-foo/285554 "2021-09-30T10:56:00Z")
I'm currently implementing 'ECS' for our in house products and I was wondering what the origin or meaning of the 'rsa' fields are? It is unusual in that they span multiple vendors and am wondering what the 'rsa' acronym… --- ## [Reindexing Api is not copying the data](https://discuss.elastic.co/t/reindexing-api-is-not-copying-the-data/285454)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 2\ **Last updated:** [September 30, 2021, 9:42am UTC](https://discuss.elastic.co/t/reindexing-api-is-not-copying-the-data/285454 "2021-09-30T09:42:19Z")
Using multi-node Elasticsearch version 7.14 I am facing an issue while doing reindexing.I want reindex all the indices given below health status index uuid pri rep docs.coun… --- ## [Logstash memcached filter plugin can not open more then 4 connection](https://discuss.elastic.co/t/logstash-memcached-filter-plugin-can-not-open-more-then-4-connection/285010)
**Author:** [@ramakrushna.sahu](https://discuss.elastic.co/u/ramakrushna.sahu)\ **Replies:** 5\ **Last updated:** [September 30, 2021, 9:27am UTC](https://discuss.elastic.co/t/logstash-memcached-filter-plugin-can-not-open-more-then-4-connection/285010 "2021-09-30T09:27:35Z")
Hi Team, Need help on logstash Memcached filter plugin. I see the logstash memcached filter plugin can not open more than 4 connections to Memcached server. is there any configuration required for this memcached filte… --- ## [How to create a visualization for node cpu usage of kubernetes](https://discuss.elastic.co/t/how-to-create-a-visualization-for-node-cpu-usage-of-kubernetes/285356)
**Author:** [@Kaushik\_R](https://discuss.elastic.co/u/Kaushik_R)\ **Replies:** 5\ **Last updated:** [September 30, 2021, 8:59am UTC](https://discuss.elastic.co/t/how-to-create-a-visualization-for-node-cpu-usage-of-kubernetes/285356 "2021-09-30T08:59:49Z")
Hi All, I'm trying to create a visualization to capture the node cpu usage of a kubernetes node. i'm trying to use kubernetes.node.cpu.usage.nanocores , kubernetes.node.cpu.usage.nanocores but none is working the outco… --- ## [How to visualise objects fields in a tab like in DISPLAY 1 (see body message)](https://discuss.elastic.co/t/how-to-visualise-objects-fields-in-a-tab-like-in-display-1-see-body-message/285561)
**Author:** [@paulpatou](https://discuss.elastic.co/u/paulpatou)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 8:59am UTC](https://discuss.elastic.co/t/how-to-visualise-objects-fields-in-a-tab-like-in-display-1-see-body-message/285561 "2021-09-30T08:59:18Z")
Hello, I am using logstash to send data into elastic. My goal is to get a visualization in kibana like : "DISPLAY 1" group\_num| name\_group |unique count of id\_document 10 |groupA |2 20 |groupB … --- ## [Logstash cannot reach Elasticsearch / filebeat security](https://discuss.elastic.co/t/logstash-cannot-reach-elasticsearch-filebeat-security/284733)
**Author:** [@smam](https://discuss.elastic.co/u/smam)\ **Replies:** 11\ **Last updated:** [September 30, 2021, 8:20am UTC](https://discuss.elastic.co/t/logstash-cannot-reach-elasticsearch-filebeat-security/284733 "2021-09-30T08:20:07Z")
Hello, I am encountering some Problems while trying to secure Filebeat&Logstash. Scenario: I created my own RootCA and signed certificates with it. My Kibana-Webserver for example is signed with this Certificate, but … --- ## [Kibana Visualization Top Hit Counts](https://discuss.elastic.co/t/kibana-visualization-top-hit-counts/285365)
**Author:** [@susmeeta](https://discuss.elastic.co/u/susmeeta)\ **Replies:** 2\ **Last updated:** [September 29, 2021, 12:54pm UTC](https://discuss.elastic.co/t/kibana-visualization-top-hit-counts/285365 "2021-09-29T12:54:21Z")
Hi , We have a scenario where each case is processed multiple times. I am trying to fetch the unique count of cases along with the latest status of each case. To get this i have used Top Hit aggregation to get the lates… --- ## [Query DSL: Aggregation of fields and also value of fields](https://discuss.elastic.co/t/query-dsl-aggregation-of-fields-and-also-value-of-fields/285083)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 2\ **Last updated:** [September 24, 2021, 3:11pm UTC](https://discuss.elastic.co/t/query-dsl-aggregation-of-fields-and-also-value-of-fields/285083 "2021-09-24T15:11:07Z")
My data structure is as follows username,department,salary bob,finance,2000 joe,HR,2200 chan,finance,3000 bobby,finance,2500 don,HR,1800 I was looking for a Splunk's equivalent command of ... | stats values(salary) b… --- ## [Query DSL: How to use WILDCARD queries on \_id](https://discuss.elastic.co/t/query-dsl-how-to-use-wildcard-queries-on-id/285547)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 7:29am UTC](https://discuss.elastic.co/t/query-dsl-how-to-use-wildcard-queries-on-id/285547 "2021-09-30T07:29:44Z")
How to use WILDCARD queries on \_id While searching for specific type of Watcher runs (triggered\_watches), I was looking for a run of a specific type of watcher which contains a keyword in the \_id. But when I do a wildca… --- ## [IndexNotFoundException during reindexing](https://discuss.elastic.co/t/indexnotfoundexception-during-reindexing/285486)
**Author:** [@Junaid\_Usman](https://discuss.elastic.co/u/Junaid_Usman)\ **Replies:** 2\ **Last updated:** [September 30, 2021, 7:22am UTC](https://discuss.elastic.co/t/indexnotfoundexception-during-reindexing/285486 "2021-09-30T07:22:49Z")
Hi All, Recently we changed the Java JDK library from Oracle to RedHat default OpenJDK. Since this change, whenever I am trying to perform reindexing I am getting the IndexNotFoundException: no such index. Does anyone h… --- ## [Setting Fortigate OID with ELK](https://discuss.elastic.co/t/setting-fortigate-oid-with-elk/285057)
**Author:** [@Jim2](https://discuss.elastic.co/u/Jim2)\ **Replies:** 1\ **Last updated:** [September 30, 2021, 7:15am UTC](https://discuss.elastic.co/t/setting-fortigate-oid-with-elk/285057 "2021-09-30T07:15:49Z")
Hi Sir I want to setting Fortigate bandwidth chart in ELK. And refer the following img that is the chart in my Fortigate. Then I have collect OID in ELK : 1.3.6.1.2.1.2.2.1.10 1.3.6.1.2.1.2.2.1.16 And Setting th… --- ## [Sliced scroll with sort](https://discuss.elastic.co/t/sliced-scroll-with-sort/284874)
**Author:** [@Burak](https://discuss.elastic.co/u/Burak)\ **Replies:** 4\ **Last updated:** [September 30, 2021, 7:05am UTC](https://discuss.elastic.co/t/sliced-scroll-with-sort/284874 "2021-09-30T07:05:05Z")
Hello guys. How does actually Elasticsearch perform sorting under the hood for shard and for index? I can find it neither in documentation or books. I am going to use a sliced scroll with sort but in some cases, I have… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1217) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1219)