# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1866 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1867 --- ## [ELASTICSEARCH - Count unique value with a condition](https://discuss.elastic.co/t/elasticsearch-count-unique-value-with-a-condition/244983)
**Author:** [@Miguel\_BA](https://discuss.elastic.co/u/Miguel_BA)\ **Replies:** 4\ **Last updated:** [August 14, 2020, 1:08pm UTC](https://discuss.elastic.co/t/elasticsearch-count-unique-value-with-a-condition/244983 "2020-08-14T13:08:20Z")
I would like a query which it returns the number of times a field is repeated, according to the unique value of another field I have this json: "name" : james, "city" : "chicago" \<----------- same … --- ## [Logstash pipelines not working together, but separately yes](https://discuss.elastic.co/t/logstash-pipelines-not-working-together-but-separately-yes/244973)
**Author:** [@Andrea\_Bozzano](https://discuss.elastic.co/u/Andrea_Bozzano)\ **Replies:** 5\ **Last updated:** [August 14, 2020, 12:46pm UTC](https://discuss.elastic.co/t/logstash-pipelines-not-working-together-but-separately-yes/244973 "2020-08-14T12:46:00Z")
Hi everyone, and sorry to bother you but i can't find a proper solution to this. I'm trying to setup a pipelines.yml file to ingest data with Logstash through 3 pipelines. I'm running Logstash on a single node ELK stac… --- ## [Logstash add field metadata not working](https://discuss.elastic.co/t/logstash-add-field-metadata-not-working/244911)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 4\ **Last updated:** [August 14, 2020, 12:40pm UTC](https://discuss.elastic.co/t/logstash-add-field-metadata-not-working/244911 "2020-08-14T12:40:42Z")
I have tried a number of different solutions and cannot get this to work. What am I missing? https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html filter { mutate { id =\> "add-pipeline" … --- ## [New to ES some questions](https://discuss.elastic.co/t/new-to-es-some-questions/244895)
**Author:** [@DevOnTie](https://discuss.elastic.co/u/DevOnTie)\ **Replies:** 1\ **Last updated:** [August 14, 2020, 12:18pm UTC](https://discuss.elastic.co/t/new-to-es-some-questions/244895 "2020-08-14T12:18:32Z")
Hello, I'm new in using elastic search (well... and programming as well), and my boss created an instance @amazon and I'm not sure of how to use es from it, where should I look, here or amazon help (if outside, it's allo… --- ## [Continuous transform is "indexing", but no new destination records](https://discuss.elastic.co/t/continuous-transform-is-indexing-but-no-new-destination-records/244692)
**Author:** [@Mark\_Duncan](https://discuss.elastic.co/u/Mark_Duncan)\ **Replies:** 3\ **Last updated:** [August 14, 2020, 12:12pm UTC](https://discuss.elastic.co/t/continuous-transform-is-indexing-but-no-new-destination-records/244692 "2020-08-14T12:12:58Z")
My last record in the destination index is for the minute, 2020-08-11 20:05:00. It is a continuous transform and it's status=indexing, but no new destination logs are being generated. I have other transforms in continu… --- ## [Grokking of the request of an apache logfile is to optimistic](https://discuss.elastic.co/t/grokking-of-the-request-of-an-apache-logfile-is-to-optimistic/245002)
**Author:** [@guequierre](https://discuss.elastic.co/u/guequierre)\ **Replies:** 0\ **Last updated:** [August 14, 2020, 11:37am UTC](https://discuss.elastic.co/t/grokking-of-the-request-of-an-apache-logfile-is-to-optimistic/245002 "2020-08-14T11:37:17Z")
Right now it expects the request always to be in a valid form of "method url version". But in practice i do see a lot of invalid requests which i actually from a siem point of view i want to know about. As these are pote… --- ## [ElasticSearch 7.7.1 plugin for custom score](https://discuss.elastic.co/t/elasticsearch-7-7-1-plugin-for-custom-score/245001)
**Author:** [@dimminbox](https://discuss.elastic.co/u/dimminbox)\ **Replies:** 0\ **Last updated:** [August 14, 2020, 11:32am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-1-plugin-for-custom-score/245001 "2020-08-14T11:32:34Z")
Hi, Now i'm using ElasticSearch 5.3 with groovy script for calculationg of score. This script use \_doc\[field\].size() for get count of tokens for field by document. This logic i'm trying to apply in custom plugin for … --- ## [I use elasticsearch in java ,and i get some questions that i can't understand](https://discuss.elastic.co/t/i-use-elasticsearch-in-java-and-i-get-some-questions-that-i-cant-understand/244890)
**Author:** [@AnswerNo2](https://discuss.elastic.co/u/AnswerNo2)\ **Replies:** 2\ **Last updated:** [August 14, 2020, 11:31am UTC](https://discuss.elastic.co/t/i-use-elasticsearch-in-java-and-i-get-some-questions-that-i-cant-understand/244890 "2020-08-14T11:31:52Z")
when i create a doc，i need use : IndexRequest indexRequest = new IndexRequest(String indexName); the question is : indexRequest .type(type) ,this method is deprecated, and what method can replace it ？ i want know! --- ## [Filebeat not able to get k8s namespace annotations](https://discuss.elastic.co/t/filebeat-not-able-to-get-k8s-namespace-annotations/245000)
**Author:** [@tocw](https://discuss.elastic.co/u/tocw)\ **Replies:** 0\ **Last updated:** [August 14, 2020, 11:30am UTC](https://discuss.elastic.co/t/filebeat-not-able-to-get-k8s-namespace-annotations/245000 "2020-08-14T11:30:41Z")
Hello, I noticed a potential issue with filebeat. I'm trying to add some annotations to the log messages and I noticed that filebeat is not able to get namespace annotations from kubernetes cluster. Pod annotations are … --- ## [Exact match not apearing on top](https://discuss.elastic.co/t/exact-match-not-apearing-on-top/244986)
**Author:** [@ankitbhai](https://discuss.elastic.co/u/ankitbhai)\ **Replies:** 3\ **Last updated:** [August 14, 2020, 11:11am UTC](https://discuss.elastic.co/t/exact-match-not-apearing-on-top/244986 "2020-08-14T11:11:07Z")
Hello, I need to get results first exact match and then other match. I've tried it with boost and should but still it's not working for me. Please give your help/suggestion for the same. {"size":20,"query":{"bool":{"mu… --- ## [Due to heavy load, Shards getting unassigned](https://discuss.elastic.co/t/due-to-heavy-load-shards-getting-unassigned/244433)
**Author:** [@het](https://discuss.elastic.co/u/het)\ **Replies:** 14\ **Last updated:** [August 14, 2020, 11:07am UTC](https://discuss.elastic.co/t/due-to-heavy-load-shards-getting-unassigned/244433 "2020-08-14T11:07:26Z")
Hi, I have a cluster contains 3 data nodes 31 GB of each, and 3 master nodes 8 GB of each. I have around a total of 252 shards. The total data size is approx. 65 GB. I am using 7.7.1 ElasticSearch When I run a load of… --- ## [Lose all my indices](https://discuss.elastic.co/t/lose-all-my-indices/244982)
**Author:** [@Alan\_Digital](https://discuss.elastic.co/u/Alan_Digital)\ **Replies:** 2\ **Last updated:** [August 14, 2020, 10:27am UTC](https://discuss.elastic.co/t/lose-all-my-indices/244982 "2020-08-14T10:27:08Z")
Hi, I've got some issues with my elastic, I have lost all my indices. I'm trying to find out, where this error is coming from? After 2 weeks all my data disappears. I'm using elasticstore to synchronize my data with … --- ## [\[timelion\] - I cannot select all of my indexes](https://discuss.elastic.co/t/timelion-i-cannot-select-all-of-my-indexes/244910)
**Author:** [@Mark\_Duncan](https://discuss.elastic.co/u/Mark_Duncan)\ **Replies:** 3\ **Last updated:** [August 14, 2020, 10:25am UTC](https://discuss.elastic.co/t/timelion-i-cannot-select-all-of-my-indexes/244910 "2020-08-14T10:25:58Z")
I have a number of indexes of data that I can select in other visualisation. But when I try to select or query them in timelion using index=blah I cannot select them all. Why would I not be able to access all indexes i… --- ## [Help Logstash XML Parsing to Xpath](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051)
**Author:** [@akim](https://discuss.elastic.co/u/akim)\ **Replies:** 22\ **Last updated:** [August 14, 2020, 10:06am UTC](https://discuss.elastic.co/t/help-logstash-xml-parsing-to-xpath/241051 "2020-08-14T10:06:52Z")
Hello! I need help how to parse this kind of XML File: \ \ \ \ **Author:** [@Ankitha\_Sathya](https://discuss.elastic.co/u/Ankitha_Sathya)\ **Replies:** 1\ **Last updated:** [August 14, 2020, 9:22am UTC](https://discuss.elastic.co/t/what-is-the-minimum-number-of-servers-necessary-for-healthy-elastic-search-cluster/244977 "2020-08-14T09:22:30Z") Hi all, I am new to Elastic search and trying to understand what is the minimum requirement for a ES cluster keeping availability in mind. Note that I am considering all the nodes are master-eligible and data node and … --- ## [Load Balancing Elasticsearch Nodes](https://discuss.elastic.co/t/load-balancing-elasticsearch-nodes/244933)
**Author:** [@jijo.john](https://discuss.elastic.co/u/jijo.john)\ **Replies:** 2\ **Last updated:** [August 14, 2020, 9:15am UTC](https://discuss.elastic.co/t/load-balancing-elasticsearch-nodes/244933 "2020-08-14T09:15:48Z")
Hi There, We are new to elasticsearch and running a POC and nodes are hosted as EC2 instance in aws. We have a query for elasticsearch node redundancy. How do we configure multiple elasticsearch nodes in auditbeat/fileb… --- ## [Passing Current time in Logstash input plugin 'exec' for executing API in curl](https://discuss.elastic.co/t/passing-current-time-in-logstash-input-plugin-exec-for-executing-api-in-curl/243933)
**Author:** [@Rebin](https://discuss.elastic.co/u/Rebin)\ **Replies:** 4\ **Last updated:** [August 14, 2020, 9:13am UTC](https://discuss.elastic.co/t/passing-current-time-in-logstash-input-plugin-exec-for-executing-api-in-curl/243933 "2020-08-14T09:13:55Z")
I want to execute API in Logstash & schedule it every 30 mins and store the data in elastic search. I'm using exec logstash plugin to do, In the curl command there is start and end query param where i need to specify epo… --- ## [Finding optimal solution to use filter in aggregations of transform index](https://discuss.elastic.co/t/finding-optimal-solution-to-use-filter-in-aggregations-of-transform-index/242980)
**Author:** [@InfiniteDreamer](https://discuss.elastic.co/u/InfiniteDreamer)\ **Replies:** 9\ **Last updated:** [August 14, 2020, 7:59am UTC](https://discuss.elastic.co/t/finding-optimal-solution-to-use-filter-in-aggregations-of-transform-index/242980 "2020-08-14T07:59:09Z")
We are trying to create transform index, Following sample data is our subset where we want to find most recent event date based our duty code. code enterprise duty visitdate 2247 HASTY&TASTY 1 Jul 27, 2020 @ … --- ## [Grok filter pattern for nginx to parse request body](https://discuss.elastic.co/t/grok-filter-pattern-for-nginx-to-parse-request-body/244898)
**Author:** [@Santu](https://discuss.elastic.co/u/Santu)\ **Replies:** 4\ **Last updated:** [August 14, 2020, 7:24am UTC](https://discuss.elastic.co/t/grok-filter-pattern-for-nginx-to-parse-request-body/244898 "2020-08-14T07:24:09Z")
Hi, i'm trying to parse the request body with logstash grok pattern for nginx generated logs. below is my grok pattern- "%{WORD:req\_body}" and i get the parsed field like below- "req\_body" : "req\_body", "@version"… --- ## [Convert all the fields to integer in csv filter](https://discuss.elastic.co/t/convert-all-the-fields-to-integer-in-csv-filter/244949)
**Author:** [@saipadu](https://discuss.elastic.co/u/saipadu)\ **Replies:** 2\ **Last updated:** [August 14, 2020, 7:02am UTC](https://discuss.elastic.co/t/convert-all-the-fields-to-integer-in-csv-filter/244949 "2020-08-14T07:02:01Z")
Hi, I'm parsing the csv output of a command to logstash which has a dynamic number of columns. All the column values are interger except one which is a time field. By default logstash takes them all as strings. How can … --- ## [How to plot correlation matrix data as heat map in kibana?](https://discuss.elastic.co/t/how-to-plot-correlation-matrix-data-as-heat-map-in-kibana/244471)
**Author:** [@apremgeorge](https://discuss.elastic.co/u/apremgeorge)\ **Replies:** 3\ **Last updated:** [August 14, 2020, 4:57am UTC](https://discuss.elastic.co/t/how-to-plot-correlation-matrix-data-as-heat-map-in-kibana/244471 "2020-08-14T04:57:56Z")
Hi, I have uploaded a correlation matrix data from python into kibana, Total 55 records, each "\_id" has 55 records Even i can view as GET atmodelleddata.cm\_v1/\_search { "aggs": { "statistics": { "matrix… --- ## [CCR - update settings for soft\_delete.enable](https://discuss.elastic.co/t/ccr-update-settings-for-soft-delete-enable/244848)
**Author:** [@aoi1](https://discuss.elastic.co/u/aoi1)\ **Replies:** 2\ **Last updated:** [August 14, 2020, 4:42am UTC](https://discuss.elastic.co/t/ccr-update-settings-for-soft-delete-enable/244848 "2020-08-14T04:42:53Z")
Hi everyone! I am now trying to use Cross Cluster Replication. It sounds powerful, good, and looks like a good way for replicating indexes. There is one question. As the document says, we need to set the leader index's… --- ## [Why not recognize simple syslog text as SYSLOGLINE in Grok Debugger?](https://discuss.elastic.co/t/why-not-recognize-simple-syslog-text-as-syslogline-in-grok-debugger/244905)
**Author:** [@patsevanton](https://discuss.elastic.co/u/patsevanton)\ **Replies:** 2\ **Last updated:** [August 14, 2020, 3:30am UTC](https://discuss.elastic.co/t/why-not-recognize-simple-syslog-text-as-syslogline-in-grok-debugger/244905 "2020-08-14T03:30:12Z")
Hello! Source log: Mar 16 00:01:25 evita postfix/smtpd\[1713\]: connect from camomile.cloud9.net\[168.100.1.3\] Grok pattern - SYSLOGLINE Output - No Matches Screenshot: --- ## [Management thread taking too much cpu](https://discuss.elastic.co/t/management-thread-taking-too-much-cpu/244935)
**Author:** [@Geeks\_Insights](https://discuss.elastic.co/u/Geeks_Insights)\ **Replies:** 2\ **Last updated:** [August 14, 2020, 2:22am UTC](https://discuss.elastic.co/t/management-thread-taking-too-much-cpu/244935 "2020-08-14T02:22:51Z")
Hi we are observing too much cpu on cluster suddenly and the hot threads shows below only on management thread pool. hotthreads 99.8% (499ms out of 500ms) cpu usage by thread 'elasticsearch\[node1\]\[management\]\[T#2\]' … --- ## [Where can we apply business logic in ELK? For instance, I don't want logs just the way they are rather some more metadata to each row. (assuming logs are in csv)](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765)
**Author:** [@AditiKhalatkar](https://discuss.elastic.co/u/AditiKhalatkar)\ **Replies:** 4\ **Last updated:** [August 14, 2020, 1:26am UTC](https://discuss.elastic.co/t/where-can-we-apply-business-logic-in-elk-for-instance-i-dont-want-logs-just-the-way-they-are-rather-some-more-metadata-to-each-row-assuming-logs-are-in-csv/243765 "2020-08-14T01:26:44Z")
Elastic search is mostly for querying data. However, I am stuck with the part where I want to ingest some logs but probably with business logic. So that the data is transformed by the time it reaches Elastic search. Can … --- ## [Generic logstash config as onprem proxy for cloud](https://discuss.elastic.co/t/generic-logstash-config-as-onprem-proxy-for-cloud/244280)
**Author:** [@NightSpark](https://discuss.elastic.co/u/NightSpark)\ **Replies:** 1\ **Last updated:** [August 14, 2020, 12:15am UTC](https://discuss.elastic.co/t/generic-logstash-config-as-onprem-proxy-for-cloud/244280 "2020-08-14T00:15:37Z")
Hi, I am running 7.8.1 and hoping that someone can provide a logstash config that will allow me to have a central server that will take requests for inputs for on prem servers and send to a elastic search running on ela… --- ## [Dynamically change the index for Elasticsearch in Logstash output](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788)
**Author:** [@udesh](https://discuss.elastic.co/u/udesh)\ **Replies:** 5\ **Last updated:** [August 13, 2020, 11:47pm UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788 "2020-08-13T23:47:31Z")
'Hi All, I'm trying to change the logstash.config file according to a log file. The log file consists three different types of log records which generated through my java application. logstash.config file is as follow… --- ## [Metricbeat ActiveMQ Broker Kibana dashboard showing blank](https://discuss.elastic.co/t/metricbeat-activemq-broker-kibana-dashboard-showing-blank/244053)
**Author:** [@Don-cloud](https://discuss.elastic.co/u/Don-cloud)\ **Replies:** 6\ **Last updated:** [August 13, 2020, 10:16pm UTC](https://discuss.elastic.co/t/metricbeat-activemq-broker-kibana-dashboard-showing-blank/244053 "2020-08-13T22:16:18Z")
Metric version: 7.7.1 Kibana : 7.6.2 how to make sure kibana dashboard is pointing to the right index? --- ## [Custom logos for Kibana 7.8.1](https://discuss.elastic.co/t/custom-logos-for-kibana-7-8-1/244097)
**Author:** [@ManuelF](https://discuss.elastic.co/u/ManuelF)\ **Replies:** 18\ **Last updated:** [August 13, 2020, 9:42pm UTC](https://discuss.elastic.co/t/custom-logos-for-kibana-7-8-1/244097 "2020-08-13T21:42:11Z")
Hi, I recently moved to ELK stack 7.8.1. It was hard for me to customize previous version 6.8.9, but after make a cocktail of workarounds, I was finally able to add own logo, colors and texts. There is an old thread to… --- ## [How to updateState in Kibana 7.8](https://discuss.elastic.co/t/how-to-updatestate-in-kibana-7-8/244936)
**Author:** [@ssimmons](https://discuss.elastic.co/u/ssimmons)\ **Replies:** 1\ **Last updated:** [August 13, 2020, 9:36pm UTC](https://discuss.elastic.co/t/how-to-updatestate-in-kibana-7-8/244936 "2020-08-13T21:36:04Z")
I'm migrating a plugin from Kibana 7.5 to 7.8 and using the new plugin/platform. In 7.5 I could call this.props.vis.updateState() from within my Editor to update the state of the visualization. I see that this cannot be… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1865) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1867)