# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=964 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 965 --- ## [How to persist data after logstash pod restart?](https://discuss.elastic.co/t/how-to-persist-data-after-logstash-pod-restart/303108)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 0\ **Last updated:** [April 25, 2022, 1:35am UTC](https://discuss.elastic.co/t/how-to-persist-data-after-logstash-pod-restart/303108 "2022-04-25T01:35:27Z")
Hello All, I am using Logstash helm chart. I want to persist a file after logstash pod restart. For that I can see an option persistence: enabled: true But there is no much documentation on it on how to pass variabl… --- ## [Logstash Kafka SASL\_PLAIN input plugin config](https://discuss.elastic.co/t/logstash-kafka-sasl-plain-input-plugin-config/303107)
**Author:** [@Prateek\_Tiwari](https://discuss.elastic.co/u/Prateek_Tiwari)\ **Replies:** 0\ **Last updated:** [April 24, 2022, 10:04pm UTC](https://discuss.elastic.co/t/logstash-kafka-sasl-plain-input-plugin-config/303107 "2022-04-24T22:04:49Z")
Hi, I have been trying to connect logstash to my remote kafka instance, which has SASL\_PLAIN authentication method, but still not able establish the connection and getting the following error : org.apache.kafka.clients… --- ## [DSL query not working post upgrade](https://discuss.elastic.co/t/dsl-query-not-working-post-upgrade/303095)
**Author:** [@Ankit\_Chauhan1](https://discuss.elastic.co/u/Ankit_Chauhan1)\ **Replies:** 3\ **Last updated:** [April 24, 2022, 6:55pm UTC](https://discuss.elastic.co/t/dsl-query-not-working-post-upgrade/303095 "2022-04-24T18:55:08Z")
Earlier below query was working well in the DSL query mode. but the same has stopped working post elastic upgrade to 7.17.1 {"query":{"bool":{"must":\[{"match":{"eT.keyword":"TRANSFER"}},{"nested": {"path": "actions","qu… --- ## [Logstash-CIDR Plugin](https://discuss.elastic.co/t/logstash-cidr-plugin/301990)
**Author:** [@leela](https://discuss.elastic.co/u/leela)\ **Replies:** 12\ **Last updated:** [April 24, 2022, 4:57pm UTC](https://discuss.elastic.co/t/logstash-cidr-plugin/301990 "2022-04-24T16:57:31Z")
How can we pass multiple IP's and Location names in Logstash config file with CIDR plugin for Source and Destination --- ## [Tuning ranking in search result](https://discuss.elastic.co/t/tuning-ranking-in-search-result/303097)
**Author:** [@kentoto](https://discuss.elastic.co/u/kentoto)\ **Replies:** 1\ **Last updated:** [April 24, 2022, 1:12pm UTC](https://discuss.elastic.co/t/tuning-ranking-in-search-result/303097 "2022-04-24T13:12:00Z")
doc1 = {"id":1,"text":"tonight"} doc2 = {"id":2,"text":"tonight tonight"} doc3 = {"id":3,"text":"tonight tonight tonight"} doc4 = {"id":4,"text":"tonight and something else"} doc5 = {"id":5,"text":"tonight and you"} es.… --- ## [How to solve index millions of files content with scalable approach?](https://discuss.elastic.co/t/how-to-solve-index-millions-of-files-content-with-scalable-approach/303084)
**Author:** [@sushilprj](https://discuss.elastic.co/u/sushilprj)\ **Replies:** 5\ **Last updated:** [April 24, 2022, 11:49am UTC](https://discuss.elastic.co/t/how-to-solve-index-millions-of-files-content-with-scalable-approach/303084 "2022-04-24T11:49:02Z")
I am working with an application that requires indexing thousands of files' content to make the content searchable. I use tika to extract the file content and then index it to Elasticsearch. I also tried to ingest pipeli… --- ## [What is the correct workflow for standalone elastic agent on docker swarm?](https://discuss.elastic.co/t/what-is-the-correct-workflow-for-standalone-elastic-agent-on-docker-swarm/303093)
**Author:** [@UchihaYuki](https://discuss.elastic.co/u/UchihaYuki)\ **Replies:** 0\ **Last updated:** [April 24, 2022, 10:08am UTC](https://discuss.elastic.co/t/what-is-the-correct-workflow-for-standalone-elastic-agent-on-docker-swarm/303093 "2022-04-24T10:08:51Z")
I don't use fleet server, because I hope everything is up and running in production environment just as what I have configured in development environment. I don't think fleet server can do this. It seems I have to config… --- ## [When the output target stops for a period of time and then resumes, filebeat is in a blocking state](https://discuss.elastic.co/t/when-the-output-target-stops-for-a-period-of-time-and-then-resumes-filebeat-is-in-a-blocking-state/303086)
**Author:** [@lhb6540](https://discuss.elastic.co/u/lhb6540)\ **Replies:** 0\ **Last updated:** [April 24, 2022, 7:51am UTC](https://discuss.elastic.co/t/when-the-output-target-stops-for-a-period-of-time-and-then-resumes-filebeat-is-in-a-blocking-state/303086 "2022-04-24T07:51:34Z")
version: 7.16 os: centos 7.6 install: rpm Steps to Reproduce: When the input of filebeat is an http endpoint, and the output is configured as a logstash. filebeat has a large qps, such as 60/s, you can use a script … --- ## [Autodiscover not working on docker swarm](https://discuss.elastic.co/t/autodiscover-not-working-on-docker-swarm/303082)
**Author:** [@UchihaYuki](https://discuss.elastic.co/u/UchihaYuki)\ **Replies:** 0\ **Last updated:** [April 24, 2022, 6:09am UTC](https://discuss.elastic.co/t/autodiscover-not-working-on-docker-swarm/303082 "2022-04-24T06:09:26Z")
If I config directly in metricbeat.yml: - module: traefik metricsets: - health hosts: \["http://traefik:8080"\] period: 10s enabled: true It will work. But If I use autodiscover, by configuring m… --- ## [Sub-aggregations in bucket Aggregation's Terms return 0 as Value](https://discuss.elastic.co/t/sub-aggregations-in-bucket-aggregations-terms-return-0-as-value/303053)
**Author:** [@Draken](https://discuss.elastic.co/u/Draken)\ **Replies:** 5\ **Last updated:** [April 24, 2022, 1:28am UTC](https://discuss.elastic.co/t/sub-aggregations-in-bucket-aggregations-terms-return-0-as-value/303053 "2022-04-24T01:28:57Z")
Hello, I'm recently introduced to Elasticsearch and currently trying to build a Monitoring application on top of it, at first everything went smoothly as we worked on elastic version 6.5.4, then we decided to move on to … --- ## [Logstash multiple output blocking: output blocking with multiple output using the isolation pattern](https://discuss.elastic.co/t/logstash-multiple-output-blocking-output-blocking-with-multiple-output-using-the-isolation-pattern/303078)
**Author:** [@yeppazu](https://discuss.elastic.co/u/yeppazu)\ **Replies:** 4\ **Last updated:** [April 23, 2022, 9:37pm UTC](https://discuss.elastic.co/t/logstash-multiple-output-blocking-output-blocking-with-multiple-output-using-the-isolation-pattern/303078 "2022-04-23T21:37:49Z")
Hello everyone, we have a trouble with a pipeline in logstash (7.17), with multiple output and isolator pattern. We have adopted the solution suggested in: In our case, input is Winlogbeat and the two outputs are: El… --- ## [How do I match by all objects in an array of objects in Elasticsearch?](https://discuss.elastic.co/t/how-do-i-match-by-all-objects-in-an-array-of-objects-in-elasticsearch/303074)
**Author:** [@codinges](https://discuss.elastic.co/u/codinges)\ **Replies:** 2\ **Last updated:** [April 23, 2022, 3:51pm UTC](https://discuss.elastic.co/t/how-do-i-match-by-all-objects-in-an-array-of-objects-in-elasticsearch/303074 "2022-04-23T15:51:03Z")
I want to get documents like below with match from Elasticsearch. { "configurationId": "dp4wb8kpw4k1s2z", "type": 1, "items": \[ { "text": "car", "rank": 1 }, { "text": "ship", "… --- ## [Which volumes of beats should I mount under docker swarm?](https://discuss.elastic.co/t/which-volumes-of-beats-should-i-mount-under-docker-swarm/302714)
**Author:** [@UchihaYuki](https://discuss.elastic.co/u/UchihaYuki)\ **Replies:** 2\ **Last updated:** [April 23, 2022, 2:54pm UTC](https://discuss.elastic.co/t/which-volumes-of-beats-should-i-mount-under-docker-swarm/302714 "2022-04-23T14:54:05Z")
I have a custom image for filebeat and metricbeat. So every time when they restart, filebeat setup and metricbeat setup will run before filebeat and metricbeat. I wonder which folders I should mount under a named volumn… --- ## [How to divide new index shard in all nodes](https://discuss.elastic.co/t/how-to-divide-new-index-shard-in-all-nodes/303070)
**Author:** [@DJ\_Zhu](https://discuss.elastic.co/u/DJ_Zhu)\ **Replies:** 1\ **Last updated:** [April 23, 2022, 2:19pm UTC](https://discuss.elastic.co/t/how-to-divide-new-index-shard-in-all-nodes/303070 "2022-04-23T14:19:02Z")
Thanks for your read first. I use Elasticsearch to store app logs which contains about 100 TB data in a single cluster. The data is store in a new index per 4 hours and old index will be deleted after 7 days. Since the … --- ## [Understanding the java code (Pattern Capture token filter)](https://discuss.elastic.co/t/understanding-the-java-code-pattern-capture-token-filter/303072)
**Author:** [@dishant.sharma](https://discuss.elastic.co/u/dishant.sharma)\ **Replies:** 0\ **Last updated:** [April 23, 2022, 11:41am UTC](https://discuss.elastic.co/t/understanding-the-java-code-pattern-capture-token-filter/303072 "2022-04-23T11:41:11Z")
--- ## [Logstash CSV mutate, split and filter](https://discuss.elastic.co/t/logstash-csv-mutate-split-and-filter/301084)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 5\ **Last updated:** [April 23, 2022, 9:44am UTC](https://discuss.elastic.co/t/logstash-csv-mutate-split-and-filter/301084 "2022-04-23T09:44:15Z")
Hello, I hope my message finds the Elastic community safe and healthy. I am trying to import CSV files wherein I want to create a new field - tld using data from one of the columns being imported. Source Column name… --- ## [Node failed to run](https://discuss.elastic.co/t/node-failed-to-run/303066)
**Author:** [@korjavin](https://discuss.elastic.co/u/korjavin)\ **Replies:** 3\ **Last updated:** [April 23, 2022, 7:30am UTC](https://discuss.elastic.co/t/node-failed-to-run/303066 "2022-04-23T07:30:50Z")
Node has been restarted and then \[2022-04-23T05:04:36,452\]\[ERROR\]\[o.e.b.ElasticsearchUncaughtExceptionHandler\] \[elastic-elasticsearch-data-2\] fatal error in thread \[elasticsearch\[elastic-elasticsearch-data-2\]\[generic\]\[T… --- ## [Extract data from Elastic search using pem cert](https://discuss.elastic.co/t/extract-data-from-elastic-search-using-pem-cert/302849)
**Author:** [@newelastic](https://discuss.elastic.co/u/newelastic)\ **Replies:** 8\ **Last updated:** [April 22, 2022, 7:59pm UTC](https://discuss.elastic.co/t/extract-data-from-elastic-search-using-pem-cert/302849 "2022-04-22T19:59:32Z")
Hello, I'm new to ELK stack and we are trying to extract data from ES via logstash pipeline from existing system. Below is the logstash input pipeline being used for extracting data and my output pipeline targeted to K… --- ## [Count Last received logs](https://discuss.elastic.co/t/count-last-received-logs/303024)
**Author:** [@Bo-Wyatt](https://discuss.elastic.co/u/Bo-Wyatt)\ **Replies:** 1\ **Last updated:** [April 22, 2022, 6:59pm UTC](https://discuss.elastic.co/t/count-last-received-logs/303024 "2022-04-22T18:59:15Z")
Hello, I have a SIEM-like flow o logs. I mean, 3 or more logs with an identifier (same value in all the logs). My last received log the one I want to "keep" or "consider". Is there any way to apply this concept to my Vis… --- ## [How to replace Security Certificate sign by Elastic CA](https://discuss.elastic.co/t/how-to-replace-security-certificate-sign-by-elastic-ca/302840)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 4\ **Last updated:** [April 22, 2022, 6:30pm UTC](https://discuss.elastic.co/t/how-to-replace-security-certificate-sign-by-elastic-ca/302840 "2022-04-22T18:30:34Z")
I had created certificate with following command in past. and it is going to expire now. bin/elasticsearch-certutil cert -out /etc/elasticsearch/config/elastic-certificates.p12 -pass "" I would like to replace that bef… --- ## [How to use the value from kubernetes metadata in S3 output](https://discuss.elastic.co/t/how-to-use-the-value-from-kubernetes-metadata-in-s3-output/302975)
**Author:** [@user2416](https://discuss.elastic.co/u/user2416)\ **Replies:** 1\ **Last updated:** [April 22, 2022, 4:38pm UTC](https://discuss.elastic.co/t/how-to-use-the-value-from-kubernetes-metadata-in-s3-output/302975 "2022-04-22T16:38:22Z")
Hi, I am trying to use the service name which will get from kubernetes metadata to use in S3 output to construct prefix, how can I use that field value in output? --- ## [java.net.NetPermission "specifyStreamHandler" not allowed for custom plugin](https://discuss.elastic.co/t/java-net-netpermission-specifystreamhandler-not-allowed-for-custom-plugin/301392)
**Author:** [@poanse](https://discuss.elastic.co/u/poanse)\ **Replies:** 2\ **Last updated:** [April 22, 2022, 3:50pm UTC](https://discuss.elastic.co/t/java-net-netpermission-specifystreamhandler-not-allowed-for-custom-plugin/301392 "2022-04-22T15:50:22Z")
I presume this issue is analogous to java.io.FilePermission is no longer allowed for custom plug-in. · Issue #69464 · elastic/elasticsearch · GitHub, but dealing with java.net.NetPermission "specifyStreamHandler" permiss… --- ## [Date conversion with Logstash](https://discuss.elastic.co/t/date-conversion-with-logstash/302947)
**Author:** [@Lynow](https://discuss.elastic.co/u/Lynow)\ **Replies:** 4\ **Last updated:** [April 22, 2022, 2:04pm UTC](https://discuss.elastic.co/t/date-conversion-with-logstash/302947 "2022-04-22T14:04:33Z")
Hello, I currently have a problem for date conversion with Logstash. I receive logs including dates in epoche format (UNIX), so I added the following filters to modify them. Namely, I have multiple date fields, as you c… --- ## [How to implement this ES curl into Java](https://discuss.elastic.co/t/how-to-implement-this-es-curl-into-java/303008)
**Author:** [@aboli](https://discuss.elastic.co/u/aboli)\ **Replies:** 1\ **Last updated:** [April 22, 2022, 1:55pm UTC](https://discuss.elastic.co/t/how-to-implement-this-es-curl-into-java/303008 "2022-04-22T13:55:57Z")
Hi, I want to translate this curl into working java code in spring - curl -XGET 'localhost:9200/catalog/\_search?pretty' -H 'Content-Type: application/json' -d' { "size": 0, "aggs": { "types": { "terms": { "field":… --- ## [Failed to parse date field](https://discuss.elastic.co/t/failed-to-parse-date-field/302812)
**Author:** [@Krunal](https://discuss.elastic.co/u/Krunal)\ **Replies:** 4\ **Last updated:** [April 22, 2022, 1:48pm UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/302812 "2022-04-22T13:48:22Z")
Blockquote I am having little problem regarding the date. What is the solution to change the date format, if i am having date format like wed 11:32 , fri 4:52, I cant able to do mapping in elasticsearch. So i got this … --- ## [Elastic - restoring index from snapshots does not work](https://discuss.elastic.co/t/elastic-restoring-index-from-snapshots-does-not-work/303001)
**Author:** [@JimJ](https://discuss.elastic.co/u/JimJ)\ **Replies:** 1\ **Last updated:** [April 22, 2022, 1:46pm UTC](https://discuss.elastic.co/t/elastic-restoring-index-from-snapshots-does-not-work/303001 "2022-04-22T13:46:52Z")
running Elastic stack 7.12.0. I configured a repository to snapshot indices. The system takes a snapshot on a daily basis and keeps 7 days. All snapshots seem to be okay. States are all "Snapshot complete". I tried … --- ## [Failed to create query - maxClauseCount is set to 1024](https://discuss.elastic.co/t/failed-to-create-query-maxclausecount-is-set-to-1024/302898)
**Author:** [@Sowmiya](https://discuss.elastic.co/u/Sowmiya)\ **Replies:** 3\ **Last updated:** [April 22, 2022, 1:39pm UTC](https://discuss.elastic.co/t/failed-to-create-query-maxclausecount-is-set-to-1024/302898 "2022-04-22T13:39:39Z")
Hi Team, I am using Elasticsearch version 7.16.2. Have set up the search on the domain-specific data around 20k records. When I am searching for a particular query, I am getting the below error. RequestError(400, 'sear… --- ## [Anomaly Detection Categorization: Kibana not showing all ml category](https://discuss.elastic.co/t/anomaly-detection-categorization-kibana-not-showing-all-ml-category/303002)
**Author:** [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Replies:** 6\ **Last updated:** [April 22, 2022, 1:36pm UTC](https://discuss.elastic.co/t/anomaly-detection-categorization-kibana-not-showing-all-ml-category/303002 "2022-04-22T13:36:39Z")
Hi, I am using Elasticsearch 8.1.0 Kibana 8.1.0 I have created a Categorization job in kibana. As input, I have given the index field which contains the log messages. After the job gets completed processing, I can s… --- ## [After restarting, all saved objects disappeared in Kibana](https://discuss.elastic.co/t/after-restarting-all-saved-objects-disappeared-in-kibana/300671)
**Author:** [@petschenek](https://discuss.elastic.co/u/petschenek)\ **Replies:** 4\ **Last updated:** [April 22, 2022, 1:18pm UTC](https://discuss.elastic.co/t/after-restarting-all-saved-objects-disappeared-in-kibana/300671 "2022-04-22T13:18:34Z")
Hi team, We are using Elasticsearch and Kibana (7.10.2) deployed on our Kubernetes cluster. After restarting the Kibana deployment, all of our saved objects were gone. They seem to be stored in Elasticsearch but somehow… --- ## [Harvester\_buffer\_size - how to determine most efficient value?](https://discuss.elastic.co/t/harvester-buffer-size-how-to-determine-most-efficient-value/303037)
**Author:** [@mikewillis](https://discuss.elastic.co/u/mikewillis)\ **Replies:** 0\ **Last updated:** [April 22, 2022, 1:02pm UTC](https://discuss.elastic.co/t/harvester-buffer-size-how-to-determine-most-efficient-value/303037 "2022-04-22T13:02:21Z")
The documentation says what the harvest\_buffer\_size setting of the log input is for, but nothing about why one might want to change it. Log input | Filebeat Reference \[7.17\] | Elastic I asked about this in another post … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=963) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=965)