# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1759 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1760 --- ## [Assistance with my Logstash filter](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342)
**Author:** [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Replies:** 9\ **Last updated:** [October 9, 2020, 1:21pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342 "2020-10-09T13:21:03Z")
Hello, I am trying to use a variation of the filter provided here - https://github.com/bromiley/olaf/blob/master/logstash/o365.config but I keep getting the following error, \[FATAL\]\[logstash.runner \] The given… --- ## [Log4j2.properties](https://discuss.elastic.co/t/log4j2-properties/251568)
**Author:** [@abhinav\_elk](https://discuss.elastic.co/u/abhinav_elk)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 1:08pm UTC](https://discuss.elastic.co/t/log4j2-properties/251568 "2020-10-09T13:08:25Z")
Can someone please explain theses two things in the Log4j.2 properties file in ElasticSearch \`appender.index\_search\_slowlog\_rolling.policies.time.type=TimeBasedTriggeringPolicy appender.index\_search\_slowlog\_rolling.pol… --- ## [How to calculate Request Rate](https://discuss.elastic.co/t/how-to-calculate-request-rate/251571)
**Author:** [@Pooja\_Nair](https://discuss.elastic.co/u/Pooja_Nair)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 1:13pm UTC](https://discuss.elastic.co/t/how-to-calculate-request-rate/251571 "2020-10-09T13:13:21Z")
Hello everyone, I am trying to setup monitoring for elasticsearch cluster using an external monitoring tool called Logic Monitor. I am trying to replicate a dashboard similar to Kibana, would like to know how to calcula… --- ## [No logstash data showing up in kibana](https://discuss.elastic.co/t/no-logstash-data-showing-up-in-kibana/251471)
**Author:** [@jimmyacuna](https://discuss.elastic.co/u/jimmyacuna)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 1:02pm UTC](https://discuss.elastic.co/t/no-logstash-data-showing-up-in-kibana/251471 "2020-10-09T13:02:19Z")
I have just installed a new ELK stack instance but I cant figure out what I am missing. By default filebeats were being sent and that worked. I dont intend to use filebeats as since Im mainly collecting syslogs from netw… --- ## [Elasticsearch Rolling Restart](https://discuss.elastic.co/t/elasticsearch-rolling-restart/251566)
**Author:** [@elasticus3r](https://discuss.elastic.co/u/elasticus3r)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 1:01pm UTC](https://discuss.elastic.co/t/elasticsearch-rolling-restart/251566 "2020-10-09T13:01:45Z")
Hello, I need to perform a rolling restart of my elasticsearch cluster which contains 3 master nodes and 7 data nodes. I believe a rolling restart is the best approach for patching and also cert renewals. From this docu… --- ## [Non writable elasticsearch.keystore path - Elasticsearch 6.8](https://discuss.elastic.co/t/non-writable-elasticsearch-keystore-path-elasticsearch-6-8/251563)
**Author:** [@BTh0mas](https://discuss.elastic.co/u/BTh0mas)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 12:50pm UTC](https://discuss.elastic.co/t/non-writable-elasticsearch-keystore-path-elasticsearch-6-8/251563 "2020-10-09T12:50:15Z")
Hi I am trying to create a custom docker container and use it in a Kubernetes cluster. I would like my build and deployment scripts to create custom Elasticsearch configuration file and mount them into the docker contai… --- ## [Datafeed query for Machine Learning, Got more than 10000 patitions](https://discuss.elastic.co/t/datafeed-query-for-machine-learning-got-more-than-10000-patitions/251504)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [October 9, 2020, 12:44pm UTC](https://discuss.elastic.co/t/datafeed-query-for-machine-learning-got-more-than-10000-patitions/251504 "2020-10-09T12:44:09Z")
Hi, Im currently using this query in the datafeed of my jobs in ML: { "bool": { "filter": \[ { "bool": { "should": \[ { "exists": { "field": "cpu\_per… --- ## [Problem with decode\_json\_fields](https://discuss.elastic.co/t/problem-with-decode-json-fields/251399)
**Author:** [@Kay\_Khan](https://discuss.elastic.co/u/Kay_Khan)\ **Replies:** 4\ **Last updated:** [October 9, 2020, 12:42pm UTC](https://discuss.elastic.co/t/problem-with-decode-json-fields/251399 "2020-10-09T12:42:42Z")
I have ECK setup and im using filebeat to ship logs from Kubernetes to elasticsearch. Ive recently added decode\_json\_fields processor to my configuration, so that im able decode the json that is usually in the message … --- ## [Stdout rubydebug not outputting to file](https://discuss.elastic.co/t/stdout-rubydebug-not-outputting-to-file/251296)
**Author:** [@elasticus3r](https://discuss.elastic.co/u/elasticus3r)\ **Replies:** 11\ **Last updated:** [October 9, 2020, 12:10pm UTC](https://discuss.elastic.co/t/stdout-rubydebug-not-outputting-to-file/251296 "2020-10-09T12:10:48Z")
Hello, I have got an nginx output with the stdout configured at the bottom as: output { if \[service\] == "nginx" and "\_grokparsefailure" in \[tags\] { elasticsearch { user =\> "user" password =\> "password" … --- ## [Automatic snapshot delete on Elasticsearch 2.4](https://discuss.elastic.co/t/automatic-snapshot-delete-on-elasticsearch-2-4/251273)
**Author:** [@Silas-91](https://discuss.elastic.co/u/Silas-91)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 11:42am UTC](https://discuss.elastic.co/t/automatic-snapshot-delete-on-elasticsearch-2-4/251273 "2020-10-09T11:42:45Z")
Hello together, i´am new to elasticsearch and i have some question about version 2.4. We take snapshots every day but now there are so many that we have to delete them. I don´t want to delete them one by one. So does a… --- ## [Re-Index to new date format. getting parse\_exception](https://discuss.elastic.co/t/re-index-to-new-date-format-getting-parse-exception/251390)
**Author:** [@ahmadsayeed3](https://discuss.elastic.co/u/ahmadsayeed3)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 7:51am UTC](https://discuss.elastic.co/t/re-index-to-new-date-format-getting-parse-exception/251390 "2020-10-08T07:51:09Z")
POST \_reindex { "source": { "index": "source" }, "dest": { "index": "dest" }, "script": { "source": """ SimpleDateFormat format = new SimpleDateFormat(… --- ## [Delay to initialize elasticsearch](https://discuss.elastic.co/t/delay-to-initialize-elasticsearch/251558)
**Author:** [@Douglas\_Ribeiro](https://discuss.elastic.co/u/Douglas_Ribeiro)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 11:38am UTC](https://discuss.elastic.co/t/delay-to-initialize-elasticsearch/251558 "2020-10-09T11:38:29Z")
Hi guys, next I use the SKELS ISO, one of the services I need is ElasticSearch, but he has a problem with Boot. Before it didn't start, now I changed the timeout time to 3 minutes, now it starts, but it takes 2 minutes. … --- ## [Сopies of shards](https://discuss.elastic.co/t/opies-of-shards/248857)
**Author:** [@djuwa4](https://discuss.elastic.co/u/djuwa4)\ **Replies:** 4\ **Last updated:** [October 9, 2020, 11:31am UTC](https://discuss.elastic.co/t/opies-of-shards/248857 "2020-10-09T11:31:29Z")
Good day! I have an elasticsearch cluster configured. Can you please tell me if copies of shards are created by default? Or does it need to be configured separately? --- ## [How to support pagination in aggregation](https://discuss.elastic.co/t/how-to-support-pagination-in-aggregation/251413)
**Author:** [@Raj3](https://discuss.elastic.co/u/Raj3)\ **Replies:** 5\ **Last updated:** [October 9, 2020, 11:16am UTC](https://discuss.elastic.co/t/how-to-support-pagination-in-aggregation/251413 "2020-10-09T11:16:09Z")
{ "aggs": { "thread": { "terms": { "field": "session\_id" } } } } i want to add pagination in field session\_id how can it? --- ## [Setting up replica shards](https://discuss.elastic.co/t/setting-up-replica-shards/251551)
**Author:** [@djuwa4](https://discuss.elastic.co/u/djuwa4)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 10:55am UTC](https://discuss.elastic.co/t/setting-up-replica-shards/251551 "2020-10-09T10:55:13Z")
How can I increase the number of shard replicas? I have five nodes in the cluster, I want each node to have a copy of each shards. I found this option on the net, is it correct? curl -X PUT http://localhost:9200/\_templ… --- ## [Can we use scripted field in kibana in term aggregation?](https://discuss.elastic.co/t/can-we-use-scripted-field-in-kibana-in-term-aggregation/251523)
**Author:** [@le\_ba\_nam](https://discuss.elastic.co/u/le_ba_nam)\ **Replies:** 1\ **Last updated:** [October 9, 2020, 9:27am UTC](https://discuss.elastic.co/t/can-we-use-scripted-field-in-kibana-in-term-aggregation/251523 "2020-10-09T09:27:23Z")
I created scripted field in kibana named service\_exception.keyword. Now I am trying to create a watcher, which aims to catch the most occurrences exception service. But it not work. I tried replace it by another field an… --- ## [Unable to run Logstash 7.9.2 as a service on Ubuntu 18.04](https://discuss.elastic.co/t/unable-to-run-logstash-7-9-2-as-a-service-on-ubuntu-18-04/251005)
**Author:** [@Philip\_Colmer](https://discuss.elastic.co/u/Philip_Colmer)\ **Replies:** 15\ **Last updated:** [October 9, 2020, 7:06am UTC](https://discuss.elastic.co/t/unable-to-run-logstash-7-9-2-as-a-service-on-ubuntu-18-04/251005 "2020-10-09T07:06:44Z")
I'm getting a warning and an error when I try to start logstash with systemctl start logstash: \[2020-10-05T14:01:19,067\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or com… --- ## [Why cant i get k8s event?](https://discuss.elastic.co/t/why-cant-i-get-k8s-event/250320)
**Author:** [@HarveyYang](https://discuss.elastic.co/u/HarveyYang)\ **Replies:** 7\ **Last updated:** [October 9, 2020, 7:04am UTC](https://discuss.elastic.co/t/why-cant-i-get-k8s-event/250320 "2020-10-09T07:04:17Z")
here is my metricbeat yaml: --- apiVersion: v1 kind: ConfigMap metadata: name: metricbeat-daemonset-config namespace: kube-system labels: k8s-app: metricbeat data: metricbeat.yml: |- metricbeat.config.mo… --- ## [How does Re-index API works?](https://discuss.elastic.co/t/how-does-re-index-api-works/251293)
**Author:** [@srbhklkrn](https://discuss.elastic.co/u/srbhklkrn)\ **Replies:** 3\ **Last updated:** [October 9, 2020, 6:30am UTC](https://discuss.elastic.co/t/how-does-re-index-api-works/251293 "2020-10-09T06:30:25Z")
We are in the process of migrating from ES 6.4 to 7.9, and around 3 TB of data. I'm looking into reindex API to start the process and do testing. My question is does reindex API will delete the original indexes? for now,… --- ## [Token Service could not get token document](https://discuss.elastic.co/t/token-service-could-not-get-token-document/251533)
**Author:** [@nitin7](https://discuss.elastic.co/u/nitin7)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 6:23am UTC](https://discuss.elastic.co/t/token-service-could-not-get-token-document/251533 "2020-10-09T06:23:15Z")
Hi all, In the cluster there has been info message \[2020-10-09T11:49:34,157\]\[INFO \]\[o.e.x.s.a.TokenService \] \[\] could not get token document \[token\_ChHbqugJxSEN47zQ-DgsvlzZXyPdPly1yRA\_VnPasUM\] that should have been c… --- ## [Logstash Comparing Field Values Using an If Statement not working](https://discuss.elastic.co/t/logstash-comparing-field-values-using-an-if-statement-not-working/251452)
**Author:** [@bardie](https://discuss.elastic.co/u/bardie)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 6:12am UTC](https://discuss.elastic.co/t/logstash-comparing-field-values-using-an-if-statement-not-working/251452 "2020-10-09T06:12:48Z")
Hi, I am trying to compare two fields using the following method but it is not working. I have used it before using an older version of logstash and it worked but it is not working on logstash 7.9.2. Logstash cannot be… --- ## [Using gsub for multi line patterns](https://discuss.elastic.co/t/using-gsub-for-multi-line-patterns/251458)
**Author:** [@Surya\_Iriventi](https://discuss.elastic.co/u/Surya_Iriventi)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 5:26am UTC](https://discuss.elastic.co/t/using-gsub-for-multi-line-patterns/251458 "2020-10-09T05:26:38Z")
Hi, I am trying to use gsub to remove the prefix for the log line. I have a filebeat configuration which matches multiline conguration. The multi line log message (input to logstash) would look line this: \*\*\*\* Error … --- ## [How to enable proxy for google\_pubsub input plugin in logstash](https://discuss.elastic.co/t/how-to-enable-proxy-for-google-pubsub-input-plugin-in-logstash/251525)
**Author:** [@nareshahi](https://discuss.elastic.co/u/nareshahi)\ **Replies:** 0\ **Last updated:** [October 9, 2020, 5:12am UTC](https://discuss.elastic.co/t/how-to-enable-proxy-for-google-pubsub-input-plugin-in-logstash/251525 "2020-10-09T05:12:17Z")
Hello Guys, i have enabled google\_pubsub plugin in logstash. it is up and runing. look like pipeline is looking for proxy to make API call with google cloud. please guide where have to set proxy setting for plugin . R… --- ## [String with spaces not is not searchable in logstash](https://discuss.elastic.co/t/string-with-spaces-not-is-not-searchable-in-logstash/251225)
**Author:** [@NARHAR\_DEV\_SHARMA](https://discuss.elastic.co/u/NARHAR_DEV_SHARMA)\ **Replies:** 4\ **Last updated:** [October 9, 2020, 4:51am UTC](https://discuss.elastic.co/t/string-with-spaces-not-is-not-searchable-in-logstash/251225 "2020-10-09T04:51:31Z")
Hi Team, Greeting for the day! I am trying to search a pattern that contain spaces and its not working. However, if i put something without space that's working absolutely fine; i am able to send mail to required addr… --- ## [Kibana plugin helpers error](https://discuss.elastic.co/t/kibana-plugin-helpers-error/250923)
**Author:** [@K\_Chaitanya](https://discuss.elastic.co/u/K_Chaitanya)\ **Replies:** 3\ **Last updated:** [October 9, 2020, 4:39am UTC](https://discuss.elastic.co/t/kibana-plugin-helpers-error/250923 "2020-10-09T04:39:14Z")
Hi, I developed a kibana plugin for kibana version 7.9.0. I setup the development environment following this. Now I want to generate the zip of the developed plugin. I tried follow the steps mentioned here but while ru… --- ## [A filter does not work in Controls visualization](https://discuss.elastic.co/t/a-filter-does-not-work-in-controls-visualization/248894)
**Author:** [@m-amano](https://discuss.elastic.co/u/m-amano)\ **Replies:** 2\ **Last updated:** [October 9, 2020, 2:56am UTC](https://discuss.elastic.co/t/a-filter-does-not-work-in-controls-visualization/248894 "2020-10-09T02:56:58Z")
I'm working on to make an options list in Controls with a filtered index by adding the filter at the upper left so that I can hide some selections. However the filtered objects are still shown in the list and I found so… --- ## [How to exclude other namespaces?](https://discuss.elastic.co/t/how-to-exclude-other-namespaces/246544)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 23\ **Last updated:** [October 9, 2020, 2:39am UTC](https://discuss.elastic.co/t/how-to-exclude-other-namespaces/246544 "2020-10-09T02:39:05Z")
filebeat.autodiscover: providers: - type: kubernetes node: ${NODE\_NAME} hints.enabled: false hints.default\_config: type: container finished: true paths: - "/var/… --- ## [Metricbeat + AWS Module: runtime error: index out of range \[0\] with length 0](https://discuss.elastic.co/t/metricbeat-aws-module-runtime-error-index-out-of-range-0-with-length-0/248592)
**Author:** [@dudumiquim](https://discuss.elastic.co/u/dudumiquim)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 11:49pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-runtime-error-index-out-of-range-0-with-length-0/248592 "2020-10-08T23:49:16Z")
I'm trying to configure the AWS module but when I try to start the metricbeat, I receive this error: FATAL \[metricbeat\] instance/beat.go:164 Failed due to panic This error are not "too clear" and I started my metricb… --- ## [Logstash count insta Filter aggregate](https://discuss.elastic.co/t/logstash-count-insta-filter-aggregate/251512)
**Author:** [@bbwolf](https://discuss.elastic.co/u/bbwolf)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 11:06pm UTC](https://discuss.elastic.co/t/logstash-count-insta-filter-aggregate/251512 "2020-10-08T23:06:49Z")
Hello, I would to know the best way to have count\_event as aggregate for task\_id equal to field1. I can create count as new event but notre like in the following format. Field1, field2, field3, field4, count\_event A,… --- ## [LOGSTASH: How to setup logstash to get all query's data not only a part of it?](https://discuss.elastic.co/t/logstash-how-to-setup-logstash-to-get-all-querys-data-not-only-a-part-of-it/251507)
**Author:** [@I\_Hathout](https://discuss.elastic.co/u/I_Hathout)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 10:29pm UTC](https://discuss.elastic.co/t/logstash-how-to-setup-logstash-to-get-all-querys-data-not-only-a-part-of-it/251507 "2020-10-08T22:29:03Z")
I've a database table which contains 400+ records, but each time, I use logstash and JDBC it only gets 126 records. I tried many times with different indices each time, but still the 126 only retrieving. Here is the out… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1758) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1760)