# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=713 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 714 --- ## [CloudWatch Integration pulling logs from multiple log groups](https://discuss.elastic.co/t/cloudwatch-integration-pulling-logs-from-multiple-log-groups/320288)
**Author:** [@salt\_ninja](https://discuss.elastic.co/u/salt_ninja)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 7:26pm UTC](https://discuss.elastic.co/t/cloudwatch-integration-pulling-logs-from-multiple-log-groups/320288 "2022-12-01T19:26:38Z")
Hi team, I'm trying to use the CloudWatch Integration to pull logs from AWS CloudWatch to Elastic. However, I have only managed to pull logs from a single log group instead of multiple log groups. I have tried to limit… --- ## [Logstash is not generating logs after I include the cmd under container](https://discuss.elastic.co/t/logstash-is-not-generating-logs-after-i-include-the-cmd-under-container/320271)
**Author:** [@rravitech](https://discuss.elastic.co/u/rravitech)\ **Replies:** 1\ **Last updated:** [December 1, 2022, 6:47pm UTC](https://discuss.elastic.co/t/logstash-is-not-generating-logs-after-i-include-the-cmd-under-container/320271 "2022-12-01T18:47:22Z")
How can print the logs in logstash, once after I add command and args in container. containers: - name: logstash env: - name: LOGSTASH\_PW image: docker.elastic.co/logstash/logstas… --- ## [Issue: Logstash with Opensearch get stucked after the pipeline start](https://discuss.elastic.co/t/issue-logstash-with-opensearch-get-stucked-after-the-pipeline-start/320282)
**Author:** [@elCoder](https://discuss.elastic.co/u/elCoder)\ **Replies:** 1\ **Last updated:** [December 1, 2022, 6:37pm UTC](https://discuss.elastic.co/t/issue-logstash-with-opensearch-get-stucked-after-the-pipeline-start/320282 "2022-12-01T18:37:21Z")
Hi Everyone, I hope you could help me. I need to set up Logstash reading as input from an AWS S3 bucket and sending as output to OpenSearch. The input is a folder containing many csv. The issue is that when Logstash is … --- ## [What can be the query for this statement in Elasticsearch DSL](https://discuss.elastic.co/t/what-can-be-the-query-for-this-statement-in-elasticsearch-dsl/320275)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 2\ **Last updated:** [December 1, 2022, 5:51pm UTC](https://discuss.elastic.co/t/what-can-be-the-query-for-this-statement-in-elasticsearch-dsl/320275 "2022-12-01T17:51:41Z")
I am trying to have this query in elasticsearch statement: (ip=xx.xx.xx.xx and status="running") or (ip=xx.xx.xx.xx and status="running") can anyone tell me how can i write this in elasticsearch. --- ## [Dell iDrac Syslog Grok Logstash](https://discuss.elastic.co/t/dell-idrac-syslog-grok-logstash/320198)
**Author:** [@rcraigncs](https://discuss.elastic.co/u/rcraigncs)\ **Replies:** 2\ **Last updated:** [December 1, 2022, 5:32pm UTC](https://discuss.elastic.co/t/dell-idrac-syslog-grok-logstash/320198 "2022-12-01T17:32:24Z")
I am looking for help to grok Syslog from Dell idrac I am having problems taking the grok below and making it work with a Logstash. Your help is deeply appreciated! Extracting additional fields from iDRAC logs I found o… --- ## [Distance query with grouping and sorting](https://discuss.elastic.co/t/distance-query-with-grouping-and-sorting/320272)
**Author:** [@dthomas](https://discuss.elastic.co/u/dthomas)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 4:44pm UTC](https://discuss.elastic.co/t/distance-query-with-grouping-and-sorting/320272 "2022-12-01T16:44:33Z")
I have a single index that represents the locations of buildings in the United States. Each buildings is owned by a company (hence the CompanyId property). The index looks something like this: (fake geo-coordinates). { … --- ## [Difference of aggregated value that grouped by two fields across time](https://discuss.elastic.co/t/difference-of-aggregated-value-that-grouped-by-two-fields-across-time/317906)
**Author:** [@amylyu](https://discuss.elastic.co/u/amylyu)\ **Replies:** 7\ **Last updated:** [November 3, 2022, 5:50pm UTC](https://discuss.elastic.co/t/difference-of-aggregated-value-that-grouped-by-two-fields-across-time/317906 "2022-11-03T17:50:25Z")
Hi, we are getting data into elastic through Kafka Prometheus endpoint and trying to create a visualization which would help us show latest ingestion rate in each kafka topics. In order to achieve this, we need to group … --- ## [Kiabana completing setup blocked](https://discuss.elastic.co/t/kiabana-completing-setup-blocked/319497)
**Author:** [@ardue](https://discuss.elastic.co/u/ardue)\ **Replies:** 2\ **Last updated:** [December 1, 2022, 3:33pm UTC](https://discuss.elastic.co/t/kiabana-completing-setup-blocked/319497 "2022-12-01T15:33:44Z")
Hello, I have a problem with kibana, when connecting elasticsearch on kibana, it remains blocked on "completing setup" Here is the error in the log Action failed with '\[index\_not\_green\_timeout\] Timeout waiting for t… --- ## [/internal/bsearch returned 405 - not allowed in Kibana Discover screen](https://discuss.elastic.co/t/internal-bsearch-returned-405-not-allowed-in-kibana-discover-screen/319366)
**Author:** [@david-n](https://discuss.elastic.co/u/david-n)\ **Replies:** 3\ **Last updated:** [December 1, 2022, 3:09pm UTC](https://discuss.elastic.co/t/internal-bsearch-returned-405-not-allowed-in-kibana-discover-screen/319366 "2022-12-01T15:09:03Z")
Hello everyone, I'm working on an application to analyze my spending. The idea is to store all bank account transactions in Elasticsearch and analyze them using Kibana. For now I have a local development environment bas… --- ## [Newly set up elastic agent does not gather anything](https://discuss.elastic.co/t/newly-set-up-elastic-agent-does-not-gather-anything/320257)
**Author:** [@SirMuffington](https://discuss.elastic.co/u/SirMuffington)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 2:41pm UTC](https://discuss.elastic.co/t/newly-set-up-elastic-agent-does-not-gather-anything/320257 "2022-12-01T14:41:56Z")
I newly setup my Elastic Agent with Fleet. I clicked thru the interface and it always can't read the logs, variables etc... Is this normal for a new setup or did I mess something up from the start? --- ## [Upgrade elasticsearch to 8.x](https://discuss.elastic.co/t/upgrade-elasticsearch-to-8-x/320250)
**Author:** [@tegerei](https://discuss.elastic.co/u/tegerei)\ **Replies:** 1\ **Last updated:** [December 1, 2022, 2:45pm UTC](https://discuss.elastic.co/t/upgrade-elasticsearch-to-8-x/320250 "2022-12-01T14:45:22Z")
Hi, I am planning to upgrade Elasticsearch from 7.10 to 8.x. However I am not sure what will happen to multiple endpoints sending logs to ES using filebeat 7.10. Will I also need to upgrade the filebeat instances? Than… --- ## [Elastic Agent - problems following the documentation, stumbled upon dead links and more](https://discuss.elastic.co/t/elastic-agent-problems-following-the-documentation-stumbled-upon-dead-links-and-more/319495)
**Author:** [@SirMuffington](https://discuss.elastic.co/u/SirMuffington)\ **Replies:** 13\ **Last updated:** [December 1, 2022, 2:26pm UTC](https://discuss.elastic.co/t/elastic-agent-problems-following-the-documentation-stumbled-upon-dead-links-and-more/319495 "2022-12-01T14:26:56Z")
In the documentation for setting up Elastic Agent the following is stated: Kibana user with All privileges on Fleet and Integrations. Since many Integrations assets are shared across spaces, users need the Kibana privi… --- ## [Logstash(6.5.4) elasticsearch output](https://discuss.elastic.co/t/logstash-6-5-4-elasticsearch-output/320219)
**Author:** [@gyrao\_72](https://discuss.elastic.co/u/gyrao_72)\ **Replies:** 5\ **Last updated:** [December 1, 2022, 1:10pm UTC](https://discuss.elastic.co/t/logstash-6-5-4-elasticsearch-output/320219 "2022-12-01T13:10:10Z")
My filebeat is on two servers. let's say app1 and app2 My logs are in the format jobID status data Example: app1 log 5hgsxyt3838 SCHEDULED data app2 log 5hgsxyt3838 COMPLETE data Here both these logs have the sam… --- ## [How to update a document using script? (es\_version=8.4.3, javaApi)](https://discuss.elastic.co/t/how-to-update-a-document-using-script-es-version-8-4-3-javaapi/320246)
**Author:** [@infinite-war](https://discuss.elastic.co/u/infinite-war)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 12:58pm UTC](https://discuss.elastic.co/t/how-to-update-a-document-using-script-es-version-8-4-3-javaapi/320246 "2022-12-01T12:58:44Z")
config elasticsearch 8.4.3 (javaApi) JDK 17 created index PUT testdb { "mappings": { "properties": { "name":{ "type":"text" }, "desc":{ "type":"text" }, "score":{ … --- ## [Trying to send logs to AWS CloudWatch via logstash output plugin](https://discuss.elastic.co/t/trying-to-send-logs-to-aws-cloudwatch-via-logstash-output-plugin/320238)
**Author:** [@Randika\_Madhushan](https://discuss.elastic.co/u/Randika_Madhushan)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 12:25pm UTC](https://discuss.elastic.co/t/trying-to-send-logs-to-aws-cloudwatch-via-logstash-output-plugin/320238 "2022-12-01T12:25:54Z")
Hello there, Currently, I'm trying to send the applications logs to AWS CloudWatch via logstash agent output plugin. So I tried the below configuration. But that did not work. Could you please help with this? input { … --- ## [Elasticsearch pods fail unexpectedly when deploying Elastic Stack through helm on minikube](https://discuss.elastic.co/t/elasticsearch-pods-fail-unexpectedly-when-deploying-elastic-stack-through-helm-on-minikube/320194)
**Author:** [@Jan\_Hein\_de\_Jong](https://discuss.elastic.co/u/Jan_Hein_de_Jong)\ **Replies:** 0\ **Last updated:** [November 30, 2022, 8:46pm UTC](https://discuss.elastic.co/t/elasticsearch-pods-fail-unexpectedly-when-deploying-elastic-stack-through-helm-on-minikube/320194 "2022-11-30T20:46:30Z")
Hi there, I'm trying to deploy ELK stack to a local minikube k8s cluster, to play around, and see if this is something we want to roll out at our company (which uses an on-prem hosted k8s cluster). I've executed the fo… --- ## [Kibana starting with fatal error no such file](https://discuss.elastic.co/t/kibana-starting-with-fatal-error-no-such-file/319983)
**Author:** [@steves](https://discuss.elastic.co/u/steves)\ **Replies:** 10\ **Last updated:** [December 1, 2022, 10:33am UTC](https://discuss.elastic.co/t/kibana-starting-with-fatal-error-no-such-file/319983 "2022-12-01T10:33:11Z")
I just installed Kibana after getting Elasticsearch running (secure), both installed with apt, on an Ubuntu 22.04 Server. I start Kibana with systemctl start kibana.service or for debugging ./bin/kibana which results … --- ## [Data stream resilience](https://discuss.elastic.co/t/data-stream-resilience/320166)
**Author:** [@lukaszst](https://discuss.elastic.co/u/lukaszst)\ **Replies:** 6\ **Last updated:** [December 1, 2022, 9:58am UTC](https://discuss.elastic.co/t/data-stream-resilience/320166 "2022-12-01T09:58:23Z")
Hello I'm migrating indices-\>datastreams in relatively large elastic cluster (15 data nodes, 80k docs per second). For our current indices-based setup we used to monitor if currently used index get red to create new in… --- ## [What is \_recovery\_source field?](https://discuss.elastic.co/t/what-is-recovery-source-field/320108)
**Author:** [@ruslaniv](https://discuss.elastic.co/u/ruslaniv)\ **Replies:** 2\ **Last updated:** [December 1, 2022, 9:54am UTC](https://discuss.elastic.co/t/what-is-recovery-source-field/320108 "2022-12-01T09:54:15Z")
I have initiated indexing operation via python client that was abnormally terminated. When I ran curl --location --request POST 'http://127.0.0.1:9200/index\_name.vec/\_disk\_usage?run\_expensive\_tasks=true to analyze the … --- ## [How to exclude dense\_vector field from being stored](https://discuss.elastic.co/t/how-to-exclude-dense-vector-field-from-being-stored/320034)
**Author:** [@ruslaniv](https://discuss.elastic.co/u/ruslaniv)\ **Replies:** 6\ **Last updated:** [December 1, 2022, 8:20am UTC](https://discuss.elastic.co/t/how-to-exclude-dense-vector-field-from-being-stored/320034 "2022-12-01T08:20:35Z")
How can I exclude dense\_vector field from being stored in the \_source? I ran an experiment indexing approximately 6\_000\_000 documents and here is what I found out after running curl --location --request POST 'http://12… --- ## [I have added a process in metricbeat.yml file to monitor but the process details are not showing in kibana](https://discuss.elastic.co/t/i-have-added-a-process-in-metricbeat-yml-file-to-monitor-but-the-process-details-are-not-showing-in-kibana/320227)
**Author:** [@sourabh\_rawat](https://discuss.elastic.co/u/sourabh_rawat)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 8:51am UTC](https://discuss.elastic.co/t/i-have-added-a-process-in-metricbeat-yml-file-to-monitor-but-the-process-details-are-not-showing-in-kibana/320227 "2022-12-01T08:51:10Z")
I have added a process in the metricbeat.yml file to monitor but the process details are not showing in kibana dashboard. I am getting the process details in the agent logs but when I try to get the process details usin… --- ## [Prevent thai tokenizer from tokenizing hashtag](https://discuss.elastic.co/t/prevent-thai-tokenizer-from-tokenizing-hashtag/320225)
**Author:** [@TFP](https://discuss.elastic.co/u/TFP)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 8:19am UTC](https://discuss.elastic.co/t/prevent-thai-tokenizer-from-tokenizing-hashtag/320225 "2022-12-01T08:19:03Z")
I'm currently using the thai tokenizer and wanted to preserve hashtag words but the tokenizer keeps on removing the hashtag symbol. ES version: 7.16.2 GET /\_analyze { "tokenizer": "thai", "text": "#รายการพ #hashtag" … --- ## [Overwriting ILM policy is disabled. Set \`setup.ilm.overwrite: true\` for enabling](https://discuss.elastic.co/t/overwriting-ilm-policy-is-disabled-set-setup-ilm-overwrite-true-for-enabling/320217)
**Author:** [@Umer\_Tahir](https://discuss.elastic.co/u/Umer_Tahir)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 7:23am UTC](https://discuss.elastic.co/t/overwriting-ilm-policy-is-disabled-set-setup-ilm-overwrite-true-for-enabling/320217 "2022-12-01T07:23:50Z")
I am using metric beat Please let me know where can I Set setup.ilm.overwrite: true for enabling. I have already searched in metricbeat.yml but still no clue I have found. --- ## [error\_class=Fluent::Plugin::ElasticsearchErrorHandler::ElasticsearchError error="400 - Rejected by Elasticsearch](https://discuss.elastic.co/t/error-class-fluent-elasticsearcherror-error-400-rejected-by-elasticsearch/320215)
**Author:** [@Somasuntaram\_MU](https://discuss.elastic.co/u/Somasuntaram_MU)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 7:03am UTC](https://discuss.elastic.co/t/error-class-fluent-elasticsearcherror-error-400-rejected-by-elasticsearch/320215 "2022-12-01T07:03:11Z")
I am setting up EFK stack using docker-compose. Fluentd is able to receive the logs from http-endpoint. But the logs are getting rejected by elasticsearch. Though I am able to receive the error as bad argument and reje… --- ## [Filebeat 8.5 pattern not applying to index + the Indices shows as .ds-\?](https://discuss.elastic.co/t/filebeat-8-5-pattern-not-applying-to-index-the-indices-shows-as-ds-name/320220)
**Author:** [@amir\_Bialek](https://discuss.elastic.co/u/amir_Bialek)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 7:46am UTC](https://discuss.elastic.co/t/filebeat-8-5-pattern-not-applying-to-index-the-indices-shows-as-ds-name/320220 "2022-12-01T07:46:19Z")
Hello all, I am using the new Filebeat Helm (8.50 to collect logs from several microservices and send them directly to elasticsearch (EFK). I've edited the Values.yaml under daemonset with the following: filebeatCon… --- ## [Query\_string search within nested types](https://discuss.elastic.co/t/query-string-search-within-nested-types/320218)
**Author:** [@vlasami](https://discuss.elastic.co/u/vlasami)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 7:26am UTC](https://discuss.elastic.co/t/query-string-search-within-nested-types/320218 "2022-12-01T07:26:47Z")
Hi, We recently introduced nested type to our document structure and found that it has some restriction with the query\_string search type, which we use extensively. The search documentation says that: "Avoid using th… --- ## [Bootstrap check failed max file descriptors \[16384\] for elasticsearch process is too low, increase to at least \[65535\]](https://discuss.elastic.co/t/bootstrap-check-failed-max-file-descriptors-16384-for-elasticsearch-process-is-too-low-increase-to-at-least-65535/320140)
**Author:** [@Dhakarlabh](https://discuss.elastic.co/u/Dhakarlabh)\ **Replies:** 2\ **Last updated:** [December 1, 2022, 6:12am UTC](https://discuss.elastic.co/t/bootstrap-check-failed-max-file-descriptors-16384-for-elasticsearch-process-is-too-low-increase-to-at-least-65535/320140 "2022-12-01T06:12:59Z")
While deploying from the helm chart on the dual stack cluster. I am facing this bootstrap check failed issue. It is working fine for single stack IPv4 cluster. I tried to configure the network.host="global:ipv6" but it i… --- ## [Include Data Streams in ILM Policy](https://discuss.elastic.co/t/include-data-streams-in-ilm-policy/319404)
**Author:** [@waleedkh](https://discuss.elastic.co/u/waleedkh)\ **Replies:** 5\ **Last updated:** [December 1, 2022, 4:28am UTC](https://discuss.elastic.co/t/include-data-streams-in-ilm-policy/319404 "2022-12-01T04:28:15Z")
Hi There, I am making use of Data Streams and an ILM Policy to delete data that is rolled over from Hot phase to delete phase, this happens after 3 days. I wanted to check if there is a way to include the actual deleti… --- ## [Would like to arrange data histogram COUNT values in descending order,Vertical bar](https://discuss.elastic.co/t/would-like-to-arrange-data-histogram-count-values-in-descending-order-vertical-bar/318072)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 1\ **Last updated:** [November 3, 2022, 12:06pm UTC](https://discuss.elastic.co/t/would-like-to-arrange-data-histogram-count-values-in-descending-order-vertical-bar/318072 "2022-11-03T12:06:15Z")
Hello All, Is it possible to arrange data histogram COUNT values in descending order(big to small). Simply need to display highestest count first for given product type on given date. I dont see any option available t… --- ## [Does nodequerycache need to be configured to be created?](https://discuss.elastic.co/t/does-nodequerycache-need-to-be-configured-to-be-created/320201)
**Author:** [@winteriscomming](https://discuss.elastic.co/u/winteriscomming)\ **Replies:** 1\ **Last updated:** [December 1, 2022, 1:30am UTC](https://discuss.elastic.co/t/does-nodequerycache-need-to-be-configured-to-be-created/320201 "2022-12-01T01:30:57Z")
A 404 not found error occurs when querying the nodequerycache index with cat api. According to the docs, it is said that node query cache can be set in elasticsearch.yml, but since the default value is enabled, I am con… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=712) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=714)