# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1099 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1100 --- ## [Using S3 as storage for elasticsearch](https://discuss.elastic.co/t/using-s3-as-storage-for-elasticsearch/294007)
**Author:** [@Yang\_Yang2](https://discuss.elastic.co/u/Yang_Yang2)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 10:35am UTC](https://discuss.elastic.co/t/using-s3-as-storage-for-elasticsearch/294007 "2022-01-11T10:35:26Z")
Is it possible to use the S3 as a data storage for Elasticsearch? --- ## [Deprecation logs not being logged to file in Elasticsearch 7.15](https://discuss.elastic.co/t/deprecation-logs-not-being-logged-to-file-in-elasticsearch-7-15/294016)
**Author:** [@Suraj\_Kumar4](https://discuss.elastic.co/u/Suraj_Kumar4)\ **Replies:** 0\ **Last updated:** [January 11, 2022, 10:18am UTC](https://discuss.elastic.co/t/deprecation-logs-not-being-logged-to-file-in-elasticsearch-7-15/294016 "2022-01-11T10:18:24Z")
Snippet of the log4j2.properties file being used with ES 7.15.2 status = error appender.console.type = Console appender.console.name = console appender.console.layout.type = PatternLayout appender.console.layout.patter… --- ## [Specify Mappings while creating index using python client](https://discuss.elastic.co/t/specify-mappings-while-creating-index-using-python-client/292433)
**Author:** [@sunny3794.ad](https://discuss.elastic.co/u/sunny3794.ad)\ **Replies:** 3\ **Last updated:** [January 11, 2022, 10:18am UTC](https://discuss.elastic.co/t/specify-mappings-while-creating-index-using-python-client/292433 "2022-01-11T10:18:09Z")
As far as I know, Elasticsearch by default takes in data and stores it as string (except for date). So, I have been using the body parameter in the create function of the Elasticsearch python client to specify the field … --- ## [Logstasch pipeline filter](https://discuss.elastic.co/t/logstasch-pipeline-filter/294009)
**Author:** [@max\_cyril](https://discuss.elastic.co/u/max_cyril)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 10:13am UTC](https://discuss.elastic.co/t/logstasch-pipeline-filter/294009 "2022-01-11T10:13:14Z")
Hi, I am new to ELK and struggling to write my first logstash pipeline. Can anyone help me to write the filter section? Thanks in advance. i want to filter and only output the maximum of completion per users fo… --- ## [Max score multi match query](https://discuss.elastic.co/t/max-score-multi-match-query/294014)
**Author:** [@Divy\_Garg](https://discuss.elastic.co/u/Divy_Garg)\ **Replies:** 0\ **Last updated:** [January 11, 2022, 10:08am UTC](https://discuss.elastic.co/t/max-score-multi-match-query/294014 "2022-01-11T10:08:16Z")
Hi, I am using multi match query of Elasticsearch. Need to understand what is the maximum \_score that multi match query can have. Please help with this. { "query": { "multi\_match" : { "query": "john wi… --- ## [Elastic service stopped automatically](https://discuss.elastic.co/t/elastic-service-stopped-automatically/293252)
**Author:** [@sandesh0209](https://discuss.elastic.co/u/sandesh0209)\ **Replies:** 4\ **Last updated:** [January 11, 2022, 9:57am UTC](https://discuss.elastic.co/t/elastic-service-stopped-automatically/293252 "2022-01-11T09:57:58Z")
Faulting application name: Elasticsearch.exe, version: 7.10.1.0, time stamp: 0x5fcb0351 Faulting module name: KERNELBASE.dll, version: 10.0.14393.4770, time stamp: 0x6180883c Exception code: 0xe0434352 Fault offset: 0… --- ## [Filebeat not picking up log files](https://discuss.elastic.co/t/filebeat-not-picking-up-log-files/293910)
**Author:** [@aamir\_mustafa](https://discuss.elastic.co/u/aamir_mustafa)\ **Replies:** 3\ **Last updated:** [January 11, 2022, 9:30am UTC](https://discuss.elastic.co/t/filebeat-not-picking-up-log-files/293910 "2022-01-11T09:30:49Z")
Hello Team, I'm using Filebeat 7.10.2 to ship logs directly to Elasticsearch. But I'm facing an issue while giving input to filebeat. Filebeat only picks up one log file even when I'm specifying to read all files from i… --- ## [How to loop through array in Logstash?](https://discuss.elastic.co/t/how-to-loop-through-array-in-logstash/291747)
**Author:** [@rijo\_joseph](https://discuss.elastic.co/u/rijo_joseph)\ **Replies:** 4\ **Last updated:** [January 11, 2022, 9:15am UTC](https://discuss.elastic.co/t/how-to-loop-through-array-in-logstash/291747 "2022-01-11T09:15:10Z")
{ "modifiedTime": "2021-12-13T06:47:11.138Z", "task\_owner\_name": null, "contents": \[ { "eligible": false, "rawResponse": "454536251", "id": 868286622 }, { "eligible": false, … --- ## [Delete index pattern](https://discuss.elastic.co/t/delete-index-pattern/293987)
**Author:** [@sunny3794.ad](https://discuss.elastic.co/u/sunny3794.ad)\ **Replies:** 2\ **Last updated:** [January 11, 2022, 8:57am UTC](https://discuss.elastic.co/t/delete-index-pattern/293987 "2022-01-11T08:57:33Z")
I am trying to automate the deletion of index patterns whenever the index is deleted. For this I am using the requests module in Python to send the delete request to the Kibana API enpoint. However, the request fails whe… --- ## [Can't start logstash after installtion because java](https://discuss.elastic.co/t/cant-start-logstash-after-installtion-because-java/293993)
**Author:** [@aviad.co1](https://discuss.elastic.co/u/aviad.co1)\ **Replies:** 0\ **Last updated:** [January 11, 2022, 8:49am UTC](https://discuss.elastic.co/t/cant-start-logstash-after-installtion-because-java/293993 "2022-01-11T08:49:51Z")
I install this java version - java -version : openjdk version "11.0.13" 2021-10-19. OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.20.04). OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.20.04… --- ## [12K fields in the mapping](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340)
**Author:** [@orips](https://discuss.elastic.co/u/orips)\ **Replies:** 6\ **Last updated:** [January 11, 2022, 8:35am UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340 "2022-01-11T08:35:28Z")
Hi, We're approaching 12K fields in our mapping. Our cluster does not have many documents (it's about 1 billion, and it's doc one per user, so it's not growing very rapidly). I'd like to know: how much further can we g… --- ## [Auto Deletion of Old index](https://discuss.elastic.co/t/auto-deletion-of-old-index/293978)
**Author:** [@triptesh1212](https://discuss.elastic.co/u/triptesh1212)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 8:17am UTC](https://discuss.elastic.co/t/auto-deletion-of-old-index/293978 "2022-01-11T08:17:42Z")
Hi community, I am exploring the index lifecycle policy and auto deletion of an index. I have gone through the following link. I have understood that we can create index template with policy where we can setup paramete… --- ## [What are the permissions necessary to import data in data visualizer? v7.12](https://discuss.elastic.co/t/what-are-the-permissions-necessary-to-import-data-in-data-visualizer-v7-12/293901)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 7:58am UTC](https://discuss.elastic.co/t/what-are-the-permissions-necessary-to-import-data-in-data-visualizer-v7-12/293901 "2022-01-11T07:58:35Z")
Hi Im getting this import persmissions error when I try to upload a file onto elastic the user has his own space and has this privigeles: machine\_learning\_user, ingest\_admin, machine\_learning\_admin the custom rol ha… --- ## [Receiving data to the ELK database using logstash](https://discuss.elastic.co/t/receiving-data-to-the-elk-database-using-logstash/292547)
**Author:** [@San9](https://discuss.elastic.co/u/San9)\ **Replies:** 5\ **Last updated:** [January 11, 2022, 7:56am UTC](https://discuss.elastic.co/t/receiving-data-to-the-elk-database-using-logstash/292547 "2022-01-11T07:56:20Z")
I have a problem with accessing the ELK database and receiving a specific field with a token to form a further request to connect to the o365 api. I can't get data from this particular field. scheme, I connect to the EL… --- ## [How to check Microsoft 365 service health](https://discuss.elastic.co/t/how-to-check-microsoft-365-service-health/290089)
**Author:** [@San9](https://discuss.elastic.co/u/San9)\ **Replies:** 6\ **Last updated:** [January 11, 2022, 7:53am UTC](https://discuss.elastic.co/t/how-to-check-microsoft-365-service-health/290089 "2022-01-11T07:53:08Z")
Hello experts! I have a question - How to check Microsoft 365 service health? As far as I know, there is a module for auditing logs in the filebeat. But I would like to get the status of o365 services. Is there such a … --- ## [Rottating policy Elastic Agent logs](https://discuss.elastic.co/t/rottating-policy-elastic-agent-logs/293880)
**Author:** [@marti1](https://discuss.elastic.co/u/marti1)\ **Replies:** 2\ **Last updated:** [January 11, 2022, 7:24am UTC](https://discuss.elastic.co/t/rottating-policy-elastic-agent-logs/293880 "2022-01-11T07:24:05Z")
Hi, Elastic Agent creates a log for each integration (Ex: /Library/Elastic/Agent/data/elastic-agent-08e204/logs/default/metricbeat-json.log). Is it possible to define rotating policy (reaching specific size or defining … --- ## [Filebeat with GCP module failed to start](https://discuss.elastic.co/t/filebeat-with-gcp-module-failed-to-start/293977)
**Author:** [@NohimX](https://discuss.elastic.co/u/NohimX)\ **Replies:** 0\ **Last updated:** [January 11, 2022, 6:46am UTC](https://discuss.elastic.co/t/filebeat-with-gcp-module-failed-to-start/293977 "2022-01-11T06:46:02Z")
I tried to configure filebeat GCP module to get audit logs from Google Cloud, and it failed to start. Failed to start crawler: creating module reloader failed: error checking input configuration: no authentication crede… --- ## [Filebeat makes too many API calls choking and bringing down the Kubernetes cluster](https://discuss.elastic.co/t/filebeat-makes-too-many-api-calls-choking-and-bringing-down-the-kubernetes-cluster/293797)
**Author:** [@ayush\_mundra](https://discuss.elastic.co/u/ayush_mundra)\ **Replies:** 2\ **Last updated:** [January 11, 2022, 6:21am UTC](https://discuss.elastic.co/t/filebeat-makes-too-many-api-calls-choking-and-bringing-down-the-kubernetes-cluster/293797 "2022-01-11T06:21:30Z")
Hi, We have filebeat version: 7.12.0 running as a daemonset on all our production K8s clusters. Recently, we faced a catastrophic situation where our whole K8s cluster with around 300+ nodes was brought down by filebeat… --- ## [Use other query next to suggester](https://discuss.elastic.co/t/use-other-query-next-to-suggester/293921)
**Author:** [@Amir\_Faramarzi](https://discuss.elastic.co/u/Amir_Faramarzi)\ **Replies:** 2\ **Last updated:** [January 11, 2022, 5:13am UTC](https://discuss.elastic.co/t/use-other-query-next-to-suggester/293921 "2022-01-11T05:13:22Z")
How to use other query next to suggester that affects the score of the data obtained from suggester For example, I want to search for products that are active ?? --- ## [Grok Pattern not working](https://discuss.elastic.co/t/grok-pattern-not-working/293159)
**Author:** [@Pranav\_M](https://discuss.elastic.co/u/Pranav_M)\ **Replies:** 5\ **Last updated:** [January 11, 2022, 4:50am UTC](https://discuss.elastic.co/t/grok-pattern-not-working/293159 "2022-01-11T04:50:00Z")
Hey Community, Received packet: Type = N5abcde8AbcAbcde9AbcdAbcde15AbcdefghAbcdefA, xx = 95, xxx = 441, xxxXxo = 2, xXxxXxx = 1, xxxx = 9 accepted. The above line is a message from my log and I want to seperate each da… --- ## [Daily indices or monthly indices for storing data?](https://discuss.elastic.co/t/daily-indices-or-monthly-indices-for-storing-data/293967)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 0\ **Last updated:** [January 11, 2022, 4:37am UTC](https://discuss.elastic.co/t/daily-indices-or-monthly-indices-for-storing-data/293967 "2022-01-11T04:37:01Z")
Hi all I have a question regarding the storage comparision between having daily indices or monthly indices. The question is very simple: is the daily indices tooks more storage space than the monthly indices. Somehow i… --- ## [Latitude and Longitude Error in logstash?](https://discuss.elastic.co/t/latitude-and-longitude-error-in-logstash/293855)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 11\ **Last updated:** [January 11, 2022, 4:35am UTC](https://discuss.elastic.co/t/latitude-and-longitude-error-in-logstash/293855 "2022-01-11T04:35:23Z")
Hello Everyone, I am trying to convert PostalCode to lat and long, I have done this task but Now I am confuse how to convert it into geopoint field I have tried in logstash but no success so for, getting error. My con… --- ## [Reduce (shrink) primary shards on older data indices of a data stream](https://discuss.elastic.co/t/reduce-shrink-primary-shards-on-older-data-indices-of-a-data-stream/293963)
**Author:** [@adriaan](https://discuss.elastic.co/u/adriaan)\ **Replies:** 0\ **Last updated:** [January 11, 2022, 2:27am UTC](https://discuss.elastic.co/t/reduce-shrink-primary-shards-on-older-data-indices-of-a-data-stream/293963 "2022-01-11T02:27:57Z")
I have a data stream with page views of multiple websites of customers. I allow customers to delete their data (this is important). When they do I remove all the documents with a certain hostname. At the same time, I wa… --- ## [Heartbeat (7.15.2) - aws\_elb autodiscover only sends data for single listener while more should be included](https://discuss.elastic.co/t/heartbeat-7-15-2-aws-elb-autodiscover-only-sends-data-for-single-listener-while-more-should-be-included/293757)
**Author:** [@ronaldkonjer](https://discuss.elastic.co/u/ronaldkonjer)\ **Replies:** 3\ **Last updated:** [January 11, 2022, 2:58am UTC](https://discuss.elastic.co/t/heartbeat-7-15-2-aws-elb-autodiscover-only-sends-data-for-single-listener-while-more-should-be-included/293757 "2022-01-11T02:58:26Z")
I try to get heartbeat with aws\_elb autodiscovery to work. Though I'm able to get data visible in kibana, it is always a single listener from a single lb. Whereas I have 3 lb setup and it logs that it fetched 16 listener… --- ## [Why cant Select an Agent policy when Add a Fleet Server](https://discuss.elastic.co/t/why-cant-select-an-agent-policy-when-add-a-fleet-server/293083)
**Author:** [@wo7387196](https://discuss.elastic.co/u/wo7387196)\ **Replies:** 3\ **Last updated:** [January 11, 2022, 2:04am UTC](https://discuss.elastic.co/t/why-cant-select-an-agent-policy-when-add-a-fleet-server/293083 "2022-01-11T02:04:30Z")
the default is not work --- ## [Unable to access snapshot and restore](https://discuss.elastic.co/t/unable-to-access-snapshot-and-restore/293526)
**Author:** [@Apoorva\_21](https://discuss.elastic.co/u/Apoorva_21)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 12:42am UTC](https://discuss.elastic.co/t/unable-to-access-snapshot-and-restore/293526 "2022-01-11T00:42:47Z")
Hi, I am unable to access snapshots and restore it says error loading snapshots . And when trying to run GET \_cat/snapshots command getting the error as Validation Failed : 1: repository missing. Can anyone help me t… --- ## [ElasticSearch for Temporal (version, incident\_raised\_date, auto\_create\_index)](https://discuss.elastic.co/t/elasticsearch-for-temporal-version-incident-raised-date-auto-create-index/293638)
**Author:** [@ridwan.santoso](https://discuss.elastic.co/u/ridwan.santoso)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 12:41am UTC](https://discuss.elastic.co/t/elasticsearch-for-temporal-version-incident-raised-date-auto-create-index/293638 "2022-01-11T00:41:29Z")
Hi, I need to use Elasticsearch for Temporal and have couple question: Will Elasticsearch 7.6 works for Temporal 1.13? If not then what is recommended version? I am using Elasticsearch 7.6 for Temporal 1.13 currently … --- ## [Only able to see 100 index in Create index pattern](https://discuss.elastic.co/t/only-able-to-see-100-index-in-create-index-pattern/293854)
**Author:** [@faisalshah20](https://discuss.elastic.co/u/faisalshah20)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 12:39am UTC](https://discuss.elastic.co/t/only-able-to-see-100-index-in-create-index-pattern/293854 "2022-01-11T00:39:37Z")
Hi Team i am running into problem i am only able to see 100 index in create index pattern. My scenario is to keep last 1 months logs in ELK. But when the 100 index are in create index pattern now new index pattern gets c… --- ## [Painless script not work](https://discuss.elastic.co/t/painless-script-not-work/293957)
**Author:** [@Jongz\_Puangput](https://discuss.elastic.co/u/Jongz_Puangput)\ **Replies:** 0\ **Last updated:** [January 11, 2022, 12:15am UTC](https://discuss.elastic.co/t/painless-script-not-work/293957 "2022-01-11T00:15:05Z")
I try to add runtime field to the current index. Name: store\_hostname Value: if (doc\['message.keyword'\].size() == 0) return; def source = doc\['message.keyword'\].value; if (source != null) { Matcher matcher = /((th… --- ## [Illegal\_argument\_exception: index.lifecycle.rollover\_alias is empty or not defined](https://discuss.elastic.co/t/illegal-argument-exception-index-lifecycle-rollover-alias-is-empty-or-not-defined/293933)
**Author:** [@dev9](https://discuss.elastic.co/u/dev9)\ **Replies:** 0\ **Last updated:** [January 10, 2022, 6:33pm UTC](https://discuss.elastic.co/t/illegal-argument-exception-index-lifecycle-rollover-alias-is-empty-or-not-defined/293933 "2022-01-10T18:33:57Z")
Hi, I noticed this error last week on our elastic illegal\_argument\_exception: setting \[index.lifecycle.rollover\_alias\] for index \[test-2022.01.07\] is empty or not defined I did some research and found this article on t… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1098) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1100)