# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1349 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1350 --- ## [Elasticsearch not saving all fields from logstash](https://discuss.elastic.co/t/elasticsearch-not-saving-all-fields-from-logstash/276957)
**Author:** [@mlassnig\_no](https://discuss.elastic.co/u/mlassnig_no)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 7:50am UTC](https://discuss.elastic.co/t/elasticsearch-not-saving-all-fields-from-logstash/276957 "2021-06-25T07:50:14Z")
I got following situation: Im sending logs from my servers to logstash using filebeat. Im running a logstash pipeline that looks like this: input { beats { port =\> 5003 } } filter { grok { match =\> … --- ## [Elasticsearch deployment](https://discuss.elastic.co/t/elasticsearch-deployment/276700)
**Author:** [@porscheme](https://discuss.elastic.co/u/porscheme)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 7:27am UTC](https://discuss.elastic.co/t/elasticsearch-deployment/276700 "2021-06-25T07:27:46Z")
We wanted to build Elasticsearch on cloud Azure. Source data size: 50TB Replicas count: 3 Shard count: 1000 For doing this, we are considering below two VM SKUs. L32sv2 is very processing we wanted to know SLAs for… --- ## [Duplication while logs are appended in Filebeat](https://discuss.elastic.co/t/duplication-while-logs-are-appended-in-filebeat/277016)
**Author:** [@connectgeeks](https://discuss.elastic.co/u/connectgeeks)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 6:32am UTC](https://discuss.elastic.co/t/duplication-while-logs-are-appended-in-filebeat/277016 "2021-06-25T06:32:05Z")
I'm using Filebeat-\> Logstash-\> ES setup. 7.x While paring the apache and json logs using, I notice every time the file is appended with new logs I see old logs also getting re-created in elasticsearch. How can I avoid … --- ## [Count unique customers based on date range](https://discuss.elastic.co/t/count-unique-customers-based-on-date-range/277014)
**Author:** [@Sanjay\_Agnani1](https://discuss.elastic.co/u/Sanjay_Agnani1)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 5:55am UTC](https://discuss.elastic.co/t/count-unique-customers-based-on-date-range/277014 "2021-06-25T05:55:53Z")
Need the stats in unique customer: index contains customers orders data. Ex: { "\_index" : "customers\_orders", "\_type" : "\_doc", "\_id" : "109126", "\_score" : 1.0, "\_source" : … --- ## [How can I start logstash from a script](https://discuss.elastic.co/t/how-can-i-start-logstash-from-a-script/277010)
**Author:** [@Mbartlett413](https://discuss.elastic.co/u/Mbartlett413)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 5:30am UTC](https://discuss.elastic.co/t/how-can-i-start-logstash-from-a-script/277010 "2021-06-25T05:30:51Z")
Background: I have logstash and elasticsearch running on a server. I have a rails application which uploads a CSV to the server and logstash then processes. It works if I manually execute the generated script. If I try t… --- ## [Elasticsearch \_update\_by\_query in logstash error \[HTTP Output Failure\] Encountered non-2xx HTTP code 400](https://discuss.elastic.co/t/elasticsearch-update-by-query-in-logstash-error-http-output-failure-encountered-non-2xx-http-code-400/274405)
**Author:** [@priyaankaa](https://discuss.elastic.co/u/priyaankaa)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 5:40am UTC](https://discuss.elastic.co/t/elasticsearch-update-by-query-in-logstash-error-http-output-failure-encountered-non-2xx-http-code-400/274405 "2021-06-25T05:40:42Z")
Properly working \_update\_by\_query call - POST /s1test-demo7/\_update\_by\_query { "script": { "source": "ctx.\_source.externaldata = params.externaldata", "lang": "painless", "params": { "externa… --- ## [Question about update sorting](https://discuss.elastic.co/t/question-about-update-sorting/277002)
**Author:** [@111486](https://discuss.elastic.co/u/111486)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 5:07am UTC](https://discuss.elastic.co/t/question-about-update-sorting/277002 "2021-06-25T05:07:32Z")
When updating the content, it was found that the updated content and the read content fields were not in the same order. version is: // 20210625115137 { "name": "372aac1d4b15", "cluster\_name": "tommy-es", "clu… --- ## [Which 'permission' for 'create user patternn'?](https://discuss.elastic.co/t/which-permission-for-create-user-patternn/277006)
**Author:** [@smm](https://discuss.elastic.co/u/smm)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 4:13am UTC](https://discuss.elastic.co/t/which-permission-for-create-user-patternn/277006 "2021-06-25T04:13:13Z")
Hi there, when I (as an cluster admin) enter the index name in the search field at "create index pattern" the index is displayed. When the customer does this, no index is found. So, which permission for him is needed t… --- ## [Highlight behavior in Elasticsearch version 6.0.1](https://discuss.elastic.co/t/highlight-behavior-in-elasticsearch-version-6-0-1/277005)
**Author:** [@shubhendra](https://discuss.elastic.co/u/shubhendra)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 4:09am UTC](https://discuss.elastic.co/t/highlight-behavior-in-elasticsearch-version-6-0-1/277005 "2021-06-25T04:09:53Z")
I'm facing this issue in Elastic search which I cannot explain. Highlight appears sometimes but other times they do not. Following is the way to reproduce it through kibana. My index and mapping - PUT test\_index { "ma… --- ## [Custom hostnames for agent deployments](https://discuss.elastic.co/t/custom-hostnames-for-agent-deployments/276997)
**Author:** [@netfire](https://discuss.elastic.co/u/netfire)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 2:49am UTC](https://discuss.elastic.co/t/custom-hostnames-for-agent-deployments/276997 "2021-06-25T02:49:17Z")
I'm testing out fleet. I would like the agents and associated beats to utilize a hostname I provide (or using a template of some sort). The issue I have is that windows hosts are not using the FQDN (in my case it would b… --- ## [\[Heartbeat\] clarify “context deadline exceeds” further](https://discuss.elastic.co/t/heartbeat-clarify-context-deadline-exceeds-further/274305)
**Author:** [@lowry](https://discuss.elastic.co/u/lowry)\ **Replies:** 3\ **Last updated:** [June 25, 2021, 2:36am UTC](https://discuss.elastic.co/t/heartbeat-clarify-context-deadline-exceeds-further/274305 "2021-06-25T02:36:18Z")
It is learned that Heartbeat employs Golang context to manage the session to do the http check, Getting “context deadline exceeds” error message while accessing a target timeout. However, there’re 2 different cases woul… --- ## [Drop\_event syntax trouble with multiple processors (7.7.1)](https://discuss.elastic.co/t/drop-event-syntax-trouble-with-multiple-processors-7-7-1/276834)
**Author:** [@Mike\_McGuire](https://discuss.elastic.co/u/Mike_McGuire)\ **Replies:** 4\ **Last updated:** [June 25, 2021, 1:30am UTC](https://discuss.elastic.co/t/drop-event-syntax-trouble-with-multiple-processors-7-7-1/276834 "2021-06-25T01:30:38Z")
This will get past the configuration check, but appears that it filters all events out. I think the OR is what is tripping me up. The other posts I've studied are close, but I can't recreate a working solution. The lo… --- ## [Question about indexing large numbers](https://discuss.elastic.co/t/question-about-indexing-large-numbers/276758)
**Author:** [@Jurgen\_Debedts](https://discuss.elastic.co/u/Jurgen_Debedts)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 1:29am UTC](https://discuss.elastic.co/t/question-about-indexing-large-numbers/276758 "2021-06-25T01:29:29Z")
This is somewhat a mix of conceptual and a technical question. I am working on a search integration between an ECM platform and ES. Everything is working fine except for 1 thing. In the ECM platform metadata field also… --- ## [Dynamically determine OS name](https://discuss.elastic.co/t/dynamically-determine-os-name/276956)
**Author:** [@jleon](https://discuss.elastic.co/u/jleon)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 12:14am UTC](https://discuss.elastic.co/t/dynamically-determine-os-name/276956 "2021-06-25T00:14:09Z")
Hi, In the agent yml file, is there a way to dynamically determine the OS name (windows, Linux, etc) to be included in the log msgs? I'm using filebeats version 7.x. I see this doc for version 6.x using Host fields, b… --- ## [GCS Keystore](https://discuss.elastic.co/t/gcs-keystore/276869)
**Author:** [@davidwebb535649](https://discuss.elastic.co/u/davidwebb535649)\ **Replies:** 3\ **Last updated:** [June 24, 2021, 11:16pm UTC](https://discuss.elastic.co/t/gcs-keystore/276869 "2021-06-24T23:16:40Z")
I have a Google Cloud instance I would like to work with. The GCS plugin is installed but I'm having trouble adding the Google credentials to the Elastic Keystore but I keep getting an error I have not been able to solve… --- ## [Aggregation to count consecutive instances of a given field value?](https://discuss.elastic.co/t/aggregation-to-count-consecutive-instances-of-a-given-field-value/276982)
**Author:** [@gpaciga](https://discuss.elastic.co/u/gpaciga)\ **Replies:** 0\ **Last updated:** [June 24, 2021, 9:08pm UTC](https://discuss.elastic.co/t/aggregation-to-count-consecutive-instances-of-a-given-field-value/276982 "2021-06-24T21:08:00Z")
I have an index that has a series of events: \[{ @timestamp: "...", jobId: "123", result: "failure", },{ @timestamp: "...", jobId: "567", result: "success", },{ @timestamp: "...", jobId: "12… --- ## [Lost All Indices After Adding Additional Pipelines](https://discuss.elastic.co/t/lost-all-indices-after-adding-additional-pipelines/276858)
**Author:** [@Steve\_K](https://discuss.elastic.co/u/Steve_K)\ **Replies:** 2\ **Last updated:** [June 24, 2021, 1:58pm UTC](https://discuss.elastic.co/t/lost-all-indices-after-adding-additional-pipelines/276858 "2021-06-24T13:58:08Z")
So after I cleared all of the indices and added an additional method to my pipeline configuration, my indices will not repopulate. I have added my code below for the original configuration and then reconfiguration. Ori… --- ## [Elastic-agent unhealthy](https://discuss.elastic.co/t/elastic-agent-unhealthy/276976)
**Author:** [@Xtrem](https://discuss.elastic.co/u/Xtrem)\ **Replies:** 1\ **Last updated:** [June 24, 2021, 8:02pm UTC](https://discuss.elastic.co/t/elastic-agent-unhealthy/276976 "2021-06-24T20:02:21Z")
Hi, I am currently testing version 7.12 of Elasticsearch and Kibana and have installed the Elastic-agent on two linux server and one Windows server. All the servers are "Unhealthy" but no datareaches elasticsearch. Whe… --- ## [More than one database in a single Elastic](https://discuss.elastic.co/t/more-than-one-database-in-a-single-elastic/276963)
**Author:** [@Glau\_Martins](https://discuss.elastic.co/u/Glau_Martins)\ **Replies:** 1\ **Last updated:** [June 24, 2021, 7:47pm UTC](https://discuss.elastic.co/t/more-than-one-database-in-a-single-elastic/276963 "2021-06-24T19:47:27Z")
Hi Guys! I need to know if I can have more than one database in a single Elastic. Can someone help me please? tks --- ## [Can I create an iframe of a Kibana dashboard without the configuration bar on the left?](https://discuss.elastic.co/t/can-i-create-an-iframe-of-a-kibana-dashboard-without-the-configuration-bar-on-the-left/276711)
**Author:** [@raylight](https://discuss.elastic.co/u/raylight)\ **Replies:** 4\ **Last updated:** [June 24, 2021, 7:35pm UTC](https://discuss.elastic.co/t/can-i-create-an-iframe-of-a-kibana-dashboard-without-the-configuration-bar-on-the-left/276711 "2021-06-24T19:35:57Z")
I know I can embed Kibana on my webpage following the procedure described on the documentation here. However, I've realized some differences when I create an iframe for visualizations and when I create an iframe for a da… --- ## [Alerting with Kibana & Elasticsearch on the same machine](https://discuss.elastic.co/t/alerting-with-kibana-elasticsearch-on-the-same-machine/276966)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 0\ **Last updated:** [June 24, 2021, 5:14pm UTC](https://discuss.elastic.co/t/alerting-with-kibana-elasticsearch-on-the-same-machine/276966 "2021-06-24T17:14:21Z")
I have a quick question for the community. I am wanting to enable Alerting on our on-prem Elastic Stack. I have set the value of xpack.encryptedSavedObjects.encryptionKey in the Kibana.yml file and now I am being prompt… --- ## [How to index newly parsed fields to be searchable - My Dynamic Template](https://discuss.elastic.co/t/how-to-index-newly-parsed-fields-to-be-searchable-my-dynamic-template/276962)
**Author:** [@rklemer](https://discuss.elastic.co/u/rklemer)\ **Replies:** 0\ **Last updated:** [June 24, 2021, 4:47pm UTC](https://discuss.elastic.co/t/how-to-index-newly-parsed-fields-to-be-searchable-my-dynamic-template/276962 "2021-06-24T16:47:45Z")
Hello, I'm trying to index custom fields that are parsed from s3 json log, it's not seeming to work, I have these processors: filebeat.inputs: - type: aws-s3 queue\_url: https://sqs.XXX\>XXXX.com/XXXX/XXXXX access\_key… --- ## [Typical Disk threshold](https://discuss.elastic.co/t/typical-disk-threshold/276960)
**Author:** [@Abhishek\_Shinde](https://discuss.elastic.co/u/Abhishek_Shinde)\ **Replies:** 0\ **Last updated:** [June 24, 2021, 4:27pm UTC](https://discuss.elastic.co/t/typical-disk-threshold/276960 "2021-06-24T16:27:34Z")
We are planning deploy ES as single node cluster. We are planning to use only flood\_stage configuration aslow and high watermark will have no impact. I know the default flood stage is set to 95% but would like to know … --- ## [A document doesn't have a value for a field error, but the field has values](https://discuss.elastic.co/t/a-document-doesnt-have-a-value-for-a-field-error-but-the-field-has-values/276839)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 4\ **Last updated:** [June 24, 2021, 3:34pm UTC](https://discuss.elastic.co/t/a-document-doesnt-have-a-value-for-a-field-error-but-the-field-has-values/276839 "2021-06-24T15:34:27Z")
Hi, i got a field with values like this InstanceName CPU ------------ --- mfemvedr 1,17 % macmnsvc 0,78 % services 0,39 % svchost 0,00 % searchindex… --- ## [Timelion top 5 not working compared to TSVB top 5](https://discuss.elastic.co/t/timelion-top-5-not-working-compared-to-tsvb-top-5/275221)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 6\ **Last updated:** [June 24, 2021, 3:01pm UTC](https://discuss.elastic.co/t/timelion-top-5-not-working-compared-to-tsvb-top-5/275221 "2021-06-24T15:01:30Z")
Hi, I have this top 5 interfaces based on the average of utilization, in TSVB last 24 hours But when I try to do the same in canvas using Timelion with this query .es(q='tenant name.keyword : LABSAV',index='metric-i… --- ## [Composable templates- how to convert Legacy?](https://discuss.elastic.co/t/composable-templates-how-to-convert-legacy/276934)
**Author:** [@P\_Larsen](https://discuss.elastic.co/u/P_Larsen)\ **Replies:** 1\ **Last updated:** [June 24, 2021, 2:56pm UTC](https://discuss.elastic.co/t/composable-templates-how-to-convert-legacy/276934 "2021-06-24T14:56:20Z")
How do I convert this to the new composable templates without breaking my ILM config? PUT \_template/99999-test\_deleteme\_policy { "index\_patterns": \[ "99999-test\_deleteme-\*" \], "settings": { "index": { "number\_of\_shards"… --- ## [Error ID Index pattern](https://discuss.elastic.co/t/error-id-index-pattern/276425)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 2\ **Last updated:** [June 24, 2021, 2:52pm UTC](https://discuss.elastic.co/t/error-id-index-pattern/276425 "2021-06-24T14:52:01Z")
hello, i need help with this error that kibana gives me when i create a filebeat index, i get this error: Error at Fetch.\_callee3$ (https://72dfe17217744236af40cc31b704a664.us-central1.gcp.cloud.es.io:9243/31997/bun… --- ## [Grok parser on amount of characters](https://discuss.elastic.co/t/grok-parser-on-amount-of-characters/276924)
**Author:** [@Soren\_vdc](https://discuss.elastic.co/u/Soren_vdc)\ **Replies:** 1\ **Last updated:** [June 24, 2021, 2:40pm UTC](https://discuss.elastic.co/t/grok-parser-on-amount-of-characters/276924 "2021-06-24T14:40:33Z")
Hi, I want to grok match on a sting where the first part contains 3 letters. For example: BUR-TEST-TEST-TEST I want to check that the first part contains 3 charachters (in this case BUR). is that possible with grok m… --- ## [How to set a custom scale (y axis) in Lens-Line?](https://discuss.elastic.co/t/how-to-set-a-custom-scale-y-axis-in-lens-line/276899)
**Author:** [@PJosse](https://discuss.elastic.co/u/PJosse)\ **Replies:** 2\ **Last updated:** [June 24, 2021, 2:34pm UTC](https://discuss.elastic.co/t/how-to-set-a-custom-scale-y-axis-in-lens-line/276899 "2021-06-24T14:34:40Z")
Hi, I starting to learn how to use Kibana, and I have a small difficulty : I want plot a line with Lens but scale for the y axis start at 0, so I cant see the real evolution of the line (my data is in \[300-340\]). I did… --- ## [Elasticsearch vs logstash ingest](https://discuss.elastic.co/t/elasticsearch-vs-logstash-ingest/276947)
**Author:** [@probson](https://discuss.elastic.co/u/probson)\ **Replies:** 0\ **Last updated:** [June 24, 2021, 2:32pm UTC](https://discuss.elastic.co/t/elasticsearch-vs-logstash-ingest/276947 "2021-06-24T14:32:40Z")
I have been using logstash for a number of years now with a pretty big pipeline. The majority of the systems we monitor are via winlogbeats. We are using elastic cloud and hosting logstash ourselves. With the introduc… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1348) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1350)