# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1713 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1714 --- ## [Logstash cloudwatch plugin error](https://discuss.elastic.co/t/logstash-cloudwatch-plugin-error/254337)
**Author:** [@deep9300](https://discuss.elastic.co/u/deep9300)\ **Replies:** 3\ **Last updated:** [November 5, 2020, 8:54pm UTC](https://discuss.elastic.co/t/logstash-cloudwatch-plugin-error/254337 "2020-11-05T20:54:52Z")
I'm trying to connect cloudwatch to logstash but get an error. this is my logstash conf file: input { cloudwatch { namespace =\> "AWS/Transfer" metrics =\> \[ "BytesIn", "BytesOut" \] filters … --- ## [Elasticsearch time to as in via python (timezone)](https://discuss.elastic.co/t/elasticsearch-time-to-as-in-via-python-timezone/254340)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 5\ **Last updated:** [November 5, 2020, 8:38pm UTC](https://discuss.elastic.co/t/elasticsearch-time-to-as-in-via-python-timezone/254340 "2020-11-05T20:38:34Z")
I have a wrote small python script where I am reading certain information from Elasticsearch and pushing that back in to ELK. what I want it to keep what ever date-time I get from ELK. I want to push same as is back in … --- ## [Compare if hosts/feeds are missing from last week](https://discuss.elastic.co/t/compare-if-hosts-feeds-are-missing-from-last-week/254412)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 3\ **Last updated:** [November 5, 2020, 8:25pm UTC](https://discuss.elastic.co/t/compare-if-hosts-feeds-are-missing-from-last-week/254412 "2020-11-05T20:25:46Z")
We have 1000's of hosts (windows/linux/solaris/network devices) coming to ELK cluster. We wanted to understand if there the hosts/devices missed sending the data. I could think of below options Get list of hosts comin… --- ## [Force replication](https://discuss.elastic.co/t/force-replication/254429)
**Author:** [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)\ **Replies:** 4\ **Last updated:** [November 5, 2020, 8:10pm UTC](https://discuss.elastic.co/t/force-replication/254429 "2020-11-05T20:10:50Z")
Hi there! I started a new fresh Elasticsearch (7.9) cluster with sample flights data. If I look up for the "number\_of\_replicas" setting of this index, I get 1. And, if I try to use the cat/shards API, I see one primary… --- ## [Index openxml structured documents with Elasticsearch](https://discuss.elastic.co/t/index-openxml-structured-documents-with-elasticsearch/254445)
**Author:** [@vhd.afr](https://discuss.elastic.co/u/vhd.afr)\ **Replies:** 0\ **Last updated:** [November 5, 2020, 5:53pm UTC](https://discuss.elastic.co/t/index-openxml-structured-documents-with-elasticsearch/254445 "2020-11-05T17:53:00Z")
We have a set of structured documents. The structure is extremely inspired by openxml data model. Briefly speaking, documents are made up of an ordered set of paragraphs, each paragraph itself has id and an ordered set o… --- ## [Filebeat for zip files](https://discuss.elastic.co/t/filebeat-for-zip-files/254385)
**Author:** [@kvtang](https://discuss.elastic.co/u/kvtang)\ **Replies:** 1\ **Last updated:** [November 5, 2020, 6:23pm UTC](https://discuss.elastic.co/t/filebeat-for-zip-files/254385 "2020-11-05T18:23:55Z")
Hi all, Is there any way to read a zipped file directly using filebeat? Thank you. --- ## [How to add up values from list and send only that value to elasticserach](https://discuss.elastic.co/t/how-to-add-up-values-from-list-and-send-only-that-value-to-elasticserach/254441)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 1\ **Last updated:** [November 5, 2020, 6:22pm UTC](https://discuss.elastic.co/t/how-to-add-up-values-from-list-and-send-only-that-value-to-elasticserach/254441 "2020-11-05T18:22:38Z")
Hello, I want to write logstash script that use snmpwalk to get cpu values form network devices average them and send one field to elasticsearch. I guess I can do this in ruby plugin but I have so fare only expiernce in… --- ## [How can we see the full output /result of a query in Metrcibeat Sql module](https://discuss.elastic.co/t/how-can-we-see-the-full-output-result-of-a-query-in-metrcibeat-sql-module/254239)
**Author:** [@Sunil18](https://discuss.elastic.co/u/Sunil18)\ **Replies:** 1\ **Last updated:** [November 5, 2020, 5:44pm UTC](https://discuss.elastic.co/t/how-can-we-see-the-full-output-result-of-a-query-in-metrcibeat-sql-module/254239 "2020-11-05T17:44:26Z")
I have configured the sql module to get custom metrices using for custom queries from Oracle ,MySQL and PostgreSQL databases. When I check the output using " metrciebat test modules sql " it does not show me full outpu… --- ## [Rendering several fields aggregations in one visualisation](https://discuss.elastic.co/t/rendering-several-fields-aggregations-in-one-visualisation/254066)
**Author:** [@Eugene\_Korepanov](https://discuss.elastic.co/u/Eugene_Korepanov)\ **Replies:** 3\ **Last updated:** [November 5, 2020, 5:28pm UTC](https://discuss.elastic.co/t/rendering-several-fields-aggregations-in-one-visualisation/254066 "2020-11-05T17:28:20Z")
Hello, dear Elastians and Kibanians. I'm looking for a decent way to build a table of several fields. In my data I have huge objects with multiple boolean properties: Doc1 Obj.Property1 = 1 Obj.Property2 = 0 ... Ob… --- ## [Random number of messages dropping on logstash import](https://discuss.elastic.co/t/random-number-of-messages-dropping-on-logstash-import/251658)
**Author:** [@Paul\_Shriner](https://discuss.elastic.co/u/Paul_Shriner)\ **Replies:** 16\ **Last updated:** [November 5, 2020, 5:18pm UTC](https://discuss.elastic.co/t/random-number-of-messages-dropping-on-logstash-import/251658 "2020-11-05T17:18:57Z")
I have Logstash 7.6 running on Ubuntu 20.04. I am importing a pretty simple CSV file. The data in the file is separated by commas. See below: email,send\_time,engaged,last\_touch\_utc,score,frequency,open\_count,click\_… --- ## [Context Suggester: measures to limit memory usage](https://discuss.elastic.co/t/context-suggester-measures-to-limit-memory-usage/254372)
**Author:** [@Bhavik92](https://discuss.elastic.co/u/Bhavik92)\ **Replies:** 0\ **Last updated:** [November 5, 2020, 8:23am UTC](https://discuss.elastic.co/t/context-suggester-measures-to-limit-memory-usage/254372 "2020-11-05T08:23:12Z")
We're using elasticsearch for our search use-case and have an index that serves both regular queries as well as autocompletion. For autocompletion, I've enabled completion suggester on it. However, there is growing con… --- ## [Using Filebeat or Logstash](https://discuss.elastic.co/t/using-filebeat-or-logstash/254332)
**Author:** [@Falikou1](https://discuss.elastic.co/u/Falikou1)\ **Replies:** 10\ **Last updated:** [November 5, 2020, 4:34pm UTC](https://discuss.elastic.co/t/using-filebeat-or-logstash/254332 "2020-11-05T16:34:20Z")
What is the benefit of using Filebeat alone? What is the advantage of using Logstash alone? What is the advantage of using Filebeat and Logstash? --- ## [GROKPARSE Failure and after Successful debug](https://discuss.elastic.co/t/grokparse-failure-and-after-successful-debug/254324)
**Author:** [@Vinay\_Kumar2](https://discuss.elastic.co/u/Vinay_Kumar2)\ **Replies:** 2\ **Last updated:** [November 5, 2020, 4:19pm UTC](https://discuss.elastic.co/t/grokparse-failure-and-after-successful-debug/254324 "2020-11-05T16:19:10Z")
Hi, i have used https://grokdebug.herokuapp.com/ to create grok match patterns and which are worked fine. However when checking with logstash, some of fields are not extracted and getting grokparse failure error. Samp… --- ## [Imported Indexes are not showing logs](https://discuss.elastic.co/t/imported-indexes-are-not-showing-logs/254391)
**Author:** [@nyquillus](https://discuss.elastic.co/u/nyquillus)\ **Replies:** 5\ **Last updated:** [November 5, 2020, 3:34pm UTC](https://discuss.elastic.co/t/imported-indexes-are-not-showing-logs/254391 "2020-11-05T15:34:40Z")
Hi, I tried importing an index to another elk stack and had no issues with it. The problem is that the index shows no logs on the discover tab. I think i need to do something else to import the logs but can't seem to fi… --- ## [Ingest Manager - Custom Files not being uploaded](https://discuss.elastic.co/t/ingest-manager-custom-files-not-being-uploaded/253397)
**Author:** [@\_Meir](https://discuss.elastic.co/u/_Meir)\ **Replies:** 8\ **Last updated:** [November 5, 2020, 3:31pm UTC](https://discuss.elastic.co/t/ingest-manager-custom-files-not-being-uploaded/253397 "2020-11-05T15:31:00Z")
I'm trying to get custom logs uploaded from my server to my elastic cloud . I'm using the Ingest Manager via fleet deployment. I've added Windows, System and Custom Logs integrations. I'm only seeing logs from MetricB… --- ## [Validating that documents conform to strict mappings](https://discuss.elastic.co/t/validating-that-documents-conform-to-strict-mappings/254421)
**Author:** [@mjuric](https://discuss.elastic.co/u/mjuric)\ **Replies:** 2\ **Last updated:** [November 5, 2020, 2:59pm UTC](https://discuss.elastic.co/t/validating-that-documents-conform-to-strict-mappings/254421 "2020-11-05T14:59:01Z")
We need to validate that documents delivered to us conform to strict mappings in ElastiSearch before they are indexed. Is there a way to use the strict mapping validation component of ElasticSearch as a library outside t… --- ## [Json template format logstash 7.6.2](https://discuss.elastic.co/t/json-template-format-logstash-7-6-2/254328)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 0\ **Last updated:** [November 4, 2020, 8:30pm UTC](https://discuss.elastic.co/t/json-template-format-logstash-7-6-2/254328 "2020-11-04T20:30:28Z")
Hi All, I am new to applying json templates. I have ELK Stack 7.6.2 My template looks like below: { "mappings": { "\_doc": { "\_all": { "enabled": true, "norms": { "enabled": fals… --- ## [Syntax changes](https://discuss.elastic.co/t/syntax-changes/254418)
**Author:** [@somethingcool](https://discuss.elastic.co/u/somethingcool)\ **Replies:** 1\ **Last updated:** [November 5, 2020, 2:31pm UTC](https://discuss.elastic.co/t/syntax-changes/254418 "2020-11-05T14:31:24Z")
Currently we are using Elastic 2.3.x and migrating to 6.8.x and noticed that one of our queries is no longer returning data. The data between the environment versions is 100% the same. We have a job that writes the same … --- ## [Negating Windows logons from managed service accounts ending with $](https://discuss.elastic.co/t/negating-windows-logons-from-managed-service-accounts-ending-with/254419)
**Author:** [@0nullbytes](https://discuss.elastic.co/u/0nullbytes)\ **Replies:** 3\ **Last updated:** [November 5, 2020, 2:11pm UTC](https://discuss.elastic.co/t/negating-windows-logons-from-managed-service-accounts-ending-with/254419 "2020-11-05T14:11:16Z")
Am using Kibana 6.8.11, using Kibana query language I want to exclude Windows Service Accounts ending with a $. Ideally the following query should work !data.win.eventdata.targetUserName:\*$ but for some reason it does no… --- ## [How to remove part of name in multiple fields with Ruby in logstash?](https://discuss.elastic.co/t/how-to-remove-part-of-name-in-multiple-fields-with-ruby-in-logstash/254263)
**Author:** [@alytkowski](https://discuss.elastic.co/u/alytkowski)\ **Replies:** 9\ **Last updated:** [November 5, 2020, 1:57pm UTC](https://discuss.elastic.co/t/how-to-remove-part-of-name-in-multiple-fields-with-ruby-in-logstash/254263 "2020-11-05T13:57:47Z")
I have a source of metrics, which are received by logstash and sent out to Elastic later. There are multiple fields starting with "prometheus.metrics.ems". Can someone help with the ruby code or some other method to cut… --- ## [For length parameter in scripted fields its returning length of unique values in a doc](https://discuss.elastic.co/t/for-length-parameter-in-scripted-fields-its-returning-length-of-unique-values-in-a-doc/254393)
**Author:** [@Pranusha119](https://discuss.elastic.co/u/Pranusha119)\ **Replies:** 3\ **Last updated:** [November 5, 2020, 1:38pm UTC](https://discuss.elastic.co/t/for-length-parameter-in-scripted-fields-its-returning-length-of-unique-values-in-a-doc/254393 "2020-11-05T13:38:16Z")
Hello, Length of string is 5 , but why is it returning 3 by counting only unique characters ? Same with words in a string --- ## [Cloned/Split Indexes Take Double Disk Space When Increasing Shards](https://discuss.elastic.co/t/cloned-split-indexes-take-double-disk-space-when-increasing-shards/254342)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 4\ **Last updated:** [November 5, 2020, 1:32pm UTC](https://discuss.elastic.co/t/cloned-split-indexes-take-double-disk-space-when-increasing-shards/254342 "2020-11-05T13:32:43Z")
Currently I have some large index's (100GB - 500GB) that have been allocated 5 shards. I would like to increase these to 10 - 20 shards each. If I use the Split API or Clone API it takes 1-2 minutes for the new index t… --- ## [Paging for multisearch (from & size are not quite doing it)](https://discuss.elastic.co/t/paging-for-multisearch-from-size-are-not-quite-doing-it/254408)
**Author:** [@LanceWalton](https://discuss.elastic.co/u/LanceWalton)\ **Replies:** 0\ **Last updated:** [November 5, 2020, 1:09pm UTC](https://discuss.elastic.co/t/paging-for-multisearch-from-size-are-not-quite-doing-it/254408 "2020-11-05T13:09:44Z")
Hi. I have a question regarding using a multisearch across many indexes with from & size parameters for the purposes of paging. I have many indexes that I want to combine into a multisearch. The problem I'm having is t… --- ## [Logstash Input TCP error Netty libraies](https://discuss.elastic.co/t/logstash-input-tcp-error-netty-libraies/254407)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 0\ **Last updated:** [November 5, 2020, 12:58pm UTC](https://discuss.elastic.co/t/logstash-input-tcp-error-netty-libraies/254407 "2020-11-05T12:58:54Z")
HI Team, Recently we upgraded our logstash-input-tcp from 5.0.9 to 5.2.0 there are SSL errors in the log file. \[2020-11-04T00:00:51,384\]\[ERROR\]\[logstash.inputs.tcp \] Error in Netty pipeline: io.netty.handler.code… --- ## [Using the Elasticsearch SQL API in Visual FoxPro (VFP) 9](https://discuss.elastic.co/t/using-the-elasticsearch-sql-api-in-visual-foxpro-vfp-9/254405)
**Author:** [@JCannon](https://discuss.elastic.co/u/JCannon)\ **Replies:** 0\ **Last updated:** [November 5, 2020, 12:55pm UTC](https://discuss.elastic.co/t/using-the-elasticsearch-sql-api-in-visual-foxpro-vfp-9/254405 "2020-11-05T12:55:42Z")
I am trying to see if I can get a program in Visual FoxPro (VFP) to work with the Elasticsearch SQL API. I have very little experience with both using REST APIs in VFP and Elasticsearch. Trying something new here. So… --- ## [Skip a screen in Kibana](https://discuss.elastic.co/t/skip-a-screen-in-kibana/254399)
**Author:** [@ErwinEnableU](https://discuss.elastic.co/u/ErwinEnableU)\ **Replies:** 2\ **Last updated:** [November 5, 2020, 12:55pm UTC](https://discuss.elastic.co/t/skip-a-screen-in-kibana/254399 "2020-11-05T12:55:21Z")
Dear community, We use Kibana for our customers to have insight in what we do using a Dashboard. The Customers have limited access to Kibana. When logging into Kibana they get a screen about Observability, Security, Vis… --- ## [Not getting data in Elastic & Kibna from logstash](https://discuss.elastic.co/t/not-getting-data-in-elastic-kibna-from-logstash/254154)
**Author:** [@Atul\_Chadha](https://discuss.elastic.co/u/Atul_Chadha)\ **Replies:** 2\ **Last updated:** [November 5, 2020, 12:10pm UTC](https://discuss.elastic.co/t/not-getting-data-in-elastic-kibna-from-logstash/254154 "2020-11-05T12:10:08Z")
I am trying to upgrade our 5.6 cluster to 7.6 and can certainly use some expertise. I have configured the filebeat with below config filebeat: config: inputs: path: /etc/filebeat/filebeat.yml filebeat: inputs: - i… --- ## [Problem with seting up of index lifecycle](https://discuss.elastic.co/t/problem-with-seting-up-of-index-lifecycle/254298)
**Author:** [@grynhtjnyuk](https://discuss.elastic.co/u/grynhtjnyuk)\ **Replies:** 2\ **Last updated:** [November 5, 2020, 11:52am UTC](https://discuss.elastic.co/t/problem-with-seting-up-of-index-lifecycle/254298 "2020-11-05T11:52:22Z")
I have the index with data from filebeat. I have set up the hot phase: maximum index size 600mb maximum age 4d Could you tell me, how to achive this behaviour: when the index come to 600mb OR age 4d , the index skip… --- ## [Custom Unpack for configuration](https://discuss.elastic.co/t/custom-unpack-for-configuration/254306)
**Author:** [@tomkirk](https://discuss.elastic.co/u/tomkirk)\ **Replies:** 5\ **Last updated:** [November 5, 2020, 11:15am UTC](https://discuss.elastic.co/t/custom-unpack-for-configuration/254306 "2020-11-05T11:15:05Z")
I'm working on a beat where I have currently in my configuration an array of string like this: fields: - FieldName - AnotherFieldName What I would like is the ability to optionally provide a type tag, like so: fie… --- ## [Problem with sending logs from filebeat to logstash when I add kafka to pipeline](https://discuss.elastic.co/t/problem-with-sending-logs-from-filebeat-to-logstash-when-i-add-kafka-to-pipeline/254285)
**Author:** [@Pavle\_Ilic](https://discuss.elastic.co/u/Pavle_Ilic)\ **Replies:** 2\ **Last updated:** [November 5, 2020, 10:18am UTC](https://discuss.elastic.co/t/problem-with-sending-logs-from-filebeat-to-logstash-when-i-add-kafka-to-pipeline/254285 "2020-11-05T10:18:32Z")
I seem to be having a problem with sending logs from filebeat to logstash via kafka, and I can't find out why. I can send them directly from filebeat to logstash without a problem, but when I add kafka to the pipeline t… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1712) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1714)