# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1516 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1517 --- ## [Wildcard index pattern matching non-system indices, 7.10.2 edition](https://discuss.elastic.co/t/wildcard-index-pattern-matching-non-system-indices-7-10-2-edition/263934)
**Author:** [@Supermathie](https://discuss.elastic.co/u/Supermathie)\ **Replies:** 4\ **Last updated:** [March 9, 2021, 5:30pm UTC](https://discuss.elastic.co/t/wildcard-index-pattern-matching-non-system-indices-7-10-2-edition/263934 "2021-03-09T17:30:12Z")
Allegedly in Kibana 7.10.2 this pattern is supposed to match only user indices: but it's definitely picking up fields from system indices: Is this supposed to work now in 7+ or do we need to keep using this w… --- ## [Logstash ternary condition when a winlogbeat field doesn't exist](https://discuss.elastic.co/t/logstash-ternary-condition-when-a-winlogbeat-field-doesnt-exist/266721)
**Author:** [@mguttula](https://discuss.elastic.co/u/mguttula)\ **Replies:** 4\ **Last updated:** [March 9, 2021, 5:17pm UTC](https://discuss.elastic.co/t/logstash-ternary-condition-when-a-winlogbeat-field-doesnt-exist/266721 "2021-03-09T17:17:04Z")
Logstash is sending winlogbeat fields to Spunk and when the fields don't exist it is send something like %{\[winlog\]\[event\_data\]\[AuthenticationPackageName\]} I am trying to find a way to either not send the field if it do… --- ## [Reindex w/field collapsing](https://discuss.elastic.co/t/reindex-w-field-collapsing/266725)
**Author:** [@F\_Doe\_FW](https://discuss.elastic.co/u/F_Doe_FW)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 4:51pm UTC](https://discuss.elastic.co/t/reindex-w-field-collapsing/266725 "2021-03-09T16:51:58Z")
Is there any way to collapse results with ReIndexing? I have a source index and want to get the most used pieces out, and only return their 'natural' key into a new index. I can't figure out how to stuff the collapse l… --- ## [Sync data from mongodb](https://discuss.elastic.co/t/sync-data-from-mongodb/266719)
**Author:** [@Gharsa\_Khouloud](https://discuss.elastic.co/u/Gharsa_Khouloud)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 4:24pm UTC](https://discuss.elastic.co/t/sync-data-from-mongodb/266719 "2021-03-09T16:24:43Z")
I just want to sync the data from mongodb to elastic using logstash. Its working good, when any new record comes in mongodb, logstash pushes into elastic. But when I update any record in mongodb then it does not change i… --- ## [Aggregation query returning single word results](https://discuss.elastic.co/t/aggregation-query-returning-single-word-results/266718)
**Author:** [@aanders50](https://discuss.elastic.co/u/aanders50)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 4:06pm UTC](https://discuss.elastic.co/t/aggregation-query-returning-single-word-results/266718 "2021-03-09T16:06:54Z")
Hello! Just wanted to preface that I am new to ElasticSearch so I apologize if I am missing any basic concepts. I have an issue and I saw some other similar articles, but they were from quite awhile back and wanted to v… --- ## [Sending data from mongodb to Elasticsearch](https://discuss.elastic.co/t/sending-data-from-mongodb-to-elasticsearch/266706)
**Author:** [@Laetitia\_RICHARD](https://discuss.elastic.co/u/Laetitia_RICHARD)\ **Replies:** 2\ **Last updated:** [March 9, 2021, 3:42pm UTC](https://discuss.elastic.co/t/sending-data-from-mongodb-to-elasticsearch/266706 "2021-03-09T15:42:47Z")
Hello, I would like to send data from mongodb in Elasticsearch, I saw a lot of different plugins like monstache, mongo-connector, but I'd like to have your recommendations for indexing this data, knowing that I will hav… --- ## [How to make difference between 2 source of logs in Logstash input](https://discuss.elastic.co/t/how-to-make-difference-between-2-source-of-logs-in-logstash-input/266709)
**Author:** [@Abdelhalim](https://discuss.elastic.co/u/Abdelhalim)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 3:35pm UTC](https://discuss.elastic.co/t/how-to-make-difference-between-2-source-of-logs-in-logstash-input/266709 "2021-03-09T15:35:50Z")
Hello everyone, I want to send log of the "auth.log" file using system module of filebeat, and I have another file that I want to parse using logstash (as there is no module for that type of log). And as I know, there … --- ## [Metrics: There is no data to display (error while fetching resource)](https://discuss.elastic.co/t/metrics-there-is-no-data-to-display-error-while-fetching-resource/266714)
**Author:** [@feva](https://discuss.elastic.co/u/feva)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 3:27pm UTC](https://discuss.elastic.co/t/metrics-there-is-no-data-to-display-error-while-fetching-resource/266714 "2021-03-09T15:27:16Z")
Hello, I have som troubles with metricbeat and Metrics App in Kibana. I am using metricbeat 7.10 and Kibana 7.10 on a Windows Server 2019. Every now and then I get an error message when I open Metrics App in Kibana: Er… --- ## [No document in index, but index stats reports numerous docs](https://discuss.elastic.co/t/no-document-in-index-but-index-stats-reports-numerous-docs/266708)
**Author:** [@zarak](https://discuss.elastic.co/u/zarak)\ **Replies:** 4\ **Last updated:** [March 9, 2021, 3:13pm UTC](https://discuss.elastic.co/t/no-document-in-index-but-index-stats-reports-numerous-docs/266708 "2021-03-09T15:13:52Z")
Hi ! I have an index that reports me some documents in Kibana in Index management, as well as in its stats: GET /v2-logs-000003/\_stats { "\_shards" : { "total" : 6, "successful" : 6, "f… --- ## [Filebeat to index with different structures](https://discuss.elastic.co/t/filebeat-to-index-with-different-structures/266565)
**Author:** [@Edi1](https://discuss.elastic.co/u/Edi1)\ **Replies:** 6\ **Last updated:** [March 9, 2021, 2:00pm UTC](https://discuss.elastic.co/t/filebeat-to-index-with-different-structures/266565 "2021-03-09T14:00:01Z")
Hello, newby here, My task is to take multiple (8) logs with different formats and using filebeat send them directly to elastic. They should all end inside the same index and each log should be searchable using "level" … --- ## [Get only one level of documents when using path-hierarchy tokenizer](https://discuss.elastic.co/t/get-only-one-level-of-documents-when-using-path-hierarchy-tokenizer/266704)
**Author:** [@avhvr](https://discuss.elastic.co/u/avhvr)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 1:53pm UTC](https://discuss.elastic.co/t/get-only-one-level-of-documents-when-using-path-hierarchy-tokenizer/266704 "2021-03-09T13:53:01Z")
Hi, I need to search documents (employee details) that are present in a tree structure. The number of levels in the hierarchy could be any. . Using path-hierarchy tokenizer and this post, I was able to index them and… --- ## [Logstash Performance Leak When Consuming Kafka Topic into Elasticsearch](https://discuss.elastic.co/t/logstash-performance-leak-when-consuming-kafka-topic-into-elasticsearch/266685)
**Author:** [@davidb](https://discuss.elastic.co/u/davidb)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 11:39am UTC](https://discuss.elastic.co/t/logstash-performance-leak-when-consuming-kafka-topic-into-elasticsearch/266685 "2021-03-09T11:39:13Z")
Greetings, I am facing a problem regarding logstash perfomance degradation over time. Any advice or hint is welcome. Thanks very much in advance. Short summary Logstash suffers from a perfomance leak over time. The data… --- ## [HPA for clients in helm chart](https://discuss.elastic.co/t/hpa-for-clients-in-helm-chart/266694)
**Author:** [@adrobisch](https://discuss.elastic.co/u/adrobisch)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 12:27pm UTC](https://discuss.elastic.co/t/hpa-for-clients-in-helm-chart/266694 "2021-03-09T12:27:30Z")
Hi, we are running the official k8s helm chart are wondering why there is no HPA support for the client node group? We are aware that it would not be a good idea on the data / master nodes and that the clients also mai… --- ## [Problem with email\_from\_field use in elastalert](https://discuss.elastic.co/t/problem-with-email-from-field-use-in-elastalert/266688)
**Author:** [@LuigiDelavega](https://discuss.elastic.co/u/LuigiDelavega)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 11:54am UTC](https://discuss.elastic.co/t/problem-with-email-from-field-use-in-elastalert/266688 "2021-03-09T11:54:15Z")
Hello, I have an issue using the email\_from\_field ... field. I created a basic alert, which is triggered (i tested without email\_from\_field and with the "email" filled). But i never received any mail on the mail transm… --- ## [Increase in CPU usage when deleting snapshots](https://discuss.elastic.co/t/increase-in-cpu-usage-when-deleting-snapshots/266682)
**Author:** [@Norman\_Khine](https://discuss.elastic.co/u/Norman_Khine)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 11:29am UTC](https://discuss.elastic.co/t/increase-in-cpu-usage-when-deleting-snapshots/266682 "2021-03-09T11:29:46Z")
Hello, I have a process which deletes outdated ES snapshots which are saved on AWS S3. Currently there are 326 snapshots which are scheduled for deletion. From the docs, https://www.elastic.co/guide/en/elasticsearch/re… --- ## [Logstash - In memory maps/Lookup tables?](https://discuss.elastic.co/t/logstash-in-memory-maps-lookup-tables/266673)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 10:56am UTC](https://discuss.elastic.co/t/logstash-in-memory-maps-lookup-tables/266673 "2021-03-09T10:56:44Z")
Filebeat sends data to logstash. I’m reading files that are up to about 500mb, about 2 million lines. On each longline I calculate certain fields based on patterns in the log file name, example, split the path, get the… --- ## [Index store size increased by 4X on new index created using slipt api](https://discuss.elastic.co/t/index-store-size-increased-by-4x-on-new-index-created-using-slipt-api/266633)
**Author:** [@sreekanth](https://discuss.elastic.co/u/sreekanth)\ **Replies:** 3\ **Last updated:** [March 9, 2021, 10:33am UTC](https://discuss.elastic.co/t/index-store-size-increased-by-4x-on-new-index-created-using-slipt-api/266633 "2021-03-09T10:33:47Z")
We had one index with 1 shard/1 replica config. As per /\_cat/indices, it shows docs.count =133213 docs.deleted=2430 store.size=819.1gb pri.store.size=409.5gb now we used split api to create new index with 40 shards. … --- ## [Kibana dashboard recovery](https://discuss.elastic.co/t/kibana-dashboard-recovery/266649)
**Author:** [@tt\_cc](https://discuss.elastic.co/u/tt_cc)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 9:50am UTC](https://discuss.elastic.co/t/kibana-dashboard-recovery/266649 "2021-03-09T09:50:05Z")
When reinstalling elk, how do I apply the existing kibana dashboard visualizations to the newly installed kibana? --- ## [Barracuda WAF Log Parsing](https://discuss.elastic.co/t/barracuda-waf-log-parsing/266665)
**Author:** [@PhilA](https://discuss.elastic.co/u/PhilA)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 9:38am UTC](https://discuss.elastic.co/t/barracuda-waf-log-parsing/266665 "2021-03-09T09:38:29Z")
Hi I would appreciate some guidance on debugging the Barracuda WAF Filebeat module. I am running this with Filebeat rather than fleet at the moment as fleet is still in Beta and when I try running it in Fleet I get zer… --- ## [Rename the index-pattern for sample visualizations](https://discuss.elastic.co/t/rename-the-index-pattern-for-sample-visualizations/266617)
**Author:** [@b133](https://discuss.elastic.co/u/b133)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 9:24am UTC](https://discuss.elastic.co/t/rename-the-index-pattern-for-sample-visualizations/266617 "2021-03-09T09:24:13Z")
Short version: In the new Kibana Lens app, how do you open existing visualizations to edit? Long version: I want to use the sample visualizations/dashboards provided by Filebeat in indices named other than filebeat-\*. … --- ## [Ruby script and boolean](https://discuss.elastic.co/t/ruby-script-and-boolean/266535)
**Author:** [@logger](https://discuss.elastic.co/u/logger)\ **Replies:** 2\ **Last updated:** [March 9, 2021, 8:25am UTC](https://discuss.elastic.co/t/ruby-script-and-boolean/266535 "2021-03-09T08:25:43Z")
Hi there, I have a question about the ruby filter. I am getting logs in Json they are working quite good, but ofcourse I have inconsistence in the mapping. Usually the logs arrive with the fields arg0, arg1, arg,2 T… --- ## [Pipeline computed Field](https://discuss.elastic.co/t/pipeline-computed-field/266558)
**Author:** [@graimato](https://discuss.elastic.co/u/graimato)\ **Replies:** 2\ **Last updated:** [March 9, 2021, 8:18am UTC](https://discuss.elastic.co/t/pipeline-computed-field/266558 "2021-03-09T08:18:28Z")
Hello, I wrote a pipeline to parse log files, I'm trying to create a computed field I have es. timerequest timeresponse It is possible to create a new field latency { "set" : { "field" : "latency… --- ## [Schema design and query - aggregation within nested document](https://discuss.elastic.co/t/schema-design-and-query-aggregation-within-nested-document/266629)
**Author:** [@cuongthamhl](https://discuss.elastic.co/u/cuongthamhl)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 8:14am UTC](https://discuss.elastic.co/t/schema-design-and-query-aggregation-within-nested-document/266629 "2021-03-09T08:14:32Z")
I have a simple schema with a nested field. A sample product document look like this: { "name": "product name", "events": \[ { "startDate": "2021-01-01 00:00:00", "endDate": "2021-01-01 00:00:00", … --- ## [Date timezone issue](https://discuss.elastic.co/t/date-timezone-issue/266644)
**Author:** [@Priyank07](https://discuss.elastic.co/u/Priyank07)\ **Replies:** 3\ **Last updated:** [March 9, 2021, 8:04am UTC](https://discuss.elastic.co/t/date-timezone-issue/266644 "2021-03-09T08:04:39Z")
Hi, I am pushing some documents to elastic and passing time as local time. So my question is.. Will that time converts to utc or it will be in local only. --- ## [How can I split string in Elasticsearch](https://discuss.elastic.co/t/how-can-i-split-string-in-elasticsearch/266576)
**Author:** [@pakwai122](https://discuss.elastic.co/u/pakwai122)\ **Replies:** 5\ **Last updated:** [March 9, 2021, 7:57am UTC](https://discuss.elastic.co/t/how-can-i-split-string-in-elasticsearch/266576 "2021-03-09T07:57:04Z")
I'm using nodejs as the client-side for Elasticsearch and got the following settings for the query. I want to split the text while users input text like "aa123 bb234" into "aa123 bb" "234". Only split the pattern like "b… --- ## [How to get date difference in weeks between today and date\_timestamp field](https://discuss.elastic.co/t/how-to-get-date-difference-in-weeks-between-today-and-date-timestamp-field/266378)
**Author:** [@Rakesh\_Rayabarapu](https://discuss.elastic.co/u/Rakesh_Rayabarapu)\ **Replies:** 4\ **Last updated:** [March 9, 2021, 7:36am UTC](https://discuss.elastic.co/t/how-to-get-date-difference-in-weeks-between-today-and-date-timestamp-field/266378 "2021-03-09T07:36:56Z")
Hello Team, could any one help me to write scripted field to get date difference in weeks between current datetime and doc\['timestamp'\]. to get current date time used datetime.new date().getTime() Used this script. (… --- ## [MultiLine read with File Beat](https://discuss.elastic.co/t/multiline-read-with-file-beat/266642)
**Author:** [@ankitji](https://discuss.elastic.co/u/ankitji)\ **Replies:** 0\ **Last updated:** [March 9, 2021, 6:59am UTC](https://discuss.elastic.co/t/multiline-read-with-file-beat/266642 "2021-03-09T06:59:15Z")
Hi, I am using filebeat on my hdfs root directories which having lots of log file inside that, for this we also using multiline parser of filebeat. So i just want to confirm on some below queries: Since we are readin… --- ## [Elasticsearch](https://discuss.elastic.co/t/elasticsearch/266528)
**Author:** [@alipujaistopo](https://discuss.elastic.co/u/alipujaistopo)\ **Replies:** 5\ **Last updated:** [March 9, 2021, 6:43am UTC](https://discuss.elastic.co/t/elasticsearch/266528 "2021-03-09T06:43:13Z")
what files are pulled from the target server to elastic and where are they stored? how long is the file stored? How do I set the file I want to drag to elastic? --- ## [CSV::MalformedCSVError: Unclosed quoted field on line 1](https://discuss.elastic.co/t/csv-unclosed-quoted-field-on-line-1/266509)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 2\ **Last updated:** [March 9, 2021, 6:10am UTC](https://discuss.elastic.co/t/csv-unclosed-quoted-field-on-line-1/266509 "2021-03-09T06:10:07Z")
After running code, getting this error. How to resolve this error. In front of date I am getting this symbol ∩╗┐ while running my configuration file. It appears once(first line). Error parsing csv {:field=\>"message", :… --- ## [Filebeat AWS Module SQS Queue Configurations Errors](https://discuss.elastic.co/t/filebeat-aws-module-sqs-queue-configurations-errors/266593)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 5:37am UTC](https://discuss.elastic.co/t/filebeat-aws-module-sqs-queue-configurations-errors/266593 "2021-03-09T05:37:04Z")
I am configuring filebeat AWS module to fetch Cloudtrail logs from an s3 bucket. I configured my settings from this article. However, somehow filebeat is unable to find SQS queue's region. here is my filebeat.yml input … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1515) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1517)