# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1915 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1916 --- ## [Find offending index causing "unknown mapping" in v6.8.6](https://discuss.elastic.co/t/find-offending-index-causing-unknown-mapping-in-v6-8-6/242007)
**Author:** [@atmosx](https://discuss.elastic.co/u/atmosx)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 10:40am UTC](https://discuss.elastic.co/t/find-offending-index-causing-unknown-mapping-in-v6-8-6/242007 "2020-07-21T10:40:55Z")
Hello, We have several ES clusters receiving logs from kubernetes apps (fluentd) and non kubernetes apps (filebeat). All clusters have the same configuration. The "sandbox" cluster contains "unknown" type fields in some… --- ## [Error while starting Logstash in WIndows](https://discuss.elastic.co/t/error-while-starting-logstash-in-windows/242006)
**Author:** [@Mitan\_Kumar\_Batu](https://discuss.elastic.co/u/Mitan_Kumar_Batu)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 10:40am UTC](https://discuss.elastic.co/t/error-while-starting-logstash-in-windows/242006 "2020-07-21T10:40:42Z")
Getting below error while starting Logstash:- \[ERROR\] 2020-07-21 16:04:29.491 \[main\] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (GemspecError) \[!\] There was an error wh… --- ## [No config files found and no configuration source found](https://discuss.elastic.co/t/no-config-files-found-and-no-configuration-source-found/242003)
**Author:** [@Bigranawab](https://discuss.elastic.co/u/Bigranawab)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 10:27am UTC](https://discuss.elastic.co/t/no-config-files-found-and-no-configuration-source-found/242003 "2020-07-21T10:27:15Z")
\ WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration… --- ## [How to execute Must and Should query that matches an individual Nested field?](https://discuss.elastic.co/t/how-to-execute-must-and-should-query-that-matches-an-individual-nested-field/239648)
**Author:** [@rd8388](https://discuss.elastic.co/u/rd8388)\ **Replies:** 9\ **Last updated:** [July 21, 2020, 10:26am UTC](https://discuss.elastic.co/t/how-to-execute-must-and-should-query-that-matches-an-individual-nested-field/239648 "2020-07-21T10:26:52Z")
A record has two items/values indexed in a nested field type addresses.fulladdress. "11 High Street" and "88 West Avenue" If a user searched for "11 West" this record should not be matched. How should we structure the … --- ## [Aggregator \[load\_time\_outlier\] of type \[percentiles\] cannot accept sub-aggregations](https://discuss.elastic.co/t/aggregator-load-time-outlier-of-type-percentiles-cannot-accept-sub-aggregations/241981)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 10:15am UTC](https://discuss.elastic.co/t/aggregator-load-time-outlier-of-type-percentiles-cannot-accept-sub-aggregations/241981 "2020-07-21T10:15:48Z")
i use percentiles aggregation for the response times low25% mid50% top25% partitions and then want to get min/max values based on each aggregated partitions. i try to write query like GET myindex-2020.06/\_search { "si… --- ## [Elasticsearch 7.7.1 ServiceConfigurationError Cannot instantiate FastCompressingCodec](https://discuss.elastic.co/t/elasticsearch-7-7-1-serviceconfigurationerror-cannot-instantiate-fastcompressingcodec/241971)
**Author:** [@Deb](https://discuss.elastic.co/u/Deb)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 10:12am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-1-serviceconfigurationerror-cannot-instantiate-fastcompressingcodec/241971 "2020-07-21T10:12:25Z")
I am trying to run a test by extending ESSingleNodeTestCase with elasticsearch 7.7.1. But I am getting the below error. I am not seeing any jar hell error and I have made the intellij settings as mentioned here java.uti… --- ## [Need suggestions on tutorials](https://discuss.elastic.co/t/need-suggestions-on-tutorials/241995)
**Author:** [@prajwalgmpp](https://discuss.elastic.co/u/prajwalgmpp)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 9:33am UTC](https://discuss.elastic.co/t/need-suggestions-on-tutorials/241995 "2020-07-21T09:33:40Z")
Is there a comprehensive tutorial that shows how to create AWS Instances as IAAS, push data through filebeat to logstash, index the data in ElasticSearch and visualize in Kibana? Please suggest. --- ## [Logstash consumes too much CPU while Redis is down](https://discuss.elastic.co/t/logstash-consumes-too-much-cpu-while-redis-is-down/241992)
**Author:** [@gabor-T-mandity](https://discuss.elastic.co/u/gabor-T-mandity)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 9:02am UTC](https://discuss.elastic.co/t/logstash-consumes-too-much-cpu-while-redis-is-down/241992 "2020-07-21T09:02:34Z")
We have multple pipelines, which has the same structure: Redis cache in seperate environments and one centralized ELK stack. To save up some money, we shut down our dev environments every day. Our problem is, that while… --- ## [Logstash encountered retryable error](https://discuss.elastic.co/t/logstash-encountered-retryable-error/241991)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 9:02am UTC](https://discuss.elastic.co/t/logstash-encountered-retryable-error/241991 "2020-07-21T09:02:08Z")
hi All, We are ingesting documents using logstash, all of a sudden i'm getting the below error. \[\[main\]\>worker1\] elasticsearch - Encountered a retryable error. Will Retry with exponential backoff {:code=\>400, :url=\>"h… --- ## [ES 7.8.0: query\_string with fields: \["\*"\] vs default\_field: "\*" performance](https://discuss.elastic.co/t/es-7-8-0-query-string-with-fields-vs-default-field-performance/241990)
**Author:** [@jojo23](https://discuss.elastic.co/u/jojo23)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 9:01am UTC](https://discuss.elastic.co/t/es-7-8-0-query-string-with-fields-vs-default-field-performance/241990 "2020-07-21T09:01:04Z")
I'm getting very different performance searching a small (\< 200MB) index with a query\_string filter, depending on how I specify the fields to be searched: fields: \["\*"\] - 1600ms consistently default\_field: "\*" - \< 100m… --- ## [Script field for search msg](https://discuss.elastic.co/t/script-field-for-search-msg/241879)
**Author:** [@Ali4](https://discuss.elastic.co/u/Ali4)\ **Replies:** 2\ **Last updated:** [July 21, 2020, 8:57am UTC](https://discuss.elastic.co/t/script-field-for-search-msg/241879 "2020-07-21T08:57:03Z")
Hello, Please i need your help. I need to create script field to search msg "error" in field message. the field message is: ''' {"@timestamp":"2020-07-20T12:14:43+02:00","@version":"1","message":" time="Jul 20 12:14… --- ## [Should we continue or stop indexing during rolling upgrades of Elastic-Search?](https://discuss.elastic.co/t/should-we-continue-or-stop-indexing-during-rolling-upgrades-of-elastic-search/240779)
**Author:** [@het](https://discuss.elastic.co/u/het)\ **Replies:** 7\ **Last updated:** [July 21, 2020, 8:56am UTC](https://discuss.elastic.co/t/should-we-continue-or-stop-indexing-during-rolling-upgrades-of-elastic-search/240779 "2020-07-21T08:56:56Z")
Hi team, Should we continue or stop indexing during rolling upgrades of Elastic-Search? I read in the upgrade documentation that it is optional steps to stop indexing and do flush sync. My query is if we do not stop in… --- ## [”\_grokparsefailure” even though the grok pattern works](https://discuss.elastic.co/t/grokparsefailure-even-though-the-grok-pattern-works/241647)
**Author:** [@Mathis\_Gourc](https://discuss.elastic.co/u/Mathis_Gourc)\ **Replies:** 4\ **Last updated:** [July 21, 2020, 8:30am UTC](https://discuss.elastic.co/t/grokparsefailure-even-though-the-grok-pattern-works/241647 "2020-07-21T08:30:35Z")
I am trying to parse different logs lines from two different type of file : slave and master. I did test my pattern in the Grok Dubugger and it is working fine but tags field in kibana is \_grokparsefailure. Here is my c… --- ## [Csv filter don't work in realtime](https://discuss.elastic.co/t/csv-filter-dont-work-in-realtime/241979)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 7:43am UTC](https://discuss.elastic.co/t/csv-filter-dont-work-in-realtime/241979 "2020-07-21T07:43:48Z")
I send a csv file with filebeat to logstash but csv filter plugin fails when it tries to parse csv with error "unclosed quoted field on line 1" If I save csv in file with logstash and with another conf I use csv filter … --- ## [How to query logstash for all incoming or outgoing records](https://discuss.elastic.co/t/how-to-query-logstash-for-all-incoming-or-outgoing-records/241977)
**Author:** [@rustyweise](https://discuss.elastic.co/u/rustyweise)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 7:40am UTC](https://discuss.elastic.co/t/how-to-query-logstash-for-all-incoming-or-outgoing-records/241977 "2020-07-21T07:40:37Z")
We are using logstash as a proxy between beats, syslog, fluentbit, etc and logz.io. I would like to inspect ('tee') the input stream, specifically from a vCSA syslog stream, or query logstash for in-memory records, or ('… --- ## [Kibana crash on re-index of own index](https://discuss.elastic.co/t/kibana-crash-on-re-index-of-own-index/236863)
**Author:** [@raulgs](https://discuss.elastic.co/u/raulgs)\ **Replies:** 5\ **Last updated:** [July 21, 2020, 7:27am UTC](https://discuss.elastic.co/t/kibana-crash-on-re-index-of-own-index/236863 "2020-07-21T07:27:07Z")
Hi this morning I found my kibana pods in non-ready state with the following log entry {"type":"log","@timestamp":"2020-06-12T08:32:29Z","tags":\["info","plugins","monitoring","monitoring"\],"pid":7,"message":"config sour… --- ## [Handle Nested data in elasticsearch](https://discuss.elastic.co/t/handle-nested-data-in-elasticsearch/241401)
**Author:** [@mathur7vidit](https://discuss.elastic.co/u/mathur7vidit)\ **Replies:** 2\ **Last updated:** [July 21, 2020, 7:04am UTC](https://discuss.elastic.co/t/handle-nested-data-in-elasticsearch/241401 "2020-07-21T07:04:08Z")
Hi All, I have a json Input file where I have a field namely error. this error is basically nested and has 3 sub-fields namely error.message, error.stacktrace and error.class. Currently i am able to use filters with the… --- ## [Aggregate distinct value based on logs in kibana](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883)
**Author:** [@Nitesh\_Chaudhary](https://discuss.elastic.co/u/Nitesh_Chaudhary)\ **Replies:** 2\ **Last updated:** [July 21, 2020, 6:55am UTC](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883 "2020-07-21T06:55:17Z")
Hi, I have logs collected in my kibana. Format is like below: {timestamp: 'July 20th 2020, 16:17:55.029', operation: 'success', 'element': '1'} {timestamp: 'July 20th 2020, 16:18:55.029', operation: 'success', 'elemen… --- ## [Help running esrally performace test on a storage SAN volume](https://discuss.elastic.co/t/help-running-esrally-performace-test-on-a-storage-san-volume/241242)
**Author:** [@dmaman](https://discuss.elastic.co/u/dmaman)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 6:41am UTC](https://discuss.elastic.co/t/help-running-esrally-performace-test-on-a-storage-san-volume/241242 "2020-07-21T06:41:59Z")
Hi, I would like to compare performance using esrally on a local SDD vs A NAS volume, how can I configure esrally to use the NAS volume? thanks, dve. --- ## [Number of shards](https://discuss.elastic.co/t/number-of-shards/241802)
**Author:** [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Replies:** 4\ **Last updated:** [July 20, 2020, 8:02am UTC](https://discuss.elastic.co/t/number-of-shards/241802 "2020-07-20T08:02:02Z")
hello I found this rule of thumb for 30-50gb per shared so, does it mean that if I have 100gb of data, I will need 2 primary shards, or 1 primary and 1 replica is good enough? thanks! --- ## [Remote Cluster Of Connection Problem With Xpack](https://discuss.elastic.co/t/remote-cluster-of-connection-problem-with-xpack/241545)
**Author:** [@3c3m](https://discuss.elastic.co/u/3c3m)\ **Replies:** 9\ **Last updated:** [July 20, 2020, 9:23am UTC](https://discuss.elastic.co/t/remote-cluster-of-connection-problem-with-xpack/241545 "2020-07-20T09:23:53Z")
Hi All, Without x-pack, I provided a seamless connection as remote clusters in Kibana. But, Xpack is enabled, remote cluster connections were broken. How can I use both together without problems? Thanks. --- ## [Elasticsearch cluster physical server](https://discuss.elastic.co/t/elasticsearch-cluster-physical-server/241970)
**Author:** [@jang](https://discuss.elastic.co/u/jang)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 6:23am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-physical-server/241970 "2020-07-21T06:23:52Z")
hi, i'm using ELK stack. I try to collect about 120 million data a day. I am considering the number of physical servers in Logstash server and Elastic Search server. I want 120 million data stored a day without error… --- ## [How to access / address nested fields on Logstash](https://discuss.elastic.co/t/how-to-access-address-nested-fields-on-logstash/241861)
**Author:** [@Yoorae\_Kim](https://discuss.elastic.co/u/Yoorae_Kim)\ **Replies:** 3\ **Last updated:** [July 21, 2020, 6:17am UTC](https://discuss.elastic.co/t/how-to-access-address-nested-fields-on-logstash/241861 "2020-07-21T06:17:18Z")
Hello, I am currently trying to convert a nested sub field that contains hexadecimal string to an int type field using Logstash filter ruby{ code =\> 'event.set("\[transactions\]\[gasprice\_int\]", event.get("\[transaction… --- ## [Kibana: unexpected character at line 1 column 1 of the JSON data](https://discuss.elastic.co/t/kibana-unexpected-character-at-line-1-column-1-of-the-json-data/239257)
**Author:** [@ade.syseng](https://discuss.elastic.co/u/ade.syseng)\ **Replies:** 6\ **Last updated:** [July 21, 2020, 6:12am UTC](https://discuss.elastic.co/t/kibana-unexpected-character-at-line-1-column-1-of-the-json-data/239257 "2020-07-21T06:12:29Z")
Hi, I'm just implemented elastic stack as a log analyzer and still learning to use it. I have a problem with kibana on a dashboard with multiple visualizations. When I'm trying to show all visualizations from 2 months … --- ## [Slowly Indexing speed](https://discuss.elastic.co/t/slowly-indexing-speed/241733)
**Author:** [@liron\_gofberg](https://discuss.elastic.co/u/liron_gofberg)\ **Replies:** 25\ **Last updated:** [July 21, 2020, 5:17am UTC](https://discuss.elastic.co/t/slowly-indexing-speed/241733 "2020-07-21T05:17:38Z")
Hi, I insert bulk into elasticsearch with c++ client each bulk contains 30,000 documents each document has 40 numeric fields indexing duration is about 13 seconds the data in each bulk is targeted to one shard in on… --- ## [Using Nested Fields in Composite Aggregation](https://discuss.elastic.co/t/using-nested-fields-in-composite-aggregation/241921)
**Author:** [@lmariselvam](https://discuss.elastic.co/u/lmariselvam)\ **Replies:** 0\ **Last updated:** [July 20, 2020, 6:03pm UTC](https://discuss.elastic.co/t/using-nested-fields-in-composite-aggregation/241921 "2020-07-20T18:03:34Z")
Basically, I'm trying to find the duplicate contacts by first name, last name & email address. For that, I've tried to use composite aggregation with the fields firstName, lastName & emails.email, the response from the q… --- ## [How to shrink an index spread over multiple nodes](https://discuss.elastic.co/t/how-to-shrink-an-index-spread-over-multiple-nodes/241814)
**Author:** [@Prash7](https://discuss.elastic.co/u/Prash7)\ **Replies:** 7\ **Last updated:** [July 21, 2020, 5:09am UTC](https://discuss.elastic.co/t/how-to-shrink-an-index-spread-over-multiple-nodes/241814 "2020-07-21T05:09:40Z")
Hey Team, How do i go about shrinking an index, which has shards over multiple nodes ? I keep getting this error { "error": { "root\_cause": \[ { "type": "remote\_transport\_exception", "reason… --- ## [How to convert EST time into UTC and add it as a field](https://discuss.elastic.co/t/how-to-convert-est-time-into-utc-and-add-it-as-a-field/241878)
**Author:** [@prabhakar\_talari](https://discuss.elastic.co/u/prabhakar_talari)\ **Replies:** 3\ **Last updated:** [July 21, 2020, 5:08am UTC](https://discuss.elastic.co/t/how-to-convert-est-time-into-utc-and-add-it-as-a-field/241878 "2020-07-21T05:08:21Z")
All, I am getting field called date with EST time, but i need to convert this date time into UTC and add it as a @timestamp through ingest pipelines. Please help me what is the correct way to do it. Thanks --- ## [Is metricbeat version related to elasticsearch?](https://discuss.elastic.co/t/is-metricbeat-version-related-to-elasticsearch/241965)
**Author:** [@CHU\_XU](https://discuss.elastic.co/u/CHU_XU)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 5:07am UTC](https://discuss.elastic.co/t/is-metricbeat-version-related-to-elasticsearch/241965 "2020-07-21T05:07:36Z")
Hi there, If I use metricbeat to monitor elasticsearch, do I need ensure they are the same version or not? For example I'm using es 7.5.2, how should I choose metricbeat version, the master or also 7.5.2? Thanks C. --- ## [How to aggregate multiple events into single output](https://discuss.elastic.co/t/how-to-aggregate-multiple-events-into-single-output/239536)
**Author:** [@pvaithee07](https://discuss.elastic.co/u/pvaithee07)\ **Replies:** 4\ **Last updated:** [July 21, 2020, 4:42am UTC](https://discuss.elastic.co/t/how-to-aggregate-multiple-events-into-single-output/239536 "2020-07-21T04:42:25Z")
Hi All - I am trying to create a logstash pipeline, that reads all the events in that pipeline, aggregates and produces one single output to elastic search. i/p looks like this - ''' { 'entity': '1212', ... }, { … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1914) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1916)