# \#none **URL:** https://discuss.elastic.co/tag/none.md?page=1321 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1322 --- ## [Handle Nested data in elasticsearch](https://discuss.elastic.co/t/handle-nested-data-in-elasticsearch/241401)
**Author:** [@mathur7vidit](https://discuss.elastic.co/u/mathur7vidit)\ **Replies:** 2\ **Last updated:** [July 21, 2020, 7:04am UTC](https://discuss.elastic.co/t/handle-nested-data-in-elasticsearch/241401 "2020-07-21T07:04:08Z")
Hi All, I have a json Input file where I have a field namely error. this error is basically nested and has 3 sub-fields namely error.message, error.stacktrace and error.class. Currently i am able to use filters with the… --- ## [Aggregate distinct value based on logs in kibana](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883)
**Author:** [@Nitesh\_Chaudhary](https://discuss.elastic.co/u/Nitesh_Chaudhary)\ **Replies:** 2\ **Last updated:** [July 21, 2020, 6:55am UTC](https://discuss.elastic.co/t/aggregate-distinct-value-based-on-logs-in-kibana/241883 "2020-07-21T06:55:17Z")
Hi, I have logs collected in my kibana. Format is like below: {timestamp: 'July 20th 2020, 16:17:55.029', operation: 'success', 'element': '1'} {timestamp: 'July 20th 2020, 16:18:55.029', operation: 'success', 'elemen… --- ## [Number of shards](https://discuss.elastic.co/t/number-of-shards/241802)
**Author:** [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Replies:** 4\ **Last updated:** [July 20, 2020, 8:02am UTC](https://discuss.elastic.co/t/number-of-shards/241802 "2020-07-20T08:02:02Z")
hello I found this rule of thumb for 30-50gb per shared so, does it mean that if I have 100gb of data, I will need 2 primary shards, or 1 primary and 1 replica is good enough? thanks! --- ## [Remote Cluster Of Connection Problem With Xpack](https://discuss.elastic.co/t/remote-cluster-of-connection-problem-with-xpack/241545)
**Author:** [@3c3m](https://discuss.elastic.co/u/3c3m)\ **Replies:** 9\ **Last updated:** [July 20, 2020, 9:23am UTC](https://discuss.elastic.co/t/remote-cluster-of-connection-problem-with-xpack/241545 "2020-07-20T09:23:53Z")
Hi All, Without x-pack, I provided a seamless connection as remote clusters in Kibana. But, Xpack is enabled, remote cluster connections were broken. How can I use both together without problems? Thanks. --- ## [Elasticsearch cluster physical server](https://discuss.elastic.co/t/elasticsearch-cluster-physical-server/241970)
**Author:** [@jang](https://discuss.elastic.co/u/jang)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 6:23am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-physical-server/241970 "2020-07-21T06:23:52Z")
hi, i'm using ELK stack. I try to collect about 120 million data a day. I am considering the number of physical servers in Logstash server and Elastic Search server. I want 120 million data stored a day without error… --- ## [How to access / address nested fields on Logstash](https://discuss.elastic.co/t/how-to-access-address-nested-fields-on-logstash/241861)
**Author:** [@Yoorae\_Kim](https://discuss.elastic.co/u/Yoorae_Kim)\ **Replies:** 3\ **Last updated:** [July 21, 2020, 6:17am UTC](https://discuss.elastic.co/t/how-to-access-address-nested-fields-on-logstash/241861 "2020-07-21T06:17:18Z")
Hello, I am currently trying to convert a nested sub field that contains hexadecimal string to an int type field using Logstash filter ruby{ code =\> 'event.set("\[transactions\]\[gasprice\_int\]", event.get("\[transaction… --- ## [Kibana: unexpected character at line 1 column 1 of the JSON data](https://discuss.elastic.co/t/kibana-unexpected-character-at-line-1-column-1-of-the-json-data/239257)
**Author:** [@ade.syseng](https://discuss.elastic.co/u/ade.syseng)\ **Replies:** 6\ **Last updated:** [July 21, 2020, 6:12am UTC](https://discuss.elastic.co/t/kibana-unexpected-character-at-line-1-column-1-of-the-json-data/239257 "2020-07-21T06:12:29Z")
Hi, I'm just implemented elastic stack as a log analyzer and still learning to use it. I have a problem with kibana on a dashboard with multiple visualizations. When I'm trying to show all visualizations from 2 months … --- ## [Slowly Indexing speed](https://discuss.elastic.co/t/slowly-indexing-speed/241733)
**Author:** [@liron\_gofberg](https://discuss.elastic.co/u/liron_gofberg)\ **Replies:** 25\ **Last updated:** [July 21, 2020, 5:17am UTC](https://discuss.elastic.co/t/slowly-indexing-speed/241733 "2020-07-21T05:17:38Z")
Hi, I insert bulk into elasticsearch with c++ client each bulk contains 30,000 documents each document has 40 numeric fields indexing duration is about 13 seconds the data in each bulk is targeted to one shard in on… --- ## [Using Nested Fields in Composite Aggregation](https://discuss.elastic.co/t/using-nested-fields-in-composite-aggregation/241921)
**Author:** [@lmariselvam](https://discuss.elastic.co/u/lmariselvam)\ **Replies:** 0\ **Last updated:** [July 20, 2020, 6:03pm UTC](https://discuss.elastic.co/t/using-nested-fields-in-composite-aggregation/241921 "2020-07-20T18:03:34Z")
Basically, I'm trying to find the duplicate contacts by first name, last name & email address. For that, I've tried to use composite aggregation with the fields firstName, lastName & emails.email, the response from the q… --- ## [How to shrink an index spread over multiple nodes](https://discuss.elastic.co/t/how-to-shrink-an-index-spread-over-multiple-nodes/241814)
**Author:** [@Prash7](https://discuss.elastic.co/u/Prash7)\ **Replies:** 7\ **Last updated:** [July 21, 2020, 5:09am UTC](https://discuss.elastic.co/t/how-to-shrink-an-index-spread-over-multiple-nodes/241814 "2020-07-21T05:09:40Z")
Hey Team, How do i go about shrinking an index, which has shards over multiple nodes ? I keep getting this error { "error": { "root\_cause": \[ { "type": "remote\_transport\_exception", "reason… --- ## [How to convert EST time into UTC and add it as a field](https://discuss.elastic.co/t/how-to-convert-est-time-into-utc-and-add-it-as-a-field/241878)
**Author:** [@prabhakar\_talari](https://discuss.elastic.co/u/prabhakar_talari)\ **Replies:** 3\ **Last updated:** [July 21, 2020, 5:08am UTC](https://discuss.elastic.co/t/how-to-convert-est-time-into-utc-and-add-it-as-a-field/241878 "2020-07-21T05:08:21Z")
All, I am getting field called date with EST time, but i need to convert this date time into UTC and add it as a @timestamp through ingest pipelines. Please help me what is the correct way to do it. Thanks --- ## [How to aggregate multiple events into single output](https://discuss.elastic.co/t/how-to-aggregate-multiple-events-into-single-output/239536)
**Author:** [@pvaithee07](https://discuss.elastic.co/u/pvaithee07)\ **Replies:** 4\ **Last updated:** [July 21, 2020, 4:42am UTC](https://discuss.elastic.co/t/how-to-aggregate-multiple-events-into-single-output/239536 "2020-07-21T04:42:25Z")
Hi All - I am trying to create a logstash pipeline, that reads all the events in that pipeline, aggregates and produces one single output to elastic search. i/p looks like this - ''' { 'entity': '1212', ... }, { … --- ## [Scoring when using wildcard search in query\_string](https://discuss.elastic.co/t/scoring-when-using-wildcard-search-in-query-string/241964)
**Author:** [@vangap](https://discuss.elastic.co/u/vangap)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 4:29am UTC](https://discuss.elastic.co/t/scoring-when-using-wildcard-search-in-query-string/241964 "2020-07-21T04:29:30Z")
Hi, This may be partly a general FTS question If I have two tokens like "abc" and "abc123" And user searches for "abc", in general is "abc" considered more relevant or "abc123"? Is there any deterministic ranking by… --- ## [Input file path wildcard is not working](https://discuss.elastic.co/t/input-file-path-wildcard-is-not-working/241617)
**Author:** [@mmk1995](https://discuss.elastic.co/u/mmk1995)\ **Replies:** 3\ **Last updated:** [July 21, 2020, 4:16am UTC](https://discuss.elastic.co/t/input-file-path-wildcard-is-not-working/241617 "2020-07-21T04:16:25Z")
Hello all, input file path wildcard is not working, As I remember, logstash 6.5.4 don't have the proble, while currently 7.6.2 encounter this problem, what could be the reason? Not working: input { file { … --- ## [Pull data periodically from an Elasticsearch index](https://discuss.elastic.co/t/pull-data-periodically-from-an-elasticsearch-index/241962)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 3:52am UTC](https://discuss.elastic.co/t/pull-data-periodically-from-an-elasticsearch-index/241962 "2020-07-21T03:52:07Z")
Hi, We have some log indices and now there is a requirement to extract some pieces of information from them and put them in another index where we can run some machine learning jobs on them. Initally I was thinking of … --- ## [Json\_lines codec is not able to parse certain jsons correctly](https://discuss.elastic.co/t/json-lines-codec-is-not-able-to-parse-certain-jsons-correctly/241593)
**Author:** [@buch11](https://discuss.elastic.co/u/buch11)\ **Replies:** 3\ **Last updated:** [July 21, 2020, 3:56am UTC](https://discuss.elastic.co/t/json-lines-codec-is-not-able-to-parse-certain-jsons-correctly/241593 "2020-07-21T03:56:51Z")
I am using json\_lines codec along with http\_input in my pipeline. One of the elements in my JSON is querystring. Usually, this codec is able to handle all the variants of valid jsons except below "querystring": { … --- ## [Logstash 7.8 doesn't re-parse existing file](https://discuss.elastic.co/t/logstash-7-8-doesnt-re-parse-existing-file/240669)
**Author:** [@Alex\_Liu](https://discuss.elastic.co/u/Alex_Liu)\ **Replies:** 7\ **Last updated:** [July 21, 2020, 3:34am UTC](https://discuss.elastic.co/t/logstash-7-8-doesnt-re-parse-existing-file/240669 "2020-07-21T03:34:53Z")
Hi team, I'm using Elastic Stack 7.8, below logstash doesn't read old files in /usr/data/nsg/ folder. input { file { path =\> "/usr/data/nsg/\*" start\_position =\> "beginning" codec =\> "json" … --- ## [How to partition response time low25% mid50% top25%](https://discuss.elastic.co/t/how-to-partition-response-time-low25-mid50-top25/241959)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 2\ **Last updated:** [July 21, 2020, 3:06am UTC](https://discuss.elastic.co/t/how-to-partition-response-time-low25-mid50-top25/241959 "2020-07-21T03:06:25Z")
hi community there is one report based on response time (req~resp) like this response time Lowest 25% and it's min/max values response time Median 50% and it's min/max values response time Top 25% and it's min/max va… --- ## [Performance of running ELK Stack on Docker or VM?](https://discuss.elastic.co/t/performance-of-running-elk-stack-on-docker-or-vm/241595)
**Author:** [@eva\_cch](https://discuss.elastic.co/u/eva_cch)\ **Replies:** 2\ **Last updated:** [July 21, 2020, 2:38am UTC](https://discuss.elastic.co/t/performance-of-running-elk-stack-on-docker-or-vm/241595 "2020-07-21T02:38:49Z")
I have tried running ELK Stack on Docker in a CentOS 7 VM with 3 different Docker containers - Elasticsearch, Logstash and Kibana containers. It took me about an hour to run 625MB as per screenshot below whereby I have a… --- ## [Agent status as a time series data](https://discuss.elastic.co/t/agent-status-as-a-time-series-data/241936)
**Author:** [@madhan\_dc](https://discuss.elastic.co/u/madhan_dc)\ **Replies:** 7\ **Last updated:** [July 21, 2020, 2:23am UTC](https://discuss.elastic.co/t/agent-status-as-a-time-series-data/241936 "2020-07-21T02:23:45Z")
Newbie question. I am exploring elastic for our needs. We have a lot of filemonitor agents. these agents can go down sometimes and we want to monitor this via Kibana my understanding, i need to save the status of the ag… --- ## [Logstash Connectivity Issue with Elastic Search on Elastic Cloud](https://discuss.elastic.co/t/logstash-connectivity-issue-with-elastic-search-on-elastic-cloud/241585)
**Author:** [@dhirajagarwal2010](https://discuss.elastic.co/u/dhirajagarwal2010)\ **Replies:** 3\ **Last updated:** [July 21, 2020, 12:51am UTC](https://discuss.elastic.co/t/logstash-connectivity-issue-with-elastic-search-on-elastic-cloud/241585 "2020-07-21T00:51:25Z")
I am not able to connect my logstash instance on my local server with Elastic Search on Elastic Cloud . The user tag in logstash encodes the @ symbol in the email address twice to %2540. I guess because of this Elastic C… --- ## [Error response in search](https://discuss.elastic.co/t/error-response-in-search/241447)
**Author:** [@Sagar\_jain](https://discuss.elastic.co/u/Sagar_jain)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 12:17am UTC](https://discuss.elastic.co/t/error-response-in-search/241447 "2020-07-21T00:17:57Z")
I am not getting the below error. string(738) "{"error":{"root\_cause":,"type":"search\_phase\_execution\_exception","reason":"", "phase":"fetch","grouped":true,"failed\_shards":,"caused\_by":{"type":"es\_rejected\_execution\_e… --- ## [Load dashboard in Kibana 5.6](https://discuss.elastic.co/t/load-dashboard-in-kibana-5-6/241639)
**Author:** [@Cesar\_Augusto](https://discuss.elastic.co/u/Cesar_Augusto)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 12:12am UTC](https://discuss.elastic.co/t/load-dashboard-in-kibana-5-6/241639 "2020-07-21T00:12:39Z")
Kibana version: 5.6.16 Elasticsearch version: 5.6.16 APM Server version: 6.8 or 7.1 (It's possible change if necessary) Steps to reproduce: Install Kibana 5.6 Install Elastic 5.6 Install APM 6.8 Problem: With the … --- ## [Helping with red shards](https://discuss.elastic.co/t/helping-with-red-shards/241748)
**Author:** [@AayushPatel](https://discuss.elastic.co/u/AayushPatel)\ **Replies:** 1\ **Last updated:** [July 21, 2020, 12:09am UTC](https://discuss.elastic.co/t/helping-with-red-shards/241748 "2020-07-21T00:09:19Z")
Hi I have elastic on 1 node and tried to reindex but it maked my node health red and after i delete the destination index to free space all of my shards get red and do not get back to run elastic the data is important … --- ## [Fluentd to External ELK stack](https://discuss.elastic.co/t/fluentd-to-external-elk-stack/241880)
**Author:** [@Lilth](https://discuss.elastic.co/u/Lilth)\ **Replies:** 1\ **Last updated:** [July 20, 2020, 11:58pm UTC](https://discuss.elastic.co/t/fluentd-to-external-elk-stack/241880 "2020-07-20T23:58:35Z")
Hello, I have a problem with Logstash. FluentD is sending logs to external ELK stack. Whichever option I have tried logstash is not creating index in ES. Config: input { tcp { port =\> codec =\> fluent } } outpu… --- ## [ElasticSearch upgrade 6.7 to 7.8](https://discuss.elastic.co/t/elasticsearch-upgrade-6-7-to-7-8/241863)
**Author:** [@starwin\_stephen](https://discuss.elastic.co/u/starwin_stephen)\ **Replies:** 1\ **Last updated:** [July 20, 2020, 11:48pm UTC](https://discuss.elastic.co/t/elasticsearch-upgrade-6-7-to-7-8/241863 "2020-07-20T23:48:26Z")
Recently i have upgraded from 6.7 to 7.8 in a single node server.Now i am facing below error while trying to clear/reindex from the application server. \*\*\[error\]\*\* \*\*Message: {"error":{"root\_cause":\[{"type":"illegal\_a… --- ## [Importing Historical Log Data](https://discuss.elastic.co/t/importing-historical-log-data/241913)
**Author:** [@phinkel](https://discuss.elastic.co/u/phinkel)\ **Replies:** 1\ **Last updated:** [July 20, 2020, 11:41pm UTC](https://discuss.elastic.co/t/importing-historical-log-data/241913 "2020-07-20T23:41:58Z")
Hello currently we are transitioning to Elastic from LogRhythm SIEM. The current logs from LR are in .LCA format. Due to compliance reasons we will need to have access to this historical log data. I assume as LR is built… --- ## [What Value of ParallelGCThreads should I set](https://discuss.elastic.co/t/what-value-of-parallelgcthreads-should-i-set/241940)
**Author:** [@khzied](https://discuss.elastic.co/u/khzied)\ **Replies:** 1\ **Last updated:** [July 20, 2020, 11:38pm UTC](https://discuss.elastic.co/t/what-value-of-parallelgcthreads-should-i-set/241940 "2020-07-20T23:38:37Z")
Hello, I have 9 servers Elasticsearch: 3 master and 6 datas. These 9 servers are identical:64 GB RAM, 8 CPU (each cpu has 4 cpu cores). Which correct value of ParallelGCThreads should I set in java.options. My Elasti… --- ## [Integrate Google Analytics with Kibana](https://discuss.elastic.co/t/integrate-google-analytics-with-kibana/241882)
**Author:** [@Tango31](https://discuss.elastic.co/u/Tango31)\ **Replies:** 2\ **Last updated:** [July 20, 2020, 10:41pm UTC](https://discuss.elastic.co/t/integrate-google-analytics-with-kibana/241882 "2020-07-20T22:41:41Z")
Hello, I want to see google analytics data into kibana dashboard..How to do it?..Can anyone help me on this topic? --- ## [Flush elasticsearch filter in output](https://discuss.elastic.co/t/flush-elasticsearch-filter-in-output/241825)
**Author:** [@Charlie\_Liang](https://discuss.elastic.co/u/Charlie_Liang)\ **Replies:** 0\ **Last updated:** [July 20, 2020, 2:13am UTC](https://discuss.elastic.co/t/flush-elasticsearch-filter-in-output/241825 "2020-07-20T02:13:25Z")
does pipeline.batch.delay control elasticsearch in output? it seems to be working on plugins in filter section. I've set the value to 5, but not able to see indices got updated in a min. Are there other controls for ela… [Previous page](https://discuss.elastic.co/tag/none.md?no_tags=true&page=1320) [Next page](https://discuss.elastic.co/tag/none.md?no_tags=true&page=1322)