# Latest **URL:** https://discuss.elastic.co/latest.md?page=1096 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1097 --- ## [Data size on disk increase 15 times when moved from hive to elasticsearch](https://discuss.elastic.co/t/data-size-on-disk-increase-15-times-when-moved-from-hive-to-elasticsearch/305654)
**Author:** [@charvi23](https://discuss.elastic.co/u/charvi23)\ **Replies:** 26\ **Last updated:** [June 9, 2022, 1:21pm UTC](https://discuss.elastic.co/t/data-size-on-disk-increase-15-times-when-moved-from-hive-to-elasticsearch/305654 "2022-06-09T13:21:56Z")
I am using hive-hadoop jar to move data from hive to Elasticsearch. The data size on disk is ~ 35GB which when moved to ES becomes ~600GB. Is this expected behaviour? Or is there that something is missing from our end. R… --- ## [Append an array element to logs coming from Logstash](https://discuss.elastic.co/t/append-an-array-element-to-logs-coming-from-logstash/306794)
**Author:** [@dishant.sharma](https://discuss.elastic.co/u/dishant.sharma)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 12:27pm UTC](https://discuss.elastic.co/t/append-an-array-element-to-logs-coming-from-logstash/306794 "2022-06-09T12:27:19Z")
how to append an element to a particular field already containing an array? Actually, I am writing a painless script in the output part of the Logstash conf file. Code inside my conf file is: input { kafka { … --- ## [Splunk vs. Elastic: Lookup Tables](https://discuss.elastic.co/t/splunk-vs-elastic-lookup-tables/306669)
**Author:** [@alaine](https://discuss.elastic.co/u/alaine)\ **Replies:** 4\ **Last updated:** [June 9, 2022, 12:07pm UTC](https://discuss.elastic.co/t/splunk-vs-elastic-lookup-tables/306669 "2022-06-09T12:07:32Z")
Good Morning, I have been working with a customer for a while and they have chosen to migrate from Splunk to Elastic. One sticking point has been lookup tables. They had very specific lookup tables that provided them wi… --- ## [Alerting error: Can't get text on a START\_OBJECT](https://discuss.elastic.co/t/alerting-error-cant-get-text-on-a-start-object/306781)
**Author:** [@Sandeep\_Raju](https://discuss.elastic.co/u/Sandeep_Raju)\ **Replies:** 4\ **Last updated:** [June 9, 2022, 11:25am UTC](https://discuss.elastic.co/t/alerting-error-cant-get-text-on-a-start-object/306781 "2022-06-09T11:25:04Z")
Hi all, I'm trying to create an alert for a query so that it sends an email when that query condition is met. When i try to create monitor & action, when i click on create -it throws an error-: \[illegal\_state\_except… --- ## [Connection problem between Logstash and Elasticsearch running on Docker](https://discuss.elastic.co/t/connection-problem-between-logstash-and-elasticsearch-running-on-docker/306757)
**Author:** [@Francesco66](https://discuss.elastic.co/u/Francesco66)\ **Replies:** 4\ **Last updated:** [June 9, 2022, 11:24am UTC](https://discuss.elastic.co/t/connection-problem-between-logstash-and-elasticsearch-running-on-docker/306757 "2022-06-09T11:24:15Z")
Hello team, I have a Logstash that I can successfully run and start by using the following docker command: ------------------------- docker run --rm -d --name logstash-Casex -p 8080:8080 -p 8081:8081 -v /Logstash\_custom\_… --- ## [Can't etablish connection with logstash and my filebeat 7.17.1](https://discuss.elastic.co/t/cant-etablish-connection-with-logstash-and-my-filebeat-7-17-1/306722)
**Author:** [@Dreinale](https://discuss.elastic.co/u/Dreinale)\ **Replies:** 5\ **Last updated:** [June 9, 2022, 11:15am UTC](https://discuss.elastic.co/t/cant-etablish-connection-with-logstash-and-my-filebeat-7-17-1/306722 "2022-06-09T11:15:48Z")
Hi all, I want to do a monitoring of my log but i don't understand why my logstash doesn't"t work with filebeat. For now i have Elasticsearch: Elasticsearch.yml node.name: master-node-1 node.master: true cluster.init… --- ## [Logstash Docker on GCP Cloud Run: Expected the service InnerService \[FAILED\] to be TERMINATED, but the service has FAILED](https://discuss.elastic.co/t/logstash-docker-on-gcp-cloud-run-expected-the-service-innerservice-failed-to-be-terminated-but-the-service-has-failed/306785)
**Author:** [@hegdesandesh25](https://discuss.elastic.co/u/hegdesandesh25)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 10:42am UTC](https://discuss.elastic.co/t/logstash-docker-on-gcp-cloud-run-expected-the-service-innerservice-failed-to-be-terminated-but-the-service-has-failed/306785 "2022-06-09T10:42:36Z")
Continuing the discussion from Logstash Docker on GCP Cloud Run: Failed to find a usable hardware address from the network interfaces: I have a similar use-case, running custom logstash container from gcloud-run with th… --- ## [Permissions for Move to lifecycle step API wrong in docs?](https://discuss.elastic.co/t/permissions-for-move-to-lifecycle-step-api-wrong-in-docs/304691)
**Author:** [@nisow95612](https://discuss.elastic.co/u/nisow95612)\ **Replies:** 1\ **Last updated:** [June 9, 2022, 10:23am UTC](https://discuss.elastic.co/t/permissions-for-move-to-lifecycle-step-api-wrong-in-docs/304691 "2022-06-09T10:23:29Z")
Hello nice people of elastic, I am confused about the permissions required to use the Move to lifecycle step API | Elasticsearch Guide \[8.2\] | Elastic The docs say you must have the manage\_ilm privileges on the indices … --- ## [Set index setting like unassigned.node\_left timeout from ILM](https://discuss.elastic.co/t/set-index-setting-like-unassigned-node-left-timeout-from-ilm/306782)
**Author:** [@nisow95612](https://discuss.elastic.co/u/nisow95612)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 10:22am UTC](https://discuss.elastic.co/t/set-index-setting-like-unassigned-node-left-timeout-from-ilm/306782 "2022-06-09T10:22:57Z")
Hello, I wonder, is there a way to apply settings to indices when transitioning betwen ILM phases? My motivation is that I would like my cold indices to have longer index.unassigned.node\_left.delayed\_timeout than the h… --- ## [How to sort buckets in Elasticsearch on their first occurrence in a search](https://discuss.elastic.co/t/how-to-sort-buckets-in-elasticsearch-on-their-first-occurrence-in-a-search/306776)
**Author:** [@Peter\_Dewachtere](https://discuss.elastic.co/u/Peter_Dewachtere)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 9:24am UTC](https://discuss.elastic.co/t/how-to-sort-buckets-in-elasticsearch-on-their-first-occurrence-in-a-search/306776 "2022-06-09T09:24:29Z")
I have an Elasticsearch query (version 7) that returns a paged result of products. The query has a specific sort on multiple fields: Product A - image 1 Product B - image 1 Product C - image 2 Product D - image 2 Prod… --- ## [Why does elasticsearch assign shards to cold nodes?](https://discuss.elastic.co/t/why-does-elasticsearch-assign-shards-to-cold-nodes/306774)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 9:04am UTC](https://discuss.elastic.co/t/why-does-elasticsearch-assign-shards-to-cold-nodes/306774 "2022-06-09T09:04:19Z")
elk: 7.17.3 01-03 nodes I want to use for hot data search and storage and 04-05 nodes as cold and frozen nodes. I have read the es documentation. By default, the data of the collector will be sent to the data\_content n… --- ## [Unable to restore snapshot](https://discuss.elastic.co/t/unable-to-restore-snapshot/306771)
**Author:** [@noob2](https://discuss.elastic.co/u/noob2)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 8:53am UTC](https://discuss.elastic.co/t/unable-to-restore-snapshot/306771 "2022-06-09T08:53:44Z")
I logged in as admin user , still not able to restore snapshot from aws s3 , while creating snapshot , the response was "Success". while restoring the snapshot, i get the following response, { "error" : { "root… --- ## [Cannot index event publisher.Event](https://discuss.elastic.co/t/cannot-index-event-publisher-event/306420)
**Author:** [@francescouk](https://discuss.elastic.co/u/francescouk)\ **Replies:** 1\ **Last updated:** [June 9, 2022, 8:44am UTC](https://discuss.elastic.co/t/cannot-index-event-publisher-event/306420 "2022-06-09T08:44:07Z")
Hi there, I got this setup using misp integration with elastic-agent and looks fine apart from there´s no data on the dashboard page. I can see on the data stream that indeed data is coming but no parsed? Maybe?: And… --- ## [Pull data from Jaeger API in to Elastic Cloud APM v8?](https://discuss.elastic.co/t/pull-data-from-jaeger-api-in-to-elastic-cloud-apm-v8/306088)
**Author:** [@Bevaldss](https://discuss.elastic.co/u/Bevaldss)\ **Replies:** 1\ **Last updated:** [June 9, 2022, 7:42am UTC](https://discuss.elastic.co/t/pull-data-from-jaeger-api-in-to-elastic-cloud-apm-v8/306088 "2022-06-09T07:42:16Z")
Hi Did anyone try to "pull" Jaeger data from Jaeger API, in to Elastic Cloud APM? (not push from Jaeger in to Elastic Cloud APM, instead: "pull") If so, could you please share some advice for how to do that in a most p… --- ## [Elastic Search nodes spitting Unrecognized SSL message error](https://discuss.elastic.co/t/elastic-search-nodes-spitting-unrecognized-ssl-message-error/306720)
**Author:** [@krishnav1](https://discuss.elastic.co/u/krishnav1)\ **Replies:** 3\ **Last updated:** [June 9, 2022, 7:34am UTC](https://discuss.elastic.co/t/elastic-search-nodes-spitting-unrecognized-ssl-message-error/306720 "2022-06-09T07:34:43Z")
Hello, I have Elasticsearch installed along with Fluentbit and Kibana in our kubernetes cluster. Of late, We have a heavy loss of logs from the kubernetes nodes to the elastic indices and there by Kibana. I have been n… --- ## [Need help with special character search](https://discuss.elastic.co/t/need-help-with-special-character-search/306692)
**Author:** [@Kapil.Lanjewar](https://discuss.elastic.co/u/Kapil.Lanjewar)\ **Replies:** 8\ **Last updated:** [June 9, 2022, 7:12am UTC](https://discuss.elastic.co/t/need-help-with-special-character-search/306692 "2022-06-09T07:12:14Z")
I have difficulties searching the special characters from my documents. One of the attribute of the document - "phrase" - contains special characters. Its value is -\> "The 2 QUICK Brown-Foxes jumped over the lazy dog's… --- ## [Snapshot Estimate](https://discuss.elastic.co/t/snapshot-estimate/306742)
**Author:** [@A\_j\_xx](https://discuss.elastic.co/u/A_j_xx)\ **Replies:** 0\ **Last updated:** [June 8, 2022, 9:50pm UTC](https://discuss.elastic.co/t/snapshot-estimate/306742 "2022-06-08T21:50:55Z")
Hi all , In order to take snapshot of index whose size is around 2 TB , how should i set { Chunk size max\_restore\_bytes\_per\_sec max\_snapshot\_bytes\_per\_sec } these parameters to get optimal and maximum performance … --- ## [Active - Active Cross Cluster Replication](https://discuss.elastic.co/t/active-active-cross-cluster-replication/306298)
**Author:** [@cagdasalagoz](https://discuss.elastic.co/u/cagdasalagoz)\ **Replies:** 4\ **Last updated:** [June 9, 2022, 6:30am UTC](https://discuss.elastic.co/t/active-active-cross-cluster-replication/306298 "2022-06-09T06:30:03Z")
Hello World! I am trying to come up with a configuration for disaster recovery for our elastic stack. We have two data-centers(DC1 and DC2) in separate cities and both will be up and running at the same time. If a prob… --- ## [Appending data to existing index](https://discuss.elastic.co/t/appending-data-to-existing-index/306533)
**Author:** [@behappy\_alwayz0401](https://discuss.elastic.co/u/behappy_alwayz0401)\ **Replies:** 2\ **Last updated:** [June 9, 2022, 6:23am UTC](https://discuss.elastic.co/t/appending-data-to-existing-index/306533 "2022-06-09T06:23:40Z")
I have an index that already has data. One of the lines in the CSV that was uploaded, had location field which reflects the co-ordinates for Vietname, showing null values due to which that particular line from the CSV d… --- ## [$ in Top N graph of TSVB](https://discuss.elastic.co/t/in-top-n-graph-of-tsvb/306666)
**Author:** [@behappy\_alwayz0401](https://discuss.elastic.co/u/behappy_alwayz0401)\ **Replies:** 2\ **Last updated:** [June 9, 2022, 6:04am UTC](https://discuss.elastic.co/t/in-top-n-graph-of-tsvb/306666 "2022-06-09T06:04:22Z")
Hi, I have a run time field "Abs Delta Value" which has format as $0,0.\[000\]. The $ sign is showing on all charts in lens and elsewhere except on TSVB. In the "Top N" chart under TSVB, it is only showing the number wi… --- ## [Docker Logstash Error: Cannot assign requested address](https://discuss.elastic.co/t/docker-logstash-error-cannot-assign-requested-address/306711)
**Author:** [@Francesco66](https://discuss.elastic.co/u/Francesco66)\ **Replies:** 2\ **Last updated:** [June 9, 2022, 5:42am UTC](https://discuss.elastic.co/t/docker-logstash-error-cannot-assign-requested-address/306711 "2022-06-09T05:42:23Z")
Hello , I am running a Logstash as Docker container using the following command: .......... docker run --rm -d --name logstash-Casex -p 8080:8080 -p 8081:8081 -v /Logstash\_custom\_configs/config/pipelines.yml:/usr/share/l… --- ## [Index pattern doesn't have a UUID](https://discuss.elastic.co/t/index-pattern-doesnt-have-a-uuid/306596)
**Author:** [@Mary2022](https://discuss.elastic.co/u/Mary2022)\ **Replies:** 3\ **Last updated:** [June 9, 2022, 5:02am UTC](https://discuss.elastic.co/t/index-pattern-doesnt-have-a-uuid/306596 "2022-06-09T05:02:46Z")
Hi, I exported all the objects (index patterns, visualizations, dashboards, etc.) to my new cluster. I need to change the index pattern to our visualizations. I noticed the old index pattern url doesn't have a UUID in i… --- ## [Memory leakage of Vega visualization when refreshing data](https://discuss.elastic.co/t/memory-leakage-of-vega-visualization-when-refreshing-data/306751)
**Author:** [@kyh119](https://discuss.elastic.co/u/kyh119)\ **Replies:** 1\ **Last updated:** [June 9, 2022, 4:59am UTC](https://discuss.elastic.co/t/memory-leakage-of-vega-visualization-when-refreshing-data/306751 "2022-06-09T04:59:20Z")
Hi. I created a dashboard with my Vega visualizations that display the latest data. It's kind of real-time monitoring dashboard so I have to refresh every certain period. When it refreshes every 10 seconds, the heap s… --- ## [Can i recover cert folder in elastic search?](https://discuss.elastic.co/t/can-i-recover-cert-folder-in-elastic-search/306602)
**Author:** [@akshay\_bhardwaj](https://discuss.elastic.co/u/akshay_bhardwaj)\ **Replies:** 6\ **Last updated:** [June 9, 2022, 4:22am UTC](https://discuss.elastic.co/t/can-i-recover-cert-folder-in-elastic-search/306602 "2022-06-09T04:22:44Z")
By mistake i deleted the cert folder present in config folder of Elasticsearch is there any command so i can recover below certificates:- transport.p12 http.p12 --- ## [Elasticsearch storage full after activate endpoint security integration](https://discuss.elastic.co/t/elasticsearch-storage-full-after-activate-endpoint-security-integration/306754)
**Author:** [@ads92](https://discuss.elastic.co/u/ads92)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 3:55am UTC](https://discuss.elastic.co/t/elasticsearch-storage-full-after-activate-endpoint-security-integration/306754 "2022-06-09T03:55:48Z")
I have 20 agent running that has the 'Endpoint Security', 'Prebuilt Security Detection Rules' and the 'System Integration'. This is running on a Windows 10/11 & Ubuntu 20.04 and reporting into ECK with fleet management … --- ## [GeoIP filter does not work with a subfield](https://discuss.elastic.co/t/geoip-filter-does-not-work-with-a-subfield/306750)
**Author:** [@austin0918](https://discuss.elastic.co/u/austin0918)\ **Replies:** 2\ **Last updated:** [June 9, 2022, 3:20am UTC](https://discuss.elastic.co/t/geoip-filter-does-not-work-with-a-subfield/306750 "2022-06-09T03:20:16Z")
I have a field client.ip that needs to get geo info. I tried below GeoIP filter but it didn't work. Looks like GeoIP does not work with a subfield. Please advise. filter { if \[client\]\[ip\] { geoip { source =\> "\[cli… --- ## [Line and barchart on same visualization for one sample point makes the linechart a single point](https://discuss.elastic.co/t/line-and-barchart-on-same-visualization-for-one-sample-point-makes-the-linechart-a-single-point/306748)
**Author:** [@tee101](https://discuss.elastic.co/u/tee101)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 12:59am UTC](https://discuss.elastic.co/t/line-and-barchart-on-same-visualization-for-one-sample-point-makes-the-linechart-a-single-point/306748 "2022-06-09T00:59:43Z")
In the following snapshot metric U is a barchart and metric A is a linechart. If there is one sample point, the linechart is just a point, and gets immersed in a big wide bar. Is there a way to have the point extrapolate… --- ## [Recommended resource setting for a Logstash Pod](https://discuss.elastic.co/t/recommended-resource-setting-for-a-logstash-pod/306747)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 12:48am UTC](https://discuss.elastic.co/t/recommended-resource-setting-for-a-logstash-pod/306747 "2022-06-09T00:48:26Z")
Is there any recommended resource setting for a Logstash Pod handling small amount(100GB) of data everyday and no filtering in place? Logstash just reads data from SQLServer and pushes it to Elasticsearch without any fil… --- ## [How to query multiple fileds in Kibana](https://discuss.elastic.co/t/how-to-query-multiple-fileds-in-kibana/306611)
**Author:** [@dev9](https://discuss.elastic.co/u/dev9)\ **Replies:** 22\ **Last updated:** [June 9, 2022, 12:41am UTC](https://discuss.elastic.co/t/how-to-query-multiple-fileds-in-kibana/306611 "2022-06-09T00:41:10Z")
Hi, I am trying to query kibana for multiple fields where the servicename is serviceworker and the correlationId is one of many. My understanding is using terms is the best option here but I cannot get this query to wo… --- ## [Elastic search snapshot size estimate](https://discuss.elastic.co/t/elastic-search-snapshot-size-estimate/306553)
**Author:** [@A\_j\_xx](https://discuss.elastic.co/u/A_j_xx)\ **Replies:** 0\ **Last updated:** [June 7, 2022, 10:39am UTC](https://discuss.elastic.co/t/elastic-search-snapshot-size-estimate/306553 "2022-06-07T10:39:27Z")
Continuing the discussion from Elasticsearch Snapshot repository size estimates: Continuing the discussion can someone with experience comment on this ? How much memory should I have assuming my data grows from 200 gb … [Previous page](https://discuss.elastic.co/latest.md?page=1095) [Next page](https://discuss.elastic.co/latest.md?page=1097)