# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=933 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 934 --- ## [How to store huge amount of unstructured data](https://discuss.elastic.co/t/how-to-store-huge-amount-of-unstructured-data/304530)
**Author:** [@Naga\_Prudhvi](https://discuss.elastic.co/u/Naga_Prudhvi)\ **Replies:** 2\ **Last updated:** [May 19, 2022, 6:18pm UTC](https://discuss.elastic.co/t/how-to-store-huge-amount-of-unstructured-data/304530 "2022-05-19T18:18:18Z")
How can i store huge amount of unstructured data for example social media chat, email, website data Please help me with storing below sample email and ideas on storing social media chat. Hi John. Thank you so much fo… --- ## [How to delete the data stream (UNASSIGNED) .fleet-actions-results?](https://discuss.elastic.co/t/how-to-delete-the-data-stream-unassigned-fleet-actions-results/304974)
**Author:** [@ajj31](https://discuss.elastic.co/u/ajj31)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 5:52pm UTC](https://discuss.elastic.co/t/how-to-delete-the-data-stream-unassigned-fleet-actions-results/304974 "2022-05-19T17:52:56Z")
sudo curl --USER elastic:xxxxxxxxxxx -X GET https://localhost:9200/\_cat/shards?h=index,shard,prirep,state,unassigned.reason --insecure | grep UNASSIGNED % Total % Received % Xferd Average Speed Time Time Ti… --- ## [Osquery Manager does not return results from curl table](https://discuss.elastic.co/t/osquery-manager-does-not-return-results-from-curl-table/305019)
**Author:** [@hwang](https://discuss.elastic.co/u/hwang)\ **Replies:** 7\ **Last updated:** [May 19, 2022, 5:28pm UTC](https://discuss.elastic.co/t/osquery-manager-does-not-return-results-from-curl-table/305019 "2022-05-19T17:28:33Z")
Hi Elastic team, Running a query from the curl table returns an empty result. I've confirmed that other queries work and curl\_certificates work as well. SELECT \* FROM curl WHERE url = 'https://www.google.com' Is there… --- ## [Machine learning can't detect changed pattern in log rate](https://discuss.elastic.co/t/machine-learning-cant-detect-changed-pattern-in-log-rate/305163)
**Author:** [@marone](https://discuss.elastic.co/u/marone)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 5:04pm UTC](https://discuss.elastic.co/t/machine-learning-cant-detect-changed-pattern-in-log-rate/305163 "2022-05-19T17:04:35Z")
Hello, I created a machine learning job using the API, I'm interested in high and low log rate for kubernetes containers, and we are trying to detect those unusual low/high log rate when they occur. Here is the job ML d… --- ## [Issue searching for first\_name and last\_name when there is a compound name](https://discuss.elastic.co/t/issue-searching-for-first-name-and-last-name-when-there-is-a-compound-name/305124)
**Author:** [@celo1](https://discuss.elastic.co/u/celo1)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 4:39pm UTC](https://discuss.elastic.co/t/issue-searching-for-first-name-and-last-name-when-there-is-a-compound-name/305124 "2022-05-19T16:39:57Z")
Hi, I am newbie to ES and I am trying to create a search as you type experience that will find clients by their first name, last name, email and phone number. I am using the code below but it is not working as expected w… --- ## [Cannot setup dashboards filebeat to kibana](https://discuss.elastic.co/t/cannot-setup-dashboards-filebeat-to-kibana/305115)
**Author:** [@francescouk](https://discuss.elastic.co/u/francescouk)\ **Replies:** 4\ **Last updated:** [May 19, 2022, 3:51pm UTC](https://discuss.elastic.co/t/cannot-setup-dashboards-filebeat-to-kibana/305115 "2022-05-19T15:51:58Z")
Hi there, I´m trying to setup kibana dashboard from filebeat but not working Follow my test output: elasticsearch: https://XX.XX.XX.XX:9200... parse url... OK connection... parse host... OK dns lookup... O… --- ## [If statement ruby filter](https://discuss.elastic.co/t/if-statement-ruby-filter/305093)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 6\ **Last updated:** [May 19, 2022, 3:42pm UTC](https://discuss.elastic.co/t/if-statement-ruby-filter/305093 "2022-05-19T15:42:50Z")
Hi, I have a small problem in if statement under ruby here my code: filter { grok { match =\> { "message" =\> "%{GREEDYDATA:log\_message}" }} mutate { split =\> {"message" =\> "|"} } ruby { code =\> "event.set('num… --- ## [Unable to split Arrays into fields -parsing XML Attributes of an unknown number of child elements](https://discuss.elastic.co/t/unable-to-split-arrays-into-fields-parsing-xml-attributes-of-an-unknown-number-of-child-elements/305144)
**Author:** [@rubberband](https://discuss.elastic.co/u/rubberband)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 3:23pm UTC](https://discuss.elastic.co/t/unable-to-split-arrays-into-fields-parsing-xml-attributes-of-an-unknown-number-of-child-elements/305144 "2022-05-19T15:23:10Z")
Hi, I'm trying to parse some XML content using Logstash. I'm trying to get the attributes as separate fields. Each \ element will have some attributes that need to be parsed as fields and an unknown number of \ **Author:** [@zubair\_aftab](https://discuss.elastic.co/u/zubair_aftab)\ **Replies:** 7\ **Last updated:** [May 19, 2022, 3:06pm UTC](https://discuss.elastic.co/t/unable-to-get-time-difference-with-ruby-in-logstash/304747 "2022-05-19T15:06:22Z") I have 2 time fields Time1 and Time2. I want to get there difference but getting error "Ruby exception occurred: undefined method \`-' for nil:NilClass". Below is the code. what is wrong in it?? \` if \[Time1\] { date{ … --- ## [Explore underlying data issue for TSVB (Bar Chart 1M intervals)](https://discuss.elastic.co/t/explore-underlying-data-issue-for-tsvb-bar-chart-1m-intervals/305207)
**Author:** [@MMurza](https://discuss.elastic.co/u/MMurza)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 3:02pm UTC](https://discuss.elastic.co/t/explore-underlying-data-issue-for-tsvb-bar-chart-1m-intervals/305207 "2022-05-19T15:02:46Z")
Hello! Is there a way to workaround this bug? Maybe it was fixed in recent versions? In TSVB visual I have a bar chart, with timeline interval set to 1 month. Add visual to dashboard, switch to view mode in dashboard. … --- ## [Create index with index-settings](https://discuss.elastic.co/t/create-index-with-index-settings/305161)
**Author:** [@Alexander\_Ott](https://discuss.elastic.co/u/Alexander_Ott)\ **Replies:** 2\ **Last updated:** [May 19, 2022, 2:58pm UTC](https://discuss.elastic.co/t/create-index-with-index-settings/305161 "2022-05-19T14:58:31Z")
We are trying to migrate from Elasticsearch 6.7.1 to Elasticsearch 8.2.0. In Elasticsearch 6.7.1 it was possible to provide index-settings within an CreateIndexRequest as follows { "index": { "mapping": { … --- ## [Elastic Agent connection to ES fails due to missing hostname in auto-configured certificate](https://discuss.elastic.co/t/elastic-agent-connection-to-es-fails-due-to-missing-hostname-in-auto-configured-certificate/305203)
**Author:** [@frederikvandeputte](https://discuss.elastic.co/u/frederikvandeputte)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 2:41pm UTC](https://discuss.elastic.co/t/elastic-agent-connection-to-es-fails-due-to-missing-hostname-in-auto-configured-certificate/305203 "2022-05-19T14:41:49Z")
My setup includes Elasticsearch server + Kibana server + Fleet server. version 8.2 Installation done via Linux APT/DEB method with security auto-configured certificates. The idea of the server setup is to install elast… --- ## [How to upgrade to kibana 8.3](https://discuss.elastic.co/t/how-to-upgrade-to-kibana-8-3/305192)
**Author:** [@Thomas\_Masquelier1](https://discuss.elastic.co/u/Thomas_Masquelier1)\ **Replies:** 2\ **Last updated:** [May 19, 2022, 2:41pm UTC](https://discuss.elastic.co/t/how-to-upgrade-to-kibana-8-3/305192 "2022-05-19T14:41:14Z")
Hey, i'm just saw on github that some new functionalities have been implemented to be able to use tags on agent and to sort them in fleet. This seems to be part of kibana 8.3. I'm using elastic cloud trial at the momen… --- ## [Why would Kibana not show all docs?](https://discuss.elastic.co/t/why-would-kibana-not-show-all-docs/305187)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 5\ **Last updated:** [May 19, 2022, 2:32pm UTC](https://discuss.elastic.co/t/why-would-kibana-not-show-all-docs/305187 "2022-05-19T14:32:15Z")
I just created an index with 235 docs all created within the same second. In devtools a search returns all 235 docs, however Kibana, on the first screen before any filtering only returns 177. I don't know where to look… --- ## [How to set cutom index in filebeat](https://discuss.elastic.co/t/how-to-set-cutom-index-in-filebeat/305086)
**Author:** [@huzaifa224](https://discuss.elastic.co/u/huzaifa224)\ **Replies:** 3\ **Last updated:** [May 19, 2022, 2:13pm UTC](https://discuss.elastic.co/t/how-to-set-cutom-index-in-filebeat/305086 "2022-05-19T14:13:41Z")
Hi, I am trying to configure custom index with logstash in filebeat so that its became easy to identify the servers with their index name here is the my filebeat config file. and logstash config file ##################… --- ## [ElasticCloud - alert on disk usage using metricbeats](https://discuss.elastic.co/t/elasticcloud-alert-on-disk-usage-using-metricbeats/304663)
**Author:** [@Thomas\_Masquelier](https://discuss.elastic.co/u/Thomas_Masquelier)\ **Replies:** 4\ **Last updated:** [May 19, 2022, 1:12pm UTC](https://discuss.elastic.co/t/elasticcloud-alert-on-disk-usage-using-metricbeats/304663 "2022-05-19T13:12:51Z")
I'm struggling to understand how to define an alert for my hosts disk usage in elastic cloud. The agent is installed on my different hosts with the "system" integration. Pretty sure this use metricbeats. I can see this… --- ## [Filebeat not started](https://discuss.elastic.co/t/filebeat-not-started/305129)
**Author:** [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Replies:** 20\ **Last updated:** [May 19, 2022, 12:53pm UTC](https://discuss.elastic.co/t/filebeat-not-started/305129 "2022-05-19T12:53:41Z")
Hi, My filebeat was not running, if can i change any settings in filebeat.yml it goes to inactive. sudo service filebeat status ● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.… --- ## [Dissect Filter, Creating Subfields](https://discuss.elastic.co/t/dissect-filter-creating-subfields/305170)
**Author:** [@shinobu](https://discuss.elastic.co/u/shinobu)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 11:59am UTC](https://discuss.elastic.co/t/dissect-filter-creating-subfields/305170 "2022-05-19T11:59:04Z")
Hello, how can i get dissect to create a structure with subfields? dissect { mapping =\> { "message" =\> "%{test.subfield}.%{test.subfield2}" } } I would expect so… --- ## [Declaring runtime fields when creating transforms](https://discuss.elastic.co/t/declaring-runtime-fields-when-creating-transforms/305169)
**Author:** [@Silver137](https://discuss.elastic.co/u/Silver137)\ **Replies:** 2\ **Last updated:** [May 19, 2022, 11:51am UTC](https://discuss.elastic.co/t/declaring-runtime-fields-when-creating-transforms/305169 "2022-05-19T11:51:14Z")
I would like to add runtime fields against the index that is created when creating a transformation. I saw that the kibana transform gui offers a text box to work with runtime fields but not sure about how it works. S… --- ## [Kibana pie chart with slices for different frequency values](https://discuss.elastic.co/t/kibana-pie-chart-with-slices-for-different-frequency-values/305175)
**Author:** [@Jaime\_Fernandez](https://discuss.elastic.co/u/Jaime_Fernandez)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 11:00am UTC](https://discuss.elastic.co/t/kibana-pie-chart-with-slices-for-different-frequency-values/305175 "2022-05-19T11:00:17Z")
Hi, I have an ES index where each document represents a user login. The document contains a "user\_id" field (the other fields are not relevant). I need to show a pie chart in kibana with the following slices: "1" wit… --- ## [Error Activating Self-Monitoring in ElasticSearch 7.17.3](https://discuss.elastic.co/t/error-activating-self-monitoring-in-elasticsearch-7-17-3/304451)
**Author:** [@IMagalashvili](https://discuss.elastic.co/u/IMagalashvili)\ **Replies:** 9\ **Last updated:** [May 19, 2022, 10:28am UTC](https://discuss.elastic.co/t/error-activating-self-monitoring-in-elasticsearch-7-17-3/304451 "2022-05-19T10:28:23Z")
Hi, I have problem with Self-Monitoring in Elasticsearch 7.17.3. It worked correctly, but I changed Stting parameters with PUT 'http: // server:9200 /\_settings { "index": { "final\_pipeline": "geoip-info" } } My f… --- ## [Socket error when starting kibana](https://discuss.elastic.co/t/socket-error-when-starting-kibana/304011)
**Author:** [@Roshan1](https://discuss.elastic.co/u/Roshan1)\ **Replies:** 3\ **Last updated:** [May 19, 2022, 10:15am UTC](https://discuss.elastic.co/t/socket-error-when-starting-kibana/304011 "2022-05-19T10:15:17Z")
Hello Team, kindly advise why I am getting the following error when starting Kibana? SSL generated using: \[elastic@rb-in-prod-kibana-01 Elasticsearch\]$ ./bin/Elasticsearch-certutil http ## Elasticsearch HTTP Certific… --- ## [Query in Runtime Field](https://discuss.elastic.co/t/query-in-runtime-field/305165)
**Author:** [@shivendra95](https://discuss.elastic.co/u/shivendra95)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 10:14am UTC](https://discuss.elastic.co/t/query-in-runtime-field/305165 "2022-05-19T10:14:28Z")
Hi, I'm creating a runtime field and want to fetch data from Elasticsearch by performing a query. Below is my query GET prod\_logistic\_parcel\_index/\_search { "query": { "query\_string": { "query": "(71116812… --- ## [MongoDB Output plugin 3.1.7 error](https://discuss.elastic.co/t/mongodb-output-plugin-3-1-7-error/305160)
**Author:** [@bhumikadave](https://discuss.elastic.co/u/bhumikadave)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 9:56am UTC](https://discuss.elastic.co/t/mongodb-output-plugin-3-1-7-error/305160 "2022-05-19T09:56:32Z")
Hello, I am not able to connect to MongoDB Atlas with below error.Using plugin version 3.1.7. \[WARN \]\[logstash.outputs.mongodb \]\[main\] MONGODB | Failed to handshake with \: 27017 ArgumentError: wrong number… --- ## [How to balance my elasticsearch cluster](https://discuss.elastic.co/t/how-to-balance-my-elasticsearch-cluster/305083)
**Author:** [@Arethusa](https://discuss.elastic.co/u/Arethusa)\ **Replies:** 12\ **Last updated:** [May 19, 2022, 9:52am UTC](https://discuss.elastic.co/t/how-to-balance-my-elasticsearch-cluster/305083 "2022-05-19T09:52:47Z")
Hello, I want to balance my Elasticsearch cluster. I am on version 7.16.3 I have 5 servers on cluster, but the data are not use symmetrically : ELS01 : 62% ELS02 : 50% ELS03 : 69% ELS04 : 76% ELS05 : 86% This is… --- ## [Permission denied (os:windows)](https://discuss.elastic.co/t/permission-denied-os-windows/304923)
**Author:** [@RohitBR\_K](https://discuss.elastic.co/u/RohitBR_K)\ **Replies:** 14\ **Last updated:** [May 19, 2022, 9:33am UTC](https://discuss.elastic.co/t/permission-denied-os-windows/304923 "2022-05-19T09:33:48Z")
Am starting Elasticsearch like this cd elasticsearch/bin/elasticsearch.bat but am getting permission denied when am accessing one of my es documents through my API. am not understanding what's the issue here . Am usin… --- ## [Creating a kibana rule to report the table of documents for the hits received in indices](https://discuss.elastic.co/t/creating-a-kibana-rule-to-report-the-table-of-documents-for-the-hits-received-in-indices/305154)
**Author:** [@vinitnair93](https://discuss.elastic.co/u/vinitnair93)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 9:18am UTC](https://discuss.elastic.co/t/creating-a-kibana-rule-to-report-the-table-of-documents-for-the-hits-received-in-indices/305154 "2022-05-19T09:18:00Z")
I have written a kibana rule (Elasticsearch query) that would check every 1 day for an index with size 1 Elasticsearch query - { "query":{ "match\_all" : {} }, "fields": \[ "@timestamp", "column\_2" \], "\_source": … --- ## [Cross Cluster Search Not Caching](https://discuss.elastic.co/t/cross-cluster-search-not-caching/304504)
**Author:** [@Liam\_B](https://discuss.elastic.co/u/Liam_B)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 9:13am UTC](https://discuss.elastic.co/t/cross-cluster-search-not-caching/304504 "2022-05-19T09:13:22Z")
Hi If I have a couple of clusters, both connected to each other as remote clusters. If I perform a search against a single cluster, I get the benefit of caching of search results within a refresh interval window, if I … --- ## [Grok logstash JSON](https://discuss.elastic.co/t/grok-logstash-json/305152)
**Author:** [@Roshan1](https://discuss.elastic.co/u/Roshan1)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 9:11am UTC](https://discuss.elastic.co/t/grok-logstash-json/305152 "2022-05-19T09:11:55Z")
EL 8.2 Kibana 8.2 Hello Team, could you please advise how to merge the 2 lines below and generate the GONK command? {"log":"0.3817706468179257\\r\\n","stream":"stdout","time":"2022-05-18T10:37:31.746257847Z"} {"log":"1… --- ## [Average response time](https://discuss.elastic.co/t/average-response-time/305149)
**Author:** [@The-morpho](https://discuss.elastic.co/u/The-morpho)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 9:02am UTC](https://discuss.elastic.co/t/average-response-time/305149 "2022-05-19T09:02:02Z")
Hello, I'm new to Elastic and Kibana, I'm working on log files that contains information about some operations on differents levels, I have a field timestamp that represent time when each operation has occured on each l… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=932) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=934)