# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=640 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 641 --- ## [Add human readable field with lookup](https://discuss.elastic.co/t/add-human-readable-field-with-lookup/325032)
**Author:** [@tonelk](https://discuss.elastic.co/u/tonelk)\ **Replies:** 1\ **Last updated:** [February 8, 2023, 3:53pm UTC](https://discuss.elastic.co/t/add-human-readable-field-with-lookup/325032 "2023-02-08T15:53:43Z")
Hi, I want to add the human readable description of error codes in my data. I've tried this processors: - add\_fields: when: equals: cisco\_code: "106015" fields: cisco\_description: 'Den… --- ## [Rank based on rarity of a field value](https://discuss.elastic.co/t/rank-based-on-rarity-of-a-field-value/323546)
**Author:** [@pheeria](https://discuss.elastic.co/u/pheeria)\ **Replies:** 3\ **Last updated:** [February 8, 2023, 3:09pm UTC](https://discuss.elastic.co/t/rank-based-on-rarity-of-a-field-value/323546 "2023-02-08T15:09:34Z")
Hi :vulcan\_salute: I'd like to know how can I rank lower items, which have fields that are frequently appearing among the results. Say, we have a similar result set: "name": "Red T-Shirt" "store": "Zara" "name": "Yel… --- ## [Recovering data from a persistent queue page file](https://discuss.elastic.co/t/recovering-data-from-a-persistent-queue-page-file/325029)
**Author:** [@peter\_west](https://discuss.elastic.co/u/peter_west)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 2:10pm UTC](https://discuss.elastic.co/t/recovering-data-from-a-persistent-queue-page-file/325029 "2023-02-08T14:10:59Z")
Is there any means by which you can reprocess data from a page file into an Elasticsearch index? My instincts tell me not because the likelihood is that the page file will probably have a partial record at the start so … --- ## [Limit user to single dashboard?](https://discuss.elastic.co/t/limit-user-to-single-dashboard/324962)
**Author:** [@willdennis](https://discuss.elastic.co/u/willdennis)\ **Replies:** 2\ **Last updated:** [February 8, 2023, 2:04pm UTC](https://discuss.elastic.co/t/limit-user-to-single-dashboard/324962 "2023-02-08T14:04:37Z")
Trying to limit a given user to only see a single dashboard. I have easily configured my system (single-node 7.17.8 ELK cluster on basic license) to have this user see only all dashboards, but I want to limit to only one… --- ## [Does elasticsearch support listening on a non root path for the URL?](https://discuss.elastic.co/t/does-elasticsearch-support-listening-on-a-non-root-path-for-the-url/324939)
**Author:** [@ghettosamson](https://discuss.elastic.co/u/ghettosamson)\ **Replies:** 2\ **Last updated:** [February 8, 2023, 2:04pm UTC](https://discuss.elastic.co/t/does-elasticsearch-support-listening-on-a-non-root-path-for-the-url/324939 "2023-02-08T14:04:37Z")
I have my elasticsearch deployed as a Docker container on AWS, sitting behind an Application Load Balancer. The container is listening on port 9200 as usual, but I'm using path based routing for all my micro-services beh… --- ## [Sum of a field in a parent node and grouping by a field in a child node](https://discuss.elastic.co/t/sum-of-a-field-in-a-parent-node-and-grouping-by-a-field-in-a-child-node/324948)
**Author:** [@learningelastic](https://discuss.elastic.co/u/learningelastic)\ **Replies:** 4\ **Last updated:** [February 8, 2023, 1:46pm UTC](https://discuss.elastic.co/t/sum-of-a-field-in-a-parent-node-and-grouping-by-a-field-in-a-child-node/324948 "2023-02-08T13:46:55Z")
I'm trying to get the sum of a field in a parent node while aggregating on a field of a child node. For example, this is what I've setup: PUT order POST order/\_mapping { "properties": { "order\_items": { "t… --- ## [Netflow data not appears in Elastic search/Kibana](https://discuss.elastic.co/t/netflow-data-not-appears-in-elastic-search-kibana/324952)
**Author:** [@canifer](https://discuss.elastic.co/u/canifer)\ **Replies:** 5\ **Last updated:** [February 8, 2023, 1:33pm UTC](https://discuss.elastic.co/t/netflow-data-not-appears-in-elastic-search-kibana/324952 "2023-02-08T13:33:31Z")
Hello, I Countered a problem when using Filebeat to forward Netflow data into Elasticsearch, I'm using filebeat 8.5.2 installed on Centos Stream 9, to send netflow data to Elastic Stack 8.5.2. The problem is, why the … --- ## [Kibana deprecated scripted fields](https://discuss.elastic.co/t/kibana-deprecated-scripted-fields/324657)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 4\ **Last updated:** [February 8, 2023, 1:33pm UTC](https://discuss.elastic.co/t/kibana-deprecated-scripted-fields/324657 "2023-02-08T13:33:16Z")
Hi, My current setup uses scripted fields for the node name in the Metricbeat index. The goal is to be able to open a specific monitoring dashboard for that VM. As I see, this was deprecated from 7.13, but I'm not sure … --- ## [Unable to get data from scripted field](https://discuss.elastic.co/t/unable-to-get-data-from-scripted-field/325021)
**Author:** [@amit\_tiwari](https://discuss.elastic.co/u/amit_tiwari)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 12:46pm UTC](https://discuss.elastic.co/t/unable-to-get-data-from-scripted-field/325021 "2023-02-08T12:46:32Z")
My Index: cls-docker-logs\* Existing mapping in index cls-docker-logs-2023-01-18 GET cls-docker-logs-2023-01-18/\_mapping "stack\_trace" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore\_ab… --- ## [How to pick up certain Logstash events so I can ignore/work on them](https://discuss.elastic.co/t/how-to-pick-up-certain-logstash-events-so-i-can-ignore-work-on-them/324935)
**Author:** [@SamuelSMendes](https://discuss.elastic.co/u/SamuelSMendes)\ **Replies:** 4\ **Last updated:** [February 8, 2023, 12:25pm UTC](https://discuss.elastic.co/t/how-to-pick-up-certain-logstash-events-so-i-can-ignore-work-on-them/324935 "2023-02-08T12:25:00Z")
So I've been working with a Logstash pipeline which deals with creating and updating a few documents. And when the schedule hits and the creation repeats the following line pop up: \[2023-02-07T17:36:07,915\]\[WARN \]\[logst… --- ## [How to remove long type](https://discuss.elastic.co/t/how-to-remove-long-type/325015)
**Author:** [@Gaurav\_kr](https://discuss.elastic.co/u/Gaurav_kr)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 11:52am UTC](https://discuss.elastic.co/t/how-to-remove-long-type/325015 "2023-02-08T11:52:56Z")
Hi Team, I am wanted to remove long data type form the fields.yml file as it is consuming a large disk space. Doing it for metrocbeat and only using system modules so have remove most of the unused fields from fields.ym… --- ## [Install elastic agent on unmanaged AWS cluster to Secured Elastic cloud fleet](https://discuss.elastic.co/t/install-elastic-agent-on-unmanaged-aws-cluster-to-secured-elastic-cloud-fleet/325011)
**Author:** [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 11:42am UTC](https://discuss.elastic.co/t/install-elastic-agent-on-unmanaged-aws-cluster-to-secured-elastic-cloud-fleet/325011 "2023-02-08T11:42:00Z")
Hi Team, We are having a Elastic cloud secured cluster and we are trying to install an elastic agent on a unmanaged AWS cluster to send data to Elasticsearch. So we have crated a private link and added the same in our E… --- ## [Eland dataframe: search\_phase\_execution\_exception](https://discuss.elastic.co/t/eland-dataframe-search-phase-execution-exception/325006)
**Author:** [@mruiter](https://discuss.elastic.co/u/mruiter)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 11:02am UTC](https://discuss.elastic.co/t/eland-dataframe-search-phase-execution-exception/325006 "2023-02-08T11:02:27Z")
Hello everybody, For our ETL process we're loading multiple indices, however for the bigger ones with 10 million+ rows, we sometimes run into an error on our server. Other times all the indices will load correctly. The … --- ## [Process json with multiple keys of same name via filebeat.yml](https://discuss.elastic.co/t/process-json-with-multiple-keys-of-same-name-via-filebeat-yml/325003)
**Author:** [@stranger](https://discuss.elastic.co/u/stranger)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 10:54am UTC](https://discuss.elastic.co/t/process-json-with-multiple-keys-of-same-name-via-filebeat-yml/325003 "2023-02-08T10:54:27Z")
I am trying to process my json log to extract all the fields. - decode\_json\_fields: fields: \["message"\] target: "" process\_array: false expand\_keys: true overwrite\_keys: false Example of l… --- ## [Elasticsearch query to SQL](https://discuss.elastic.co/t/elasticsearch-query-to-sql/324814)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [February 8, 2023, 10:35am UTC](https://discuss.elastic.co/t/elasticsearch-query-to-sql/324814 "2023-02-08T10:35:42Z")
I know there is SQL API which can convert SQL query to REST is there a way around. convert REST query to SQL ? --- ## [Line Chart - How to filter for one single Term](https://discuss.elastic.co/t/line-chart-how-to-filter-for-one-single-term/324983)
**Author:** [@tinrik](https://discuss.elastic.co/u/tinrik)\ **Replies:** 1\ **Last updated:** [February 8, 2023, 10:29am UTC](https://discuss.elastic.co/t/line-chart-how-to-filter-for-one-single-term/324983 "2023-02-08T10:29:15Z")
Hi, I have a dataset where files can belong to categories a and b that looks like follows: { file: foo, category = \[a\], value = 123, timestamp = 1 } { file: bar, category = \[a, b\], value = 321, timestamp = 1 } ... … --- ## [Roller over at mid night with Index per 30days in ILM](https://discuss.elastic.co/t/roller-over-at-mid-night-with-index-per-30days-in-ilm/324982)
**Author:** [@pruthvi](https://discuss.elastic.co/u/pruthvi)\ **Replies:** 4\ **Last updated:** [February 8, 2023, 10:04am UTC](https://discuss.elastic.co/t/roller-over-at-mid-night-with-index-per-30days-in-ilm/324982 "2023-02-08T10:04:10Z")
Hi There, Need to create Index per 30 days interval, total of 60+ index for 5 years of data with ILM. So while searching within the date range I could search only the index created within that date range which would in… --- ## [ElasticsearchClient didn't close socket connect and file open](https://discuss.elastic.co/t/elasticsearchclient-didnt-close-socket-connect-and-file-open/324705)
**Author:** [@Adam\_Lin](https://discuss.elastic.co/u/Adam_Lin)\ **Replies:** 6\ **Last updated:** [February 8, 2023, 6:14am UTC](https://discuss.elastic.co/t/elasticsearchclient-didnt-close-socket-connect-and-file-open/324705 "2023-02-08T06:14:18Z")
Hi all, I tried to query logs from the ELK server through java RestClient. This is how I set up search client try { restClient = RestClient .builder(new HttpHost(host, port)) .setRequestConfigCallback(new Res… --- ## [Is there a way to log or to check the log with time taken to complete the transition from Phases in ILM](https://discuss.elastic.co/t/is-there-a-way-to-log-or-to-check-the-log-with-time-taken-to-complete-the-transition-from-phases-in-ilm/324357)
**Author:** [@pruthvi](https://discuss.elastic.co/u/pruthvi)\ **Replies:** 2\ **Last updated:** [February 8, 2023, 8:34am UTC](https://discuss.elastic.co/t/is-there-a-way-to-log-or-to-check-the-log-with-time-taken-to-complete-the-transition-from-phases-in-ilm/324357 "2023-02-08T08:34:20Z")
I have implemented ILM(index lifecycle management) with roller over from Hot to warm and move to cold after period of time. I have indexed few thousand of document into index. Rollover is also happing with max age in po… --- ## [Is there a limit on number of fields on which we can aggregate?](https://discuss.elastic.co/t/is-there-a-limit-on-number-of-fields-on-which-we-can-aggregate/324947)
**Author:** [@mattkallo](https://discuss.elastic.co/u/mattkallo)\ **Replies:** 3\ **Last updated:** [February 8, 2023, 7:13am UTC](https://discuss.elastic.co/t/is-there-a-limit-on-number-of-fields-on-which-we-can-aggregate/324947 "2023-02-08T07:13:06Z")
Is there a limit on number of fields on which we can aggregate? I could not find any specific info on this. I could find details on # of buckets for a given field, but not for the number of fields itself. Currently I h… --- ## [How to setup ILM for filebeat & metricbeat?](https://discuss.elastic.co/t/how-to-setup-ilm-for-filebeat-metricbeat/322798)
**Author:** [@Siavash\_Fazli](https://discuss.elastic.co/u/Siavash_Fazli)\ **Replies:** 2\ **Last updated:** [February 8, 2023, 6:17am UTC](https://discuss.elastic.co/t/how-to-setup-ilm-for-filebeat-metricbeat/322798 "2023-02-08T06:17:24Z")
Hi Everyone. I need to set ILM for some beat services. As you know saving beats docs from the index changed to Datastream in ELK version 8. I could enable index rollup for heartbeat From Stack Management \> Index Lifec… --- ## [Dynamic Generate CSV in Logstash csv output plugin](https://discuss.elastic.co/t/dynamic-generate-csv-in-logstash-csv-output-plugin/324301)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 2\ **Last updated:** [February 8, 2023, 6:09am UTC](https://discuss.elastic.co/t/dynamic-generate-csv-in-logstash-csv-output-plugin/324301 "2023-02-08T06:09:19Z")
Hello, I want to dynamically generate my csv file with current date. My conf---- csv { fields =\> \["id" , "name"\] path =\> "test-%{+YYYY-MM-dd}.csv" } but it is not working.. It is generating file as : test-.… --- ## [Compression algorithm for best Search Performance in Elastic](https://discuss.elastic.co/t/compression-algorithm-for-best-search-performance-in-elastic/324972)
**Author:** [@siddhartha\_c](https://discuss.elastic.co/u/siddhartha_c)\ **Replies:** 1\ **Last updated:** [February 8, 2023, 6:02am UTC](https://discuss.elastic.co/t/compression-algorithm-for-best-search-performance-in-elastic/324972 "2023-02-08T06:02:55Z")
Hi Team, Which is the best compression algorithm in Elastic for the best search performance. We have 100 TB of data to be indexed in an Index. The idea is to have best search performance as a trade off with the index s… --- ## [Netflow data not appears in Elastic search/Kibana](https://discuss.elastic.co/t/netflow-data-not-appears-in-elastic-search-kibana/324971)
**Author:** [@canifer](https://discuss.elastic.co/u/canifer)\ **Replies:** 1\ **Last updated:** [February 8, 2023, 5:52am UTC](https://discuss.elastic.co/t/netflow-data-not-appears-in-elastic-search-kibana/324971 "2023-02-08T05:52:54Z")
Hello, I capture Netflow data using Filebeat 8.5.2 and send it directly into Elasticsearch version 8.5.2 The problem is, why the Kibana cannot show any data? Please take a look at the filebeat.yml: ###############… --- ## [Zombie process continuous generated by elastic-agent](https://discuss.elastic.co/t/zombie-process-continuous-generated-by-elastic-agent/324966)
**Author:** [@Xu\_Danny](https://discuss.elastic.co/u/Xu_Danny)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 5:17am UTC](https://discuss.elastic.co/t/zombie-process-continuous-generated-by-elastic-agent/324966 "2023-02-08T05:17:12Z")
found zombie process which parent process is elastic-agent, after kill elastic-agent for a while, it automatically start and generate new zombie child process. root@cdh1:~# ps -A -ostat,ppid,pid,cmd|grep elastic-agent S… --- ## [Adding Custom processors with Elastic-Agent](https://discuss.elastic.co/t/adding-custom-processors-with-elastic-agent/324958)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 3\ **Last updated:** [February 8, 2023, 3:47am UTC](https://discuss.elastic.co/t/adding-custom-processors-with-elastic-agent/324958 "2023-02-08T03:47:20Z")
Hi, I wrote a custom parser with dissect processor for collecting fail2ban-logs. Those are working fine for the servers where logs are being collected using filebeat however I am not sure how do I add those with where I… --- ## [Running percolate query against document date value](https://discuss.elastic.co/t/running-percolate-query-against-document-date-value/324950)
**Author:** [@Matthew\_Greenfield](https://discuss.elastic.co/u/Matthew_Greenfield)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 1:49am UTC](https://discuss.elastic.co/t/running-percolate-query-against-document-date-value/324950 "2023-02-08T01:49:04Z")
I have a query that checks to see if a date is more than a year old with something like: "range": { "some\_date": { "gt": "now-1y" } } But I need to also figure out if that document WILL BE more than a year old … --- ## [Self-signed certificate in the chain](https://discuss.elastic.co/t/self-signed-certificate-in-the-chain/324715)
**Author:** [@Leonardo\_Henrique](https://discuss.elastic.co/u/Leonardo_Henrique)\ **Replies:** 1\ **Last updated:** [February 8, 2023, 12:16am UTC](https://discuss.elastic.co/t/self-signed-certificate-in-the-chain/324715 "2023-02-08T00:16:55Z")
Hey everyone! How have you been? I configured the SSL for the Elastic cluster using the elasticsearch-certutil. So far, elasticsearch nodes can reach out each other, the Kibana can communicate with them and both are bei… --- ## [Accent with edge ngram token filter](https://discuss.elastic.co/t/accent-with-edge-ngram-token-filter/324863)
**Author:** [@Bob\_Guo](https://discuss.elastic.co/u/Bob_Guo)\ **Replies:** 2\ **Last updated:** [February 8, 2023, 12:15am UTC](https://discuss.elastic.co/t/accent-with-edge-ngram-token-filter/324863 "2023-02-08T00:15:52Z")
Hi, I have a custom analyzer which uses the edge\_ngram token filer. Below is the setup: "analysis": { "filter": { "my\_filter": { "type": "edge\_ngram", "min\_gram": "1", … --- ## [Failed to index dataframe](https://discuss.elastic.co/t/failed-to-index-dataframe/324944)
**Author:** [@OnTheRoad](https://discuss.elastic.co/u/OnTheRoad)\ **Replies:** 1\ **Last updated:** [February 7, 2023, 11:29pm UTC](https://discuss.elastic.co/t/failed-to-index-dataframe/324944 "2023-02-07T23:29:25Z")
Hello everyone, I'm starting using ES and I'm really having trouble to index a dataframe. dataframe looks like : So far I wrote the following code : es\_client = Elasticsearch(hosts = "http://xxx.xxx.xxx.xxx:9200",… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=639) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=641)