# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=633 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 634 --- ## [How to dynamically specify a url formatter](https://discuss.elastic.co/t/how-to-dynamically-specify-a-url-formatter/325520)
**Author:** [@kohkaw](https://discuss.elastic.co/u/kohkaw)\ **Replies:** 0\ **Last updated:** [February 15, 2023, 1:58am UTC](https://discuss.elastic.co/t/how-to-dynamically-specify-a-url-formatter/325520 "2023-02-15T01:58:25Z")
I want to dynamically specify a url formatter for a document that contains an ever-increasing number of URL strings. Is there any other way than manually setting Set format=url from Index pattern? --- ## [Reindexing in Production Environment](https://discuss.elastic.co/t/reindexing-in-production-environment/323543)
**Author:** [@vishnu\_teja](https://discuss.elastic.co/u/vishnu_teja)\ **Replies:** 1\ **Last updated:** [February 15, 2023, 12:18am UTC](https://discuss.elastic.co/t/reindexing-in-production-environment/323543 "2023-02-15T00:18:19Z")
Hi Everyone, Currently in our Elasticsearch cluster we have a lot of documents which need to deleted, so we are looking to re-index the used documents to a new index and delete the old index. We will be doing this in pro… --- ## [Editing a managed policy can break Kibana caution](https://discuss.elastic.co/t/editing-a-managed-policy-can-break-kibana-caution/325515)
**Author:** [@David41](https://discuss.elastic.co/u/David41)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 10:54pm UTC](https://discuss.elastic.co/t/editing-a-managed-policy-can-break-kibana-caution/325515 "2023-02-14T22:54:04Z")
Hi, I am wanting to edit an existing policy and I notice that there is a caution symbol that states that "Editing a managed policy can break Kibana" I am not sure if it will break or not . However there is an option tha… --- ## [Message: app heartbeat--8.4.3-d6501b26: Missed two check-in elastic-agent standalone](https://discuss.elastic.co/t/message-app-heartbeat-8-4-3-d6501b26-missed-two-check-in-elastic-agent-standalone/325050)
**Author:** [@Bhrugu\_Sharma](https://discuss.elastic.co/u/Bhrugu_Sharma)\ **Replies:** 3\ **Last updated:** [February 14, 2023, 9:34pm UTC](https://discuss.elastic.co/t/message-app-heartbeat-8-4-3-d6501b26-missed-two-check-in-elastic-agent-standalone/325050 "2023-02-14T21:34:13Z")
I've used the code mentioned below, and when i log into one of the agents and do ./elastic-agent status i get the following error and the logs for kibana states sample code apiVersion: v1 kind: ConfigMap metadata… --- ## [How to parse csv via Elastic Agent?](https://discuss.elastic.co/t/how-to-parse-csv-via-elastic-agent/324121)
**Author:** [@test\_qweqwe](https://discuss.elastic.co/u/test_qweqwe)\ **Replies:** 16\ **Last updated:** [February 14, 2023, 9:05pm UTC](https://discuss.elastic.co/t/how-to-parse-csv-via-elastic-agent/324121 "2023-02-14T21:05:22Z")
Hi! I want to collects csv logs and if I understand correct, I need to add new integration based on "Custom Logs"? I'm not sure how to do it. I have this config for logstash under conf.d folder and everything work fin… --- ## [Elasticsearch query for \`SELECT id FROM foo WHERE id NOT IN (SELECT id FROM foo WHERE ...)](https://discuss.elastic.co/t/elasticsearch-query-for-select-id-from-foo-where-id-not-in-select-id-from-foo-where/325302)
**Author:** [@learningelastic](https://discuss.elastic.co/u/learningelastic)\ **Replies:** 4\ **Last updated:** [February 14, 2023, 7:51pm UTC](https://discuss.elastic.co/t/elasticsearch-query-for-select-id-from-foo-where-id-not-in-select-id-from-foo-where/325302 "2023-02-14T19:51:34Z")
I want to do a "not in present index" type of operation. For example, let's say I have an index called customer\_subscription with just these 4 records: | customer\_id | pay\_date | +-------------+------------+ | … --- ## [Why is multiline not working for this unstructured log?](https://discuss.elastic.co/t/why-is-multiline-not-working-for-this-unstructured-log/325510)
**Author:** [@erwin339](https://discuss.elastic.co/u/erwin339)\ **Replies:** 0\ **Last updated:** [February 14, 2023, 7:32pm UTC](https://discuss.elastic.co/t/why-is-multiline-not-working-for-this-unstructured-log/325510 "2023-02-14T19:32:18Z")
My multiline: parsers: -multiline: type: pattern pattern: '^\\{' negate: true match: after The format of my log can be like this: { C-FLOW-ID-CAB APN101MQ C-OPERATION-CAB P T-EVENTO-CAB RUNN… --- ## [Restarting logstash container sends events again to elastic, despite sincedb](https://discuss.elastic.co/t/restarting-logstash-container-sends-events-again-to-elastic-despite-sincedb/324675)
**Author:** [@paul\_chrlt](https://discuss.elastic.co/u/paul_chrlt)\ **Replies:** 7\ **Last updated:** [February 14, 2023, 6:03pm UTC](https://discuss.elastic.co/t/restarting-logstash-container-sends-events-again-to-elastic-despite-sincedb/324675 "2023-02-14T18:03:09Z")
Hi all, Can you help us ? We use elk 7.17.7 in docker containers hosted on a server with persistent shared volumes for path (read only), file\_completed\_log\_path, sincedb\_path. Each time we stop and start our logstash … --- ## [No data streams and logs under fleet server](https://discuss.elastic.co/t/no-data-streams-and-logs-under-fleet-server/325493)
**Author:** [@Leonardo\_Henrique](https://discuss.elastic.co/u/Leonardo_Henrique)\ **Replies:** 0\ **Last updated:** [February 14, 2023, 3:21pm UTC](https://discuss.elastic.co/t/no-data-streams-and-logs-under-fleet-server/325493 "2023-02-14T15:21:35Z")
Hey everyone. I have configured fleet server but I can not see any log messages or data streams in the Kibana UI. Elasticsearch nodes are configured with SSL (with elasticsearch-certutil) and fleet-server is using the … --- ## [Implement User interface buttons for Logstash](https://discuss.elastic.co/t/implement-user-interface-buttons-for-logstash/323641)
**Author:** [@Sandeep\_Raju](https://discuss.elastic.co/u/Sandeep_Raju)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 3:42pm UTC](https://discuss.elastic.co/t/implement-user-interface-buttons-for-logstash/323641 "2023-02-14T15:42:39Z")
Hi all, I am currently pushing data to indices in my elasticsearch 8.4 using logstash in my terminal. However , One of my friends does not know logstash and wants to work with logstash in User interface and push data i… --- ## [Optimizer fails when running yarn start in Kibana version 8](https://discuss.elastic.co/t/optimizer-fails-when-running-yarn-start-in-kibana-version-8/324142)
**Author:** [@ssimmons](https://discuss.elastic.co/u/ssimmons)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 4:49pm UTC](https://discuss.elastic.co/t/optimizer-fails-when-running-yarn-start-in-kibana-version-8/324142 "2023-02-14T16:49:19Z")
I'm in the process of upgrading our custom plugins to Kibana version 8. I'm trying to setup Kibana and Elasticsearch through docker. In the past, I built Kibana using a Dockerfile that would clone Kibana and then run yar… --- ## [Index Patterns](https://discuss.elastic.co/t/index-patterns/325496)
**Author:** [@hnclientes\_HN](https://discuss.elastic.co/u/hnclientes_HN)\ **Replies:** 2\ **Last updated:** [February 14, 2023, 4:48pm UTC](https://discuss.elastic.co/t/index-patterns/325496 "2023-02-14T16:48:48Z")
Hi, I'm testing this version (cloud), and I can't find the option to create an index pattern for an index that I create using devtools. Could you tell me how I can activate this option please? Thank you --- ## [Add a custom tooltip to charts in Kibana?](https://discuss.elastic.co/t/add-a-custom-tooltip-to-charts-in-kibana/325487)
**Author:** [@Senol\_Kurt](https://discuss.elastic.co/u/Senol_Kurt)\ **Replies:** 2\ **Last updated:** [February 14, 2023, 3:50pm UTC](https://discuss.elastic.co/t/add-a-custom-tooltip-to-charts-in-kibana/325487 "2023-02-14T15:50:03Z")
I want to add a custom tooltip that explains charts created with Lens. Is it possible with Kibana? Elasticsearch v.8.3.3 --- ## [Filebeat autodiscover stopping too early when kubernetes pod terminates](https://discuss.elastic.co/t/filebeat-autodiscover-stopping-too-early-when-kubernetes-pod-terminates/325491)
**Author:** [@cpaton](https://discuss.elastic.co/u/cpaton)\ **Replies:** 0\ **Last updated:** [February 14, 2023, 2:49pm UTC](https://discuss.elastic.co/t/filebeat-autodiscover-stopping-too-early-when-kubernetes-pod-terminates/325491 "2023-02-14T14:49:14Z")
I am using filebeat with autodiscover within a Kubernetes cluster to capture logs. When a Kubernetes pod terminates filebeat immediately stops reading log entries which can result in log lines at the end of the logs not… --- ## [Logstash fetched data not available in elastic search](https://discuss.elastic.co/t/logstash-fetched-data-not-available-in-elastic-search/325216)
**Author:** [@ekambaram\_varathan](https://discuss.elastic.co/u/ekambaram_varathan)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 3:26am UTC](https://discuss.elastic.co/t/logstash-fetched-data-not-available-in-elastic-search/325216 "2023-02-14T03:26:58Z")
Hi Team, I am using ELK version: 6.8.23. Though Logstash uploaded csv file data are not present in elasticsearch. my logstash conf file content as follows, input { file { path =\> "/home/data/reports/\*.csv" … --- ## [Yum is unable to update/install from elastic repo](https://discuss.elastic.co/t/yum-is-unable-to-update-install-from-elastic-repo/325221)
**Author:** [@Thomas3](https://discuss.elastic.co/u/Thomas3)\ **Replies:** 3\ **Last updated:** [February 14, 2023, 2:20pm UTC](https://discuss.elastic.co/t/yum-is-unable-to-update-install-from-elastic-repo/325221 "2023-02-14T14:20:51Z")
Hello, when trying to perform updates in RHEL8, I'm getting Failed to download metadata for repo 'logstash-7.x': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried with the follo… --- ## [Rule type Log threshold](https://discuss.elastic.co/t/rule-type-log-threshold/325436)
**Author:** [@maxxl](https://discuss.elastic.co/u/maxxl)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 12:44pm UTC](https://discuss.elastic.co/t/rule-type-log-threshold/325436 "2023-02-14T12:44:22Z")
Kibana 8.4.3 Stack Management \\ Rules and Connectors Connectors type - Server Log Rule type - Log threshold The rule works well. How do I send the hostname and the original log (error.message) to the message? --- ## [SnapShot with select indices is still using all indices](https://discuss.elastic.co/t/snapshot-with-select-indices-is-still-using-all-indices/325437)
**Author:** [@tymercer](https://discuss.elastic.co/u/tymercer)\ **Replies:** 2\ **Last updated:** [February 14, 2023, 12:26pm UTC](https://discuss.elastic.co/t/snapshot-with-select-indices-is-still-using-all-indices/325437 "2023-02-14T12:26:00Z")
Hello, pretty new to ES and everything related to it. Just trying to do a snapshot and only selecting a set of things from 2023 and everything with .xxx in the name as part of it, tons of old data we aren't capable of d… --- ## [Parse different records in 1 document](https://discuss.elastic.co/t/parse-different-records-in-1-document/325471)
**Author:** [@Bart-d-sdlr](https://discuss.elastic.co/u/Bart-d-sdlr)\ **Replies:** 0\ **Last updated:** [February 14, 2023, 12:25pm UTC](https://discuss.elastic.co/t/parse-different-records-in-1-document/325471 "2023-02-14T12:25:52Z")
Hi, I have a (maybe stupid) question concerning parsing of custom logfile where the Date is the first record followed by the detailed lines (time,....) I don't use logstash, but filebeat and pipelines Simple example: … --- ## [500Gb text data per day - how to design elk solution](https://discuss.elastic.co/t/500gb-text-data-per-day-how-to-design-elk-solution/325432)
**Author:** [@coldcoder8502](https://discuss.elastic.co/u/coldcoder8502)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 12:19pm UTC](https://discuss.elastic.co/t/500gb-text-data-per-day-how-to-design-elk-solution/325432 "2023-02-14T12:19:15Z")
I have a device which is sending 500GB text data (logs) per day to my central server. I want to design a system using which user can: Apply exact-match filters and go through data using pagination Export PDF/CSV report… --- ## [How do I retrieve statistics from two CEPH clusters using metricbeat autodiscover?](https://discuss.elastic.co/t/how-do-i-retrieve-statistics-from-two-ceph-clusters-using-metricbeat-autodiscover/325386)
**Author:** [@eeijlar](https://discuss.elastic.co/u/eeijlar)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 11:54am UTC](https://discuss.elastic.co/t/how-do-i-retrieve-statistics-from-two-ceph-clusters-using-metricbeat-autodiscover/325386 "2023-02-14T11:54:15Z")
I am retrieving statistics from a CEPH cluster running in a kubernetes deployment using this values file: metricbeat: extraEnvs: - name: CEPH\_API\_USERNAME value: monitoring-ceph - name: CEPH\_API\_PASSWOR… --- ## [Remove json object from nested log](https://discuss.elastic.co/t/remove-json-object-from-nested-log/325468)
**Author:** [@Sharoze\_Meraj](https://discuss.elastic.co/u/Sharoze_Meraj)\ **Replies:** 0\ **Last updated:** [February 14, 2023, 12:03pm UTC](https://discuss.elastic.co/t/remove-json-object-from-nested-log/325468 "2023-02-14T12:03:08Z")
How can I use ruby code or some other filter plugin to detect and remove json object fields from my nested json logs. This is required because the fields can either be json objects or strings. If I remove the json objec… --- ## [Elasticsearch Master Quorum is lost](https://discuss.elastic.co/t/elasticsearch-master-quorum-is-lost/325459)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 3\ **Last updated:** [February 14, 2023, 11:46am UTC](https://discuss.elastic.co/t/elasticsearch-master-quorum-is-lost/325459 "2023-02-14T11:46:57Z")
Hello : We are having Elasticsearch version 7.8.0 , running with 6 node cluster. All 6 nodes were data and master nodes. We have lost 3 Nodes out of 6 , and now we have message in logfile \[2023-02-14T10:58:47,994\]\[WA… --- ## [Retrieve data having date from 1st, Jan to current date on a date field](https://discuss.elastic.co/t/retrieve-data-having-date-from-1st-jan-to-current-date-on-a-date-field/325451)
**Author:** [@kajalp](https://discuss.elastic.co/u/kajalp)\ **Replies:** 3\ **Last updated:** [February 14, 2023, 10:24am UTC](https://discuss.elastic.co/t/retrieve-data-having-date-from-1st-jan-to-current-date-on-a-date-field/325451 "2023-02-14T10:24:20Z")
Hi All, I have data with a date field having dates from last 3 years. What I want? I want to get all the records from Jan1st to current day at any given time over a year. I tried below query: GET index\_name/\_search … --- ## [Query performance measuring tool](https://discuss.elastic.co/t/query-performance-measuring-tool/325446)
**Author:** [@siddhartha\_c](https://discuss.elastic.co/u/siddhartha_c)\ **Replies:** 2\ **Last updated:** [February 14, 2023, 10:20am UTC](https://discuss.elastic.co/t/query-performance-measuring-tool/325446 "2023-02-14T10:20:06Z")
How do I monitor the search performance for an Index, wherein data is constantly getting indexed. I want to restrict the size of the Index wherein the search performance starts degrading with new incoming data. We have… --- ## [Logstash issue](https://discuss.elastic.co/t/logstash-issue/325414)
**Author:** [@namdev](https://discuss.elastic.co/u/namdev)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 10:00am UTC](https://discuss.elastic.co/t/logstash-issue/325414 "2023-02-14T10:00:44Z")
Hi , I am new to elk stack,I want to create a new field which is the difference between two dates field. I want to do it in logstash. The two dates field data is given. I am using filter like this but not getting the … --- ## [Kibana Visualize - Display count even if field not exists](https://discuss.elastic.co/t/kibana-visualize-display-count-even-if-field-not-exists/325246)
**Author:** [@Pedro\_Ventura](https://discuss.elastic.co/u/Pedro_Ventura)\ **Replies:** 3\ **Last updated:** [February 14, 2023, 9:41am UTC](https://discuss.elastic.co/t/kibana-visualize-display-count-even-if-field-not-exists/325246 "2023-02-14T09:41:02Z")
Hello! First time posting here, I've been looking around but haven't found anything to point me towards the right direction to solve my issue. I'm creating a data table which contains an aggregation by Terms for a date … --- ## [Ruby exception occurred: undefined method \`\*' for nil:NilClass](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-for-nil-nilclass/325411)
**Author:** [@mc96](https://discuss.elastic.co/u/mc96)\ **Replies:** 2\ **Last updated:** [February 14, 2023, 9:25am UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-for-nil-nilclass/325411 "2023-02-14T09:25:03Z")
I have a filter that is as follows: event.set('\[json\]\[event\]\[packetloss1\]', event.get('\[packets-received\]') / event.get('\[packets-sent\]') \* 100) event.set('\[json\]\[event\]\[packetlosspercentage\]', 100 - event.get('\[json\]… --- ## [Elastic keeps creating indices with replica:1](https://discuss.elastic.co/t/elastic-keeps-creating-indices-with-replica-1/325390)
**Author:** [@martijnomoda](https://discuss.elastic.co/u/martijnomoda)\ **Replies:** 2\ **Last updated:** [February 14, 2023, 9:26am UTC](https://discuss.elastic.co/t/elastic-keeps-creating-indices-with-replica-1/325390 "2023-02-14T09:26:01Z")
I have an cloud Elastic cluster with 1 hot node and 1 cold node. Because of this, I limited my replica to 0 when an index has been created. I have done this trough a index template with a high prioritity (99999), with th… --- ## [What is the time complexity of query a word in lucene?](https://discuss.elastic.co/t/what-is-the-time-complexity-of-query-a-word-in-lucene/325385)
**Author:** [@dan\_kim](https://discuss.elastic.co/u/dan_kim)\ **Replies:** 7\ **Last updated:** [February 14, 2023, 9:10am UTC](https://discuss.elastic.co/t/what-is-the-time-complexity-of-query-a-word-in-lucene/325385 "2023-02-14T09:10:22Z")
Hello! please let me know what is the time complexity of query in lucene index . for example, jus simple query to a index like "localhost:9200/index1/\_search?q={searchWord}" I know it is inverted index , but i think … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=632) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=634)