# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=481 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 482 --- ## [Kafka logstash logs are showing into filebeat logstash index](https://discuss.elastic.co/t/kafka-logstash-logs-are-showing-into-filebeat-logstash-index/337419)
**Author:** [@lalchand\_rajak](https://discuss.elastic.co/u/lalchand_rajak)\ **Replies:** 10\ **Last updated:** [July 14, 2023, 2:27pm UTC](https://discuss.elastic.co/t/kafka-logstash-logs-are-showing-into-filebeat-logstash-index/337419 "2023-07-14T14:27:36Z")
Hello, I have kafka-logstash conf and logstash reciveing the logs from kafka. here is the config. input { kafka { topics =\> \["sitlogtopic","locallogtopic"\] bootstrap\_servers =\> "ddr-kafkadev.pvt.cci… --- ## [Parsing firewall logs in logstash](https://discuss.elastic.co/t/parsing-firewall-logs-in-logstash/338405)
**Author:** [@secsec](https://discuss.elastic.co/u/secsec)\ **Replies:** 1\ **Last updated:** [July 14, 2023, 1:31pm UTC](https://discuss.elastic.co/t/parsing-firewall-logs-in-logstash/338405 "2023-07-14T13:31:25Z")
Hello, our sophos firewall are sending logs to filebeat, then filebeat send to logstash. In logstash im trying to separate field called "action" to be able to filter it under elasticsearch. So far no luck. I managed to … --- ## [Elastich search:unassigned shards: status yellow](https://discuss.elastic.co/t/elastich-search-unassigned-shards-status-yellow/338120)
**Author:** [@Deepika\_Gupta](https://discuss.elastic.co/u/Deepika_Gupta)\ **Replies:** 5\ **Last updated:** [July 14, 2023, 12:58pm UTC](https://discuss.elastic.co/t/elastich-search-unassigned-shards-status-yellow/338120 "2023-07-14T12:58:38Z")
Hello Team, My team is trying to create an Elasticsearch -cluster with Kibana. on one server Elasticsearch node works fine with zero shards. But another server is complaining about the status "yellow". Below is the u… --- ## [Facing java.io.EOFException: read past EOF exception and org.apache.lucene.index.CorruptIndexException: compound sub-files must have a valid codec header and footer: file is too small (0 bytes) in elastic 7.17.5](https://discuss.elastic.co/t/facing-java-io-eofexception-read-past-eof-exception-and-org-apache-lucene-index-corruptindexexception-compound-sub-files-must-have-a-valid-codec-header-and-footer-file-is-too-small-0-bytes-in-elastic-7-17-5/338370)
**Author:** [@Kesavan](https://discuss.elastic.co/u/Kesavan)\ **Replies:** 4\ **Last updated:** [July 14, 2023, 10:34am UTC](https://discuss.elastic.co/t/facing-java-io-eofexception-read-past-eof-exception-and-org-apache-lucene-index-corruptindexexception-compound-sub-files-must-have-a-valid-codec-header-and-footer-file-is-too-small-0-bytes-in-elastic-7-17-5/338370 "2023-07-14T10:34:03Z")
In one our environment we are facing the "CorruptIndexException". While analyzing the elastic log we found the below are the list of exception details: infinity\_infinity-elasticsearch.1.862455ajz1ca@WorkerNode03Prod … --- ## [Eck stack helm install, expose ingress](https://discuss.elastic.co/t/eck-stack-helm-install-expose-ingress/338399)
**Author:** [@simonebenati](https://discuss.elastic.co/u/simonebenati)\ **Replies:** 0\ **Last updated:** [July 14, 2023, 10:31am UTC](https://discuss.elastic.co/t/eck-stack-helm-install-expose-ingress/338399 "2023-07-14T10:31:30Z")
Hello, I installed eck operator via helm and then the eck stack via helm. Now I want to expose via ingress Elasticsearch but I am not able to find anywhere in the helm values or docs the value in order to expose an ingr… --- ## [Kibana data-table visualization not displaying all data rows](https://discuss.elastic.co/t/kibana-data-table-visualization-not-displaying-all-data-rows/337861)
**Author:** [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Replies:** 6\ **Last updated:** [July 14, 2023, 5:25am UTC](https://discuss.elastic.co/t/kibana-data-table-visualization-not-displaying-all-data-rows/337861 "2023-07-14T05:25:18Z")
I’m trying to create a Data Table visualization and I have below issue. My buckets selections as follows. In discover page, i can able to view the data , each fields value and its rows without any problem. But in v… --- ## [Logstash grok pattern for apache error log](https://discuss.elastic.co/t/logstash-grok-pattern-for-apache-error-log/337676)
**Author:** [@sanjeev1895](https://discuss.elastic.co/u/sanjeev1895)\ **Replies:** 4\ **Last updated:** [July 14, 2023, 4:59am UTC](https://discuss.elastic.co/t/logstash-grok-pattern-for-apache-error-log/337676 "2023-07-14T04:59:24Z")
Hi experts, Can any one tell me that how to configure the logstash grok custom pattern for apache web server error log. below is my apache web server sample error log, \[Fri Jun 09 08:26:38.311375 2023\] \[proxy\_fcgi:err… --- ## [In Elastic Cloud field type is correct i.e. keyword](https://discuss.elastic.co/t/in-elastic-cloud-field-type-is-correct-i-e-keyword/338368)
**Author:** [@Nishant\_Chauhan](https://discuss.elastic.co/u/Nishant_Chauhan)\ **Replies:** 0\ **Last updated:** [July 14, 2023, 4:35am UTC](https://discuss.elastic.co/t/in-elastic-cloud-field-type-is-correct-i-e-keyword/338368 "2023-07-14T04:35:13Z")
In Elastic Cloud, when I install any integration data ingest in proper field type i.e. keyword. However, in local deployment, when i install any integration data always comes in text field type. --- ## [How to update service account which is used to create snapshot](https://discuss.elastic.co/t/how-to-update-service-account-which-is-used-to-create-snapshot/337128)
**Author:** [@Aysh14](https://discuss.elastic.co/u/Aysh14)\ **Replies:** 5\ **Last updated:** [July 14, 2023, 3:38am UTC](https://discuss.elastic.co/t/how-to-update-service-account-which-is-used-to-create-snapshot/337128 "2023-07-14T03:38:16Z")
How to update service account which is used to create snapshot. I created repository from Kibana to snapshot the Elastic search indices. The snapshot location is GCS bucket. However, the repository is not getting verifie… --- ## [Snapshotter setup](https://discuss.elastic.co/t/snapshotter-setup/338365)
**Author:** [@Aysh14](https://discuss.elastic.co/u/Aysh14)\ **Replies:** 0\ **Last updated:** [July 14, 2023, 3:33am UTC](https://discuss.elastic.co/t/snapshotter-setup/338365 "2023-07-14T03:33:39Z")
Can I setup a new repository to take snapshots today onwards without having to restart the data and master nodes on the Elastic Search cluster ? I am currently using ES 7.16 . The old snapshots are not available and ther… --- ## [Kibana Watcher to trigger email by checking aggregation results with dynamic threshold value](https://discuss.elastic.co/t/kibana-watcher-to-trigger-email-by-checking-aggregation-results-with-dynamic-threshold-value/338357)
**Author:** [@Santosh1667](https://discuss.elastic.co/u/Santosh1667)\ **Replies:** 0\ **Last updated:** [July 13, 2023, 8:30pm UTC](https://discuss.elastic.co/t/kibana-watcher-to-trigger-email-by-checking-aggregation-results-with-dynamic-threshold-value/338357 "2023-07-13T20:30:17Z")
Hi , I had a Kibana watcher which will give aggregation buckets in below format distinct\_error\_count:\[ { key:"Error 1 Occured", distinct\_count:6 }, { key:"Error 2 Occured", distinct\_count:4 }, { key:"Error 3 Occured", d… --- ## [Charts are not properly embeding in the dash board](https://discuss.elastic.co/t/charts-are-not-properly-embeding-in-the-dash-board/338261)
**Author:** [@Kumar\_Abhinav](https://discuss.elastic.co/u/Kumar_Abhinav)\ **Replies:** 2\ **Last updated:** [July 13, 2023, 7:39pm UTC](https://discuss.elastic.co/t/charts-are-not-properly-embeding-in-the-dash-board/338261 "2023-07-13T19:39:14Z")
Hi, I have edited a field name with a Custom Label and edited the Format to Title Case. While it is showing well in the visualization i.e. (With Changes) when I import the visual into the dashboard it goes back to its d… --- ## [Is there a way to identify which visualization type was used to in a dashboard besides deducing and testing?](https://discuss.elastic.co/t/is-there-a-way-to-identify-which-visualization-type-was-used-to-in-a-dashboard-besides-deducing-and-testing/338043)
**Author:** [@brunofl](https://discuss.elastic.co/u/brunofl)\ **Replies:** 3\ **Last updated:** [July 13, 2023, 7:26pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-identify-which-visualization-type-was-used-to-in-a-dashboard-besides-deducing-and-testing/338043 "2023-07-13T19:26:31Z")
Hi, I am working on migration of kibana dashboards to a new instance and having trouble to find what type of table was used in certain visualizations. Is there a way to check this in the existing dashboard what was the e… --- ## [Terms aggregation over section of a keyword](https://discuss.elastic.co/t/terms-aggregation-over-section-of-a-keyword/338351)
**Author:** [@tmslara.a](https://discuss.elastic.co/u/tmslara.a)\ **Replies:** 1\ **Last updated:** [July 13, 2023, 5:55pm UTC](https://discuss.elastic.co/t/terms-aggregation-over-section-of-a-keyword/338351 "2023-07-13T17:55:59Z")
Hi, I have an index with a keyword field. The values are in the form '\\' where I know the code is a three figures number (e.g., in '123456789' the code is 123). I want to perform an aggregation like a term… --- ## [Kibana savej object giving error on import: migrating from 7.9.1 to 8.7.1 version full elk stack](https://discuss.elastic.co/t/kibana-savej-object-giving-error-on-import-migrating-from-7-9-1-to-8-7-1-version-full-elk-stack/338220)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 8\ **Last updated:** [July 13, 2023, 6:35pm UTC](https://discuss.elastic.co/t/kibana-savej-object-giving-error-on-import-migrating-from-7-9-1-to-8-7-1-version-full-elk-stack/338220 "2023-07-13T18:35:47Z")
Hello All, I'm migrating my full elk stack stack from 7.9.1 to 8.7.1 and facing issues while importing saved object in 8.7.1,below is the error in second image: How do I save saved object and download -shown below 1st … --- ## [Elasticsearch creating different indices with identical data](https://discuss.elastic.co/t/elasticsearch-creating-different-indices-with-identical-data/338352)
**Author:** [@mfisher](https://discuss.elastic.co/u/mfisher)\ **Replies:** 5\ **Last updated:** [July 13, 2023, 6:30pm UTC](https://discuss.elastic.co/t/elasticsearch-creating-different-indices-with-identical-data/338352 "2023-07-13T18:30:00Z")
I recently moved from ELK stack 7.X to 8.8.2. I'm using my old Logstash pipline confs. For some reason Elasticsearch/Kibana is showing each individual index but each index as the same data. I don't think its a datavie… --- ## [Methods to Enroll Kibana](https://discuss.elastic.co/t/methods-to-enroll-kibana/338164)
**Author:** [@ANUBHAV\_GUPTA](https://discuss.elastic.co/u/ANUBHAV_GUPTA)\ **Replies:** 5\ **Last updated:** [July 13, 2023, 6:12pm UTC](https://discuss.elastic.co/t/methods-to-enroll-kibana/338164 "2023-07-13T18:12:03Z")
Hi there, Is there a way to enroll kibana other than the mentioned 2 below:- bin/elasticsearch-create-enrollment-token copy the enrollment token from the elasticsearch stdout Note: I am using docker to deploy these t… --- ## [Scripted field was used to show traffic light image up or down in index pattern 7.9.1 kibana,8.8.2 dont support,wht is alternative to implement?](https://discuss.elastic.co/t/scripted-field-was-used-to-show-traffic-light-image-up-or-down-in-index-pattern-7-9-1-kibana-8-8-2-dont-support-wht-is-alternative-to-implement/338345)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 1\ **Last updated:** [July 13, 2023, 3:35pm UTC](https://discuss.elastic.co/t/scripted-field-was-used-to-show-traffic-light-image-up-or-down-in-index-pattern-7-9-1-kibana-8-8-2-dont-support-wht-is-alternative-to-implement/338345 "2023-07-13T15:35:08Z")
In kinbana 7.9.1 I used to use scripted fields in index pattern option ,now going forward its not supported in future version.We want this feature of tarffic light image show green or red. Its recommended to use run tim… --- ## [Filebeat Syslog no listening port](https://discuss.elastic.co/t/filebeat-syslog-no-listening-port/336969)
**Author:** [@mc.gyver.reboot](https://discuss.elastic.co/u/mc.gyver.reboot)\ **Replies:** 15\ **Last updated:** [July 13, 2023, 2:35pm UTC](https://discuss.elastic.co/t/filebeat-syslog-no-listening-port/336969 "2023-07-13T14:35:45Z")
Good morning, Configuration: Ubuntu version 22 Filebeat version 8.8.1 Aucun message d'erreur au lancement de Filebeat After hours of searching and testing, I can't find why Filebeat isn't listening on the ports I te… --- ## [Anomaly Detection Rule Won't Send Email](https://discuss.elastic.co/t/anomaly-detection-rule-wont-send-email/338244)
**Author:** [@McJava1967](https://discuss.elastic.co/u/McJava1967)\ **Replies:** 6\ **Last updated:** [July 13, 2023, 2:02pm UTC](https://discuss.elastic.co/t/anomaly-detection-rule-wont-send-email/338244 "2023-07-13T14:02:45Z")
Hi all. I'm evaluating Anomaly alerting using a locally hosted Platinum trial. In short, the anomaly detection Job itself is working. I can see anomalies in the results. And I have set up a Rule with a Connector. I… --- ## [Elasticsearch Cluster Health watch Watcher](https://discuss.elastic.co/t/elasticsearch-cluster-health-watch-watcher/338321)
**Author:** [@Shalinicts](https://discuss.elastic.co/u/Shalinicts)\ **Replies:** 8\ **Last updated:** [July 13, 2023, 1:36pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-health-watch-watcher/338321 "2023-07-13T13:36:55Z")
Hi Team, I am trying to create a watcher for cluster health check (Clluster is 3 master and 5 data node ) as per Elastic documentation In the input section it is referred to provide host as host:localhost "input" :… --- ## [ECE & Watcher: Trouble sending API key to ECE](https://discuss.elastic.co/t/ece-watcher-trouble-sending-api-key-to-ece/300980)
**Author:** [@Apprentice](https://discuss.elastic.co/u/Apprentice)\ **Replies:** 1\ **Last updated:** [July 13, 2023, 1:05pm UTC](https://discuss.elastic.co/t/ece-watcher-trouble-sending-api-key-to-ece/300980 "2023-07-13T13:05:45Z")
I am trying to create a Watcher using information from the ECE API as input. However I am having trouble getting authenticated. This is the Input for the watcher: "input": { "http" : { "request" : { "s… --- ## [Problem to add new date field in filter logstash](https://discuss.elastic.co/t/problem-to-add-new-date-field-in-filter-logstash/338107)
**Author:** [@shayn](https://discuss.elastic.co/u/shayn)\ **Replies:** 3\ **Last updated:** [July 13, 2023, 12:27pm UTC](https://discuss.elastic.co/t/problem-to-add-new-date-field-in-filter-logstash/338107 "2023-07-13T12:27:22Z")
i have date field called case\_start\_time in format of date and time . i am trying to add new field called case\_day which will cut the date without the time from case\_start\_time . case\_start\_time: 09/07/23 23:54:26 ca… --- ## [Logstash forwarding connection refused](https://discuss.elastic.co/t/logstash-forwarding-connection-refused/338293)
**Author:** [@willsy](https://discuss.elastic.co/u/willsy)\ **Replies:** 2\ **Last updated:** [July 13, 2023, 11:29am UTC](https://discuss.elastic.co/t/logstash-forwarding-connection-refused/338293 "2023-07-13T11:29:53Z")
Hello, I am trying to forward logs to any other location for the moment however i have the following error when trying to forward any data what so ever. I have a netcat listener on the opposite end and can see the incom… --- ## [Controlled rotation of elasticsearch data nodes while enabling the shard allocation awareness](https://discuss.elastic.co/t/controlled-rotation-of-elasticsearch-data-nodes-while-enabling-the-shard-allocation-awareness/338269)
**Author:** [@veerachenna](https://discuss.elastic.co/u/veerachenna)\ **Replies:** 7\ **Last updated:** [July 13, 2023, 10:44am UTC](https://discuss.elastic.co/t/controlled-rotation-of-elasticsearch-data-nodes-while-enabling-the-shard-allocation-awareness/338269 "2023-07-13T10:44:16Z")
Hi All, We are trying to enable the shard allocation awareness on the elasticsearch cluster on "zone" attribute while rotating the data nodes one after the other. We wanted to achieve this in more controlled manner. Ini… --- ## [Logstash pipeline Http output plugin error "\[HTTP Output Failure\] Encountered non-2xx HTTP code 400"](https://discuss.elastic.co/t/logstash-pipeline-http-output-plugin-error-http-output-failure-encountered-non-2xx-http-code-400/338116)
**Author:** [@PodarcisMuralis](https://discuss.elastic.co/u/PodarcisMuralis)\ **Replies:** 1\ **Last updated:** [July 13, 2023, 9:55am UTC](https://discuss.elastic.co/t/logstash-pipeline-http-output-plugin-error-http-output-failure-encountered-non-2xx-http-code-400/338116 "2023-07-13T09:55:29Z")
Hi all, I have a logstash output http plugin: output { if \[@metadata\]\[index\_to\_delete\] == "first\_index" or \[@metadata\]\[index\_to\_delete\] == "second\_index" { http { id =\> "http\_index\_delete" … --- ## [Reduce storage taken by specific index ? Freeze index ? Frozen tier ? Cold tier?](https://discuss.elastic.co/t/reduce-storage-taken-by-specific-index-freeze-index-frozen-tier-cold-tier/338311)
**Author:** [@mlng54](https://discuss.elastic.co/u/mlng54)\ **Replies:** 0\ **Last updated:** [July 13, 2023, 9:53am UTC](https://discuss.elastic.co/t/reduce-storage-taken-by-specific-index-freeze-index-frozen-tier-cold-tier/338311 "2023-07-13T09:53:13Z")
Hi everyone, I recently experienced a DDoS attack on my Apache server. The logs are sent to Elasticsearch, so my last indices are around 70Gb/day. I have not configured ILM on my ELK stack yet but I would like to reduce… --- ## [Metricbeat - how to create two different index templates from me metricbeat.yml](https://discuss.elastic.co/t/metricbeat-how-to-create-two-different-index-templates-from-me-metricbeat-yml/338298)
**Author:** [@Terkea](https://discuss.elastic.co/u/Terkea)\ **Replies:** 0\ **Last updated:** [July 13, 2023, 9:01am UTC](https://discuss.elastic.co/t/metricbeat-how-to-create-two-different-index-templates-from-me-metricbeat-yml/338298 "2023-07-13T09:01:53Z")
Hello guys, I want to create two different index templates with different ILM policies for each module that I use in metricbeat. My metricbeat.yml metricbeat: modules: - hosts: - http://localhost:5067 metr… --- ## [Elasticsearch Stack monitoring feature does not work when using metricbeat to monitor elasticsearch cluster](https://discuss.elastic.co/t/elasticsearch-stack-monitoring-feature-does-not-work-when-using-metricbeat-to-monitor-elasticsearch-cluster/333451)
**Author:** [@ramdas](https://discuss.elastic.co/u/ramdas)\ **Replies:** 13\ **Last updated:** [July 13, 2023, 8:51am UTC](https://discuss.elastic.co/t/elasticsearch-stack-monitoring-feature-does-not-work-when-using-metricbeat-to-monitor-elasticsearch-cluster/333451 "2023-07-13T08:51:07Z")
Hi, I am using elasticsearch/kibana 8.7 and also using metricbeat 8.7 to monitor elasticsearch cluster in kubernetes environment. following is the elasticsearch module config in metricbeat: - module: elasticsearch x… --- ## [How to support complex filters in nested aggregation?](https://discuss.elastic.co/t/how-to-support-complex-filters-in-nested-aggregation/337444)
**Author:** [@crowod](https://discuss.elastic.co/u/crowod)\ **Replies:** 1\ **Last updated:** [July 13, 2023, 8:51am UTC](https://discuss.elastic.co/t/how-to-support-complex-filters-in-nested-aggregation/337444 "2023-07-13T08:51:56Z")
Here is my index mapping: { "mappings": { "properties": { "non\_nested\_field": { "type": "keyword" }, "nested\_field": { "type": "nested", "properties": { "subfiel… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=480) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=482)