# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=791 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 792 --- ## [After filebeat upgrades & preserving registry it sending duplicate events](https://discuss.elastic.co/t/after-filebeat-upgrades-preserving-registry-it-sending-duplicate-events/313701)
**Author:** [@krish0608](https://discuss.elastic.co/u/krish0608)\ **Replies:** 14\ **Last updated:** [September 16, 2022, 4:15pm UTC](https://discuss.elastic.co/t/after-filebeat-upgrades-preserving-registry-it-sending-duplicate-events/313701 "2022-09-16T16:15:01Z")
I am having a trouble with duplicate events in elasticsearch after updating it to 7.17.3 from 5.6.5. In case of Linux its working as expected but in windows its sending duplicate events. Below are my configurations: ===… --- ## [Query DSL in kibana discover](https://discuss.elastic.co/t/query-dsl-in-kibana-discover/314526)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 2\ **Last updated:** [September 16, 2022, 6:58pm UTC](https://discuss.elastic.co/t/query-dsl-in-kibana-discover/314526 "2022-09-16T18:58:46Z")
I want to run this below query in kibana discover filter "aggs": { "response\_term": { "terms": { "field": "ResponseCode.keyword", "size": 100000 }, "aggs": { "response": { "… --- ## [Create a table where the columns change depending on which fields exist for the data returned (given filters and searches)](https://discuss.elastic.co/t/create-a-table-where-the-columns-change-depending-on-which-fields-exist-for-the-data-returned-given-filters-and-searches/314596)
**Author:** [@akitatak](https://discuss.elastic.co/u/akitatak)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 5:55pm UTC](https://discuss.elastic.co/t/create-a-table-where-the-columns-change-depending-on-which-fields-exist-for-the-data-returned-given-filters-and-searches/314596 "2022-09-16T17:55:21Z")
Hello, So in my index pattern objects\* has multiple different object types within it that all have their own fields. Example: Object 1: type=house, id=0, color=blue, size=2000, city=Portland) Object 2: type=car, id=1… --- ## [Xpack security troubles](https://discuss.elastic.co/t/xpack-security-troubles/314594)
**Author:** [@Mo\_the\_literate](https://discuss.elastic.co/u/Mo_the_literate)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 5:30pm UTC](https://discuss.elastic.co/t/xpack-security-troubles/314594 "2022-09-16T17:30:58Z")
Hi guys, I have installed graylog-server 4.3.5-1 on Debian 11.4 as well as elasticsearch 7.17.5 When I run the setup without uncommenting #xpack.security.enabled: true I get these here warnings in logs: "Elasticsear… --- ## [Unable to see winlogbeat logs in the analytics discover](https://discuss.elastic.co/t/unable-to-see-winlogbeat-logs-in-the-analytics-discover/314590)
**Author:** [@Neyo](https://discuss.elastic.co/u/Neyo)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 4:58pm UTC](https://discuss.elastic.co/t/unable-to-see-winlogbeat-logs-in-the-analytics-discover/314590 "2022-09-16T16:58:28Z")
I'm unable to see the logs in the dashboard, Please give some advice: winlogbeat.yml file: ###################### Winlogbeat Configuration Example ######################## # This file is an example configuration file … --- ## [Break down log messages to keywords](https://discuss.elastic.co/t/break-down-log-messages-to-keywords/314588)
**Author:** [@Mark\_S](https://discuss.elastic.co/u/Mark_S)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 3:39pm UTC](https://discuss.elastic.co/t/break-down-log-messages-to-keywords/314588 "2022-09-16T15:39:40Z")
I want to index logs from different applications in one index. However some applications don't use exactly the same log format.... How can I use a grok pattern in the input section of the config file to break down the … --- ## [Kibana Dashboard Enlarge image on click](https://discuss.elastic.co/t/kibana-dashboard-enlarge-image-on-click/314404)
**Author:** [@mirokrastev](https://discuss.elastic.co/u/mirokrastev)\ **Replies:** 5\ **Last updated:** [September 16, 2022, 3:24pm UTC](https://discuss.elastic.co/t/kibana-dashboard-enlarge-image-on-click/314404 "2022-09-16T15:24:28Z")
Hello, Please take a look at the attached snippet. Basically we have a table, which has images into it. My goal here is to be able to enlarge the image, when I click on it. Can you let me know if there is such functiona… --- ## [Logstash from SQL Server to Elasticsearch character encoding problem](https://discuss.elastic.co/t/logstash-from-sql-server-to-elasticsearch-character-encoding-problem/314587)
**Author:** [@Startech](https://discuss.elastic.co/u/Startech)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 3:23pm UTC](https://discuss.elastic.co/t/logstash-from-sql-server-to-elasticsearch-character-encoding-problem/314587 "2022-09-16T15:23:25Z")
Hi, I am using ELK stack v8.4.1 and trying to integrate data between SQL Server and Elasticsearch via Logstash. My source table includes Turkish characters (collation SQL\_Latin1\_General\_CP1\_CI\_AS). When Logstash writes … --- ## [How to run a pipeline from a pipeline](https://discuss.elastic.co/t/how-to-run-a-pipeline-from-a-pipeline/314562)
**Author:** [@reed](https://discuss.elastic.co/u/reed)\ **Replies:** 4\ **Last updated:** [September 16, 2022, 2:37pm UTC](https://discuss.elastic.co/t/how-to-run-a-pipeline-from-a-pipeline/314562 "2022-09-16T14:37:30Z")
Hi All, in my project I have 3 pipelines, I know that a pipeline can be scheduled but in my case I need to run the third one at the end of other two. There is a way to do this? run the pipelines in order: pipeline1 -\>… --- ## [Filebeat hogged the IO](https://discuss.elastic.co/t/filebeat-hogged-the-io/314581)
**Author:** [@silence-linhl](https://discuss.elastic.co/u/silence-linhl)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 2:35pm UTC](https://discuss.elastic.co/t/filebeat-hogged-the-io/314581 "2022-09-16T14:35:20Z")
filebeat hogged the IO After starting filebeat, I found that the IO of the machine became very high. After the digging, it was found that filebeat kept writing to disk a file named checkpoint.new after it was started. S… --- ## [UDP Listener died, address already in use](https://discuss.elastic.co/t/udp-listener-died-address-already-in-use/314281)
**Author:** [@sirineb](https://discuss.elastic.co/u/sirineb)\ **Replies:** 4\ **Last updated:** [September 16, 2022, 2:24pm UTC](https://discuss.elastic.co/t/udp-listener-died-address-already-in-use/314281 "2022-09-16T14:24:49Z")
Hi, I have two pipeline configurations located in /etc/logstash/conf.d. I can make work both seperately by running /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/\*\*.conf -r, and it sends data to elasticsearch … --- ## [How to handle inconsistent field types](https://discuss.elastic.co/t/how-to-handle-inconsistent-field-types/314064)
**Author:** [@elastimatic](https://discuss.elastic.co/u/elastimatic)\ **Replies:** 3\ **Last updated:** [September 16, 2022, 2:12pm UTC](https://discuss.elastic.co/t/how-to-handle-inconsistent-field-types/314064 "2022-09-16T14:12:51Z")
I am trying to index a JSON object (comes from Postgres) that has sometimes-inconsistent types that is tripping me up. an example would be: { "financials": { "amount": "100" } }, { "financials": { … --- ## [Elastic Agent Tomcat Integration Error](https://discuss.elastic.co/t/elastic-agent-tomcat-integration-error/314579)
**Author:** [@DougR](https://discuss.elastic.co/u/DougR)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 1:53pm UTC](https://discuss.elastic.co/t/elastic-agent-tomcat-integration-error/314579 "2022-09-16T13:53:40Z")
What versions of Tomcat are supported by the Elastic Agent Tomcat integration? I have an instance which runs an older version of Tomcat (8.5), and I get dissect\_parsing\_error in my events, which are pushed to Elasticsear… --- ## [Issue with logstash agent](https://discuss.elastic.co/t/issue-with-logstash-agent/314578)
**Author:** [@yas](https://discuss.elastic.co/u/yas)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 1:34pm UTC](https://discuss.elastic.co/t/issue-with-logstash-agent/314578 "2022-09-16T13:34:02Z")
Having an issue with Logstash agents ( linux hosts ) sending files to a server - what seems to happen is on the agent starting up , the first few rows of the logs are sent after which it just stops and nothing further is… --- ## [Custom API integration](https://discuss.elastic.co/t/custom-api-integration/314538)
**Author:** [@alextg](https://discuss.elastic.co/u/alextg)\ **Replies:** 1\ **Last updated:** [September 16, 2022, 11:51am UTC](https://discuss.elastic.co/t/custom-api-integration/314538 "2022-09-16T11:51:19Z")
Hello, I'm running into issues while implementing the Custom API integration which sends API get requests. I have setup successfully other integrations using the Custom Logs integration, however, the Custom API integrat… --- ## [Auditbeat logs write to /var/log/messages](https://discuss.elastic.co/t/auditbeat-logs-write-to-var-log-messages/314361)
**Author:** [@masonlu2014](https://discuss.elastic.co/u/masonlu2014)\ **Replies:** 3\ **Last updated:** [September 16, 2022, 12:32pm UTC](https://discuss.elastic.co/t/auditbeat-logs-write-to-var-log-messages/314361 "2022-09-16T12:32:09Z")
hi guys, we are sawing weird issue, we are using auditbeat as pod running on our kubenets cluster, however in some day, we saw there is a large audit events has been write to our work node /var/log/message, and i can co… --- ## [Unable to acquire permit to use snapshot files during recovery](https://discuss.elastic.co/t/unable-to-acquire-permit-to-use-snapshot-files-during-recovery/313935)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 10\ **Last updated:** [September 16, 2022, 12:14pm UTC](https://discuss.elastic.co/t/unable-to-acquire-permit-to-use-snapshot-files-during-recovery/313935 "2022-09-16T12:14:28Z")
Hello there, i want to ask something about this. i haven't made any changes and i just wanted to check my cluster but suddenly one of my data node was offline and my cluster state from green to yellow. i found much log … --- ## [Process events from split](https://discuss.elastic.co/t/process-events-from-split/314564)
**Author:** [@sirReeall](https://discuss.elastic.co/u/sirReeall)\ **Replies:** 1\ **Last updated:** [September 16, 2022, 10:01am UTC](https://discuss.elastic.co/t/process-events-from-split/314564 "2022-09-16T10:01:34Z")
Is it possible to have logstash process the events generated from a split? For example, give the following: 2022-09-06 23:39:01.034+0000 INFO \[my.java.class\] Process started \\n some lines \\n some more lines \\n … --- ## [Create elastic field alias](https://discuss.elastic.co/t/create-elastic-field-alias/313966)
**Author:** [@sirReeall](https://discuss.elastic.co/u/sirReeall)\ **Replies:** 4\ **Last updated:** [September 16, 2022, 9:25am UTC](https://discuss.elastic.co/t/create-elastic-field-alias/313966 "2022-09-16T09:25:34Z")
Hello, Is it possible for logstash to alias fields as described in the elastic docs below? --- ## [Spring + ES 8 server, can't make it compatible](https://discuss.elastic.co/t/spring-es-8-server-cant-make-it-compatible/314557)
**Author:** [@lhberg](https://discuss.elastic.co/u/lhberg)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 8:16am UTC](https://discuss.elastic.co/t/spring-es-8-server-cant-make-it-compatible/314557 "2022-09-16T08:16:36Z")
Hi! So, I'm new to ES and been running ES locally through docker on ES 7.17.3 with RestHighLevelClient, Spring Data ES and running queries with QueryBuilders etc. Now I've been hosting Elastic Search through your/their… --- ## [Invalid Index Name-Index name must be lowercase error](https://discuss.elastic.co/t/invalid-index-name-index-name-must-be-lowercase-error/314287)
**Author:** [@tapas](https://discuss.elastic.co/u/tapas)\ **Replies:** 15\ **Last updated:** [September 16, 2022, 7:55am UTC](https://discuss.elastic.co/t/invalid-index-name-index-name-must-be-lowercase-error/314287 "2022-09-16T07:55:52Z")
I am getting error in logstash logs saying that the index is invalid and reason is it should be lower case.but the index name in the log is different from the index i declared in the logstash.config file.in log file it i… --- ## [Kibana visualization to show intime value count of different fieldski](https://discuss.elastic.co/t/kibana-visualization-to-show-intime-value-count-of-different-fieldski/312642)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 3\ **Last updated:** [September 16, 2022, 7:42am UTC](https://discuss.elastic.co/t/kibana-visualization-to-show-intime-value-count-of-different-fieldski/312642 "2022-09-16T07:42:32Z")
Hello All, How to make TSVB visualization displaying all the count values for given filelds. Here I'm trying to dispaly the index updated,nodes travered,entrie process and chunk "intime" count value wrt statistic time.… --- ## [Slow performance Logstash/Filebeat](https://discuss.elastic.co/t/slow-performance-logstash-filebeat/314040)
**Author:** [@AsapNoCare](https://discuss.elastic.co/u/AsapNoCare)\ **Replies:** 6\ **Last updated:** [September 16, 2022, 7:19am UTC](https://discuss.elastic.co/t/slow-performance-logstash-filebeat/314040 "2022-09-16T07:19:12Z")
Hi everyone, It’s the first time I ask for help so feel free to tell me if you need more information from me. Something else, English is not my native language so sorry in advance for the mistakes. Here is my data flo… --- ## [Logstash filter taking. high cpu](https://discuss.elastic.co/t/logstash-filter-taking-high-cpu/314550)
**Author:** [@krishan.kumar](https://discuss.elastic.co/u/krishan.kumar)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 6:35am UTC](https://discuss.elastic.co/t/logstash-filter-taking-high-cpu/314550 "2022-09-16T06:35:54Z")
Hi, My logstash grok pattern taking high CPU This is my log format I, \[2022-09-16T06:27:40.386003 #13375\] INFO -- : \[s45g0-e726-56y7-adcf-3382276c1a77\] \[user\_id: xxxxxx, portfolio\_id: xxxxxxx\] {"method":"GET","path":… --- ## [Kibana No monitoring data found](https://discuss.elastic.co/t/kibana-no-monitoring-data-found/313677)
**Author:** [@WenhaoXu](https://discuss.elastic.co/u/WenhaoXu)\ **Replies:** 15\ **Last updated:** [September 16, 2022, 6:46am UTC](https://discuss.elastic.co/t/kibana-no-monitoring-data-found/313677 "2022-09-16T06:46:37Z")
version : 8.2.3 kibana No monitoring data found get cluster info not found . is it the reason? And GET /\_cat/indices?v&s=index has .monitoring-kibana-7-2022.09.05 .monitoring-es-7-2022.09.05 --- ## [Nodes do not release RAM when idle](https://discuss.elastic.co/t/nodes-do-not-release-ram-when-idle/314321)
**Author:** [@Thijsvdp](https://discuss.elastic.co/u/Thijsvdp)\ **Replies:** 17\ **Last updated:** [September 16, 2022, 6:17am UTC](https://discuss.elastic.co/t/nodes-do-not-release-ram-when-idle/314321 "2022-09-16T06:17:51Z")
Hi all, I have experienced some weird behavior on my development cluster while reading an index using the Elasticsearch-Hadoop connector. I am benchmarking the read performance, by running some scripts that read the ful… --- ## [Transform Data - Vega Kibana](https://discuss.elastic.co/t/transform-data-vega-kibana/314545)
**Author:** [@Irwan\_Kurniawan](https://discuss.elastic.co/u/Irwan_Kurniawan)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 3:24am UTC](https://discuss.elastic.co/t/transform-data-vega-kibana/314545 "2022-09-16T03:24:49Z")
How to transform data that I already get from index like this: Into the format like this: So the Type and Value can be separated in different column. I only know how to do this in Vega-Lite but not in Vega. --- ## [When does the pending count become 0](https://discuss.elastic.co/t/when-does-the-pending-count-become-0/313649)
**Author:** [@hanhee](https://discuss.elastic.co/u/hanhee)\ **Replies:** 3\ **Last updated:** [September 16, 2022, 12:36am UTC](https://discuss.elastic.co/t/when-does-the-pending-count-become-0/313649 "2022-09-16T00:36:58Z")
hi I do not speak English well. sorry my servers log file create 100/1min every day and delete all 12:00 pm but registry json not show flag "remove" and log.json, number.json just keeps getting bigger. i wonder Wh… --- ## [How elastic bulk data](https://discuss.elastic.co/t/how-elastic-bulk-data/314513)
**Author:** [@avnere](https://discuss.elastic.co/u/avnere)\ **Replies:** 3\ **Last updated:** [September 16, 2022, 12:30am UTC](https://discuss.elastic.co/t/how-elastic-bulk-data/314513 "2022-09-16T00:30:00Z")
How Java client send the data source to the cluster? bulkRequest.add(new IndexRequest().source(object)); client.bulk(\_bulkRequest, RequestOptions.DEFAULT); What exactly it does after converting the data to JSON? Does… --- ## [Problem with aggs queries returning null?](https://discuss.elastic.co/t/problem-with-aggs-queries-returning-null/314529)
**Author:** [@JamesD\_7](https://discuss.elastic.co/u/JamesD_7)\ **Replies:** 1\ **Last updated:** [September 15, 2022, 11:17pm UTC](https://discuss.elastic.co/t/problem-with-aggs-queries-returning-null/314529 "2022-09-15T23:17:09Z")
Hello, I'm having trouble getting a simple aggs query to return data. GET mydata1/\_search { "aggs": { "avg\_num\_days": { "avg": { "field": "myField" } } } } I was getting an error: "error"… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=790) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=792)