# Latest **URL:** https://discuss.elastic.co/latest.md?page=1350 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1351 --- ## [Kibana Visualization filtering data based on aggregation](https://discuss.elastic.co/t/kibana-visualization-filtering-data-based-on-aggregation/291610)
**Author:** [@anas\_pp](https://discuss.elastic.co/u/anas_pp)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 9:34am UTC](https://discuss.elastic.co/t/kibana-visualization-filtering-data-based-on-aggregation/291610 "2021-12-13T09:34:42Z")
I have requirement that to show the count of running jobs and completed in Pie chart . I am receiving status of jobs in real time . To show the count of the jobs that are currently in running state I have to filter those… --- ## [Alternative to capture\_body](https://discuss.elastic.co/t/alternative-to-capture-body/291607)
**Author:** [@Bhavani\_Ananth](https://discuss.elastic.co/u/Bhavani_Ananth)\ **Replies:** 1\ **Last updated:** [December 13, 2021, 9:26am UTC](https://discuss.elastic.co/t/alternative-to-capture-body/291607 "2021-12-13T09:26:04Z")
Hello Team, There is a new requirement wherein we need to parse a certain tracking number that is part of the http url or part of the request body. The one in the url can be easily retrieved.(http.url and using logstas… --- ## [Github URL to set on op.jwkset\_path parameter](https://discuss.elastic.co/t/github-url-to-set-on-op-jwkset-path-parameter/291474)
**Author:** [@Fabio\_Peruchi](https://discuss.elastic.co/u/Fabio_Peruchi)\ **Replies:** 5\ **Last updated:** [December 13, 2021, 8:12am UTC](https://discuss.elastic.co/t/github-url-to-set-on-op-jwkset-path-parameter/291474 "2021-12-13T08:12:38Z")
Hello. I'm struggling to find where is the GitHub URL to set here on this parameter from this documentation: Configuring single sign-on to the Elastic Stack using OpenID Connect | Elasticsearch Guide \[7.16\] | Elastic o… --- ## [Logstash Error: "Your settings are invalid"](https://discuss.elastic.co/t/logstash-error-your-settings-are-invalid/291544)
**Author:** [@neerajg](https://discuss.elastic.co/u/neerajg)\ **Replies:** 2\ **Last updated:** [December 13, 2021, 7:52am UTC](https://discuss.elastic.co/t/logstash-error-your-settings-are-invalid/291544 "2021-12-13T07:52:03Z")
Hi guys, I just upgraded elk stack from 7.15.0 to 7.16.0. Everything is working fine except Logstash. It ends up failing to start. I checked the system logs and found the following error. Error: logstash\[20212\]: Ope… --- ## [Fscrawler - update existing record](https://discuss.elastic.co/t/fscrawler-update-existing-record/291597)
**Author:** [@Andrej\_K](https://discuss.elastic.co/u/Andrej_K)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 7:37am UTC](https://discuss.elastic.co/t/fscrawler-update-existing-record/291597 "2021-12-13T07:37:18Z")
Hello, I need to search documents by its content and some custom customer data. So my idea is to use fscrawler to extract content from documents and keep it with custom customer data in one index for easy searching. So… --- ## [How to get the field names which have no data or null in elasticsearch dynamically?](https://discuss.elastic.co/t/how-to-get-the-field-names-which-have-no-data-or-null-in-elasticsearch-dynamically/291591)
**Author:** [@Saurabh\_A](https://discuss.elastic.co/u/Saurabh_A)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 6:17am UTC](https://discuss.elastic.co/t/how-to-get-the-field-names-which-have-no-data-or-null-in-elasticsearch-dynamically/291591 "2021-12-13T06:17:53Z")
How do we get the field names with no data in them? For example, in an index, there is a field that is present in the mapping but no data is indexed in this case. So I want a list of all the field names or keys where the… --- ## [Metricbeat showing same ip of elasticsearch nodes](https://discuss.elastic.co/t/metricbeat-showing-same-ip-of-elasticsearch-nodes/291586)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 5:40am UTC](https://discuss.elastic.co/t/metricbeat-showing-same-ip-of-elasticsearch-nodes/291586 "2021-12-13T05:40:04Z")
1 week ago i upgrade elk stack from 7.13 to 7.15.Everything is fine but today i notice that in stack monitoring page when i click on nodes ,the ip of nodes is same for cross verification i ran GET /cat/nodes it showing m… --- ## [Zero-day-exploit in log4j2](https://discuss.elastic.co/t/zero-day-exploit-in-log4j2/291583)
**Author:** [@josh12](https://discuss.elastic.co/u/josh12)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 4:30am UTC](https://discuss.elastic.co/t/zero-day-exploit-in-log4j2/291583 "2021-12-13T04:30:33Z")
Hi all, I'm bringing up my elastic stack via docker which consists of Elasticsearch, logstash and kibana. Is this sufficient or is there any other options needed to be performed. I'm using 7.15 version. What version are … --- ## [Output to different elastic search Indices based on the string](https://discuss.elastic.co/t/output-to-different-elastic-search-indices-based-on-the-string/291568)
**Author:** [@mussa572](https://discuss.elastic.co/u/mussa572)\ **Replies:** 3\ **Last updated:** [December 13, 2021, 3:40am UTC](https://discuss.elastic.co/t/output-to-different-elastic-search-indices-based-on-the-string/291568 "2021-12-13T03:40:17Z")
Hi I have a Category field (cat) which can have following value and future expect to add more. BL LA KA Currently I have following code to output to the different Elasticsearch indices from our Logstash pipeline . I… --- ## [Collecting logs using filebeat over http](https://discuss.elastic.co/t/collecting-logs-using-filebeat-over-http/291578)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 3:14am UTC](https://discuss.elastic.co/t/collecting-logs-using-filebeat-over-http/291578 "2021-12-13T03:14:37Z")
Hi Team, Wondering if the filebeat.log.type can be http or https? I need to pull the logs using https path and using basic auth. Please let me know if that is possible? I am aware it can be possible with httpjson; howe… --- ## [Resources for learning ElasticSearch](https://discuss.elastic.co/t/resources-for-learning-elasticsearch/291536)
**Author:** [@rav3n](https://discuss.elastic.co/u/rav3n)\ **Replies:** 2\ **Last updated:** [December 12, 2021, 7:19pm UTC](https://discuss.elastic.co/t/resources-for-learning-elasticsearch/291536 "2021-12-12T19:19:50Z")
Hey people, Just started using Elasticsearch for my personal use. So, last night I picked one of the tool from Elastic stack and started reading about it. I came across to this book named "Elasticsearch: The Definitive … --- ## [Kibana Single Click Drilldown events.value Array Issue](https://discuss.elastic.co/t/kibana-single-click-drilldown-events-value-array-issue/290925)
**Author:** [@azulgrana](https://discuss.elastic.co/u/azulgrana)\ **Replies:** 3\ **Last updated:** [December 12, 2021, 6:56pm UTC](https://discuss.elastic.co/t/kibana-single-click-drilldown-events-value-array-issue/290925 "2021-12-12T18:56:37Z")
Hi there! I have an aggregation-based data table where I would like to use the drilldown capabilities. My table shows the following fields Id, Description, Severity, Hostname, Count My goal is to have a single click d… --- ## [ILM not reclaiming back space in ELK Host server](https://discuss.elastic.co/t/ilm-not-reclaiming-back-space-in-elk-host-server/290218)
**Author:** [@Anagha\_nambiar](https://discuss.elastic.co/u/Anagha_nambiar)\ **Replies:** 20\ **Last updated:** [December 12, 2021, 6:46pm UTC](https://discuss.elastic.co/t/ilm-not-reclaiming-back-space-in-elk-host-server/290218 "2021-12-12T18:46:07Z")
We have a ELK cluster with single node having 50gb disk space. We are sending metricbeat data from 200 servers to Elasticsearch. Created an ILM policy for metricbeat index to do a rollover and delete the index after it … --- ## [Increase timeouts between ES and logstash](https://discuss.elastic.co/t/increase-timeouts-between-es-and-logstash/291550)
**Author:** [@dominbdg](https://discuss.elastic.co/u/dominbdg)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 4:15pm UTC](https://discuss.elastic.co/t/increase-timeouts-between-es-and-logstash/291550 "2021-12-12T16:15:52Z")
hello, I'm facing the issues means: \[\[main\]\>worker5\] elasticsearch - Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error\_message=\>"Elasticsearch Unreachable:… --- ## [Can't access our cluster through Grafana](https://discuss.elastic.co/t/cant-access-our-cluster-through-grafana/291548)
**Author:** [@mertkaant](https://discuss.elastic.co/u/mertkaant)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 3:43pm UTC](https://discuss.elastic.co/t/cant-access-our-cluster-through-grafana/291548 "2021-12-12T15:43:15Z")
Hello, dear community, We recently have upgraded our ELK to 7.15.2. from 7.7.0. Here are some details from our cluster This is one of our cluster's health: { "cluster\_name" : "xxxxxxxxxxx", "status" : "green", … --- ## [Raise bulkindexerror](https://discuss.elastic.co/t/raise-bulkindexerror/291546)
**Author:** [@FUNZAR](https://discuss.elastic.co/u/FUNZAR)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 2:19pm UTC](https://discuss.elastic.co/t/raise-bulkindexerror/291546 "2021-12-12T14:19:44Z")
Hello everyone! Need a help here. Here is my code: import json import os from elasticsearch import Elasticsearch, RequestError from elasticsearch.helpers import bulk def create\_index(es\_client, index\_name, setting… --- ## [Does watcher works on a higher version?](https://discuss.elastic.co/t/does-watcher-works-on-a-higher-version/291503)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 2\ **Last updated:** [December 12, 2021, 12:22pm UTC](https://discuss.elastic.co/t/does-watcher-works-on-a-higher-version/291503 "2021-12-12T12:22:19Z")
Hi I have a cluster with version 7.6.2 I want to upgrade the cluster with rpm package and just want to know, If I upgrade Elasticsearch and Kibana to 7.15.2 , after upgrading, all of my watcher work as same as the curr… --- ## [Logstash ingest pipeline - Data from one pipeline going to another](https://discuss.elastic.co/t/logstash-ingest-pipeline-data-from-one-pipeline-going-to-another/289131)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 8\ **Last updated:** [December 12, 2021, 9:35am UTC](https://discuss.elastic.co/t/logstash-ingest-pipeline-data-from-one-pipeline-going-to-another/289131 "2021-12-12T09:35:42Z")
Hello, I hope my message finds the members of the community and their loved ones safe and healthy. I am running logstash (7.15.2) on a Raspberry Pi 4B running Ubuntu 20.04.3 LTS. I have seven pipelines, all listening … --- ## [Split index in runtime](https://discuss.elastic.co/t/split-index-in-runtime/291329)
**Author:** [@valentineg](https://discuss.elastic.co/u/valentineg)\ **Replies:** 2\ **Last updated:** [December 12, 2021, 8:01am UTC](https://discuss.elastic.co/t/split-index-in-runtime/291329 "2021-12-12T08:01:11Z")
HI, we have logstash and elastic on a k8s environment. in our topology logstash receives data from multiple applications while part of the data is application name \\ id. is there a way to add a dynamic variable to the … --- ## [Metricbeat postgresql 13.x, statement module needs update](https://discuss.elastic.co/t/metricbeat-postgresql-13-x-statement-module-needs-update/291530)
**Author:** [@Robin\_Goussey](https://discuss.elastic.co/u/Robin_Goussey)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 7:57am UTC](https://discuss.elastic.co/t/metricbeat-postgresql-13-x-statement-module-needs-update/291530 "2021-12-12T07:57:47Z")
Hi, I've been trying to get metricbeat to work with postgres. It works fine, however, I noticed some of the default dashboards had no data, after further investigation, I've" found that the time fields are always empty. … --- ## [Filebeat is leaking memory](https://discuss.elastic.co/t/filebeat-is-leaking-memory/291528)
**Author:** [@cva](https://discuss.elastic.co/u/cva)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 6:37am UTC](https://discuss.elastic.co/t/filebeat-is-leaking-memory/291528 "2021-12-12T06:37:33Z")
There are about 100,000 log files on the VM, the characteristics of the VM OS Windows Server 2019, CPU 4, RAM 4GB. To collect logs and then send them to logstash, filebeat 7.16.0 is used. After starting filebeat, it begi… --- ## [Filebeat harvesting](https://discuss.elastic.co/t/filebeat-harvesting/291514)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 1\ **Last updated:** [December 11, 2021, 10:04pm UTC](https://discuss.elastic.co/t/filebeat-harvesting/291514 "2021-12-11T22:04:31Z")
Hi all I am using filebeat to harvest logs and send data to Elasticsearch. It is noted that log4net is used to create log and log will rotate by size for example in 10m, when log.log size received to 10m, the current l… --- ## [Copy an engine](https://discuss.elastic.co/t/copy-an-engine/291513)
**Author:** [@miki\_haiat](https://discuss.elastic.co/u/miki_haiat)\ **Replies:** 1\ **Last updated:** [December 11, 2021, 7:46pm UTC](https://discuss.elastic.co/t/copy-an-engine/291513 "2021-12-11T19:46:06Z")
Im trying to figure out if there a way to copy an existing engine with all his properties. the API doesn't support this feature. It will be very useful if you create a "DEV" engine and you convert it to a "PROD" en… --- ## [How distinguish records at file](https://discuss.elastic.co/t/how-distinguish-records-at-file/291518)
**Author:** [@Alpork](https://discuss.elastic.co/u/Alpork)\ **Replies:** 0\ **Last updated:** [December 11, 2021, 7:34pm UTC](https://discuss.elastic.co/t/how-distinguish-records-at-file/291518 "2021-12-11T19:34:05Z")
Hello I have these records at file. 1639243031955|120|0|100|1|344405511|1|1906||sw|34esae3d||sdsw| 1639044203391|207|0|2009030129|http:\\126.198.81.51|5|1|10500| I use Processors-\>CSV for extracting values for these r… --- ## [Elastic Agent broken](https://discuss.elastic.co/t/elastic-agent-broken/278490)
**Author:** [@Henry943](https://discuss.elastic.co/u/Henry943)\ **Replies:** 2\ **Last updated:** [December 11, 2021, 6:31pm UTC](https://discuss.elastic.co/t/elastic-agent-broken/278490 "2021-12-11T18:31:44Z")
Thank you for providing such an wonderful tool to monitor and logs. But unfortunately recent times Elastic agent corrupted frequently. Elastic Agent is installed but currently broken: service exists but installation pa… --- ## [Filebeat утекает память](https://discuss.elastic.co/t/filebeat/291510)
**Author:** [@cva](https://discuss.elastic.co/u/cva)\ **Replies:** 0\ **Last updated:** [December 11, 2021, 3:50pm UTC](https://discuss.elastic.co/t/filebeat/291510 "2021-12-11T15:50:22Z")
На ВМке имеется примерно около 100 000 лог файлов, характеристики ВМ OS Windows Server 2019, CPU 4, ОЗУ 4Гб. Для сбора логов и последующей их отправки в logstash используется filebeat 7.16.0. После запуска filebeat начин… --- ## [Index roll over every 30 minutes](https://discuss.elastic.co/t/index-roll-over-every-30-minutes/290788)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 10\ **Last updated:** [December 11, 2021, 5:35pm UTC](https://discuss.elastic.co/t/index-roll-over-every-30-minutes/290788 "2021-12-11T17:35:24Z")
Hello All, We have a logstash set up where input is based on a file. Logstash parses the file and an index is created. A new file ends up for processing every 30 minutes. We want a new index to be created every time a… --- ## [Filebeat not working](https://discuss.elastic.co/t/filebeat-not-working/291337)
**Author:** [@Shubham\_Singh](https://discuss.elastic.co/u/Shubham_Singh)\ **Replies:** 2\ **Last updated:** [December 11, 2021, 4:25pm UTC](https://discuss.elastic.co/t/filebeat-not-working/291337 "2021-12-11T16:25:19Z")
Filebeat is not logging logs from a path. Here is my filebeat config - filebeat.inputs: type: log enabled: true paths: /var/log/xyz/\*/ \*.log input\_type: log json.keys\_under\_root: true json.add\_error\_key: true j… --- ## [Ingest processor: remove a line from a CSV?](https://discuss.elastic.co/t/ingest-processor-remove-a-line-from-a-csv/291137)
**Author:** [@QuartoStato](https://discuss.elastic.co/u/QuartoStato)\ **Replies:** 8\ **Last updated:** [December 11, 2021, 12:58pm UTC](https://discuss.elastic.co/t/ingest-processor-remove-a-line-from-a-csv/291137 "2021-12-11T12:58:40Z")
Hi all, I'm trying to create a processor for an ingest pipeline. I'm checking all available existing processor but I can't understand if it's possible to remove a line from a CSV. The data we need to import has 2 header… --- ## [Deploying E+K onto AWD EKS cluster, multiple subnets and nodegroups across 2 az's](https://discuss.elastic.co/t/deploying-e-k-onto-awd-eks-cluster-multiple-subnets-and-nodegroups-across-2-azs/291413)
**Author:** [@George\_Leonard](https://discuss.elastic.co/u/George_Leonard)\ **Replies:** 1\ **Last updated:** [December 11, 2021, 10:01am UTC](https://discuss.elastic.co/t/deploying-e-k-onto-awd-eks-cluster-multiple-subnets-and-nodegroups-across-2-azs/291413 "2021-12-11T10:01:57Z")
Hi all I"m wanting to deploy Elasticsearc and Kibana onto a AWS EKS cluster. The cluster has 4 subnets, across 2 AZ's. Each of the AZ's have a dedicated set of nodes corralled into a node\_group. 2 subnets = app\_ng 2… [Previous page](https://discuss.elastic.co/latest.md?page=1349) [Next page](https://discuss.elastic.co/latest.md?page=1351)