# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1130 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1131 --- ## [Kibana suricata integration](https://discuss.elastic.co/t/kibana-suricata-integration/290363)
**Author:** [@algira37](https://discuss.elastic.co/u/algira37)\ **Replies:** 1\ **Last updated:** [December 13, 2021, 9:47am UTC](https://discuss.elastic.co/t/kibana-suricata-integration/290363 "2021-12-13T09:47:45Z")
Hello, I added suricata integration to my system, but I can't use filebeat suricata module so I configured filebeat to send logs without suricata module to ELK, I just wondering is there any solution to use kibana inte… --- ## [Kibana dashboard spacing between values in an array](https://discuss.elastic.co/t/kibana-dashboard-spacing-between-values-in-an-array/289402)
**Author:** [@hellotty](https://discuss.elastic.co/u/hellotty)\ **Replies:** 2\ **Last updated:** [December 13, 2021, 9:45am UTC](https://discuss.elastic.co/t/kibana-dashboard-spacing-between-values-in-an-array/289402 "2021-12-13T09:45:52Z")
Hello I am passing values as an array to elasticsearch and when I saw the result I got a question. Is it possible to make line spacing in an array so that it can be easier to read? At the moment, the kibana dashboard … --- ## [Kibana Visualization filtering data based on aggregation](https://discuss.elastic.co/t/kibana-visualization-filtering-data-based-on-aggregation/291610)
**Author:** [@anas\_pp](https://discuss.elastic.co/u/anas_pp)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 9:34am UTC](https://discuss.elastic.co/t/kibana-visualization-filtering-data-based-on-aggregation/291610 "2021-12-13T09:34:42Z")
I have requirement that to show the count of running jobs and completed in Pie chart . I am receiving status of jobs in real time . To show the count of the jobs that are currently in running state I have to filter those… --- ## [Github URL to set on op.jwkset\_path parameter](https://discuss.elastic.co/t/github-url-to-set-on-op-jwkset-path-parameter/291474)
**Author:** [@Fabio\_Peruchi](https://discuss.elastic.co/u/Fabio_Peruchi)\ **Replies:** 5\ **Last updated:** [December 13, 2021, 8:12am UTC](https://discuss.elastic.co/t/github-url-to-set-on-op-jwkset-path-parameter/291474 "2021-12-13T08:12:38Z")
Hello. I'm struggling to find where is the GitHub URL to set here on this parameter from this documentation: Configuring single sign-on to the Elastic Stack using OpenID Connect | Elasticsearch Guide \[7.16\] | Elastic o… --- ## [Logstash Error: "Your settings are invalid"](https://discuss.elastic.co/t/logstash-error-your-settings-are-invalid/291544)
**Author:** [@neerajg](https://discuss.elastic.co/u/neerajg)\ **Replies:** 2\ **Last updated:** [December 13, 2021, 7:52am UTC](https://discuss.elastic.co/t/logstash-error-your-settings-are-invalid/291544 "2021-12-13T07:52:03Z")
Hi guys, I just upgraded elk stack from 7.15.0 to 7.16.0. Everything is working fine except Logstash. It ends up failing to start. I checked the system logs and found the following error. Error: logstash\[20212\]: Ope… --- ## [Fscrawler - update existing record](https://discuss.elastic.co/t/fscrawler-update-existing-record/291597)
**Author:** [@Andrej\_K](https://discuss.elastic.co/u/Andrej_K)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 7:37am UTC](https://discuss.elastic.co/t/fscrawler-update-existing-record/291597 "2021-12-13T07:37:18Z")
Hello, I need to search documents by its content and some custom customer data. So my idea is to use fscrawler to extract content from documents and keep it with custom customer data in one index for easy searching. So… --- ## [How to get the field names which have no data or null in elasticsearch dynamically?](https://discuss.elastic.co/t/how-to-get-the-field-names-which-have-no-data-or-null-in-elasticsearch-dynamically/291591)
**Author:** [@Saurabh\_A](https://discuss.elastic.co/u/Saurabh_A)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 6:17am UTC](https://discuss.elastic.co/t/how-to-get-the-field-names-which-have-no-data-or-null-in-elasticsearch-dynamically/291591 "2021-12-13T06:17:53Z")
How do we get the field names with no data in them? For example, in an index, there is a field that is present in the mapping but no data is indexed in this case. So I want a list of all the field names or keys where the… --- ## [Metricbeat showing same ip of elasticsearch nodes](https://discuss.elastic.co/t/metricbeat-showing-same-ip-of-elasticsearch-nodes/291586)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 5:40am UTC](https://discuss.elastic.co/t/metricbeat-showing-same-ip-of-elasticsearch-nodes/291586 "2021-12-13T05:40:04Z")
1 week ago i upgrade elk stack from 7.13 to 7.15.Everything is fine but today i notice that in stack monitoring page when i click on nodes ,the ip of nodes is same for cross verification i ran GET /cat/nodes it showing m… --- ## [Output to different elastic search Indices based on the string](https://discuss.elastic.co/t/output-to-different-elastic-search-indices-based-on-the-string/291568)
**Author:** [@mussa572](https://discuss.elastic.co/u/mussa572)\ **Replies:** 3\ **Last updated:** [December 13, 2021, 3:40am UTC](https://discuss.elastic.co/t/output-to-different-elastic-search-indices-based-on-the-string/291568 "2021-12-13T03:40:17Z")
Hi I have a Category field (cat) which can have following value and future expect to add more. BL LA KA Currently I have following code to output to the different Elasticsearch indices from our Logstash pipeline . I… --- ## [Collecting logs using filebeat over http](https://discuss.elastic.co/t/collecting-logs-using-filebeat-over-http/291578)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 3:14am UTC](https://discuss.elastic.co/t/collecting-logs-using-filebeat-over-http/291578 "2021-12-13T03:14:37Z")
Hi Team, Wondering if the filebeat.log.type can be http or https? I need to pull the logs using https path and using basic auth. Please let me know if that is possible? I am aware it can be possible with httpjson; howe… --- ## [Resources for learning ElasticSearch](https://discuss.elastic.co/t/resources-for-learning-elasticsearch/291536)
**Author:** [@rav3n](https://discuss.elastic.co/u/rav3n)\ **Replies:** 2\ **Last updated:** [December 12, 2021, 7:19pm UTC](https://discuss.elastic.co/t/resources-for-learning-elasticsearch/291536 "2021-12-12T19:19:50Z")
Hey people, Just started using Elasticsearch for my personal use. So, last night I picked one of the tool from Elastic stack and started reading about it. I came across to this book named "Elasticsearch: The Definitive … --- ## [Kibana Single Click Drilldown events.value Array Issue](https://discuss.elastic.co/t/kibana-single-click-drilldown-events-value-array-issue/290925)
**Author:** [@azulgrana](https://discuss.elastic.co/u/azulgrana)\ **Replies:** 3\ **Last updated:** [December 12, 2021, 6:56pm UTC](https://discuss.elastic.co/t/kibana-single-click-drilldown-events-value-array-issue/290925 "2021-12-12T18:56:37Z")
Hi there! I have an aggregation-based data table where I would like to use the drilldown capabilities. My table shows the following fields Id, Description, Severity, Hostname, Count My goal is to have a single click d… --- ## [Increase timeouts between ES and logstash](https://discuss.elastic.co/t/increase-timeouts-between-es-and-logstash/291550)
**Author:** [@dominbdg](https://discuss.elastic.co/u/dominbdg)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 4:15pm UTC](https://discuss.elastic.co/t/increase-timeouts-between-es-and-logstash/291550 "2021-12-12T16:15:52Z")
hello, I'm facing the issues means: \[\[main\]\>worker5\] elasticsearch - Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error\_message=\>"Elasticsearch Unreachable:… --- ## [Can't access our cluster through Grafana](https://discuss.elastic.co/t/cant-access-our-cluster-through-grafana/291548)
**Author:** [@mertkaant](https://discuss.elastic.co/u/mertkaant)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 3:43pm UTC](https://discuss.elastic.co/t/cant-access-our-cluster-through-grafana/291548 "2021-12-12T15:43:15Z")
Hello, dear community, We recently have upgraded our ELK to 7.15.2. from 7.7.0. Here are some details from our cluster This is one of our cluster's health: { "cluster\_name" : "xxxxxxxxxxx", "status" : "green", … --- ## [Raise bulkindexerror](https://discuss.elastic.co/t/raise-bulkindexerror/291546)
**Author:** [@FUNZAR](https://discuss.elastic.co/u/FUNZAR)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 2:19pm UTC](https://discuss.elastic.co/t/raise-bulkindexerror/291546 "2021-12-12T14:19:44Z")
Hello everyone! Need a help here. Here is my code: import json import os from elasticsearch import Elasticsearch, RequestError from elasticsearch.helpers import bulk def create\_index(es\_client, index\_name, setting… --- ## [Does watcher works on a higher version?](https://discuss.elastic.co/t/does-watcher-works-on-a-higher-version/291503)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 2\ **Last updated:** [December 12, 2021, 12:22pm UTC](https://discuss.elastic.co/t/does-watcher-works-on-a-higher-version/291503 "2021-12-12T12:22:19Z")
Hi I have a cluster with version 7.6.2 I want to upgrade the cluster with rpm package and just want to know, If I upgrade Elasticsearch and Kibana to 7.15.2 , after upgrading, all of my watcher work as same as the curr… --- ## [Logstash ingest pipeline - Data from one pipeline going to another](https://discuss.elastic.co/t/logstash-ingest-pipeline-data-from-one-pipeline-going-to-another/289131)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 8\ **Last updated:** [December 12, 2021, 9:35am UTC](https://discuss.elastic.co/t/logstash-ingest-pipeline-data-from-one-pipeline-going-to-another/289131 "2021-12-12T09:35:42Z")
Hello, I hope my message finds the members of the community and their loved ones safe and healthy. I am running logstash (7.15.2) on a Raspberry Pi 4B running Ubuntu 20.04.3 LTS. I have seven pipelines, all listening … --- ## [Split index in runtime](https://discuss.elastic.co/t/split-index-in-runtime/291329)
**Author:** [@valentineg](https://discuss.elastic.co/u/valentineg)\ **Replies:** 2\ **Last updated:** [December 12, 2021, 8:01am UTC](https://discuss.elastic.co/t/split-index-in-runtime/291329 "2021-12-12T08:01:11Z")
HI, we have logstash and elastic on a k8s environment. in our topology logstash receives data from multiple applications while part of the data is application name \\ id. is there a way to add a dynamic variable to the … --- ## [Metricbeat postgresql 13.x, statement module needs update](https://discuss.elastic.co/t/metricbeat-postgresql-13-x-statement-module-needs-update/291530)
**Author:** [@Robin\_Goussey](https://discuss.elastic.co/u/Robin_Goussey)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 7:57am UTC](https://discuss.elastic.co/t/metricbeat-postgresql-13-x-statement-module-needs-update/291530 "2021-12-12T07:57:47Z")
Hi, I've been trying to get metricbeat to work with postgres. It works fine, however, I noticed some of the default dashboards had no data, after further investigation, I've" found that the time fields are always empty. … --- ## [Filebeat is leaking memory](https://discuss.elastic.co/t/filebeat-is-leaking-memory/291528)
**Author:** [@cva](https://discuss.elastic.co/u/cva)\ **Replies:** 0\ **Last updated:** [December 12, 2021, 6:37am UTC](https://discuss.elastic.co/t/filebeat-is-leaking-memory/291528 "2021-12-12T06:37:33Z")
There are about 100,000 log files on the VM, the characteristics of the VM OS Windows Server 2019, CPU 4, RAM 4GB. To collect logs and then send them to logstash, filebeat 7.16.0 is used. After starting filebeat, it begi… --- ## [Filebeat harvesting](https://discuss.elastic.co/t/filebeat-harvesting/291514)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 1\ **Last updated:** [December 11, 2021, 10:04pm UTC](https://discuss.elastic.co/t/filebeat-harvesting/291514 "2021-12-11T22:04:31Z")
Hi all I am using filebeat to harvest logs and send data to Elasticsearch. It is noted that log4net is used to create log and log will rotate by size for example in 10m, when log.log size received to 10m, the current l… --- ## [How distinguish records at file](https://discuss.elastic.co/t/how-distinguish-records-at-file/291518)
**Author:** [@Alpork](https://discuss.elastic.co/u/Alpork)\ **Replies:** 0\ **Last updated:** [December 11, 2021, 7:34pm UTC](https://discuss.elastic.co/t/how-distinguish-records-at-file/291518 "2021-12-11T19:34:05Z")
Hello I have these records at file. 1639243031955|120|0|100|1|344405511|1|1906||sw|34esae3d||sdsw| 1639044203391|207|0|2009030129|http:\\126.198.81.51|5|1|10500| I use Processors-\>CSV for extracting values for these r… --- ## [Index roll over every 30 minutes](https://discuss.elastic.co/t/index-roll-over-every-30-minutes/290788)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 10\ **Last updated:** [December 11, 2021, 5:35pm UTC](https://discuss.elastic.co/t/index-roll-over-every-30-minutes/290788 "2021-12-11T17:35:24Z")
Hello All, We have a logstash set up where input is based on a file. Logstash parses the file and an index is created. A new file ends up for processing every 30 minutes. We want a new index to be created every time a… --- ## [Filebeat not working](https://discuss.elastic.co/t/filebeat-not-working/291337)
**Author:** [@Shubham\_Singh](https://discuss.elastic.co/u/Shubham_Singh)\ **Replies:** 2\ **Last updated:** [December 11, 2021, 4:25pm UTC](https://discuss.elastic.co/t/filebeat-not-working/291337 "2021-12-11T16:25:19Z")
Filebeat is not logging logs from a path. Here is my filebeat config - filebeat.inputs: type: log enabled: true paths: /var/log/xyz/\*/ \*.log input\_type: log json.keys\_under\_root: true json.add\_error\_key: true j… --- ## [Ingest processor: remove a line from a CSV?](https://discuss.elastic.co/t/ingest-processor-remove-a-line-from-a-csv/291137)
**Author:** [@QuartoStato](https://discuss.elastic.co/u/QuartoStato)\ **Replies:** 8\ **Last updated:** [December 11, 2021, 12:58pm UTC](https://discuss.elastic.co/t/ingest-processor-remove-a-line-from-a-csv/291137 "2021-12-11T12:58:40Z")
Hi all, I'm trying to create a processor for an ingest pipeline. I'm checking all available existing processor but I can't understand if it's possible to remove a line from a CSV. The data we need to import has 2 header… --- ## [What is the correct way to support Global search across multiple document types?](https://discuss.elastic.co/t/what-is-the-correct-way-to-support-global-search-across-multiple-document-types/291388)
**Author:** [@abm\_a](https://discuss.elastic.co/u/abm_a)\ **Replies:** 3\ **Last updated:** [December 11, 2021, 9:23am UTC](https://discuss.elastic.co/t/what-is-the-correct-way-to-support-global-search-across-multiple-document-types/291388 "2021-12-11T09:23:36Z")
Let's say in a Microservices architecture there are 100s of different entities and search needs to be supported that can look for matches in all those entities. I see a few ways of doing this. Create one index for each… --- ## [Nested Script Sorting - Access all matching nested documents inside script](https://discuss.elastic.co/t/nested-script-sorting-access-all-matching-nested-documents-inside-script/291497)
**Author:** [@Ruan23](https://discuss.elastic.co/u/Ruan23)\ **Replies:** 0\ **Last updated:** [December 11, 2021, 8:26am UTC](https://discuss.elastic.co/t/nested-script-sorting-access-all-matching-nested-documents-inside-script/291497 "2021-12-11T08:26:23Z")
I am trying to find a way to sort root documents, based on the cumulative price of all the matching nested documents. Each vendor is stored as the root document, with all their products stored as nested documents: { … --- ## [MySQL Query Equivalent In JDBC Static](https://discuss.elastic.co/t/mysql-query-equivalent-in-jdbc-static/290347)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 3\ **Last updated:** [December 11, 2021, 6:28am UTC](https://discuss.elastic.co/t/mysql-query-equivalent-in-jdbc-static/290347 "2021-12-11T06:28:42Z")
hello, I have a query in MySQL and I was wondering what would the equal query in JDBC static with parameters be. Here's my query: select name from my\_table where inet\_aton('1.1.1.1') between ip\_from and ip\_to limit 1;… --- ## [Rolling up Data Streams](https://discuss.elastic.co/t/rolling-up-data-streams/288328)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 7\ **Last updated:** [December 11, 2021, 6:21am UTC](https://discuss.elastic.co/t/rolling-up-data-streams/288328 "2021-12-11T06:21:24Z")
Hi there, I'm curious if making rollups from data streams is recommended/possible? It would be great after rolling over data from my data stream if I could also roll it up into bigger time chunks. Thanks! --- ## [How to connect Elasticsearch correctly in a Kafka connector?](https://discuss.elastic.co/t/how-to-connect-elasticsearch-correctly-in-a-kafka-connector/290949)
**Author:** [@Hongbo-Miao](https://discuss.elastic.co/u/Hongbo-Miao)\ **Replies:** 2\ **Last updated:** [December 11, 2021, 5:29am UTC](https://discuss.elastic.co/t/how-to-connect-elasticsearch-correctly-in-a-kafka-connector/290949 "2021-12-11T05:29:12Z")
Step 1 - Deploy Elasticsearch in Kubernetes (succeed) I deployed Elasticsearch in Kubernetes by following the tutorial apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: hm-elasticsearch… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1129) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1131)