# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1284 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1285 --- ## [How to define what data is sent to Elastic from Logstash?](https://discuss.elastic.co/t/how-to-define-what-data-is-sent-to-elastic-from-logstash/279995)
**Author:** [@profsteve](https://discuss.elastic.co/u/profsteve)\ **Replies:** 8\ **Last updated:** [August 12, 2021, 1:37pm UTC](https://discuss.elastic.co/t/how-to-define-what-data-is-sent-to-elastic-from-logstash/279995 "2021-08-12T13:37:12Z")
Hello, I have ELK running on Ubuntu Server with the intention of it receiving Syslog from firewalls and Filebeat from DNS/DHCP/AD services. The primary issue is the firewalls make a lot of Syslog noise without there be… --- ## [Failed to parse field](https://discuss.elastic.co/t/failed-to-parse-field/281112)
**Author:** [@Senthil\_ak](https://discuss.elastic.co/u/Senthil_ak)\ **Replies:** 4\ **Last updated:** [August 12, 2021, 1:25pm UTC](https://discuss.elastic.co/t/failed-to-parse-field/281112 "2021-08-12T13:25:01Z")
Hi admin, I am getting some strange exception in the logstash all of a sudden. I try to debug but unable to do so. Need your kind help. \[2021-08-11T15:37:35,189\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index e… --- ## [Filtering the data list from Discover to only see Unique Ids](https://discuss.elastic.co/t/filtering-the-data-list-from-discover-to-only-see-unique-ids/280472)
**Author:** [@bailarch](https://discuss.elastic.co/u/bailarch)\ **Replies:** 10\ **Last updated:** [August 12, 2021, 1:10pm UTC](https://discuss.elastic.co/t/filtering-the-data-list-from-discover-to-only-see-unique-ids/280472 "2021-08-12T13:10:25Z")
Hello guys - not sure if the subcategory for this issue is docker (not sure what this is but the other options seem incorrect for me) I am trying to get the list of all the users name from Discovery. However I am seein… --- ## [Does kibana understand package registry via proxy?](https://discuss.elastic.co/t/does-kibana-understand-package-registry-via-proxy/281086)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 1:04pm UTC](https://discuss.elastic.co/t/does-kibana-understand-package-registry-via-proxy/281086 "2021-08-12T13:04:01Z")
Hi, Appearantly since 7.14 (fleet = GA) I am required to have internet access for my selfmanaged kibana's to reach some packet registry (or host one myself). However, I set a proxy in the environment and am still not… --- ## [Filebeat doesn't catchup with suricata eve.json](https://discuss.elastic.co/t/filebeat-doesnt-catchup-with-suricata-eve-json/281153)
**Author:** [@3c2c2ff5](https://discuss.elastic.co/u/3c2c2ff5)\ **Replies:** 6\ **Last updated:** [August 12, 2021, 12:20pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-catchup-with-suricata-eve-json/281153 "2021-08-12T12:20:02Z")
Hello ELK community, I am fairly new to the subject ELK stack, I am trying to setup an IDS with suricata and ELK, the initial setup went pretty good, but I realized that the suricata events from eve.json are not getting… --- ## [Unclear how logstash fingerprint treats empty/non-existing fields](https://discuss.elastic.co/t/unclear-how-logstash-fingerprint-treats-empty-non-existing-fields/281157)
**Author:** [@AdmireDirac](https://discuss.elastic.co/u/AdmireDirac)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 12:00pm UTC](https://discuss.elastic.co/t/unclear-how-logstash-fingerprint-treats-empty-non-existing-fields/281157 "2021-08-12T12:00:19Z")
Hello everyone, I have a hard time to understand the behaviour fo the logstash fingerprint filter if a field is empty or non-existent. My scenario is the following: I'm reading a json-file with logstash and create a d… --- ## [ILM rollover for "Logstash Elasticsearch output plugin"](https://discuss.elastic.co/t/ilm-rollover-for-logstash-elasticsearch-output-plugin/281176)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 6\ **Last updated:** [August 12, 2021, 12:00pm UTC](https://discuss.elastic.co/t/ilm-rollover-for-logstash-elasticsearch-output-plugin/281176 "2021-08-12T12:00:13Z")
Hello My logstash code uses the elasticsearch output plugin to ingest docs into elasticsearch . However the index size tends to get larger in size , i came across ILM concepts in the elasticsearch documents, that says t… --- ## [How to use aggregate in inner hits](https://discuss.elastic.co/t/how-to-use-aggregate-in-inner-hits/281195)
**Author:** [@Amir\_Faramarzi](https://discuss.elastic.co/u/Amir_Faramarzi)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 11:22am UTC](https://discuss.elastic.co/t/how-to-use-aggregate-in-inner-hits/281195 "2021-08-12T11:22:49Z")
Hello friends Is it possible to use aggregation in inner hits? If so, how? --- ## [Index filebeat panw error](https://discuss.elastic.co/t/index-filebeat-panw-error/281192)
**Author:** [@newbie313](https://discuss.elastic.co/u/newbie313)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 10:47am UTC](https://discuss.elastic.co/t/index-filebeat-panw-error/281192 "2021-08-12T10:47:28Z")
I'm using filebeat-7.3.2 and configure panw.yml open port 514, the data from palo alto already received ( i check using tshark) but there is error come in the index management shows: Index lifecycle error illegal\_argum… --- ## [Getting error while starting elastic](https://discuss.elastic.co/t/getting-error-while-starting-elastic/281108)
**Author:** [@Faizan\_Khurshid](https://discuss.elastic.co/u/Faizan_Khurshid)\ **Replies:** 3\ **Last updated:** [August 12, 2021, 10:24am UTC](https://discuss.elastic.co/t/getting-error-while-starting-elastic/281108 "2021-08-12T10:24:17Z")
Hi Folk I am new to ELK , i was trying to install ELK on centos 8.4 below is my error when i was trying to start sudo systemctl start elasticsearch \[root@ ~\]# systemctl status elasticsearch.service ● elasticsearch.… --- ## [Setting up logstash with mongodb not starting](https://discuss.elastic.co/t/setting-up-logstash-with-mongodb-not-starting/280998)
**Author:** [@Bugsbee](https://discuss.elastic.co/u/Bugsbee)\ **Replies:** 5\ **Last updated:** [August 12, 2021, 10:12am UTC](https://discuss.elastic.co/t/setting-up-logstash-with-mongodb-not-starting/280998 "2021-08-12T10:12:05Z")
hi community, need your expertise / advise in this issue im facing when starting logstash which im trying to connect wth mongodb. input { uri =\> 'mongodb://userhere:passwordhere@1234.234.123.12:8081' p… --- ## [Grouping/filtering host servers](https://discuss.elastic.co/t/grouping-filtering-host-servers/281030)
**Author:** [@Garry](https://discuss.elastic.co/u/Garry)\ **Replies:** 3\ **Last updated:** [August 12, 2021, 9:48am UTC](https://discuss.elastic.co/t/grouping-filtering-host-servers/281030 "2021-08-12T09:48:24Z")
I have a situation where the I have a variety of hostnames for each client. E.g. Client 1: host1, host2, host3 Client 2: host4, host5, host6, host7 What would be the most effective way to group these client machines … --- ## [Elasticsearch kibana](https://discuss.elastic.co/t/elasticsearch-kibana/281151)
**Author:** [@Dilnawaz\_Khan](https://discuss.elastic.co/u/Dilnawaz_Khan)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 9:29am UTC](https://discuss.elastic.co/t/elasticsearch-kibana/281151 "2021-08-12T09:29:55Z")
Hello I have 2 master nodes of elasticsearch and 3 data nodes and one seprate node is for kibana Now I have a question if I give host ip of those 2 master node in kibana.yml file then where do I need to configure those … --- ## [Creating Vega visualization with Image in kibana v 7.13](https://discuss.elastic.co/t/creating-vega-visualization-with-image-in-kibana-v-7-13/280942)
**Author:** [@ErSumit](https://discuss.elastic.co/u/ErSumit)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 9:15am UTC](https://discuss.elastic.co/t/creating-vega-visualization-with-image-in-kibana-v-7-13/280942 "2021-08-12T09:15:25Z")
My vega visualization is working perfectly on vega editor, but its failing on kibana. On elastic stack version 7.9.2, the exception message is: “External URLs are not enabled. Add vis\_type\_vega.enableExternalUrls: true… --- ## [Max aggregation on .keyword field with both string and number values](https://discuss.elastic.co/t/max-aggregation-on-keyword-field-with-both-string-and-number-values/281168)
**Author:** [@Daniele\_Renda](https://discuss.elastic.co/u/Daniele_Renda)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 8:16am UTC](https://discuss.elastic.co/t/max-aggregation-on-keyword-field-with-both-string-and-number-values/281168 "2021-08-12T08:16:27Z")
Hi, I've a quite common situation: I've an index for products with a barcode field: "barcode": { "type": "text", "fields": { "ngram": { "type": "tex… --- ## [Pipeline worker error, the pipeline will be stopped {:pipeline\_id=\>"main", :error=\>"(TypeError) no implicit conversion of nil into Integer](https://discuss.elastic.co/t/pipeline-worker-error-the-pipeline-will-be-stopped-pipeline-id-main-error-typeerror-no-implicit-conversion-of-nil-into-integer/281167)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 8:16am UTC](https://discuss.elastic.co/t/pipeline-worker-error-the-pipeline-will-be-stopped-pipeline-id-main-error-typeerror-no-implicit-conversion-of-nil-into-integer/281167 "2021-08-12T08:16:12Z")
I have configured SQS as input, not sure why I am getting this error. I am not performing any conversation from my Logstash config file. Can anyone help to identify the issue? The following error message comes right af… --- ## [Fleet/Elastic-Agent trusted information from certificate](https://discuss.elastic.co/t/fleet-elastic-agent-trusted-information-from-certificate/281028)
**Author:** [@wsnet](https://discuss.elastic.co/u/wsnet)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 7:21am UTC](https://discuss.elastic.co/t/fleet-elastic-agent-trusted-information-from-certificate/281028 "2021-08-12T07:21:12Z")
Hello, With 7.14.0 Elasticsearch has started validating agent.id using the .fleet\_final\_pipeline-1 (see the PR), which greatly improves security. While using Elastic-Agent to fetch information from servers, e.g. web-lo… --- ## [Filebeats can see new lines in file only after restart](https://discuss.elastic.co/t/filebeats-can-see-new-lines-in-file-only-after-restart/281159)
**Author:** [@station72](https://discuss.elastic.co/u/station72)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 6:48am UTC](https://discuss.elastic.co/t/filebeats-can-see-new-lines-in-file-only-after-restart/281159 "2021-08-12T06:48:13Z")
I have a log file. It named access.log Each 30min new lines appears in this file from remote server (by synchronization). My filebeat config filebeat.inputs: - type: filestream scan\_frequency: 10s paths: - /sr… --- ## [Changing / Modifying / Customizing Visualize Library](https://discuss.elastic.co/t/changing-modifying-customizing-visualize-library/280946)
**Author:** [@bailarch](https://discuss.elastic.co/u/bailarch)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 6:47am UTC](https://discuss.elastic.co/t/changing-modifying-customizing-visualize-library/280946 "2021-08-12T06:47:50Z")
Hello everyone, I would like to create a set of dataset from Visualize Library since some of the templates that I see can be more powerful than Lens / Dashboard since you can set infinite about of data/values to show - … --- ## [Kibana startup failure in Windows](https://discuss.elastic.co/t/kibana-startup-failure-in-windows/280709)
**Author:** [@xubisid](https://discuss.elastic.co/u/xubisid)\ **Replies:** 3\ **Last updated:** [August 12, 2021, 6:47am UTC](https://discuss.elastic.co/t/kibana-startup-failure-in-windows/280709 "2021-08-12T06:47:05Z")
I have download the updated 7.14.0 kibana version and wait for the whole zip download but unfortunately getting the same error message Cannot find module './persistence\_google\_workspace\_api\_access\_granted\_via\_domain\_wid… --- ## [Question about Kibana Visualize](https://discuss.elastic.co/t/question-about-kibana-visualize/281111)
**Author:** [@FayeLu](https://discuss.elastic.co/u/FayeLu)\ **Replies:** 3\ **Last updated:** [August 12, 2021, 6:36am UTC](https://discuss.elastic.co/t/question-about-kibana-visualize/281111 "2021-08-12T06:36:24Z")
I am new to Kibana and I am learning how to create custom Kibana visualization chart. Is that possible we can create visualization of Elastic Search schema data using Kibana? eg. I have a data of colleges students with… --- ## [ELK cluster is not stable due to several issues related to shards and JVM configuration](https://discuss.elastic.co/t/elk-cluster-is-not-stable-due-to-several-issues-related-to-shards-and-jvm-configuration/281059)
**Author:** [@Dawood](https://discuss.elastic.co/u/Dawood)\ **Replies:** 2\ **Last updated:** [August 12, 2021, 6:25am UTC](https://discuss.elastic.co/t/elk-cluster-is-not-stable-due-to-several-issues-related-to-shards-and-jvm-configuration/281059 "2021-08-12T06:25:42Z")
Hi, First it is not the first time i am sharing the issues that i am encountering and hopefully this time somebody will help me with that !!! So i have an ELK cluster that is running on premise on a rancher k8s based … --- ## [Disable Alerts and Actions for some roles](https://discuss.elastic.co/t/disable-alerts-and-actions-for-some-roles/281155)
**Author:** [@Ankur\_Mahajan](https://discuss.elastic.co/u/Ankur_Mahajan)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 6:21am UTC](https://discuss.elastic.co/t/disable-alerts-and-actions-for-some-roles/281155 "2021-08-12T06:21:04Z")
I have set up Kibana alerts for multiple events for my production. I have other users also who can monitor the logs and dashboards. I've created a role for them with specific privileges but I couldn't find the option to … --- ## [Can we send jenkins Build console logs to Kibana using filebeat](https://discuss.elastic.co/t/can-we-send-jenkins-build-console-logs-to-kibana-using-filebeat/280547)
**Author:** [@Harindra\_Balanagired](https://discuss.elastic.co/u/Harindra_Balanagired)\ **Replies:** 7\ **Last updated:** [August 12, 2021, 5:47am UTC](https://discuss.elastic.co/t/can-we-send-jenkins-build-console-logs-to-kibana-using-filebeat/280547 "2021-08-12T05:47:08Z")
Can we send Jenkins Build console output to Kibana using the filebeat you figure out in the image? based on timestamp see highlighted time in that image. Based on the timestamp can we create a dashboard --- ## [Split Timelion Graph](https://discuss.elastic.co/t/split-timelion-graph/281152)
**Author:** [@Phanikumar\_21](https://discuss.elastic.co/u/Phanikumar_21)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 6:03am UTC](https://discuss.elastic.co/t/split-timelion-graph/281152 "2021-08-12T06:03:26Z")
Hi, We had ingested job monitoring information… start time / end time / target end time of job for SLA. However, some jobs start in the evening EST time and run into the next day – i.e. 11pm and ends at 2am . We are… --- ## [Kibana overrides original time format from Elasticsearch](https://discuss.elastic.co/t/kibana-overrides-original-time-format-from-elasticsearch/281124)
**Author:** [@rtemotfuorco123](https://discuss.elastic.co/u/rtemotfuorco123)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 5:29am UTC](https://discuss.elastic.co/t/kibana-overrides-original-time-format-from-elasticsearch/281124 "2021-08-12T05:29:10Z")
Hi, I have the index below and when I visualise it in Kibana, the log\_field.time format is changed. I want to keep it as the original one which is RFC3339. How do I get this sorted? See image at the bottom please. By th… --- ## [How to re run the failed transform job for that specific failed duration?](https://discuss.elastic.co/t/how-to-re-run-the-failed-transform-job-for-that-specific-failed-duration/281085)
**Author:** [@Mojahid\_Ahmad](https://discuss.elastic.co/u/Mojahid_Ahmad)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 5:23am UTC](https://discuss.elastic.co/t/how-to-re-run-the-failed-transform-job-for-that-specific-failed-duration/281085 "2021-08-12T05:23:17Z")
I have created one transform job, so the incoming document is passed to transform job and do aggregation, and then it gets ingested into a new index. It was running fine until I see the error failed to gather field map… --- ## [Grok filter by values](https://discuss.elastic.co/t/grok-filter-by-values/280988)
**Author:** [@IsAa](https://discuss.elastic.co/u/IsAa)\ **Replies:** 4\ **Last updated:** [August 12, 2021, 5:19am UTC](https://discuss.elastic.co/t/grok-filter-by-values/280988 "2021-08-12T05:19:42Z")
Hi Everyone, I have a text in my input as follows, The following text contains {"Food":"Fruit","Type":"Apple"} I am trying to structure my output data to also contain fields Food: Fruit Type: Apple I was looking int… --- ## [Dump elasticsearch documents into avro format](https://discuss.elastic.co/t/dump-elasticsearch-documents-into-avro-format/281146)
**Author:** [@Sagar\_kadu](https://discuss.elastic.co/u/Sagar_kadu)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 4:43am UTC](https://discuss.elastic.co/t/dump-elasticsearch-documents-into-avro-format/281146 "2021-08-12T04:43:10Z")
I want to store elasticsearch query results into avro files using logstash..As far as I know there file output logstash plugin which by default stores JSON documents. after going through logstash documentation,I co… --- ## [Setup Elasticsearch cluster result in two clusers](https://discuss.elastic.co/t/setup-elasticsearch-cluster-result-in-two-clusers/281135)
**Author:** [@Benli](https://discuss.elastic.co/u/Benli)\ **Replies:** 3\ **Last updated:** [August 12, 2021, 3:22am UTC](https://discuss.elastic.co/t/setup-elasticsearch-cluster-result-in-two-clusers/281135 "2021-08-12T03:22:35Z")
Hi there, I'm setting up Elasticsearch cluster by using two separate Linux machines. On the first Linux, elasticsearch.yml: # ---------------------------------- Cluster ----------------------------------- cluster.name:… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1283) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1285)