# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1420 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1421 --- ## [Can we edit Kibana's own fancy dashboards or make similar?](https://discuss.elastic.co/t/can-we-edit-kibanas-own-fancy-dashboards-or-make-similar/272718)
**Author:** [@Mark\_Duncan](https://discuss.elastic.co/u/Mark_Duncan)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 5:39pm UTC](https://discuss.elastic.co/t/can-we-edit-kibanas-own-fancy-dashboards-or-make-similar/272718 "2021-05-11T17:39:12Z")
Is it possible to make pages like these ourselves in Kibana, i.e. these layouts, and the nice min/max/current views? thanks in advance! Mark --- ## [Custom message filter with logstash](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626)
**Author:** [@cuongnp](https://discuss.elastic.co/u/cuongnp)\ **Replies:** 6\ **Last updated:** [May 11, 2021, 5:31pm UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626 "2021-05-11T17:31:47Z")
Hi all, I have a message log below: \<85\>1 2021-05-11T09:25:02+07:00 172.19.16.241 CP-GW - Log \[Fields@1.3.6.1.4.1.2620 Log delay="1620699902" src="172.19.9.18" dst="172.19.11.13" proto="6" UP\_match\_table="TABLE\_START" … --- ## [Logstash to output syslog in Snare format](https://discuss.elastic.co/t/logstash-to-output-syslog-in-snare-format/271405)
**Author:** [@Ravi342883](https://discuss.elastic.co/u/Ravi342883)\ **Replies:** 4\ **Last updated:** [May 11, 2021, 5:10pm UTC](https://discuss.elastic.co/t/logstash-to-output-syslog-in-snare-format/271405 "2021-05-11T17:10:58Z")
Hi All, For a PoC portion, I would like to know whether logstash can output the syslogs in Snare format. Any reference? If anyone has got insights around this, please shed some light here. Thanks, Ravi --- ## [Standalone cluster appears when monitoring logstash with 27 pipeines ELK 7.12](https://discuss.elastic.co/t/standalone-cluster-appears-when-monitoring-logstash-with-27-pipeines-elk-7-12/272730)
**Author:** [@Philip\_Thomson1](https://discuss.elastic.co/u/Philip_Thomson1)\ **Replies:** 0\ **Last updated:** [May 11, 2021, 4:57pm UTC](https://discuss.elastic.co/t/standalone-cluster-appears-when-monitoring-logstash-with-27-pipeines-elk-7-12/272730 "2021-05-11T16:57:29Z")
I have upgraded to 7.12, and then dutifully setup metricbeats for monitoring ES, Kibana and logstash Its a simple setup, with one instance of logstash, however in the monitoring page I see my cluster and a "standalone c… --- ## [Logstash JSON parse error](https://discuss.elastic.co/t/logstash-json-parse-error/272533)
**Author:** [@Richard\_Phillips\_Roy](https://discuss.elastic.co/u/Richard_Phillips_Roy)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 4:59pm UTC](https://discuss.elastic.co/t/logstash-json-parse-error/272533 "2021-05-11T16:59:49Z")
Hi, how do i fix this parsing error in logstash \[ERROR\] 2021-05-08 15:30:46.678 \[\[main\]\#\ **Author:** [@marone](https://discuss.elastic.co/u/marone)\ **Replies:** 4\ **Last updated:** [May 11, 2021, 4:58pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-a-secured-cluster/272218 "2021-05-11T16:58:10Z") Hi I'm using ELK 7.12.0 and I'm struggling to connect my logstash with my secured cluster, without xpack logstash send data to elasticsearch using twitter plugin, but when I enable xpack for elasticsearch and try to conf… --- ## [Recalculate times?](https://discuss.elastic.co/t/recalculate-times/272680)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 4:03pm UTC](https://discuss.elastic.co/t/recalculate-times/272680 "2021-05-11T16:03:00Z")
Hi, Due to a number of reasons we have servers logging with UTC en servers logging with timestamps in CEST. In kibana/es I would like to see everything in CESt to easily connect the dots with events in clientdevices. Is… --- ## [Error: Failed to decrypt report job data. Please ensure that xpack.reporting.encryptionKey is set and re-generate this report. Error: Unsupported state or unable to authenticate data](https://discuss.elastic.co/t/error-failed-to-decrypt-report-job-data-please-ensure-that-xpack-reporting-encryptionkey-is-set-and-re-generate-this-report-error-unsupported-state-or-unable-to-authenticate-data/272713)
**Author:** [@testsemd\_email](https://discuss.elastic.co/u/testsemd_email)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 4:02pm UTC](https://discuss.elastic.co/t/error-failed-to-decrypt-report-job-data-please-ensure-that-xpack-reporting-encryptionkey-is-set-and-re-generate-this-report-error-unsupported-state-or-unable-to-authenticate-data/272713 "2021-05-11T16:02:59Z")
Hi, I'm using elastic open source version 7.10.1 in my elastic cluster. xpack.reporting.encryptionKey has configured in kibana.yml I got this error while generating a CSV file using kibana. Also it occurs randomly. … --- ## [Parse very complicated logs with logstash (braces and random keys names)](https://discuss.elastic.co/t/parse-very-complicated-logs-with-logstash-braces-and-random-keys-names/272715)
**Author:** [@gmbynkrkrqz](https://discuss.elastic.co/u/gmbynkrkrqz)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 3:59pm UTC](https://discuss.elastic.co/t/parse-very-complicated-logs-with-logstash-braces-and-random-keys-names/272715 "2021-05-11T15:59:30Z")
Hello, I'm trying to parse very complicated logs as below but it's very hard. Could someone help me ? 2020-05-29T16:28:49.051146+02:00 machinehostname APP\_NAME\[31161\] @cee: {"category": "USER", "context": {"authMode": … --- ## [\[7.12.0\] Kibana Line Chart creates Duplicates when more than 10 lines](https://discuss.elastic.co/t/7-12-0-kibana-line-chart-creates-duplicates-when-more-than-10-lines/272640)
**Author:** [@pruoff](https://discuss.elastic.co/u/pruoff)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 3:46pm UTC](https://discuss.elastic.co/t/7-12-0-kibana-line-chart-creates-duplicates-when-more-than-10-lines/272640 "2021-05-11T15:46:45Z")
Hi! When creating a Kibana Line Chart Visualization in v7.12.0 or v7.12.1, I encounter an issue that duplicate lines are shown, when adding an eleventh line. In my case, I have 3 y-axes but I observe the same issue for … --- ## [Index "\_source.enabled": false does not remove source data and index size does not change at all due to this setting](https://discuss.elastic.co/t/index-source-enabled-false-does-not-remove-source-data-and-index-size-does-not-change-at-all-due-to-this-setting/272677)
**Author:** [@raulk89](https://discuss.elastic.co/u/raulk89)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 3:38pm UTC](https://discuss.elastic.co/t/index-source-enabled-false-does-not-remove-source-data-and-index-size-does-not-change-at-all-due-to-this-setting/272677 "2021-05-11T15:38:03Z")
Hi Elastic 7.11.2 I am indexing documents through pipelines (using ingest\_attachment plugin) My pipeline looks like this: PUT \_ingest/pipeline/my\_pipeline { "description": "pipeline for indexing documents", "… --- ## [ERROR: Pipelines YAML file must contain an array of pipeline configs. Found "Hash"](https://discuss.elastic.co/t/error-pipelines-yaml-file-must-contain-an-array-of-pipeline-configs-found-hash/272708)
**Author:** [@andcur](https://discuss.elastic.co/u/andcur)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 3:33pm UTC](https://discuss.elastic.co/t/error-pipelines-yaml-file-must-contain-an-array-of-pipeline-configs-found-hash/272708 "2021-05-11T15:33:10Z")
Windows server: I have the following pipeline.yml file -pipeline.id: policylocalexposurehub path.config: "config/logstash-sample.conf" pipeline.workers: 1 error ERROR: Pipelines YAML file must contain an array of pip… --- ## [Search multiple words using wildcard query](https://discuss.elastic.co/t/search-multiple-words-using-wildcard-query/272648)
**Author:** [@schawla](https://discuss.elastic.co/u/schawla)\ **Replies:** 5\ **Last updated:** [May 11, 2021, 3:32pm UTC](https://discuss.elastic.co/t/search-multiple-words-using-wildcard-query/272648 "2021-05-11T15:32:59Z")
I have documents indexed to elastic search in the following structure - \[{ "firstname" : "john", "lastname" : "cena" } , { "firstname" : "peter", "lastname" : "parker" }\] I am trying to use a bool query that contains… --- ## [Kibana Sort in visualization is not correct](https://discuss.elastic.co/t/kibana-sort-in-visualization-is-not-correct/272607)
**Author:** [@lachlann562](https://discuss.elastic.co/u/lachlann562)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 3:11pm UTC](https://discuss.elastic.co/t/kibana-sort-in-visualization-is-not-correct/272607 "2021-05-11T15:11:10Z")
In some visualizations, the sorting appears to not respect the selected values. for example, when using metricbeat i have a scripted field for "hour of day", "day of week" and "sundayOfWeek". Depending on how i filter my… --- ## [Elasticsearch - 'network.host': is not allowed](https://discuss.elastic.co/t/elasticsearch-network-host-is-not-allowed/272669)
**Author:** [@mfa](https://discuss.elastic.co/u/mfa)\ **Replies:** 4\ **Last updated:** [May 11, 2021, 2:52pm UTC](https://discuss.elastic.co/t/elasticsearch-network-host-is-not-allowed/272669 "2021-05-11T14:52:00Z")
I am using ElasticCloud , version v7.12.1, I have sat up a watcher for health check of cluster that it looks like below: PUT \_watcher/watch/cluster\_health\_watch { "trigger" : { "schedule" : { "cron… --- ## [Removing a field with dots in an ingest pipeline](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 2:50pm UTC](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570 "2021-05-11T14:50:21Z")
Hi all, I'm using filebeat with an ingest simple ingest pipeline and I want to remove some of the host's field, for example the host.os I've defined a ingest pipeline PUT \_ingest/pipeline/appsalutmw { "description":… --- ## [Path not found error when try to add on-prem S3 compatible storage](https://discuss.elastic.co/t/path-not-found-error-when-try-to-add-on-prem-s3-compatible-storage/271463)
**Author:** [@qiratnahraf](https://discuss.elastic.co/u/qiratnahraf)\ **Replies:** 0\ **Last updated:** [April 28, 2021, 4:55am UTC](https://discuss.elastic.co/t/path-not-found-error-when-try-to-add-on-prem-s3-compatible-storage/271463 "2021-04-28T04:55:45Z")
Hi, I am facing path not found error when try to register S3 compatible storage as repository. API call return with "path is not accessible on master node" following is my environment detail ELK Version 7.12 S3 plugi… --- ## [Graphing Interface Utilization with Timelion or TVSB](https://discuss.elastic.co/t/graphing-interface-utilization-with-timelion-or-tvsb/272459)
**Author:** [@nickr](https://discuss.elastic.co/u/nickr)\ **Replies:** 5\ **Last updated:** [May 11, 2021, 2:37pm UTC](https://discuss.elastic.co/t/graphing-interface-utilization-with-timelion-or-tvsb/272459 "2021-05-11T14:37:55Z")
Hi, I'm trying to follow this blogpost to normalize network traffic collected by the SNMP filter (ifHCInOctects/ifHCOutOctets) on a 7.12 stack. I'm sure I had this working on a previous version 6 months or so ago. At … --- ## [Filebeat 7.12 is collecting the events very slowly](https://discuss.elastic.co/t/filebeat-7-12-is-collecting-the-events-very-slowly/272579)
**Author:** [@bhavaniprasad\_reddy](https://discuss.elastic.co/u/bhavaniprasad_reddy)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 8:52am UTC](https://discuss.elastic.co/t/filebeat-7-12-is-collecting-the-events-very-slowly/272579 "2021-05-11T08:52:30Z")
I have deployed filebeat 7.12 in my cluster to collect events from kubernetes logs using autodiscover. The events are being collected in NRT (Near Real Time) for all the pods that I wish but stopped collecting only for … --- ## [\_id like var in output email](https://discuss.elastic.co/t/id-like-var-in-output-email/272696)
**Author:** [@cassiopee](https://discuss.elastic.co/u/cassiopee)\ **Replies:** 4\ **Last updated:** [May 11, 2021, 2:09pm UTC](https://discuss.elastic.co/t/id-like-var-in-output-email/272696 "2021-05-11T14:09:46Z")
hi everyone, I would use \_id (metadata) like variable in my output mail. I make my output on this way : \`output { elasticsearch { hosts =\> \[ "https://192.168.1.160:9200" \] ssl =\> true ssl\_certificate\_ve… --- ## [Combine different log events in one data table](https://discuss.elastic.co/t/combine-different-log-events-in-one-data-table/272403)
**Author:** [@bab](https://discuss.elastic.co/u/bab)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 2:04pm UTC](https://discuss.elastic.co/t/combine-different-log-events-in-one-data-table/272403 "2021-05-11T14:04:40Z")
Hi all, given are two log files from two different sources. log 1 2021-05-03 10:00:38,656 INFO \[Service TP\] : | M\_ID(3867) | ABC trade received from AFG via MM 2021-05-03 10:00:38,662 INFO \[Service TP\] : | M\_ID(3… --- ## [Beats indexes not being created after upgrading stack from oss-7.9.3 =\> basic-7.12.1](https://discuss.elastic.co/t/beats-indexes-not-being-created-after-upgrading-stack-from-oss-7-9-3-basic-7-12-1/271798)
**Author:** [@danpatdav](https://discuss.elastic.co/u/danpatdav)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 1:55pm UTC](https://discuss.elastic.co/t/beats-indexes-not-being-created-after-upgrading-stack-from-oss-7-9-3-basic-7-12-1/271798 "2021-05-11T13:55:08Z")
First post, so please feel free to let me know how things work around here. :slight\_smile: For the past several months I have been using OSS Elasticsearch + Beats to pull metrics from a few systems. The beats run in d… --- ## [ConnectionError while creating track in Rally with docker image](https://discuss.elastic.co/t/connectionerror-while-creating-track-in-rally-with-docker-image/272448)
**Author:** [@akm](https://discuss.elastic.co/u/akm)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 1:49pm UTC](https://discuss.elastic.co/t/connectionerror-while-creating-track-in-rally-with-docker-image/272448 "2021-05-11T13:49:03Z")
Hi, I am trying to create a custom track from an existing Elasticsearch cluster using the instructions in the Rally documentation page. I am running Rally as a Docker Image. I am running elasticsearch:7.12.0 as well a… --- ## [How can I delete an index after one month automatically?](https://discuss.elastic.co/t/how-can-i-delete-an-index-after-one-month-automatically/272695)
**Author:** [@Martin\_perez](https://discuss.elastic.co/u/Martin_perez)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 12:47pm UTC](https://discuss.elastic.co/t/how-can-i-delete-an-index-after-one-month-automatically/272695 "2021-05-11T12:47:22Z")
Hello everybody. I have a docker mounted elk stack server, which collects logs from various filebeats from other servers. Every month, I start a new index pattern,and I would like to make the previous index pattern auto… --- ## [Storage usage when maintaining data over 2 seperate indexes](https://discuss.elastic.co/t/storage-usage-when-maintaining-data-over-2-seperate-indexes/272257)
**Author:** [@Koren\_Molcho](https://discuss.elastic.co/u/Koren_Molcho)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 11:54am UTC](https://discuss.elastic.co/t/storage-usage-when-maintaining-data-over-2-seperate-indexes/272257 "2021-05-11T11:54:02Z")
Hello, I am looking into a solution for producing a dedup version of an existing index, similar to the solution offered here: Little Logstash Lessons: Handling Duplicates | Elastic Blog I have an index containing all … --- ## [Kibana Dashboard Drilldown](https://discuss.elastic.co/t/kibana-dashboard-drilldown/272554)
**Author:** [@Nisha2297](https://discuss.elastic.co/u/Nisha2297)\ **Replies:** 6\ **Last updated:** [May 11, 2021, 11:20am UTC](https://discuss.elastic.co/t/kibana-dashboard-drilldown/272554 "2021-05-11T11:20:54Z")
Is it possible to create drilldowns on a Table visualization in Kibana version 7.11? I want to create a drilldown on a table viz which redirects me to another table containing details in the same dashboard. --- ## [ILM hot/warm policy is not moving shards](https://discuss.elastic.co/t/ilm-hot-warm-policy-is-not-moving-shards/272603)
**Author:** [@vkatsikaros](https://discuss.elastic.co/u/vkatsikaros)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 10:25am UTC](https://discuss.elastic.co/t/ilm-hot-warm-policy-is-not-moving-shards/272603 "2021-05-11T10:25:20Z")
Hi I have created an ILM policy and template, and I create daily indices with Logstash on an ES cluster 7.10. I have some fast and expensive nodes as hot (and other types) and a few slow and inexpensive as warm only. I … --- ## [Machine learning - Data visualiser unable to determine file structure](https://discuss.elastic.co/t/machine-learning-data-visualiser-unable-to-determine-file-structure/272654)
**Author:** [@AQ\_Amra](https://discuss.elastic.co/u/AQ_Amra)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 9:56am UTC](https://discuss.elastic.co/t/machine-learning-data-visualiser-unable-to-determine-file-structure/272654 "2021-05-11T09:56:18Z")
Hi, I am attempting to load a CSV using the Machine learning - Data visualiser, however if i upload a file with more than 30K records, the Machine learning - Data visualiser fails to determine the file structure. I hav… --- ## [Elastic Search update api - updating complex data including array objects and parts of document](https://discuss.elastic.co/t/elastic-search-update-api-updating-complex-data-including-array-objects-and-parts-of-document/272644)
**Author:** [@ab.umarjikar](https://discuss.elastic.co/u/ab.umarjikar)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 9:20am UTC](https://discuss.elastic.co/t/elastic-search-update-api-updating-complex-data-including-array-objects-and-parts-of-document/272644 "2021-05-11T09:20:40Z")
I have complex structure of data coming in as bellow: { ​​​​​ "id": 1, "other\_details": \[{ ​​​​​ "address\_details": { ​​​​​ "city": "pune", … --- ## [Please explain how rewrite top\_terms\_n works](https://discuss.elastic.co/t/please-explain-how-rewrite-top-terms-n-works/270128)
**Author:** [@carantunes](https://discuss.elastic.co/u/carantunes)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 8:57am UTC](https://discuss.elastic.co/t/please-explain-how-rewrite-top-terms-n-works/270128 "2021-05-11T08:57:59Z")
Can someone expand a bit on how rewrite top\_terms\_n works? Related to https://discuss.elastic.co/t/missing-results-using-query-string-wildcard-query-with-rewrite-top-terms-n/269680 Thanks in advance! [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1419) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1421)