# Latest **URL:** https://discuss.elastic.co/latest.md?page=1684 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1685 --- ## [Parse very complicated logs with logstash (braces and random keys names)](https://discuss.elastic.co/t/parse-very-complicated-logs-with-logstash-braces-and-random-keys-names/272715)
**Author:** [@gmbynkrkrqz](https://discuss.elastic.co/u/gmbynkrkrqz)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 3:59pm UTC](https://discuss.elastic.co/t/parse-very-complicated-logs-with-logstash-braces-and-random-keys-names/272715 "2021-05-11T15:59:30Z")
Hello, I'm trying to parse very complicated logs as below but it's very hard. Could someone help me ? 2020-05-29T16:28:49.051146+02:00 machinehostname APP\_NAME\[31161\] @cee: {"category": "USER", "context": {"authMode": … --- ## [\[7.12.0\] Kibana Line Chart creates Duplicates when more than 10 lines](https://discuss.elastic.co/t/7-12-0-kibana-line-chart-creates-duplicates-when-more-than-10-lines/272640)
**Author:** [@pruoff](https://discuss.elastic.co/u/pruoff)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 3:46pm UTC](https://discuss.elastic.co/t/7-12-0-kibana-line-chart-creates-duplicates-when-more-than-10-lines/272640 "2021-05-11T15:46:45Z")
Hi! When creating a Kibana Line Chart Visualization in v7.12.0 or v7.12.1, I encounter an issue that duplicate lines are shown, when adding an eleventh line. In my case, I have 3 y-axes but I observe the same issue for … --- ## [Index "\_source.enabled": false does not remove source data and index size does not change at all due to this setting](https://discuss.elastic.co/t/index-source-enabled-false-does-not-remove-source-data-and-index-size-does-not-change-at-all-due-to-this-setting/272677)
**Author:** [@raulk89](https://discuss.elastic.co/u/raulk89)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 3:38pm UTC](https://discuss.elastic.co/t/index-source-enabled-false-does-not-remove-source-data-and-index-size-does-not-change-at-all-due-to-this-setting/272677 "2021-05-11T15:38:03Z")
Hi Elastic 7.11.2 I am indexing documents through pipelines (using ingest\_attachment plugin) My pipeline looks like this: PUT \_ingest/pipeline/my\_pipeline { "description": "pipeline for indexing documents", "… --- ## [ERROR: Pipelines YAML file must contain an array of pipeline configs. Found "Hash"](https://discuss.elastic.co/t/error-pipelines-yaml-file-must-contain-an-array-of-pipeline-configs-found-hash/272708)
**Author:** [@andcur](https://discuss.elastic.co/u/andcur)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 3:33pm UTC](https://discuss.elastic.co/t/error-pipelines-yaml-file-must-contain-an-array-of-pipeline-configs-found-hash/272708 "2021-05-11T15:33:10Z")
Windows server: I have the following pipeline.yml file -pipeline.id: policylocalexposurehub path.config: "config/logstash-sample.conf" pipeline.workers: 1 error ERROR: Pipelines YAML file must contain an array of pip… --- ## [Search multiple words using wildcard query](https://discuss.elastic.co/t/search-multiple-words-using-wildcard-query/272648)
**Author:** [@schawla](https://discuss.elastic.co/u/schawla)\ **Replies:** 5\ **Last updated:** [May 11, 2021, 3:32pm UTC](https://discuss.elastic.co/t/search-multiple-words-using-wildcard-query/272648 "2021-05-11T15:32:59Z")
I have documents indexed to elastic search in the following structure - \[{ "firstname" : "john", "lastname" : "cena" } , { "firstname" : "peter", "lastname" : "parker" }\] I am trying to use a bool query that contains… --- ## [Kibana Sort in visualization is not correct](https://discuss.elastic.co/t/kibana-sort-in-visualization-is-not-correct/272607)
**Author:** [@lachlann562](https://discuss.elastic.co/u/lachlann562)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 3:11pm UTC](https://discuss.elastic.co/t/kibana-sort-in-visualization-is-not-correct/272607 "2021-05-11T15:11:10Z")
In some visualizations, the sorting appears to not respect the selected values. for example, when using metricbeat i have a scripted field for "hour of day", "day of week" and "sundayOfWeek". Depending on how i filter my… --- ## [Elasticsearch - 'network.host': is not allowed](https://discuss.elastic.co/t/elasticsearch-network-host-is-not-allowed/272669)
**Author:** [@mfa](https://discuss.elastic.co/u/mfa)\ **Replies:** 4\ **Last updated:** [May 11, 2021, 2:52pm UTC](https://discuss.elastic.co/t/elasticsearch-network-host-is-not-allowed/272669 "2021-05-11T14:52:00Z")
I am using ElasticCloud , version v7.12.1, I have sat up a watcher for health check of cluster that it looks like below: PUT \_watcher/watch/cluster\_health\_watch { "trigger" : { "schedule" : { "cron… --- ## [Removing a field with dots in an ingest pipeline](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 2:50pm UTC](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570 "2021-05-11T14:50:21Z")
Hi all, I'm using filebeat with an ingest simple ingest pipeline and I want to remove some of the host's field, for example the host.os I've defined a ingest pipeline PUT \_ingest/pipeline/appsalutmw { "description":… --- ## [Path not found error when try to add on-prem S3 compatible storage](https://discuss.elastic.co/t/path-not-found-error-when-try-to-add-on-prem-s3-compatible-storage/271463)
**Author:** [@qiratnahraf](https://discuss.elastic.co/u/qiratnahraf)\ **Replies:** 0\ **Last updated:** [April 28, 2021, 4:55am UTC](https://discuss.elastic.co/t/path-not-found-error-when-try-to-add-on-prem-s3-compatible-storage/271463 "2021-04-28T04:55:45Z")
Hi, I am facing path not found error when try to register S3 compatible storage as repository. API call return with "path is not accessible on master node" following is my environment detail ELK Version 7.12 S3 plugi… --- ## [Graphing Interface Utilization with Timelion or TVSB](https://discuss.elastic.co/t/graphing-interface-utilization-with-timelion-or-tvsb/272459)
**Author:** [@nickr](https://discuss.elastic.co/u/nickr)\ **Replies:** 5\ **Last updated:** [May 11, 2021, 2:37pm UTC](https://discuss.elastic.co/t/graphing-interface-utilization-with-timelion-or-tvsb/272459 "2021-05-11T14:37:55Z")
Hi, I'm trying to follow this blogpost to normalize network traffic collected by the SNMP filter (ifHCInOctects/ifHCOutOctets) on a 7.12 stack. I'm sure I had this working on a previous version 6 months or so ago. At … --- ## [Filebeat 7.12 is collecting the events very slowly](https://discuss.elastic.co/t/filebeat-7-12-is-collecting-the-events-very-slowly/272579)
**Author:** [@bhavaniprasad\_reddy](https://discuss.elastic.co/u/bhavaniprasad_reddy)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 8:52am UTC](https://discuss.elastic.co/t/filebeat-7-12-is-collecting-the-events-very-slowly/272579 "2021-05-11T08:52:30Z")
I have deployed filebeat 7.12 in my cluster to collect events from kubernetes logs using autodiscover. The events are being collected in NRT (Near Real Time) for all the pods that I wish but stopped collecting only for … --- ## [\_id like var in output email](https://discuss.elastic.co/t/id-like-var-in-output-email/272696)
**Author:** [@cassiopee](https://discuss.elastic.co/u/cassiopee)\ **Replies:** 4\ **Last updated:** [May 11, 2021, 2:09pm UTC](https://discuss.elastic.co/t/id-like-var-in-output-email/272696 "2021-05-11T14:09:46Z")
hi everyone, I would use \_id (metadata) like variable in my output mail. I make my output on this way : \`output { elasticsearch { hosts =\> \[ "https://192.168.1.160:9200" \] ssl =\> true ssl\_certificate\_ve… --- ## [Combine different log events in one data table](https://discuss.elastic.co/t/combine-different-log-events-in-one-data-table/272403)
**Author:** [@bab](https://discuss.elastic.co/u/bab)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 2:04pm UTC](https://discuss.elastic.co/t/combine-different-log-events-in-one-data-table/272403 "2021-05-11T14:04:40Z")
Hi all, given are two log files from two different sources. log 1 2021-05-03 10:00:38,656 INFO \[Service TP\] : | M\_ID(3867) | ABC trade received from AFG via MM 2021-05-03 10:00:38,662 INFO \[Service TP\] : | M\_ID(3… --- ## [Automatic instrumentation of function calls](https://discuss.elastic.co/t/automatic-instrumentation-of-function-calls/272707)
**Author:** [@lingwood](https://discuss.elastic.co/u/lingwood)\ **Replies:** 0\ **Last updated:** [May 11, 2021, 2:00pm UTC](https://discuss.elastic.co/t/automatic-instrumentation-of-function-calls/272707 "2021-05-11T14:00:04Z")
Kibana version: 7.12.1 Elasticsearch version: 7.12.1 APM Server version: 7.12.1 APM Agent language and version: php agent 1.01. Php 7.3.27 Browser version: Edge 90.0.818.56 Original install method (e.g. download pag… --- ## [Beats indexes not being created after upgrading stack from oss-7.9.3 =\> basic-7.12.1](https://discuss.elastic.co/t/beats-indexes-not-being-created-after-upgrading-stack-from-oss-7-9-3-basic-7-12-1/271798)
**Author:** [@danpatdav](https://discuss.elastic.co/u/danpatdav)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 1:55pm UTC](https://discuss.elastic.co/t/beats-indexes-not-being-created-after-upgrading-stack-from-oss-7-9-3-basic-7-12-1/271798 "2021-05-11T13:55:08Z")
First post, so please feel free to let me know how things work around here. :slight\_smile: For the past several months I have been using OSS Elasticsearch + Beats to pull metrics from a few systems. The beats run in d… --- ## [ConnectionError while creating track in Rally with docker image](https://discuss.elastic.co/t/connectionerror-while-creating-track-in-rally-with-docker-image/272448)
**Author:** [@akm](https://discuss.elastic.co/u/akm)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 1:49pm UTC](https://discuss.elastic.co/t/connectionerror-while-creating-track-in-rally-with-docker-image/272448 "2021-05-11T13:49:03Z")
Hi, I am trying to create a custom track from an existing Elasticsearch cluster using the instructions in the Rally documentation page. I am running Rally as a Docker Image. I am running elasticsearch:7.12.0 as well a… --- ## [How can I delete an index after one month automatically?](https://discuss.elastic.co/t/how-can-i-delete-an-index-after-one-month-automatically/272695)
**Author:** [@Martin\_perez](https://discuss.elastic.co/u/Martin_perez)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 12:47pm UTC](https://discuss.elastic.co/t/how-can-i-delete-an-index-after-one-month-automatically/272695 "2021-05-11T12:47:22Z")
Hello everybody. I have a docker mounted elk stack server, which collects logs from various filebeats from other servers. Every month, I start a new index pattern,and I would like to make the previous index pattern auto… --- ## [Storage usage when maintaining data over 2 seperate indexes](https://discuss.elastic.co/t/storage-usage-when-maintaining-data-over-2-seperate-indexes/272257)
**Author:** [@Koren\_Molcho](https://discuss.elastic.co/u/Koren_Molcho)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 11:54am UTC](https://discuss.elastic.co/t/storage-usage-when-maintaining-data-over-2-seperate-indexes/272257 "2021-05-11T11:54:02Z")
Hello, I am looking into a solution for producing a dedup version of an existing index, similar to the solution offered here: Little Logstash Lessons: Handling Duplicates | Elastic Blog I have an index containing all … --- ## [Kibana Dashboard Drilldown](https://discuss.elastic.co/t/kibana-dashboard-drilldown/272554)
**Author:** [@Nisha2297](https://discuss.elastic.co/u/Nisha2297)\ **Replies:** 6\ **Last updated:** [May 11, 2021, 11:20am UTC](https://discuss.elastic.co/t/kibana-dashboard-drilldown/272554 "2021-05-11T11:20:54Z")
Is it possible to create drilldowns on a Table visualization in Kibana version 7.11? I want to create a drilldown on a table viz which redirects me to another table containing details in the same dashboard. --- ## [Display the DNS of the visiting IP](https://discuss.elastic.co/t/display-the-dns-of-the-visiting-ip/272174)
**Author:** [@ethical20](https://discuss.elastic.co/u/ethical20)\ **Replies:** 6\ **Last updated:** [May 11, 2021, 10:38am UTC](https://discuss.elastic.co/t/display-the-dns-of-the-visiting-ip/272174 "2021-05-11T10:38:40Z")
Hi, In SEIM I can see the DNS requests coming out from my machine. Like when I'm visiting google it is logged here in SIEM --\> Network --\> DNS I need the contrary, How can I see the DNS of the IP's visiting my machin… --- ## [ILM hot/warm policy is not moving shards](https://discuss.elastic.co/t/ilm-hot-warm-policy-is-not-moving-shards/272603)
**Author:** [@vkatsikaros](https://discuss.elastic.co/u/vkatsikaros)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 10:25am UTC](https://discuss.elastic.co/t/ilm-hot-warm-policy-is-not-moving-shards/272603 "2021-05-11T10:25:20Z")
Hi I have created an ILM policy and template, and I create daily indices with Logstash on an ES cluster 7.10. I have some fast and expensive nodes as hot (and other types) and a few slow and inexpensive as warm only. I … --- ## [Separate engines per product type?](https://discuss.elastic.co/t/separate-engines-per-product-type/272163)
**Author:** [@Peter418](https://discuss.elastic.co/u/Peter418)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 10:14am UTC](https://discuss.elastic.co/t/separate-engines-per-product-type/272163 "2021-05-11T10:14:44Z")
Hi, just want to hear your opinion; what is the best way to index multiple product types (for example, books, lawn mowers and televisions) by using app search? Each type has their own properties... should you use one en… --- ## [Machine learning - Data visualiser unable to determine file structure](https://discuss.elastic.co/t/machine-learning-data-visualiser-unable-to-determine-file-structure/272654)
**Author:** [@AQ\_Amra](https://discuss.elastic.co/u/AQ_Amra)\ **Replies:** 2\ **Last updated:** [May 11, 2021, 9:56am UTC](https://discuss.elastic.co/t/machine-learning-data-visualiser-unable-to-determine-file-structure/272654 "2021-05-11T09:56:18Z")
Hi, I am attempting to load a CSV using the Machine learning - Data visualiser, however if i upload a file with more than 30K records, the Machine learning - Data visualiser fails to determine the file structure. I hav… --- ## [Elastic Search update api - updating complex data including array objects and parts of document](https://discuss.elastic.co/t/elastic-search-update-api-updating-complex-data-including-array-objects-and-parts-of-document/272644)
**Author:** [@ab.umarjikar](https://discuss.elastic.co/u/ab.umarjikar)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 9:20am UTC](https://discuss.elastic.co/t/elastic-search-update-api-updating-complex-data-including-array-objects-and-parts-of-document/272644 "2021-05-11T09:20:40Z")
I have complex structure of data coming in as bellow: { ​​​​​ "id": 1, "other\_details": \[{ ​​​​​ "address\_details": { ​​​​​ "city": "pune", … --- ## [Please explain how rewrite top\_terms\_n works](https://discuss.elastic.co/t/please-explain-how-rewrite-top-terms-n-works/270128)
**Author:** [@carantunes](https://discuss.elastic.co/u/carantunes)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 8:57am UTC](https://discuss.elastic.co/t/please-explain-how-rewrite-top-terms-n-works/270128 "2021-05-11T08:57:59Z")
Can someone expand a bit on how rewrite top\_terms\_n works? Related to https://discuss.elastic.co/t/missing-results-using-query-string-wildcard-query-with-rewrite-top-terms-n/269680 Thanks in advance! --- ## [Can't get disk information (Windows) - Metricbeat 7.12.0](https://discuss.elastic.co/t/cant-get-disk-information-windows-metricbeat-7-12-0/270735)
**Author:** [@ruivit](https://discuss.elastic.co/u/ruivit)\ **Replies:** 3\ **Last updated:** [May 11, 2021, 8:42am UTC](https://discuss.elastic.co/t/cant-get-disk-information-windows-metricbeat-7-12-0/270735 "2021-05-11T08:42:28Z")
Hi everyone. After a couple of minutes searching and trying out solutions, I still can't get information about the filesystem and fsstat of, for example of the C drive on my Windows Server. Tried the following … --- ## [KQL and scripted fields](https://discuss.elastic.co/t/kql-and-scripted-fields/272260)
**Author:** [@thomasv95](https://discuss.elastic.co/u/thomasv95)\ **Replies:** 5\ **Last updated:** [May 11, 2021, 8:15am UTC](https://discuss.elastic.co/t/kql-and-scripted-fields/272260 "2021-05-11T08:15:41Z")
Hello, I've been having issues with KQL queries using scripted fields. Specifically, queries on numerical fields using operators such as "\<=" or "\>" throw the following error when using a Lens : \[lens\_merge\_tables\] \> \[e… --- ## [Transforms - using painless for terms aggs on nested objects](https://discuss.elastic.co/t/transforms-using-painless-for-terms-aggs-on-nested-objects/272501)
**Author:** [@OmarDacca](https://discuss.elastic.co/u/OmarDacca)\ **Replies:** 1\ **Last updated:** [May 11, 2021, 7:31am UTC](https://discuss.elastic.co/t/transforms-using-painless-for-terms-aggs-on-nested-objects/272501 "2021-05-11T07:31:42Z")
I understand that Transforms does not support nested aggregations, but what is the alternative ? here is the mappings: "properties" : { "clientId" : { "type" : "integer" }, "clientIdAsString" : { "type" : "keywor… --- ## [ElasaticSearch Killed](https://discuss.elastic.co/t/elasaticsearch-killed/272580)
**Author:** [@Sohaib\_Gondal](https://discuss.elastic.co/u/Sohaib_Gondal)\ **Replies:** 6\ **Last updated:** [May 11, 2021, 7:16am UTC](https://discuss.elastic.co/t/elasaticsearch-killed/272580 "2021-05-11T07:16:04Z")
When I try to run the Elasticsearch it says "Killed" and shows the following error. Error Log \[2021-05-10T11:56:50.233+0000\]\[15547\]\[gc\] Using Concurrent Mark Sweep \[2021-05-10T11:56:50.233+0000\]\[15547\]\[gc,heap,coops\] … --- ## [String Scripted field conversion to keyword](https://discuss.elastic.co/t/string-scripted-field-conversion-to-keyword/272531)
**Author:** [@SHUBHAM\_SALUNKHE](https://discuss.elastic.co/u/SHUBHAM_SALUNKHE)\ **Replies:** 5\ **Last updated:** [May 11, 2021, 5:35am UTC](https://discuss.elastic.co/t/string-scripted-field-conversion-to-keyword/272531 "2021-05-11T05:35:40Z")
Hi, I have created one scripted field of string format. Need to convert that in keyword format to perform aggregation operation on that scripted field. How can I convert that string? Thanks in advance. [Previous page](https://discuss.elastic.co/latest.md?page=1683) [Next page](https://discuss.elastic.co/latest.md?page=1685)