# Latest **URL:** https://discuss.elastic.co/latest.md?page=1734 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1735 --- ## [ILM Policy through logstash dynamic index name](https://discuss.elastic.co/t/ilm-policy-through-logstash-dynamic-index-name/268011)
**Author:** [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Replies:** 9\ **Last updated:** [April 13, 2021, 7:05pm UTC](https://discuss.elastic.co/t/ilm-policy-through-logstash-dynamic-index-name/268011 "2021-04-13T19:05:07Z")
Hello Team, I am trying to implement the ILM Policy but I ran into an issue because my index name is dynamically created through logstash. Let me give you an explain of my setup and wat I am trying to do I create th… --- ## [Duplicated Log](https://discuss.elastic.co/t/duplicated-log/269931)
**Author:** [@Paulogbr](https://discuss.elastic.co/u/Paulogbr)\ **Replies:** 9\ **Last updated:** [April 13, 2021, 6:19pm UTC](https://discuss.elastic.co/t/duplicated-log/269931 "2021-04-13T18:19:41Z")
Hello Guys, My firewall device is sending duplicated log. I tried filter with logstash. I need write the output in file. I test with logger but don´t work, I tried use fingerprint. Can you have any ideia ? inpu… --- ## [Как отображать в выдаче только дубликаты - аггрегация, поиск дубликатов](https://discuss.elastic.co/t/topic/269874)
**Author:** [@Sergei\_Frolov](https://discuss.elastic.co/u/Sergei_Frolov)\ **Replies:** 9\ **Last updated:** [April 13, 2021, 6:12pm UTC](https://discuss.elastic.co/t/topic/269874 "2021-04-13T18:12:51Z")
Добрый день! Выполняю аггрегацию для поиска дублей: GET app\_log-\*/\_search { "size": 0, "aggs": { "duplicateCount": { "terms": { "field": "context.id", "min\_doc\_count": 2 }, "ag… --- ## [Logstash doesn't automatically collect all Zeek fields without grok pattern](https://discuss.elastic.co/t/logstash-doesnt-automatically-collect-all-zeek-fields-without-grok-pattern/269613)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 3\ **Last updated:** [April 13, 2021, 6:01pm UTC](https://discuss.elastic.co/t/logstash-doesnt-automatically-collect-all-zeek-fields-without-grok-pattern/269613 "2021-04-13T18:01:49Z")
Hi, Is there a setting I need to provide in order to enable the automatically collection of all the Zeek's log fields? I can collect the fields message only through a grok filter. My assumption is that logstash is smar… --- ## [Issues with Elastic Agent and Defender?](https://discuss.elastic.co/t/issues-with-elastic-agent-and-defender/269818)
**Author:** [@nemhods](https://discuss.elastic.co/u/nemhods)\ **Replies:** 6\ **Last updated:** [April 13, 2021, 5:27pm UTC](https://discuss.elastic.co/t/issues-with-elastic-agent-and-defender/269818 "2021-04-13T17:27:09Z")
After an upgrade to 7.12, a few machines experience high CPU load from Elastic endpoint security. The load seems to be periodic: The three processes/services that seem to cause this load are Elastic Endpoint, MS D… --- ## [Сравнение документов внутри индекса](https://discuss.elastic.co/t/topic/268688)
**Author:** [@frostwar](https://discuss.elastic.co/u/frostwar)\ **Replies:** 11\ **Last updated:** [April 13, 2021, 5:16pm UTC](https://discuss.elastic.co/t/topic/268688 "2021-04-13T17:16:23Z")
Всем привет. В индексе есть несколько типов документов. Одни документы - Input, другие - Output. В каждом документе есть поле session\_ID, которое отвечает за уникальность передачи сесии. Вопросы: Как связать/сопост… --- ## [Index is not showing to create index pattern](https://discuss.elastic.co/t/index-is-not-showing-to-create-index-pattern/269973)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 1\ **Last updated:** [April 13, 2021, 4:41pm UTC](https://discuss.elastic.co/t/index-is-not-showing-to-create-index-pattern/269973 "2021-04-13T16:41:23Z")
I have created index in kibana Dev Tools but it is not visible in stack Management to create its index pattern for analysis. --- ## [Setting multiple pipelines for elasticserach.output](https://discuss.elastic.co/t/setting-multiple-pipelines-for-elasticserach-output/269999)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 1\ **Last updated:** [April 13, 2021, 3:52pm UTC](https://discuss.elastic.co/t/setting-multiple-pipelines-for-elasticserach-output/269999 "2021-04-13T15:52:22Z")
Hello, Is It possible to use for example metricbeat.yml config file for setting multiple ingest node pipelines for the same document/index? I triend something like this but only the first one work. #-------------------… --- ## [Unable to find elasticsearch, logstash, kibana metrics on stackmonitoring even after having all the .monitoring-\* indices created and the Cluster\_uuid is right](https://discuss.elastic.co/t/unable-to-find-elasticsearch-logstash-kibana-metrics-on-stackmonitoring-even-after-having-all-the-monitoring-indices-created-and-the-cluster-uuid-is-right/269513)
**Author:** [@Pooja](https://discuss.elastic.co/u/Pooja)\ **Replies:** 3\ **Last updated:** [April 8, 2021, 2:19pm UTC](https://discuss.elastic.co/t/unable-to-find-elasticsearch-logstash-kibana-metrics-on-stackmonitoring-even-after-having-all-the-monitoring-indices-created-and-the-cluster-uuid-is-right/269513 "2021-04-08T14:19:09Z")
Unable to find elasticsearch, logstash, kibana metrics on stackmonitoring even after having all the .monitoring-\* indices created and the Cluster\_uuid is right. --- ## [Report Generation issue in from Kibana Visualize](https://discuss.elastic.co/t/report-generation-issue-in-from-kibana-visualize/269322)
**Author:** [@jancodenew](https://discuss.elastic.co/u/jancodenew)\ **Replies:** 8\ **Last updated:** [April 13, 2021, 3:26pm UTC](https://discuss.elastic.co/t/report-generation-issue-in-from-kibana-visualize/269322 "2021-04-13T15:26:32Z")
Hello, I want send the report with event count of each devices in last 24 hours by email. I have created new visualization by aggregating with src.ip feild. When i am downloading the report as PDF, full infromation is n… --- ## [Getting master\_not\_discovered\_exception from node](https://discuss.elastic.co/t/getting-master-not-discovered-exception-from-node/269840)
**Author:** [@arielman2304](https://discuss.elastic.co/u/arielman2304)\ **Replies:** 6\ **Last updated:** [April 13, 2021, 3:20pm UTC](https://discuss.elastic.co/t/getting-master-not-discovered-exception-from-node/269840 "2021-04-13T15:20:19Z")
Hi I created 3 EC2 clusters in AWS, and I'm trying to set 2 of them as nodes. I followed this guideline: However, nothing helps and I'm still getting this exception from the nodes. all ES are up and return response t… --- ## [Missing certain log entries (json from k8s using filebeat)](https://discuss.elastic.co/t/missing-certain-log-entries-json-from-k8s-using-filebeat/269948)
**Author:** [@geoaxis](https://discuss.elastic.co/u/geoaxis)\ **Replies:** 2\ **Last updated:** [April 13, 2021, 3:18pm UTC](https://discuss.elastic.co/t/missing-certain-log-entries-json-from-k8s-using-filebeat/269948 "2021-04-13T15:18:42Z")
I am a consumer of an elastic stack (filebeat, es, kibana) that gets structured json logs from a kubernetes cluster (writing to stdout). The stack is slef hosted by a group at my workplace and unfortunately it does not … --- ## [Xml with optionnal fields not added in the result array by logstash](https://discuss.elastic.co/t/xml-with-optionnal-fields-not-added-in-the-result-array-by-logstash/269891)
**Author:** [@meepmeep](https://discuss.elastic.co/u/meepmeep)\ **Replies:** 7\ **Last updated:** [April 13, 2021, 3:17pm UTC](https://discuss.elastic.co/t/xml-with-optionnal-fields-not-added-in-the-result-array-by-logstash/269891 "2021-04-13T15:17:09Z")
Hi :slight\_smile: I'm moving slowly in this new wold of elastic stack. I'm currently stuck on a parsing issue with my xml file (ip and nickname removed) : \ \ \ \User1\ … --- ## [Can I get a specific part of the referrer in the nginx log when parsing it via logstash](https://discuss.elastic.co/t/can-i-get-a-specific-part-of-the-referrer-in-the-nginx-log-when-parsing-it-via-logstash/269865)
**Author:** [@pradeep\_gadkari](https://discuss.elastic.co/u/pradeep_gadkari)\ **Replies:** 3\ **Last updated:** [April 13, 2021, 3:13pm UTC](https://discuss.elastic.co/t/can-i-get-a-specific-part-of-the-referrer-in-the-nginx-log-when-parsing-it-via-logstash/269865 "2021-04-13T15:13:04Z")
My grok pattern is as below: grok{ match =\> { "message" =\> \["%{IPV4:IP\_address} (?:-|(%{WORD}.%{WORD})) %{USER:ident} \\\[%{HTTPDATE:message\_timestamp}\\\] \\"(?:%{WORD:message\_type} %{NOTSPACE:request}(?: HTTP/%{NUMB… --- ## [An error occurred during rule execution](https://discuss.elastic.co/t/an-error-occurred-during-rule-execution/269536)
**Author:** [@austinsonger](https://discuss.elastic.co/u/austinsonger)\ **Replies:** 1\ **Last updated:** [April 13, 2021, 2:06pm UTC](https://discuss.elastic.co/t/an-error-occurred-during-rule-execution/269536 "2021-04-13T14:06:31Z")
I've been getting this error on all of my Machine Learning Rules. --- ## [Invalid value for MonthOfYear and nested JSON mapping](https://discuss.elastic.co/t/invalid-value-for-monthofyear-and-nested-json-mapping/270019)
**Author:** [@tommycahir](https://discuss.elastic.co/u/tommycahir)\ **Replies:** 0\ **Last updated:** [April 13, 2021, 2:02pm UTC](https://discuss.elastic.co/t/invalid-value-for-monthofyear-and-nested-json-mapping/270019 "2021-04-13T14:02:41Z")
Hi Folks, I am very new to Elastic Stack and am trying to insert a JSON list of JSON API responses to an index from a python script and am getting the error: elasticsearch.exceptions.RequestError: RequestError(400, 'ma… --- ## [Generate report in Kibana results in 404](https://discuss.elastic.co/t/generate-report-in-kibana-results-in-404/269050)
**Author:** [@eeijlar](https://discuss.elastic.co/u/eeijlar)\ **Replies:** 8\ **Last updated:** [April 13, 2021, 2:00pm UTC](https://discuss.elastic.co/t/generate-report-in-kibana-results-in-404/269050 "2021-04-13T14:00:22Z")
If I generate a pdf report from Kibana web UI, using Share \> PDF Reports \> Generate PDF. Then I got to : Stack Management \> Reporting \> Download report The resulting report just has: ELK Stack version 7.11.2 The sa… --- ## [Reindex nested to non-nested](https://discuss.elastic.co/t/reindex-nested-to-non-nested/269961)
**Author:** [@ajitha](https://discuss.elastic.co/u/ajitha)\ **Replies:** 0\ **Last updated:** [April 13, 2021, 2:07am UTC](https://discuss.elastic.co/t/reindex-nested-to-non-nested/269961 "2021-04-13T02:07:55Z")
Does the script in the "reindex API" support mapping of nested fields to non nested fields? If i do use the script I keep getting cannot create source field in dest index, even though I have "remove". What is the issu… --- ## [Kibana: Scripted Field: Dynamic date (now - 15d / now - 30d) not working](https://discuss.elastic.co/t/kibana-scripted-field-dynamic-date-now-15d-now-30d-not-working/269781)
**Author:** [@nshari.24](https://discuss.elastic.co/u/nshari.24)\ **Replies:** 4\ **Last updated:** [April 13, 2021, 11:11am UTC](https://discuss.elastic.co/t/kibana-scripted-field-dynamic-date-now-15d-now-30d-not-working/269781 "2021-04-13T11:11:47Z")
I have a scripted field of the format "Date". When I try using this field in Kibana visualization and apply a filter like "date \> now-30d", I am getting an error: caused\_by": { "type": "date\_time\_pa… --- ## [Watchers and Elastic Security](https://discuss.elastic.co/t/watchers-and-elastic-security/268360)
**Author:** [@Anna\_Foxx](https://discuss.elastic.co/u/Anna_Foxx)\ **Replies:** 1\ **Last updated:** [April 13, 2021, 11:01am UTC](https://discuss.elastic.co/t/watchers-and-elastic-security/268360 "2021-04-13T11:01:28Z")
Hi! I have a lot of detections on my watchers. Is it possible to have the alert generated by watchers in the Detection section of Elastic Security SIEM? --- ## [DDoS attach detection using Elastic stack](https://discuss.elastic.co/t/ddos-attach-detection-using-elastic-stack/258984)
**Author:** [@atharvak](https://discuss.elastic.co/u/atharvak)\ **Replies:** 1\ **Last updated:** [April 13, 2021, 10:54am UTC](https://discuss.elastic.co/t/ddos-attach-detection-using-elastic-stack/258984 "2021-04-13T10:54:49Z")
Hello all, I am currently using Elastic stack to parse Apache and Nginx web logs with Filebeat installed on multiple remote servers shipping logs to my Elastic server. I want to use the same infrastructure to be able to… --- ## [How to use the full tracks instead of 1k](https://discuss.elastic.co/t/how-to-use-the-full-tracks-instead-of-1k/269994)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 1\ **Last updated:** [April 13, 2021, 10:53am UTC](https://discuss.elastic.co/t/how-to-use-the-full-tracks-instead-of-1k/269994 "2021-04-13T10:53:46Z")
hey there, I am using the Docker image to try several benchmarks. Can you say to me how to use the full tracks instead of just 1k docs? Running a basic command elastic/rally race --track=nyc\_taxis --test-mode --pipelin… --- ## [Elastic logging plugin doesn’t work with self-signed certificate](https://discuss.elastic.co/t/elastic-logging-plugin-doesn-t-work-with-self-signed-certificate/268406)
**Author:** [@nvb97](https://discuss.elastic.co/u/nvb97)\ **Replies:** 4\ **Last updated:** [April 13, 2021, 10:05am UTC](https://discuss.elastic.co/t/elastic-logging-plugin-doesn-t-work-with-self-signed-certificate/268406 "2021-04-13T10:05:41Z")
Hi all, I have installed Elastic Logging Plugin for Docker (v.7.11.1) which should send the data to Elasticsearch where the security is enabled, but the log events from the docker container don’t arrive in elastic index… --- ## [Timestamp issue with Cisco ios module](https://discuss.elastic.co/t/timestamp-issue-with-cisco-ios-module/269987)
**Author:** [@rs-patrick](https://discuss.elastic.co/u/rs-patrick)\ **Replies:** 0\ **Last updated:** [April 13, 2021, 9:56am UTC](https://discuss.elastic.co/t/timestamp-issue-with-cisco-ios-module/269987 "2021-04-13T09:56:24Z")
Hello together, If i use the Cisco ios module, i have an issue with the timezone. The field @timestamp contains Z instead of +2.00 (from MEST). Here my example: Here the input logline: Apr 13 08:59:29 test.ch 2916048… --- ## [Restart Elastic Lab Training](https://discuss.elastic.co/t/restart-elastic-lab-training/269985)
**Author:** [@artsius](https://discuss.elastic.co/u/artsius)\ **Replies:** 0\ **Last updated:** [April 13, 2021, 9:36am UTC](https://discuss.elastic.co/t/restart-elastic-lab-training/269985 "2021-04-13T09:36:45Z")
May I know how to restart lab training on strigo.io? I have already do restart machine 3 times and nothing is changed. I want to do the lab from start again. Thanks --- ## [Corrélation log](https://discuss.elastic.co/t/correlation-log/269495)
**Author:** [@david\_oliver](https://discuss.elastic.co/u/david_oliver)\ **Replies:** 2\ **Last updated:** [April 13, 2021, 9:35am UTC](https://discuss.elastic.co/t/correlation-log/269495 "2021-04-13T09:35:49Z")
Bonjour, Je cherche des piste sur comment faire matcher des valeur de diffèrent index. le cas d'usage : un index avec des logs apache par exemple un index avec une liste ip si une ip est dans les log apache le faire… --- ## [How to change field type of filebeat modules?](https://discuss.elastic.co/t/how-to-change-field-type-of-filebeat-modules/269966)
**Author:** [@new2\_elk](https://discuss.elastic.co/u/new2_elk)\ **Replies:** 4\ **Last updated:** [April 13, 2021, 9:33am UTC](https://discuss.elastic.co/t/how-to-change-field-type-of-filebeat-modules/269966 "2021-04-13T09:33:57Z")
Hi guys, I am using the panw module on filebeat to pass log to logstash then pass to Elasticsearch. Then I added 1 more extract field in the "/usr/share/filebeat/module/panw/panos/config/input.yml" but the default type… --- ## [Bulk insert with python](https://discuss.elastic.co/t/bulk-insert-with-python/269883)
**Author:** [@eliase](https://discuss.elastic.co/u/eliase)\ **Replies:** 1\ **Last updated:** [April 13, 2021, 9:08am UTC](https://discuss.elastic.co/t/bulk-insert-with-python/269883 "2021-04-13T09:08:15Z")
Hi everyone, just as a foreword: I'm new to elasticsearch (and this forum :slightly\_smiling\_face:) and still figuring out a lot. So, I successfully tried to use bulk insertion from a file with the following content: {… --- ## [How can we configure ELK Stack and Filebeat for production environment?](https://discuss.elastic.co/t/how-can-we-configure-elk-stack-and-filebeat-for-production-environment/269968)
**Author:** [@Husnain](https://discuss.elastic.co/u/Husnain)\ **Replies:** 3\ **Last updated:** [April 13, 2021, 8:45am UTC](https://discuss.elastic.co/t/how-can-we-configure-elk-stack-and-filebeat-for-production-environment/269968 "2021-04-13T08:45:00Z")
Hello.Currently I am running ELK Stack and Filebeat for development and testing purpose.After some time I want to run ELK Stack and Filebeat in production environment.How can I do this?What are the configurations to run … --- ## [Time series kibana](https://discuss.elastic.co/t/time-series-kibana/268805)
**Author:** [@Jose\_Manuel\_Tirado](https://discuss.elastic.co/u/Jose_Manuel_Tirado)\ **Replies:** 5\ **Last updated:** [April 13, 2021, 8:24am UTC](https://discuss.elastic.co/t/time-series-kibana/268805 "2021-04-13T08:24:46Z")
Hi!!: I want to compare a data set from the last 15 min with the total mean accumulated over time. How could i do this in kibana? I have tried as TSVB and Timelion and cannot get the desired result. Can anybody help … [Previous page](https://discuss.elastic.co/latest.md?page=1733) [Next page](https://discuss.elastic.co/latest.md?page=1735)