# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1472 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1473 --- ## [Filebeat modules not user friendly and missing real references to version and format requirements](https://discuss.elastic.co/t/filebeat-modules-not-user-friendly-and-missing-real-references-to-version-and-format-requirements/268810)
**Author:** [@Mischa\_Diehm](https://discuss.elastic.co/u/Mischa_Diehm)\ **Replies:** 1\ **Last updated:** [April 8, 2021, 3:10am UTC](https://discuss.elastic.co/t/filebeat-modules-not-user-friendly-and-missing-real-references-to-version-and-format-requirements/268810 "2021-04-08T03:10:41Z")
Hi, many of the filebeat modules have a hint that they were converted from RSA NetWitness log parser XML. After some search and trying to understand how to get to the logformat required by the module I must say that al… --- ## [Use Kafka input in filebeat modules](https://discuss.elastic.co/t/use-kafka-input-in-filebeat-modules/268693)
**Author:** [@LeonT123](https://discuss.elastic.co/u/LeonT123)\ **Replies:** 3\ **Last updated:** [April 8, 2021, 2:46am UTC](https://discuss.elastic.co/t/use-kafka-input-in-filebeat-modules/268693 "2021-04-08T02:46:14Z")
Hi, Is there anyway to use Kafka input instead of file or other types (.i.e. UDP SYSLOG) in a module? I mean consider the events are available as a Kafka topic (instead of a file). As I checked it seems only file and U… --- ## [Include reference to prebuilt rules for SIEM](https://discuss.elastic.co/t/include-reference-to-prebuilt-rules-for-siem/268700)
**Author:** [@EvanGertis](https://discuss.elastic.co/u/EvanGertis)\ **Replies:** 2\ **Last updated:** [April 8, 2021, 2:43am UTC](https://discuss.elastic.co/t/include-reference-to-prebuilt-rules-for-siem/268700 "2021-04-08T02:43:04Z")
I think it would be very valuable to include a directory that loads the prebuilt SIEM rules\_export.ndjson file in the filebeat configuration. This way if we need to change the default indicies we could modify the json as… --- ## [Unable to change the elastic user password](https://discuss.elastic.co/t/unable-to-change-the-elastic-user-password/269508)
**Author:** [@Harsha7071](https://discuss.elastic.co/u/Harsha7071)\ **Replies:** 2\ **Last updated:** [April 8, 2021, 1:54am UTC](https://discuss.elastic.co/t/unable-to-change-the-elastic-user-password/269508 "2021-04-08T01:54:40Z")
We used the below command to change the password however it shows an error as per the screen shot attached. /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive Elastic version- 7.9 --- ## [Filebeat 403 error to SQS](https://discuss.elastic.co/t/filebeat-403-error-to-sqs/269205)
**Author:** [@paano](https://discuss.elastic.co/u/paano)\ **Replies:** 3\ **Last updated:** [April 8, 2021, 12:16am UTC](https://discuss.elastic.co/t/filebeat-403-error-to-sqs/269205 "2021-04-08T00:16:20Z")
Following the document from elastic blog, S3 log to Elastic using filebeat and SQS, i keep getting error 2021-04-03T13:45:04.022Z ERROR \[input.aws-s3\] awss3/collector.go:101 SQS ReceiveMessageRequest failed: AccessDenie… --- ## [Can Logstash read multiple text files , store config data available in first lines and add it to all other lines](https://discuss.elastic.co/t/can-logstash-read-multiple-text-files-store-config-data-available-in-first-lines-and-add-it-to-all-other-lines/267830)
**Author:** [@Svett](https://discuss.elastic.co/u/Svett)\ **Replies:** 2\ **Last updated:** [April 8, 2021, 12:00am UTC](https://discuss.elastic.co/t/can-logstash-read-multiple-text-files-store-config-data-available-in-first-lines-and-add-it-to-all-other-lines/267830 "2021-04-08T00:00:44Z")
Hi, I have multiple log files that I want to load in Kibana. Each file has at the beginning few lines with configuration information, that has to be added to each log line from this file. These lines look like that: … --- ## [Reverse proxy + XPACK](https://discuss.elastic.co/t/reverse-proxy-xpack/267433)
**Author:** [@macaulaysouza](https://discuss.elastic.co/u/macaulaysouza)\ **Replies:** 2\ **Last updated:** [April 7, 2021, 11:36pm UTC](https://discuss.elastic.co/t/reverse-proxy-xpack/267433 "2021-04-07T23:36:34Z")
Hi, I'm trying to setup Nginx to do reverse proxy but I'm struggling to make the X-PACK authentication work. My infrastruture is composed by HTTPS load balancer + NGINX + Kibana. I'm currently accessing my Kibana by I… --- ## [Kubernetes - populate deployment name](https://discuss.elastic.co/t/kubernetes-populate-deployment-name/269308)
**Author:** [@Szymon\_Przepiora](https://discuss.elastic.co/u/Szymon_Przepiora)\ **Replies:** 0\ **Last updated:** [April 6, 2021, 9:06am UTC](https://discuss.elastic.co/t/kubernetes-populate-deployment-name/269308 "2021-04-06T09:06:46Z")
Hi, I'm trying to populate a field kubernetes.deployment.name. I'm getting only the higher level kubernetes.replicaset.name . There isn't any error in the log. My config: filebeat: enabled: true filebeatConfig: … --- ## [Elasticsearch 7.9.3 fail to start](https://discuss.elastic.co/t/elasticsearch-7-9-3-fail-to-start/269463)
**Author:** [@FOMihai](https://discuss.elastic.co/u/FOMihai)\ **Replies:** 1\ **Last updated:** [April 7, 2021, 10:07pm UTC](https://discuss.elastic.co/t/elasticsearch-7-9-3-fail-to-start/269463 "2021-04-07T22:07:26Z")
Hello, I'm trying to install Elasticsearch 7.9.3 but when I'm trying to start elasticsearch I get this error: \[2021-04-07T14:31:59,501\]\[ERROR\]\[o.e.b.Bootstrap \] \[node-inopvdn23i\] Exception java.lang.IllegalSta… --- ## [Elastic - Logstash: Copy data from one field to another field within same index - both the fields are nested](https://discuss.elastic.co/t/elastic-logstash-copy-data-from-one-field-to-another-field-within-same-index-both-the-fields-are-nested/269529)
**Author:** [@Siri2](https://discuss.elastic.co/u/Siri2)\ **Replies:** 0\ **Last updated:** [April 7, 2021, 9:49pm UTC](https://discuss.elastic.co/t/elastic-logstash-copy-data-from-one-field-to-another-field-within-same-index-both-the-fields-are-nested/269529 "2021-04-07T21:49:10Z")
Hi, I am trying to copy data from one field to another field within the same index using filter -\> mutate -\> copy I want to mention that both the fields are nested. When I run the script it doesn't show me if it copied… --- ## [Kibana saved objects](https://discuss.elastic.co/t/kibana-saved-objects/269418)
**Author:** [@Mohith\_Vegi](https://discuss.elastic.co/u/Mohith_Vegi)\ **Replies:** 3\ **Last updated:** [April 7, 2021, 9:06pm UTC](https://discuss.elastic.co/t/kibana-saved-objects/269418 "2021-04-07T21:06:12Z")
Hi, For the kibana saved objects API , when we run the curl command for exporting saved objects, where does the json file be stored. curl -X POST "localhost:5601/api/saved\_objects/\_export -H 'kbn-xsrf: true' -H 'Conten… --- ## [Facing issue with occasional slow queries on 5 node ES cluster](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228)
**Author:** [@Vikash](https://discuss.elastic.co/u/Vikash)\ **Replies:** 11\ **Last updated:** [April 7, 2021, 9:04pm UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228 "2021-04-07T21:04:57Z")
I have a 5 node ES cluster and it's performing well on an average but sometime 0.5% of the queries are taking 1 sec to respond. I have tried all the possible solutions/suggestions, not sure what is causing issue. Overvi… --- ## [Getting NodeClosedException while running the ES 6.8.14 through the systemctl start elasticsearch.service custom made for the zip installable ES](https://discuss.elastic.co/t/getting-nodeclosedexception-while-running-the-es-6-8-14-through-the-systemctl-start-elasticsearch-service-custom-made-for-the-zip-installable-es/269399)
**Author:** [@Amit\_raj\_khurdhara](https://discuss.elastic.co/u/Amit_raj_khurdhara)\ **Replies:** 5\ **Last updated:** [April 7, 2021, 9:01pm UTC](https://discuss.elastic.co/t/getting-nodeclosedexception-while-running-the-es-6-8-14-through-the-systemctl-start-elasticsearch-service-custom-made-for-the-zip-installable-es/269399 "2021-04-07T21:01:35Z")
\[2021-04-06T17:45:52,825\]\[WARN \]\[o.e.x.m.e.l.LocalExporter\] \[FN06jOJ\] unexpected error while indexing monitoring document org.elasticsearch.xpack.monitoring.exporter.ExportException: NodeClosedException\[node closed {FN0… --- ## [Remove duplicates based on the timestamp and another field](https://discuss.elastic.co/t/remove-duplicates-based-on-the-timestamp-and-another-field/269519)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [April 7, 2021, 8:05pm UTC](https://discuss.elastic.co/t/remove-duplicates-based-on-the-timestamp-and-another-field/269519 "2021-04-07T20:05:22Z")
What method do you recommend to remove duplicates in an index, based on @timestamp and a field? somthing like this: if @timestamp and interface\_name are equal, delete one of the document --- ## [\[ERROR\]\[logstash.agent\] "Expected one of \[ \\\\t\\\\r\\\\n\], \\"#\\", \\"input\\", \\"filter\\", \\"output\\" at line 1, column 1 (byte 1)"](https://discuss.elastic.co/t/error-logstash-agent-expected-one-of-t-r-n-input-filter-output-at-line-1-column-1-byte-1/269515)
**Author:** [@Eng.Lucy](https://discuss.elastic.co/u/Eng.Lucy)\ **Replies:** 1\ **Last updated:** [April 7, 2021, 8:04pm UTC](https://discuss.elastic.co/t/error-logstash-agent-expected-one-of-t-r-n-input-filter-output-at-line-1-column-1-byte-1/269515 "2021-04-07T20:04:35Z")
I run logstash in a Docker container and occurs a error: \`\`\[ERROR\]\[logstash.agent\] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :messa… --- ## [Anomaly detection job failing to pull any data in](https://discuss.elastic.co/t/anomaly-detection-job-failing-to-pull-any-data-in/268940)
**Author:** [@timothyhutz](https://discuss.elastic.co/u/timothyhutz)\ **Replies:** 3\ **Last updated:** [April 7, 2021, 7:38pm UTC](https://discuss.elastic.co/t/anomaly-detection-job-failing-to-pull-any-data-in/268940 "2021-04-07T19:38:19Z")
So our ingestion mechanism for ingesting data had stopped for an hour, now my machine learning jobs refuse to pull in any new documents. I have verified it can see new latest document dates but just doesn't pull them in … --- ## [System fields for CPU load](https://discuss.elastic.co/t/system-fields-for-cpu-load/269485)
**Author:** [@hohooss](https://discuss.elastic.co/u/hohooss)\ **Replies:** 2\ **Last updated:** [April 7, 2021, 7:29pm UTC](https://discuss.elastic.co/t/system-fields-for-cpu-load/269485 "2021-04-07T19:29:50Z")
I have the following linux server specs and get the CPU load from two different fields, as shown further below: # lscpu Architecture: x86\_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Littl… --- ## [Where data nodes stored in ES Kuberenetes Statefulset](https://discuss.elastic.co/t/where-data-nodes-stored-in-es-kuberenetes-statefulset/269516)
**Author:** [@MdRashid](https://discuss.elastic.co/u/MdRashid)\ **Replies:** 0\ **Last updated:** [April 7, 2021, 7:28pm UTC](https://discuss.elastic.co/t/where-data-nodes-stored-in-es-kuberenetes-statefulset/269516 "2021-04-07T19:28:27Z")
Hi All, I have deploy Elasticsearch statefulset in my ubuntu terminal after that i check pod of elasticsearch and found that there is not folder with name of /usr/share/elasticsearch/data in ES pod. When i check logs it… --- ## [Check if string is in field](https://discuss.elastic.co/t/check-if-string-is-in-field/269479)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 9\ **Last updated:** [April 7, 2021, 7:20pm UTC](https://discuss.elastic.co/t/check-if-string-is-in-field/269479 "2021-04-07T19:20:18Z")
Hi, I need some help checking if a string is contained within the value of a field. my json has a key "log.file.path.keyword":\["myfolder/mypath/mylog.log" I tried to parse it with filter{ if "mylog.log" in \[log.file… --- ## [Configure translog parameters and flush period](https://discuss.elastic.co/t/configure-translog-parameters-and-flush-period/269509)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 0\ **Last updated:** [April 7, 2021, 6:29pm UTC](https://discuss.elastic.co/t/configure-translog-parameters-and-flush-period/269509 "2021-04-07T18:29:32Z")
hey there, I'm trying to understand correctly how to configure translog parameters. Taking a look to official documentation I have a doubt regarding index.translog.sync\_interval I am moving in this scenario: index.tra… --- ## [Connecting Kibana with Runscope](https://discuss.elastic.co/t/connecting-kibana-with-runscope/269460)
**Author:** [@JaroslawNiec](https://discuss.elastic.co/u/JaroslawNiec)\ **Replies:** 2\ **Last updated:** [April 7, 2021, 5:37pm UTC](https://discuss.elastic.co/t/connecting-kibana-with-runscope/269460 "2021-04-07T17:37:27Z")
Hi, I would like to create dashboard in Kibana that will display status of our API tests in Runscope. Runscope has some REST API that could return test metrics in json format. Is this somehow possible to configure Kib… --- ## [Filebeat azure module error code 409 BlobAlreadyExists](https://discuss.elastic.co/t/filebeat-azure-module-error-code-409-blobalreadyexists/269251)
**Author:** [@sguerrero](https://discuss.elastic.co/u/sguerrero)\ **Replies:** 1\ **Last updated:** [April 7, 2021, 5:03pm UTC](https://discuss.elastic.co/t/filebeat-azure-module-error-code-409-blobalreadyexists/269251 "2021-04-07T17:03:38Z")
Hi, I'm getting this error from filebeat: Apr 05 15:14:43 SOMETHING filebeat\[410843\]: / \_\_\_\_/ \_\_\_\_\_ \_\_\_\_ / /\_/ / / /\_ \_\_/ /\_ \_\_\_\_\_ Apr 05 15:14:43 SOMETHING filebeat\[410843\]: / \_\_/ | | / / \_ \\/ \_\_ \\/ \_\_/ /\_/… --- ## [Custom match pattern based on log message type](https://discuss.elastic.co/t/custom-match-pattern-based-on-log-message-type/269423)
**Author:** [@sergle](https://discuss.elastic.co/u/sergle)\ **Replies:** 1\ **Last updated:** [April 7, 2021, 4:53pm UTC](https://discuss.elastic.co/t/custom-match-pattern-based-on-log-message-type/269423 "2021-04-07T16:53:46Z")
Hey there, Would very much appreciate some input on something I am trying to achieve. The setup: NLog -\> (UDP) -\> Logstash (ELK) The NLog is configured to send different message structures to Logstash, all in the for… --- ## [Rollover\_alias errors on logstash daily indices from client application (fluentd)](https://discuss.elastic.co/t/rollover-alias-errors-on-logstash-daily-indices-from-client-application-fluentd/267519)
**Author:** [@cotjoey](https://discuss.elastic.co/u/cotjoey)\ **Replies:** 3\ **Last updated:** [April 7, 2021, 4:51pm UTC](https://discuss.elastic.co/t/rollover-alias-errors-on-logstash-daily-indices-from-client-application-fluentd/267519 "2021-04-07T16:51:29Z")
This is the error I am getting on the indices that fluentd is sending directly to Elasticsearch: illegal\_argument\_exception: index.lifecycle.rollover\_alias \[logstash\] does not point to index \[logstash-2021.02.09\] The i… --- ## [How can i create different indices for different module](https://discuss.elastic.co/t/how-can-i-create-different-indices-for-different-module/269095)
**Author:** [@Shriram\_Wasule](https://discuss.elastic.co/u/Shriram_Wasule)\ **Replies:** 11\ **Last updated:** [April 7, 2021, 4:45pm UTC](https://discuss.elastic.co/t/how-can-i-create-different-indices-for-different-module/269095 "2021-04-07T16:45:03Z")
Hello, i have two modules activated Apache and tomcat,, i want to store logs of this two in two different indices for better result but i am not able to get how to do that. please help me to set this configuration in m… --- ## [Safely remove tcp input plugin's "host" and "port" fields](https://discuss.elastic.co/t/safely-remove-tcp-input-plugins-host-and-port-fields/269492)
**Author:** [@Supermathie](https://discuss.elastic.co/u/Supermathie)\ **Replies:** 3\ **Last updated:** [April 7, 2021, 4:44pm UTC](https://discuss.elastic.co/t/safely-remove-tcp-input-plugins-host-and-port-fields/269492 "2021-04-07T16:44:56Z")
The tcp input plugin adds host and port fields to received events. Ideally we don't want to add these but since this isn't documented, there doesn't seem to be a way to remove them. What is the best filter to safely re… --- ## [Filebeat won't get geo\_points field type](https://discuss.elastic.co/t/filebeat-wont-get-geo-points-field-type/269501)
**Author:** [@Chris6](https://discuss.elastic.co/u/Chris6)\ **Replies:** 0\ **Last updated:** [April 7, 2021, 4:40pm UTC](https://discuss.elastic.co/t/filebeat-wont-get-geo-points-field-type/269501 "2021-04-07T16:40:39Z")
I have a json line of input that foes through filebeat. The line is in this format: {"first": "text", "second": "text", "@timestamp": "2021-02-25T08:54:04Z", "location": \[-71.34, 41.12\] } I am trying to get the location… --- ## [Troubles with German Umlaute ä ö ü ß](https://discuss.elastic.co/t/troubles-with-german-umlaute-a-o-u-ss/269428)
**Author:** [@feva](https://discuss.elastic.co/u/feva)\ **Replies:** 1\ **Last updated:** [April 7, 2021, 4:26pm UTC](https://discuss.elastic.co/t/troubles-with-german-umlaute-a-o-u-ss/269428 "2021-04-07T16:26:40Z")
I have problems with German Umlaute ä ü ö ß. I send a json logmessage from IBMi (AS400) to logstash encoded in ISO8859-1. It is not possible to send them in UTF-8. So I tried to set input { http { host =\> "0.0.0.… --- ## [Filebeat autodiscovery in Kubernetes/EKS and multiple outputs](https://discuss.elastic.co/t/filebeat-autodiscovery-in-kubernetes-eks-and-multiple-outputs/268272)
**Author:** [@bplies](https://discuss.elastic.co/u/bplies)\ **Replies:** 1\ **Last updated:** [April 7, 2021, 4:02pm UTC](https://discuss.elastic.co/t/filebeat-autodiscovery-in-kubernetes-eks-and-multiple-outputs/268272 "2021-04-07T16:02:26Z")
Scenario A shared K8s cluster with several different apps all needing their app-specific logs collected by Filebeat. Each different app's logs are substantially different (some plaintext, some json with differing field… --- ## [Download CSV from dashboard panel](https://discuss.elastic.co/t/download-csv-from-dashboard-panel/268638)
**Author:** [@AlessandroKP](https://discuss.elastic.co/u/AlessandroKP)\ **Replies:** 2\ **Last updated:** [April 7, 2021, 3:59pm UTC](https://discuss.elastic.co/t/download-csv-from-dashboard-panel/268638 "2021-04-07T15:59:49Z")
Hi, I'm having trouble exporting saved searches from a Kibana dashboard in CSV format through the Download CSV tool. The download works as usual when I set the time interval (through the time picker of the dashboard) t… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1471) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1473)