# Latest **URL:** https://discuss.elastic.co/latest.md?page=1548 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1549 --- ## [Filebeat AWS Module No found](https://discuss.elastic.co/t/filebeat-aws-module-no-found/280119)
**Author:** [@mibfb23](https://discuss.elastic.co/u/mibfb23)\ **Replies:** 5\ **Last updated:** [July 31, 2021, 11:49pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-no-found/280119 "2021-07-31T23:49:02Z")
Hello, having the need to store the cloudtrail logs in elasticsearch, I used the appropriate filebeat module providing it with all the options in this way: - module: aws cloudtrail: enabled: true # AWS SQS q… --- ## [Can't upgrade beats / kibana after enabling security](https://discuss.elastic.co/t/cant-upgrade-beats-kibana-after-enabling-security/279547)
**Author:** [@DPattee](https://discuss.elastic.co/u/DPattee)\ **Replies:** 3\ **Last updated:** [July 31, 2021, 8:04pm UTC](https://discuss.elastic.co/t/cant-upgrade-beats-kibana-after-enabling-security/279547 "2021-07-31T20:04:25Z")
Prior to enabling security on elasticsearch cluster, I was able to easily upgrade beats by just upgrading the beat binary and letting it run. Since my beat config files (eg filebeat-config.yml) were set with setup.temp… --- ## [Best strategy to deal with one event that contain multiple events, and graph it on kibana](https://discuss.elastic.co/t/best-strategy-to-deal-with-one-event-that-contain-multiple-events-and-graph-it-on-kibana/280137)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 0\ **Last updated:** [July 31, 2021, 7:42pm UTC](https://discuss.elastic.co/t/best-strategy-to-deal-with-one-event-that-contain-multiple-events-and-graph-it-on-kibana/280137 "2021-07-31T19:42:38Z")
Hi, after parsing a string field . it, I got several fields with different names, this names will change constantly and probably will appear new ones { "mfemve"=\>"0,39 ", "vmtoolsd"=\>"0,39 ", "zabbix\_agentd"=\>"0,32 ", … --- ## [Beats for solaris 10](https://discuss.elastic.co/t/beats-for-solaris-10/280037)
**Author:** [@alipujaistopo](https://discuss.elastic.co/u/alipujaistopo)\ **Replies:** 1\ **Last updated:** [July 31, 2021, 5:43pm UTC](https://discuss.elastic.co/t/beats-for-solaris-10/280037 "2021-07-31T17:43:48Z")
is there any beats who compatible with solaris 10? --- ## [Multiple logs parsing to elastic search directly through filebeat](https://discuss.elastic.co/t/multiple-logs-parsing-to-elastic-search-directly-through-filebeat/280134)
**Author:** [@maduranga\_nessus](https://discuss.elastic.co/u/maduranga_nessus)\ **Replies:** 0\ **Last updated:** [July 31, 2021, 5:24pm UTC](https://discuss.elastic.co/t/multiple-logs-parsing-to-elastic-search-directly-through-filebeat/280134 "2021-07-31T17:24:23Z")
I need to parse multiple log files to an elastic search through a file beat. File beat already installed in this pc and through that file beat nginx already parsing directly to elastic search. I need to parse apache logs… --- ## [Is it possible to monitor elasticsearch using metricbeat and custom index name?](https://discuss.elastic.co/t/is-it-possible-to-monitor-elasticsearch-using-metricbeat-and-custom-index-name/280084)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 3\ **Last updated:** [July 31, 2021, 4:44pm UTC](https://discuss.elastic.co/t/is-it-possible-to-monitor-elasticsearch-using-metricbeat-and-custom-index-name/280084 "2021-07-31T16:44:40Z")
Hello ! I use metricbeat on the whole stack members (Elasticsearch/Logstash/Kibana) to collect system metrics (enabled by default) I use logstash output and custom index name metricbeat.yml : # ----------------------… --- ## [Reindex by grouping documents by Unique ID](https://discuss.elastic.co/t/reindex-by-grouping-documents-by-unique-id/280129)
**Author:** [@mrunalini](https://discuss.elastic.co/u/mrunalini)\ **Replies:** 0\ **Last updated:** [July 31, 2021, 3:15pm UTC](https://discuss.elastic.co/t/reindex-by-grouping-documents-by-unique-id/280129 "2021-07-31T15:15:35Z")
Hi Everyone, I want to reindex my Index which has different documents with unique id . I want to reindex all the data into single documents against unique id. is it possible to do that. --- ## [MySQL 8: Could not create connection to database server](https://discuss.elastic.co/t/mysql-8-could-not-create-connection-to-database-server/280064)
**Author:** [@Justin\_Morgan](https://discuss.elastic.co/u/Justin_Morgan)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 11:57am UTC](https://discuss.elastic.co/t/mysql-8-could-not-create-connection-to-database-server/280064 "2021-07-30T11:57:29Z")
After upgrading to MySQL 8 I get the following error from my logstash pod: Loading class \`com.mysql.jdbc.Driver'. This is deprecated. The new driver class is \`com.mysql.cj.jdbc.Driver'. The driver is automatically regis… --- ## [Fluentd performance is degraded](https://discuss.elastic.co/t/fluentd-performance-is-degraded/280060)
**Author:** [@Voula\_Mikr](https://discuss.elastic.co/u/Voula_Mikr)\ **Replies:** 3\ **Last updated:** [July 31, 2021, 6:05am UTC](https://discuss.elastic.co/t/fluentd-performance-is-degraded/280060 "2021-07-31T06:05:31Z")
In a deployment with elasticsearch 8.x, fluentd pod performance has been degraded and following log is printed whenever a new message is received and it is about to be delivered in elastic. warning: 299 Elasticsearch-7.… --- ## [Pre-index data in ingest pipeline](https://discuss.elastic.co/t/pre-index-data-in-ingest-pipeline/280104)
**Author:** [@jjammin92](https://discuss.elastic.co/u/jjammin92)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 10:31pm UTC](https://discuss.elastic.co/t/pre-index-data-in-ingest-pipeline/280104 "2021-07-30T22:31:44Z")
Good afternoon, I have some access style logs with a numeric field "player.score." I'm looking to pre-index these fields to leverage patterns for faster retrieval The values can range anywhere from 0 to 5000 being the… --- ## [Ruby hash to fields of an event](https://discuss.elastic.co/t/ruby-hash-to-fields-of-an-event/280102)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 10:03pm UTC](https://discuss.elastic.co/t/ruby-hash-to-fields-of-an-event/280102 "2021-07-30T22:03:53Z")
Hi I got the following ruby hash after parsing a long string with ruby code, { "mfemve"=\>"0,39 ", "vmtoolsd"=\>"0,39 ", "zabbix\_agentd"=\>"0,39 ", "powershell"=\>"0,39 ", "ir\_agent"=\>"0,00 ", "wmiprvse"=\>"0,00 ", "unsecap… --- ## [Is @onMethodEnter and @onMethodExit syncronized?](https://discuss.elastic.co/t/is-onmethodenter-and-onmethodexit-syncronized/280016)
**Author:** [@milanvdm](https://discuss.elastic.co/u/milanvdm)\ **Replies:** 2\ **Last updated:** [July 30, 2021, 7:37am UTC](https://discuss.elastic.co/t/is-onmethodenter-and-onmethodexit-syncronized/280016 "2021-07-30T07:37:32Z")
For example, I instrument a method foo which is implemented in a certain way so that only 1 thread at the time will call it. If I instrument foo with onMethodEnter and onMethodExit, will onMethodExit always finish befor… --- ## [AWSFargate module for Filebeat not working?](https://discuss.elastic.co/t/awsfargate-module-for-filebeat-not-working/279949)
**Author:** [@stec00](https://discuss.elastic.co/u/stec00)\ **Replies:** 4\ **Last updated:** [July 30, 2021, 9:19pm UTC](https://discuss.elastic.co/t/awsfargate-module-for-filebeat-not-working/279949 "2021-07-30T21:19:15Z")
I am attempting to use the AWSFargate module for Filebeat. Ran the following commands (with placeholder values substituted): ./filebeat.exe modules enable awsfargate ./filebeat -e -E cloud.id=myElasticCloudID -E cloud… --- ## [How to compare exported objects in .ndjson format from two envirornments](https://discuss.elastic.co/t/how-to-compare-exported-objects-in-ndjson-format-from-two-envirornments/279846)
**Author:** [@Kunal\_Sharma](https://discuss.elastic.co/u/Kunal_Sharma)\ **Replies:** 2\ **Last updated:** [July 30, 2021, 8:19pm UTC](https://discuss.elastic.co/t/how-to-compare-exported-objects-in-ndjson-format-from-two-envirornments/279846 "2021-07-30T20:19:11Z")
Dear Team, We have multiple Kibana stacks (dev, test, prod) which are manually managed and have some differences. We are looking for some quick help on comparing the Kibana objects from two environments and finding the… --- ## [Output to json saving only specific fields](https://discuss.elastic.co/t/output-to-json-saving-only-specific-fields/280098)
**Author:** [@Hamza\_El\_Aouane](https://discuss.elastic.co/u/Hamza_El_Aouane)\ **Replies:** 3\ **Last updated:** [July 30, 2021, 7:57pm UTC](https://discuss.elastic.co/t/output-to-json-saving-only-specific-fields/280098 "2021-07-30T19:57:42Z")
Hello everyone I am new logstash and I am sorry if the question is so basic. I have been saving my syslog into a csv file, as follow: csv { path =\> "C:\\path\\Desktop\\Adib\\adib-syslogs-%{+yyyy.MM.dd}.csv" csv\_o… --- ## [Logstash mutate field](https://discuss.elastic.co/t/logstash-mutate-field/280097)
**Author:** [@Hamza\_El\_Aouane](https://discuss.elastic.co/u/Hamza_El_Aouane)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 7:56pm UTC](https://discuss.elastic.co/t/logstash-mutate-field/280097 "2021-07-30T19:56:27Z")
Hi everyone, I hope somebody can help me with this issue I am facing with logstash. I am receiving some syslog, and trying to put in place some filtering and merging together 2 fields. the simple structure that I want… --- ## [Elasticsearch not doing local primary indices recovery after node restart](https://discuss.elastic.co/t/elasticsearch-not-doing-local-primary-indices-recovery-after-node-restart/280101)
**Author:** [@pranav\_sah](https://discuss.elastic.co/u/pranav_sah)\ **Replies:** 0\ **Last updated:** [July 30, 2021, 7:45pm UTC](https://discuss.elastic.co/t/elasticsearch-not-doing-local-primary-indices-recovery-after-node-restart/280101 "2021-07-30T19:45:41Z")
We have a cluster of 6 nodes of which one node was restarted due to some patch activity. Before the node was brought down we had set "cluster.routing.allocation.enable" : "primaries" on persistent level so as to have f… --- ## [How to prevent clobbering in translate filter](https://discuss.elastic.co/t/how-to-prevent-clobbering-in-translate-filter/280008)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 4\ **Last updated:** [July 30, 2021, 4:50pm UTC](https://discuss.elastic.co/t/how-to-prevent-clobbering-in-translate-filter/280008 "2021-07-30T16:50:51Z")
Hello , I'm using translate filter with a set of mappings in dictionary . { "@timestamp" =\> 2021-07-23T11:52:08.000Z, "Flag" =\> "16464", "Flag\_Definition" =\> "liveness test timed out… --- ## [Kibana discover tab is lagging](https://discuss.elastic.co/t/kibana-discover-tab-is-lagging/277902)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 17\ **Last updated:** [July 30, 2021, 3:51pm UTC](https://discuss.elastic.co/t/kibana-discover-tab-is-lagging/277902 "2021-07-30T15:51:53Z")
Running elk stack 7.11 Issue is while searching 2 months data in kibana's discover it is taking more than 5 minutes to show the result the index size is 100gb we did top command and we didn't see any serious thing cpu i… --- ## [JDBC last run is updated, but logstash always start from the beggining and duplicate documents](https://discuss.elastic.co/t/jdbc-last-run-is-updated-but-logstash-always-start-from-the-beggining-and-duplicate-documents/280091)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 0\ **Last updated:** [July 30, 2021, 3:11pm UTC](https://discuss.elastic.co/t/jdbc-last-run-is-updated-but-logstash-always-start-from-the-beggining-and-duplicate-documents/280091 "2021-07-30T15:11:53Z")
Hi, I have a pipeline with JDBC input with schedule of 5 min, I ran logstash as a service sudo systemctl start logstash and the JDBC last run is updated, but when logstash start the last run is not used an duplicate do… --- ## [Extract selected properties from a nested JSON with logstash?](https://discuss.elastic.co/t/extract-selected-properties-from-a-nested-json-with-logstash/280090)
**Author:** [@Samurai1](https://discuss.elastic.co/u/Samurai1)\ **Replies:** 0\ **Last updated:** [July 30, 2021, 2:53pm UTC](https://discuss.elastic.co/t/extract-selected-properties-from-a-nested-json-with-logstash/280090 "2021-07-30T14:53:53Z")
{ "billToLocation": { "prop1": "iwant", "prop2": { "prop2prop1": "iwant", "prop2prop2": { }, "prop2prop3:"iDontWant }, … --- ## [Match any character regular expression does not match as expected](https://discuss.elastic.co/t/match-any-character-regular-expression-does-not-match-as-expected/278918)
**Author:** [@alexleen](https://discuss.elastic.co/u/alexleen)\ **Replies:** 6\ **Last updated:** [July 30, 2021, 2:31pm UTC](https://discuss.elastic.co/t/match-any-character-regular-expression-does-not-match-as-expected/278918 "2021-07-30T14:31:45Z")
POST /search\_test/\_doc { "text":"Jack and Jill went up the hill." } GET /search\_test/\_search?q=j%2A GET /search\_test/\_search?q=j... I would expect those two queries to return the same document since both of those regu… --- ## [Want to add multiple log files as a input in filebeat.yml and send to logstash](https://discuss.elastic.co/t/want-to-add-multiple-log-files-as-a-input-in-filebeat-yml-and-send-to-logstash/280087)
**Author:** [@Aboli\_77](https://discuss.elastic.co/u/Aboli_77)\ **Replies:** 0\ **Last updated:** [July 30, 2021, 2:23pm UTC](https://discuss.elastic.co/t/want-to-add-multiple-log-files-as-a-input-in-filebeat-yml-and-send-to-logstash/280087 "2021-07-30T14:23:38Z")
Hi All, My filebeat.yml file input is: filebeat.inputs: type: log enabled: true paths: /home/XXXX/XYZ/abc.send.2\* /home/XXXX/XYZ/abc\_xyz.target\* /home/XXXX/XYZ/abc.sender.6\* /home/XXXX/XYZ/abc.log\* fields: log… --- ## [ECE Setup - Failed to detect that a master was elected](https://discuss.elastic.co/t/ece-setup-failed-to-detect-that-a-master-was-elected/280041)
**Author:** [@Kishore](https://discuss.elastic.co/u/Kishore)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 2:10pm UTC](https://discuss.elastic.co/t/ece-setup-failed-to-detect-that-a-master-was-elected/280041 "2021-07-30T14:10:47Z")
Hi Elastic Team, We have setup ECE, trying to move default clusters to Allocators. We are getting following error while moving the cluster from primary node. Plan change failed: \[ClusterFailure:MasterNotDetectedInTime… --- ## [Grok fails to write events on a file](https://discuss.elastic.co/t/grok-fails-to-write-events-on-a-file/280081)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 2\ **Last updated:** [July 30, 2021, 2:04pm UTC](https://discuss.elastic.co/t/grok-fails-to-write-events-on-a-file/280081 "2021-07-30T14:04:46Z")
I have the following test configuration. Basically I want to write the messages in either the fail or the notfail files depending on whether the grok filter would succeed. I've created the files proactively under a folde… --- ## [ML: difference between partition\_field\_name and by\_field\_name?](https://discuss.elastic.co/t/ml-difference-between-partition-field-name-and-by-field-name/280013)
**Author:** [@CameronCenic](https://discuss.elastic.co/u/CameronCenic)\ **Replies:** 3\ **Last updated:** [July 30, 2021, 2:01pm UTC](https://discuss.elastic.co/t/ml-difference-between-partition-field-name-and-by-field-name/280013 "2021-07-30T14:01:24Z")
I do not really understand the difference between these two settings. They seem to perform the same function from my perspective. --- ## [Logstash filter take everything after](https://discuss.elastic.co/t/logstash-filter-take-everything-after/280061)
**Author:** [@Betorov](https://discuss.elastic.co/u/Betorov)\ **Replies:** 3\ **Last updated:** [July 30, 2021, 9:56am UTC](https://discuss.elastic.co/t/logstash-filter-take-everything-after/280061 "2021-07-30T09:56:21Z")
Hi avrey one, I wanted to filter this type of message: ip ip text I don't know the format of the text and what is inside of it.I only know that there is 2 ip and the text. How i can geto something that I don't know i… --- ## [Centralization Logstash](https://discuss.elastic.co/t/centralization-logstash/279938)
**Author:** [@Ilya\_21bA](https://discuss.elastic.co/u/Ilya_21bA)\ **Replies:** 3\ **Last updated:** [July 30, 2021, 1:10pm UTC](https://discuss.elastic.co/t/centralization-logstash/279938 "2021-07-30T13:10:47Z")
Hi, tell me, is it necessary to install the entire ELK stack on a new virtual machine, if I want to collect logs from several machines, tell me how it can be done, there are instructions. --- ## [Shard allocation on single node causes cluster overload](https://discuss.elastic.co/t/shard-allocation-on-single-node-causes-cluster-overload/279794)
**Author:** [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\ **Replies:** 3\ **Last updated:** [July 30, 2021, 1:06pm UTC](https://discuss.elastic.co/t/shard-allocation-on-single-node-causes-cluster-overload/279794 "2021-07-30T13:06:12Z")
Cluster creates index with 5 shards on same node. And it picks the node with least disk space. High performance ETL process writes all the data on to single node instead of to 5 multiple nodes. This causes overloads of … --- ## [Working with List Fields in Kibana](https://discuss.elastic.co/t/working-with-list-fields-in-kibana/279860)
**Author:** [@Sandeep\_Raju](https://discuss.elastic.co/u/Sandeep_Raju)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 11:34am UTC](https://discuss.elastic.co/t/working-with-list-fields-in-kibana/279860 "2021-07-30T11:34:13Z")
Hi all, I'm having difficulty in adding list fields in my project. The fields are already defined & cannot be changed. For ex: Groups: QA, audit, domain, company, admin, ip These are values given for a field in a s… [Previous page](https://discuss.elastic.co/latest.md?page=1547) [Next page](https://discuss.elastic.co/latest.md?page=1549)