# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=322 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 323 --- ## [How to parse Proxmox log format?](https://discuss.elastic.co/t/how-to-parse-proxmox-log-format/243089)
**Author:** [@gmbynkrkrqz](https://discuss.elastic.co/u/gmbynkrkrqz)\ **Replies:** 2\ **Last updated:** [July 30, 2020, 12:04pm UTC](https://discuss.elastic.co/t/how-to-parse-proxmox-log-format/243089 "2020-07-30T12:04:50Z")
Hello, I would like to store all my Proxmox logs in Elasticsearch. I know there are Groks and I already use its for some other logs. However, as you can see below, the log format is often different : the fields are not … --- ## [Field is not renaming](https://discuss.elastic.co/t/field-is-not-renaming/243026)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 5\ **Last updated:** [July 29, 2020, 12:19pm UTC](https://discuss.elastic.co/t/field-is-not-renaming/243026 "2020-07-29T12:19:43Z")
I am using winlogbeat and i want to create an index which stores user sessions info. but i am doing this step by step that is i am storing successfull login of user. Everything is going good but now i am facing some prob… --- ## [Logstash Filebeat module Configuration - No data has been received from this module yet](https://discuss.elastic.co/t/logstash-filebeat-module-configuration-no-data-has-been-received-from-this-module-yet/242990)
**Author:** [@GraceYu](https://discuss.elastic.co/u/GraceYu)\ **Replies:** 11\ **Last updated:** [July 30, 2020, 8:44am UTC](https://discuss.elastic.co/t/logstash-filebeat-module-configuration-no-data-has-been-received-from-this-module-yet/242990 "2020-07-30T08:44:54Z")
Hi, I have been trying to configure Logstash Filebeat module. However I received message "No data has been received from this module yet". I have installed Logstash. The following log from console: Sending Logstash… --- ## [Split a JSON array into individual documents in Logstash](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151)
**Author:** [@jclemons7](https://discuss.elastic.co/u/jclemons7)\ **Replies:** 7\ **Last updated:** [July 30, 2020, 6:29am UTC](https://discuss.elastic.co/t/split-a-json-array-into-individual-documents-in-logstash/243151 "2020-07-30T06:29:08Z")
I'm sorry to ask this honestly.. normally I'm a decently smart fella.. but I can't seem to grasp this. I have some JSON coming that I need to split into individual documents. { "lan": { "inkbps": 756.27034713779… --- ## [Best method in fingerprint](https://discuss.elastic.co/t/best-method-in-fingerprint/243055)
**Author:** [@Saravana37](https://discuss.elastic.co/u/Saravana37)\ **Replies:** 3\ **Last updated:** [July 30, 2020, 4:32am UTC](https://discuss.elastic.co/t/best-method-in-fingerprint/243055 "2020-07-30T04:32:24Z")
Hello All , Can any one please let me know which is the best method used in Fingerprint filter ? I am beginner in fingerprint filter and not able to figure-out . Thanks and Regards Saravana S --- ## [Using credentials manager to store Logstash's keystore password](https://discuss.elastic.co/t/using-credentials-manager-to-store-logstashs-keystore-password/243131)
**Author:** [@GusAntoniassi](https://discuss.elastic.co/u/GusAntoniassi)\ **Replies:** 0\ **Last updated:** [July 29, 2020, 9:43pm UTC](https://discuss.elastic.co/t/using-credentials-manager-to-store-logstashs-keystore-password/243131 "2020-07-29T21:43:49Z")
Hello! I am currently using Logstash 7.8.0, running as a service and installed via package manager (yum) on an Amazon Linux OS. I am the Logstash Keystore to store some sensitive credentials used by my pipelines. I man… --- ## [How to set HTTP filter body to whole event](https://discuss.elastic.co/t/how-to-set-http-filter-body-to-whole-event/243041)
**Author:** [@bradley.king](https://discuss.elastic.co/u/bradley.king)\ **Replies:** 2\ **Last updated:** [July 29, 2020, 9:16pm UTC](https://discuss.elastic.co/t/how-to-set-http-filter-body-to-whole-event/243041 "2020-07-29T21:16:07Z")
Hi guys, just a quick question. I can't seem to find the answer on the docs :confused: How do I reference the whole event in logstash? I want to use the HTTP filter to send the whole event to one of our APIs for some q… --- ## [Kibana Index time field does not match @timestamp field](https://discuss.elastic.co/t/kibana-index-time-field-does-not-match-timestamp-field/242922)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 2\ **Last updated:** [July 29, 2020, 8:20pm UTC](https://discuss.elastic.co/t/kibana-index-time-field-does-not-match-timestamp-field/242922 "2020-07-29T20:20:39Z")
Hello, Within my Kibana, I have an index showing @timestamp and another called "time". The first issue is the field @timestamp does not match the time in Europe/London. The second is issue is that the time field in lo… --- ## [Is there a way to match Timestamps with the time in the message of the log](https://discuss.elastic.co/t/is-there-a-way-to-match-timestamps-with-the-time-in-the-message-of-the-log/242906)
**Author:** [@Archie\_Crawford](https://discuss.elastic.co/u/Archie_Crawford)\ **Replies:** 9\ **Last updated:** [July 29, 2020, 7:32pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-match-timestamps-with-the-time-in-the-message-of-the-log/242906 "2020-07-29T19:32:29Z")
I have tried to use things like : Filter{ grok { match =\> \["message", "%{TIMESTAMP\_ISO8601:timestamp}"\] } date { match =\> \["timestamp", "ISO8601"\] } } I have been trying to do this for java stack trace logs ... … --- ## [Creating a sub field of GROK filter pattern](https://discuss.elastic.co/t/creating-a-sub-field-of-grok-filter-pattern/238860)
**Author:** [@hemant\_472](https://discuss.elastic.co/u/hemant_472)\ **Replies:** 4\ **Last updated:** [July 29, 2020, 6:52pm UTC](https://discuss.elastic.co/t/creating-a-sub-field-of-grok-filter-pattern/238860 "2020-07-29T18:52:00Z")
Hi Everyone, i have a logstash configuration to parse my current logs but i would like to create a sub field from my message field and that data should be parsed as a fieldname, please suggest how that is possible: For… --- ## [Encrypt filebeat to logstash](https://discuss.elastic.co/t/encrypt-filebeat-to-logstash/242944)
**Author:** [@elk6](https://discuss.elastic.co/u/elk6)\ **Replies:** 3\ **Last updated:** [July 29, 2020, 5:04pm UTC](https://discuss.elastic.co/t/encrypt-filebeat-to-logstash/242944 "2020-07-29T17:04:37Z")
I have a small setup. Filebeat was able to successfully send logs to logstash. I was then asked to encrypt communication between filebeat to logstash. On logstash I've added to the following to the input{ } block: ssl\_… --- ## [Cidr Plugin - Warn and skipping data... Missing Data](https://discuss.elastic.co/t/cidr-plugin-warn-and-skipping-data-missing-data/242875)
**Author:** [@Apollyon](https://discuss.elastic.co/u/Apollyon)\ **Replies:** 3\ **Last updated:** [July 29, 2020, 3:57pm UTC](https://discuss.elastic.co/t/cidr-plugin-warn-and-skipping-data-missing-data/242875 "2020-07-29T15:57:57Z")
Hello, I've been playing around with the logstash cidr filter plugin and have been coming into a couple of issues. For the most part, with most of the data sets that are coming in, this filter is working well and is doi… --- ## [Logstash filter plugin (Grok) for custom pattern Not match curly bracket!](https://discuss.elastic.co/t/logstash-filter-plugin-grok-for-custom-pattern-not-match-curly-bracket/243053)
**Author:** [@AliMaleki21](https://discuss.elastic.co/u/AliMaleki21)\ **Replies:** 17\ **Last updated:** [July 29, 2020, 3:42pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-grok-for-custom-pattern-not-match-curly-bracket/243053 "2020-07-29T15:42:57Z")
i have problem with grok plugin, curly bracket not match on Grok filter plugin sample log: {Allowed/Denied} .... regex: ACTION\_FIREWALL (Allowed|Denied) Pattern: {%{ACTION\_FIREWALL:action\_firewall}} when try this … --- ## [Can't get encryption to Logstash working](https://discuss.elastic.co/t/cant-get-encryption-to-logstash-working/243078)
**Author:** [@elk6](https://discuss.elastic.co/u/elk6)\ **Replies:** 0\ **Last updated:** [July 29, 2020, 2:44pm UTC](https://discuss.elastic.co/t/cant-get-encryption-to-logstash-working/243078 "2020-07-29T14:44:23Z")
I've created elasticsearch users to secure communications between the stack (all on one machine). Now I need to encrypt communications to logstash from remote machines. I'm redoing my step because I just don't know wha… --- ## [Is anyone using logstash-output-datadog\_metrics with logstash 7.8.0?](https://discuss.elastic.co/t/is-anyone-using-logstash-output-datadog-metrics-with-logstash-7-8-0/243064)
**Author:** [@windowsrefund\_refund](https://discuss.elastic.co/u/windowsrefund_refund)\ **Replies:** 0\ **Last updated:** [July 29, 2020, 1:43pm UTC](https://discuss.elastic.co/t/is-anyone-using-logstash-output-datadog-metrics-with-logstash-7-8-0/243064 "2020-07-29T13:43:18Z")
Full description of the issue I'm seeing: https://github.com/logstash-plugins/logstash-output-datadog\_metrics/issues/16 --- ## [Ingesting Oracle Cloud Infrastructure Logs from It's Object Storage](https://discuss.elastic.co/t/ingesting-oracle-cloud-infrastructure-logs-from-its-object-storage/243046)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 0\ **Last updated:** [July 29, 2020, 11:59am UTC](https://discuss.elastic.co/t/ingesting-oracle-cloud-infrastructure-logs-from-its-object-storage/243046 "2020-07-29T11:59:08Z")
Hello All, I recently raised a Topic: https://discuss.elastic.co/t/oracle-cloud-infrastructure-logstash-plugin/239813. This is regarding if anyone has come across a way of ingesting logs from OCI object storage. Simil… --- ## [How to use jdbc to import data into nested objects?](https://discuss.elastic.co/t/how-to-use-jdbc-to-import-data-into-nested-objects/52871)
**Author:** [@DaanCreemers](https://discuss.elastic.co/u/DaanCreemers)\ **Replies:** 7\ **Last updated:** [July 29, 2020, 10:59am UTC](https://discuss.elastic.co/t/how-to-use-jdbc-to-import-data-into-nested-objects/52871 "2020-07-29T10:59:17Z")
Hi everybody, I have multiple tables in my database which I would like to import into Elasticsearch using a single document type. One of the tables has a one-to-many relation to another table. I will use the example… --- ## [Logstash Pipeline Monitoring Data Is Intermittently Blank In Kibana](https://discuss.elastic.co/t/logstash-pipeline-monitoring-data-is-intermittently-blank-in-kibana/243008)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 2\ **Last updated:** [July 29, 2020, 10:54am UTC](https://discuss.elastic.co/t/logstash-pipeline-monitoring-data-is-intermittently-blank-in-kibana/243008 "2020-07-29T10:54:15Z")
In my setup I have 3 Logstash(7.5.0) nodes, each node has 20 pipelines. I'm trying to troubleshoot some performance issues with one of my pipelines, but find it difficult because the data is not always viewable in Kiban… --- ## [Unable to read parse log entries in Logstash](https://discuss.elastic.co/t/unable-to-read-parse-log-entries-in-logstash/243007)
**Author:** [@Dark](https://discuss.elastic.co/u/Dark)\ **Replies:** 1\ **Last updated:** [July 29, 2020, 7:57am UTC](https://discuss.elastic.co/t/unable-to-read-parse-log-entries-in-logstash/243007 "2020-07-29T07:57:59Z")
I have filebeat pushing data to logstash. Logstash is only able to parse the first entry pushed by beats and the rest are not getting filtered. This is the record present in log file: 2020-07-29 08:35:18,042 INFO \[gov… --- ## [Logstash filter conditionals](https://discuss.elastic.co/t/logstash-filter-conditionals/243005)
**Author:** [@smm](https://discuss.elastic.co/u/smm)\ **Replies:** 2\ **Last updated:** [July 29, 2020, 8:05am UTC](https://discuss.elastic.co/t/logstash-filter-conditionals/243005 "2020-07-29T08:05:12Z")
Hi there, I have 2 simple questions reagarding this logstash conditionals example. I have found something like this: ... if \[fields\]\[some\_id\] { mutate { add\_field =\> {"some\_id" =\> "%{\[fields\]\[some\_id\]}"} } } ... … --- ## [Logstash - Multiple outputs with different content](https://discuss.elastic.co/t/logstash-multiple-outputs-with-different-content/242300)
**Author:** [@vasek](https://discuss.elastic.co/u/vasek)\ **Replies:** 3\ **Last updated:** [July 29, 2020, 7:55am UTC](https://discuss.elastic.co/t/logstash-multiple-outputs-with-different-content/242300 "2020-07-29T07:55:28Z")
Hello, I would like to achieve to have multiple outputs withing one pipeline but with different content of documents. ONE PIPELINE OUTPUT FOR CUSTOMER (REDIS QUEUE or LOGSTASH) only one field: \[@metadata\]\[message\]\[… --- ## [Cant analyze byte string ( b'{... \\\\u0633 ... }' )](https://discuss.elastic.co/t/cant-analyze-byte-string-b-u0633/242605)
**Author:** [@momora9809](https://discuss.elastic.co/u/momora9809)\ **Replies:** 6\ **Last updated:** [July 29, 2020, 4:27am UTC](https://discuss.elastic.co/t/cant-analyze-byte-string-b-u0633/242605 "2020-07-29T04:27:11Z")
First of all, sorry for the title. I don't know what should I call that (byte string?). I have a data that is like this: b'{"id": "2", "words": \["\\\\u0633\\\\u0647\\\\u0627\\\\u0645\\\\u062f\\\\u0627\\\\u0631\\\\u06cc", "\\\\u0634\\\\u06… --- ## [HOW TO logstash convert all fields to String](https://discuss.elastic.co/t/how-to-logstash-convert-all-fields-to-string/242844)
**Author:** [@thaopt](https://discuss.elastic.co/u/thaopt)\ **Replies:** 2\ **Last updated:** [July 29, 2020, 4:12am UTC](https://discuss.elastic.co/t/how-to-logstash-convert-all-fields-to-string/242844 "2020-07-29T04:12:30Z")
Hello I am a foreign developer. My process is: Json \>\> Filebeat \>\> logstash \>\> Kibana So how can i convert all of fileds to String in Json via Logstash Filter config? P/s: I added this config to pipeline but not work… --- ## [Syntax error for XPath](https://discuss.elastic.co/t/syntax-error-for-xpath/242846)
**Author:** [@bevano](https://discuss.elastic.co/u/bevano)\ **Replies:** 5\ **Last updated:** [July 29, 2020, 2:07am UTC](https://discuss.elastic.co/t/syntax-error-for-xpath/242846 "2020-07-29T02:07:18Z")
Hi All, Trying to get the xml-xpath working, but logstash is crashing out at a certain part of the config file. The error points to it being a syntax problem Jul 28 17:09:30 QPS-HPI-TESTELK1 logstash\[730767\]: \[2020-0… --- ## [Loading Oracle into Elastic with Logstash](https://discuss.elastic.co/t/loading-oracle-into-elastic-with-logstash/242957)
**Author:** [@Queyon](https://discuss.elastic.co/u/Queyon)\ **Replies:** 4\ **Last updated:** [July 29, 2020, 12:02am UTC](https://discuss.elastic.co/t/loading-oracle-into-elastic-with-logstash/242957 "2020-07-29T00:02:13Z")
I'm trying to load Oracle data into Elastic with Logstash. The query returns a number of rows, but it only loaded one record into Elastic. Is there anything wrong with my config file? input { jdbc { jdbc\_driver\_libr… --- ## [Logstash stops forwarding to elasticsearch after short run](https://discuss.elastic.co/t/logstash-stops-forwarding-to-elasticsearch-after-short-run/242841)
**Author:** [@amhulli](https://discuss.elastic.co/u/amhulli)\ **Replies:** 3\ **Last updated:** [July 28, 2020, 10:27pm UTC](https://discuss.elastic.co/t/logstash-stops-forwarding-to-elasticsearch-after-short-run/242841 "2020-07-28T22:27:13Z")
i notice that my issue is similar to https://discuss.elastic.co/t/logstash-runs-as-service-but-stops-sending-logs-over-http-after-some-time/102938 but also note there was no resolution at that time. I'm hoping someone … --- ## [Logs de las búsquedas en ElasticSearch](https://discuss.elastic.co/t/logs-de-las-busquedas-en-elasticsearch/242179)
**Author:** [@Jesus2121](https://discuss.elastic.co/u/Jesus2121)\ **Replies:** 1\ **Last updated:** [July 28, 2020, 6:12pm UTC](https://discuss.elastic.co/t/logs-de-las-busquedas-en-elasticsearch/242179 "2020-07-28T18:12:08Z")
Buenas. Estoy intentando auditar las búsquedas que se realizan en elasticsearch ( las realizo desde python con la libreria elasticsearch o desde rest en cerebro). ¿Donde se guardan estos datos? Hasta ahora solo consig… --- ## [Kibana Index - Splitting Index Values and Adding to New Field](https://discuss.elastic.co/t/kibana-index-splitting-index-values-and-adding-to-new-field/242896)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 13\ **Last updated:** [July 28, 2020, 1:49pm UTC](https://discuss.elastic.co/t/kibana-index-splitting-index-values-and-adding-to-new-field/242896 "2020-07-28T13:49:44Z")
Hello, I am trying to figure out how to split the value of an index field and add that to a new field. Basically what I have on Kibana is as shown: This is the json document output of the index fields: I am trying… --- ## [Combine fields in Elasticsearch from logs with same ID](https://discuss.elastic.co/t/combine-fields-in-elasticsearch-from-logs-with-same-id/242880)
**Author:** [@egray](https://discuss.elastic.co/u/egray)\ **Replies:** 6\ **Last updated:** [July 28, 2020, 1:06pm UTC](https://discuss.elastic.co/t/combine-fields-in-elasticsearch-from-logs-with-same-id/242880 "2020-07-28T13:06:03Z")
I am working on a log aggregation solution for my company's web requests, using Logstash and Elasticsearch. There are several logs produced for each web request containing slightly different information, and they would l… --- ## [Logstash Unvailable When Syncing with Elastic's Yum Repository](https://discuss.elastic.co/t/logstash-unvailable-when-syncing-with-elastics-yum-repository/242904)
**Author:** [@alpo](https://discuss.elastic.co/u/alpo)\ **Replies:** 1\ **Last updated:** [July 28, 2020, 1:01pm UTC](https://discuss.elastic.co/t/logstash-unvailable-when-syncing-with-elastics-yum-repository/242904 "2020-07-28T13:01:56Z")
All, Looks like when we attempt to sync with Elastic's yum repository, the logstash rpm is not downloaded. Here is our full reposync command: /usr/bin/dnf reposync \\ --repoid=elastic-7.x \\ --arch=x86\_64 \\ --down… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=321) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=323)