# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=36 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 37 --- ## [Logstash unnest nested components using ruby](https://discuss.elastic.co/t/logstash-unnest-nested-components-using-ruby/356044)
**Author:** [@chowpay](https://discuss.elastic.co/u/chowpay)\ **Replies:** 0\ **Last updated:** [March 24, 2024, 12:19am UTC](https://discuss.elastic.co/t/logstash-unnest-nested-components-using-ruby/356044 "2024-03-24T00:19:22Z")
I have fields that come into the system with nested components. Using this code in logstash: if \[receivers\] { split { field =\> "receivers" } } ruby { … --- ## [Fingerprint no consistent on combination of same type of value](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038)
**Author:** [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\ **Replies:** 10\ **Last updated:** [March 23, 2024, 7:23pm UTC](https://discuss.elastic.co/t/fingerprint-no-consistent-on-combination-of-same-type-of-value/356038 "2024-03-23T19:23:09Z")
Hello Everyone , i am trying to create a unique fingerprint figure print according to upc + displayName and upc + selectedModifiersCombined with the conditions Here are 4 orders after splitting in logstash { orderId: … --- ## [Logstash filter http body with boolean values error](https://discuss.elastic.co/t/logstash-filter-http-body-with-boolean-values-error/355852)
**Author:** [@Dallas\_Toth](https://discuss.elastic.co/u/Dallas_Toth)\ **Replies:** 1\ **Last updated:** [March 23, 2024, 5:43pm UTC](https://discuss.elastic.co/t/logstash-filter-http-body-with-boolean-values-error/355852 "2024-03-23T17:43:09Z")
Having issues dealing with a value that needs to be a boolean in the body. Errors body\_format json errors {"type":"/request/invalid","title":"Bad Request","status":400,"detail":"Format of field article is invalid"}"} … --- ## [Issue with Case Sensitivity and Double Quotes in PostgreSQL Query through Logstash](https://discuss.elastic.co/t/issue-with-case-sensitivity-and-double-quotes-in-postgresql-query-through-logstash/355990)
**Author:** [@Atakan](https://discuss.elastic.co/u/Atakan)\ **Replies:** 1\ **Last updated:** [March 22, 2024, 4:57pm UTC](https://discuss.elastic.co/t/issue-with-case-sensitivity-and-double-quotes-in-postgresql-query-through-logstash/355990 "2024-03-22T16:57:14Z")
Hello, I'm encountering an issue while trying to fetch data from a PostgreSQL database using Logstash's JDBC input. The problem seems to revolve around case sensitivity and the use of double quotes in column names. Desp… --- ## [Ruby for loop logstash](https://discuss.elastic.co/t/ruby-for-loop-logstash/355946)
**Author:** [@Johnson\_will](https://discuss.elastic.co/u/Johnson_will)\ **Replies:** 4\ **Last updated:** [March 22, 2024, 3:27am UTC](https://discuss.elastic.co/t/ruby-for-loop-logstash/355946 "2024-03-22T03:27:18Z")
I am trying to convert the epoch timestamps which comes in an array. "steps": \[ { "number": 1, "status": "COMPLETE", "doRetry": null, "progress": null, "startTime": 1710431698… --- ## [JSON parse failure](https://discuss.elastic.co/t/json-parse-failure/355821)
**Author:** [@Johnson\_will](https://discuss.elastic.co/u/Johnson_will)\ **Replies:** 2\ **Last updated:** [March 21, 2024, 9:10pm UTC](https://discuss.elastic.co/t/json-parse-failure/355821 "2024-03-21T21:10:46Z")
Error parsing json {:source=\>"message", :raw=\>"Error running job", :exception=\>#\ **Author:** [@Lucas\_Giusti](https://discuss.elastic.co/u/Lucas_Giusti)\ **Replies:** 1\ **Last updated:** [March 21, 2024, 6:56pm UTC](https://discuss.elastic.co/t/does-running-two-pipelines-update-the-same-events/355933 "2024-03-21T18:56:58Z") Good afternoon friends, I'm new to logstash and I have the following question: I have two pipelines that I created separately. They are different processes. I'm using Docker and, when I up logstash with just one of th… --- ## [Logstash split event messages to different syslog servers](https://discuss.elastic.co/t/logstash-split-event-messages-to-different-syslog-servers/355917)
**Author:** [@Forsaken\_Sherbert\_81](https://discuss.elastic.co/u/Forsaken_Sherbert_81)\ **Replies:** 0\ **Last updated:** [March 21, 2024, 3:11pm UTC](https://discuss.elastic.co/t/logstash-split-event-messages-to-different-syslog-servers/355917 "2024-03-21T15:11:05Z")
Hi all I have an specific use-case scenario were I'm testing out Elastic Agent on windows clients. Some of these logs will be sent to the ELK SIEM for analysis, more debuging and version related information. Other logs… --- ## [How to check if a string is in an array element's property - no ruby](https://discuss.elastic.co/t/how-to-check-if-a-string-is-in-an-array-elements-property-no-ruby/355907)
**Author:** [@richfish](https://discuss.elastic.co/u/richfish)\ **Replies:** 0\ **Last updated:** [March 21, 2024, 1:33pm UTC](https://discuss.elastic.co/t/how-to-check-if-a-string-is-in-an-array-elements-property-no-ruby/355907 "2024-03-21T13:33:39Z")
Without Ruby, I would like to find if a string is in a particular property of any of the elements of an array. Is there any way to do this without using Ruby? Let's say an array contains objects each with a 'name' prope… --- ## [Is there any way to remove log.syslog.structured\_data](https://discuss.elastic.co/t/is-there-any-way-to-remove-log-syslog-structured-data/355899)
**Author:** [@Madiha\_Samad](https://discuss.elastic.co/u/Madiha_Samad)\ **Replies:** 2\ **Last updated:** [March 21, 2024, 1:00pm UTC](https://discuss.elastic.co/t/is-there-any-way-to-remove-log-syslog-structured-data/355899 "2024-03-21T13:00:30Z")
I am trying to parse checkpoint and fortinet logs through logstash and get log.syslog.structured\_data field in kibana/logscale .Is there any way to remove it ? I know it represents structured data expressed in RFC 5424 … --- ## [Grok parse failure](https://discuss.elastic.co/t/grok-parse-failure/355882)
**Author:** [@Anca\_Linca](https://discuss.elastic.co/u/Anca_Linca)\ **Replies:** 2\ **Last updated:** [March 21, 2024, 12:44pm UTC](https://discuss.elastic.co/t/grok-parse-failure/355882 "2024-03-21T12:44:09Z")
Hi! I am trying to parse a log that looks like this: 2024-03-21T09:33:14.187Z\\t77bfdcb9-15439-5d254-96r4a-12c543f7\\tDEBUG\\t\[logMetrics\] \[171101645.89459\] Metrics: {\\n fieldOne: 311,\\n fieldTwo: 36,\\n fieldThree: 34… --- ## [Sync multiple mysql tables with logstash](https://discuss.elastic.co/t/sync-multiple-mysql-tables-with-logstash/355810)
**Author:** [@abhinavtyagi](https://discuss.elastic.co/u/abhinavtyagi)\ **Replies:** 4\ **Last updated:** [March 21, 2024, 11:25am UTC](https://discuss.elastic.co/t/sync-multiple-mysql-tables-with-logstash/355810 "2024-03-21T11:25:22Z")
Hello, please take a look at my conf file for logstash: input { jdbc { jdbc\_driver\_library =\> "/usr/share/logstash/mysql-connector-java-8.0.22.jar" jdbc\_driver\_class =\> "com.mysql.cj.jdbc.Driver" jdbc\_conn… --- ## [How to update logstash 8.9.1 to 8.12.2](https://discuss.elastic.co/t/how-to-update-logstash-8-9-1-to-8-12-2/355891)
**Author:** [@Michael\_Mathan\_S](https://discuss.elastic.co/u/Michael_Mathan_S)\ **Replies:** 0\ **Last updated:** [March 21, 2024, 10:30am UTC](https://discuss.elastic.co/t/how-to-update-logstash-8-9-1-to-8-12-2/355891 "2024-03-21T10:30:55Z")
Here's the corrected text: "Hi, this is Michael Mathan S. I'm trying to upgrade Logstash from version 8.9.1 to 8.12.2, but I'm encountering several errors. Could you please advise me on how to fix the following errors? … --- ## [Multiline codec behavior on large input files](https://discuss.elastic.co/t/multiline-codec-behavior-on-large-input-files/355838)
**Author:** [@Mahdi\_Moazami](https://discuss.elastic.co/u/Mahdi_Moazami)\ **Replies:** 2\ **Last updated:** [March 20, 2024, 9:31pm UTC](https://discuss.elastic.co/t/multiline-codec-behavior-on-large-input-files/355838 "2024-03-20T21:31:00Z")
Hi there hope you're doing well. I have a question about the details about how multiline codec processes the input file. based on the file input docs, the file is chunked and lines are being read from the chunks. say w… --- ## [Logstash configuration remove everything after "-"](https://discuss.elastic.co/t/logstash-configuration-remove-everything-after/355672)
**Author:** [@ska](https://discuss.elastic.co/u/ska)\ **Replies:** 3\ **Last updated:** [March 20, 2024, 8:23pm UTC](https://discuss.elastic.co/t/logstash-configuration-remove-everything-after/355672 "2024-03-20T20:23:34Z")
ELK 7.16.x In my logstash file, I have the following configuration: .... if \[kubernetes\]\[namespace\] == 'webservices' or \[kubernetes\]\[namespace\] =~ /webservices-frontend\\d+$/ { mutate { replace =\> { 'p… --- ## [Logstash - ES index mixed up](https://discuss.elastic.co/t/logstash-es-index-mixed-up/355796)
**Author:** [@maskrider1111](https://discuss.elastic.co/u/maskrider1111)\ **Replies:** 7\ **Last updated:** [March 20, 2024, 4:49pm UTC](https://discuss.elastic.co/t/logstash-es-index-mixed-up/355796 "2024-03-20T16:49:16Z")
Hey folks, Currently im trying to ingest multiple firewall log source by differencing the folders and index. I created conf file for each source so that it will be ingested separately into different index. However i stu… --- ## [Unable to push to elasticsearch getting 400](https://discuss.elastic.co/t/unable-to-push-to-elasticsearch-getting-400/355492)
**Author:** [@Johnson\_will](https://discuss.elastic.co/u/Johnson_will)\ **Replies:** 2\ **Last updated:** [March 20, 2024, 2:08pm UTC](https://discuss.elastic.co/t/unable-to-push-to-elasticsearch-getting-400/355492 "2024-03-20T14:08:59Z")
\[2024-03-15T13:47:54,432\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>"64836160a3a2926d6a156148", :\_index=\>"job", :routing=\>nil}, {"startDate"=\>n… --- ## [Can I replace the @timestamp of the dashboard with the time at which the event get logged in the file](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607)
**Author:** [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)\ **Replies:** 16\ **Last updated:** [March 20, 2024, 2:12am UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607 "2024-03-20T02:12:23Z")
Hi there, I wanted to replace the @timestamp value with the log time. I am using filebeat as a logs collector. Here is my filter value which I am trying grok { match =\> { "message" =\> "\\\[%{DATA:datetime}\\\] I… --- ## [Upgraded to Elasticsearch 7.10 on AWS, enabled HTTPS only. Logstash is not connecting](https://discuss.elastic.co/t/upgraded-to-elasticsearch-7-10-on-aws-enabled-https-only-logstash-is-not-connecting/355709)
**Author:** [@scolilay](https://discuss.elastic.co/u/scolilay)\ **Replies:** 5\ **Last updated:** [March 19, 2024, 2:45pm UTC](https://discuss.elastic.co/t/upgraded-to-elasticsearch-7-10-on-aws-enabled-https-only-logstash-is-not-connecting/355709 "2024-03-19T14:45:48Z")
Hello I recentl upgraded Elasticsearch to 7.10 on my AWS account. I also upgraded logstash to 7.17.18. I also enabled the following settings because of my customer request. Required HTTPS Node-to-node encryption Enc… --- ## [New to ELK cant get logstash to work](https://discuss.elastic.co/t/new-to-elk-cant-get-logstash-to-work/355639)
**Author:** [@elastic\_user2](https://discuss.elastic.co/u/elastic_user2)\ **Replies:** 3\ **Last updated:** [March 19, 2024, 1:33pm UTC](https://discuss.elastic.co/t/new-to-elk-cant-get-logstash-to-work/355639 "2024-03-19T13:33:02Z")
Hello, I have an elk on an on prem cluster on VMs. One is running norconex web crawler (windows) which is working and creating logs and filebeats. The other is running logstash (RHEL). I installed the lastest versions… --- ## [Need help with grok](https://discuss.elastic.co/t/need-help-with-grok/355674)
**Author:** [@roman-tasi](https://discuss.elastic.co/u/roman-tasi)\ **Replies:** 2\ **Last updated:** [March 19, 2024, 10:27am UTC](https://discuss.elastic.co/t/need-help-with-grok/355674 "2024-03-19T10:27:46Z")
I need to perform a grok on the directory field with example value: C:\\Users\\takuya\\Desktop\\\_Summary presentation\\references Where the Desktop value is pulled out and saved as a new field containing the Desktop value. --- ## [Separate syslog](https://discuss.elastic.co/t/separate-syslog/355689)
**Author:** [@Frances\_Chu](https://discuss.elastic.co/u/Frances_Chu)\ **Replies:** 0\ **Last updated:** [March 19, 2024, 8:13am UTC](https://discuss.elastic.co/t/separate-syslog/355689 "2024-03-19T08:13:06Z")
I need to receive syslog from different systems. All syslogs are using UDP 514 port and cannot be changed in the source Now I need to separate different syslogs in logstash. logstash system reserved port below 1024. S… --- ## [Logstash filter aggregate with 2 nested fields](https://discuss.elastic.co/t/logstash-filter-aggregate-with-2-nested-fields/355548)
**Author:** [@Lucas\_Giusti](https://discuss.elastic.co/u/Lucas_Giusti)\ **Replies:** 4\ **Last updated:** [March 18, 2024, 9:23pm UTC](https://discuss.elastic.co/t/logstash-filter-aggregate-with-2-nested-fields/355548 "2024-03-18T21:23:22Z")
Goodnight. Could you help me with the problem below? I have the mysql table below: productId productName categoryId categoryName locationId locationPrice --------------------------------------------… --- ## [Logstash not started after change path.logs](https://discuss.elastic.co/t/logstash-not-started-after-change-path-logs/355616)
**Author:** [@Exdar](https://discuss.elastic.co/u/Exdar)\ **Replies:** 4\ **Last updated:** [March 18, 2024, 3:43pm UTC](https://discuss.elastic.co/t/logstash-not-started-after-change-path-logs/355616 "2024-03-18T15:43:49Z")
Hi all! I am newbie in ES\\logstash and etc. I installed ELK on Windows Server 2022 and if I not change logstash.yml - logstash starting and I got "Starting server on port..." But if I change section path log.level: w… --- ## [Incorrect ELASTIC\_HOSTS variable in Logstash container](https://discuss.elastic.co/t/incorrect-elastic-hosts-variable-in-logstash-container/355599)
**Author:** [@thibaut\_a](https://discuss.elastic.co/u/thibaut_a)\ **Replies:** 1\ **Last updated:** [March 18, 2024, 1:54pm UTC](https://discuss.elastic.co/t/incorrect-elastic-hosts-variable-in-logstash-container/355599 "2024-03-18T13:54:57Z")
Hi, I have a docker compose file with which I start 3 ES nodes (+ 1 for the setup), 1 Kibana, and 1 Logstash. The 4th ones works well when I deploy, but Logstash crashes. Here is the error I get from logs: \[2024-03-18T1… --- ## [Logstash conf file error](https://discuss.elastic.co/t/logstash-conf-file-error/355566)
**Author:** [@deathofangel](https://discuss.elastic.co/u/deathofangel)\ **Replies:** 1\ **Last updated:** [March 17, 2024, 11:36pm UTC](https://discuss.elastic.co/t/logstash-conf-file-error/355566 "2024-03-17T23:36:27Z")
Hello I need to constantly listen to a port and direct the json format logs coming there to elastic. I created a logstash.conf for this situation. But I always get the same error. logstash verison 8.12.2 input { tcp … --- ## [Logstash ERROR: (NameError) cannot initialize Java class org.logstash.plugins.AliasRegistry (java.lang.ExceptionInInitializerError)](https://discuss.elastic.co/t/logstash-error-nameerror-cannot-initialize-java-class-org-logstash-plugins-aliasregistry-java-lang-exceptionininitializererror/355350)
**Author:** [@Abdulberk](https://discuss.elastic.co/u/Abdulberk)\ **Replies:** 14\ **Last updated:** [March 16, 2024, 2:25pm UTC](https://discuss.elastic.co/t/logstash-error-nameerror-cannot-initialize-java-class-org-logstash-plugins-aliasregistry-java-lang-exceptionininitializererror/355350 "2024-03-16T14:25:46Z")
I just got both Kibana and Elasticsearch up and running as a windows service and now i want to build a very basic logstash pipeline to figure how it will perform sending the results to Elasticsearch so that i can make su… --- ## [Netflow codec: how to decode and define a field outside a flowset?](https://discuss.elastic.co/t/netflow-codec-how-to-decode-and-define-a-field-outside-a-flowset/355538)
**Author:** [@Catwoolfii](https://discuss.elastic.co/u/Catwoolfii)\ **Replies:** 0\ **Last updated:** [March 16, 2024, 2:15pm UTC](https://discuss.elastic.co/t/netflow-codec-how-to-decode-and-define-a-field-outside-a-flowset/355538 "2024-03-16T14:15:26Z")
Hello everyone! I'm decoding a netflow stream, here's an example: СпойлерCisco NetFlow/IPFIX Version: 10 Length: 380 Timestamp: Jan 2, 2024 14:21:25.000000000 RTZ 2 (winter) ExportTime: 1704194485 FlowSequence: 2… --- ## [Unable to enable port 5044 for logstash](https://discuss.elastic.co/t/unable-to-enable-port-5044-for-logstash/355449)
**Author:** [@Quan\_Chu](https://discuss.elastic.co/u/Quan_Chu)\ **Replies:** 3\ **Last updated:** [March 15, 2024, 7:32pm UTC](https://discuss.elastic.co/t/unable-to-enable-port-5044-for-logstash/355449 "2024-03-15T19:32:28Z")
here is my config: input { beats { port =\> 5044 } } filter { if \[fileset\]\[module\] == "system" { if \[fileset\]\[name\] == "auth" { grok { match =\> { "message" =\> \["%{SYSLOGTIMESTAMP:\[system\]\[auth\]\[timestamp\]} %{SYS… --- ## [Missing data when logstash Aggregate](https://discuss.elastic.co/t/missing-data-when-logstash-aggregate/355511)
**Author:** [@Minh\_S\_D](https://discuss.elastic.co/u/Minh_S_D)\ **Replies:** 2\ **Last updated:** [March 15, 2024, 5:06pm UTC](https://discuss.elastic.co/t/missing-data-when-logstash-aggregate/355511 "2024-03-15T17:06:48Z")
Input: Output: Code: Can someone explain to me why am i missing data? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=35) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=37)