# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=139 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 140 --- ## [Logstash how to parse and split nested json file](https://discuss.elastic.co/t/logstash-how-to-parse-and-split-nested-json-file/305308)
**Author:** [@Iss](https://discuss.elastic.co/u/Iss)\ **Replies:** 9\ **Last updated:** [May 23, 2022, 10:52am UTC](https://discuss.elastic.co/t/logstash-how-to-parse-and-split-nested-json-file/305308 "2022-05-23T10:52:32Z")
Hi! I have a nested json file in a list read as single event in Elasticsearch like this : \[{"header": {"id": "idvalue", "datestamp": "YYYY-MM-DD" }, "metas": {"dc:title": "text...", "dc:id": "idvalue… --- ## [Can't parse Laravel log with json content](https://discuss.elastic.co/t/cant-parse-laravel-log-with-json-content/305174)
**Author:** [@Robert\_Garcia\_Torren](https://discuss.elastic.co/u/Robert_Garcia_Torren)\ **Replies:** 3\ **Last updated:** [May 23, 2022, 8:31am UTC](https://discuss.elastic.co/t/cant-parse-laravel-log-with-json-content/305174 "2022-05-23T08:31:50Z")
Hello! I'm trying to send this Laravel Log to Elasticksearch using filebeat and logstash visualizing it with Kibana: \[2022-05-16 12:03:50\] dev.INFO: Update successful for user {"idmember":"37774", "idcard":"0000000H","… --- ## [How to parse log file with different log types with json](https://discuss.elastic.co/t/how-to-parse-log-file-with-different-log-types-with-json/305269)
**Author:** [@Robert\_Garcia\_Torren](https://discuss.elastic.co/u/Robert_Garcia_Torren)\ **Replies:** 3\ **Last updated:** [May 23, 2022, 8:29am UTC](https://discuss.elastic.co/t/how-to-parse-log-file-with-different-log-types-with-json/305269 "2022-05-23T08:29:15Z")
Hello I'm trying to parse a log file where different log types coexist. Hope someone can help :slight\_smile: This are all log types: \[2022-05-18 11:09:41\] dev.INFO: Inserting new User... \[2022-05-18 11:09:41\] dev.INFO:… --- ## [Process log for logstash pipelines](https://discuss.elastic.co/t/process-log-for-logstash-pipelines/305376)
**Author:** [@karlk](https://discuss.elastic.co/u/karlk)\ **Replies:** 0\ **Last updated:** [May 23, 2022, 8:05am UTC](https://discuss.elastic.co/t/process-log-for-logstash-pipelines/305376 "2022-05-23T08:05:50Z")
My goal is to make process log for any running process in ELK stack. Initially processes can be logged from Logstash pipelines and different Java processes. Sample index in elastic for process logs could be something li… --- ## [Log4j2.properties scripts and JavaScript error version \> 8.0](https://discuss.elastic.co/t/log4j2-properties-scripts-and-javascript-error-version-8-0/304054)
**Author:** [@Yustas](https://discuss.elastic.co/u/Yustas)\ **Replies:** 1\ **Last updated:** [May 23, 2022, 8:02am UTC](https://discuss.elastic.co/t/log4j2-properties-scripts-and-javascript-error-version-8-0/304054 "2022-05-23T08:02:09Z")
From version 8.0.0 JavaScript in log4j2.properties rise error. So i tryed find any script to filter logs with enabled pipeline splitting (pipeline.separate\_logs: true), but dont found any replacement for code (worked … --- ## [Blocking logs with certain conditions](https://discuss.elastic.co/t/blocking-logs-with-certain-conditions/305369)
**Author:** [@nyquillus](https://discuss.elastic.co/u/nyquillus)\ **Replies:** 1\ **Last updated:** [May 23, 2022, 6:59am UTC](https://discuss.elastic.co/t/blocking-logs-with-certain-conditions/305369 "2022-05-23T06:59:27Z")
Hi, There are some clients in our cluster that are sending logs longer than 1500 characters, these logs are basically useless. I was wondering if there was a way to block these kinds of logs entirely. Thanks in advance… --- ## [Failed to add field from ECS](https://discuss.elastic.co/t/failed-to-add-field-from-ecs/305342)
**Author:** [@Arunas\_Saltis](https://discuss.elastic.co/u/Arunas_Saltis)\ **Replies:** 1\ **Last updated:** [May 22, 2022, 4:30pm UTC](https://discuss.elastic.co/t/failed-to-add-field-from-ecs/305342 "2022-05-22T16:30:59Z")
Hi all, I am trying to add information about TLS session state between filebeats and logstash into logstash beats input from ECS, but no success. I tested all the described ECS fields of the beats input module, only one… --- ## [Copy metricbeat data from one elasticsearch to another on different machines](https://discuss.elastic.co/t/copy-metricbeat-data-from-one-elasticsearch-to-another-on-different-machines/304813)
**Author:** [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Replies:** 2\ **Last updated:** [May 22, 2022, 9:03am UTC](https://discuss.elastic.co/t/copy-metricbeat-data-from-one-elasticsearch-to-another-on-different-machines/304813 "2022-05-22T09:03:00Z")
Hi, source: Elasticsearch-8.1.0 destination: Elasticsearch-8.1.0 Both the source and destination are on different machines. In the source metricbeat indices are created daily. metricbeat.yml configuration: output.e… --- ## [Logstash problems with splitting on field value](https://discuss.elastic.co/t/logstash-problems-with-splitting-on-field-value/305328)
**Author:** [@edwardc](https://discuss.elastic.co/u/edwardc)\ **Replies:** 2\ **Last updated:** [May 22, 2022, 4:14am UTC](https://discuss.elastic.co/t/logstash-problems-with-splitting-on-field-value/305328 "2022-05-22T04:14:17Z")
Hi All, i have a problem with the logstash split filter. Any advice or input is greatly appreciated. I have the input as such: {"SONG\_A":\[ { "MD5":"a", "filename": "x.txt" }, { "MD5":"b", "filename": "y.txt" } \] } I a… --- ## [Parse basic xml file](https://discuss.elastic.co/t/parse-basic-xml-file/305331)
**Author:** [@Ely\_96](https://discuss.elastic.co/u/Ely_96)\ **Replies:** 11\ **Last updated:** [May 21, 2022, 10:42pm UTC](https://discuss.elastic.co/t/parse-basic-xml-file/305331 "2022-05-21T22:42:01Z")
Hi All, Could you please help me? I'm in trouble on parsing a basic xml file. This is an example of my xml: \\Hotel 1\\22\\1\\2\\ \… --- ## [Logstash =\> influxdb authentication failed](https://discuss.elastic.co/t/logstash-influxdb-authentication-failed/305323)
**Author:** [@mibrahim](https://discuss.elastic.co/u/mibrahim)\ **Replies:** 0\ **Last updated:** [May 21, 2022, 3:51am UTC](https://discuss.elastic.co/t/logstash-influxdb-authentication-failed/305323 "2022-05-21T03:51:14Z")
I am trying to make my logstash report matrics to influxdb but i got authentication failed :frowning: Here is my config for logstash.conf: input { exec { command =\> "echo 'Hello'" interval =\> 30 } } output… --- ## [Handle json filter field name cannot be an empty string error](https://discuss.elastic.co/t/handle-json-filter-field-name-cannot-be-an-empty-string-error/304973)
**Author:** [@austin0918](https://discuss.elastic.co/u/austin0918)\ **Replies:** 9\ **Last updated:** [May 21, 2022, 2:39am UTC](https://discuss.elastic.co/t/handle-json-filter-field-name-cannot-be-an-empty-string-error/304973 "2022-05-21T02:39:13Z")
I have some Proofpoint logs in JSON format likes... { "metadata":{ "tzOffsetMins":-240, "tsEpochMs":1650294078380 }, "ts":"2022-05-17T01:02:10.380410-0400", "guid":"T1sZJIJ-YGTdBDzraMXIw6rea9pB78Qw", "msgParts":\[ { … --- ## [Rename Field with \[\]](https://discuss.elastic.co/t/rename-field-with/305117)
**Author:** [@cris](https://discuss.elastic.co/u/cris)\ **Replies:** 8\ **Last updated:** [May 20, 2022, 6:04pm UTC](https://discuss.elastic.co/t/rename-field-with/305117 "2022-05-20T18:04:37Z")
Hello I want to rename some field without affect the other data. I tried with. input { elasticsearch { hosts =\> "localhost:9200" index =\> "services" size =\> 1 docinfo =\> true } } filter { … --- ## [Logstash elasticsearch-filter module return the wrong document](https://discuss.elastic.co/t/logstash-elasticsearch-filter-module-return-the-wrong-document/305304)
**Author:** [@CyaTo](https://discuss.elastic.co/u/CyaTo)\ **Replies:** 0\ **Last updated:** [May 20, 2022, 5:05pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-module-return-the-wrong-document/305304 "2022-05-20T17:05:47Z")
Hi, I'm trying to run a logstash ingestion pipeline that is able to enrich an event by querying an Elasticsearch node But so far I can't get it to work 100%, here is what I did: My logstash pipeline look like that : i… --- ## [\]\[FATAL\]\[org.logstash.Logstash \] Logstash stopped processing because of an error: (SystemExit) exit org.jruby.exceptions.SystemExit: (SystemExit) exit](https://discuss.elastic.co/t/fatal-org-logstash-logstash-logstash-stopped-processing-because-of-an-error-systemexit-exit-org-jruby-exceptions-systemexit-systemexit-exit/305283)
**Author:** [@MOUNA1](https://discuss.elastic.co/u/MOUNA1)\ **Replies:** 5\ **Last updated:** [May 20, 2022, 11:18am UTC](https://discuss.elastic.co/t/fatal-org-logstash-logstash-logstash-stopped-processing-because-of-an-error-systemexit-exit-org-jruby-exceptions-systemexit-systemexit-exit/305283 "2022-05-20T11:18:20Z")
I want to configure logstash to receive logs from pfsense but i have an error when i tested the configuration via : /usr/share/logstash/bin/logstash --config.test\_and\_exit --log.level=debug -f /etc/logstash/conf.d/ --pa… --- ## [Logstash Error - Unknown setting 'chedule' for jdbc](https://discuss.elastic.co/t/logstash-error-unknown-setting-chedule-for-jdbc/305268)
**Author:** [@Dr.Rask](https://discuss.elastic.co/u/Dr.Rask)\ **Replies:** 2\ **Last updated:** [May 20, 2022, 10:00am UTC](https://discuss.elastic.co/t/logstash-error-unknown-setting-chedule-for-jdbc/305268 "2022-05-20T10:00:37Z")
Hi, Getting this error when running logstash 8.2.0 on a windows machine PS C:\\logstash-oss-8.2.0\\bin\> .\\logstash -f logstash.config \[logstash.inputs.jdbc \] Unknown setting 'chedule' for jdbc input { jdbc{ jdbc… --- ## [Filtered Logstash for desired part](https://discuss.elastic.co/t/filtered-logstash-for-desired-part/305142)
**Author:** [@Zerra\_Triani](https://discuss.elastic.co/u/Zerra_Triani)\ **Replies:** 2\ **Last updated:** [May 20, 2022, 7:55am UTC](https://discuss.elastic.co/t/filtered-logstash-for-desired-part/305142 "2022-05-20T07:55:24Z")
0 I have a log, and I only want to retrieve the ones I have marked in red boxes. I've tried putting it in Elasticsearch but it inserts it every row in that log. How do you take the data that I have marked and combine it… --- ## [Logstash with 'ecs-compatibility' unable to accept particular CEF Extension field](https://discuss.elastic.co/t/logstash-with-ecs-compatibility-unable-to-accept-particular-cef-extension-field/305246)
**Author:** [@wii](https://discuss.elastic.co/u/wii)\ **Replies:** 2\ **Last updated:** [May 20, 2022, 7:38am UTC](https://discuss.elastic.co/t/logstash-with-ecs-compatibility-unable-to-accept-particular-cef-extension-field/305246 "2022-05-20T07:38:57Z")
Hi, I am still new with Elastic Stack. I have a problem with Logstash configuration which I have found no answer to for days. I have two Logstash configuration, 'ecs.conf' and 'cef.conf'. These two configs are created fo… --- ## [Help needed with ingesting custom log data](https://discuss.elastic.co/t/help-needed-with-ingesting-custom-log-data/305075)
**Author:** [@Sakis](https://discuss.elastic.co/u/Sakis)\ **Replies:** 1\ **Last updated:** [May 20, 2022, 5:15am UTC](https://discuss.elastic.co/t/help-needed-with-ingesting-custom-log-data/305075 "2022-05-20T05:15:14Z")
Hey there! I'm new to Elastic Stack (currently using the 14-day cloud trial) and I have a few questions about parsing log data. I work at an web development company and we provide customers with shop systems from Shopwa… --- ## [Parsing date in logstsh](https://discuss.elastic.co/t/parsing-date-in-logstsh/305118)
**Author:** [@jaikunwar](https://discuss.elastic.co/u/jaikunwar)\ **Replies:** 3\ **Last updated:** [May 19, 2022, 7:40pm UTC](https://discuss.elastic.co/t/parsing-date-in-logstsh/305118 "2022-05-19T19:40:39Z")
Hi team, I havee csv file with following data: name,age,gender,country,logtime John,34,male,China,2019-05-12 08:10 Basil,43,male,Taiwan,2020-05-13 8:10 Bella,25,female,USA,2018-05-14 8:10 I am using date parser as: … --- ## [Logstash error](https://discuss.elastic.co/t/logstash-error/305217)
**Author:** [@Naga\_Prudhvi](https://discuss.elastic.co/u/Naga_Prudhvi)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 6:52pm UTC](https://discuss.elastic.co/t/logstash-error/305217 "2022-05-19T18:52:34Z")
I am getting below error, while loading data into old index. \[2022-05-19T23:38:51,354\]\[ERROR\]\[logstash.filters.ruby \]\[main\]\[0c9121c6ce91a23303b26f6c1387d90c28a872d644b33037968422ec670a60c1\] Ruby exception occurred: u… --- ## [How to store huge amount of unstructured data](https://discuss.elastic.co/t/how-to-store-huge-amount-of-unstructured-data/304530)
**Author:** [@Naga\_Prudhvi](https://discuss.elastic.co/u/Naga_Prudhvi)\ **Replies:** 2\ **Last updated:** [May 19, 2022, 6:18pm UTC](https://discuss.elastic.co/t/how-to-store-huge-amount-of-unstructured-data/304530 "2022-05-19T18:18:18Z")
How can i store huge amount of unstructured data for example social media chat, email, website data Please help me with storing below sample email and ideas on storing social media chat. Hi John. Thank you so much fo… --- ## [If statement ruby filter](https://discuss.elastic.co/t/if-statement-ruby-filter/305093)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 6\ **Last updated:** [May 19, 2022, 3:42pm UTC](https://discuss.elastic.co/t/if-statement-ruby-filter/305093 "2022-05-19T15:42:50Z")
Hi, I have a small problem in if statement under ruby here my code: filter { grok { match =\> { "message" =\> "%{GREEDYDATA:log\_message}" }} mutate { split =\> {"message" =\> "|"} } ruby { code =\> "event.set('num… --- ## [Unable to split Arrays into fields -parsing XML Attributes of an unknown number of child elements](https://discuss.elastic.co/t/unable-to-split-arrays-into-fields-parsing-xml-attributes-of-an-unknown-number-of-child-elements/305144)
**Author:** [@rubberband](https://discuss.elastic.co/u/rubberband)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 3:23pm UTC](https://discuss.elastic.co/t/unable-to-split-arrays-into-fields-parsing-xml-attributes-of-an-unknown-number-of-child-elements/305144 "2022-05-19T15:23:10Z")
Hi, I'm trying to parse some XML content using Logstash. I'm trying to get the attributes as separate fields. Each \ element will have some attributes that need to be parsed as fields and an unknown number of \ **Author:** [@zubair\_aftab](https://discuss.elastic.co/u/zubair_aftab)\ **Replies:** 7\ **Last updated:** [May 19, 2022, 3:06pm UTC](https://discuss.elastic.co/t/unable-to-get-time-difference-with-ruby-in-logstash/304747 "2022-05-19T15:06:22Z") I have 2 time fields Time1 and Time2. I want to get there difference but getting error "Ruby exception occurred: undefined method \`-' for nil:NilClass". Below is the code. what is wrong in it?? \` if \[Time1\] { date{ … --- ## [Dissect Filter, Creating Subfields](https://discuss.elastic.co/t/dissect-filter-creating-subfields/305170)
**Author:** [@shinobu](https://discuss.elastic.co/u/shinobu)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 11:59am UTC](https://discuss.elastic.co/t/dissect-filter-creating-subfields/305170 "2022-05-19T11:59:04Z")
Hello, how can i get dissect to create a structure with subfields? dissect { mapping =\> { "message" =\> "%{test.subfield}.%{test.subfield2}" } } I would expect so… --- ## [MongoDB Output plugin 3.1.7 error](https://discuss.elastic.co/t/mongodb-output-plugin-3-1-7-error/305160)
**Author:** [@bhumikadave](https://discuss.elastic.co/u/bhumikadave)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 9:56am UTC](https://discuss.elastic.co/t/mongodb-output-plugin-3-1-7-error/305160 "2022-05-19T09:56:32Z")
Hello, I am not able to connect to MongoDB Atlas with below error.Using plugin version 3.1.7. \[WARN \]\[logstash.outputs.mongodb \]\[main\] MONGODB | Failed to handshake with \: 27017 ArgumentError: wrong number… --- ## [Grok logstash JSON](https://discuss.elastic.co/t/grok-logstash-json/305152)
**Author:** [@Roshan1](https://discuss.elastic.co/u/Roshan1)\ **Replies:** 0\ **Last updated:** [May 19, 2022, 9:11am UTC](https://discuss.elastic.co/t/grok-logstash-json/305152 "2022-05-19T09:11:55Z")
EL 8.2 Kibana 8.2 Hello Team, could you please advise how to merge the 2 lines below and generate the GONK command? {"log":"0.3817706468179257\\r\\n","stream":"stdout","time":"2022-05-18T10:37:31.746257847Z"} {"log":"1… --- ## [Logstash parse logfiles](https://discuss.elastic.co/t/logstash-parse-logfiles/305128)
**Author:** [@Roshan1](https://discuss.elastic.co/u/Roshan1)\ **Replies:** 1\ **Last updated:** [May 19, 2022, 7:43am UTC](https://discuss.elastic.co/t/logstash-parse-logfiles/305128 "2022-05-19T07:43:31Z")
Hello Team, kindly advise how I can parse the following logfile entry on Kibana/logstash? {"log":"LOG: aborting any active transactions\\n","stream":"stderr","time":"2022-05-17T11:04:05.353977974Z"} {"log":"FATAL: ter… --- ## [Logstash input file configuration](https://discuss.elastic.co/t/logstash-input-file-configuration/305081)
**Author:** [@Iss](https://discuss.elastic.co/u/Iss)\ **Replies:** 10\ **Last updated:** [May 19, 2022, 2:07am UTC](https://discuss.elastic.co/t/logstash-input-file-configuration/305081 "2022-05-19T02:07:05Z")
Hi! I use logstash to import a json file into Elasticsearch. I use the following configuration but the index is not created in Elasticsearch (or the consol). input{ file{ codec=\>multiline{ pattern =\>"^{header" … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=138) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=140)