# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=690 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 691 --- ## [How many times does a character appear?](https://discuss.elastic.co/t/how-many-times-does-a-character-appear/145454)
**Author:** [@JimP](https://discuss.elastic.co/u/JimP)\ **Replies:** 2\ **Last updated:** [September 5, 2018, 6:54pm UTC](https://discuss.elastic.co/t/how-many-times-does-a-character-appear/145454 "2018-09-05T18:54:22Z")
I would like to be able to find out how many times a character appears in a field. Is this possible with grok or some other filter? To explain, we are getting beats from multiple sources, and multiple departments in my… --- ## [Merge extendedStackTrace received from log4j to regular stack trace](https://discuss.elastic.co/t/merge-extendedstacktrace-received-from-log4j-to-regular-stack-trace/147393)
**Author:** [@andrsuh13](https://discuss.elastic.co/u/andrsuh13)\ **Replies:** 1\ **Last updated:** [September 5, 2018, 6:35pm UTC](https://discuss.elastic.co/t/merge-extendedstacktrace-received-from-log4j-to-regular-stack-trace/147393 "2018-09-05T18:35:52Z")
I am trying integrate our product with ELK. I use log4j and socket appender with JsonLayout and I see all my logs in Kibana. But stacktraces I get look like: "thrown": { "extendedStackTrace": \[ { "file… --- ## [Parse with different grok filter problem](https://discuss.elastic.co/t/parse-with-different-grok-filter-problem/147387)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 2\ **Last updated:** [September 5, 2018, 1:25pm UTC](https://discuss.elastic.co/t/parse-with-different-grok-filter-problem/147387 "2018-09-05T13:25:28Z")
Hello everybody, I want to parse these events : INFO;0000;000003;data/EDT/batchs/files/logs/REL015MT-20180807-20180807-2004.log INFO;0000;000003; /data/EDT/batchs/files/logs/WKF997-20180904-20180904-2029-42.log INFO;00… --- ## [Data sending to ES without using Filebeat](https://discuss.elastic.co/t/data-sending-to-es-without-using-filebeat/147361)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 5\ **Last updated:** [September 5, 2018, 12:36pm UTC](https://discuss.elastic.co/t/data-sending-to-es-without-using-filebeat/147361 "2018-09-05T12:36:16Z")
HI, I want to send data from logstash to ES without filebeat. I use docker command to start the logstash service and log is JSON. pLease find the below code and tell me where im commiting mistake, input{ file{ path =\>… --- ## [How to install Azure event hub plugin in Elastic cloud](https://discuss.elastic.co/t/how-to-install-azure-event-hub-plugin-in-elastic-cloud/147381)
**Author:** [@Theraghav](https://discuss.elastic.co/u/Theraghav)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 11:04am UTC](https://discuss.elastic.co/t/how-to-install-azure-event-hub-plugin-in-elastic-cloud/147381 "2018-09-05T11:04:21Z")
I am new to Elastic Cloud. The plugin for Azure event Hub as an input works all well in local environment. I need to know how to install the plugin in Cloud. It isn't listed in the list of installable plugins in Cloud c… --- ## [How to send json log from Raspberry to ELK Stack?(Filebeat alternartive)](https://discuss.elastic.co/t/how-to-send-json-log-from-raspberry-to-elk-stack-filebeat-alternartive/147379)
**Author:** [@Rahmat\_Agung\_W](https://discuss.elastic.co/u/Rahmat_Agung_W)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 11:01am UTC](https://discuss.elastic.co/t/how-to-send-json-log-from-raspberry-to-elk-stack-filebeat-alternartive/147379 "2018-09-05T11:01:58Z")
Hi, i just want to know how to send my json log from my raspberry to my ELK Stack remotely? Because i've been try using Filebeat but it is not compatible with arm architecture. PS : It is json log from honeypot Thanks. … --- ## [How work Logstash with Filebeat](https://discuss.elastic.co/t/how-work-logstash-with-filebeat/147353)
**Author:** [@Jose\_Luis\_Sanchez\_Bo](https://discuss.elastic.co/u/Jose_Luis_Sanchez_Bo)\ **Replies:** 2\ **Last updated:** [September 5, 2018, 10:54am UTC](https://discuss.elastic.co/t/how-work-logstash-with-filebeat/147353 "2018-09-05T10:54:44Z")
Hi everyone, I have a ELK Server, and i use Filebeat for input. Example: 2018-06-30 20:18:32,137 Authenticate a=1234 2018-06-30 20:18:32,137 Values b=4 c=5 Procedure in logstash: The first "Authenticate" traces, pro… --- ## [Logstash 6.4 startup error in Windows ... Please suggest](https://discuss.elastic.co/t/logstash-6-4-startup-error-in-windows-please-suggest/147370)
**Author:** [@samuraichetan](https://discuss.elastic.co/u/samuraichetan)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 10:28am UTC](https://discuss.elastic.co/t/logstash-6-4-startup-error-in-windows-please-suggest/147370 "2018-09-05T10:28:49Z")
HI Team , Couls you suggest how to resolve the error.. THis seems a known error since v6.0 in logstash but could not really get a resolution. Env: Win 7 64 bit Issue :- Logstash startup error in Windows ...SThis is on… --- ## [Feature: max\_age of events in persistent queue / time based cleanup](https://discuss.elastic.co/t/feature-max-age-of-events-in-persistent-queue-time-based-cleanup/147367)
**Author:** [@Suny](https://discuss.elastic.co/u/Suny)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 10:20am UTC](https://discuss.elastic.co/t/feature-max-age-of-events-in-persistent-queue-time-based-cleanup/147367 "2018-09-05T10:20:37Z")
The persistent queue can be configured to hold a specific number of events, and it can be limited in size (see persistent queue doc) , the parameters are called queue.max\_events and queue.max\_bytes. Due to regulations, … --- ## [Is there a way to groupby all the events processed by Logstash?](https://discuss.elastic.co/t/is-there-a-way-to-groupby-all-the-events-processed-by-logstash/147362)
**Author:** [@Sahaya\_Agnel](https://discuss.elastic.co/u/Sahaya_Agnel)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 9:51am UTC](https://discuss.elastic.co/t/is-there-a-way-to-groupby-all-the-events-processed-by-logstash/147362 "2018-09-05T09:51:25Z")
Ideally Logstash will process row by row if we read data from SQL and process via Logstash. Is there a way to group by all the events processed by Logstash and do aggregations ? --- ## [Logstash xml filter](https://discuss.elastic.co/t/logstash-xml-filter/147285)
**Author:** [@xodn0812](https://discuss.elastic.co/u/xodn0812)\ **Replies:** 1\ **Last updated:** [September 5, 2018, 9:12am UTC](https://discuss.elastic.co/t/logstash-xml-filter/147285 "2018-09-05T09:12:57Z")
I want to store the following xml format data at once. \name\ \my name\ \hi~ my name is taewoo!\ \2018-09-04T09:00:00\ However, each field is stored separatel… --- ## [Kafka output plugin not working](https://discuss.elastic.co/t/kafka-output-plugin-not-working/147187)
**Author:** [@asatsi](https://discuss.elastic.co/u/asatsi)\ **Replies:** 1\ **Last updated:** [September 5, 2018, 5:38am UTC](https://discuss.elastic.co/t/kafka-output-plugin-not-working/147187 "2018-09-05T05:38:40Z")
Hi, I am trying to push logs onto a kafka topic. Has anybody successfully used Kafka output plugin in logstash? Tried with logstash version: 6.3.2/6.4.0 but same result. Below is my config file: input { file { path… --- ## [How to grok field appearing multiple times in a log line](https://discuss.elastic.co/t/how-to-grok-field-appearing-multiple-times-in-a-log-line/146287)
**Author:** [@Prakash\_Sharma](https://discuss.elastic.co/u/Prakash_Sharma)\ **Replies:** 15\ **Last updated:** [September 5, 2018, 8:54am UTC](https://discuss.elastic.co/t/how-to-grok-field-appearing-multiple-times-in-a-log-line/146287 "2018-09-05T08:54:15Z")
Hi , I have a syslog coming from cisco ISE which has multiple entry of "Step" in one line . Like ++++++++++++++++++++++ Step=11001, Step=11017, Step=15049, Step=15008, Step=15048, Step=15048, Step=15048, Step=15048,… --- ## [Failed to create monitoring event Map keys: \[:pipelines, :reloads\]](https://discuss.elastic.co/t/failed-to-create-monitoring-event-map-keys-pipelines-reloads/147331)
**Author:** [@Girish\_N](https://discuss.elastic.co/u/Girish_N)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 8:05am UTC](https://discuss.elastic.co/t/failed-to-create-monitoring-event-map-keys-pipelines-reloads/147331 "2018-09-05T08:05:11Z")
Hi After installing x-pack, i have the followinf error in my logstash \[2018-09-05T09:14:40,717\]\[ERROR\]\[logstash.inputs.metrics \] Failed to create monitoring event {:message=\>"For path: events. Map keys: \[:pipelines, :r… --- ## [How can we distinguish the known issue?](https://discuss.elastic.co/t/how-can-we-distinguish-the-known-issue/147325)
**Author:** [@terrymu](https://discuss.elastic.co/u/terrymu)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 7:42am UTC](https://discuss.elastic.co/t/how-can-we-distinguish-the-known-issue/147325 "2018-09-05T07:42:17Z")
Dear all experts, I have do the maintainance work of one huge system, and I use the logstash pattern to parse the related logs to find the abnormal information. Of course logstash will grep all of abnormal logs accordin… --- ## [SinceDB not updated properly](https://discuss.elastic.co/t/sincedb-not-updated-properly/147324)
**Author:** [@5b0eafdfbfb313f6ed17](https://discuss.elastic.co/u/5b0eafdfbfb313f6ed17)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 7:31am UTC](https://discuss.elastic.co/t/sincedb-not-updated-properly/147324 "2018-09-05T07:31:23Z")
My logstash ingest log from /var/log/audit/audit.log After running for one day, the cpu usage is very high (80%). I found that i have many same events. It is wired. I have delete file in /var/log/audit/ and sincedb fi… --- ## [Protobuf to avro](https://discuss.elastic.co/t/protobuf-to-avro/147317)
**Author:** [@laurent\_telle](https://discuss.elastic.co/u/laurent_telle)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 6:25am UTC](https://discuss.elastic.co/t/protobuf-to-avro/147317 "2018-09-05T06:25:28Z")
Hi, Actually i use a personal java program to take nginx log protobuf format from kafka , then transform it into avro format , then resend it to another topic in kafka and then use logstash to take it and inject it in e… --- ## [Logstash 6.2 not able to execute oracle database procedures](https://discuss.elastic.co/t/logstash-6-2-not-able-to-execute-oracle-database-procedures/147049)
**Author:** [@raulk89](https://discuss.elastic.co/u/raulk89)\ **Replies:** 2\ **Last updated:** [September 5, 2018, 6:04am UTC](https://discuss.elastic.co/t/logstash-6-2-not-able-to-execute-oracle-database-procedures/147049 "2018-09-05T06:04:31Z")
Hi Logstash 6.2 Oracle database 12.1.0.2.0 Using ojdbc6.jar drivers to connect to oracle database (have tried also ojdbc7.jar). Connection attempt succeeds with both drivers, since select statements run successfully, … --- ## [Ealasticsearch filter](https://discuss.elastic.co/t/ealasticsearch-filter/147261)
**Author:** [@Pierre2](https://discuss.elastic.co/u/Pierre2)\ **Replies:** 0\ **Last updated:** [September 4, 2018, 7:30pm UTC](https://discuss.elastic.co/t/ealasticsearch-filter/147261 "2018-09-04T19:30:57Z")
Hi, I use Logstash with two pipelines. The first will process log file comming from SVI (Vocal server) and index fields into the index called "svi".The second pipeline process CDR (call details records) and index fields… --- ## [Not Seeing the Index created in Elastic. Even No errors are shown](https://discuss.elastic.co/t/not-seeing-the-index-created-in-elastic-even-no-errors-are-shown/147260)
**Author:** [@Damodara\_Gottipalli](https://discuss.elastic.co/u/Damodara_Gottipalli)\ **Replies:** 0\ **Last updated:** [September 4, 2018, 7:16pm UTC](https://discuss.elastic.co/t/not-seeing-the-index-created-in-elastic-even-no-errors-are-shown/147260 "2018-09-04T19:16:09Z")
Hi, This is my configuration file: input { file { path =\> \["C:\\study\\elasticsearch\\data.csv"\] start\_position =\> "beginning" sincedb\_path =\> "nul" codec =\> rubydebug } } filter { csv { separator =\> "," column… --- ## [HTM / Logstasg Configuration / Pipeline Config](https://discuss.elastic.co/t/htm-logstasg-configuration-pipeline-config/146244)
**Author:** [@robegome](https://discuss.elastic.co/u/robegome)\ **Replies:** 31\ **Last updated:** [September 4, 2018, 6:38pm UTC](https://discuss.elastic.co/t/htm-logstasg-configuration-pipeline-config/146244 "2018-09-04T18:38:00Z")
Hello everyone, I would like to know if there are some place (thread) to create a pipeline (.conf file) to get the input/filter and output from a HTM file or if using filebeat I could get some view in Kibana... I am new… --- ## [Need to parse the xml data via logstash](https://discuss.elastic.co/t/need-to-parse-the-xml-data-via-logstash/147247)
**Author:** [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\ **Replies:** 0\ **Last updated:** [September 4, 2018, 5:12pm UTC](https://discuss.elastic.co/t/need-to-parse-the-xml-data-via-logstash/147247 "2018-09-04T17:12:34Z")
I am trying to parse an XML using 5.2.1 logstatsh version. Sample xml am getting error below: WARN logstash.filter.split - Only String and Array Types are splittable.field:apps\[app\] is of type = NilClass ERRO… --- ## [Logstash fingerprint plugin with JSON data](https://discuss.elastic.co/t/logstash-fingerprint-plugin-with-json-data/147239)
**Author:** [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\ **Replies:** 1\ **Last updated:** [September 4, 2018, 3:54pm UTC](https://discuss.elastic.co/t/logstash-fingerprint-plugin-with-json-data/147239 "2018-09-04T15:54:57Z")
Hi, I am parsing the JSON logs using below config: file { type =\>"logstash" path =\> \["/logstash/\*.json"\] sincedb\_path =\>"/logstash/var/json.sincedb" codec =\> json start\_positio… --- ## [Event send Kafka to logstash](https://discuss.elastic.co/t/event-send-kafka-to-logstash/147142)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 3\ **Last updated:** [September 4, 2018, 1:45pm UTC](https://discuss.elastic.co/t/event-send-kafka-to-logstash/147142 "2018-09-04T13:45:54Z")
Hi, Hope all doing good. I have some doubt about using Kafka. In my end, we are just planning to receive data from Kafka to Logstash. In Logstash configuration I have enclosed the Kafka as an input filter and Logstash … --- ## [Logstash translate filter - multiple matches](https://discuss.elastic.co/t/logstash-translate-filter-multiple-matches/144401)
**Author:** [@lens](https://discuss.elastic.co/u/lens)\ **Replies:** 7\ **Last updated:** [September 4, 2018, 1:37pm UTC](https://discuss.elastic.co/t/logstash-translate-filter-multiple-matches/144401 "2018-09-04T13:37:31Z")
I have a situation where I am using a dictionary file with the translate filter. The keys are IPs or regexes for the IPs in IP blocks. What happens if an IP matches more than one dictionary entry, eg it's listed individu… --- ## [Logstash handling duplicates using UUID](https://discuss.elastic.co/t/logstash-handling-duplicates-using-uuid/147215)
**Author:** [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\ **Replies:** 0\ **Last updated:** [September 4, 2018, 1:30pm UTC](https://discuss.elastic.co/t/logstash-handling-duplicates-using-uuid/147215 "2018-09-04T13:30:54Z")
I have requirement to use a mechanism to avoid duplicates on events received from log data. These duplicates are being caused by ES or Logstash indexer crashes. I was going through this link: https://www.elastic.co/blo… --- ## [Logstash getting fileds from another event](https://discuss.elastic.co/t/logstash-getting-fileds-from-another-event/146865)
**Author:** [@vasudevan](https://discuss.elastic.co/u/vasudevan)\ **Replies:** 3\ **Last updated:** [September 4, 2018, 1:10pm UTC](https://discuss.elastic.co/t/logstash-getting-fileds-from-another-event/146865 "2018-09-04T13:10:36Z")
My Logstash config: input { beats { port =\> 5002 } } filter { grok { add\_tag =\> \[ "first" \] match =\> { "message" =\> "". .definitionName": "(?\. ?)"" } } grok { add\_tag =\> \[ "first" \] ma… --- ## [Logstash-data between start and end transaction](https://discuss.elastic.co/t/logstash-data-between-start-and-end-transaction/143735)
**Author:** [@agrawal](https://discuss.elastic.co/u/agrawal)\ **Replies:** 7\ **Last updated:** [September 4, 2018, 11:21am UTC](https://discuss.elastic.co/t/logstash-data-between-start-and-end-transaction/143735 "2018-09-04T11:21:24Z")
I need to index the data between start and end transaction. I tried lots of combination. but no one worked. In logs, I want Logstash to parse the text between start and end transaction. is it possible? 2018-07-25 13:1… --- ## [XML to JSON and parsing the last one](https://discuss.elastic.co/t/xml-to-json-and-parsing-the-last-one/146824)
**Author:** [@JPelastic](https://discuss.elastic.co/u/JPelastic)\ **Replies:** 3\ **Last updated:** [September 4, 2018, 8:40am UTC](https://discuss.elastic.co/t/xml-to-json-and-parsing-the-last-one/146824 "2018-09-04T08:40:54Z")
If I am correct, after the xml filter the field invoice contains JSON. xml { source =\> "documentout" target =\> "invoice" } I did confirm this in the stdout "invoice" =\> { "DeliveryDate" =\>… --- ## [The Kv plugin, field\_split uses spaces to cut, but the value contains spaces. Value cutting is incomplete](https://discuss.elastic.co/t/the-kv-plugin-field-split-uses-spaces-to-cut-but-the-value-contains-spaces-value-cutting-is-incomplete/146834)
**Author:** [@sun\_changlong](https://discuss.elastic.co/u/sun_changlong)\ **Replies:** 0\ **Last updated:** [August 31, 2018, 8:57am UTC](https://discuss.elastic.co/t/the-kv-plugin-field-split-uses-spaces-to-cut-but-the-value-contains-spaces-value-cutting-is-incomplete/146834 "2018-08-31T08:57:04Z")
My data format is: evel="2" treatment="3" cmd="\\"D:\\Program Files (x86)\\360\\360safe\\modules\\setup.exe\\" /s /smartsilence" type="sys" Use the kv plugin, use " " to cut, and then use "=" to parse, but the value of the cm… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=689) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=691)