# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=426 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 427 --- ## [Two groks?](https://discuss.elastic.co/t/two-groks/213397)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 2\ **Last updated:** [December 31, 2019, 4:39pm UTC](https://discuss.elastic.co/t/two-groks/213397 "2019-12-31T16:39:33Z")
I have logs of memory usage, the problem is that some logs description are in spanish and others in english Logs: 12/30/19 15:34:50 host-name Uso de Memoria % vm.memory.size\[pused\] 67.6928 12/30/19 16:03:58 host-na… --- ## [Multiline codec is not breaking messages correctly](https://discuss.elastic.co/t/multiline-codec-is-not-breaking-messages-correctly/213460)
**Author:** [@sparkblaze](https://discuss.elastic.co/u/sparkblaze)\ **Replies:** 0\ **Last updated:** [December 31, 2019, 3:10pm UTC](https://discuss.elastic.co/t/multiline-codec-is-not-breaking-messages-correctly/213460 "2019-12-31T15:10:05Z")
Afternoon, I'm having some issues with the multiline codec config and it adding events to the previous message so not being very helpful. in logstash I have the following input definition syslog { id =\> "input… --- ## [Qradar to Elastic](https://discuss.elastic.co/t/qradar-to-elastic/213443)
**Author:** [@Sahil.zxc](https://discuss.elastic.co/u/Sahil.zxc)\ **Replies:** 0\ **Last updated:** [December 31, 2019, 11:46am UTC](https://discuss.elastic.co/t/qradar-to-elastic/213443 "2019-12-31T11:46:15Z")
We want to extend our Qradar to Elastic for threat hunting, so we intend on forwarding logs from Qradar to Elastic via log forwarding, via syslog or via JSON. Since I cannot install any beats on QRadar Server, will forw… --- ## [Logstash ruby filter is not working](https://discuss.elastic.co/t/logstash-ruby-filter-is-not-working/213432)
**Author:** [@Sajal](https://discuss.elastic.co/u/Sajal)\ **Replies:** 0\ **Last updated:** [December 31, 2019, 9:48am UTC](https://discuss.elastic.co/t/logstash-ruby-filter-is-not-working/213432 "2019-12-31T09:48:08Z")
Hi All , I am using ruby filter in logstash . Xpack is enabled in my local machine. I am not able to put username and password in ruby filter like i am putting after index name(elastic:elastic). Hence, my ruby filter lo… --- ## [Snow API to avoid duplicates](https://discuss.elastic.co/t/snow-api-to-avoid-duplicates/213074)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 13\ **Last updated:** [December 31, 2019, 9:31am UTC](https://discuss.elastic.co/t/snow-api-to-avoid-duplicates/213074 "2019-12-31T09:31:28Z")
Hi Team, im working with a Snow API on my ELK setup (7.2.0), and below is the configuration of my logsatsh config file. input { http\_poller { urls =\> { snowinc =\> { url =\> "https://service-now.com/api/n… --- ## [Logstash Data Recovery point](https://discuss.elastic.co/t/logstash-data-recovery-point/213408)
**Author:** [@rkcharlie](https://discuss.elastic.co/u/rkcharlie)\ **Replies:** 0\ **Last updated:** [December 31, 2019, 5:18am UTC](https://discuss.elastic.co/t/logstash-data-recovery-point/213408 "2019-12-31T05:18:41Z")
Hello Team, How to handle Logstash Data Recovery point if you have any. We are connecting logstash with DB Source ( MSSQL,PostgreSql). We are able to make connectivity but not sure if source or logstash goes down then f… --- ## [How to access IIS request source IP from within pipeline.conf?](https://discuss.elastic.co/t/how-to-access-iis-request-source-ip-from-within-pipeline-conf/213387)
**Author:** [@dRm42](https://discuss.elastic.co/u/dRm42)\ **Replies:** 2\ **Last updated:** [December 31, 2019, 1:35am UTC](https://discuss.elastic.co/t/how-to-access-iis-request-source-ip-from-within-pipeline-conf/213387 "2019-12-31T01:35:09Z")
Hi Guys, I've spent more than a day trying to figure this out but just couldn't get it working. I'm using Filebeat to collect IIS logs, push them through Logstash and then into ES. Filebeat has the IIS module enabled an… --- ## [Special characters in regex pattern](https://discuss.elastic.co/t/special-characters-in-regex-pattern/213380)
**Author:** [@Jeremy\_Andrews](https://discuss.elastic.co/u/Jeremy_Andrews)\ **Replies:** 1\ **Last updated:** [December 30, 2019, 9:40pm UTC](https://discuss.elastic.co/t/special-characters-in-regex-pattern/213380 "2019-12-30T21:40:00Z")
I’m trying to write an if statement in Logstash to check if the message starts with a certain pattern, but Logstash crashes without any useful reason given in the log. I’m wondering if I’m not escaping the special charac… --- ## [Two kinds of logs, the same grok](https://discuss.elastic.co/t/two-kinds-of-logs-the-same-grok/213369)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 2\ **Last updated:** [December 30, 2019, 6:02pm UTC](https://discuss.elastic.co/t/two-kinds-of-logs-the-same-grok/213369 "2019-12-30T18:02:53Z")
Hi, I have two kinds of logs. if the server is linux it display : "$1" after the string "Used disk space on", and if the server is windows display the letter of the drive: "C:" Logs: 12/28/19 21:59:10 server-hostname U… --- ## [Logstash does not forward logs](https://discuss.elastic.co/t/logstash-does-not-forward-logs/213368)
**Author:** [@d.shulinin](https://discuss.elastic.co/u/d.shulinin)\ **Replies:** 0\ **Last updated:** [December 30, 2019, 4:44pm UTC](https://discuss.elastic.co/t/logstash-does-not-forward-logs/213368 "2019-12-30T16:44:22Z")
Hi all. I'm facing a strange issue with logstash 7.2.0. I have two winlogbeats with a similar config sending logs to logstash (elk-1). Logstash then sends the logs to elasticsearch node (elk-2). I can confirm that log… --- ## [Logstash reads only 10 rows](https://discuss.elastic.co/t/logstash-reads-only-10-rows/213338)
**Author:** [@Lubos\_Marek](https://discuss.elastic.co/u/Lubos_Marek)\ **Replies:** 1\ **Last updated:** [December 30, 2019, 2:24pm UTC](https://discuss.elastic.co/t/logstash-reads-only-10-rows/213338 "2019-12-30T14:24:31Z")
Hello, I am trying to read csv file with logstash but logstash reads only firs 10 rows. Where could be a problem? Structure of my CSV file: 500;message 1 200;message 2 500;message 1 200;message 2 500;message 1 20… --- ## [How to configure Logstash to get live tails logs in kibana dashboard?](https://discuss.elastic.co/t/how-to-configure-logstash-to-get-live-tails-logs-in-kibana-dashboard/213246)
**Author:** [@ppantang](https://discuss.elastic.co/u/ppantang)\ **Replies:** 2\ **Last updated:** [December 30, 2019, 2:24pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-get-live-tails-logs-in-kibana-dashboard/213246 "2019-12-30T14:24:18Z")
I have one custom log file that updates every one min. I am using flebeat to push this logfile to logstash . I want to see live logs with tailing in kibana dashboard how to configure at filebeat end. Please help me on … --- ## [Fortinet firewall logs in ELK](https://discuss.elastic.co/t/fortinet-firewall-logs-in-elk/213343)
**Author:** [@rguptarg](https://discuss.elastic.co/u/rguptarg)\ **Replies:** 3\ **Last updated:** [December 30, 2019, 1:49pm UTC](https://discuss.elastic.co/t/fortinet-firewall-logs-in-elk/213343 "2019-12-30T13:49:34Z")
HI Team, I am getting logs from Fortinet firewall and config file for this input { tcp { port =\> 5014 type =\> syslog } udp { port =\> 5014 type =\> syslog } } filter { if \[type\] == "syslog" { grok { match =\> … --- ## [Logstash do not allow absolute file path](https://discuss.elastic.co/t/logstash-do-not-allow-absolute-file-path/213339)
**Author:** [@Vo\_Minh\_Tri](https://discuss.elastic.co/u/Vo_Minh_Tri)\ **Replies:** 0\ **Last updated:** [December 30, 2019, 11:47am UTC](https://discuss.elastic.co/t/logstash-do-not-allow-absolute-file-path/213339 "2019-12-30T11:47:38Z")
I'm running elk stack on my Windows environment. It's work well with path =\> "/opt/in/\*.log.\*" but It always throws File paths must be absolute, relative path specified when I change to absolute path "C:/Users/vmtri/De… --- ## [Network devices logs monitoring](https://discuss.elastic.co/t/network-devices-logs-monitoring/213340)
**Author:** [@rguptarg](https://discuss.elastic.co/u/rguptarg)\ **Replies:** 0\ **Last updated:** [December 30, 2019, 11:49am UTC](https://discuss.elastic.co/t/network-devices-logs-monitoring/213340 "2019-12-30T11:49:46Z")
Hi Team, I am new in ELK, and I am looking to monitor my network infra logs, We are using below vendors devices :- Cisco Sonic wall Fortinet At network end all log forwarding is configured on UDP 5014 port for all d… --- ## [Oniguruma patern will not work in grok debuger](https://discuss.elastic.co/t/oniguruma-patern-will-not-work-in-grok-debuger/213190)
**Author:** [@Marta\_Zagrajek](https://discuss.elastic.co/u/Marta_Zagrajek)\ **Replies:** 3\ **Last updated:** [December 30, 2019, 11:29am UTC](https://discuss.elastic.co/t/oniguruma-patern-will-not-work-in-grok-debuger/213190 "2019-12-30T11:29:00Z")
I have a filters like that: filter { grok { match =\> { "message" =\> '%{TIME:timestamp} (\\\[)?(%{DATA:logger})?(\\\])? \\{%{DATA:thread}\\} %{LOGLEVEL:level} : (?\(.|\\r|\\n)\*)' } remove\_field =\> \["messag… --- ## [Splitting field in logstash](https://discuss.elastic.co/t/splitting-field-in-logstash/213184)
**Author:** [@Nisha.MP](https://discuss.elastic.co/u/Nisha.MP)\ **Replies:** 3\ **Last updated:** [December 30, 2019, 6:06am UTC](https://discuss.elastic.co/t/splitting-field-in-logstash/213184 "2019-12-30T06:06:06Z")
Hi I'm trying to split a particular field in CloudTrail log and this is my filter pattern filter{ json{ source =\> "message" } split{ field =\> "resources" add\_tag =\> "splitted" } } This is working when the field… --- ## [How to take selected fields from nested array](https://discuss.elastic.co/t/how-to-take-selected-fields-from-nested-array/213188)
**Author:** [@namali](https://discuss.elastic.co/u/namali)\ **Replies:** 2\ **Last updated:** [December 30, 2019, 5:27am UTC](https://discuss.elastic.co/t/how-to-take-selected-fields-from-nested-array/213188 "2019-12-30T05:27:11Z")
sample json data { Name: "Cesar Barroso" ageBands : \[ { ageFrom :25, ageTo :64, adult :true, treatAsAdult :true, description :"Adult"}, { ageFrom :35, ageTo :25, adult :true, treatAsAdult :true, descri… --- ## [How to poll thousands of devices using SNMP input Plugin?](https://discuss.elastic.co/t/how-to-poll-thousands-of-devices-using-snmp-input-plugin/212926)
**Author:** [@subash](https://discuss.elastic.co/u/subash)\ **Replies:** 1\ **Last updated:** [December 30, 2019, 5:21am UTC](https://discuss.elastic.co/t/how-to-poll-thousands-of-devices-using-snmp-input-plugin/212926 "2019-12-30T05:21:51Z")
Is it possible to poll thousands of devices using Logstash SNMP input plugin without any latency? I tried with one device, but when it comes to 1000s of devices I'm not sure whether it feasible or not. Please suggest T… --- ## [Cannot get Logstash to connect with TLS on Elastic Docker Cluster](https://discuss.elastic.co/t/cannot-get-logstash-to-connect-with-tls-on-elastic-docker-cluster/212886)
**Author:** [@greengeek](https://discuss.elastic.co/u/greengeek)\ **Replies:** 1\ **Last updated:** [December 29, 2019, 11:36pm UTC](https://discuss.elastic.co/t/cannot-get-logstash-to-connect-with-tls-on-elastic-docker-cluster/212886 "2019-12-29T23:36:28Z")
I am unable to get Logstash to connect over TLS to my Elastic/Kibana Docker Cluster. I followed the instructions to setup a multi node elastic stack on docker found here, https://www.elastic.co/guide/en/elastic-stack-g… --- ## [How to store XML input to multiples indices](https://discuss.elastic.co/t/how-to-store-xml-input-to-multiples-indices/213271)
**Author:** [@razimandom](https://discuss.elastic.co/u/razimandom)\ **Replies:** 0\ **Last updated:** [December 28, 2019, 11:51pm UTC](https://discuss.elastic.co/t/how-to-store-xml-input-to-multiples-indices/213271 "2019-12-28T23:51:48Z")
I have a deeply nested XML with few entities that I would to store in XML. Single XML file should be store to multiple indices. But I don't know how to split it to store to multiple indices. Output from this file, it wi… --- ## [Logstash performance bottleneck](https://discuss.elastic.co/t/logstash-performance-bottleneck/212855)
**Author:** [@carrot](https://discuss.elastic.co/u/carrot)\ **Replies:** 1\ **Last updated:** [December 28, 2019, 12:20pm UTC](https://discuss.elastic.co/t/logstash-performance-bottleneck/212855 "2019-12-28T12:20:08Z")
I'm using Kafka input with JSON Codec. Parsing is already done (dissect) from Filebeat. What I'm seeing is a thread called "Ruby-0-Thread-3" using 100% CPU. Overall CPU utilization isn't much (only that thread using 100… --- ## [Logstash from command line works, but not from service?](https://discuss.elastic.co/t/logstash-from-command-line-works-but-not-from-service/213234)
**Author:** [@dris](https://discuss.elastic.co/u/dris)\ **Replies:** 1\ **Last updated:** [December 28, 2019, 11:21am UTC](https://discuss.elastic.co/t/logstash-from-command-line-works-but-not-from-service/213234 "2019-12-28T11:21:50Z")
So, I have a conf file for checking a mysql DB and adding new records. It works great when I run it via the terminal (bin/logstash -f \) but when I move the .conf in the conf.d folder, it never runs. I check t… --- ## [Number of Pipelines logstash can have?](https://discuss.elastic.co/t/number-of-pipelines-logstash-can-have/213165)
**Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 1\ **Last updated:** [December 28, 2019, 8:07am UTC](https://discuss.elastic.co/t/number-of-pipelines-logstash-can-have/213165 "2019-12-28T08:07:37Z")
Hi Team, How many pipelines can we have in pipelines.yml in logstash Thanks --- ## [How to aggregate a complicated json log?](https://discuss.elastic.co/t/how-to-aggregate-a-complicated-json-log/213249)
**Author:** [@Ilia](https://discuss.elastic.co/u/Ilia)\ **Replies:** 0\ **Last updated:** [December 28, 2019, 7:14am UTC](https://discuss.elastic.co/t/how-to-aggregate-a-complicated-json-log/213249 "2019-12-28T07:14:40Z")
I want to aggregate some logs with different number, type and level of fields that have one common id field. Each line of log is relatively big and has complicated nested json structure. I've provided a simplified exampl… --- ## [How to do more extensive unit testing](https://discuss.elastic.co/t/how-to-do-more-extensive-unit-testing/213240)
**Author:** [@Brian\_Seel](https://discuss.elastic.co/u/Brian_Seel)\ **Replies:** 0\ **Last updated:** [December 28, 2019, 2:29am UTC](https://discuss.elastic.co/t/how-to-do-more-extensive-unit-testing/213240 "2019-12-28T02:29:13Z")
I am doing some extensive data enrichment in my logstash pipeline, where I pull in data from a rest endpoint, generate 50-100 events, and need to enrich each event with data from a text file I load into memory. This all … --- ## [Logstash Exec input plugin doesn't run two scripts at the same time](https://discuss.elastic.co/t/logstash-exec-input-plugin-doesnt-run-two-scripts-at-the-same-time/211548)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 2\ **Last updated:** [December 28, 2019, 1:47am UTC](https://discuss.elastic.co/t/logstash-exec-input-plugin-doesnt-run-two-scripts-at-the-same-time/211548 "2019-12-28T01:47:29Z")
Here is my scenario for using Logstash Exec input plugin, I have two scripts scheduled at 15 21 \* \* \* UTC time, but the Logstash Ruby output plugin result only shows one of the scripts as an output. Does each Exec plugi… --- ## [Use the log time and date in the timestamp](https://discuss.elastic.co/t/use-the-log-time-and-date-in-the-timestamp/213223)
**Author:** [@Daniel\_Calisto](https://discuss.elastic.co/u/Daniel_Calisto)\ **Replies:** 1\ **Last updated:** [December 28, 2019, 12:38am UTC](https://discuss.elastic.co/t/use-the-log-time-and-date-in-the-timestamp/213223 "2019-12-28T00:38:45Z")
Hi, Im trying to use the date and time in the log as the timestamp in elastic. this is part of the log 12/27/19 06:46:39 this is the grok and the date filter to use the log date and time to use in the timestamp grok{ … --- ## [Parsing log with logstash](https://discuss.elastic.co/t/parsing-log-with-logstash/212280)
**Author:** [@volcano](https://discuss.elastic.co/u/volcano)\ **Replies:** 34\ **Last updated:** [December 27, 2019, 11:59am UTC](https://discuss.elastic.co/t/parsing-log-with-logstash/212280 "2019-12-27T11:59:25Z")
How would you parse this key value data ? ( // this log data is in a single line) myapp.myproject.notice.student.request-time = 2019-12-13 12:37:01.4 # myapp.myproject.notice.student.response-time = 2019-12-13 12:37:19… --- ## [Send logs From Filebeat to 2 different logstash servers simultaneously](https://discuss.elastic.co/t/send-logs-from-filebeat-to-2-different-logstash-servers-simultaneously/212572)
**Author:** [@saravananveera](https://discuss.elastic.co/u/saravananveera)\ **Replies:** 2\ **Last updated:** [December 27, 2019, 3:27pm UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-2-different-logstash-servers-simultaneously/212572 "2019-12-27T15:27:03Z")
Hi Team, My requirement is to configure filebeat output as two logstash servers and I need to send logs to defined 2 servers simultaneously. It is not load balance scenario. Both logstash servers should received same da… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=425) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=427)