# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=591 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 592 --- ## [Configuration Logstash in windows machine](https://discuss.elastic.co/t/configuration-logstash-in-windows-machine/167342)
**Author:** [@NIKHIL\_KULSHRESTHA](https://discuss.elastic.co/u/NIKHIL_KULSHRESTHA)\ **Replies:** 2\ **Last updated:** [March 1, 2019, 7:06pm UTC](https://discuss.elastic.co/t/configuration-logstash-in-windows-machine/167342 "2019-03-01T19:06:15Z")
I have downloaded the logstash and extract and run the command logstash -f logstash-simple.conf and getting Error: Could not find or load main class Kulshrestha\\Desktop\\Dev Could you please assist. --- ## [Possible to convert exceptions from multiple logs into one single log in elastic?](https://discuss.elastic.co/t/possible-to-convert-exceptions-from-multiple-logs-into-one-single-log-in-elastic/170524)
**Author:** [@roblopes](https://discuss.elastic.co/u/roblopes)\ **Replies:** 2\ **Last updated:** [March 1, 2019, 3:36pm UTC](https://discuss.elastic.co/t/possible-to-convert-exceptions-from-multiple-logs-into-one-single-log-in-elastic/170524 "2019-03-01T15:36:43Z")
We are planning to stream Pivotal Cloud Foundry logs to Logstash, however looking into some log examples when an exception is logged, instead of sending one single log, it is sending each line of the exception as one log… --- ## [Latency with 2 Elasticsearch systems](https://discuss.elastic.co/t/latency-with-2-elasticsearch-systems/170074)
**Author:** [@krknopp](https://discuss.elastic.co/u/krknopp)\ **Replies:** 2\ **Last updated:** [March 1, 2019, 1:41pm UTC](https://discuss.elastic.co/t/latency-with-2-elasticsearch-systems/170074 "2019-03-01T13:41:31Z")
I have the requirement of collecting logs and then sending some (based on source log file path name) to one elasticsearch server and the rest to another one. In order to accomplish this, I'm using Filebeat to send to a … --- ## [Filter logs using logstash](https://discuss.elastic.co/t/filter-logs-using-logstash/170459)
**Author:** [@Costi](https://discuss.elastic.co/u/Costi)\ **Replies:** 8\ **Last updated:** [March 1, 2019, 1:36pm UTC](https://discuss.elastic.co/t/filter-logs-using-logstash/170459 "2019-03-01T13:36:03Z")
Hi! I am using logstash to ingest some data from a sql DB. I am able to get the data in elastic but i want to get a substring from a filed and put it into another field. For example i have the following line: {"Timesta… --- ## [Grok gives an error, causing cpu to be high](https://discuss.elastic.co/t/grok-gives-an-error-causing-cpu-to-be-high/170465)
**Author:** [@shenshensinian](https://discuss.elastic.co/u/shenshensinian)\ **Replies:** 6\ **Last updated:** [March 1, 2019, 1:02pm UTC](https://discuss.elastic.co/t/grok-gives-an-error-causing-cpu-to-be-high/170465 "2019-03-01T13:02:53Z")
Hello, I don't know why I set the matching specifications, logstash keeps getting error log and cpu usage rate rises rapidly. The error log is as follows: \[WARN \]\[logstash.filters.grok \] Timeout executing grok '(.?)… --- ## [Logstash config reload is taking too much time and logs are being lost during reload](https://discuss.elastic.co/t/logstash-config-reload-is-taking-too-much-time-and-logs-are-being-lost-during-reload/170460)
**Author:** [@liptanbiswas](https://discuss.elastic.co/u/liptanbiswas)\ **Replies:** 1\ **Last updated:** [March 1, 2019, 12:39pm UTC](https://discuss.elastic.co/t/logstash-config-reload-is-taking-too-much-time-and-logs-are-being-lost-during-reload/170460 "2019-03-01T12:39:42Z")
Hello, We are using logspout to ship container logs to logstash from where logstash is filtering and sending to Elastic cluster hosted on elastic cloud. We have only one input in which all container logs are being pushe… --- ## [Sending Watchguard Firewall logs to the logstash](https://discuss.elastic.co/t/sending-watchguard-firewall-logs-to-the-logstash/170496)
**Author:** [@DavidLevi](https://discuss.elastic.co/u/DavidLevi)\ **Replies:** 0\ **Last updated:** [March 1, 2019, 12:36pm UTC](https://discuss.elastic.co/t/sending-watchguard-firewall-logs-to-the-logstash/170496 "2019-03-01T12:36:41Z")
Hello guys, How can I send my firewall Watchguard Firebox X505 logs to the Logstash? Many Thanks, --- ## [How to mask sensitive data in logstash](https://discuss.elastic.co/t/how-to-mask-sensitive-data-in-logstash/170333)
**Author:** [@manickalai](https://discuss.elastic.co/u/manickalai)\ **Replies:** 2\ **Last updated:** [March 1, 2019, 12:27pm UTC](https://discuss.elastic.co/t/how-to-mask-sensitive-data-in-logstash/170333 "2019-03-01T12:27:51Z")
i have to mask the Card no : 712345678901236, OTP : 1234, expiryDate:01/01/2018, cvv:000, MobileNo:1234567890 the masked data should be Card no : 71231236, OTP : 14, expiryDate://20, cvv:\*\*, MobileNo:12\*\*\*\*\*\*\*90 kindl… --- ## [Increment field no work](https://discuss.elastic.co/t/increment-field-no-work/170457)
**Author:** [@hermann102](https://discuss.elastic.co/u/hermann102)\ **Replies:** 1\ **Last updated:** [March 1, 2019, 11:27am UTC](https://discuss.elastic.co/t/increment-field-no-work/170457 "2019-03-01T11:27:02Z")
Hello All, I would like to know how to increment a field when running an event in logstash. I tried this, but it does not work. if \[Process\] == "status"{ script =\> "ctx.\_source.Rej\_status++" } I want to incremen… --- ## [Can't merge a non object mapping ... with an object mapping ...but mapping still do not exists](https://discuss.elastic.co/t/cant-merge-a-non-object-mapping-with-an-object-mapping-but-mapping-still-do-not-exists/170446)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 2\ **Last updated:** [March 1, 2019, 10:57am UTC](https://discuss.elastic.co/t/cant-merge-a-non-object-mapping-with-an-object-mapping-but-mapping-still-do-not-exists/170446 "2019-03-01T10:57:45Z")
Hi all, When parsing an XML, I have an error "Can't merge a non object mapping \[poc.ports.port.script.elem\] with an object mapping \[poc.ports.port.script.elem\]" Searching in the forum I found that the error appears wh… --- ## [Moving from logstash-1.5.4 to 6.x | how do i convert filter?](https://discuss.elastic.co/t/moving-from-logstash-1-5-4-to-6-x-how-do-i-convert-filter/170454)
**Author:** [@Pierre\_es](https://discuss.elastic.co/u/Pierre_es)\ **Replies:** 1\ **Last updated:** [March 1, 2019, 8:45am UTC](https://discuss.elastic.co/t/moving-from-logstash-1-5-4-to-6-x-how-do-i-convert-filter/170454 "2019-03-01T08:45:15Z")
We have been running elk on Windows for years and its come to the point where we cant upgrade. The filters dont work at all. instead of using beats i tried to mount the shares again but the performance is terrible. If … --- ## [Logstash Conditional Indexes](https://discuss.elastic.co/t/logstash-conditional-indexes/170080)
**Author:** [@luke14](https://discuss.elastic.co/u/luke14)\ **Replies:** 4\ **Last updated:** [March 1, 2019, 3:57am UTC](https://discuss.elastic.co/t/logstash-conditional-indexes/170080 "2019-03-01T03:57:25Z")
I am having great difficulty parsing my Filebeat input into separate Indices dependant on some custom fields. My IIS server has filebeat installed and the filebeat.yml looks like this: filebeat.inputs: enabled: true … --- ## [Logstash open\_file handle issue For Linux](https://discuss.elastic.co/t/logstash-open-file-handle-issue-for-linux/168959)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 22\ **Last updated:** [March 1, 2019, 3:14am UTC](https://discuss.elastic.co/t/logstash-open-file-handle-issue-for-linux/168959 "2019-03-01T03:14:21Z")
I'm facing until recently a very peculiar warning on the logstash log file \[filewatch.tailmode.handlers.createinitial\] open\_file OPEN\_WARN\_INTERVAL is '300', i've searched around the web for every possible help to dig i… --- ## [How to compare dates without timestamp](https://discuss.elastic.co/t/how-to-compare-dates-without-timestamp/170102)
**Author:** [@adityaPsl](https://discuss.elastic.co/u/adityaPsl)\ **Replies:** 9\ **Last updated:** [March 1, 2019, 1:32am UTC](https://discuss.elastic.co/t/how-to-compare-dates-without-timestamp/170102 "2019-03-01T01:32:07Z")
Hello Team, I want to compare dates with @timestamp , but I don't want to compare the time , only the date part I want to compare. I tried using date filter but could not set date in dd/MM/YY format any pointers to re… --- ## [Logstash not able to create index on Elastic](https://discuss.elastic.co/t/logstash-not-able-to-create-index-on-elastic/170417)
**Author:** [@vijayakrishna.rg](https://discuss.elastic.co/u/vijayakrishna.rg)\ **Replies:** 6\ **Last updated:** [March 1, 2019, 12:03am UTC](https://discuss.elastic.co/t/logstash-not-able-to-create-index-on-elastic/170417 "2019-03-01T00:03:52Z")
Hi, I have installed logstash on my elastic machine and pointed to one of the log file as input, i have started logstash service, i don't see any logs coming into elastic either index in elastic, when i tail logstash lo… --- ## [Logstash StatsD Plugin caches DNS resolutions when using UDP forever](https://discuss.elastic.co/t/logstash-statsd-plugin-caches-dns-resolutions-when-using-udp-forever/170413)
**Author:** [@SpencerMalone](https://discuss.elastic.co/u/SpencerMalone)\ **Replies:** 0\ **Last updated:** [February 28, 2019, 10:06pm UTC](https://discuss.elastic.co/t/logstash-statsd-plugin-caches-dns-resolutions-when-using-udp-forever/170413 "2019-02-28T22:06:19Z")
The recent(ish) release of https://github.com/logstash-plugins/logstash-output-statsd to 3.2.0 used the latest release of https://github.com/reinh/statsd/blob/master/lib/statsd.rb, which changed the socket logic to reuse… --- ## [File input randomly skipping files](https://discuss.elastic.co/t/file-input-randomly-skipping-files/170228)
**Author:** [@Bob\_Nicksic](https://discuss.elastic.co/u/Bob_Nicksic)\ **Replies:** 9\ **Last updated:** [February 28, 2019, 10:02pm UTC](https://discuss.elastic.co/t/file-input-randomly-skipping-files/170228 "2019-02-28T22:02:50Z")
I'm using the file input to ingest cloudflare logs into elasticsearch. I have a cronjob that runs every 10 minutes and dumps a gzip file ranging from a few hundred KB to a couple MB in size into the read path. Logstash r… --- ## [Write grok filter conf file](https://discuss.elastic.co/t/write-grok-filter-conf-file/170405)
**Author:** [@Rajashekar](https://discuss.elastic.co/u/Rajashekar)\ **Replies:** 0\ **Last updated:** [February 28, 2019, 8:37pm UTC](https://discuss.elastic.co/t/write-grok-filter-conf-file/170405 "2019-02-28T20:37:47Z")
Hello all, I need your help writing a correct parsing filter configuration file of one of my log. Below is the message log: \<102\>Feb 10 18:22:27 abc-rbjyyyb5demo2a info logger: \[xxl\_acc\] 120.0.0.1 - - \[10/Feb/2019:18:2… --- ## [\[resolved\] No AWS ELB logs in the elasticsearch](https://discuss.elastic.co/t/resolved-no-aws-elb-logs-in-the-elasticsearch/170266)
**Author:** [@divyakaushik](https://discuss.elastic.co/u/divyakaushik)\ **Replies:** 1\ **Last updated:** [February 28, 2019, 8:13pm UTC](https://discuss.elastic.co/t/resolved-no-aws-elb-logs-in-the-elasticsearch/170266 "2019-02-28T20:13:20Z")
Stack Version: 6.6 I am not able to see logs in elasticsearch/kibana, even though they get successfully pasred in logstash. In the logs, I can see that the logstash is able to get logs from the ELB. It also prints outp… --- ## [Unexpected character '\<'](https://discuss.elastic.co/t/unexpected-character/170366)
**Author:** [@colix](https://discuss.elastic.co/u/colix)\ **Replies:** 2\ **Last updated:** [February 28, 2019, 8:00pm UTC](https://discuss.elastic.co/t/unexpected-character/170366 "2019-02-28T20:00:15Z")
Hi, i sent json log to logstash. Logstash config: input { udp { port =\> 5001 codec =\> "json" } } filter { json { source =\> "message" } } output { if \[type\] == "rsyslog" { elasticsearch { hosts =\> \[ "elastic… --- ## [\[resolved\]RSS filter and dedupe](https://discuss.elastic.co/t/resolved-rss-filter-and-dedupe/168856)
**Author:** [@devmal044](https://discuss.elastic.co/u/devmal044)\ **Replies:** 1\ **Last updated:** [February 28, 2019, 7:54pm UTC](https://discuss.elastic.co/t/resolved-rss-filter-and-dedupe/168856 "2019-02-28T19:54:05Z")
Im trying to pull information from an RSS feed. I am running into 2 problems. The first is I am getting duplicates entries every time the Logstash interval is hit. What is the best way to handle this? The second is… --- ## [Possibility to select "NULL" in a log file](https://discuss.elastic.co/t/possibility-to-select-null-in-a-log-file/170390)
**Author:** [@Gegi\_K](https://discuss.elastic.co/u/Gegi_K)\ **Replies:** 2\ **Last updated:** [February 28, 2019, 5:40pm UTC](https://discuss.elastic.co/t/possibility-to-select-null-in-a-log-file/170390 "2019-02-28T17:40:25Z")
Dear All, I may ask for the impossible, but maybe you can help me out... Even formulating the title bit hard and it may be misleading. I have a logfile structure of a firewall that has some possible actions: Allow; Dr… --- ## [Create grok filter conf file](https://discuss.elastic.co/t/create-grok-filter-conf-file/170253)
**Author:** [@Rajashekar](https://discuss.elastic.co/u/Rajashekar)\ **Replies:** 0\ **Last updated:** [February 27, 2019, 10:21pm UTC](https://discuss.elastic.co/t/create-grok-filter-conf-file/170253 "2019-02-27T22:21:36Z")
Hi Everyone, I'm new to Log stash and Grok Debugger. I need your help writing a correct parsing filter configuration file of one of my log. Below is the message log: \<102\>Feb 10 18:22:27 abc-rbjyyyb5demo2a info logger:… --- ## [How to pass ROWNUM dynamically in LS command line argument](https://discuss.elastic.co/t/how-to-pass-rownum-dynamically-in-ls-command-line-argument/170274)
**Author:** [@senanup](https://discuss.elastic.co/u/senanup)\ **Replies:** 2\ **Last updated:** [February 28, 2019, 3:21pm UTC](https://discuss.elastic.co/t/how-to-pass-rownum-dynamically-in-ls-command-line-argument/170274 "2019-02-28T15:21:12Z")
Hi, I want to pass a dynamic parameter like ROWNUM in the command line argument so, that the relevant query can fetch the number of rows based on that. Is this achievable ? I am using Windows 10. Kindly help. Below th… --- ## [Grok parse rule succeeding in debugger, but not in Logstash](https://discuss.elastic.co/t/grok-parse-rule-succeeding-in-debugger-but-not-in-logstash/170329)
**Author:** [@Nik\_Gupta](https://discuss.elastic.co/u/Nik_Gupta)\ **Replies:** 4\ **Last updated:** [February 28, 2019, 3:14pm UTC](https://discuss.elastic.co/t/grok-parse-rule-succeeding-in-debugger-but-not-in-logstash/170329 "2019-02-28T15:14:14Z")
Hey, would really appreciate some help debugging this. Example log line: 2019-02-24 17:58:30,396 INFO \[i.b.logging.LoggingComponent\] \[\] \[\] \[\] : The log message Parse rule that succeeds in the Grok debugger but fails f… --- ## [Formatting jdbc-streaming filter result as a field instead of a list](https://discuss.elastic.co/t/formatting-jdbc-streaming-filter-result-as-a-field-instead-of-a-list/170305)
**Author:** [@yayun](https://discuss.elastic.co/u/yayun)\ **Replies:** 1\ **Last updated:** [February 28, 2019, 2:23pm UTC](https://discuss.elastic.co/t/formatting-jdbc-streaming-filter-result-as-a-field-instead-of-a-list/170305 "2019-02-28T14:23:19Z")
Hi, I trying to enrich the log with data (username) from postgres by user id. However I didn't find a way to add the result as a field instead of a list. Is there a way to do so with the jdbc\_streaming filter? Actual … --- ## [Creating index depends on source](https://discuss.elastic.co/t/creating-index-depends-on-source/167061)
**Author:** [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Replies:** 6\ **Last updated:** [February 28, 2019, 2:15pm UTC](https://discuss.elastic.co/t/creating-index-depends-on-source/167061 "2019-02-28T14:15:21Z")
I am new to Elastic Stack.Using version :6.5.4 Data flow like : Filebeat -\> Logstash -\> ES-\> Kibana Configured Filebeat on SIT and UAT environment to common Log server. Would like to create separate index for SIT and … --- ## [Multiple Logstash config files causing field leak despite conditionals](https://discuss.elastic.co/t/multiple-logstash-config-files-causing-field-leak-despite-conditionals/170341)
**Author:** [@Anish\_Sujanani](https://discuss.elastic.co/u/Anish_Sujanani)\ **Replies:** 1\ **Last updated:** [February 28, 2019, 2:08pm UTC](https://discuss.elastic.co/t/multiple-logstash-config-files-causing-field-leak-despite-conditionals/170341 "2019-02-28T14:08:09Z")
Hi everyone, I have 2 different application files to be monitored and on-boarded daily. I started off with creating 2 separate config files for the 2 applications, but have now moved to having 3 files - for input, filte… --- ## [Filter mutate add\_field from nested field](https://discuss.elastic.co/t/filter-mutate-add-field-from-nested-field/170231)
**Author:** [@Welton\_Leao\_Machado](https://discuss.elastic.co/u/Welton_Leao_Machado)\ **Replies:** 3\ **Last updated:** [February 28, 2019, 2:06pm UTC](https://discuss.elastic.co/t/filter-mutate-add-field-from-nested-field/170231 "2019-02-28T14:06:19Z")
i'm trying to catch a nested field to add in a new field with mutate add\_field So, i have the follow data "beat" =\> { "name" =\> "LBR001001-172.net.mapfre.com.br", "hostname" =\> "LBR001001-172.net.mapfre.com.br", "v… --- ## [Logstash not ingesting data to elasticsearch when run as daemon](https://discuss.elastic.co/t/logstash-not-ingesting-data-to-elasticsearch-when-run-as-daemon/170356)
**Author:** [@itssameer](https://discuss.elastic.co/u/itssameer)\ **Replies:** 0\ **Last updated:** [February 28, 2019, 1:38pm UTC](https://discuss.elastic.co/t/logstash-not-ingesting-data-to-elasticsearch-when-run-as-daemon/170356 "2019-02-28T13:38:44Z")
Logstash not ingesting data to elasticsearch when run as daemon and it does show any error as well. But its ingesting data to elasticsearch when ran from terminal as /bin/logstash -f /path/to/conf/file. Below is the con… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=590) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=592)