# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=603 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 604 --- ## [Logstash IF Statement](https://discuss.elastic.co/t/logstash-if-statement/167869)
**Author:** [@luiz.miguelsl](https://discuss.elastic.co/u/luiz.miguelsl)\ **Replies:** 2\ **Last updated:** [February 11, 2019, 2:10pm UTC](https://discuss.elastic.co/t/logstash-if-statement/167869 "2019-02-11T14:10:03Z")
What's up everyone?! I wish you're all doing great. Take a look at this problem I'm facing. I'm loading a few log files from an Apache to ELK via Logstash. The process is pretty simple, I use the input to read the file … --- ## [Error while loading data via logstash](https://discuss.elastic.co/t/error-while-loading-data-via-logstash/167859)
**Author:** [@Kushagra\_Makharia](https://discuss.elastic.co/u/Kushagra_Makharia)\ **Replies:** 5\ **Last updated:** [February 11, 2019, 2:01pm UTC](https://discuss.elastic.co/t/error-while-loading-data-via-logstash/167859 "2019-02-11T14:01:04Z")
(base) apples-imac:logstash-6.6.0 apple$ bin/logstash -f /Users/apple/Downloads/logstash-6.6.0/data/app.config Sending Logstash logs to /Users/apple/Downloads/logstash-6.6.0/logs which is now configured via log4j2.prope… --- ## [Can I reference a variable set in my GROK statement in an if statement later on in the filter?](https://discuss.elastic.co/t/can-i-reference-a-variable-set-in-my-grok-statement-in-an-if-statement-later-on-in-the-filter/167786)
**Author:** [@sdvincent](https://discuss.elastic.co/u/sdvincent)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 12:54pm UTC](https://discuss.elastic.co/t/can-i-reference-a-variable-set-in-my-grok-statement-in-an-if-statement-later-on-in-the-filter/167786 "2019-02-11T12:54:56Z")
The goal is to parse a line from syslog and if the program is sshd, parse the extracted message to grab the source IP. My grok line looks like this: filter { if \[fields\]\[type\] == "syslog" { grok { match =\> {… --- ## [Wrong format of MAC address using SNMPTRAP input](https://discuss.elastic.co/t/wrong-format-of-mac-address-using-snmptrap-input/167836)
**Author:** [@rudok](https://discuss.elastic.co/u/rudok)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 12:53pm UTC](https://discuss.elastic.co/t/wrong-format-of-mac-address-using-snmptrap-input/167836 "2019-02-11T12:53:12Z")
Our Logstash output contains "invalid" characters on field containing mac address. Is there any filter how to covert it to readable format? Thx logstash output: { "HP-ICF-MACNOTIFY-MIB::hpicfMacNotifyFromPortId.0" =\> … --- ## [\[JSON filter\] Split Dynamic fields](https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774)
**Author:** [@manunc](https://discuss.elastic.co/u/manunc)\ **Replies:** 3\ **Last updated:** [February 11, 2019, 12:37pm UTC](https://discuss.elastic.co/t/json-filter-split-dynamic-fields/167774 "2019-02-11T12:37:13Z")
Hello, I try to create an event for each fields named "XXX.YY" coming from the following JSON message (data object): {"data":{ "316.2":\[{"sensorId":"316.2.3","sensorTime":1549599391,"sensorValue":0.08}\], "315.10":\[{"… --- ## [Missing Mapping caused by an updated Translation file - Possible to Reindex without Restart all?](https://discuss.elastic.co/t/missing-mapping-caused-by-an-updated-translation-file-possible-to-reindex-without-restart-all/165895)
**Author:** [@Datakids](https://discuss.elastic.co/u/Datakids)\ **Replies:** 6\ **Last updated:** [February 11, 2019, 12:28pm UTC](https://discuss.elastic.co/t/missing-mapping-caused-by-an-updated-translation-file-possible-to-reindex-without-restart-all/165895 "2019-02-11T12:28:58Z")
Hi there, I just checked the ES Put Mapping API but that's like "Böhmische Dörfer" for me atm. What I try to do is following: My ES Index name = "dict\_c" In my logstash conf: translate { field =\> "\[Präfix\_Coun… --- ## [IOError in WebHDFS - Output Plugin](https://discuss.elastic.co/t/ioerror-in-webhdfs-output-plugin/167845)
**Author:** [@alenthampi](https://discuss.elastic.co/u/alenthampi)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 12:24pm UTC](https://discuss.elastic.co/t/ioerror-in-webhdfs-output-plugin/167845 "2019-02-11T12:24:06Z")
I am facing the following error while shipping data into hadoop. \[2019-02-11T17:45:49,731\]\[WARN \]\[logstash.outputs.webhdfs \] Failed to flush outgoing items {:outgoing\_count=\>1, :exception=\>"WebHDFS::IOError", :backtrace… --- ## [Memory issues or is it?](https://discuss.elastic.co/t/memory-issues-or-is-it/167808)
**Author:** [@Jasna\_Bencic](https://discuss.elastic.co/u/Jasna_Bencic)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 9:06am UTC](https://discuss.elastic.co/t/memory-issues-or-is-it/167808 "2019-02-11T09:06:09Z")
You can close this topic, I don't see a button for deletion. I have posted a note to discover the problem for memory but then I have realized that ELK ate all the disk, not memory. Sorry. --- ## [Sending Logstash data to multiple servers](https://discuss.elastic.co/t/sending-logstash-data-to-multiple-servers/167639)
**Author:** [@The1](https://discuss.elastic.co/u/The1)\ **Replies:** 9\ **Last updated:** [February 11, 2019, 8:37am UTC](https://discuss.elastic.co/t/sending-logstash-data-to-multiple-servers/167639 "2019-02-11T08:37:17Z")
Hi! I would like to know if it is possible to send collected data from Logstash to multiple destinations outside of my cluster which run Elasticsearch. Many thanks! --- ## [Offline plugin install from github .gem](https://discuss.elastic.co/t/offline-plugin-install-from-github-gem/167804)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 7:52am UTC](https://discuss.elastic.co/t/offline-plugin-install-from-github-gem/167804 "2019-02-11T07:52:13Z")
\[ Hi, I am installing cloudwatch-logs plugin given at https://github.com/lukewaite/logstash-input-cloudwatch-logs I followed the steps https://github.com/lukewaite/logstash-input-cloudwatch-logs/blob/master/DEVELOPER.… --- ## [How to split into multiple events dynamically for a given json? Tried from various question in forums](https://discuss.elastic.co/t/how-to-split-into-multiple-events-dynamically-for-a-given-json-tried-from-various-question-in-forums/167745)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 9\ **Last updated:** [February 11, 2019, 7:28am UTC](https://discuss.elastic.co/t/how-to-split-into-multiple-events-dynamically-for-a-given-json-tried-from-various-question-in-forums/167745 "2019-02-11T07:28:29Z")
Hi All These question has been asked multiple times in forum as i see but no definite answer. I saw one of the Elastic member answered in a nice way, but it is not working for me. I have a json data in this format com… --- ## [Time Diff Calculation in ruby not working](https://discuss.elastic.co/t/time-diff-calculation-in-ruby-not-working/167775)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [February 10, 2019, 11:37pm UTC](https://discuss.elastic.co/t/time-diff-calculation-in-ruby-not-working/167775 "2019-02-10T23:37:53Z")
Hi, What is wrong in the following filter lines that I am not able to find the TimeDiff field in the output? date { match =\> \[ "Criado em", "dd/MM/yyyy HH:mm:ss", "ISO8601"\] target =\> "Create… --- ## [Could not find or load main class Files\\logstash-6.6.0\\logstash-core\\lib\\jars\\commons-codec-1.11.jar](https://discuss.elastic.co/t/could-not-find-or-load-main-class-files-logstash-6-6-0-logstash-core-lib-jars-commons-codec-1-11-jar/167738)
**Author:** [@Joji\_P\_Cherian](https://discuss.elastic.co/u/Joji_P_Cherian)\ **Replies:** 0\ **Last updated:** [February 10, 2019, 6:01am UTC](https://discuss.elastic.co/t/could-not-find-or-load-main-class-files-logstash-6-6-0-logstash-core-lib-jars-commons-codec-1-11-jar/167738 "2019-02-10T06:01:36Z")
Hi, When I run the cmd bin\\logstash.bat -f logstash.conf , I get the error " Could not find or load main class Files\\logstash-6.6.0\\logstash-core\\lib\\jars\\commons-codec-1.11.jar" . Kindly Help --- ## [Cannot Start Logstash](https://discuss.elastic.co/t/cannot-start-logstash/167697)
**Author:** [@Murr](https://discuss.elastic.co/u/Murr)\ **Replies:** 4\ **Last updated:** [February 9, 2019, 3:19pm UTC](https://discuss.elastic.co/t/cannot-start-logstash/167697 "2019-02-09T15:19:16Z")
I cannot install logstash successfully. I tried to do this sudo /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd The output is Using provided startup.options file: /etc/logstash/startup.o… --- ## [I cannot start logstash](https://discuss.elastic.co/t/i-cannot-start-logstash/167720)
**Author:** [@Murr](https://discuss.elastic.co/u/Murr)\ **Replies:** 1\ **Last updated:** [February 9, 2019, 3:06pm UTC](https://discuss.elastic.co/t/i-cannot-start-logstash/167720 "2019-02-09T15:06:53Z")
1.I tried to do this sudo /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd The output is Using provided startup.options file: /etc/logstash/startup.options Manually creating startup for spe… --- ## [Upgrade Logstash 6.4.2 to 6.6](https://discuss.elastic.co/t/upgrade-logstash-6-4-2-to-6-6/167663)
**Author:** [@bharatchawla87](https://discuss.elastic.co/u/bharatchawla87)\ **Replies:** 1\ **Last updated:** [February 8, 2019, 11:42pm UTC](https://discuss.elastic.co/t/upgrade-logstash-6-4-2-to-6-6/167663 "2019-02-08T23:42:50Z")
I am planning to upgrade the elasticsearch cloud cluster to latest 6.6. I have logstash installed on ubuntu which is 6.4.2. Should I need to upgrade that as well. --- ## [Using Pipelines or not .. a shakespearean dilemma!](https://discuss.elastic.co/t/using-pipelines-or-not-a-shakespearean-dilemma/167668)
**Author:** [@nandrik](https://discuss.elastic.co/u/nandrik)\ **Replies:** 0\ **Last updated:** [February 8, 2019, 4:50pm UTC](https://discuss.elastic.co/t/using-pipelines-or-not-a-shakespearean-dilemma/167668 "2019-02-08T16:50:51Z")
I have two different applications running on two different servers, shipping logs and exposing server API information to Logstash. The output from each application has a different data rate. Let's say: App1 sends data… --- ## [Interval Setting in http\_poller is invalid](https://discuss.elastic.co/t/interval-setting-in-http-poller-is-invalid/167553)
**Author:** [@imortalsolitude](https://discuss.elastic.co/u/imortalsolitude)\ **Replies:** 7\ **Last updated:** [February 8, 2019, 8:50pm UTC](https://discuss.elastic.co/t/interval-setting-in-http-poller-is-invalid/167553 "2019-02-08T20:50:25Z")
I am trying to use http\_poller plugin to get data at 30 sec internval however its giving me an erorr ""The setting interval in plugin http\_poller is obsolete and is no longer available. The interval options is obsolete."… --- ## [One input to multiple outputs with different transforms](https://discuss.elastic.co/t/one-input-to-multiple-outputs-with-different-transforms/167675)
**Author:** [@jenyphur](https://discuss.elastic.co/u/jenyphur)\ **Replies:** 2\ **Last updated:** [February 8, 2019, 6:24pm UTC](https://discuss.elastic.co/t/one-input-to-multiple-outputs-with-different-transforms/167675 "2019-02-08T18:24:38Z")
Hey there, pretty sure this can be done but just want to get some direction. I need Logstash to take in one input and transform it in 2 different ways to end up in 2 different outputs. I know I can use two outputs just… --- ## [How to test if fieldNames contains specific caracter?](https://discuss.elastic.co/t/how-to-test-if-fieldnames-contains-specific-caracter/167645)
**Author:** [@Isilve28](https://discuss.elastic.co/u/Isilve28)\ **Replies:** 1\ **Last updated:** [February 8, 2019, 3:22pm UTC](https://discuss.elastic.co/t/how-to-test-if-fieldnames-contains-specific-caracter/167645 "2019-02-08T15:22:26Z")
Hello everyone, I want to use date filter for every fieldName ending with "Date" (and keeping the value in the same fieldName) because I'm using a kv filter at first and I don't know all the time if a field is numeric … --- ## [Help with starting logstash with my config](https://discuss.elastic.co/t/help-with-starting-logstash-with-my-config/167442)
**Author:** [@hHelen](https://discuss.elastic.co/u/hHelen)\ **Replies:** 3\ **Last updated:** [February 8, 2019, 1:44pm UTC](https://discuss.elastic.co/t/help-with-starting-logstash-with-my-config/167442 "2019-02-08T13:44:12Z")
I have installed logstash but struggling to get it working. my config is located here /etc/logstash/logstash-test.conf And the content is: input { beats { port =\> "5044" } } output { elasticsearch { hosts =\> \["l… --- ## [Corrupt gzipped input file causes index to grow and saturate storage](https://discuss.elastic.co/t/corrupt-gzipped-input-file-causes-index-to-grow-and-saturate-storage/167635)
**Author:** [@makibroshett](https://discuss.elastic.co/u/makibroshett)\ **Replies:** 0\ **Last updated:** [February 8, 2019, 1:38pm UTC](https://discuss.elastic.co/t/corrupt-gzipped-input-file-causes-index-to-grow-and-saturate-storage/167635 "2019-02-08T13:38:12Z")
Hi, I have a bug I'd like to report/feature I'd like to request (but where do we do this?). But before I'd like to know if some of you found a work-around. I've observed that on logstash 6.5.0 and 6.6.0. I'm using a lo… --- ## [After executing sudo bin/logstash -f /home/web/ghost\_files/trans\_client\_auth.config I get this error](https://discuss.elastic.co/t/after-executing-sudo-bin-logstash-f-home-web-ghost-files-trans-client-auth-config-i-get-this-error/167282)
**Author:** [@sachin\_augustine](https://discuss.elastic.co/u/sachin_augustine)\ **Replies:** 2\ **Last updated:** [February 8, 2019, 11:30am UTC](https://discuss.elastic.co/t/after-executing-sudo-bin-logstash-f-home-web-ghost-files-trans-client-auth-config-i-get-this-error/167282 "2019-02-08T11:30:00Z")
WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at p… --- ## [Logstash-plugin.bat install logstash-input-jdbc](https://discuss.elastic.co/t/logstash-plugin-bat-install-logstash-input-jdbc/167601)
**Author:** [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Replies:** 0\ **Last updated:** [February 8, 2019, 10:06am UTC](https://discuss.elastic.co/t/logstash-plugin-bat-install-logstash-input-jdbc/167601 "2019-02-08T10:06:42Z")
Hi, I am not able to install input jdbc plugin for logstash. i am getting an following error: ERROR: Something went wrong when installing logstash-input-jdbc, message: An established connection was aborted by the soft… --- ## [Unable to use if condition and calculate percenatge inside ruby code](https://discuss.elastic.co/t/unable-to-use-if-condition-and-calculate-percenatge-inside-ruby-code/167586)
**Author:** [@Sapna](https://discuss.elastic.co/u/Sapna)\ **Replies:** 0\ **Last updated:** [February 8, 2019, 8:38am UTC](https://discuss.elastic.co/t/unable-to-use-if-condition-and-calculate-percenatge-inside-ruby-code/167586 "2019-02-08T08:38:11Z")
Hello, I am trying to first filter my data with only 3 KPI and only with hostname which domain is ".xxx.com". then I want to calculate percentage from value\_current and value\_max columns. filter{ mutate {convert =\> \[… --- ## [Import Data from SQL Server 2014 to Elasticsearch - Driver not loaded](https://discuss.elastic.co/t/import-data-from-sql-server-2014-to-elasticsearch-driver-not-loaded/167424)
**Author:** [@Sergio\_PR](https://discuss.elastic.co/u/Sergio_PR)\ **Replies:** 2\ **Last updated:** [February 8, 2019, 7:39am UTC](https://discuss.elastic.co/t/import-data-from-sql-server-2014-to-elasticsearch-driver-not-loaded/167424 "2019-02-08T07:39:28Z")
Hi, I want to import data from SQL Server table to elasticsearch using logstash. But when I launch Logstash with my config file, the next message appears : My config file lauched with logstash is : I have specifi… --- ## [How to read two different timestamp](https://discuss.elastic.co/t/how-to-read-two-different-timestamp/167410)
**Author:** [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Replies:** 2\ **Last updated:** [February 8, 2019, 6:07am UTC](https://discuss.elastic.co/t/how-to-read-two-different-timestamp/167410 "2019-02-08T06:07:40Z")
Hi, I am getting issue while indexing the data My Log file : 2019-01-02 14:31:02 \[00000001\] info \[native\] Apache reports revision 2.2.8() 2019-01-02 14:31:02 \[00000001\] info \[native\] Apache reports version stri… --- ## [Parsing Pentaho logs using Grok in Logstash config](https://discuss.elastic.co/t/parsing-pentaho-logs-using-grok-in-logstash-config/167572)
**Author:** [@prasannnn](https://discuss.elastic.co/u/prasannnn)\ **Replies:** 0\ **Last updated:** [February 8, 2019, 5:50am UTC](https://discuss.elastic.co/t/parsing-pentaho-logs-using-grok-in-logstash-config/167572 "2019-02-08T05:50:24Z")
HI Team, I have installed ELK 6.4.2. I have Pentaho logs generated from my applications. I have successfully installed and configured Elastic, Kibana, Logstash and Filebeat. The generated logs are consumed by the Filebe… --- ## [Add new field to index using Logstash getting null values](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370)
**Author:** [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)\ **Replies:** 6\ **Last updated:** [February 7, 2019, 10:41pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-using-logstash-getting-null-values/167370 "2019-02-07T22:41:19Z")
Hi, currently i would like to add a new field gain in a current index based on a field value as follows: input { elasticsearch { hosts =\> "http://localhost:9200" index =\> "my\_index" } } filter { if \[ga… --- ## [Need help with parsing logs](https://discuss.elastic.co/t/need-help-with-parsing-logs/167367)
**Author:** [@dips](https://discuss.elastic.co/u/dips)\ **Replies:** 2\ **Last updated:** [February 7, 2019, 10:27pm UTC](https://discuss.elastic.co/t/need-help-with-parsing-logs/167367 "2019-02-07T22:27:04Z")
Hello I am trying to parse log file and wanted to check how I can stop at first encounter of ":" for Tag. 02-05 00:00:13.199 3162 3162 I audit : test: Loaded service\_contexts from /service\_contexts. Grok filter I ha… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=602) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=604)