# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1134 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1135 --- ## [Example for how logstash works](https://discuss.elastic.co/t/example-for-how-logstash-works/54309)
**Author:** [@a\_soong](https://discuss.elastic.co/u/a_soong)\ **Replies:** 1\ **Last updated:** [June 30, 2016, 3:00am UTC](https://discuss.elastic.co/t/example-for-how-logstash-works/54309 "2016-06-30T03:00:28Z")
does anyone have a resource regarding how logstash works. I have a high level understanding, but when I try to implement a configuration file, all I get is errors. The only thing I have working the way it is supposed to… --- ## [Gelf parsing failing](https://discuss.elastic.co/t/gelf-parsing-failing/54351)
**Author:** [@skebo5150](https://discuss.elastic.co/u/skebo5150)\ **Replies:** 0\ **Last updated:** [June 30, 2016, 1:13am UTC](https://discuss.elastic.co/t/gelf-parsing-failing/54351 "2016-06-30T01:13:50Z")
I get the following message in logstash for all of my gelf logging: message=\>"Gelfd failed to parse a message skipping", :exception=\>#, :backtrace=\>\["/opt/logstash/vendor/bundle/jruby/1.9/gems/gelfd-0.2.0/l ib/gelfd/chu… --- ## [Population](https://discuss.elastic.co/t/population/53698)
**Author:** [@Julio](https://discuss.elastic.co/u/Julio)\ **Replies:** 3\ **Last updated:** [June 29, 2016, 8:19pm UTC](https://discuss.elastic.co/t/population/53698 "2016-06-29T20:19:28Z")
My doubt is, How do I will populate the IPs with their correct lat/long to call the heat map correctly? Info: I just have the IP of my client when he/her open the e-mail and I get the lat/long. --- ## [How to write NULL value for a field](https://discuss.elastic.co/t/how-to-write-null-value-for-a-field/53885)
**Author:** [@trondhindenes](https://discuss.elastic.co/u/trondhindenes)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 8:14pm UTC](https://discuss.elastic.co/t/how-to-write-null-value-for-a-field/53885 "2016-06-29T20:14:32Z")
We're ingesting some logfiles where the system writes a "-" for fields without value. I'd like to replace this with a proper NULL in order to keep data as clean as possible. I've tried mutations with the following valu… --- ## [Logstash output conditions doesn't works](https://discuss.elastic.co/t/logstash-output-conditions-doesnt-works/53779)
**Author:** [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 8:07pm UTC](https://discuss.elastic.co/t/logstash-output-conditions-doesnt-works/53779 "2016-06-29T20:07:55Z")
Hi, My problem is that the conditions in my logstash conf file in the output section, seems to not work. here is my output section: output { if "topbeat" in \[tags\] { if "\_grokparsefailure"… --- ## [Number Format Exception when parsing mac address from Netflow](https://discuss.elastic.co/t/number-format-exception-when-parsing-mac-address-from-netflow/54192)
**Author:** [@Fidel\_Netflow](https://discuss.elastic.co/u/Fidel_Netflow)\ **Replies:** 3\ **Last updated:** [June 29, 2016, 4:52pm UTC](https://discuss.elastic.co/t/number-format-exception-when-parsing-mac-address-from-netflow/54192 "2016-06-29T16:52:29Z")
Hi, I am receiving the following error when trying to index netflow records: "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse \[netflow.staMacAddress\]", "caused\_by"=\>{"type"=\>"number\_format\_excep… --- ## [Ruby filter](https://discuss.elastic.co/t/ruby-filter/54313)
**Author:** [@DMJ](https://discuss.elastic.co/u/DMJ)\ **Replies:** 0\ **Last updated:** [June 29, 2016, 4:16pm UTC](https://discuss.elastic.co/t/ruby-filter/54313 "2016-06-29T16:16:01Z")
Is it possible to assign a class variable to an event field using ruby ? --- ## [Logstash : Grok : Out of sequence key:value filter](https://discuss.elastic.co/t/logstash-grok-out-of-sequence-key-value-filter/54269)
**Author:** [@hassanfarid](https://discuss.elastic.co/u/hassanfarid)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 3:17pm UTC](https://discuss.elastic.co/t/logstash-grok-out-of-sequence-key-value-filter/54269 "2016-06-29T15:17:14Z")
Hello, I have log lines that may report key:value pair out of order. 2016-06-01 08:52:52 \[info\]: key1:value1, key2:value2 2016-06-01 08:52:52 \[info\]: key3:value3, key2:value2 2016-06-01 08:52:52 \[info\]: key2:value2, ke… --- ## [\[SOLVED\] Doing conversion byte to mega byte with logstash](https://discuss.elastic.co/t/solved-doing-conversion-byte-to-mega-byte-with-logstash/54002)
**Author:** [@Alex\_Lum](https://discuss.elastic.co/u/Alex_Lum)\ **Replies:** 3\ **Last updated:** [June 29, 2016, 3:09pm UTC](https://discuss.elastic.co/t/solved-doing-conversion-byte-to-mega-byte-with-logstash/54002 "2016-06-29T15:09:41Z")
Hello all. I m doing a real time monitor for fortinet bandwith but sent and received packets are in byte. I make a sum of all but it's not easy readable in byte. So i need to convert all bytes in megabytes. Here's my… --- ## [Beats adds a tag](https://discuss.elastic.co/t/beats-adds-a-tag/54288)
**Author:** [@muhamadli302](https://discuss.elastic.co/u/muhamadli302)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 3:04pm UTC](https://discuss.elastic.co/t/beats-adds-a-tag/54288 "2016-06-29T15:04:01Z")
Hi, I'm using Logstash 2.1. This is my input conf: input { beats { port =\> 5044 type =\> "l2tcsv" codec =\> "plain" } } and in kibana, I see that it created a tag "beats\_input\_codec\_pla… --- ## [Logstash service stop working](https://discuss.elastic.co/t/logstash-service-stop-working/54051)
**Author:** [@William\_Leclerc](https://discuss.elastic.co/u/William_Leclerc)\ **Replies:** 3\ **Last updated:** [June 29, 2016, 2:45pm UTC](https://discuss.elastic.co/t/logstash-service-stop-working/54051 "2016-06-29T14:45:06Z")
Here is my configuration that works for us at the moment ( Filebeat -\> Logstash -\> Elasticsearch) input { beats { port =\> 5044 ssl =\> true ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt" … --- ## [Trigger logstash JDBC input plugin](https://discuss.elastic.co/t/trigger-logstash-jdbc-input-plugin/54278)
**Author:** [@lnx](https://discuss.elastic.co/u/lnx)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 12:54pm UTC](https://discuss.elastic.co/t/trigger-logstash-jdbc-input-plugin/54278 "2016-06-29T12:54:23Z")
If I have logstash running with a JDBC input plugin in the configuration, is it possible to send a call to logstash to run that input plugin again? i.e. I have a database which is periodically updated by scripts, I wan… --- ## [Grok add\_field creates two values](https://discuss.elastic.co/t/grok-add-field-creates-two-values/54277)
**Author:** [@muhamadli302](https://discuss.elastic.co/u/muhamadli302)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 12:37pm UTC](https://discuss.elastic.co/t/grok-add-field-creates-two-values/54277 "2016-06-29T12:37:52Z")
Hi, I'm using logstash 2.1, and I have the following filter: if ("sha1\_hash" in \[hash\_field\]) { grok { patterns\_dir =\> \["/opt/logstash/patterns"\] match =\> {"sha1\_hash: %{SHA1:sha1}"} add\_fie… --- ## [Unknown setting 'definitions' for netflow!](https://discuss.elastic.co/t/unknown-setting-definitions-for-netflow/54120)
**Author:** [@bzaki](https://discuss.elastic.co/u/bzaki)\ **Replies:** 11\ **Last updated:** [June 29, 2016, 10:43am UTC](https://discuss.elastic.co/t/unknown-setting-definitions-for-netflow/54120 "2016-06-29T10:43:04Z")
Hello, I'm trying to use ELK to collect netflows. I'm using logstash 2.3.3 . But when I use the configuration option 'definitions' I get the error "Unknown setting 'definitions' for netflow". Here's my config file belo… --- ## [Struggling to parse JSON key/value pairs](https://discuss.elastic.co/t/struggling-to-parse-json-key-value-pairs/42400)
**Author:** [@StephenGoodall](https://discuss.elastic.co/u/StephenGoodall)\ **Replies:** 16\ **Last updated:** [June 29, 2016, 11:39am UTC](https://discuss.elastic.co/t/struggling-to-parse-json-key-value-pairs/42400 "2016-06-29T11:39:59Z")
Hi, I'm struggling to parse a JSON Log4J2 output as the context map (MDC) looks to be stored as an array of JSON (i think!). Whats the best way of taking the KV pairs out of this: {"timeMillis":1456156644547,"thread":"… --- ## [Problem getting Windows Event log inside ELK](https://discuss.elastic.co/t/problem-getting-windows-event-log-inside-elk/54043)
**Author:** [@Gennady\_Sorochan](https://discuss.elastic.co/u/Gennady_Sorochan)\ **Replies:** 2\ **Last updated:** [June 29, 2016, 9:28am UTC](https://discuss.elastic.co/t/problem-getting-windows-event-log-inside-elk/54043 "2016-06-29T09:28:57Z")
Hey. We're test ELK for for accumulating logs from various systems. We're using ELK on Windows and nxlog to forward logs from the Windows servers to the ELK server. Getting IIS logs works fine. Once we're starting fo… --- ## [During loadtesting Logstash failed to load files with permission denied](https://discuss.elastic.co/t/during-loadtesting-logstash-failed-to-load-files-with-permission-denied/53268)
**Author:** [@devibala](https://discuss.elastic.co/u/devibala)\ **Replies:** 4\ **Last updated:** [June 29, 2016, 8:10am UTC](https://discuss.elastic.co/t/during-loadtesting-logstash-failed-to-load-files-with-permission-denied/53268 "2016-06-29T08:10:41Z")
Trying to do load testing .When logstash reaches certain limit it start throwing the "failed to open "..."permission denied " error ....but full permission given for the file. In single index "xyz" the total hits 334… --- ## [Dynamic Tag names for InfluxDB output based on field values](https://discuss.elastic.co/t/dynamic-tag-names-for-influxdb-output-based-on-field-values/54250)
**Author:** [@rbrandstaedter](https://discuss.elastic.co/u/rbrandstaedter)\ **Replies:** 0\ **Last updated:** [June 29, 2016, 7:58am UTC](https://discuss.elastic.co/t/dynamic-tag-names-for-influxdb-output-based-on-field-values/54250 "2016-06-29T07:58:38Z")
Hello, this is not really an issue but more a feature request or idea. I'm trying to create tag keys in the influx output that are based on a dynamic value. Let me explain. Here is my message. What I'm interested in … --- ## [Init.d file is missing in Logstash 5 Alpha3 Setup](https://discuss.elastic.co/t/init-d-file-is-missing-in-logstash-5-alpha3-setup/54132)
**Author:** [@gaurav\_S\_javheri](https://discuss.elastic.co/u/gaurav_S_javheri)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 6:58am UTC](https://discuss.elastic.co/t/init-d-file-is-missing-in-logstash-5-alpha3-setup/54132 "2016-06-29T06:58:58Z")
Hi, Last day I installed Logstash 5 Alpha3 version using installation guide https://www.elastic.co/guide/en/logstash/5.0/installing-logstash.html on RHEL 6.x. After installation found that /etc/init.d/logstash scrip… --- ## [Error Handling in Logstash](https://discuss.elastic.co/t/error-handling-in-logstash/54179)
**Author:** [@Achilles](https://discuss.elastic.co/u/Achilles)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 6:58am UTC](https://discuss.elastic.co/t/error-handling-in-logstash/54179 "2016-06-29T06:58:20Z")
Hi team, I am a newbie and I am using ELK as a reporting engine for the data from csv exported files. I am currently using 2.3.2 version of logstash to load csv data from a remote server through file beat. Since m… --- ## [Stuck with something related to Date](https://discuss.elastic.co/t/stuck-with-something-related-to-date/54093)
**Author:** [@kansei1983](https://discuss.elastic.co/u/kansei1983)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 6:55am UTC](https://discuss.elastic.co/t/stuck-with-something-related-to-date/54093 "2016-06-29T06:55:19Z")
I am new to using the ELK stack and running into the following problem -\> I have a CSV file with the following format -\> "string1" , "string2" , "string3" , "2016-06-26" , "string4" My mapping in elasticsearch cre… --- ## [How to solve grokparsefailure in Logstash](https://discuss.elastic.co/t/how-to-solve-grokparsefailure-in-logstash/54055)
**Author:** [@keya](https://discuss.elastic.co/u/keya)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 6:06am UTC](https://discuss.elastic.co/t/how-to-solve-grokparsefailure-in-logstash/54055 "2016-06-29T06:06:42Z")
Hi, I am trying to parse nginx error logs in Logstash. I have tested the grok pattern on http://grokdebug.herokuapp.com/ and it does not give any error. But when I try to use the same log sample and pattern in logstash… --- ## [XML filter help](https://discuss.elastic.co/t/xml-filter-help/54050)
**Author:** [@sar](https://discuss.elastic.co/u/sar)\ **Replies:** 4\ **Last updated:** [June 29, 2016, 6:03am UTC](https://discuss.elastic.co/t/xml-filter-help/54050 "2016-06-29T06:03:30Z")
Hi, I'm trying to import a log file which contains an XML message embedded in each line in to elasticsearch using logstash. I use KV filter to extract the XML message. Then I'm trying to extract several values from t… --- ## [Logstash replace escape chars in messages](https://discuss.elastic.co/t/logstash-replace-escape-chars-in-messages/54030)
**Author:** [@AyliD](https://discuss.elastic.co/u/AyliD)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 5:50am UTC](https://discuss.elastic.co/t/logstash-replace-escape-chars-in-messages/54030 "2016-06-29T05:50:39Z")
What is the best way and syntax to replace ESC chars from my logs.... 06/27/16 14:11:30.888 D S10.10.64.83 STREAMER 0 @IAAAAAAAJOGABHHF ^\[\[01;37;44mFX\_Ingestion\_Media\_Session\_Stop\_Command::dump\_co… --- ## [How to trouble shoot logstash crashing](https://discuss.elastic.co/t/how-to-trouble-shoot-logstash-crashing/53970)
**Author:** [@AyliD](https://discuss.elastic.co/u/AyliD)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 5:48am UTC](https://discuss.elastic.co/t/how-to-trouble-shoot-logstash-crashing/53970 "2016-06-29T05:48:33Z")
Hi, I have 2 nodes that has the same log stash configuration, one of them keep failing on this error: TypeError: can't convert nil into String Is there a way for me to understand on what message it failed?On what l… --- ## [Logstash only wants to target ONE file?](https://discuss.elastic.co/t/logstash-only-wants-to-target-one-file/54178)
**Author:** [@Robert\_Mascitelli](https://discuss.elastic.co/u/Robert_Mascitelli)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 5:27am UTC](https://discuss.elastic.co/t/logstash-only-wants-to-target-one-file/54178 "2016-06-29T05:27:57Z")
Hi guys, so I'm trying to send some dummy data to Elastic via Logstash. I am able to send kibana logs (kibana.log), but ONLY those kibana logs for some reason! Whenever I change my "path" in my config to another file Log… --- ## [Logstash Conf OpenVPN](https://discuss.elastic.co/t/logstash-conf-openvpn/54097)
**Author:** [@killmasta93](https://discuss.elastic.co/u/killmasta93)\ **Replies:** 2\ **Last updated:** [June 29, 2016, 3:19am UTC](https://discuss.elastic.co/t/logstash-conf-openvpn/54097 "2016-06-29T03:19:43Z")
Hi, I was want to say that im close to logstash grok patterns for OpenVPN pfSense Logs. I just have one tiny issue I cannot get or understand why maybe im missing a } not sure. So after implementing the OpenVPN pater… --- ## [The sync principle of logstash-input-jdbc?](https://discuss.elastic.co/t/the-sync-principle-of-logstash-input-jdbc/54017)
**Author:** [@laoyang360](https://discuss.elastic.co/u/laoyang360)\ **Replies:** 1\ **Last updated:** [June 29, 2016, 12:44am UTC](https://discuss.elastic.co/t/the-sync-principle-of-logstash-input-jdbc/54017 "2016-06-29T00:44:07Z")
I want to know the sync principle of logstash-input-jdbc? I know it is based on the time compare method like : select \* from cc where cc.modified\_at \> :sql\_last\_value, Does it also based on database trig… --- ## [Logstash s3 elb config into existing ELK stack](https://discuss.elastic.co/t/logstash-s3-elb-config-into-existing-elk-stack/52914)
**Author:** [@meathouse](https://discuss.elastic.co/u/meathouse)\ **Replies:** 2\ **Last updated:** [June 28, 2016, 11:19pm UTC](https://discuss.elastic.co/t/logstash-s3-elb-config-into-existing-elk-stack/52914 "2016-06-28T23:19:11Z")
I recently set up an ELK stack in AWS with 1 elasticsearch master, and 1 data node. Running fine for a few weeks with filebeat and some basic auth log and syslog filters. Now I'm trying to add my ELB logs from s3. I h… --- ## [Logstash error on windows7](https://discuss.elastic.co/t/logstash-error-on-windows7/49837)
**Author:** [@dvazarkar](https://discuss.elastic.co/u/dvazarkar)\ **Replies:** 1\ **Last updated:** [June 28, 2016, 5:47pm UTC](https://discuss.elastic.co/t/logstash-error-on-windows7/49837 "2016-06-28T17:47:00Z")
Hello I am getting the following error The signal HUP is in use by the JVM and will not work correctly on this platform when executing the following logstash statement.. C:\\tools\\ELK\\LogStash\\logstash-2.3.2\\bin\>lo… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1133) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1135)