# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=780 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 781 --- ## [Comparing to null in logstash](https://discuss.elastic.co/t/comparing-to-null-in-logstash/128008)
**Author:** [@snewman](https://discuss.elastic.co/u/snewman)\ **Replies:** 1\ **Last updated:** [April 13, 2018, 3:49pm UTC](https://discuss.elastic.co/t/comparing-to-null-in-logstash/128008 "2018-04-13T15:49:30Z")
I have a beats\>logstash\>elasticearch\>kibana pipeline to parse logfiles, some lines of which contain location data, but they all have a field for location data. Attempting to convert the empty fields to a geo\_point gives … --- ## [KV Pair Filter with empty values - Logstash 6.2.3](https://discuss.elastic.co/t/kv-pair-filter-with-empty-values-logstash-6-2-3/128000)
**Author:** [@jdnewhouse](https://discuss.elastic.co/u/jdnewhouse)\ **Replies:** 0\ **Last updated:** [April 13, 2018, 2:50pm UTC](https://discuss.elastic.co/t/kv-pair-filter-with-empty-values-logstash-6-2-3/128000 "2018-04-13T14:50:51Z")
My data comes as follow with the = as the value split and a tab character (shown here as \\t) as the field split. key1=value1\\tkey2=value2 \\tkey3=\\tkey4=\\tkey5=value5\\ Several of the KV pairs may contain empty values. I… --- ## [Unzip MQTT Message](https://discuss.elastic.co/t/unzip-mqtt-message/127855)
**Author:** [@CasMeiron](https://discuss.elastic.co/u/CasMeiron)\ **Replies:** 2\ **Last updated:** [April 13, 2018, 1:10pm UTC](https://discuss.elastic.co/t/unzip-mqtt-message/127855 "2018-04-13T13:10:41Z")
Hi, I'm using Mqtt plugin (Eclipse-Paho) to get messages from a MQTT topic, using the follow snippet: input { mqtt { topic =\> "/some/topic" host =\> "${MQTT\_HOST}" username =\> "${MQTT\_US… --- ## [With variable output file name : "The starting part of the path should not be dynamic"](https://discuss.elastic.co/t/with-variable-output-file-name-the-starting-part-of-the-path-should-not-be-dynamic/127966)
**Author:** [@AJ\_NOURI](https://discuss.elastic.co/u/AJ_NOURI)\ **Replies:** 2\ **Last updated:** [April 13, 2018, 12:04pm UTC](https://discuss.elastic.co/t/with-variable-output-file-name-the-starting-part-of-the-path-should-not-be-dynamic/127966 "2018-04-13T12:04:03Z")
When I add a output file with variable in the pipiline.conf path =\> "%{type}\_%{+yyyy\_MM\_dd}.log" I get the logstash error : The starting part of the path should not be dynamic It works fine when I remove the path l… --- ## [One Input Event - 2 Output Events](https://discuss.elastic.co/t/one-input-event-2-output-events/127920)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 4\ **Last updated:** [April 13, 2018, 10:31am UTC](https://discuss.elastic.co/t/one-input-event-2-output-events/127920 "2018-04-13T10:31:32Z")
We have tomcat access logs which we read from a FileBeat input and pass to an ElasticSearch output IndexA (fields a, b, c). We would however like to have some of these log entries (the 400 and 500 errors) additionally i… --- ## [Parsing Text Data from a txt file without Key Value Pair using Grok](https://discuss.elastic.co/t/parsing-text-data-from-a-txt-file-without-key-value-pair-using-grok/126005)
**Author:** [@vmashok85](https://discuss.elastic.co/u/vmashok85)\ **Replies:** 12\ **Last updated:** [April 13, 2018, 10:22am UTC](https://discuss.elastic.co/t/parsing-text-data-from-a-txt-file-without-key-value-pair-using-grok/126005 "2018-04-13T10:22:58Z")
Hi All, I need help to parse data separated by ' ; ' from a txt file without key value pair using grok. Kindly advice. --- ## [Encrypt incoming beats](https://discuss.elastic.co/t/encrypt-incoming-beats/127726)
**Author:** [@Kissa\_Koira](https://discuss.elastic.co/u/Kissa_Koira)\ **Replies:** 5\ **Last updated:** [April 13, 2018, 9:56am UTC](https://discuss.elastic.co/t/encrypt-incoming-beats/127726 "2018-04-13T09:56:44Z")
So far I have been able to encode some of the parts from my message (Mask parts of the message) now I would like to make this even better and use encryption instead. Could someone show me an example how to do this? Encry… --- ## [Add Column in csv file logstash](https://discuss.elastic.co/t/add-column-in-csv-file-logstash/127853)
**Author:** [@rouchad\_rouchad](https://discuss.elastic.co/u/rouchad_rouchad)\ **Replies:** 4\ **Last updated:** [April 13, 2018, 9:07am UTC](https://discuss.elastic.co/t/add-column-in-csv-file-logstash/127853 "2018-04-13T09:07:48Z")
hello everyone , please i need ur help i want add new columns in my logstash config file but i don't how do it ?? there is my config file : input { file { path =\> "C:\\elk\\logstash\\data.csv" start\_position =\> "begin… --- ## [Netflow Codec : UDP receive errors](https://discuss.elastic.co/t/netflow-codec-udp-receive-errors/127111)
**Author:** [@Julzor](https://discuss.elastic.co/u/Julzor)\ **Replies:** 11\ **Last updated:** [April 13, 2018, 8:43am UTC](https://discuss.elastic.co/t/netflow-codec-udp-receive-errors/127111 "2018-04-13T08:43:50Z")
Hi, I have installed LogStash with the Netflow module to catch my flows and output them in ES and it works great, i can see my stats in Kibana without problems. I have between 5k and 12k flows/sec depending of the time… --- ## [Index nested json](https://discuss.elastic.co/t/index-nested-json/127854)
**Author:** [@chrigui94](https://discuss.elastic.co/u/chrigui94)\ **Replies:** 3\ **Last updated:** [April 13, 2018, 8:32am UTC](https://discuss.elastic.co/t/index-nested-json/127854 "2018-04-13T08:32:08Z")
I have a nested file each document is like the following (contains a field "gateways" contains diffents number of gateways each one contains a latitude and longitude) { field1: "aa", field2:"bb", "gateways"\[ {antenn… --- ## [Dateformat exception](https://discuss.elastic.co/t/dateformat-exception/127774)
**Author:** [@tim.van.rooijen](https://discuss.elastic.co/u/tim.van.rooijen)\ **Replies:** 2\ **Last updated:** [April 13, 2018, 8:10am UTC](https://discuss.elastic.co/t/dateformat-exception/127774 "2018-04-13T08:10:01Z")
Hi Working on some dashboards and I’m stuck on how to handle datetimes. Beginner with elasticsearch Have a bunch off data from a database which I import with logstash. All datetimes without a timezone and the whole pro… --- ## [Logstash - netflow codec vs module](https://discuss.elastic.co/t/logstash-netflow-codec-vs-module/127290)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 1\ **Last updated:** [April 13, 2018, 8:01am UTC](https://discuss.elastic.co/t/logstash-netflow-codec-vs-module/127290 "2018-04-13T08:01:38Z")
Hi All, Iam trying to find the difference between the netflow codec and netflow module , I have tried both but i could see lot of additional fields in netflow module than in netflow codec, for example geoip.as\_org.keywo… --- ## [I want to save index by my own time zone](https://discuss.elastic.co/t/i-want-to-save-index-by-my-own-time-zone/127918)
**Author:** [@chenzilong444](https://discuss.elastic.co/u/chenzilong444)\ **Replies:** 2\ **Last updated:** [April 13, 2018, 7:59am UTC](https://discuss.elastic.co/t/i-want-to-save-index-by-my-own-time-zone/127918 "2018-04-13T07:59:07Z")
output { elasticsearch { hosts =\> localhost index =\> "logstash(%{+yyyy.MM.dd})" document\_type =\>"logstash" } stdout { codec =\> rubydebug } } index =\> "logstash(%{+… --- ## [Error while starting logstash agent in version 6.2.3](https://discuss.elastic.co/t/error-while-starting-logstash-agent-in-version-6-2-3/127816)
**Author:** [@amritesh\_mishra](https://discuss.elastic.co/u/amritesh_mishra)\ **Replies:** 2\ **Last updated:** [April 13, 2018, 6:59am UTC](https://discuss.elastic.co/t/error-while-starting-logstash-agent-in-version-6-2-3/127816 "2018-04-13T06:59:40Z")
\[2018-04-12T11:05:18,164\]\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash: :ConfigurationError", :message=\>"Expected one of #… --- ## [How can I encode unicode characters in config file for logstash?](https://discuss.elastic.co/t/how-can-i-encode-unicode-characters-in-config-file-for-logstash/127845)
**Author:** [@nneul](https://discuss.elastic.co/u/nneul)\ **Replies:** 1\ **Last updated:** [April 13, 2018, 6:19am UTC](https://discuss.elastic.co/t/how-can-i-encode-unicode-characters-in-config-file-for-logstash/127845 "2018-04-13T06:19:24Z")
i.e. I'd like to be able to do something like \\u1234 instead of having the raw utf8 character in the config. Is that possible with the config parser? I'm using 5.5 still in case it makes a difference. --- ## [Logstash configuration error (wants to check if value is null)](https://discuss.elastic.co/t/logstash-configuration-error-wants-to-check-if-value-is-null/127801)
**Author:** [@PolterFox](https://discuss.elastic.co/u/PolterFox)\ **Replies:** 5\ **Last updated:** [April 13, 2018, 6:14am UTC](https://discuss.elastic.co/t/logstash-configuration-error-wants-to-check-if-value-is-null/127801 "2018-04-13T06:14:42Z")
Hey everybody, I have an error in my configuration-file. I still think that the problem is the check if the value of \[\_source\]\[machine\_name\] ist null, but i dont know why?! Logstash configuration fileinput { http { … --- ## [Logstash plugin for reference data](https://discuss.elastic.co/t/logstash-plugin-for-reference-data/127808)
**Author:** [@arunvijayam](https://discuss.elastic.co/u/arunvijayam)\ **Replies:** 3\ **Last updated:** [April 13, 2018, 6:12am UTC](https://discuss.elastic.co/t/logstash-plugin-for-reference-data/127808 "2018-04-13T06:12:43Z")
Hi All, I am reading 4 CSV files in csv input plugins with different type attribute with different headers. Is there any option/provision to merge multiple CSV files columns into single row of records in logstash. Or … --- ## [How to generate the field\_split\_pattern?](https://discuss.elastic.co/t/how-to-generate-the-field-split-pattern/127814)
**Author:** [@Pinno\_Lin](https://discuss.elastic.co/u/Pinno_Lin)\ **Replies:** 1\ **Last updated:** [April 13, 2018, 2:24am UTC](https://discuss.elastic.co/t/how-to-generate-the-field-split-pattern/127814 "2018-04-13T02:24:45Z")
Hi, I want to make the filed\_split\_pattern in kv plugin, it's not like grok have some web site can let me try and error. Agent\_Local\_Date="2018.03.26" Agent\_Local\_Time="2018/3/26 David 05:43:55" Computer\_Name="zootek\\z… --- ## [Installing logstash jdbc plugin](https://discuss.elastic.co/t/installing-logstash-jdbc-plugin/127885)
**Author:** [@squanchydata](https://discuss.elastic.co/u/squanchydata)\ **Replies:** 3\ **Last updated:** [April 13, 2018, 1:44am UTC](https://discuss.elastic.co/t/installing-logstash-jdbc-plugin/127885 "2018-04-13T01:44:09Z")
In order for me to link my mysql database with elasticsearch, I need the jdbc plugin with logstash. Where/how do I install this plugin and how do I use it? --- ## [Logstash 1.5.4 : No password supplied for PKCS#12 KeyStore](https://discuss.elastic.co/t/logstash-1-5-4-no-password-supplied-for-pkcs-12-keystore/127697)
**Author:** [@meghnatolani](https://discuss.elastic.co/u/meghnatolani)\ **Replies:** 3\ **Last updated:** [April 12, 2018, 11:08pm UTC](https://discuss.elastic.co/t/logstash-1-5-4-no-password-supplied-for-pkcs-12-keystore/127697 "2018-04-12T23:08:26Z")
Hi, I am using logstash version 1.5.4 I am getting the bellow error - WARNING: Default JAVA\_OPTS will be overridden by the JAVA\_OPTS defined in the environment. Environment JAVA\_OPTS are -Djava.io.tmpdir=/var/lib/logst… --- ## [Date is not getting converted into right timezone](https://discuss.elastic.co/t/date-is-not-getting-converted-into-right-timezone/127856)
**Author:** [@sachinvy](https://discuss.elastic.co/u/sachinvy)\ **Replies:** 3\ **Last updated:** [April 12, 2018, 8:53pm UTC](https://discuss.elastic.co/t/date-is-not-getting-converted-into-right-timezone/127856 "2018-04-12T20:53:16Z")
Hi all, Here is my filter Input: 03-27-18 09:32:10,590 \[2184\] TRACE SFTP \<\> - \[139C9318\] msg: 2024502473 Sending SSH\_MSG\_KEX\_31 (139 bytes, seq nr 1) Filter: grok{ match =\> {"message" =\> "%{DATESTAMP:dat… --- ## [(v6.2.3) Kafka plugin: Compatibility](https://discuss.elastic.co/t/v6-2-3-kafka-plugin-compatibility/127550)
**Author:** [@rfrink](https://discuss.elastic.co/u/rfrink)\ **Replies:** 5\ **Last updated:** [April 12, 2018, 8:28pm UTC](https://discuss.elastic.co/t/v6-2-3-kafka-plugin-compatibility/127550 "2018-04-12T20:28:49Z")
The compatability link is broken, and what I saw from Confluence is extremely dated.... I see reference to "Client 1.0.0" for version 6.2.x. So, that gives rise to questions: 1.) Will it also work+support for Kafka 1.… --- ## [Convert a datetime value (not timestamp) to UTC](https://discuss.elastic.co/t/convert-a-datetime-value-not-timestamp-to-utc/127842)
**Author:** [@roblopes](https://discuss.elastic.co/u/roblopes)\ **Replies:** 3\ **Last updated:** [April 12, 2018, 6:16pm UTC](https://discuss.elastic.co/t/convert-a-datetime-value-not-timestamp-to-utc/127842 "2018-04-12T18:16:01Z")
Hi, I know that there are tons of topics on this subject, but the only ones that I found didn't work for me.\\ I have a date field with a date and time that has to be converted in Logstash to the UTC value. I tried lot… --- ## [Unable to find propper jdbc driver for sql server (ubuntu 12.04, 64 bit)](https://discuss.elastic.co/t/unable-to-find-propper-jdbc-driver-for-sql-server-ubuntu-12-04-64-bit/127340)
**Author:** [@jurquiza](https://discuss.elastic.co/u/jurquiza)\ **Replies:** 2\ **Last updated:** [April 12, 2018, 3:36pm UTC](https://discuss.elastic.co/t/unable-to-find-propper-jdbc-driver-for-sql-server-ubuntu-12-04-64-bit/127340 "2018-04-12T15:36:59Z")
Hi! Im currently trying to deploy an ES server on Ubuntu. The goal is to obtain data from an Sql Server (remote) via Logstash. Im running into some issues when installing the Jdbc Drivers. Im pulling the drivers with t… --- ## [Logstash Throwing Error](https://discuss.elastic.co/t/logstash-throwing-error/127840)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 0\ **Last updated:** [April 12, 2018, 2:37pm UTC](https://discuss.elastic.co/t/logstash-throwing-error/127840 "2018-04-12T14:37:25Z")
Sorry for the vague title, not really sure how to be more specific, below is the error message being logged. This is Elastic Stack 6.2.3 on a Windows server \[2018-04-12T02:21:41,279\]\[WARN \]\[io.netty.channel.DefaultChan… --- ## [Creating config file](https://discuss.elastic.co/t/creating-config-file/127821)
**Author:** [@Nada\_I](https://discuss.elastic.co/u/Nada_I)\ **Replies:** 3\ **Last updated:** [April 12, 2018, 2:21pm UTC](https://discuss.elastic.co/t/creating-config-file/127821 "2018-04-12T14:21:15Z")
What should i write in the config file to install logstash? --- ## [Logstash service starting error in log file](https://discuss.elastic.co/t/logstash-service-starting-error-in-log-file/127828)
**Author:** [@amritesh\_mishra](https://discuss.elastic.co/u/amritesh_mishra)\ **Replies:** 1\ **Last updated:** [April 12, 2018, 2:13pm UTC](https://discuss.elastic.co/t/logstash-service-starting-error-in-log-file/127828 "2018-04-12T14:13:02Z")
\[2018-04-12T11:05:18,066\]\[INFO \]\[logstash.runner \] Starting Logstash {"logstash.version"=\>"6.2.3"} \[2018-04-12T11:05:18,149\]\[INFO \]\[logstash.agent \] Successfully started Logstash API endpoint {:port=\>9… --- ## [Windows 7 Logstash JRUBY Error](https://discuss.elastic.co/t/windows-7-logstash-jruby-error/124152)
**Author:** [@AloysiusParedes](https://discuss.elastic.co/u/AloysiusParedes)\ **Replies:** 2\ **Last updated:** [April 12, 2018, 2:00pm UTC](https://discuss.elastic.co/t/windows-7-logstash-jruby-error/124152 "2018-04-12T14:00:05Z")
Hi, I am new to the entire ELK Stack, and I am trying to set up Logstash. I followed all of the instructions (unzipping, setting up config file, starting Logstash). My setup is Windows 7, and my java version is 1.8.0\_51. … --- ## [Logstash Simple JSON](https://discuss.elastic.co/t/logstash-simple-json/127351)
**Author:** [@AloysiusParedes](https://discuss.elastic.co/u/AloysiusParedes)\ **Replies:** 5\ **Last updated:** [April 12, 2018, 1:59pm UTC](https://discuss.elastic.co/t/logstash-simple-json/127351 "2018-04-12T13:59:06Z")
Trying to parse the following JSON file with Logstash: { "glossary": { "title": "example glossary", "GlossDiv": { "title": "S", "GlossList": { "GlossEntry": { … --- ## [Define a no existent field](https://discuss.elastic.co/t/define-a-no-existent-field/127673)
**Author:** [@PAU\_SUREDA\_COLL](https://discuss.elastic.co/u/PAU_SUREDA_COLL)\ **Replies:** 5\ **Last updated:** [April 12, 2018, 1:57pm UTC](https://discuss.elastic.co/t/define-a-no-existent-field/127673 "2018-04-12T13:57:24Z")
how can I define that there is no field called dir? I tried that and it doesn't worked if !\[dir\] { } [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=779) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=781)