# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=814 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 815 --- ## [Problem to install logstash using deb file on ubuntu 16.0.4](https://discuss.elastic.co/t/problem-to-install-logstash-using-deb-file-on-ubuntu-16-0-4/121092)
**Author:** [@Arab.Soheyl](https://discuss.elastic.co/u/Arab.Soheyl)\ **Replies:** 7\ **Last updated:** [February 24, 2018, 8:38pm UTC](https://discuss.elastic.co/t/problem-to-install-logstash-using-deb-file-on-ubuntu-16-0-4/121092 "2018-02-24T20:38:33Z")
I have an ubuntu server 16.0.4. I was installed elastic and kibana (both 6.2.2) with Debian file package. Now I want to install logstash 6.2.2 with .deb file, but some error occurred. elk@elk:~$sudo dpkg --install logs… --- ## [Problem with Grok debugger](https://discuss.elastic.co/t/problem-with-grok-debugger/121333)
**Author:** [@franco.federico](https://discuss.elastic.co/u/franco.federico)\ **Replies:** 3\ **Last updated:** [February 24, 2018, 3:52pm UTC](https://discuss.elastic.co/t/problem-with-grok-debugger/121333 "2018-02-24T15:52:42Z")
Hello I have the problem with the Grok debugger. I'm reading your post https://www.elastic.co/blog/grokking-the-linux-authorization-logs and I try to parse this sample line Feb 21 21:56:12 localhost sshd\[3430\]: Invali… --- ## [Csv file with escape character](https://discuss.elastic.co/t/csv-file-with-escape-character/120140)
**Author:** [@krrohit0604](https://discuss.elastic.co/u/krrohit0604)\ **Replies:** 1\ **Last updated:** [February 24, 2018, 3:15pm UTC](https://discuss.elastic.co/t/csv-file-with-escape-character/120140 "2018-02-24T15:15:28Z")
i have a csv file to index. some records are split in multiple lines as one of the column contains double quotes as escape character. eg: lin1: 870525,abc,Inner User,Minor,2018-01-14 02:22:34,Management,Log in to the … --- ## [Parsing nessus XML in Logstash](https://discuss.elastic.co/t/parsing-nessus-xml-in-logstash/120861)
**Author:** [@wdaburu](https://discuss.elastic.co/u/wdaburu)\ **Replies:** 22\ **Last updated:** [February 24, 2018, 2:10pm UTC](https://discuss.elastic.co/t/parsing-nessus-xml-in-logstash/120861 "2018-02-24T14:10:05Z")
Hi, i've come upon a problem on parsing the xml file to be displayed on kibana. Below are the configuration for my logstash input { file { path =\> "/home/user/nessus/\*" start\_position =\> "beginning" tags =\> … --- ## [Node.name not reflected in "host"](https://discuss.elastic.co/t/node-name-not-reflected-in-host/121305)
**Author:** [@vikasgb](https://discuss.elastic.co/u/vikasgb)\ **Replies:** 1\ **Last updated:** [February 24, 2018, 6:46am UTC](https://discuss.elastic.co/t/node-name-not-reflected-in-host/121305 "2018-02-24T06:46:41Z")
In logstash.yml file, we have set node.name to a specific hostname explicitly. But the host field is displayed as "localhost". $HOSTNAME is set to a valid hostname. If I sepecifically hardcode the hostname like below:: … --- ## [Add array items as prefixes for another array](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134)
**Author:** [@Maher\_Ben\_Taleb\_Ali](https://discuss.elastic.co/u/Maher_Ben_Taleb_Ali)\ **Replies:** 6\ **Last updated:** [February 24, 2018, 1:33am UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134 "2018-02-24T01:33:47Z")
Hi, I am trying to push articles into the corresponding Kafka Topics. article : { "topics": \[ "Barclays", "Companies", "Government", "Warrant" \], "categories": \[ "economy", … --- ## [Configuring logstash for use with Elastic Cloud](https://discuss.elastic.co/t/configuring-logstash-for-use-with-elastic-cloud/121303)
**Author:** [@mrubin](https://discuss.elastic.co/u/mrubin)\ **Replies:** 0\ **Last updated:** [February 23, 2018, 11:59pm UTC](https://discuss.elastic.co/t/configuring-logstash-for-use-with-elastic-cloud/121303 "2018-02-23T23:59:14Z")
I have some questions regarding configuring logstash to work with my Elastic Cloud cluster (see https://www.elastic.co/guide/en/cloud/master/ec-cloud-id.html). "In our examples, we use the elastic superuser that every … --- ## [Logstash - reading from mysql and loading to elastic search index](https://discuss.elastic.co/t/logstash-reading-from-mysql-and-loading-to-elastic-search-index/121298)
**Author:** [@binut](https://discuss.elastic.co/u/binut)\ **Replies:** 0\ **Last updated:** [February 23, 2018, 10:26pm UTC](https://discuss.elastic.co/t/logstash-reading-from-mysql-and-loading-to-elastic-search-index/121298 "2018-02-23T22:26:15Z")
We are using logstash which queries from mysql database and load it to elastic search index using the logstash jdbc connector. This a nested index when we load approx 1,000 records some of the documents have multiple v… --- ## [Logstash output to es](https://discuss.elastic.co/t/logstash-output-to-es/121274)
**Author:** [@David9](https://discuss.elastic.co/u/David9)\ **Replies:** 4\ **Last updated:** [February 23, 2018, 9:53pm UTC](https://discuss.elastic.co/t/logstash-output-to-es/121274 "2018-02-23T21:53:03Z")
New user of ES Cloud and logstash. I'm told this is where to post such questions... config is below, nothing's happening (no new data arrives in es). index has been created with mapping. any suggestions? i'm assumin… --- ## [Parse text value from message but keep original text in message](https://discuss.elastic.co/t/parse-text-value-from-message-but-keep-original-text-in-message/121263)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 3\ **Last updated:** [February 23, 2018, 6:59pm UTC](https://discuss.elastic.co/t/parse-text-value-from-message-but-keep-original-text-in-message/121263 "2018-02-23T18:59:57Z")
Good Afternoon All, I am currently using the Logstash Grok filter to parse out values from a log entry and populate index fields with the values and it is working like a champ but now I want to parse out certain values … --- ## [Parsing an array in Ruby](https://discuss.elastic.co/t/parsing-an-array-in-ruby/121254)
**Author:** [@Karanushah.1992](https://discuss.elastic.co/u/Karanushah.1992)\ **Replies:** 4\ **Last updated:** [February 23, 2018, 7:04pm UTC](https://discuss.elastic.co/t/parsing-an-array-in-ruby/121254 "2018-02-23T19:04:40Z")
Hi, I have an array field which has epoch time values. I am trying to use Ruby filter to parse it. I am getting a syntax error. I am new to Ruby so I am unable to spot the error. Can you help me out. Below is my code i… --- ## [Help with removing parts of log output](https://discuss.elastic.co/t/help-with-removing-parts-of-log-output/121259)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 2\ **Last updated:** [February 23, 2018, 5:53pm UTC](https://discuss.elastic.co/t/help-with-removing-parts-of-log-output/121259 "2018-02-23T17:53:57Z")
I would like to remove, if possible, the \\r\\n data and replace with a whitespace from the log file below.... Log file: Feb 20 13:00:10 172.19.1.181 1 2018-02-20T11:50:23.000Z removed-data-vices.com KES|10.2.4.0 - 00000… --- ## [Importing the same file](https://discuss.elastic.co/t/importing-the-same-file/120716)
**Author:** [@Pororo](https://discuss.elastic.co/u/Pororo)\ **Replies:** 18\ **Last updated:** [February 23, 2018, 7:15am UTC](https://discuss.elastic.co/t/importing-the-same-file/120716 "2018-02-23T07:15:01Z")
Hi, I just have a conceptual question about how logstash import same files. Let's say I have done with the previous data ingesting. I terminated the process of logstash after that.And I do not change the data source fi… --- ## [How to hold a state in Logstash?](https://discuss.elastic.co/t/how-to-hold-a-state-in-logstash/120887)
**Author:** [@daved](https://discuss.elastic.co/u/daved)\ **Replies:** 6\ **Last updated:** [February 23, 2018, 3:29pm UTC](https://discuss.elastic.co/t/how-to-hold-a-state-in-logstash/120887 "2018-02-23T15:29:22Z")
Hi, I'm trying to apply a state (with a field in ES) to subsequent log lines after seeing \[message\] =~ "foo" but have not had luck using class variables with the ruby filter (@@classVariable) as described in Keeping glo… --- ## [Logstah stopping pipeline](https://discuss.elastic.co/t/logstah-stopping-pipeline/121239)
**Author:** [@1\_Maximys\_1](https://discuss.elastic.co/u/1_Maximys_1)\ **Replies:** 0\ **Last updated:** [February 23, 2018, 3:23pm UTC](https://discuss.elastic.co/t/logstah-stopping-pipeline/121239 "2018-02-23T15:23:56Z")
Hi. I know that this topic was discussing many time, but I need you help. I want to start logstash in docker. When I run docker container with parameter -it, I see that everything is working. But when I start this co… --- ## [Grok Pattern works on grok debugger but \_GROKPARSE failure in Logstash](https://discuss.elastic.co/t/grok-pattern-works-on-grok-debugger-but-grokparse-failure-in-logstash/120399)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 3\ **Last updated:** [February 23, 2018, 2:58pm UTC](https://discuss.elastic.co/t/grok-pattern-works-on-grok-debugger-but-grokparse-failure-in-logstash/120399 "2018-02-23T14:58:58Z")
Hi I am trying to parse a multi-line log file(CSV) using Logstash/grok pattern. The pattern works inside grok debugger but when i run it in the logstash, it produces \_grokparsefailuer. The \_grokparsefailuer is there whe… --- ## [Persistent Queues Never Writes Tail Files](https://discuss.elastic.co/t/persistent-queues-never-writes-tail-files/121231)
**Author:** [@simondawe](https://discuss.elastic.co/u/simondawe)\ **Replies:** 0\ **Last updated:** [February 23, 2018, 2:29pm UTC](https://discuss.elastic.co/t/persistent-queues-never-writes-tail-files/121231 "2018-02-23T14:29:16Z")
Hi I have just started playing with logstash 5.4.0 persistent queues. I have configured logstash to use persistent queues though this always writes to head and never rolls the head over to tail. My logstash.yml is as f… --- ## [FileMaker jdbc input error](https://discuss.elastic.co/t/filemaker-jdbc-input-error/120599)
**Author:** [@Rami.Atieyeh](https://discuss.elastic.co/u/Rami.Atieyeh)\ **Replies:** 1\ **Last updated:** [February 23, 2018, 1:21pm UTC](https://discuss.elastic.co/t/filemaker-jdbc-input-error/120599 "2018-02-23T13:21:54Z")
Hi I am running Logstash with FileMaker JDBC connection as the input. Here is the config file I am using: input { jdbc { jdbc\_driver\_library =\> "C:\\fmjdbc.jar" jdbc\_driver\_class =\> "com.filemaker.jdbc.Driver" jdbc… --- ## [Compress output from logstash](https://discuss.elastic.co/t/compress-output-from-logstash/121104)
**Author:** [@yofa](https://discuss.elastic.co/u/yofa)\ **Replies:** 6\ **Last updated:** [February 23, 2018, 11:53am UTC](https://discuss.elastic.co/t/compress-output-from-logstash/121104 "2018-02-23T11:53:36Z")
Greetings, I'm new to ELK and all the stuff. I've been reading the documentation to have a better understanding of elastic's products and, as far I've seen, couldn't find a solution for the case below: I'm trying to re… --- ## [Logstash JDBC input - no suitable driver](https://discuss.elastic.co/t/logstash-jdbc-input-no-suitable-driver/120701)
**Author:** [@db2oracle](https://discuss.elastic.co/u/db2oracle)\ **Replies:** 3\ **Last updated:** [February 23, 2018, 11:15am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-no-suitable-driver/120701 "2018-02-23T11:15:27Z")
Hi All Does Logstash JDBC plugin ONLY support certain JDBC drivers - and that is fixed in code somehow. Using an input configuration for Mysql - I get past an error relating to no suitable driver - but I am trying to … --- ## [Output configuration question](https://discuss.elastic.co/t/output-configuration-question/121186)
**Author:** [@elkstarter](https://discuss.elastic.co/u/elkstarter)\ **Replies:** 6\ **Last updated:** [February 23, 2018, 11:08am UTC](https://discuss.elastic.co/t/output-configuration-question/121186 "2018-02-23T11:08:50Z")
Hi everyone, I am curious about the code described here : output { elasticsearch { hosts =\> \["http://localhost:9200"\] index =\> "%{\[@metadata\]\[beat\]}-%{+YYYY.MM.dd}" document\_type =\> "%{\[@metadata\]\[type\]}… --- ## [Logstash not showing any output](https://discuss.elastic.co/t/logstash-not-showing-any-output/120850)
**Author:** [@student](https://discuss.elastic.co/u/student)\ **Replies:** 4\ **Last updated:** [February 23, 2018, 10:32am UTC](https://discuss.elastic.co/t/logstash-not-showing-any-output/120850 "2018-02-23T10:32:02Z")
Hi I am a student working on a project and decided to use ELK Stack as a logging tool to present its value. I am using SUSE Linux Enterprise Server 12 SP3 (x86\_64) as OS and running it on a virtual machine. I need a… --- ## [Hi Team, Could someone help me as mentioned below, I'm unable to parse below mentioned logs. Thanks](https://discuss.elastic.co/t/hi-team-could-someone-help-me-as-mentioned-below-im-unable-to-parse-below-mentioned-logs-thanks/118567)
**Author:** [@Manjunatha](https://discuss.elastic.co/u/Manjunatha)\ **Replies:** 3\ **Last updated:** [February 23, 2018, 10:31am UTC](https://discuss.elastic.co/t/hi-team-could-someone-help-me-as-mentioned-below-im-unable-to-parse-below-mentioned-logs-thanks/118567 "2018-02-23T10:31:07Z")
\ **Author:** [@Manjunatha](https://discuss.elastic.co/u/Manjunatha)\ **Replies:** 0\ **Last updated:** [February 23, 2018, 10:27am UTC](https://discuss.elastic.co/t/enable-to-pick-values-from-below-parsed-logs/121197 "2018-02-23T10:27:27Z") Hi Team, I have parsed the XML based logs can you please help me to get the elements as mentioned below. Parsed Logs: "payload" =\> "\ **Author:** [@s\_b92](https://discuss.elastic.co/u/s_b92)\ **Replies:** 0\ **Last updated:** [February 23, 2018, 9:42am UTC](https://discuss.elastic.co/t/extract-timestamp-from-xml-comment/121192 "2018-02-23T09:42:44Z") Hi guys, I have a problem to extract a timestamp from a xml document. The timestamp information is nested in a xml comment. Here is an example of this XML comment. Export created at 2017-11-10 14:01:59Z by SPIDSS IMS T… --- ## [Best practices for "debugging" and problem solving logstash problems](https://discuss.elastic.co/t/best-practices-for-debugging-and-problem-solving-logstash-problems/121154)
**Author:** [@digirave](https://discuss.elastic.co/u/digirave)\ **Replies:** 1\ **Last updated:** [February 23, 2018, 7:10am UTC](https://discuss.elastic.co/t/best-practices-for-debugging-and-problem-solving-logstash-problems/121154 "2018-02-23T07:10:08Z")
Hello, I'm new to logstash and testing filebeat -\> logstash -\> elasticsearch -\> kibana for viewing log data. I have managed to use logstash to parse and format log formats, but I am worried that some log lines may b… --- ## [How to parse json into logstash to display data in kibana](https://discuss.elastic.co/t/how-to-parse-json-into-logstash-to-display-data-in-kibana/121169)
**Author:** [@Rajkumar\_idsil](https://discuss.elastic.co/u/Rajkumar_idsil)\ **Replies:** 1\ **Last updated:** [February 23, 2018, 7:07am UTC](https://discuss.elastic.co/t/how-to-parse-json-into-logstash-to-display-data-in-kibana/121169 "2018-02-23T07:07:26Z")
logstash.conf input { file { path =\> "E:\\Rajkumar\\ELK-InputOutput\\Input.log" start\_position =\> "beginning" type =\> "json" } } filter { } output { elasticsearch { hosts =\> "http://192.168.77.59:9200" index =\> … --- ## [Ignoring the 'pipelines.yml' file because modules or command line options are specified](https://discuss.elastic.co/t/ignoring-the-pipelines-yml-file-because-modules-or-command-line-options-are-specified/120606)
**Author:** [@Rajkumar\_idsil](https://discuss.elastic.co/u/Rajkumar_idsil)\ **Replies:** 4\ **Last updated:** [February 23, 2018, 6:53am UTC](https://discuss.elastic.co/t/ignoring-the-pipelines-yml-file-because-modules-or-command-line-options-are-specified/120606 "2018-02-23T06:53:46Z")
Initializing module {:module\_name=\>"fb\_apache", :directory=\>"G:/Rajkumar/ELK/logstash-6.2.1/modules/fb\_apache/configuration"} \[logstash.modules.scaffold\] Initializing module {:module\_name=\>"netflow", :directory=\>"G:/Raj… --- ## ["java.lang.OutOfMemoryError: Java heap space" after running for a few minutes](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-after-running-for-a-few-minutes/121165)
**Author:** [@joshuaspence](https://discuss.elastic.co/u/joshuaspence)\ **Replies:** 1\ **Last updated:** [February 23, 2018, 5:10am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-after-running-for-a-few-minutes/121165 "2018-02-23T05:10:05Z")
We are seeing Logstash dying with an out of memory error after running for anywhere from 5 to 15 minutes. We are using Logstash 6.2.2 with the following configuration: input { beats { include\_codec\_tag =\> false … --- ## [Multipipeline Issues](https://discuss.elastic.co/t/multipipeline-issues/117347)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 14\ **Last updated:** [February 23, 2018, 4:36am UTC](https://discuss.elastic.co/t/multipipeline-issues/117347 "2018-02-23T04:36:20Z")
ELK 6.1.1 stack running on Windows Server 2012R2 .\\logstash -f path\\pipelines.yml pipelines.yml pipeline.id: Beats path.config: "D:/ELKStack/LogStash/config/pipeline\_beats.config" pipeline.id: NMap path.config: "… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=813) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=815)