# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=825 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 826 --- ## [Logstash running out of Memory and crashing](https://discuss.elastic.co/t/logstash-running-out-of-memory-and-crashing/119114)
**Author:** [@Bhanu\_Karumudi](https://discuss.elastic.co/u/Bhanu_Karumudi)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 8:01pm UTC](https://discuss.elastic.co/t/logstash-running-out-of-memory-and-crashing/119114 "2018-02-08T20:01:21Z")
Logstash running out of memory. it looked like a slow leak and memory reaching max. hprofs are generated and see following as suspects for leaks. The class "org.jruby.util.io.ChannelDescriptor", loaded by "", occupies… --- ## [\_dateparsefailure ISO8106](https://discuss.elastic.co/t/dateparsefailure-iso8106/119105)
**Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\ **Replies:** 2\ **Last updated:** [February 8, 2018, 7:47pm UTC](https://discuss.elastic.co/t/dateparsefailure-iso8106/119105 "2018-02-08T19:47:37Z")
Elasticstack 6.1 I am trying to parse the following field/date: "EventTime":"2018-02-08 12:43:35.951854Z" I have used: filter { date { match =\> \[ "EventTime", "yyyy-MM-dd HH:mm:ss:SSSZ" \] } } filter { … --- ## [Monitoring Logstash with XPACK](https://discuss.elastic.co/t/monitoring-logstash-with-xpack/117165)
**Author:** [@uirobot](https://discuss.elastic.co/u/uirobot)\ **Replies:** 16\ **Last updated:** [February 8, 2018, 6:06pm UTC](https://discuss.elastic.co/t/monitoring-logstash-with-xpack/117165 "2018-02-08T18:06:26Z")
Hi! I follow the instruction about installing and setting up logstash monitoring. No errors. I using cloud ES and Kibana. When I navigate to Monitoring - nothing happends - "No Monitor Data Found", but when I go to De… --- ## [Need to create array from deep level JSON structure](https://discuss.elastic.co/t/need-to-create-array-from-deep-level-json-structure/117720)
**Author:** [@mnaveen\_m](https://discuss.elastic.co/u/mnaveen_m)\ **Replies:** 6\ **Last updated:** [February 8, 2018, 6:02pm UTC](https://discuss.elastic.co/t/need-to-create-array-from-deep-level-json-structure/117720 "2018-02-08T18:02:03Z")
Here is our JSON structure: "data": { "OTAHotelAvailRS": { "errorsOrHotelStaysOrRoomStays": \[ {}, { "roomStay": \[ { "responseType": "PropertyList", "basicPropertyInfo": { "hotelCode": "H23428", } },{ "respon… --- ## [Logstash java.lang.OutOfMemoryError: Java heap space](https://discuss.elastic.co/t/logstash-java-lang-outofmemoryerror-java-heap-space/118708)
**Author:** [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\ **Replies:** 3\ **Last updated:** [February 8, 2018, 5:05pm UTC](https://discuss.elastic.co/t/logstash-java-lang-outofmemoryerror-java-heap-space/118708 "2018-02-08T17:05:19Z")
I recently upgraded to logstash 6.1.3 and I am now getting Java Heap Space Out of Memory errors. I have 32 Gb of memory and I gave the jvm 16Gb. I did not have this error on the previous version of 6.0.1. Is this a bug… --- ## [Logstash out put format issue](https://discuss.elastic.co/t/logstash-out-put-format-issue/119027)
**Author:** [@Sivaprakash\_Shanmuga](https://discuss.elastic.co/u/Sivaprakash_Shanmuga)\ **Replies:** 6\ **Last updated:** [February 8, 2018, 4:41pm UTC](https://discuss.elastic.co/t/logstash-out-put-format-issue/119027 "2018-02-08T16:41:56Z")
Hi, Here is my logstash configuration input { http { host =\> "127.0.0.1" port =\> 8001 codec =\> json add\_field =\> { "raw\_message" =\> "%{message}" } response\_headers =\> { "Access-Control-Allow-Origin" =\> "\*" "Co… --- ## [Elasticsearch output not respecting retry\_on\_conflict](https://discuss.elastic.co/t/elasticsearch-output-not-respecting-retry-on-conflict/118964)
**Author:** [@shnhrrsn](https://discuss.elastic.co/u/shnhrrsn)\ **Replies:** 1\ **Last updated:** [February 8, 2018, 4:26pm UTC](https://discuss.elastic.co/t/elasticsearch-output-not-respecting-retry-on-conflict/118964 "2018-02-08T16:26:05Z")
Unsure if I'm doing something wrong, but I have the following ES output setup: output { stdout { codec =\> json\_lines } elasticsearch { hosts =\> "search:9200" index =\> "posts" template =\> "/indexer/templates/post… --- ## [How to migrate indices from old .raw template to new .keyword mapping](https://discuss.elastic.co/t/how-to-migrate-indices-from-old-raw-template-to-new-keyword-mapping/118610)
**Author:** [@T3rm11](https://discuss.elastic.co/u/T3rm11)\ **Replies:** 4\ **Last updated:** [February 8, 2018, 3:13pm UTC](https://discuss.elastic.co/t/how-to-migrate-indices-from-old-raw-template-to-new-keyword-mapping/118610 "2018-02-08T15:13:57Z")
I think I ran into a very big problem when I upgraded from ELK 5 to ELK 6. When I look at the Logstash template in ELK 5 I see a keyword mapping with .raw. After the upgrade to ELK 6 the new template has a keyword mappin… --- ## [Logstash Grok Filter IIS parsing to ElasticSearch](https://discuss.elastic.co/t/logstash-grok-filter-iis-parsing-to-elasticsearch/119067)
**Author:** [@Edouard\_Fazenda](https://discuss.elastic.co/u/Edouard_Fazenda)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 2:28pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-iis-parsing-to-elasticsearch/119067 "2018-02-08T14:28:28Z")
Dear Comunity, I explain the situation, I have Filebeat installed in a server where there is an IIS configured , I retrieve the logs and send them to a logstash server which will output to a elasticsearch node. This is… --- ## [Can't figure out how to get Logstash to talk to Elastic Cloud](https://discuss.elastic.co/t/cant-figure-out-how-to-get-logstash-to-talk-to-elastic-cloud/118942)
**Author:** [@Josh\_A](https://discuss.elastic.co/u/Josh_A)\ **Replies:** 11\ **Last updated:** [February 8, 2018, 1:55pm UTC](https://discuss.elastic.co/t/cant-figure-out-how-to-get-logstash-to-talk-to-elastic-cloud/118942 "2018-02-08T13:55:27Z")
Hi there... I'm trying to run Logstash on Ubuntu 16.04 and ship some syslog data to Elastic Cloud. Here's my /etc/logstash/conf.d/logstash.conf file... input { udp { host =\> "172.31.27.139" port =\> 10514 … --- ## [How to handle a complex event with newlines](https://discuss.elastic.co/t/how-to-handle-a-complex-event-with-newlines/119054)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [February 8, 2018, 1:52pm UTC](https://discuss.elastic.co/t/how-to-handle-a-complex-event-with-newlines/119054 "2018-02-08T13:52:50Z")
Hi, My event looks like that: ####\ \ \ \ \*\*\* Starting Rule Debug Messages February 5, 2018 2:57:30 PM CST6CDT \*\*\* \[1\] \*\*\* Rule Name \[ Set expiration date when eligibi… --- ## [Split command in ruby by new line](https://discuss.elastic.co/t/split-command-in-ruby-by-new-line/118999)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 4\ **Last updated:** [February 8, 2018, 1:11pm UTC](https://discuss.elastic.co/t/split-command-in-ruby-by-new-line/118999 "2018-02-08T13:11:18Z")
Hi, I am trying to split the message where ever a new line exist by ruby filter. The command in the ruby code I use: message\_array = event.get('message').split("\\n") But the logstash doesn't start because of it. It … --- ## [Archive Conf file after run](https://discuss.elastic.co/t/archive-conf-file-after-run/119018)
**Author:** [@sn001](https://discuss.elastic.co/u/sn001)\ **Replies:** 3\ **Last updated:** [February 8, 2018, 1:10pm UTC](https://discuss.elastic.co/t/archive-conf-file-after-run/119018 "2018-02-08T13:10:50Z")
Hi there I am working on a large application that ingests a lot of data using logstash. We have logstash looking for a CSV file and a conf file coming in a pair in a specific folder. When a new file gets created in that… --- ## [Problem tag \_grokparsefailure](https://discuss.elastic.co/t/problem-tag-grokparsefailure/119044)
**Author:** [@samsam](https://discuss.elastic.co/u/samsam)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 1:09pm UTC](https://discuss.elastic.co/t/problem-tag-grokparsefailure/119044 "2018-02-08T13:09:28Z")
Hello everyone, I have logstash's problème on TAG. I have this filter in conf : \`filter { if "kibana" in \[message\] { drop { } } grok { match =\> \["message", "%{SYSLOGTIMESTAMP:log\_timestamp} %{GREEDYDATA:message}"\] … --- ## [Issue while connecting to Sql server using losgtash](https://discuss.elastic.co/t/issue-while-connecting-to-sql-server-using-losgtash/119043)
**Author:** [@Mahesh\_Kumar](https://discuss.elastic.co/u/Mahesh_Kumar)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 12:53pm UTC](https://discuss.elastic.co/t/issue-while-connecting-to-sql-server-using-losgtash/119043 "2018-02-08T12:53:28Z")
I am trying to index data in elastic search using logstash from SQL server DB and iam getting this following error Error: undefined method to\_sym' for nil:NilClass Did you mean? to\_s Exception: NoMethodError Stack: C:/L… --- ## [Windows Security Log Events Remain Stuck in Queue](https://discuss.elastic.co/t/windows-security-log-events-remain-stuck-in-queue/118481)
**Author:** [@cappy](https://discuss.elastic.co/u/cappy)\ **Replies:** 1\ **Last updated:** [February 8, 2018, 12:45pm UTC](https://discuss.elastic.co/t/windows-security-log-events-remain-stuck-in-queue/118481 "2018-02-08T12:45:38Z")
I am collecting Windows Security Event Logs from a Windows machine using Winlogbeat, and that is sending to my box that hosts Logstash fine, but for some reason or another, the Security log events are getting stuck in L… --- ## [No implicit conversion of nil into string](https://discuss.elastic.co/t/no-implicit-conversion-of-nil-into-string/119040)
**Author:** [@yosi\_ela](https://discuss.elastic.co/u/yosi_ela)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 12:37pm UTC](https://discuss.elastic.co/t/no-implicit-conversion-of-nil-into-string/119040 "2018-02-08T12:37:47Z")
when i am runing the logstash conf i am getting this exception someone know why? ATAL\] 2018-02-08 14:14:59.163 \[main\] runner - An unexpected error occurred! {:error=\>#\ **Author:** [@miki\_haiat](https://discuss.elastic.co/u/miki_haiat)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 11:52am UTC](https://discuss.elastic.co/t/logstash-event-log-windows-question-mark-issue/119031 "2018-02-08T11:52:29Z") Hi im trying to read the event log with the event log plugin . Unfortunately all the incoming text is written with question mark I tried various of charset but it was the same this is the conf file input { eventlog… --- ## [Filtring tweets with logstash filter plugin](https://discuss.elastic.co/t/filtring-tweets-with-logstash-filter-plugin/119008)
**Author:** [@soukaina](https://discuss.elastic.co/u/soukaina)\ **Replies:** 6\ **Last updated:** [February 8, 2018, 11:31am UTC](https://discuss.elastic.co/t/filtring-tweets-with-logstash-filter-plugin/119008 "2018-02-08T11:31:25Z")
Hi, i'm working on a ELK project to extract data from twitter to elasticsearch and build a sentimental analyses engin. I need to know how can I filter spammed tweets with logstash ? Is that possible or not ? for exam… --- ## [Split value in logstash](https://discuss.elastic.co/t/split-value-in-logstash/119019)
**Author:** [@youssef](https://discuss.elastic.co/u/youssef)\ **Replies:** 2\ **Last updated:** [February 8, 2018, 11:21am UTC](https://discuss.elastic.co/t/split-value-in-logstash/119019 "2018-02-08T11:21:51Z")
Hi all, i have field called CONGESTION\_LEVEL that may contain the values 0 , 1 , 2 , 0;N , 1;N , 2;N or may be empty, i want to add integer field that may contain 0 ,1 ,2 values: if the CONGESTION\_LEVEL field is 0 1 o… --- ## [CSV filter how to replace new line with ","](https://discuss.elastic.co/t/csv-filter-how-to-replace-new-line-with/119001)
**Author:** [@scch](https://discuss.elastic.co/u/scch)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 9:06am UTC](https://discuss.elastic.co/t/csv-filter-how-to-replace-new-line-with/119001 "2018-02-08T09:06:53Z")
Hi Magnus, I am parsing csv file which contain new line in a field. i want to replace it with ",". Tried below code. so far no success, please advice mutate {gsub =\> \["Standard", "\[\\n\]", ""\]} below image shows how dat… --- ## [Issue with my logstash conf file](https://discuss.elastic.co/t/issue-with-my-logstash-conf-file/118983)
**Author:** [@rattisyam](https://discuss.elastic.co/u/rattisyam)\ **Replies:** 5\ **Last updated:** [February 8, 2018, 9:39am UTC](https://discuss.elastic.co/t/issue-with-my-logstash-conf-file/118983 "2018-02-08T09:39:59Z")
Hi Team, I am very new to ELK stack. I am trying to parse jenkins XML (buil.xml) logs through xml parser. but i could not make it. could you please help me on this. below is my conf file: input { file { path =\> "… --- ## [PubSub Logstash input stuck on authentication](https://discuss.elastic.co/t/pubsub-logstash-input-stuck-on-authentication/118837)
**Author:** [@Amr\_Motan](https://discuss.elastic.co/u/Amr_Motan)\ **Replies:** 1\ **Last updated:** [February 8, 2018, 9:36am UTC](https://discuss.elastic.co/t/pubsub-logstash-input-stuck-on-authentication/118837 "2018-02-08T09:36:41Z")
Hi, I am trying to pull logs from GCP Stackdriver logs through pubsub. I followed the instructions in the plugin documentation, I created the topic and the subscriber and made sure that I am authenticated to access the … --- ## [Error in Logstash conditional expression](https://discuss.elastic.co/t/error-in-logstash-conditional-expression/118322)
**Author:** [@iamsaurabhrk](https://discuss.elastic.co/u/iamsaurabhrk)\ **Replies:** 4\ **Last updated:** [February 8, 2018, 9:21am UTC](https://discuss.elastic.co/t/error-in-logstash-conditional-expression/118322 "2018-02-08T09:21:38Z")
I am using Logstash 5.5.0. I have the following configuration: input { stdin { } } filter { grok { match =\> { "message" =\> "^(%{TIMESTAMP\_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:LogLevel}%{SPACE}(?:\\\[)%{NOTSPACE:Thr… --- ## [Logstash conf error - amazon\_es](https://discuss.elastic.co/t/logstash-conf-error-amazon-es/118998)
**Author:** [@Anthony\_Maffert](https://discuss.elastic.co/u/Anthony_Maffert)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 8:56am UTC](https://discuss.elastic.co/t/logstash-conf-error-amazon-es/118998 "2018-02-08T08:56:35Z")
Hi, I am trying to configure for the first time my logstash.conf file with an output to amazon\_es. However, i get the following error: \[ERROR\]\[logstash.outputs.amazones\] Failed to install template: \[400\] {"error":{"ro… --- ## [Logstash is not loading all the records from SQL server](https://discuss.elastic.co/t/logstash-is-not-loading-all-the-records-from-sql-server/118997)
**Author:** [@Mrin](https://discuss.elastic.co/u/Mrin)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 8:44am UTC](https://discuss.elastic.co/t/logstash-is-not-loading-all-the-records-from-sql-server/118997 "2018-02-08T08:44:11Z")
I have been using logstash to load a couple of millions of data into Elasticsearch from SQL server. However, every time I load the data, I am short of few thousands of records. input { jdbc { jdbc\_driver\_library =\> … --- ## [Logstash is not creating index](https://discuss.elastic.co/t/logstash-is-not-creating-index/118968)
**Author:** [@Somarjun\_Chandolu](https://discuss.elastic.co/u/Somarjun_Chandolu)\ **Replies:** 1\ **Last updated:** [February 8, 2018, 8:06am UTC](https://discuss.elastic.co/t/logstash-is-not-creating-index/118968 "2018-02-08T08:06:17Z")
Hi, This is what my conf file looks like \[52%20pm\] The data is flowing and is visible when I debug the output. But it is not creating an index. Command used to debug: usr/share/logstash/bin/logstash -f etc/logstash/… --- ## [How to replace @timestamp with logtime?](https://discuss.elastic.co/t/how-to-replace-timestamp-with-logtime/118949)
**Author:** [@MichaelKutsenko](https://discuss.elastic.co/u/MichaelKutsenko)\ **Replies:** 3\ **Last updated:** [February 8, 2018, 7:32am UTC](https://discuss.elastic.co/t/how-to-replace-timestamp-with-logtime/118949 "2018-02-08T07:32:49Z")
I've read a lot of topics about this but I did not manage with this task. Please, help. I'm using filebeat, logstash, elasticsearch and kibana to show my logs. Client and server run on Ubuntu 16.04. When I try to replac… --- ## [LogStash Exception: Net::IMAP::ResponseParseError](https://discuss.elastic.co/t/logstash-exception-net-imap-responseparseerror/118976)
**Author:** [@baskera](https://discuss.elastic.co/u/baskera)\ **Replies:** 0\ **Last updated:** [February 8, 2018, 6:23am UTC](https://discuss.elastic.co/t/logstash-exception-net-imap-responseparseerror/118976 "2018-02-08T06:23:15Z")
C:\\Logstash\\logstash-6.1.2\\logstash-6.1.2\\bin\>logstash -f emaillog.conf after execute the conf file: Following Config file using to get the email data: input { imap { type =\> mail host =\> "webmail.mydomain.com" u… --- ## [Logstash ms sql connector](https://discuss.elastic.co/t/logstash-ms-sql-connector/118806)
**Author:** [@Anuar\_Mukatov](https://discuss.elastic.co/u/Anuar_Mukatov)\ **Replies:** 3\ **Last updated:** [February 8, 2018, 4:18am UTC](https://discuss.elastic.co/t/logstash-ms-sql-connector/118806 "2018-02-08T04:18:35Z")
Hello! I have some trouble with jdbc input plugin. How i can connect to ms sql database with windows domain authentication? I know how connect with sql login name, but i can't understand how to connect with domain user… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=824) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=826)