# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=338 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 339 --- ## [Csv Column -\> Splitstring array and -\> Nested Object Problem](https://discuss.elastic.co/t/csv-column-splitstring-array-and-nested-object-problem/237609)
**Author:** [@Bender](https://discuss.elastic.co/u/Bender)\ **Replies:** 3\ **Last updated:** [June 24, 2020, 9:18am UTC](https://discuss.elastic.co/t/csv-column-splitstring-array-and-nested-object-problem/237609 "2020-06-24T09:18:44Z")
Dear Community, iam very new in the logstash and Ruby Environment and i have task that i have to solve. Input is a CSV file and on Column has a String with the following content (content length is dynamic but structure… --- ## [Problem with logstash: java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/problem-with-logstash-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/238390)
**Author:** [@kase](https://discuss.elastic.co/u/kase)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 6:29am UTC](https://discuss.elastic.co/t/problem-with-logstash-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/238390 "2020-06-24T06:29:53Z")
Hi there, I'm using elastiflow with logstash, I've done these command in this url(https://www.catapultsystems.com/blogs/install-elastiflow-on-ubuntu-18-04-part-3/) And netflow data could came through logstash to elasti… --- ## [How to connect to 32-bit Sybase from Logstash Windows](https://discuss.elastic.co/t/how-to-connect-to-32-bit-sybase-from-logstash-windows/238389)
**Author:** [@qkhanhpro](https://discuss.elastic.co/u/qkhanhpro)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 6:14am UTC](https://discuss.elastic.co/t/how-to-connect-to-32-bit-sybase-from-logstash-windows/238389 "2020-06-24T06:14:07Z")
Hi. I am trying to push some data from 32-bit Sybase to ES but at the moment, using Logstash for Windows, I am not sure which kind of configuration I should use for this to work? I already have a some experience with L… --- ## [Logstash-plain.log stop recording and logstash failed to start](https://discuss.elastic.co/t/logstash-plain-log-stop-recording-and-logstash-failed-to-start/238201)
**Author:** [@kase](https://discuss.elastic.co/u/kase)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 3:06am UTC](https://discuss.elastic.co/t/logstash-plain-log-stop-recording-and-logstash-failed-to-start/238201 "2020-06-24T03:06:29Z")
Hi there, I'm using logstash to filter netflow data to save to elasticsearch, it works fine but I also want to filter data in elasticsearch to save to redis, so I wrote a file to do logstash pipeline. Here is my file i… --- ## [Logstash Google Cloud Storage Output Plugin Error Msg](https://discuss.elastic.co/t/logstash-google-cloud-storage-output-plugin-error-msg/238355)
**Author:** [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 1:42am UTC](https://discuss.elastic.co/t/logstash-google-cloud-storage-output-plugin-error-msg/238355 "2020-06-24T01:42:51Z")
I'm trying to set up the logstash-output-google\_cloud\_storage plugin with a very simple configuration set. The input works just fine, I tested it writing to a local file. output { google\_cloud\_storage { bucket… --- ## [\[WARN \]\[org.logstash.common.io.DeadLetterQueueWriter\] Event previously submitted to dead letter queue. Skipping](https://discuss.elastic.co/t/warn-org-logstash-common-io-deadletterqueuewriter-event-previously-submitted-to-dead-letter-queue-skipping/113250)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 2\ **Last updated:** [June 23, 2020, 11:07pm UTC](https://discuss.elastic.co/t/warn-org-logstash-common-io-deadletterqueuewriter-event-previously-submitted-to-dead-letter-queue-skipping/113250 "2020-06-23T23:07:03Z")
I just started working with the Dead Letter Queue, but after restarting my pipelines a few times I am now getting this message repeatedly in my logs: \[WARN \]\[org.logstash.common.io.DeadLetterQueueWriter\] Event previousl… --- ## [How to create patterns\_dir in elastic logstash helm chart?](https://discuss.elastic.co/t/how-to-create-patterns-dir-in-elastic-logstash-helm-chart/238346)
**Author:** [@Phillip\_Boushy](https://discuss.elastic.co/u/Phillip_Boushy)\ **Replies:** 0\ **Last updated:** [June 23, 2020, 9:53pm UTC](https://discuss.elastic.co/t/how-to-create-patterns-dir-in-elastic-logstash-helm-chart/238346 "2020-06-23T21:53:39Z")
I'm deploy the elastic.co logstash helm chart 7.7.1 to kubernetes. I'm using a ConfigMap to create an example file in /usr/share/logstash/pipeline/patterns/ But it seems like it's not being read. I keep getting an err… --- ## [ELK - SO Security correlations](https://discuss.elastic.co/t/elk-so-security-correlations/237313)
**Author:** [@YASSER\_MEMO](https://discuss.elastic.co/u/YASSER_MEMO)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 8:34pm UTC](https://discuss.elastic.co/t/elk-so-security-correlations/237313 "2020-06-23T20:34:57Z")
Hey guys, I installed Security onion with ELK to monitor enterprise network. I want to send Cisco Firewall logs , IPS, F5 and antivirus logs to logstash. I've already sent them however, I'm facing two issues. 1- non of… --- ## [Grok filter extracting fields from message](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298)
**Author:** [@heinrich](https://discuss.elastic.co/u/heinrich)\ **Replies:** 8\ **Last updated:** [June 23, 2020, 8:21pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298 "2020-06-23T20:21:41Z")
Hi guys, I've never used grok patterns before, after hours of looking into this i'm making very slow progress. Here's a sample of what i have. 2020-06-23 15:30:02.568 \[https-jsse-nio-0.0.0.0-8443-exec-58\] INFO c.c.d.… --- ## [Grok custom pattern for space delimited file](https://discuss.elastic.co/t/grok-custom-pattern-for-space-delimited-file/238291)
**Author:** [@atchyuth](https://discuss.elastic.co/u/atchyuth)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 5:23pm UTC](https://discuss.elastic.co/t/grok-custom-pattern-for-space-delimited-file/238291 "2020-06-23T17:23:15Z")
I'm trying to load a file to structured table in Athena. I am using GROK pattern to load it to the table but not able to find the correct pattern. The file format is as below: L1127 ACTUALS 214171 ON 27649075 -000000000… --- ## [How to update an existing es document and add a new value received from logstash event, to the existing array field at es at indexing time](https://discuss.elastic.co/t/how-to-update-an-existing-es-document-and-add-a-new-value-received-from-logstash-event-to-the-existing-array-field-at-es-at-indexing-time/238112)
**Author:** [@darkwarrior](https://discuss.elastic.co/u/darkwarrior)\ **Replies:** 6\ **Last updated:** [June 23, 2020, 5:12pm UTC](https://discuss.elastic.co/t/how-to-update-an-existing-es-document-and-add-a-new-value-received-from-logstash-event-to-the-existing-array-field-at-es-at-indexing-time/238112 "2020-06-23T17:12:46Z")
Hello guys, I am running logstash with es as output and my logstash input looks like this: Date time \[some-fields\] country\_name city\_name sample input values=\> 22-06-2020 02:19:27 \[some-fields\] US NewYork 22-06-202… --- ## [Gelf in Kafka out for logstash](https://discuss.elastic.co/t/gelf-in-kafka-out-for-logstash/238294)
**Author:** [@pointonj](https://discuss.elastic.co/u/pointonj)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 5:06pm UTC](https://discuss.elastic.co/t/gelf-in-kafka-out-for-logstash/238294 "2020-06-23T17:06:16Z")
Im trying to use Logstash with a GELF input and outputting to Kafka and not getting any logs. My config file looks like this - input { udp { host =\> "ip of loadbalancer" port =\> 9900 type =\> gelf … --- ## [Where's the % coming from?](https://discuss.elastic.co/t/wheres-the-coming-from/238277)
**Author:** [@mykael](https://discuss.elastic.co/u/mykael)\ **Replies:** 2\ **Last updated:** [June 23, 2020, 4:46pm UTC](https://discuss.elastic.co/t/wheres-the-coming-from/238277 "2020-06-23T16:46:20Z")
I'm using the following to add an ingestion timestamp to my records: ruby { code =\> 'event.set("ingest\_time", Time.now.strftime("%Y-%m-%d %H:%M:%S.%3"))' } What I'm getting in my data, thought, is: ingest\_tim… --- ## [Logstash Performance Tuning](https://discuss.elastic.co/t/logstash-performance-tuning/238088)
**Author:** [@Jib\_Suwanachote](https://discuss.elastic.co/u/Jib_Suwanachote)\ **Replies:** 3\ **Last updated:** [June 23, 2020, 3:48pm UTC](https://discuss.elastic.co/t/logstash-performance-tuning/238088 "2020-06-23T15:48:22Z")
Hi all Currently our team use logstash run over container with K8S cluster for purpose consume data from KAFKA messaging and push data to Elasticsearch .As we're testing performance not good enough. we're setup only 3… --- ## [Elasticsearch not indexing after update (only a specific index)](https://discuss.elastic.co/t/elasticsearch-not-indexing-after-update-only-a-specific-index/238271)
**Author:** [@user9](https://discuss.elastic.co/u/user9)\ **Replies:** 0\ **Last updated:** [June 23, 2020, 1:51pm UTC](https://discuss.elastic.co/t/elasticsearch-not-indexing-after-update-only-a-specific-index/238271 "2020-06-23T13:51:44Z")
I've been for days trying to solve this issue, there is no support neither troubleshoot about these issues. All indexes are being created, except this one. Before the update it worked, now it doesn't. Things I checked: … --- ## [Netflow logs](https://discuss.elastic.co/t/netflow-logs/238272)
**Author:** [@amr\_hamdy](https://discuss.elastic.co/u/amr_hamdy)\ **Replies:** 0\ **Last updated:** [June 23, 2020, 1:55pm UTC](https://discuss.elastic.co/t/netflow-logs/238272 "2020-06-23T13:55:49Z")
when i run logstash to collect netflow logs and send it to sentinel it works well but message in logs is in unreadable format my input configuration file input { udp { port =\> 2055 type =\> "netflow" } } --- ## [Http\_poller input plugin: please no http\_poller\_metadata](https://discuss.elastic.co/t/http-poller-input-plugin-please-no-http-poller-metadata/238200)
**Author:** [@smm](https://discuss.elastic.co/u/smm)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 1:32pm UTC](https://discuss.elastic.co/t/http-poller-input-plugin-please-no-http-poller-metadata/238200 "2020-06-23T13:32:27Z")
Hi there, is there a 'trick' to disable collecting the http\_poller\_metadata fields when using this plugin? I am only interested in the acutal data I get. Right now I am doing this: http\_poller { urls =\> { url =\> "ht… --- ## [Filter Sonicwall](https://discuss.elastic.co/t/filter-sonicwall/238257)
**Author:** [@Danilo\_Visual](https://discuss.elastic.co/u/Danilo_Visual)\ **Replies:** 0\ **Last updated:** [June 23, 2020, 12:34pm UTC](https://discuss.elastic.co/t/filter-sonicwall/238257 "2020-06-23T12:34:13Z")
I'm using ELK and I wanted to know where I can insert filters to analyze Sonicwall logs and view them in the graphana. Would you put it in the filter.conf of the logstash? Can anybody help me? --- ## [Unable to connect to mssql using logstash jdbc static filter](https://discuss.elastic.co/t/unable-to-connect-to-mssql-using-logstash-jdbc-static-filter/238237)
**Author:** [@Matthew\_Field](https://discuss.elastic.co/u/Matthew_Field)\ **Replies:** 0\ **Last updated:** [June 23, 2020, 11:13am UTC](https://discuss.elastic.co/t/unable-to-connect-to-mssql-using-logstash-jdbc-static-filter/238237 "2020-06-23T11:13:31Z")
The log output doesnt give me much of a clue what the problem is: My config: jdbc\_static { loaders =\> \[ \ \] local\_db\_objects =\> \[ \… --- ## [Confused about the behaviour of last\_run\_metadata\_path in JDBC input](https://discuss.elastic.co/t/confused-about-the-behaviour-of-last-run-metadata-path-in-jdbc-input/237669)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 2\ **Last updated:** [June 23, 2020, 8:00am UTC](https://discuss.elastic.co/t/confused-about-the-behaviour-of-last-run-metadata-path-in-jdbc-input/237669 "2020-06-23T08:00:27Z")
I am trying to get some data out of a SQL database. Here is the configuration: input { jdbc { jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc\_con… --- ## [Performance of logstash degrading](https://discuss.elastic.co/t/performance-of-logstash-degrading/238229)
**Author:** [@juliolb](https://discuss.elastic.co/u/juliolb)\ **Replies:** 0\ **Last updated:** [June 23, 2020, 10:49am UTC](https://discuss.elastic.co/t/performance-of-logstash-degrading/238229 "2020-06-23T10:49:33Z")
Hello, I have been using logstash for some time now and I recently upgraded from an old version (5.3.2) to a newer one (6.2.4). Using basically the same parameters I noticed a dramatic decrease in the rate of processed … --- ## [Grok pattern to get the whole text without considering space](https://discuss.elastic.co/t/grok-pattern-to-get-the-whole-text-without-considering-space/238095)
**Author:** [@Mohan\_vel](https://discuss.elastic.co/u/Mohan_vel)\ **Replies:** 2\ **Last updated:** [June 23, 2020, 6:55am UTC](https://discuss.elastic.co/t/grok-pattern-to-get-the-whole-text-without-considering-space/238095 "2020-06-23T06:55:15Z")
I am trying to grok a message text with my grok pattern if i pass the sample data without space my grok pattern works fine but if there is any space in between text it fetch only the first word before space and after spa… --- ## [Integrate ElasticSearch, Logstash and fscrawler](https://discuss.elastic.co/t/integrate-elasticsearch-logstash-and-fscrawler/238185)
**Author:** [@akshaybhuradia](https://discuss.elastic.co/u/akshaybhuradia)\ **Replies:** 3\ **Last updated:** [June 23, 2020, 9:00am UTC](https://discuss.elastic.co/t/integrate-elasticsearch-logstash-and-fscrawler/238185 "2020-06-23T09:00:04Z")
I have meta data for document in database. Example : filename, file description, file\_tags and etc Earlier i was using this data for search. elasticsearch--for search and logstash--for accessing data source(db --\> tabl… --- ## [Grok filter not working on logstash](https://discuss.elastic.co/t/grok-filter-not-working-on-logstash/238092)
**Author:** [@Abdur\_Raqeeb2](https://discuss.elastic.co/u/Abdur_Raqeeb2)\ **Replies:** 3\ **Last updated:** [June 23, 2020, 7:42am UTC](https://discuss.elastic.co/t/grok-filter-not-working-on-logstash/238092 "2020-06-23T07:42:04Z")
For my below filebeat config im not able to filter the logs based on tags in logstash Filebeat Config filebeat.inputs: - type: log enabled: true fields: env: xyz-production app\_name: xyz tags: \["nginx"\]… --- ## [Stack-trace issue](https://discuss.elastic.co/t/stack-trace-issue/238016)
**Author:** [@vshivare](https://discuss.elastic.co/u/vshivare)\ **Replies:** 7\ **Last updated:** [June 23, 2020, 6:51am UTC](https://discuss.elastic.co/t/stack-trace-issue/238016 "2020-06-23T06:51:32Z")
This is my json message: { "module":"MODULE", "timeStamp":"Jun 22 12:20:46.785", "containerId":INDLT1534", "correlationId":"", "level":"ERROR", "logger":"c.m.r.HelloWorldService", "message":"This is due to java.la… --- ## [Is ELK itself capable of keeping track of certain events & alerting?](https://discuss.elastic.co/t/is-elk-itself-capable-of-keeping-track-of-certain-events-alerting/238188)
**Author:** [@Aj\_t](https://discuss.elastic.co/u/Aj_t)\ **Replies:** 4\ **Last updated:** [June 23, 2020, 6:25am UTC](https://discuss.elastic.co/t/is-elk-itself-capable-of-keeping-track-of-certain-events-alerting/238188 "2020-06-23T06:25:57Z")
I realize that the question is super vague, but here's what I am trying to do. I've already set up ELK and am sending some logs via Filebeat to the ELK server. What I ultimately want to do is a proof of concept system t… --- ## [Date filter plugin error](https://discuss.elastic.co/t/date-filter-plugin-error/238176)
**Author:** [@mhinds](https://discuss.elastic.co/u/mhinds)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 5:34am UTC](https://discuss.elastic.co/t/date-filter-plugin-error/238176 "2020-06-23T05:34:38Z")
The date filter plugin is incorrectly overwriting the month of my date with January, see example output below and configuration. Logstash Conf File input { stdin {} } filter { date { tar… --- ## [Logstash error "there are no living connections in the connection pool"](https://discuss.elastic.co/t/logstash-error-there-are-no-living-connections-in-the-connection-pool/238171)
**Author:** [@SNJY](https://discuss.elastic.co/u/SNJY)\ **Replies:** 0\ **Last updated:** [June 23, 2020, 2:51am UTC](https://discuss.elastic.co/t/logstash-error-there-are-no-living-connections-in-the-connection-pool/238171 "2020-06-23T02:51:29Z")
Logstash instance is connected to an ES cluster behind a load balancer. The load balancer has an idle timeout of 5 minutes. Logstash is configured with the ES url corresponding to the loadbalancer ip. There are random… --- ## [Pipeline does not work](https://discuss.elastic.co/t/pipeline-does-not-work/238142)
**Author:** [@impkos](https://discuss.elastic.co/u/impkos)\ **Replies:** 5\ **Last updated:** [June 23, 2020, 12:57am UTC](https://discuss.elastic.co/t/pipeline-does-not-work/238142 "2020-06-23T00:57:54Z")
Help understand why it does not work pipeline pipelines.yml - pipeline.id: main path.config: "/usr/share/logstash/config/conf.d/\*.conf" - pipeline.id: ms001-backup01 path.config: "/usr/share/logstash/config/conf.d… --- ## [Logstash csv / First Line Index](https://discuss.elastic.co/t/logstash-csv-first-line-index/237755)
**Author:** [@xennn](https://discuss.elastic.co/u/xennn)\ **Replies:** 1\ **Last updated:** [June 22, 2020, 11:03pm UTC](https://discuss.elastic.co/t/logstash-csv-first-line-index/237755 "2020-06-22T23:03:56Z")
Hello, I use Logstash CSV Filter Plugin to index data using Powershell scripts. I want the first line not to be indexed as a record. Do you know how to do that? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=337) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=339)