# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=491 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 492 --- ## [Facing issue with logstash command java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/facing-issue-with-logstash-command-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/194709)
**Author:** [@jainabhishek0406](https://discuss.elastic.co/u/jainabhishek0406)\ **Replies:** 6\ **Last updated:** [August 19, 2019, 9:04am UTC](https://discuss.elastic.co/t/facing-issue-with-logstash-command-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/194709 "2019-08-19T09:04:54Z")
C:\\Users\\Abhishek\\Desktop\\elasticSearch\\projectdata\\logstash-6.4.3\\bin\>logstash -f logstash-simple.conf Sending Logstash logs to C:/Users/Abhishek/Desktop/elasticSearch/projectdata/logstash-6.4.3/logs which is now confi… --- ## [Parse Modsecurity json logs](https://discuss.elastic.co/t/parse-modsecurity-json-logs/195609)
**Author:** [@mattia\_conversano](https://discuss.elastic.co/u/mattia_conversano)\ **Replies:** 5\ **Last updated:** [August 19, 2019, 8:19am UTC](https://discuss.elastic.co/t/parse-modsecurity-json-logs/195609 "2019-08-19T08:19:49Z")
Hi team, First of all thanks for this awesome tool. I have a problem: I'm trying to send Modsecurity's (JSON) logs to Elasticsearch through Filebeat and Logstash. Log example: { ... "audit\_data":{ "message… --- ## [Uninitialized constant logstash::Filters::Ruby::JSON Did you mean? LogStash::Filters::Json](https://discuss.elastic.co/t/uninitialized-constant-logstash-json-did-you-mean-logstash-json/195685)
**Author:** [@Alex-Victor](https://discuss.elastic.co/u/Alex-Victor)\ **Replies:** 0\ **Last updated:** [August 19, 2019, 7:15am UTC](https://discuss.elastic.co/t/uninitialized-constant-logstash-json-did-you-mean-logstash-json/195685 "2019-08-19T07:15:47Z")
as the topic, I was traped with this problem, I have tried serveral methods,but it still does not work at all. anybody ever met that like me ? below is my settings if \[tags\]\[0\] == "ad\_api-logs" { mutate { add\_field =\> … --- ## [Convert Date String Into Date Format](https://discuss.elastic.co/t/convert-date-string-into-date-format/193913)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 110\ **Last updated:** [August 19, 2019, 4:02am UTC](https://discuss.elastic.co/t/convert-date-string-into-date-format/193913 "2019-08-19T04:02:51Z")
I have a column date when i write a config file and parse data into logstash the date column save by default into string format i want to convert date string column into date format can someone help really appreciate i… --- ## [Making the best of an existing Syslog stream](https://discuss.elastic.co/t/making-the-best-of-an-existing-syslog-stream/195662)
**Author:** [@Brent\_Ashley](https://discuss.elastic.co/u/Brent_Ashley)\ **Replies:** 2\ **Last updated:** [August 18, 2019, 10:24pm UTC](https://discuss.elastic.co/t/making-the-best-of-an-existing-syslog-stream/195662 "2019-08-18T22:24:54Z")
My task is to introduce log analysis to an environment where I am a drop-in replacement for Syslog and I do not have the option of asking clients to use a different shipper. In this environment logs are forwarded to my … --- ## [Want to remove some field from GROK filter](https://discuss.elastic.co/t/want-to-remove-some-field-from-grok-filter/195555)
**Author:** [@sumansaha](https://discuss.elastic.co/u/sumansaha)\ **Replies:** 4\ **Last updated:** [August 18, 2019, 11:37am UTC](https://discuss.elastic.co/t/want-to-remove-some-field-from-grok-filter/195555 "2019-08-18T11:37:25Z")
I've this log format: QID: 13636 from: 127.0.0.1 qtype: A qclass: IN qname: acep.top. rcode: NXDOMAIN rrcount: 0 policy: 'dns-name.host.sum' I've used this GROK filter : %{WORD:qid}: %{NUMBER:id} %{WORD:from}: %{IP:cl… --- ## [Elasticsearch not reachable](https://discuss.elastic.co/t/elasticsearch-not-reachable/195419)
**Author:** [@muzamil47](https://discuss.elastic.co/u/muzamil47)\ **Replies:** 8\ **Last updated:** [August 18, 2019, 11:29am UTC](https://discuss.elastic.co/t/elasticsearch-not-reachable/195419 "2019-08-18T11:29:20Z")
I am trying to load CSV data into Elasticsearch using Logstash and visualize the data in Kibana, but when I run logstash config file, I am getting Elasticseach not reachable error, though my elasticsearch server is runni… --- ## [Create Grok Filter for multiple log entries](https://discuss.elastic.co/t/create-grok-filter-for-multiple-log-entries/195625)
**Author:** [@rakeshthakur](https://discuss.elastic.co/u/rakeshthakur)\ **Replies:** 0\ **Last updated:** [August 18, 2019, 2:55am UTC](https://discuss.elastic.co/t/create-grok-filter-for-multiple-log-entries/195625 "2019-08-18T02:55:48Z")
Hi I am newbie here and i am stuck on one issue here to generate GROK pattern here. i want to configure Hardware Logs which is in multiple line format. i already got GROK pattern but not able to combine in one file. … --- ## [Reading a file with file input](https://discuss.elastic.co/t/reading-a-file-with-file-input/195604)
**Author:** [@Lavinia\_Maties](https://discuss.elastic.co/u/Lavinia_Maties)\ **Replies:** 6\ **Last updated:** [August 17, 2019, 7:50pm UTC](https://discuss.elastic.co/t/reading-a-file-with-file-input/195604 "2019-08-17T19:50:23Z")
Hi everyone, I have a problem parsing my file using Logstash on windows 10. this is my .conf file : input { file { path =\> "C:\\ProgramData\\ELK\\Logs\\CBS\\CBS.log" } } filter { grok { match =\> { "message" =\> "%{DAT… --- ## [7.2.0: Unknown setting 'ssl' for redis -\> canot connect to redis via TLS](https://discuss.elastic.co/t/7-2-0-unknown-setting-ssl-for-redis-canot-connect-to-redis-via-tls/190184)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 4\ **Last updated:** [August 17, 2019, 4:55pm UTC](https://discuss.elastic.co/t/7-2-0-unknown-setting-ssl-for-redis-canot-connect-to-redis-via-tls/190184 "2019-08-17T16:55:34Z")
Hi, I am using redis behind stunnel to gain TLS encryption for redis. Filebeat is running fine, ships it's data sucessfully to redis database. Now I want to establish secured connection from logstash to redis. I updat… --- ## [If expression confusion](https://discuss.elastic.co/t/if-expression-confusion/195409)
**Author:** [@nnet](https://discuss.elastic.co/u/nnet)\ **Replies:** 3\ **Last updated:** [August 17, 2019, 4:27pm UTC](https://discuss.elastic.co/t/if-expression-confusion/195409 "2019-08-17T16:27:59Z")
Hi, in trying to create an conditional if expression, I started with: if \[tags\] == "blah" or \[tags\] == "blah2" or \[tags\] == "blah3" { blah } but event data wasn't being "caught" by this expression. I then changed the … --- ## [Logstash only binding to IPv6 in Docker](https://discuss.elastic.co/t/logstash-only-binding-to-ipv6-in-docker/195608)
**Author:** [@thedraketaylor](https://discuss.elastic.co/u/thedraketaylor)\ **Replies:** 0\ **Last updated:** [August 17, 2019, 4:20pm UTC](https://discuss.elastic.co/t/logstash-only-binding-to-ipv6-in-docker/195608 "2019-08-17T16:20:50Z")
I've been working with logstash for the past week and so far it's been great. Yesterday I decided to switch from using it in a vagrant box to docker, so I downloaded the official docker image from docker.elastic.co/logst… --- ## [Nosql data ingestion using logstash](https://discuss.elastic.co/t/nosql-data-ingestion-using-logstash/195486)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 2\ **Last updated:** [August 17, 2019, 8:09am UTC](https://discuss.elastic.co/t/nosql-data-ingestion-using-logstash/195486 "2019-08-17T08:09:37Z")
Hi All, Anybody have ever tried data ingestion using logstash from a "nosql couchbase" database? I'm unable to find out exact driver for this, can someone advice on how to proceed in this. Thanks Gauti --- ## [I have the date time in the first log line.. I am trying to add this date to all following events after it has been matched once, making it act like a global variable in some ways](https://discuss.elastic.co/t/i-have-the-date-time-in-the-first-log-line-i-am-trying-to-add-this-date-to-all-following-events-after-it-has-been-matched-once-making-it-act-like-a-global-variable-in-some-ways/195584)
**Author:** [@satyam2593](https://discuss.elastic.co/u/satyam2593)\ **Replies:** 0\ **Last updated:** [August 17, 2019, 4:41am UTC](https://discuss.elastic.co/t/i-have-the-date-time-in-the-first-log-line-i-am-trying-to-add-this-date-to-all-following-events-after-it-has-been-matched-once-making-it-act-like-a-global-variable-in-some-ways/195584 "2019-08-17T04:41:52Z")
Here is the sample log lines START the log printing 08/08/19 09:10:06 343 7066|DestroyJavaVM|0.0|11.3 7072|Reference Handler|0.0|11.3 31204|qtp1459076321-3197|0.0|11.3 There is no unique values to use the aggregat… --- ## [Can I enrich lookup source(nested) with jdbc\_static filter?](https://discuss.elastic.co/t/can-i-enrich-lookup-source-nested-with-jdbc-static-filter/194783)
**Author:** [@sung-sang\_kang](https://discuss.elastic.co/u/sung-sang_kang)\ **Replies:** 6\ **Last updated:** [August 17, 2019, 2:49am UTC](https://discuss.elastic.co/t/can-i-enrich-lookup-source-nested-with-jdbc-static-filter/194783 "2019-08-17T02:49:23Z")
I some have source with generator Here generator { lines =\> \[ '{ "macAddr": "d880397a3b0f", "sensors": \[ {"sensor-id": "s01", "sensor-val": 31}, {"sensor-id": "s02", "se… --- ## [Best approach to split arrays of objects?](https://discuss.elastic.co/t/best-approach-to-split-arrays-of-objects/195567)
**Author:** [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Replies:** 0\ **Last updated:** [August 16, 2019, 9:47pm UTC](https://discuss.elastic.co/t/best-approach-to-split-arrays-of-objects/195567 "2019-08-16T21:47:13Z")
I've been trying to ingest audit logs from Google Cloud, but the logs contain several arrays of objects. What would be the best way to split the arrays? Let's say if there are 10-20 different arrays, not in a single log … --- ## [Time Series Index](https://discuss.elastic.co/t/time-series-index/195541)
**Author:** [@mhare](https://discuss.elastic.co/u/mhare)\ **Replies:** 6\ **Last updated:** [August 16, 2019, 9:03pm UTC](https://discuss.elastic.co/t/time-series-index/195541 "2019-08-16T21:03:41Z")
Using ELK 7.3 on Ubuntu 18.04 I am gathering memory and cpu data every 5 min from several devices under test, looking for memory leaks in the firmware. This is written to disk as a csv file. Since my visualizers are al… --- ## [Logstash - csv parsing failure](https://discuss.elastic.co/t/logstash-csv-parsing-failure/195415)
**Author:** [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Replies:** 2\ **Last updated:** [August 16, 2019, 8:52pm UTC](https://discuss.elastic.co/t/logstash-csv-parsing-failure/195415 "2019-08-16T20:52:22Z")
Hi Team, I have below data in a csv file which has two columns only. As you see, the content in msg columns actually are a few of key-vlaue pairs, and I want logstash to parse these key-value pairs as well. indent pre… --- ## [How to connect to an API using Logstash?](https://discuss.elastic.co/t/how-to-connect-to-an-api-using-logstash/192123)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 8\ **Last updated:** [August 14, 2019, 11:21pm UTC](https://discuss.elastic.co/t/how-to-connect-to-an-api-using-logstash/192123 "2019-08-14T23:21:08Z")
I have an API library written in Python, and I also have data that comes from AWS Athena through queries. Is there any way to merge these two data sources together using only Logstash? If not what are other suggestions? … --- ## [How can I send syslogs to ELK Cloud with rsyslog?](https://discuss.elastic.co/t/how-can-i-send-syslogs-to-elk-cloud-with-rsyslog/195509)
**Author:** [@debjit](https://discuss.elastic.co/u/debjit)\ **Replies:** 4\ **Last updated:** [August 16, 2019, 7:59pm UTC](https://discuss.elastic.co/t/how-can-i-send-syslogs-to-elk-cloud-with-rsyslog/195509 "2019-08-16T19:59:34Z")
How can I send syslogs to ELK Cloud with rsyslog without using filebeat ? I am evaluating ELK at the moment and using Elastic Cloud trial. I can see there are Pipelines for Logstash ? What is the endpoint for Logstash on… --- ## [Which ways to copy subfields of kubernetes.container.\_module.labels.\* to kubernetes.lables.\*?](https://discuss.elastic.co/t/which-ways-to-copy-subfields-of-kubernetes-container-module-labels-to-kubernetes-lables/195546)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 4\ **Last updated:** [August 16, 2019, 7:46pm UTC](https://discuss.elastic.co/t/which-ways-to-copy-subfields-of-kubernetes-container-module-labels-to-kubernetes-lables/195546 "2019-08-16T19:46:22Z")
Hi, to simplify filtering I want to copy all substructure of kubernetes.container.\_module.labels.\* to kubernetes.lables.\*. Just moving a copy levels up. The structure below .labels is dynamic, so I cannot hardcode the … --- ## [Logstash data transit time and possible line loss before elasticsearch](https://discuss.elastic.co/t/logstash-data-transit-time-and-possible-line-loss-before-elasticsearch/195451)
**Author:** [@Igorr](https://discuss.elastic.co/u/Igorr)\ **Replies:** 1\ **Last updated:** [August 16, 2019, 6:34pm UTC](https://discuss.elastic.co/t/logstash-data-transit-time-and-possible-line-loss-before-elasticsearch/195451 "2019-08-16T18:34:49Z")
Data on time leaves logstash (tracked by tcpdump) and is delayed in kibana. What monitoring tools are best used to track at what stage data is delayed? There are no errors in logstash logs. To check for possible loss o… --- ## [Help wtih KV Plugin](https://discuss.elastic.co/t/help-wtih-kv-plugin/195265)
**Author:** [@uklipse](https://discuss.elastic.co/u/uklipse)\ **Replies:** 6\ **Last updated:** [August 16, 2019, 6:19pm UTC](https://discuss.elastic.co/t/help-wtih-kv-plugin/195265 "2019-08-16T18:19:18Z")
I have a cef log being sent to us but I can't get the kv plugin to work. When I apply this config, I don't get any errors but I don't see any logs from it so it's breaking it somehow. Below is the config I'm giving it an… --- ## [How do I configure logstash for an Ubuntu service?](https://discuss.elastic.co/t/how-do-i-configure-logstash-for-an-ubuntu-service/195518)
**Author:** [@mhare](https://discuss.elastic.co/u/mhare)\ **Replies:** 5\ **Last updated:** [August 16, 2019, 5:34pm UTC](https://discuss.elastic.co/t/how-do-i-configure-logstash-for-an-ubuntu-service/195518 "2019-08-16T17:34:05Z")
Specifically, when I was running logstash as a process I ran it like: bin/logstash -f logstash-simple.conf But I have stopped that madness and want to run it as a service I have the service installed from the debian p… --- ## [I want to add new field and replace the one of the existing field value to the newly added field](https://discuss.elastic.co/t/i-want-to-add-new-field-and-replace-the-one-of-the-existing-field-value-to-the-newly-added-field/195303)
**Author:** [@satyam2593](https://discuss.elastic.co/u/satyam2593)\ **Replies:** 0\ **Last updated:** [August 15, 2019, 6:31am UTC](https://discuss.elastic.co/t/i-want-to-add-new-field-and-replace-the-one-of-the-existing-field-value-to-the-newly-added-field/195303 "2019-08-15T06:31:39Z")
The below is the input for the config file.. Here i want to grep the date from the one of the line and store into a field(called logdate).. Later i want to create the new field(called log\_timestamp) with null value and … --- ## [Parsing log message optional fields](https://discuss.elastic.co/t/parsing-log-message-optional-fields/195516)
**Author:** [@mahmoud\_shiri\_varami](https://discuss.elastic.co/u/mahmoud_shiri_varami)\ **Replies:** 3\ **Last updated:** [August 16, 2019, 4:27pm UTC](https://discuss.elastic.co/t/parsing-log-message-optional-fields/195516 "2019-08-16T16:27:38Z")
I'm supporting an application that generates log messages that contain 65 fields. More than 40 fields are optional that means sometimes according to application logic, optional fields exits on log message and sometimes n… --- ## [Logstash "idle" CPU is 100%](https://discuss.elastic.co/t/logstash-idle-cpu-is-100/195517)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 2\ **Last updated:** [August 16, 2019, 3:46pm UTC](https://discuss.elastic.co/t/logstash-idle-cpu-is-100/195517 "2019-08-16T15:46:43Z")
I've found several similar posts about logstash using "100%" CPU but none seem to be answered unless it was caused by a bad config. I think our several logstash systems are functioning correctly, but all use at least al… --- ## [Help to handle the nested JSON array](https://discuss.elastic.co/t/help-to-handle-the-nested-json-array/195469)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 3\ **Last updated:** [August 16, 2019, 2:54pm UTC](https://discuss.elastic.co/t/help-to-handle-the-nested-json-array/195469 "2019-08-16T14:54:57Z")
Dear Elastic folks, We have a complex JSON data with multiple nested JSON arrays need to send to elastic search, Basically, we want to split the JSON array. In short, we want to split a single big event with multiple… --- ## [\[Share\] On mapper \[geoip. location\] of different type, current\_type \[geo\_point\], merged\_type \[ObjectMapper\]the problem of processing](https://discuss.elastic.co/t/share-on-mapper-geoip-location-of-different-type-current-type-geo-point-merged-type-objectmapper-the-problem-of-processing/195512)
**Author:** [@xudahuo](https://discuss.elastic.co/u/xudahuo)\ **Replies:** 0\ **Last updated:** [August 16, 2019, 2:27pm UTC](https://discuss.elastic.co/t/share-on-mapper-geoip-location-of-different-type-current-type-geo-point-merged-type-objectmapper-the-problem-of-processing/195512 "2019-08-16T14:27:55Z")
Today, I use the geop module of logstash\_7.3 to write geographic information to elasticsearch, but I have been unable to write data. The logstash output: mapper \[geoip.location\] of different type, current\_type \[Preform… --- ## [How to import newly added record from sql server to elasticsearch](https://discuss.elastic.co/t/how-to-import-newly-added-record-from-sql-server-to-elasticsearch/195508)
**Author:** [@rameshkr1994](https://discuss.elastic.co/u/rameshkr1994)\ **Replies:** 0\ **Last updated:** [August 16, 2019, 2:02pm UTC](https://discuss.elastic.co/t/how-to-import-newly-added-record-from-sql-server-to-elasticsearch/195508 "2019-08-16T14:02:59Z")
Dear All. Hope doing well. need some help from your side:- Case:-1 i have three different table in sql\_server side and i am using join here based on single column that column is matching with all table records. ca… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=490) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=492)