# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=636 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 637 --- ## [Logstash -\> multiple match patterns increase my load average](https://discuss.elastic.co/t/logstash-multiple-match-patterns-increase-my-load-average/159792)
**Author:** [@Sebastian\_Herrera](https://discuss.elastic.co/u/Sebastian_Herrera)\ **Replies:** 4\ **Last updated:** [December 6, 2018, 8:04pm UTC](https://discuss.elastic.co/t/logstash-multiple-match-patterns-increase-my-load-average/159792 "2018-12-06T20:04:01Z")
Hello, i am doing some parsing on my logstash. I have adde multiple patterns to my grok match, but the thing that is happening is that once i did that my load average increase too much. It is possible that the number of … --- ## [How to parse Haproxy log with grok](https://discuss.elastic.co/t/how-to-parse-haproxy-log-with-grok/159217)
**Author:** [@Momo](https://discuss.elastic.co/u/Momo)\ **Replies:** 5\ **Last updated:** [December 6, 2018, 6:00pm UTC](https://discuss.elastic.co/t/how-to-parse-haproxy-log-with-grok/159217 "2018-12-06T18:00:56Z")
Hello, I am a new elastic stack user. I want to parse the following log : Dec 3 14:05:17 f774bd25748a local0.info haproxy\[1\]: unix:1 \[03/Dec/2018:14:05:17.559\] frontend.https.services~ https.services.tac/https.servi… --- ## [Logstash: Problem with dissect](https://discuss.elastic.co/t/logstash-problem-with-dissect/159738)
**Author:** [@boutout.soukaina](https://discuss.elastic.co/u/boutout.soukaina)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 3:23pm UTC](https://discuss.elastic.co/t/logstash-problem-with-dissect/159738 "2018-12-06T15:23:44Z")
when i start logstash i have this warning: "Dissector mapping, pattern not found" and my logs can not be dissected. --- ## ['pipelines' on elastic host - where are they monitored?](https://discuss.elastic.co/t/pipelines-on-elastic-host-where-are-they-monitored/159719)
**Author:** [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 2:15pm UTC](https://discuss.elastic.co/t/pipelines-on-elastic-host-where-are-they-monitored/159719 "2018-12-06T14:15:28Z")
Looking at the 'pipelines' entries on kibana / clusters / elasticsearch / logstash. I can see the various kubernetes entries here but not the main logstash service running on the same host as elasticsearch. Is there som… --- ## [How can i send all the events in one email using logstash](https://discuss.elastic.co/t/how-can-i-send-all-the-events-in-one-email-using-logstash/159087)
**Author:** [@naga\_kunchala](https://discuss.elastic.co/u/naga_kunchala)\ **Replies:** 3\ **Last updated:** [December 6, 2018, 2:13pm UTC](https://discuss.elastic.co/t/how-can-i-send-all-the-events-in-one-email-using-logstash/159087 "2018-12-06T14:13:08Z")
how can i send all the events in one email using logstash --- ## [Jdbc client is not working in elastic search 6.5.1](https://discuss.elastic.co/t/jdbc-client-is-not-working-in-elastic-search-6-5-1/159331)
**Author:** [@afeef1915](https://discuss.elastic.co/u/afeef1915)\ **Replies:** 7\ **Last updated:** [December 6, 2018, 2:01pm UTC](https://discuss.elastic.co/t/jdbc-client-is-not-working-in-elastic-search-6-5-1/159331 "2018-12-06T14:01:52Z")
cannot sync data from mysql to elastic search . I tried to follow https://www.elastic.co/blog/logstash-jdbc-input-plugin asked in stack overflow logstash config # file: logstash-sample.conf input { jdbc { … --- ## [I am unable to use logstash to load csv files to elastic search](https://discuss.elastic.co/t/i-am-unable-to-use-logstash-to-load-csv-files-to-elastic-search/159496)
**Author:** [@lungelo\_zondo](https://discuss.elastic.co/u/lungelo_zondo)\ **Replies:** 16\ **Last updated:** [December 6, 2018, 1:27pm UTC](https://discuss.elastic.co/t/i-am-unable-to-use-logstash-to-load-csv-files-to-elastic-search/159496 "2018-12-06T13:27:03Z")
I get this on CMD but no data can be seen on elastic search. !image|689x305 C:\\Users\\zondol\\logstash-6.5.1\\bin\>logstash -f /Users/zondol/data/logstash.config Sending Logstash logs to C:/Users/zondol/logstash-6.5.1/log… --- ## [Gelf cannot parse](https://discuss.elastic.co/t/gelf-cannot-parse/159732)
**Author:** [@nr2017](https://discuss.elastic.co/u/nr2017)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 1:13pm UTC](https://discuss.elastic.co/t/gelf-cannot-parse/159732 "2018-12-06T13:13:22Z")
\<\[2018-12-05T17:29:03,698\]\[WARN \]\[logstash.inputs.gelf \] Gelfd failed to parse a message skipping {:exception=\>#\, :backtrace=\>\["/opt/logstash/ve… --- ## [How can I merge two events in logstash?](https://discuss.elastic.co/t/how-can-i-merge-two-events-in-logstash/159520)
**Author:** [@vinodanumarla](https://discuss.elastic.co/u/vinodanumarla)\ **Replies:** 4\ **Last updated:** [December 6, 2018, 12:42pm UTC](https://discuss.elastic.co/t/how-can-i-merge-two-events-in-logstash/159520 "2018-12-06T12:42:33Z")
I'm trying to parse a log file into elasticsearch through logstash. I want to send the following log as single event(i.e. as a single document) into elasticsearch. ######################################################… --- ## [Configure mongodb as input in logstash config file](https://discuss.elastic.co/t/configure-mongodb-as-input-in-logstash-config-file/159724)
**Author:** [@nandakumar\_k\_6693](https://discuss.elastic.co/u/nandakumar_k_6693)\ **Replies:** 0\ **Last updated:** [December 6, 2018, 12:11pm UTC](https://discuss.elastic.co/t/configure-mongodb-as-input-in-logstash-config-file/159724 "2018-12-06T12:11:52Z")
We want to sync data from mongodb into elastic search using logstash. we created an config file as below input { mongodb { uri =\> 'mongodb://demouser:demo@password%@localhost:27017/demodb' placeholder\_db\_dir =\> 'C:\\… --- ## [Postgresql problem parse geomretry](https://discuss.elastic.co/t/postgresql-problem-parse-geomretry/159723)
**Author:** [@elasitcboy](https://discuss.elastic.co/u/elasitcboy)\ **Replies:** 0\ **Last updated:** [December 6, 2018, 12:09pm UTC](https://discuss.elastic.co/t/postgresql-problem-parse-geomretry/159723 "2018-12-06T12:09:13Z")
Hello, need help with logstash parser... and postgresql query: SELECT st\_asgeojson(st\_makevalid(jj.point)) AS location FROM table { "\_index": "test", "\_type": "doc", "\_id": "1111", "\_version": 1, "\_score": … --- ## [Logstash out of memory with large gz input (caused by json filter)](https://discuss.elastic.co/t/logstash-out-of-memory-with-large-gz-input-caused-by-json-filter/157537)
**Author:** [@adaisley](https://discuss.elastic.co/u/adaisley)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 8:59am UTC](https://discuss.elastic.co/t/logstash-out-of-memory-with-large-gz-input-caused-by-json-filter/157537 "2018-12-06T08:59:14Z")
Hi. I've currently got a Logstash instance in which I'm trying to ingest a fairly large set of gzipped data every day (600-800mb compressed, 6m-11m events). I initially started with the file input plugin on v6.4.0 (and … --- ## [Logstash and Java 11](https://discuss.elastic.co/t/logstash-and-java-11/159570)
**Author:** [@gbourke](https://discuss.elastic.co/u/gbourke)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 10:21am UTC](https://discuss.elastic.co/t/logstash-and-java-11/159570 "2018-12-06T10:21:34Z")
Hi, I'm looking for info on the plans to move Logstash to Java 11. The only thread I see is under Elasticsearch The support matrix referenced shows support for Java 11 will be part of Elastic 6.5.x Logstash shows n… --- ## [Logstash failed to write in ES!](https://discuss.elastic.co/t/logstash-failed-to-write-in-es/159544)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 5\ **Last updated:** [December 6, 2018, 9:54am UTC](https://discuss.elastic.co/t/logstash-failed-to-write-in-es/159544 "2018-12-06T09:54:30Z")
WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at pa… --- ## [Logstash lose connection with elasticsearch](https://discuss.elastic.co/t/logstash-lose-connection-with-elasticsearch/159689)
**Author:** [@Niyaz\_Sagadiev](https://discuss.elastic.co/u/Niyaz_Sagadiev)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 9:25am UTC](https://discuss.elastic.co/t/logstash-lose-connection-with-elasticsearch/159689 "2018-12-06T09:25:30Z")
Hi guess! I have problem with logstash in docker. Sometimes my logstash container lose connection with all elasticsearch master node's. I see in logstash logs this message: Logstash logs\[2018-12-06T07:04:24,880\]\[WARN… --- ## [How to install logstash jdbc plugin](https://discuss.elastic.co/t/how-to-install-logstash-jdbc-plugin/157842)
**Author:** [@vishruti](https://discuss.elastic.co/u/vishruti)\ **Replies:** 5\ **Last updated:** [December 6, 2018, 9:08am UTC](https://discuss.elastic.co/t/how-to-install-logstash-jdbc-plugin/157842 "2018-12-06T09:08:50Z")
I have installed ELK, I installed Jruby and Bundler with gems as described byhttps://github.com/logstash-plugins/logstash-input-jdbc I seperately downloaded Jruby and installed gems. I think the next step is to install… --- ## [Updating log file is effecting date value on kibana dashboard](https://discuss.elastic.co/t/updating-log-file-is-effecting-date-value-on-kibana-dashboard/158316)
**Author:** [@Sanchit\_Gupta](https://discuss.elastic.co/u/Sanchit_Gupta)\ **Replies:** 23\ **Last updated:** [December 6, 2018, 6:02am UTC](https://discuss.elastic.co/t/updating-log-file-is-effecting-date-value-on-kibana-dashboard/158316 "2018-12-06T06:02:56Z")
Hi, I am new in ElasticSearch, please help me on this. When my log is updating(SOA Admin log), even single line added, that time My complete Kibana dashboard (discover tab) time option is updating with the current time… --- ## [Best way to handle different date formats?](https://discuss.elastic.co/t/best-way-to-handle-different-date-formats/159642)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 2\ **Last updated:** [December 6, 2018, 4:28am UTC](https://discuss.elastic.co/t/best-way-to-handle-different-date-formats/159642 "2018-12-06T04:28:21Z")
Hi, I have a a bunch of .csv files but the providing party has been nice enough to use a different date format on just about every file... Apart from using lots of if statements, is there a more practical way to have t… --- ## [Anyway to exclude IP ranges in discover?](https://discuss.elastic.co/t/anyway-to-exclude-ip-ranges-in-discover/159622)
**Author:** [@ciphee](https://discuss.elastic.co/u/ciphee)\ **Replies:** 3\ **Last updated:** [December 6, 2018, 12:17am UTC](https://discuss.elastic.co/t/anyway-to-exclude-ip-ranges-in-discover/159622 "2018-12-06T00:17:32Z")
Hello, I was wondering if there was a way to exclude a whole range of IP's from the discover search area. I can easily exclude a single IP, but I have not been able to find a reliable way to exclude a whole range, is th… --- ## [Logstash output to Elasticsearch SSL 6.3 issue](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-ssl-6-3-issue/159426)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 1\ **Last updated:** [December 5, 2018, 9:47pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-ssl-6-3-issue/159426 "2018-12-05T21:47:53Z")
I generated the cert and keays using "bin/elasticsearch-certutil" with " CERT mode" and let it generate the CA as well. A 2 node cluster and kibana are tsl and http.ssl enabled fine. Logstash output to elasticsearch … --- ## [How to parse my json logs as grok pattern in logstash's ".config" file](https://discuss.elastic.co/t/how-to-parse-my-json-logs-as-grok-pattern-in-logstashs-config-file/157915)
**Author:** [@vnallamothu](https://discuss.elastic.co/u/vnallamothu)\ **Replies:** 2\ **Last updated:** [December 3, 2018, 8:37pm UTC](https://discuss.elastic.co/t/how-to-parse-my-json-logs-as-grok-pattern-in-logstashs-config-file/157915 "2018-12-03T20:37:28Z")
Hi, my log file looks like the following: {"version":"0","id":"37a685ca-2e11-6b2-85d5-f09101abc58cec7","detail-type":"GuardDuty Finding","source":"aws.guardduty","account":"\*\*\*\*\*\*1326","time":"2018-11-09T12:07:07Z","r… --- ## ["Expected \[START\_OBJECT\] but found \[VALUE\_STRING\]"](https://discuss.elastic.co/t/expected-start-object-but-found-value-string/159558)
**Author:** [@Jrcnp](https://discuss.elastic.co/u/Jrcnp)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 2:49pm UTC](https://discuss.elastic.co/t/expected-start-object-but-found-value-string/159558 "2018-12-05T14:49:09Z")
I'm really new to the logstash concept and it's use so the problems i'm dealing with, are probably really easy. I have a folder with json files where logstash is getting the info and sending it to elasticsearch. At this… --- ## [16000 threads, then death](https://discuss.elastic.co/t/16000-threads-then-death/159554)
**Author:** [@cy\_lir](https://discuss.elastic.co/u/cy_lir)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 2:38pm UTC](https://discuss.elastic.co/t/16000-threads-then-death/159554 "2018-12-05T14:38:02Z")
Hi ! I'm currently facing an issue with logstash making it impossible to use : When it starts, logstash creates some threads (like 10 or 20) to initialize, and right after the pipelines are set up, it creates threads w… --- ## [Filter-code errors apparently cause the filter not to be applied and I get nothing in Elasticsearch](https://discuss.elastic.co/t/filter-code-errors-apparently-cause-the-filter-not-to-be-applied-and-i-get-nothing-in-elasticsearch/159262)
**Author:** [@RussellBateman](https://discuss.elastic.co/u/RussellBateman)\ **Replies:** 0\ **Last updated:** [December 4, 2018, 12:50am UTC](https://discuss.elastic.co/t/filter-code-errors-apparently-cause-the-filter-not-to-be-applied-and-i-get-nothing-in-elasticsearch/159262 "2018-12-04T00:50:31Z")
I'm trying to enhance a working filter I wrote that does the first dissect with only attempting that from now on if field source contains audit.log or using a different dissect if source contains debug.log. Then, if eith… --- ## [Xml multi elemts parsing](https://discuss.elastic.co/t/xml-multi-elemts-parsing/159552)
**Author:** [@povisx1](https://discuss.elastic.co/u/povisx1)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 2:09pm UTC](https://discuss.elastic.co/t/xml-multi-elemts-parsing/159552 "2018-12-05T14:09:28Z")
Hello, I have a problem with multi elements XML I try take values like this: xml { source =\> "parsedXML" remove\_namespaces =\> "true" force\_array =\> "false" force\_content =\> "true" xpath =\> \["//msisdn/text()", "… --- ## [Using relative paths in Logstash](https://discuss.elastic.co/t/using-relative-paths-in-logstash/159529)
**Author:** [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 12:29pm UTC](https://discuss.elastic.co/t/using-relative-paths-in-logstash/159529 "2018-12-05T12:29:00Z")
Hello, I have a Logstash exec-input that executes a bash script everyday. This Bash script is located in the logstash folder. I want to distribute that logastash installation to different machines and therefore the Logs… --- ## [Get Name of a file in parent directory and put it into a field](https://discuss.elastic.co/t/get-name-of-a-file-in-parent-directory-and-put-it-into-a-field/159534)
**Author:** [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 12:45pm UTC](https://discuss.elastic.co/t/get-name-of-a-file-in-parent-directory-and-put-it-into-a-field/159534 "2018-12-05T12:45:55Z")
Hello, I want to retrieve the name of a (the only) .jar file in the parent folder of a directory which is already named in the source field because the log came through filebeat. So my /path/to/application/ looks like … --- ## [Auto delete consumed messages in Kafka](https://discuss.elastic.co/t/auto-delete-consumed-messages-in-kafka/159533)
**Author:** [@florentmair](https://discuss.elastic.co/u/florentmair)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 12:45pm UTC](https://discuss.elastic.co/t/auto-delete-consumed-messages-in-kafka/159533 "2018-12-05T12:45:07Z")
Hello I know that i can tune the log retention directly with the config of each topy (by date, by space) but i prefer that's the consumer (input-kafka plugin) who delete consumed messages. Does it possible withe the In… --- ## [Integration arcsight with logstash](https://discuss.elastic.co/t/integration-arcsight-with-logstash/159527)
**Author:** [@sandeepnandula](https://discuss.elastic.co/u/sandeepnandula)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 12:22pm UTC](https://discuss.elastic.co/t/integration-arcsight-with-logstash/159527 "2018-12-05T12:22:22Z")
Hi All, please suggest me how do i resolve the following error when i try to run the below command to get the smart connector for arcsight the command: bin/logstash --modules arcsight --setup -M "arcsight.var.input.sm… --- ## [Ruby cipher-decrypt pad block corrupted](https://discuss.elastic.co/t/ruby-cipher-decrypt-pad-block-corrupted/159521)
**Author:** [@jmkim](https://discuss.elastic.co/u/jmkim)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 11:51am UTC](https://discuss.elastic.co/t/ruby-cipher-decrypt-pad-block-corrupted/159521 "2018-12-05T11:51:31Z")
hi! Is there a solution? I would like to decrypt the encrypted code in another script through logstash's ruby ​​filter. By the way, an error has been issued. "Could not process event: pad block corrupted" Could no… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=635) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=637)