# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=895 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 896 --- ## [Multipul level nested objects from JDBC](https://discuss.elastic.co/t/multipul-level-nested-objects-from-jdbc/103323)
**Author:** [@baggiogamp](https://discuss.elastic.co/u/baggiogamp)\ **Replies:** 0\ **Last updated:** [October 10, 2017, 8:44am UTC](https://discuss.elastic.co/t/multipul-level-nested-objects-from-jdbc/103323 "2017-10-10T08:44:49Z")
I am trying to import relational DBMS data into ES by this filter(logstash-filter-aggregate), i am not sure it is should be used as this purpose. In DB there are 3 tables, P parent, child c1, child c2 P -\> c1 one to… --- ## [High CPU usage with no config](https://discuss.elastic.co/t/high-cpu-usage-with-no-config/103322)
**Author:** [@Frizz\_Supertramp](https://discuss.elastic.co/u/Frizz_Supertramp)\ **Replies:** 0\ **Last updated:** [October 10, 2017, 8:38am UTC](https://discuss.elastic.co/t/high-cpu-usage-with-no-config/103322 "2017-10-10T08:38:36Z")
I have a Logstash installation (5.6.2) where the config files are deployed at some later stage, so at the moment /etc/logstash/config.d is empty. Unfortunately this causes 100% CPU load. Why is that? --- ## [Mssql logs through elasticsearch stack](https://discuss.elastic.co/t/mssql-logs-through-elasticsearch-stack/101586)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 4\ **Last updated:** [October 10, 2017, 8:21am UTC](https://discuss.elastic.co/t/mssql-logs-through-elasticsearch-stack/101586 "2017-10-10T08:21:20Z")
Hi Guys, Is there any use case for monitoring sql logs like Oracle, mysql, MSsql? through logstash --- ## [Date format failing in grok](https://discuss.elastic.co/t/date-format-failing-in-grok/103295)
**Author:** [@sreejiths](https://discuss.elastic.co/u/sreejiths)\ **Replies:** 2\ **Last updated:** [October 10, 2017, 7:04am UTC](https://discuss.elastic.co/t/date-format-failing-in-grok/103295 "2017-10-10T07:04:47Z")
We are using ELK 5.2 for Network device syslog Analytics . Some devices is sending date format with a special character "" in the beginning of date format for some reasons due to which grok is failing to match . Anyway… --- ## [SNMP Trap listener died Address already in use](https://discuss.elastic.co/t/snmp-trap-listener-died-address-already-in-use/103146)
**Author:** [@gilisade](https://discuss.elastic.co/u/gilisade)\ **Replies:** 5\ **Last updated:** [October 10, 2017, 6:25am UTC](https://discuss.elastic.co/t/snmp-trap-listener-died-address-already-in-use/103146 "2017-10-10T06:25:44Z")
HI I'm encountering a problem with snmptrap input using LS 5.2 this is my conf file: input{ snmptrap{ port =\> "5042" type =\> "test" community =\> "Public" … --- ## [Help needed with multiline codec, current pattern too slow](https://discuss.elastic.co/t/help-needed-with-multiline-codec-current-pattern-too-slow/102516)
**Author:** [@nikhilsangvikar](https://discuss.elastic.co/u/nikhilsangvikar)\ **Replies:** 4\ **Last updated:** [October 10, 2017, 6:08am UTC](https://discuss.elastic.co/t/help-needed-with-multiline-codec-current-pattern-too-slow/102516 "2017-10-10T06:08:24Z")
Hello, for these set of input lines: SAMPLE INPUT INFO | jvm 1 | 2017/08/28 17:54:14 | 25306.601: Total time for which application threads were stopped: 0.0008632 seconds, Stopping threads took: 0.0000905 seconds I… --- ## [Trying to fetch values from xml and feed into influxdb](https://discuss.elastic.co/t/trying-to-fetch-values-from-xml-and-feed-into-influxdb/102988)
**Author:** [@Niranjan\_kaup](https://discuss.elastic.co/u/Niranjan_kaup)\ **Replies:** 3\ **Last updated:** [October 10, 2017, 5:55am UTC](https://discuss.elastic.co/t/trying-to-fetch-values-from-xml-and-feed-into-influxdb/102988 "2017-10-10T05:55:53Z")
Sample xml file: Tove Jani Reminder Don't forget me this weekend! Below is the configuration: input { file { path =\> "/mnt/users.xml" start\_position =\> "beginning" type =\> xml } } filter { xml { source =\> … --- ## [Tcp logstash encoder](https://discuss.elastic.co/t/tcp-logstash-encoder/103051)
**Author:** [@visioncoder](https://discuss.elastic.co/u/visioncoder)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 5:34am UTC](https://discuss.elastic.co/t/tcp-logstash-encoder/103051 "2017-10-10T05:34:18Z")
Hello, I am using tcp-logstash-encoder 4.11 with SpringBoot 1.5.6. to push json log messages to Logstash. Some select messages are coming through fine but most are not. Console is printing all the messages. The TCP con… --- ## [Visualize/Obtain Logstash stats per data source](https://discuss.elastic.co/t/visualize-obtain-logstash-stats-per-data-source/103279)
**Author:** [@thestudentoftheearth](https://discuss.elastic.co/u/thestudentoftheearth)\ **Replies:** 1\ **Last updated:** [October 10, 2017, 5:30am UTC](https://discuss.elastic.co/t/visualize-obtain-logstash-stats-per-data-source/103279 "2017-10-10T05:30:25Z")
Hi. I would like to ask help regarding Logstash stats. I have configured Logstash to receive data from multiple data sources via syslog through designated ports. For example, I receive AD logs through port 5001 of Logs… --- ## [Logstash Idle Time](https://discuss.elastic.co/t/logstash-idle-time/103182)
**Author:** [@ramesh205](https://discuss.elastic.co/u/ramesh205)\ **Replies:** 2\ **Last updated:** [October 10, 2017, 5:21am UTC](https://discuss.elastic.co/t/logstash-idle-time/103182 "2017-10-10T05:21:15Z")
Hi, I am using ELK and file beat 5.5.2 version,when ever we inserted data in log file filebeat push to Logstash and kibana able to see data but we are facing an issue that every week saturday and sunday there will no lo… --- ## [Grok username in iis log](https://discuss.elastic.co/t/grok-username-in-iis-log/102543)
**Author:** [@frumat\_l](https://discuss.elastic.co/u/frumat_l)\ **Replies:** 8\ **Last updated:** [October 10, 2017, 5:17am UTC](https://discuss.elastic.co/t/grok-username-in-iis-log/102543 "2017-10-10T05:17:47Z")
Hi, I have an IIS log, example: "2017-10-01 08:29:29 10.70.4.52 GET /Shared+Documents/Forms/AllItems.aspx AjaxDelta=1&isStartPlt1=1506846557836 80 0#.w|auto\\user\_sppano\_dev 10.70.4.52 Mozilla/4.0+(compatible;+MSIE+7.0;… --- ## [Geo ip in tile map](https://discuss.elastic.co/t/geo-ip-in-tile-map/103261)
**Author:** [@bm1391](https://discuss.elastic.co/u/bm1391)\ **Replies:** 5\ **Last updated:** [October 9, 2017, 9:40pm UTC](https://discuss.elastic.co/t/geo-ip-in-tile-map/103261 "2017-10-09T21:40:10Z")
Good afternoon, I continue to receive the error " index pattern does not contain any of the following field types: geo\_point " when trying to map via tile map. Here is what my data looks like: { "\_index": "combine-in… --- ## [Logstash missing from monitoring after 5.6.2 upgrade](https://discuss.elastic.co/t/logstash-missing-from-monitoring-after-5-6-2-upgrade/102030)
**Author:** [@flalar](https://discuss.elastic.co/u/flalar)\ **Replies:** 5\ **Last updated:** [October 9, 2017, 9:07pm UTC](https://discuss.elastic.co/t/logstash-missing-from-monitoring-after-5-6-2-upgrade/102030 "2017-10-09T21:07:17Z")
After upgrading elasticsearch, kibana, logstash and corresponding x-pack plugins, Logstash monitoring is missing from Kibana. When looking at the monitoring indexes it seems the .monitoring-logstash-6-2017.09.27 index … --- ## [eMail Notification via Logstash](https://discuss.elastic.co/t/email-notification-via-logstash/103218)
**Author:** [@baldy2811](https://discuss.elastic.co/u/baldy2811)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 7:06pm UTC](https://discuss.elastic.co/t/email-notification-via-logstash/103218 "2017-10-09T19:06:08Z")
Hi there, i wanted to have email Notifications when some issues appears. I created such Output: output { elasticsearch { hosts =\> \["10.10.5.100:9200"\] manage\_template =\> false index =\> "%{\[@metadata\]\[beat\]}-%{+YYY… --- ## [Shipping Logstash to Logstash with encryption](https://discuss.elastic.co/t/shipping-logstash-to-logstash-with-encryption/103215)
**Author:** [@Mark\_van\_Rossum](https://discuss.elastic.co/u/Mark_van_Rossum)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 7:01pm UTC](https://discuss.elastic.co/t/shipping-logstash-to-logstash-with-encryption/103215 "2017-10-09T19:01:56Z")
Currently we have an ELK cluster, where Logstash has a "beats" input with SSL certificates to authenticate incoming data (mainly from metricbeat). I have a 2nd remote logstash instance that inputs data over UDP from wit… --- ## [Logistash pipline ended without any error](https://discuss.elastic.co/t/logistash-pipline-ended-without-any-error/102940)
**Author:** [@sidra\_89](https://discuss.elastic.co/u/sidra_89)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 6:50pm UTC](https://discuss.elastic.co/t/logistash-pipline-ended-without-any-error/102940 "2017-10-09T18:50:00Z")
i am using logistash to transfer data from MYSQL \[wamp\] to Elasticsearch. the query i am using is extremely complex will lots of joins and multiple values in select statement something like this select CONVERT\_TZ(so.p… --- ## [Grok parse failures for ssl\_request\_log (but none in debugger)](https://discuss.elastic.co/t/grok-parse-failures-for-ssl-request-log-but-none-in-debugger/103248)
**Author:** [@adam.stallard](https://discuss.elastic.co/u/adam.stallard)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 6:08pm UTC](https://discuss.elastic.co/t/grok-parse-failures-for-ssl-request-log-but-none-in-debugger/103248 "2017-10-09T18:08:02Z")
I have a log file from apache ssl\_request\_log which is a slightly different format from every other one in that it swaps the ordering of various fields. So I copied the macro from %{COMBINEDAPACHELOG} and made a few chan… --- ## [Gsub syntax to remove all before \\](https://discuss.elastic.co/t/gsub-syntax-to-remove-all-before/103243)
**Author:** [@Callahan](https://discuss.elastic.co/u/Callahan)\ **Replies:** 2\ **Last updated:** [October 9, 2017, 5:08pm UTC](https://discuss.elastic.co/t/gsub-syntax-to-remove-all-before/103243 "2017-10-09T17:08:31Z")
Hi, I'm trying to write up a gsub to remove everything before a slash. My input will come in the format of something similar to: LP-45XP\\joe.bloggs 3G49-2AP\\john.doe random\_PC-domain\_name\\random\_firstname.random\_last… --- ## [Environment variables error for AWS secret\_access\_key](https://discuss.elastic.co/t/environment-variables-error-for-aws-secret-access-key/103255)
**Author:** [@erikkn](https://discuss.elastic.co/u/erikkn)\ **Replies:** 2\ **Last updated:** [October 9, 2017, 5:07pm UTC](https://discuss.elastic.co/t/environment-variables-error-for-aws-secret-access-key/103255 "2017-10-09T17:07:43Z")
Hey guys :),, I am using the S3 plugin (logstash-input-s3) . I would like to use environment variables for the AWS\_ACCESS\_KEY\_ID and the AWS\_SECRET\_ACCESS\_KEY variables, but I am getting an error with the secret acces… --- ## [Can't find correct Joda Time Documentation](https://discuss.elastic.co/t/cant-find-correct-joda-time-documentation/103062)
**Author:** [@dolanmk](https://discuss.elastic.co/u/dolanmk)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 3:21pm UTC](https://discuss.elastic.co/t/cant-find-correct-joda-time-documentation/103062 "2017-10-09T15:21:44Z")
Hi, I am using Logstash to parse some install logs. I have a field named LOG\_DATE and the grok pattern is correct but I need to change the field from a string to a date. I am using the date filter plugin to do this. I h… --- ## [Grok - Positive Lookbehind](https://discuss.elastic.co/t/grok-positive-lookbehind/103005)
**Author:** [@paolo1](https://discuss.elastic.co/u/paolo1)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 1:21pm UTC](https://discuss.elastic.co/t/grok-positive-lookbehind/103005 "2017-10-09T13:21:49Z")
Hi, Does grok match support 'positive lookbehind'? I'm take everything after 'Details=' in the following json example: { "Field":"Details=xyz" } match =\> \["Field", "(?\< Field2 \>(?\<=Details=).\*)"\] I am getting grok … --- ## [Logstash v.6 error after x-pack installation](https://discuss.elastic.co/t/logstash-v-6-error-after-x-pack-installation/102712)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 7\ **Last updated:** [October 9, 2017, 1:12pm UTC](https://discuss.elastic.co/t/logstash-v-6-error-after-x-pack-installation/102712 "2017-10-09T13:12:20Z")
Hello, I have installed x-pack for Logstash. When I try to run configuration file, I get the following error: \[FATAL\] 2017-10-04 15:41:55.607 \[LogStash::Runner\] runner - An unexpected error occurred! {:error=\>#\ **Author:** [@Guirlande](https://discuss.elastic.co/u/Guirlande)\ **Replies:** 2\ **Last updated:** [October 9, 2017, 12:01pm UTC](https://discuss.elastic.co/t/logstash-log4j2-properties-gets-overriden-by-rpm/102983 "2017-10-09T12:01:22Z") We've got an issue with logstash package when updating it. There are the version informations that would be useful about OS : RedHat 7.4 LogStash 5.5.1 After a yum update and logstash is updated, if log4j2.properties… --- ## [After restart logstash the s3 output plugin not work](https://discuss.elastic.co/t/after-restart-logstash-the-s3-output-plugin-not-work/103154)
**Author:** [@sockaddr\_in](https://discuss.elastic.co/u/sockaddr_in)\ **Replies:** 2\ **Last updated:** [October 9, 2017, 11:18am UTC](https://discuss.elastic.co/t/after-restart-logstash-the-s3-output-plugin-not-work/103154 "2017-10-09T11:18:12Z")
Hello, everyone! I use the s3 as the logstash output. When i first start the logstash, it works well, but after i restart logstash is does not work at all and generated many ERROR log as following, so how to solve this … --- ## [Logstash stopped transmitting events](https://discuss.elastic.co/t/logstash-stopped-transmitting-events/103213)
**Author:** [@dandrestor](https://discuss.elastic.co/u/dandrestor)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 10:51am UTC](https://discuss.elastic.co/t/logstash-stopped-transmitting-events/103213 "2017-10-09T10:51:36Z")
Hello, I need help troubleshooting a problem that occurred Saturday night, with no change to the configuration: Logstash stopped emitting events to Elasticsearch. There is nothing in the logs that I can see to indicate … --- ## [Monitoring API value is updated only after logstash restart](https://discuss.elastic.co/t/monitoring-api-value-is-updated-only-after-logstash-restart/100971)
**Author:** [@dandrestor](https://discuss.elastic.co/u/dandrestor)\ **Replies:** 2\ **Last updated:** [October 9, 2017, 10:22am UTC](https://discuss.elastic.co/t/monitoring-api-value-is-updated-only-after-logstash-restart/100971 "2017-10-09T10:22:08Z")
Hi there I am trying to monitor the number of events that are currently pending in the persisted queue of my Logstash instance. This number seems to update only when I restart Logstash. I looked at the metric in three d… --- ## [Logstash: filter to convert string to array of values](https://discuss.elastic.co/t/logstash-filter-to-convert-string-to-array-of-values/103210)
**Author:** [@geexee](https://discuss.elastic.co/u/geexee)\ **Replies:** 0\ **Last updated:** [October 9, 2017, 10:13am UTC](https://discuss.elastic.co/t/logstash-filter-to-convert-string-to-array-of-values/103210 "2017-10-09T10:13:37Z")
Hello, I have an input log string which contains some easy to regex-matching keywords. I want to index them as an array of values. E.g. "message"="User's locations: locERTY locRUIE locFGHD locTYUS" The below code succ… --- ## [Problem with grok and multiline pattern](https://discuss.elastic.co/t/problem-with-grok-and-multiline-pattern/103201)
**Author:** [@Rodolphe\_Redoute](https://discuss.elastic.co/u/Rodolphe_Redoute)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 10:02am UTC](https://discuss.elastic.co/t/problem-with-grok-and-multiline-pattern/103201 "2017-10-09T10:02:41Z")
Hi, I have a touchy problem, I need to make a filter for a log file but here is the thing : inside this file there is some log with Multiline and the begining of theses lines is not the same than the "one line" here i… --- ## [Help needed to filter unwanted charators from kafka message in logstash](https://discuss.elastic.co/t/help-needed-to-filter-unwanted-charators-from-kafka-message-in-logstash/103203)
**Author:** [@Rameswar\_m](https://discuss.elastic.co/u/Rameswar_m)\ **Replies:** 0\ **Last updated:** [October 9, 2017, 9:38am UTC](https://discuss.elastic.co/t/help-needed-to-filter-unwanted-charators-from-kafka-message-in-logstash/103203 "2017-10-09T09:38:44Z")
hi , i am new to logstash , i am trying to pull the dta from mongo to kibana via logstash from kafka i am getting message below and my logstash conf is : message: �{"\_id" : {"$oid" : "59da4346140f1dc01e861e4b"},"nam… --- ## [Can we use ElasticSearch as the input and kafka as the output in Logstash configuration file? using kafka output plugin?](https://discuss.elastic.co/t/can-we-use-elasticsearch-as-the-input-and-kafka-as-the-output-in-logstash-configuration-file-using-kafka-output-plugin/103185)
**Author:** [@manu\_john](https://discuss.elastic.co/u/manu_john)\ **Replies:** 1\ **Last updated:** [October 9, 2017, 7:30am UTC](https://discuss.elastic.co/t/can-we-use-elasticsearch-as-the-input-and-kafka-as-the-output-in-logstash-configuration-file-using-kafka-output-plugin/103185 "2017-10-09T07:30:30Z")
Can we use ElasticSearch as the input and kafka as the output in Logstash configuration file? using kafka output plugin? If so can anyone provide the config file for that? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=894) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=896)