# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=783 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 784 --- ## [I want to strip out the beginning and ending parts of the log entry](https://discuss.elastic.co/t/i-want-to-strip-out-the-beginning-and-ending-parts-of-the-log-entry/125447)
**Author:** [@JimP](https://discuss.elastic.co/u/JimP)\ **Replies:** 5\ **Last updated:** [April 10, 2018, 8:52pm UTC](https://discuss.elastic.co/t/i-want-to-strip-out-the-beginning-and-ending-parts-of-the-log-entry/125447 "2018-04-10T20:52:47Z")
Logstash is receiving this, and I want to remove " --------------- Event Log Start Here ---------------\\n" and "\\n--------------- Event Log End Here ---------------" from it. And, I want to pull Timestamp value and … --- ## [Tracking\_column in JDBC input plugin](https://discuss.elastic.co/t/tracking-column-in-jdbc-input-plugin/127362)
**Author:** [@sujeetKumar](https://discuss.elastic.co/u/sujeetKumar)\ **Replies:** 2\ **Last updated:** [April 10, 2018, 8:41pm UTC](https://discuss.elastic.co/t/tracking-column-in-jdbc-input-plugin/127362 "2018-04-10T20:41:46Z")
I just want to understand how it really works in deep. In my case, I have a table where I have a datetime column last\_modified\_date. I kept this as tracking column and I want to achieve: If the row is getting updated… --- ## [Getting Type Error but not setting a Type](https://discuss.elastic.co/t/getting-type-error-but-not-setting-a-type/127215)
**Author:** [@GregMeadows](https://discuss.elastic.co/u/GregMeadows)\ **Replies:** 6\ **Last updated:** [April 10, 2018, 8:09pm UTC](https://discuss.elastic.co/t/getting-type-error-but-not-setting-a-type/127215 "2018-04-10T20:09:20Z")
I've read the issues, solutions and blog article about removal of type fields I am NOT SETTING A TYPE FIELD but still get a type error I have a geo\_point type which I must create using an index template The following n… --- ## [Is there a way to start logstash remotely?](https://discuss.elastic.co/t/is-there-a-way-to-start-logstash-remotely/127381)
**Author:** [@elasticfunatic](https://discuss.elastic.co/u/elasticfunatic)\ **Replies:** 3\ **Last updated:** [April 10, 2018, 8:09pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-start-logstash-remotely/127381 "2018-04-10T20:09:19Z")
Here's the use case:- Windows Environment. logstash is on Server A and runs using a simple configuration file. The configuration file uses JDBC input plugin and elasticsearch output plugin. JDBC is connecting to SQL on … --- ## [Logstash with Kafka compression and decompression](https://discuss.elastic.co/t/logstash-with-kafka-compression-and-decompression/127412)
**Author:** [@hat\_trick](https://discuss.elastic.co/u/hat_trick)\ **Replies:** 5\ **Last updated:** [April 10, 2018, 8:05pm UTC](https://discuss.elastic.co/t/logstash-with-kafka-compression-and-decompression/127412 "2018-04-10T20:05:47Z")
Hi, I'm new to Elastic and very interested in this pipeline: Data Sources --\>LogStash --\> Kafka --\>LogStash --\> ElasticSearch, where the first LS specifies gzip data compression with Kafka output plugin and the second L… --- ## [Logstash downloadable filter plugins are not found by Logstash](https://discuss.elastic.co/t/logstash-downloadable-filter-plugins-are-not-found-by-logstash/127539)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 0\ **Last updated:** [April 10, 2018, 7:35pm UTC](https://discuss.elastic.co/t/logstash-downloadable-filter-plugins-are-not-found-by-logstash/127539 "2018-04-10T19:35:51Z")
Hello, I am using Logstash 6.2 currently and I am trying to get any downloadable filters to work, but when starting Logstash, I get an error stating that all of the downloaded filters cannot be found. Here is an example… --- ## [No suitable mariadb driver found](https://discuss.elastic.co/t/no-suitable-mariadb-driver-found/127380)
**Author:** [@rsaeks](https://discuss.elastic.co/u/rsaeks)\ **Replies:** 2\ **Last updated:** [April 10, 2018, 7:03pm UTC](https://discuss.elastic.co/t/no-suitable-mariadb-driver-found/127380 "2018-04-10T19:03:06Z")
Hi there, I've been trying to get logstash to connect to a mariadb instance without any luck. My config is pretty basic with: input { jdbc { jdbc\_connection\_string =\> "jdbc:mariadb://192.168.40.202:3306/munkirepo… --- ## [Need help on Grok Pattern](https://discuss.elastic.co/t/need-help-on-grok-pattern/127534)
**Author:** [@Babul\_Kumar\_Sahoo](https://discuss.elastic.co/u/Babul_Kumar_Sahoo)\ **Replies:** 0\ **Last updated:** [April 10, 2018, 6:59pm UTC](https://discuss.elastic.co/t/need-help-on-grok-pattern/127534 "2018-04-10T18:59:17Z")
Hello, I have below log format. 18:14:45.273 VERB t@-1864464640 input params: personName=Babul, groupRoleName=Engineer Group I want the grok filter for this to give the output like below: { "Time": \[ "18:14:45.27… --- ## [Grok newbie requesting guidance](https://discuss.elastic.co/t/grok-newbie-requesting-guidance/127492)
**Author:** [@camerodity](https://discuss.elastic.co/u/camerodity)\ **Replies:** 2\ **Last updated:** [April 10, 2018, 2:55pm UTC](https://discuss.elastic.co/t/grok-newbie-requesting-guidance/127492 "2018-04-10T14:55:25Z")
Greetings! I am using the grok filter in Logstash to parse some metrics output that were normally sent to Ganglia, and we want to now use in Kibana for various Dashboard/Visualizations. A typical log file entry is alon… --- ## [Logstash couchdb\_changes plugin with a host different from localhost](https://discuss.elastic.co/t/logstash-couchdb-changes-plugin-with-a-host-different-from-localhost/127498)
**Author:** [@nyakundigeorge](https://discuss.elastic.co/u/nyakundigeorge)\ **Replies:** 0\ **Last updated:** [April 10, 2018, 1:58pm UTC](https://discuss.elastic.co/t/logstash-couchdb-changes-plugin-with-a-host-different-from-localhost/127498 "2018-04-10T13:58:10Z")
I would like to connect to a Couchdb database that has a url with a slash i.e hosted in a different server. ELK version 6.0.0 Here's my config file input { couchdb\_changes{ host =\> \['app.mysite.org/db/' \] username … --- ## [Unhelpful error message for duplicate xpath reference in xml filter](https://discuss.elastic.co/t/unhelpful-error-message-for-duplicate-xpath-reference-in-xml-filter/127389)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 2\ **Last updated:** [April 10, 2018, 1:24pm UTC](https://discuss.elastic.co/t/unhelpful-error-message-for-duplicate-xpath-reference-in-xml-filter/127389 "2018-04-10T13:24:00Z")
In 6.2 this config input { generator { message =\> '\1\' count =\> 1 } } output { stdout { codec =\> rubydebug } } filter { xml { source =\> "message" xpath =\> { "/a" =\> "a" "/a" =\> "b" … --- ## [Possibility of running different versions of same plugin simultaneously](https://discuss.elastic.co/t/possibility-of-running-different-versions-of-same-plugin-simultaneously/127459)
**Author:** [@Pratheek](https://discuss.elastic.co/u/Pratheek)\ **Replies:** 2\ **Last updated:** [April 10, 2018, 1:01pm UTC](https://discuss.elastic.co/t/possibility-of-running-different-versions-of-same-plugin-simultaneously/127459 "2018-04-10T13:01:10Z")
Is it possible to run different versions of the same plugin (for instance kafka input plugin) in a single machine simultaneously? It can be run even in a separate instance of logstash. --- ## [Forwarding Logstash Alerts to Azure OMS](https://discuss.elastic.co/t/forwarding-logstash-alerts-to-azure-oms/127479)
**Author:** [@charan.gandra](https://discuss.elastic.co/u/charan.gandra)\ **Replies:** 0\ **Last updated:** [April 10, 2018, 12:22pm UTC](https://discuss.elastic.co/t/forwarding-logstash-alerts-to-azure-oms/127479 "2018-04-10T12:22:25Z")
Hi, I am not sure whether it is possible to forward the alerts from Logstash to Azure OMS portal? My case is if there is any ALERT in log files While Logstash outputting these logs to Elasticsearch I want to send it to… --- ## [Logstash is not capturing data from file beat](https://discuss.elastic.co/t/logstash-is-not-capturing-data-from-file-beat/127312)
**Author:** [@Nagu\_R\_Pujari](https://discuss.elastic.co/u/Nagu_R_Pujari)\ **Replies:** 6\ **Last updated:** [April 10, 2018, 12:00pm UTC](https://discuss.elastic.co/t/logstash-is-not-capturing-data-from-file-beat/127312 "2018-04-10T12:00:21Z")
\[2018-04-09T16:57:23,836\]\[INFO \]\[org.logstash.beats.BeatsHandler\] \[local: 192.168.36.198:5044, remote: 192.168.36.37:47240\] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame … --- ## [Logstash 6.2.3 constantly restarting](https://discuss.elastic.co/t/logstash-6-2-3-constantly-restarting/127470)
**Author:** [@souf\_el\_badraoui](https://discuss.elastic.co/u/souf_el_badraoui)\ **Replies:** 0\ **Last updated:** [April 10, 2018, 11:22am UTC](https://discuss.elastic.co/t/logstash-6-2-3-constantly-restarting/127470 "2018-04-10T11:22:28Z")
Hi, I used Centos 7 with elk stack 6.2.3. After installing x-pack, logstash contantly restarting (systemd). the logs show \[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>#\ **Author:** [@Amit\_Jaiswal](https://discuss.elastic.co/u/Amit_Jaiswal)\ **Replies:** 3\ **Last updated:** [April 10, 2018, 11:14am UTC](https://discuss.elastic.co/t/pagination-logic-in-input-of-http-poller-plugin/127427 "2018-04-10T11:14:01Z") I have to bulk import data via LogStash as below: Source: Restful Get APIs Destination : ElasticSearch My .conf file looks as below: input { http\_poller { urls =\> { test1 =\> { method =\> get url =\> "https://someu… --- ## [What does Expected one of #, {, ,, \] mean?](https://discuss.elastic.co/t/what-does-expected-one-of-mean/127269)
**Author:** [@Wuzu85](https://discuss.elastic.co/u/Wuzu85)\ **Replies:** 2\ **Last updated:** [April 10, 2018, 7:45am UTC](https://discuss.elastic.co/t/what-does-expected-one-of-mean/127269 "2018-04-10T07:45:44Z")
Hi im new to logstash what does this error mean? (version 5.6.8) Cannot create pipeline {:reason=\>"Expected one of #, {, ,, \] at line 33, column 26 input { file { path =\> \["/var/log/collectd.logstash.log"\] start\_pos… --- ## [Logstash output to tcp error it a signlge line that not ok](https://discuss.elastic.co/t/logstash-output-to-tcp-error-it-a-signlge-line-that-not-ok/127416)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 8\ **Last updated:** [April 10, 2018, 7:29am UTC](https://discuss.elastic.co/t/logstash-output-to-tcp-error-it-a-signlge-line-that-not-ok/127416 "2018-04-10T07:29:04Z")
i have josn data use udp listener and output to tcp server . to tcp server is the date is a single line . i don't want to a signlge line. need many lines. i find this at https://www.elastic.co/guide/en/logstash/5.6/… --- ## [Issue with modelling data while bulk import](https://discuss.elastic.co/t/issue-with-modelling-data-while-bulk-import/127032)
**Author:** [@Amit\_Jaiswal](https://discuss.elastic.co/u/Amit_Jaiswal)\ **Replies:** 3\ **Last updated:** [April 10, 2018, 6:55am UTC](https://discuss.elastic.co/t/issue-with-modelling-data-while-bulk-import/127032 "2018-04-10T06:55:02Z")
Hello, I have to bulk import data via LogStash as below: Source: Restful Get APIs Destination : ElasticSearch My .conf file looks as below: input { http\_poller { urls =\> { test1 =\> { method =\> get url =\> "http… --- ## [My conf file not working](https://discuss.elastic.co/t/my-conf-file-not-working/127419)
**Author:** [@Faenor2105](https://discuss.elastic.co/u/Faenor2105)\ **Replies:** 0\ **Last updated:** [April 10, 2018, 6:19am UTC](https://discuss.elastic.co/t/my-conf-file-not-working/127419 "2018-04-10T06:19:27Z")
Hi. i need help with my log. Need to parse local log information to Elasticsearch about errors in web app. I'm novice and my superiors asked me to study logstash work and conf files. Time is almost up, but i'm stuck with… --- ## [Switch case in Logstash](https://discuss.elastic.co/t/switch-case-in-logstash/127408)
**Author:** [@seungtae](https://discuss.elastic.co/u/seungtae)\ **Replies:** 2\ **Last updated:** [April 10, 2018, 5:53am UTC](https://discuss.elastic.co/t/switch-case-in-logstash/127408 "2018-04-10T05:53:25Z")
Can I use switch~case in Logstash? I can't find it in reference.. --- ## [I getting this error while logstash.conf file](https://discuss.elastic.co/t/i-getting-this-error-while-logstash-conf-file/127213)
**Author:** [@kartheek91](https://discuss.elastic.co/u/kartheek91)\ **Replies:** 2\ **Last updated:** [April 10, 2018, 5:25am UTC](https://discuss.elastic.co/t/i-getting-this-error-while-logstash-conf-file/127213 "2018-04-10T05:25:42Z")
\[image\] Please help me --- ## [Lumberjack Output Client Write Error](https://discuss.elastic.co/t/lumberjack-output-client-write-error/127409)
**Author:** [@karnamonkster](https://discuss.elastic.co/u/karnamonkster)\ **Replies:** 0\ **Last updated:** [April 10, 2018, 3:04am UTC](https://discuss.elastic.co/t/lumberjack-output-client-write-error/127409 "2018-04-10T03:04:58Z")
We have been strugling with one specific issue which comes intermittently when trying to send data from one Logstash instance to other using Lumberjack Output -Input plugin. The recieveing end never shows an error. Howe… --- ## [Kafka input with different deserializer classes](https://discuss.elastic.co/t/kafka-input-with-different-deserializer-classes/127352)
**Author:** [@guidoilbaldo](https://discuss.elastic.co/u/guidoilbaldo)\ **Replies:** 1\ **Last updated:** [April 10, 2018, 12:57am UTC](https://discuss.elastic.co/t/kafka-input-with-different-deserializer-classes/127352 "2018-04-10T00:57:53Z")
Hi, I'm trying to consume events from Kafka with Logstash to send them to Elastic. I'm currently using a pipeline with the following input: kafka { type =\> "standard\_event" bootstrap\_servers =\> "xxx01:9092,x… --- ## [Logstash S3 Input - How does it know where to start?](https://discuss.elastic.co/t/logstash-s3-input-how-does-it-know-where-to-start/127398)
**Author:** [@Shahzaib\_Gill](https://discuss.elastic.co/u/Shahzaib_Gill)\ **Replies:** 1\ **Last updated:** [April 10, 2018, 12:39am UTC](https://discuss.elastic.co/t/logstash-s3-input-how-does-it-know-where-to-start/127398 "2018-04-10T00:39:20Z")
If I define an S3 bucket as input, and logstash processes a few before being turned off, how does it know where to begin so that it doesn't process the same data again? For example, if I have 2 files in my S3 bucket , an… --- ## [Kibana can't parse the logs in discover tab](https://discuss.elastic.co/t/kibana-cant-parse-the-logs-in-discover-tab/126565)
**Author:** [@keremcan](https://discuss.elastic.co/u/keremcan)\ **Replies:** 1\ **Last updated:** [April 10, 2018, 12:26am UTC](https://discuss.elastic.co/t/kibana-cant-parse-the-logs-in-discover-tab/126565 "2018-04-10T00:26:05Z")
Hey! I'm working on nginx logs analysing with ELK stack. I'm using filebeat\>logstash\>elasticsearch\>kibana in order. this is my custom pattern NGINX %{DATA:ipadress} %{DATA:ident} %{DATA:auth} \\\[%{HTTPDATE:timestamp}\\\] … --- ## [Filter out passwords with mutate](https://discuss.elastic.co/t/filter-out-passwords-with-mutate/127345)
**Author:** [@joshiegy](https://discuss.elastic.co/u/joshiegy)\ **Replies:** 1\ **Last updated:** [April 9, 2018, 11:53pm UTC](https://discuss.elastic.co/t/filter-out-passwords-with-mutate/127345 "2018-04-09T23:53:22Z")
Hi, I'm trying to filter out passwords that icingabeat sends out. The field with password is "check\_result.command" and the password is always after "-p," or "-P,". Example line: /usr/lib/nagios/plugins/check\_mysql, … --- ## [Help with parsing xml data](https://discuss.elastic.co/t/help-with-parsing-xml-data/127383)
**Author:** [@kattula](https://discuss.elastic.co/u/kattula)\ **Replies:** 2\ **Last updated:** [April 9, 2018, 11:50pm UTC](https://discuss.elastic.co/t/help-with-parsing-xml-data/127383 "2018-04-09T23:50:01Z")
Hi everyone, I'm new to using ELK stack. I'm trying to parse an xml audit file. Even though I get the output as I need, because of the length of code I'm assuming there could be a simpler solution than what I've written… --- ## [Loading csv file using logstash in windows 7](https://discuss.elastic.co/t/loading-csv-file-using-logstash-in-windows-7/127302)
**Author:** [@Pratap\_Kinjarapu](https://discuss.elastic.co/u/Pratap_Kinjarapu)\ **Replies:** 1\ **Last updated:** [April 9, 2018, 10:05pm UTC](https://discuss.elastic.co/t/loading-csv-file-using-logstash-in-windows-7/127302 "2018-04-09T22:05:39Z")
I am trying to load csv file using logstash in windows 7 machine. My elastic search and kibana are running. I am getting below error. Please help me in resolving this error. Error: Error: Could not find or load main cl… --- ## [Logstash elasticsearch output plugin filtering](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-filtering/126795)
**Author:** [@Ferdous\_Shibly](https://discuss.elastic.co/u/Ferdous_Shibly)\ **Replies:** 7\ **Last updated:** [April 9, 2018, 8:52pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-filtering/126795 "2018-04-09T20:52:00Z")
The application are sending logs in JSON format with some additional tags for example logType, \[attribute\]\[APPLICATION\] etc. and we want apply filter on logstash based on those tags so that logstash will not send some ty… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=782) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=784)