# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1015 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1016 --- ## [Xml filter : remove a useless node name before array](https://discuss.elastic.co/t/xml-filter-remove-a-useless-node-name-before-array/79284)
**Author:** [@Lemec\_Cinq](https://discuss.elastic.co/u/Lemec_Cinq)\ **Replies:** 1\ **Last updated:** [March 20, 2017, 9:00pm UTC](https://discuss.elastic.co/t/xml-filter-remove-a-useless-node-name-before-array/79284 "2017-03-20T21:00:33Z")
I have a microsoft sql input and i parse an xml column thanks to the filter xml My sql column contains xml like bellow: \ \ \ **Author:** [@jaijhala](https://discuss.elastic.co/u/jaijhala)\ **Replies:** 2\ **Last updated:** [March 20, 2017, 7:59pm UTC](https://discuss.elastic.co/t/twitter-plugin-giving-errors/78964 "2017-03-20T19:59:46Z") I'm trying to implement this example in my lab -\> https://github.com/elastic/examples/tree/master/ElasticStack\_twitter I'm using version 5.2.1 But when I start logstash I'm seeing below errors -\> \[2017-03-16T22:32… --- ## [Logstash elasticsearch output plugin hangs at health check](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-hangs-at-health-check/77040)
**Author:** [@addyk](https://discuss.elastic.co/u/addyk)\ **Replies:** 2\ **Last updated:** [March 20, 2017, 6:24pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-hangs-at-health-check/77040 "2017-03-20T18:24:52Z")
I am using a Logstash ver 5.2 docker image and outputting to Elastic Cloud with ver 5.1.2. The appropriate environment variables have been set to correspond with my output. output { stdout { codec =\> rubydebug } … --- ## [HTTP Plugin port numbers](https://discuss.elastic.co/t/http-plugin-port-numbers/79153)
**Author:** [@rpazhyannur](https://discuss.elastic.co/u/rpazhyannur)\ **Replies:** 4\ **Last updated:** [March 20, 2017, 4:28pm UTC](https://discuss.elastic.co/t/http-plugin-port-numbers/79153 "2017-03-20T16:28:03Z")
I have two scripts using the HTTP plug listening on different ports (log stash is running on centos on an AWS instance) However, when I post something on one port both scripts are getting the HTTP request. script 1 in… --- ## [Good filter/output for Tomcat logs](https://discuss.elastic.co/t/good-filter-output-for-tomcat-logs/79109)
**Author:** [@alfredo.deluca](https://discuss.elastic.co/u/alfredo.deluca)\ **Replies:** 2\ **Last updated:** [March 20, 2017, 12:28pm UTC](https://discuss.elastic.co/t/good-filter-output-for-tomcat-logs/79109 "2017-03-20T12:28:09Z")
Hi all. ELK newbie here. We have tomcat6 and I'd like to filter catalina.out to 2 different alarming tools Nagios and Zabbix. Any suggestions/clue/docs? Appreciated Alfredo --- ## [Logstash field split](https://discuss.elastic.co/t/logstash-field-split/78693)
**Author:** [@wolfghost](https://discuss.elastic.co/u/wolfghost)\ **Replies:** 20\ **Last updated:** [March 20, 2017, 8:49am UTC](https://discuss.elastic.co/t/logstash-field-split/78693 "2017-03-20T08:49:34Z")
How to split "content" =\> "command hitesh.restaurants command: drop { drop: \\"restaurants\\" } keyUpdates:0 writeConflicts:0 numYields:0 reslen:81 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireC… --- ## [Getting errors while running tests for plugin](https://discuss.elastic.co/t/getting-errors-while-running-tests-for-plugin/74300)
**Author:** [@mkelkar](https://discuss.elastic.co/u/mkelkar)\ **Replies:** 3\ **Last updated:** [March 20, 2017, 11:08am UTC](https://discuss.elastic.co/t/getting-errors-while-running-tests-for-plugin/74300 "2017-03-20T11:08:44Z")
Hi All, I am trying to write tests for my logstash plugin. When I try to run rspec, this is what I get bundle exec rspec NameError: missing class name (\`org.apache.logging.log4j.Level') method\_missing at … --- ## [How can I schedule Logstash jdbc input ro run on specific minutes?](https://discuss.elastic.co/t/how-can-i-schedule-logstash-jdbc-input-ro-run-on-specific-minutes/79229)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [March 20, 2017, 11:04am UTC](https://discuss.elastic.co/t/how-can-i-schedule-logstash-jdbc-input-ro-run-on-specific-minutes/79229 "2017-03-20T11:04:12Z")
Hi, I am using ELK GA 5.0.0. I have set Logsrash jdbc to run on every 15th minute like; schedule =\> "15 \* \* \* \*" Now, I want to change this scheduler, to run on every 0th, 15th, 30th and 45th minute. How can I do th… --- ## [Running logstash automatically on Updating Database](https://discuss.elastic.co/t/running-logstash-automatically-on-updating-database/79206)
**Author:** [@Varun\_Patel](https://discuss.elastic.co/u/Varun_Patel)\ **Replies:** 2\ **Last updated:** [March 20, 2017, 8:51am UTC](https://discuss.elastic.co/t/running-logstash-automatically-on-updating-database/79206 "2017-03-20T08:51:58Z")
Hi, Is there any way that whenever my Database is updated logstash run's automatically and data is pushed to ealsticsearch. Note: Right now whenever i have new data i am manually running logstash and data is pushed… --- ## [How to capture .net error logs with grok](https://discuss.elastic.co/t/how-to-capture-net-error-logs-with-grok/79219)
**Author:** [@Radiatelabs](https://discuss.elastic.co/u/Radiatelabs)\ **Replies:** 0\ **Last updated:** [March 20, 2017, 8:40am UTC](https://discuss.elastic.co/t/how-to-capture-net-error-logs-with-grok/79219 "2017-03-20T08:40:07Z")
Hi, I am new to ELK i am trying to setup grok pattern to capture some .net error logs. The logs look like this : 2017-03-20 17:52:31,020 \[00000000-0000-0000-0000-000000000000\] ERROR ABC.XYZervices.Logger.Log4NetLogg… --- ## [Logstash metrics](https://discuss.elastic.co/t/logstash-metrics/79022)
**Author:** [@bush](https://discuss.elastic.co/u/bush)\ **Replies:** 2\ **Last updated:** [March 20, 2017, 8:17am UTC](https://discuss.elastic.co/t/logstash-metrics/79022 "2017-03-20T08:17:37Z")
Good day everyone. I have a problem configuring logstash to store metrics. So the goal is to have statistics based on connected servers. I want to see how many messages each of my servers emit in a point of time. loo… --- ## [Logstash configuration](https://discuss.elastic.co/t/logstash-configuration/74982)
**Author:** [@Nikparab](https://discuss.elastic.co/u/Nikparab)\ **Replies:** 46\ **Last updated:** [March 20, 2017, 8:04am UTC](https://discuss.elastic.co/t/logstash-configuration/74982 "2017-03-20T08:04:32Z")
I need to parse mongodb logs in elasticsearch using logstash. I need to index only command that fired in mongodb logs. I have following config file of logstash:- input { file { path =\> "C:/Data/log/mongod.log" … --- ## [How to use logstash-jdbc-input-plugin and also denormalize data?](https://discuss.elastic.co/t/how-to-use-logstash-jdbc-input-plugin-and-also-denormalize-data/79166)
**Author:** [@abhinavzspace](https://discuss.elastic.co/u/abhinavzspace)\ **Replies:** 1\ **Last updated:** [March 20, 2017, 7:22am UTC](https://discuss.elastic.co/t/how-to-use-logstash-jdbc-input-plugin-and-also-denormalize-data/79166 "2017-03-20T07:22:20Z")
Suppose I have two tables like this in postgres: User \* id \* name \* country\_id Country \* id \* name Now when I do: jdbc { ... statement =\> "SELECT \* FROM user JOIN country ON country.id = user.country\_id… --- ## [Logstash Statment Dynamic Parameter](https://discuss.elastic.co/t/logstash-statment-dynamic-parameter/79037)
**Author:** [@Yur\_Gasparyan](https://discuss.elastic.co/u/Yur_Gasparyan)\ **Replies:** 1\ **Last updated:** [March 20, 2017, 6:22am UTC](https://discuss.elastic.co/t/logstash-statment-dynamic-parameter/79037 "2017-03-20T06:22:46Z")
Hi all. I have tried to connect logstash with mysql and now I am getting error associated with dynamic parameter of statment. jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/mydb" jdbc\_user =\> "vuser" jdbc\_… --- ## [How to use filter in logstash for particular field with dynamic query string value?](https://discuss.elastic.co/t/how-to-use-filter-in-logstash-for-particular-field-with-dynamic-query-string-value/36883)
**Author:** [@Aravinthan\_Asokan](https://discuss.elastic.co/u/Aravinthan_Asokan)\ **Replies:** 5\ **Last updated:** [March 20, 2017, 6:14am UTC](https://discuss.elastic.co/t/how-to-use-filter-in-logstash-for-particular-field-with-dynamic-query-string-value/36883 "2017-03-20T06:14:48Z")
I just able to search to my index using below query xx:9200/myindex/\_search?q=keyword for above search i can able to get results .. how to do another dynamic value in the url and search into particular field dynamica… --- ## [Too many open files while running logstash](https://discuss.elastic.co/t/too-many-open-files-while-running-logstash/78883)
**Author:** [@spraveenjd](https://discuss.elastic.co/u/spraveenjd)\ **Replies:** 1\ **Last updated:** [March 20, 2017, 2:52am UTC](https://discuss.elastic.co/t/too-many-open-files-while-running-logstash/78883 "2017-03-20T02:52:20Z")
not able to post the data to elastic search. It is error out with too many open files. Please help to find the issue The logs "type"=\>"exception", "reason"=\>"failed to sync translog", "caused\_by"=\>{"type"=\>"file\_syst… --- ## [Logstash 5 alpha5-1 - Failed to load settings file from "path.settings". Aborting](https://discuss.elastic.co/t/logstash-5-alpha5-1-failed-to-load-settings-file-from-path-settings-aborting/58360)
**Author:** [@rwagner](https://discuss.elastic.co/u/rwagner)\ **Replies:** 15\ **Last updated:** [March 19, 2017, 7:50pm UTC](https://discuss.elastic.co/t/logstash-5-alpha5-1-failed-to-load-settings-file-from-path-settings-aborting/58360 "2017-03-19T19:50:44Z")
I'm getting the following error when try run: # /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.1… --- ## [Logstash Date Filter: Translation Missing](https://discuss.elastic.co/t/logstash-date-filter-translation-missing/79182)
**Author:** [@BenMatheja](https://discuss.elastic.co/u/BenMatheja)\ **Replies:** 0\ **Last updated:** [March 19, 2017, 5:58pm UTC](https://discuss.elastic.co/t/logstash-date-filter-translation-missing/79182 "2017-03-19T17:58:57Z")
I'm trying to use the date filter to parse an nginx access log. filter { if \[type\] == "nginx\_access" { grok { match =\> { "message" =\> "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra\_fields}" } } #locale is … --- ## [How to debug further? Logstash not reading Kafka input](https://discuss.elastic.co/t/how-to-debug-further-logstash-not-reading-kafka-input/79140)
**Author:** [@GregDalton](https://discuss.elastic.co/u/GregDalton)\ **Replies:** 5\ **Last updated:** [March 19, 2017, 3:27pm UTC](https://discuss.elastic.co/t/how-to-debug-further-logstash-not-reading-kafka-input/79140 "2017-03-19T15:27:09Z")
Hey all, looking for some help on where to look next: I'm playing with a pipeline that has filebeat push some log data onto a kafka topic, and then logstash read that and do some filtering and ultimately push to Elasti… --- ## [How to set \_parent properly using Logstash 2.0](https://discuss.elastic.co/t/how-to-set--parent-properly-using-logstash-2-0/33497)
**Author:** [@drdebian](https://discuss.elastic.co/u/drdebian)\ **Replies:** 3\ **Last updated:** [March 19, 2017, 10:43am UTC](https://discuss.elastic.co/t/how-to-set--parent-properly-using-logstash-2-0/33497 "2017-03-19T10:43:56Z")
I'm trying to load some tables from a relational database into ES 2.0 using logstash. I've created the apropriate mappings in ES to reflect the relationships between the tables (basically one is the parent with the other… --- ## [Append an object (nested) from Logstash to Elasticsearch](https://discuss.elastic.co/t/append-an-object-nested-from-logstash-to-elasticsearch/62162)
**Author:** [@adrianolimit](https://discuss.elastic.co/u/adrianolimit)\ **Replies:** 2\ **Last updated:** [March 19, 2017, 7:57am UTC](https://discuss.elastic.co/t/append-an-object-nested-from-logstash-to-elasticsearch/62162 "2017-03-19T07:57:32Z")
Hello Elasticusers, I have a little issue, I have a the following configuration in Logstash : input : jdbc to execute a SQL query filter : mutate to rename the fields of the SQL query and to create some objects outpu… --- ## [CSV output has been broken since 5.0](https://discuss.elastic.co/t/csv-output-has-been-broken-since-5-0/77188)
**Author:** [@j2man](https://discuss.elastic.co/u/j2man)\ **Replies:** 1\ **Last updated:** [March 19, 2017, 5:25am UTC](https://discuss.elastic.co/t/csv-output-has-been-broken-since-5-0/77188 "2017-03-19T05:25:26Z")
We need to stop letting this fall off due to inactivity. Also bug report has been generated here: -j --- ## [Pipeline.batch.delay maximum delay clarification](https://discuss.elastic.co/t/pipeline-batch-delay-maximum-delay-clarification/79158)
**Author:** [@Exocomp](https://discuss.elastic.co/u/Exocomp)\ **Replies:** 0\ **Last updated:** [March 19, 2017, 2:41am UTC](https://discuss.elastic.co/t/pipeline-batch-delay-maximum-delay-clarification/79158 "2017-03-19T02:41:58Z")
Hi, Reading the documentation here it says that: Pipeline batch delay is the maximum amount of time in milliseconds that Logstash waits for new messages after receiving an event in the current pipeline worker thread. … --- ## [Elasticsearch reindex using logstash](https://discuss.elastic.co/t/elasticsearch-reindex-using-logstash/79143)
**Author:** [@Srivastavark](https://discuss.elastic.co/u/Srivastavark)\ **Replies:** 4\ **Last updated:** [March 18, 2017, 9:42pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-using-logstash/79143 "2017-03-18T21:42:56Z")
Hello experts, I am re-indexing data using Logstash 2.3.3 from ES 1.4.4 to ES 2.4.0. I have 70000 doc type in my old index and 14000 doc type has been re-indexed in ES 2.4.0 but due to some error Logstash crashed. M… --- ## [Elasticsearch input is not finding any documents](https://discuss.elastic.co/t/elasticsearch-input-is-not-finding-any-documents/78984)
**Author:** [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Replies:** 1\ **Last updated:** [March 18, 2017, 8:08pm UTC](https://discuss.elastic.co/t/elasticsearch-input-is-not-finding-any-documents/78984 "2017-03-18T20:08:59Z")
Hi all, My configuration looks like this: input{ rabbitmq { key =\> "mykey" queue =\> "myqueue" host =\> "myhost" exchange =\> "amq.topic" ack =\> true auto\_delete =\> f… --- ## [How create dynamic query in logstash?](https://discuss.elastic.co/t/how-create-dynamic-query-in-logstash/79119)
**Author:** [@Yur\_Gasparyan](https://discuss.elastic.co/u/Yur_Gasparyan)\ **Replies:** 1\ **Last updated:** [March 18, 2017, 7:44pm UTC](https://discuss.elastic.co/t/how-create-dynamic-query-in-logstash/79119 "2017-03-18T19:44:09Z")
Hi all. I need your help. In my db config I have code look like this jdbc { jdbc\_connection\_string =\> "jdbc:mysql//localhost:3306/mydb" jdbc\_user =\> "root" jdbc\_password =\> "" jdbc\_driver\_library =\> "C:/server… --- ## [Logstash extract multiple regex](https://discuss.elastic.co/t/logstash-extract-multiple-regex/79039)
**Author:** [@Ollm4c](https://discuss.elastic.co/u/Ollm4c)\ **Replies:** 1\ **Last updated:** [March 18, 2017, 1:47pm UTC](https://discuss.elastic.co/t/logstash-extract-multiple-regex/79039 "2017-03-18T13:47:48Z")
Hi to everyone, I need to extract some regex (I don't know how much) in a log line such as the following "2017-01-01 2:04:08 - DEBUG --\> SELECT like "%aaaaa%" )) and like "%bbbb%" )) GROUP BY order by asc LIMIT " I… --- ## [How to match string in grok](https://discuss.elastic.co/t/how-to-match-string-in-grok/79101)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 4\ **Last updated:** [March 17, 2017, 11:59pm UTC](https://discuss.elastic.co/t/how-to-match-string-in-grok/79101 "2017-03-17T23:59:52Z")
Hi, the last part of my messages looks like these: Beginn Bürgerliche Dämmerung HH:MM:SS,05:56:19 Zustand: Vor Ende Dämmerung,0 Ende Nautische Dämmerung HH:MM:SS,19:39:04 Windgeschwindigkeit km/h,1.1 Beginn Bürgerli… --- ## [Logstash Grok Filter Issue](https://discuss.elastic.co/t/logstash-grok-filter-issue/79051)
**Author:** [@peter.alfafara](https://discuss.elastic.co/u/peter.alfafara)\ **Replies:** 1\ **Last updated:** [March 17, 2017, 11:57pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-issue/79051 "2017-03-17T23:57:02Z")
Hi, Can anyone help us in creating the proper filter for our logs. See sample below as we see in Kibana: @timestamp:March 17th 2017, 23:22:49.774 port:52,718 @version:1 host:127.0.0.1 message:{"client":{"name":"Clien… --- ## [Parsing json from http\_poller and CiscoAMP API](https://discuss.elastic.co/t/parsing-json-from-http-poller-and-ciscoamp-api/78390)
**Author:** [@Karn\_Griffen](https://discuss.elastic.co/u/Karn_Griffen)\ **Replies:** 3\ **Last updated:** [March 17, 2017, 11:14pm UTC](https://discuss.elastic.co/t/parsing-json-from-http-poller-and-ciscoamp-api/78390 "2017-03-17T23:14:07Z")
I am attempting to pull data from CiscoAMP, I am able to connect and grab the data with http\_poller, but I am having a hard time understanding how to break out the subfields. Products: Logstash (5.2.0) Elastic Cloud … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1014) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1016)