# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1023 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1024 --- ## [What is the best way to config multiple sql tables in logstash](https://discuss.elastic.co/t/what-is-the-best-way-to-config-multiple-sql-tables-in-logstash/62529)
**Author:** [@saisn](https://discuss.elastic.co/u/saisn)\ **Replies:** 2\ **Last updated:** [March 7, 2017, 8:17pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-config-multiple-sql-tables-in-logstash/62529 "2017-03-07T20:17:38Z")
Hi, I've multiple tables in SQL server that needs to be imported to elasticsearch using logstash. So far, I'm doing single conf for single table but it's getting hard to maintain several config files. Is it better to… --- ## [How to configure heap size on 5.x installed from .deb?](https://discuss.elastic.co/t/how-to-configure-heap-size-on-5-x-installed-from-deb/77373)
**Author:** [@FlorinAndrei](https://discuss.elastic.co/u/FlorinAndrei)\ **Replies:** 2\ **Last updated:** [March 7, 2017, 6:53pm UTC](https://discuss.elastic.co/t/how-to-configure-heap-size-on-5-x-installed-from-deb/77373 "2017-03-07T18:53:10Z")
I've installed Logstash-5.2.2 from the .deb file on Ubuntu 16.04 In /etc/default/logstash I have this: LS\_JAVA\_OPTS="-Xms8g -Xmx8g" And I can verify that the Logstash Java process indeed has the -Xmx8g -Xms8g optio… --- ## [Yum install logstash 5.2 no daemon](https://discuss.elastic.co/t/yum-install-logstash-5-2-no-daemon/77628)
**Author:** [@shaonbean](https://discuss.elastic.co/u/shaonbean)\ **Replies:** 2\ **Last updated:** [March 7, 2017, 6:41pm UTC](https://discuss.elastic.co/t/yum-install-logstash-5-2-no-daemon/77628 "2017-03-07T18:41:34Z")
logstash I use logstash repo yum install logstash, when install complete, i want to use /etc/init.d/logstash start logstash,but no logstash daemon. it's right ? --- ## [Best way to parse json input?](https://discuss.elastic.co/t/best-way-to-parse-json-input/29809)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 11\ **Last updated:** [March 7, 2017, 5:57pm UTC](https://discuss.elastic.co/t/best-way-to-parse-json-input/29809 "2017-03-07T17:57:47Z")
I have json in a file in the format: { events : \[ { prop1 : val1, prop2 : val2 }, { prop1 : val3, prop2 : val4 } \] } I've tried multiline input with no success. I tried a mutate filt… --- ## [How to check logstash sends logs?](https://discuss.elastic.co/t/how-to-check-logstash-sends-logs/77314)
**Author:** [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Replies:** 4\ **Last updated:** [March 7, 2017, 5:56pm UTC](https://discuss.elastic.co/t/how-to-check-logstash-sends-logs/77314 "2017-03-07T17:56:55Z")
Hi, In Kibana I don't recieve any logs so I can't start with the dashboard. How can I check that Logstash is sending something to Elasticsearch? Thanks --- ## [Multiline handling for log4net](https://discuss.elastic.co/t/multiline-handling-for-log4net/77700)
**Author:** [@CLLI](https://discuss.elastic.co/u/CLLI)\ **Replies:** 0\ **Last updated:** [March 7, 2017, 4:53pm UTC](https://discuss.elastic.co/t/multiline-handling-for-log4net/77700 "2017-03-07T16:53:42Z")
Hello there, I am looking at ways of centralizing my logs and the ELK stack is very impressive. While trying to evaluate it I've gotten stuck on the handling of multiline logs. I've googled and read posts from up to a f… --- ## [Logstash can not parse azure activity logs](https://discuss.elastic.co/t/logstash-can-not-parse-azure-activity-logs/76801)
**Author:** [@111148](https://discuss.elastic.co/u/111148)\ **Replies:** 1\ **Last updated:** [March 7, 2017, 4:08pm UTC](https://discuss.elastic.co/t/logstash-can-not-parse-azure-activity-logs/76801 "2017-03-07T16:08:16Z")
Hello, I have faced with issue during parsing Azure activity logs (multiline json) by Logstash. Tried several codecs but all of them are useless. They can not parse multiline logs with coma delimiters. Please advise ho… --- ## [What's the real difference between filter and codec and which I have to use?](https://discuss.elastic.co/t/whats-the-real-difference-between-filter-and-codec-and-which-i-have-to-use/77590)
**Author:** [@Ranieri\_Mazili\_de\_Ol](https://discuss.elastic.co/u/Ranieri_Mazili_de_Ol)\ **Replies:** 5\ **Last updated:** [March 7, 2017, 2:46pm UTC](https://discuss.elastic.co/t/whats-the-real-difference-between-filter-and-codec-and-which-i-have-to-use/77590 "2017-03-07T14:46:56Z")
Hello, I didn't understand very well the difference between filter and codec and I'm in doubt about which one I have to use to solve my problem. My logstash is listening a JMS queue where XML messages are received. T… --- ## [Why my Logstash is consuming more than 80% of CPU?](https://discuss.elastic.co/t/why-my-logstash-is-consuming-more-than-80-of-cpu/77640)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 9\ **Last updated:** [March 7, 2017, 2:19pm UTC](https://discuss.elastic.co/t/why-my-logstash-is-consuming-more-than-80-of-cpu/77640 "2017-03-07T14:19:57Z")
Hi, I use ELK GA 5.0.0. I have 2 Logstash instances in my Linux box. Both of them are designed to consume from Kafka. One logstash consume from a Kafka topic while the other Logstash consume from 3 other Kafka topics. … --- ## [Where the events are processed?](https://discuss.elastic.co/t/where-the-events-are-processed/77673)
**Author:** [@rrs](https://discuss.elastic.co/u/rrs)\ **Replies:** 0\ **Last updated:** [March 7, 2017, 1:53pm UTC](https://discuss.elastic.co/t/where-the-events-are-processed/77673 "2017-03-07T13:53:39Z")
Hi, In Logstash, where the events are processed? in system's main memory or in defined heap memory? Is there any tools to check the memory/events load being processed? We are using CentOS-7. --- ## [Elasticsearch filter Logstash - tag matching queries?](https://discuss.elastic.co/t/elasticsearch-filter-logstash-tag-matching-queries/77670)
**Author:** [@es435](https://discuss.elastic.co/u/es435)\ **Replies:** 0\ **Last updated:** [March 7, 2017, 1:32pm UTC](https://discuss.elastic.co/t/elasticsearch-filter-logstash-tag-matching-queries/77670 "2017-03-07T13:32:23Z")
I'm using the Elasticsearch filter and a percolate query to search documents as I index them with Logstash. My ruby debug output currently lists each log as it is indexed and the ones which have matched a search have an … --- ## [Loaded template is not appearing in Elastic](https://discuss.elastic.co/t/loaded-template-is-not-appearing-in-elastic/77499)
**Author:** [@111148](https://discuss.elastic.co/u/111148)\ **Replies:** 11\ **Last updated:** [March 7, 2017, 1:06pm UTC](https://discuss.elastic.co/t/loaded-template-is-not-appearing-in-elastic/77499 "2017-03-07T13:06:04Z")
Hello, I am currently trying to load existing template from Previous Elastic version 2.\* to the new nodes with version 5.\*. Using this command: C:\\Users\\admin\> Invoke-RestMethod -Uri 'http://localhost:9200/\_template/n… --- ## [Logs created on default folder](https://discuss.elastic.co/t/logs-created-on-default-folder/44610)
**Author:** [@evasu](https://discuss.elastic.co/u/evasu)\ **Replies:** 7\ **Last updated:** [March 7, 2017, 12:50pm UTC](https://discuss.elastic.co/t/logs-created-on-default-folder/44610 "2017-03-07T12:50:07Z")
Hi, I am new to ELK, when I ran my logstash conf, all my logs created on default folder "/var/log/logstash". Is there any why to keeps this log files on customize folder as "/home/folder ". Here is conf ig file input … --- ## [What is the mean of the sqljdbc usage?](https://discuss.elastic.co/t/what-is-the-mean-of-the-sqljdbc-usage/77656)
**Author:** [@LazyerHong](https://discuss.elastic.co/u/LazyerHong)\ **Replies:** 0\ **Last updated:** [March 7, 2017, 12:43pm UTC](https://discuss.elastic.co/t/what-is-the-mean-of-the-sqljdbc-usage/77656 "2017-03-07T12:43:45Z")
When i connect to the sqlserver, i get the error: what is the meaning of language between the "elk\_monitor"???It is so werid. Here is my config: input { jdbc { jdbc\_driver\_library =\> "../ojdbc/sqljdbc4.jar" jdbc\_… --- ## [Logstash/Grok Parsing CSV with columns including Semicolons and escaped doublequotes](https://discuss.elastic.co/t/logstash-grok-parsing-csv-with-columns-including-semicolons-and-escaped-doublequotes/77612)
**Author:** [@ChrisGeorg](https://discuss.elastic.co/u/ChrisGeorg)\ **Replies:** 3\ **Last updated:** [March 7, 2017, 12:08pm UTC](https://discuss.elastic.co/t/logstash-grok-parsing-csv-with-columns-including-semicolons-and-escaped-doublequotes/77612 "2017-03-07T12:08:35Z")
Hi all, I am trying to write a grok parser for a log in CSV format. The different columns are separated by semicolons and each column is included in double quotes.here is an example: "2017-03-07T08:07:39.585Z";"WARN";"… --- ## [Time-difference between multiple occurrences of a custom pattern](https://discuss.elastic.co/t/time-difference-between-multiple-occurrences-of-a-custom-pattern/77642)
**Author:** [@Ruthu\_Ramya](https://discuss.elastic.co/u/Ruthu_Ramya)\ **Replies:** 0\ **Last updated:** [March 7, 2017, 11:14am UTC](https://discuss.elastic.co/t/time-difference-between-multiple-occurrences-of-a-custom-pattern/77642 "2017-03-07T11:14:44Z")
Hi, I have the interesting log-lines as shown below: Nov 30,14:49:56.785 INFO Plane 1A landed Nov 30,14:49:57.099 INFO Plane 2B landed Nov 30,14:49:57.368 INFO Plane 2C landed Nov 30,14:49:57.620 INFO Plane 2D land… --- ## [Remove fields with value: nil](https://discuss.elastic.co/t/remove-fields-with-value-nil/42243)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 8\ **Last updated:** [March 7, 2017, 10:51am UTC](https://discuss.elastic.co/t/remove-fields-with-value-nil/42243 "2017-03-07T10:51:09Z")
Hi, I'm using the csv filter to split my lines up. Working quite well apart from one problem... Some fields are being populated with a value of nil and I'd like to remove those fields:- "SkillGroupSix" =\> nil I have … --- ## [Logstash forwarder settings](https://discuss.elastic.co/t/logstash-forwarder-settings/77478)
**Author:** [@emilio](https://discuss.elastic.co/u/emilio)\ **Replies:** 8\ **Last updated:** [March 7, 2017, 10:27am UTC](https://discuss.elastic.co/t/logstash-forwarder-settings/77478 "2017-03-07T10:27:41Z")
Dear all, I have a lot of application logs on different hosts. I am going transfer all my application logs into time based index. So, i have couple of questions: 1.How and where i can mark transferring logs that … --- ## [Logstash logs configuration](https://discuss.elastic.co/t/logstash-logs-configuration/77614)
**Author:** [@emilio](https://discuss.elastic.co/u/emilio)\ **Replies:** 0\ **Last updated:** [March 7, 2017, 8:38am UTC](https://discuss.elastic.co/t/logstash-logs-configuration/77614 "2017-03-07T08:38:57Z")
Dear all, I need your advice. I would like to place all logstash logs on the same location - /data01/logstash/logs Where i should configure it? I need all levels of logging: info, warn, error. Currently my log… --- ## [Extract data from xml file](https://discuss.elastic.co/t/extract-data-from-xml-file/77534)
**Author:** [@Mohamed](https://discuss.elastic.co/u/Mohamed)\ **Replies:** 1\ **Last updated:** [March 7, 2017, 8:38am UTC](https://discuss.elastic.co/t/extract-data-from-xml-file/77534 "2017-03-07T08:38:42Z")
Hello, I want to dynamically extract information from an xml file, something like this: \ \ \T1\ \ \ \T2\ **Author:** [@Takitani](https://discuss.elastic.co/u/Takitani)\ **Replies:** 1\ **Last updated:** [March 7, 2017, 8:33am UTC](https://discuss.elastic.co/t/can-you-help-me-to-discover-if-this-is-possible-filter-a-mixed-log/77566 "2017-03-07T08:33:40Z") I receive logs from a heroku drain, like this: 295 \<158\>1 2017-03-06T18:54:32.485983+00:00 host heroku router - at=info method=GET path="/admin" host=myhost.com request\_id=0ba313f9-9e88-449e-b4b6-7e34205cf3fe fwd="2a01… --- ## [Http poller logstash plugin - auth problem](https://discuss.elastic.co/t/http-poller-logstash-plugin-auth-problem/77607)
**Author:** [@Jonas](https://discuss.elastic.co/u/Jonas)\ **Replies:** 0\ **Last updated:** [March 7, 2017, 7:46am UTC](https://discuss.elastic.co/t/http-poller-logstash-plugin-auth-problem/77607 "2017-03-07T07:46:23Z")
Hello, I try to use http poller plugin with ssl and auth. I have problem with user authentication on server using http poller. CURL WORKS PERFECT: curl --cacert downloaded\_cert.pem -X POST -u USER:PASSWORD "https:… --- ## [Logstash does nothing for valid json input](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595)
**Author:** [@tushartyagi](https://discuss.elastic.co/u/tushartyagi)\ **Replies:** 8\ **Last updated:** [March 7, 2017, 7:28am UTC](https://discuss.elastic.co/t/logstash-does-nothing-for-valid-json-input/77595 "2017-03-07T07:28:03Z")
Hi, Since yesterday I am trying to work with logstash but I am not able to make much progress. I have some sentiment data in json which I am trying to read in. If there is a syntax error in the file, or if I use "jso… --- ## [Alpine-based Docker-image?](https://discuss.elastic.co/t/alpine-based-docker-image/77311)
**Author:** [@TommySedin](https://discuss.elastic.co/u/TommySedin)\ **Replies:** 1\ **Last updated:** [March 7, 2017, 7:06am UTC](https://discuss.elastic.co/t/alpine-based-docker-image/77311 "2017-03-07T07:06:48Z")
Hi, I noticed that after the switch to the docker.elastic.co/logstash registry, there's no longer any Alpine based images available. Will this be added in the future? Thanks, Tommy Sedin --- ## [Config Logstash for Syslog](https://discuss.elastic.co/t/config-logstash-for-syslog/77325)
**Author:** [@Senninp](https://discuss.elastic.co/u/Senninp)\ **Replies:** 4\ **Last updated:** [March 7, 2017, 6:58am UTC](https://discuss.elastic.co/t/config-logstash-for-syslog/77325 "2017-03-07T06:58:28Z")
Hi everyone, I would like to know here is my mistake. My config is : Kibana 4.1.1 Elasticsearch 1.7.1 Logstash 1.5.4 I want to catch syslog in my elasticsearch. To do this i configure my logstash like : input { … --- ## [How to group space-separated parts of a message into one field in Grok?](https://discuss.elastic.co/t/how-to-group-space-separated-parts-of-a-message-into-one-field-in-grok/77181)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 2\ **Last updated:** [March 7, 2017, 6:24am UTC](https://discuss.elastic.co/t/how-to-group-space-separated-parts-of-a-message-into-one-field-in-grok/77181 "2017-03-07T06:24:58Z")
I have this log message 2017-03-02 15:52:08,455 INFO :RMI TCP Connection(80920)-10.53.123.123 \[c om.company.app\] Hello world I want to group the "RMI TCP onnection(80920)-10.53.123.123" into one field, called my… --- ## [Error when installing plugin](https://discuss.elastic.co/t/error-when-installing-plugin/77407)
**Author:** [@Ranieri\_Mazili\_de\_Ol](https://discuss.elastic.co/u/Ranieri_Mazili_de_Ol)\ **Replies:** 3\ **Last updated:** [March 7, 2017, 2:58am UTC](https://discuss.elastic.co/t/error-when-installing-plugin/77407 "2017-03-07T02:58:48Z")
Hello, I'm trying to install logstash-input-jms plugin but when I run the commando logstash-plugin install --no-verify logstash-input-jms I'm receiving the following message: Installing logstash-input-jms Error Bundl… --- ## [How to set a default start up option?](https://discuss.elastic.co/t/how-to-set-a-default-start-up-option/77557)
**Author:** [@Rich\_Johnson](https://discuss.elastic.co/u/Rich_Johnson)\ **Replies:** 1\ **Last updated:** [March 6, 2017, 7:34pm UTC](https://discuss.elastic.co/t/how-to-set-a-default-start-up-option/77557 "2017-03-06T19:34:27Z")
I am new to Elasticstack, and am currently plowing my way through Logstash configuration according to this guide: https://www.elastic.co/guide/en/logstash/5.2/advanced-pipeline.html On this page, it shows how you can… --- ## [Logstash 5.2.0 - A plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/logstash-5-2-0-a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/77569)
**Author:** [@Rich\_Johnson](https://discuss.elastic.co/u/Rich_Johnson)\ **Replies:** 0\ **Last updated:** [March 6, 2017, 7:24pm UTC](https://discuss.elastic.co/t/logstash-5-2-0-a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/77569 "2017-03-06T19:24:17Z")
my /var/log/logstash/logstash-plain.log is filled with this message: \[2017-03-06T12:20:19,552\]\[ERROR\]\[logstash.pipeline \] A plugin had an unrecoverable error. Will restart this plugin. Plugin: \ **Author:** [@mr\_chang7](https://discuss.elastic.co/u/mr_chang7)\ **Replies:** 0\ **Last updated:** [March 6, 2017, 6:05pm UTC](https://discuss.elastic.co/t/any-performance-breakthrough-plan-with-logstash-aggregate-plugin/77559 "2017-03-06T18:05:21Z") Hi, We're using logstash-aggregate a lot but it really hurts the performance lowered by "5 times slower" because the plugin only works using a single worker. I remember the plugin developer once mentioned that if logst… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1022) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1024)