# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1153 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1154 --- ## [Unable to install the logstash-filter-grok plugin](https://discuss.elastic.co/t/unable-to-install-the-logstash-filter-grok-plugin/50259)
**Author:** [@Hobbit](https://discuss.elastic.co/u/Hobbit)\ **Replies:** 4\ **Last updated:** [May 18, 2016, 5:51am UTC](https://discuss.elastic.co/t/unable-to-install-the-logstash-filter-grok-plugin/50259 "2016-05-18T05:51:13Z")
I followed the installation steps but when I install the plugin it shows this error: ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core-plugin-api": In snapshot (Gem… --- ## [Uploading data from Sql Server](https://discuss.elastic.co/t/uploading-data-from-sql-server/50293)
**Author:** [@Prateek\_Kshtriya](https://discuss.elastic.co/u/Prateek_Kshtriya)\ **Replies:** 1\ **Last updated:** [May 18, 2016, 5:47am UTC](https://discuss.elastic.co/t/uploading-data-from-sql-server/50293 "2016-05-18T05:47:27Z")
Hi All, Please suggest how to index data from SQL server. Regards, Prateek Divya --- ## [\[2.3.1\] Multiline - ConcurrencyError](https://discuss.elastic.co/t/2-3-1-multiline-concurrencyerror/50292)
**Author:** [@Romey-Rome](https://discuss.elastic.co/u/Romey-Rome)\ **Replies:** 0\ **Last updated:** [May 18, 2016, 4:27am UTC](https://discuss.elastic.co/t/2-3-1-multiline-concurrencyerror/50292 "2016-05-18T04:27:52Z")
Get this error, then Logstash stops processing anything comping in on that port. These are incoming Java stack traces. It works for a while, then: {:timestamp=\>"2016-05-17T21:07:36.970000-0700", :message=\>"Exceptio… --- ## [Codec msgpack : Trouble parsing msgpack input](https://discuss.elastic.co/t/codec-msgpack-trouble-parsing-msgpack-input/50284)
**Author:** [@qiangxiao](https://discuss.elastic.co/u/qiangxiao)\ **Replies:** 0\ **Last updated:** [May 18, 2016, 2:42am UTC](https://discuss.elastic.co/t/codec-msgpack-trouble-parsing-msgpack-input/50284 "2016-05-18T02:42:45Z")
i am trying to collector data from kafka by logstash. The data is encoded as msgpack format. however, i use codec msgpack in input-plugin but failed. here is my logstash config: input { kafka { z… --- ## [Kafka input problem](https://discuss.elastic.co/t/kafka-input-problem/50107)
**Author:** [@Vtamins](https://discuss.elastic.co/u/Vtamins)\ **Replies:** 2\ **Last updated:** [May 18, 2016, 2:05am UTC](https://discuss.elastic.co/t/kafka-input-problem/50107 "2016-05-18T02:05:29Z")
I use kafka as input for my logstash But it have a problem. \[2016-05-16 17:56:56,151\] INFO Got user-level KeeperException when processing sessioni d:0x154b859cdaa0018 type:create cxid:0x15 zxid:0x24b txn… --- ## [Bug with logstash-filter-elasticsearch](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161)
**Author:** [@Andrew\_Shved](https://discuss.elastic.co/u/Andrew_Shved)\ **Replies:** 5\ **Last updated:** [May 17, 2016, 9:10pm UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161 "2016-05-17T21:10:28Z")
I have tried example straight from documentation and it seems to fail as elasticsearch expects " around field names. elasticsearch { hosts =\> \["localhost"\] query =\> "statuscode:200" fields =\> \["c… --- ## [Just starting - Advanced Pipeline tutorial issue](https://discuss.elastic.co/t/just-starting-advanced-pipeline-tutorial-issue/50152)
**Author:** [@Jamen\_McGranahan](https://discuss.elastic.co/u/Jamen_McGranahan)\ **Replies:** 4\ **Last updated:** [May 17, 2016, 7:37pm UTC](https://discuss.elastic.co/t/just-starting-advanced-pipeline-tutorial-issue/50152 "2016-05-17T19:37:00Z")
I am using Redhat 5 with an installation of Logstash 2.3.1 (since it appears 2.3.2 does not have the plugins and I could not figure out how to get those installed) and I am trying to work through the Advanced Pipeline tu… --- ## [Possible bug? Using variables for UDP output host causes crash](https://discuss.elastic.co/t/possible-bug-using-variables-for-udp-output-host-causes-crash/50143)
**Author:** [@srfoley](https://discuss.elastic.co/u/srfoley)\ **Replies:** 1\ **Last updated:** [May 17, 2016, 6:54pm UTC](https://discuss.elastic.co/t/possible-bug-using-variables-for-udp-output-host-causes-crash/50143 "2016-05-17T18:54:51Z")
Using ruby to set event fields in filter.conf: ruby { code =\> " case Integer(event\['buildingnumber'\]); when 1..50, 250..300, 500..550, 750..800; event\['lcp1'\] = '(FQDN removed)'; event\['lcp2'\] = '(FQDN re… --- ## [Grok filter if value is void](https://discuss.elastic.co/t/grok-filter-if-value-is-void/50234)
**Author:** [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Replies:** 3\ **Last updated:** [May 17, 2016, 6:02pm UTC](https://discuss.elastic.co/t/grok-filter-if-value-is-void/50234 "2016-05-17T18:02:35Z")
Hello Guys, I have Logstash-2.2.2-1 and I have a filter in grok to add fields, for example: Input: "2016-04-01 00:00:00" "value1" "value2" 100 Grok: %{TIMESTAMP\_ISO8601:EventTime} %{WORD:value1} %{WORD… --- ## [Logstash kv filter include\_keys](https://discuss.elastic.co/t/logstash-kv-filter-include-keys/50103)
**Author:** [@sundar](https://discuss.elastic.co/u/sundar)\ **Replies:** 2\ **Last updated:** [May 17, 2016, 4:04pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-include-keys/50103 "2016-05-17T16:04:27Z")
Hello Logstash experts, Is it ok to have multiple include\_keys in my kv filter config? I want to avoid parsing all KVs since it is adding a lot of junk KV pairs to my list of fields, so I am whitelisting the keys I am in… --- ## [Input Elasticsearch Query in logstash](https://discuss.elastic.co/t/input-elasticsearch-query-in-logstash/50229)
**Author:** [@billy6](https://discuss.elastic.co/u/billy6)\ **Replies:** 1\ **Last updated:** [May 17, 2016, 2:54pm UTC](https://discuss.elastic.co/t/input-elasticsearch-query-in-logstash/50229 "2016-05-17T14:54:47Z")
Hello! I would like to use a query filtering these parameters in logstash's elasticsearch input\> host.raw = host 1 OR host 2 & code != "123" How could I do the query? I have been trying several things for a while w… --- ## [CPU use configuration](https://discuss.elastic.co/t/cpu-use-configuration/50227)
**Author:** [@Maher\_Ben\_Taleb\_Ali](https://discuss.elastic.co/u/Maher_Ben_Taleb_Ali)\ **Replies:** 3\ **Last updated:** [May 17, 2016, 2:50pm UTC](https://discuss.elastic.co/t/cpu-use-configuration/50227 "2016-05-17T14:50:53Z")
Hi, Is there a way to configure the CPU use for logstash, so it can't pass a threshold? what I am seing logstash uses all the available resources. Thank you! --- ## [Parent/Child indexing in config file](https://discuss.elastic.co/t/parent-child-indexing-in-config-file/50218)
**Author:** [@PascalR](https://discuss.elastic.co/u/PascalR)\ **Replies:** 1\ **Last updated:** [May 17, 2016, 2:43pm UTC](https://discuss.elastic.co/t/parent-child-indexing-in-config-file/50218 "2016-05-17T14:43:41Z")
Hi everyone! Is there a way to setup a parent child relation in Logstash? I have a MySQL database as input and I want to use the foreign key of some of my tables as index for a parent\_id. NOTE: Using Logstash 2.3.1… --- ## [Filtering incoming data based on field](https://discuss.elastic.co/t/filtering-incoming-data-based-on-field/50223)
**Author:** [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)\ **Replies:** 1\ **Last updated:** [May 17, 2016, 2:24pm UTC](https://discuss.elastic.co/t/filtering-incoming-data-based-on-field/50223 "2016-05-17T14:24:54Z")
Hi, I implemented an ELK Stack about a week ago and we have received over 16 million entries which a majority is not necessarily useful and we are looking into filtering it out. Our incoming data is syslog and there … --- ## [Logstash consuming huge amount of CPU - mem leak? garbage collector?](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310)
**Author:** [@Eduard\_Camaj](https://discuss.elastic.co/u/Eduard_Camaj)\ **Replies:** 7\ **Last updated:** [May 17, 2016, 9:58am UTC](https://discuss.elastic.co/t/logstash-consuming-huge-amount-of-cpu-mem-leak-garbage-collector/49310 "2016-05-17T09:58:46Z")
Details: dockerized environment, isolated, only ELK logstash version 2.3.1 (Apr 4, 2016) logstash-input-kafka-2.0.6 elasticsearch output 200 records/second 8 cores, 8GB of RAM, 2GB logstash heap (cpu consumation ("ni"… --- ## [Logstash performance](https://discuss.elastic.co/t/logstash-performance/50109)
**Author:** [@Maher\_Ben\_Taleb\_Ali](https://discuss.elastic.co/u/Maher_Ben_Taleb_Ali)\ **Replies:** 1\ **Last updated:** [May 17, 2016, 6:11am UTC](https://discuss.elastic.co/t/logstash-performance/50109 "2016-05-17T06:11:31Z")
We are trying to deploy logstash in production. So I need a detailed answer for these questions: -Logstash extracts data from database using SQL. How exactly logstash get notified about new data? -Is there a way to c… --- ## [How to delete data complete?](https://discuss.elastic.co/t/how-to-delete-data-complete/50123)
**Author:** [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Replies:** 3\ **Last updated:** [May 17, 2016, 5:55am UTC](https://discuss.elastic.co/t/how-to-delete-data-complete/50123 "2016-05-17T05:55:29Z")
hi all, i have using curl -XDELETE to delete indices and delete (logstash-log-\*) index from kibana , but when i create the index (logstash-log-) again , why the old field for the index (logstash-log-) is still exis… --- ## [Trying to add a field with the ip address of the host](https://discuss.elastic.co/t/trying-to-add-a-field-with-the-ip-address-of-the-host/50061)
**Author:** [@Harley\_Burton](https://discuss.elastic.co/u/Harley_Burton)\ **Replies:** 2\ **Last updated:** [May 16, 2016, 4:22pm UTC](https://discuss.elastic.co/t/trying-to-add-a-field-with-the-ip-address-of-the-host/50061 "2016-05-16T16:22:21Z")
I'm sure this is in the docs, and/or here somewhere, but the closest I have been able to find was a thread here from Oct. 2015 with a link to a doc with info that doesn't work on the current version. I want to add a fi… --- ## [Logstash can not consume messages from kafka](https://discuss.elastic.co/t/logstash-can-not-consume-messages-from-kafka/664)
**Author:** [@wivernband](https://discuss.elastic.co/u/wivernband)\ **Replies:** 4\ **Last updated:** [May 16, 2016, 10:03am UTC](https://discuss.elastic.co/t/logstash-can-not-consume-messages-from-kafka/664 "2016-05-16T10:03:44Z")
We configured logstash to use "kafka" as input and "elasticsearch" as output. But it seems that the logstash instance can not consume the messages from kafka. Kindly check the logstash version and its configuration as b… --- ## [Logs are not generated for logstash](https://discuss.elastic.co/t/logs-are-not-generated-for-logstash/49672)
**Author:** [@Roshan\_r](https://discuss.elastic.co/u/Roshan_r)\ **Replies:** 6\ **Last updated:** [May 16, 2016, 8:27am UTC](https://discuss.elastic.co/t/logs-are-not-generated-for-logstash/49672 "2016-05-16T08:27:05Z")
I am using filebeat to ship the logs to another server where my logstash is residing. Starting the logstash shows like this: \[root@astroELK bin\]# ./logstash -f /etc/logstash/conf.d/astro\_pdl.conf --- jar coordinate com… --- ## [Parsing static log file - tracking parsing completion](https://discuss.elastic.co/t/parsing-static-log-file-tracking-parsing-completion/50082)
**Author:** [@Miloni\_Mehta](https://discuss.elastic.co/u/Miloni_Mehta)\ **Replies:** 0\ **Last updated:** [May 15, 2016, 6:05pm UTC](https://discuss.elastic.co/t/parsing-static-log-file-tracking-parsing-completion/50082 "2016-05-15T18:05:40Z")
I have a use case of parsing static log files. I need a way to determine if a file is completely parsed in order to delete it and download next file from the server. I am currently using the sincedb file to track this, … --- ## [Checking if a static file is parsed](https://discuss.elastic.co/t/checking-if-a-static-file-is-parsed/50081)
**Author:** [@Miloni\_Mehta](https://discuss.elastic.co/u/Miloni_Mehta)\ **Replies:** 0\ **Last updated:** [May 15, 2016, 5:57pm UTC](https://discuss.elastic.co/t/checking-if-a-static-file-is-parsed/50081 "2016-05-15T17:57:49Z")
I have a use case of parsing static log files. I need a way to determine if a file is completely parsed in order to delete it and download next file from the server. I am currently using the sincedb file to track this, … --- ## [How to change sender](https://discuss.elastic.co/t/how-to-change-sender/50069)
**Author:** [@netsuro](https://discuss.elastic.co/u/netsuro)\ **Replies:** 3\ **Last updated:** [May 15, 2016, 2:06pm UTC](https://discuss.elastic.co/t/how-to-change-sender/50069 "2016-05-15T14:06:45Z")
Hello, I am new in using logstash. I would just like to change the sender. My conf file is very basic : input { file { path =\> "/srv/log//" start\_position =\> "beginning" } } output { gelf { host =\> \["loca… --- ## [Logstash Crashed:NoMethodError: undefined method \`map\_cleanup' for nil:NilClass start at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-multiline-2.0.11/lib/logstash/codecs/identity\_map\_codec.rb:55](https://discuss.elastic.co/t/logstash-crashed-nomethoderror-undefined-method-map-cleanup-for-nil-nilclass-start-at-opt-logstash-vendor-bundle-jruby-1-9-gems-logstash-codec-multiline-2-0-11-lib-logstash-codecs-identity-map-codec-rb-55/49693)
**Author:** [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\ **Replies:** 1\ **Last updated:** [May 15, 2016, 11:30am UTC](https://discuss.elastic.co/t/logstash-crashed-nomethoderror-undefined-method-map-cleanup-for-nil-nilclass-start-at-opt-logstash-vendor-bundle-jruby-1-9-gems-logstash-codec-multiline-2-0-11-lib-logstash-codecs-identity-map-codec-rb-55/49693 "2016-05-15T11:30:57Z")
Logstash crashed with the following error. NoMethodError: undefined method \`map\_cleanup' for nil:NilClass start at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-multiline-2.0.11/lib/logstash/codecs/identi… --- ## [MIB OID Translation for the same oid?](https://discuss.elastic.co/t/mib-oid-translation-for-the-same-oid/50065)
**Author:** [@mattcrawfoord](https://discuss.elastic.co/u/mattcrawfoord)\ **Replies:** 3\ **Last updated:** [May 15, 2016, 9:37am UTC](https://discuss.elastic.co/t/mib-oid-translation-for-the-same-oid/50065 "2016-05-15T09:37:30Z")
hi all, i have two MIB file : TCP-MIB.yaml and RFC1213-MIB.yam all have defined the oid as follows, tcpConnState: 1.3.6.1.2.1.6.13.1.1 from kibana , RFC1213-MIB::tcpConnState.14.32.100.75.2065.172.18.86.111.23092… --- ## [Logstash does not work since version 2](https://discuss.elastic.co/t/logstash-does-not-work-since-version-2/50063)
**Author:** [@Urun\_Icin](https://discuss.elastic.co/u/Urun_Icin)\ **Replies:** 2\ **Last updated:** [May 15, 2016, 9:17am UTC](https://discuss.elastic.co/t/logstash-does-not-work-since-version-2/50063 "2016-05-15T09:17:33Z")
Hi, I am using version since logstash 1.5 for windows collect logs. Everything was very good but problems occurred after the 2 version I use ELK + Nxlog My config: nxlog: Module im\_msvistalog Module … --- ## [\[RESOLVED\] Double option problem -\> -Djava.io.tmpdir=/var/lib/logstash](https://discuss.elastic.co/t/resolved-double-option-problem-djava-io-tmpdir-var-lib-logstash/50047)
**Author:** [@netsuro](https://discuss.elastic.co/u/netsuro)\ **Replies:** 1\ **Last updated:** [May 15, 2016, 6:39am UTC](https://discuss.elastic.co/t/resolved-double-option-problem-djava-io-tmpdir-var-lib-logstash/50047 "2016-05-15T06:39:15Z")
Hi all, My system : Distributor ID: CentOS Description: CentOS Linux release 7.2.1511 (Core) Release: 7.2.1511 Codename: Core version : logstash-2.3.2-1.noarch I use logstash for send my logs with gelf on a graylo… --- ## [Logstash add\_field](https://discuss.elastic.co/t/logstash-add-field/50036)
**Author:** [@piyush](https://discuss.elastic.co/u/piyush)\ **Replies:** 6\ **Last updated:** [May 14, 2016, 1:53pm UTC](https://discuss.elastic.co/t/logstash-add-field/50036 "2016-05-14T13:53:55Z")
Hi Team, I am trying to add a field but not getting expected result please assist, surely i am overlooking something. It's logstash-2.3.2-1 PFB Details: grok{ match =\> \["message", "%{WORD:appname}"\] } muta… --- ## [Logstash included grok patterns dir?](https://discuss.elastic.co/t/logstash-included-grok-patterns-dir/50024)
**Author:** [@vchav73](https://discuss.elastic.co/u/vchav73)\ **Replies:** 1\ **Last updated:** [May 14, 2016, 8:37am UTC](https://discuss.elastic.co/t/logstash-included-grok-patterns-dir/50024 "2016-05-14T08:37:21Z")
I'm running logstash 2.3 on a CentOS 6 installation. I used the Elasticsearch yum repo to install logstash. My question is where in Linux are the included grok pattern files stored? I looked in /opt/logstash but couldn… --- ## [Quote\_char in csv filter plugin](https://discuss.elastic.co/t/quote-char-in-csv-filter-plugin/50042)
**Author:** [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\ **Replies:** 1\ **Last updated:** [May 14, 2016, 8:21am UTC](https://discuss.elastic.co/t/quote-char-in-csv-filter-plugin/50042 "2016-05-14T08:21:51Z")
"^" as quote\_char not working. in the file fields are enclosed with the "^"; What are the possible values for the quote\_char Thanks in advance. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1152) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1154)