# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=962 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 963 --- ## [If condition matching json with "in"](https://discuss.elastic.co/t/if-condition-matching-json-with-in/89946)
**Author:** [@Enrico\_Maria\_Fusi](https://discuss.elastic.co/u/Enrico_Maria_Fusi)\ **Replies:** 5\ **Last updated:** [June 19, 2017, 9:00pm UTC](https://discuss.elastic.co/t/if-condition-matching-json-with-in/89946 "2017-06-19T21:00:08Z")
Hello all, I have problem with a configuration which is supposed to select only lines containing a specific text inside. The source is a json\_line file. What I want is that only the messages matching a given string are … --- ## [Aggregate logstash filter config does not work](https://discuss.elastic.co/t/aggregate-logstash-filter-config-does-not-work/89976)
**Author:** [@pratheek\_k](https://discuss.elastic.co/u/pratheek_k)\ **Replies:** 0\ **Last updated:** [June 19, 2017, 5:34pm UTC](https://discuss.elastic.co/t/aggregate-logstash-filter-config-does-not-work/89976 "2017-06-19T17:34:03Z")
@magnusbaeck My goal is to combine events based on pId. Events are not being combined. Please help Log looks like this: June 1st 2017, 11:51:26.992 {id} {pId} ClassName:methodName:99 \[DEBUG\] - Received request: June 1… --- ## [Dynamically adding fields with ES field mapping template enabled](https://discuss.elastic.co/t/dynamically-adding-fields-with-es-field-mapping-template-enabled/89634)
**Author:** [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\ **Replies:** 2\ **Last updated:** [June 19, 2017, 4:58pm UTC](https://discuss.elastic.co/t/dynamically-adding-fields-with-es-field-mapping-template-enabled/89634 "2017-06-19T16:58:05Z")
I am unable to decide between dynamic mapping and giving a template before hand. I have configured logstash with file input and ES output. Say, ES output plugin is configured with field mapping template. Currently, I… --- ## [Logstash executable accepts filebeat data, but logstash service not working with same conf files](https://discuss.elastic.co/t/logstash-executable-accepts-filebeat-data-but-logstash-service-not-working-with-same-conf-files/89956)
**Author:** [@Garrett\_Huff](https://discuss.elastic.co/u/Garrett_Huff)\ **Replies:** 0\ **Last updated:** [June 19, 2017, 3:28pm UTC](https://discuss.elastic.co/t/logstash-executable-accepts-filebeat-data-but-logstash-service-not-working-with-same-conf-files/89956 "2017-06-19T15:28:12Z")
Hello everyone, I am running the ELK stack in openstack and am having a bit of trouble getting the daemon to work properly. For some reason, if I run the executable version using the command ./bin/logstash -f config/be… --- ## [Use Solr instead of ELK](https://discuss.elastic.co/t/use-solr-instead-of-elk/89959)
**Author:** [@Leela\_Kodali](https://discuss.elastic.co/u/Leela_Kodali)\ **Replies:** 1\ **Last updated:** [June 19, 2017, 3:40pm UTC](https://discuss.elastic.co/t/use-solr-instead-of-elk/89959 "2017-06-19T15:40:07Z")
Hi, I already configured the process, Filebeat --\>Logstash--\> Elastic Search --\>Kibana, it is working good, but we have the requirement to use Solr instead of Elastic Search, while researching on internet i found about S… --- ## [How to collect windows event logs using Logstash?](https://discuss.elastic.co/t/how-to-collect-windows-event-logs-using-logstash/89948)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 4\ **Last updated:** [June 19, 2017, 3:20pm UTC](https://discuss.elastic.co/t/how-to-collect-windows-event-logs-using-logstash/89948 "2017-06-19T15:20:34Z")
Hi, I am trying to collect Windows event logs through logstash, but it's giving me an error as, 10:19:40.614 \[LogStash::Runner\] ERROR logstash.agent - Cannot load an invalid configuration {:reason=\>"Couldn't find any … --- ## [Logstash parses configs but not as service](https://discuss.elastic.co/t/logstash-parses-configs-but-not-as-service/89440)
**Author:** [@Callahan](https://discuss.elastic.co/u/Callahan)\ **Replies:** 1\ **Last updated:** [June 19, 2017, 2:50pm UTC](https://discuss.elastic.co/t/logstash-parses-configs-but-not-as-service/89440 "2017-06-19T14:50:57Z")
Hi, So this is very odd. I have a small conf file that pulls data from log file and does a little mutating, and key/value pair matching then outputs to elastic. That conf is shown below. input { file { … --- ## [LogStash 2.2 start as a service does not read config file](https://discuss.elastic.co/t/logstash-2-2-start-as-a-service-does-not-read-config-file/43675)
**Author:** [@Jeremy\_Colton](https://discuss.elastic.co/u/Jeremy_Colton)\ **Replies:** 10\ **Last updated:** [June 19, 2017, 2:35pm UTC](https://discuss.elastic.co/t/logstash-2-2-start-as-a-service-does-not-read-config-file/43675 "2017-06-19T14:35:26Z")
Hi, I am running logstash 2.2 on ubuntu 14.0.4. I SSH into my linux box using a cert and sudo to root user. My config file is under /etc/logstash/conf.d/default.conf. a) When I run logstash from the command line: … --- ## [Environment variable of type array isn't accepted in logstash elasticsearch output plugin](https://discuss.elastic.co/t/environment-variable-of-type-array-isnt-accepted-in-logstash-elasticsearch-output-plugin/89772)
**Author:** [@saisn](https://discuss.elastic.co/u/saisn)\ **Replies:** 1\ **Last updated:** [June 19, 2017, 1:41pm UTC](https://discuss.elastic.co/t/environment-variable-of-type-array-isnt-accepted-in-logstash-elasticsearch-output-plugin/89772 "2017-06-19T13:41:08Z")
Hi, I'm trying to output to elasticsearch using logstash and specifying hosts as environment variable is not working. I see there's any issue as per this link https://github.com/elastic/logstash/issues/6366 I'd like … --- ## [Optimizing kafka input plugin](https://discuss.elastic.co/t/optimizing-kafka-input-plugin/89930)
**Author:** [@andreynikk](https://discuss.elastic.co/u/andreynikk)\ **Replies:** 0\ **Last updated:** [June 19, 2017, 12:39pm UTC](https://discuss.elastic.co/t/optimizing-kafka-input-plugin/89930 "2017-06-19T12:39:39Z")
Hi, I'm successfully consume kafka topic and send it to elasticsearch but my problem is the throughput. at first I thought ES is the bottle neck, but after install logstash xpack metrics I saw that the Events Received Ra… --- ## [Message size added as Attribute](https://discuss.elastic.co/t/message-size-added-as-attribute/89922)
**Author:** [@raisinpit](https://discuss.elastic.co/u/raisinpit)\ **Replies:** 2\ **Last updated:** [June 19, 2017, 12:20pm UTC](https://discuss.elastic.co/t/message-size-added-as-attribute/89922 "2017-06-19T12:20:52Z")
Hi, I want to add the size in Bytes of a message to as an attribute of that message, is this possible? I have seen a some try this before with the ruby plugin, but I have been unable to get this to work. Thanks in ad… --- ## [Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down](https://discuss.elastic.co/t/attempted-to-send-a-bulk-request-to-elasticsearch-but-elasticsearch-appears-to-be-unreachable-or-down/89918)
**Author:** [@Zeeshan\_Alam](https://discuss.elastic.co/u/Zeeshan_Alam)\ **Replies:** 0\ **Last updated:** [June 19, 2017, 11:09am UTC](https://discuss.elastic.co/t/attempted-to-send-a-bulk-request-to-elasticsearch-but-elasticsearch-appears-to-be-unreachable-or-down/89918 "2017-06-19T11:09:32Z")
I am using Elasticsearch and Logstash 5.2.0, I am getting these constant error and warnings in my logstash logs even though my elastcisearch is always up. \[2017-06-19T07:00:37,497\]\[ERROR\]\[logstash.outputs.elasticsearch… --- ## [Configure multi line in logstash](https://discuss.elastic.co/t/configure-multi-line-in-logstash/89899)
**Author:** [@Mohamed](https://discuss.elastic.co/u/Mohamed)\ **Replies:** 2\ **Last updated:** [June 19, 2017, 8:49am UTC](https://discuss.elastic.co/t/configure-multi-line-in-logstash/89899 "2017-06-19T08:49:55Z")
Hello, This is my config file: input { file { path =\> "/tmp/CIPAV/result/\*.xml" start\_position =\> "beginning" sincedb\_path =\> "NUL" codec =\> multiline { pattern =\> "^\" negate =\> … --- ## [Parse errors for IIS logs using ELK](https://discuss.elastic.co/t/parse-errors-for-iis-logs-using-elk/89385)
**Author:** [@desai.amogh](https://discuss.elastic.co/u/desai.amogh)\ **Replies:** 4\ **Last updated:** [June 19, 2017, 8:07am UTC](https://discuss.elastic.co/t/parse-errors-for-iis-logs-using-elk/89385 "2017-06-19T08:07:12Z")
My Logs are like below: 2017-06-14T10:39:30.105Z x.x.x.x 2017-06-14 10:39:27,827 Thread:'9' Level:'ERROR' Message:System error: Get security settings list service call failed. \[\[some.app\]\]some.app.ServiceResponseWrappe… --- ## [Logstash Stop Working : Invalid netflow packet format](https://discuss.elastic.co/t/logstash-stop-working-invalid-netflow-packet-format/88598)
**Author:** [@Diyal](https://discuss.elastic.co/u/Diyal)\ **Replies:** 1\ **Last updated:** [June 19, 2017, 7:53am UTC](https://discuss.elastic.co/t/logstash-stop-working-invalid-netflow-packet-format/88598 "2017-06-19T07:53:36Z")
Hi all, I configured Logstash to collect netflow packet. Since few weeks ago, routinely Logstash stop directly when we receive Invalid netflow packet Configurations on device side about sending netflow packet for sp… --- ## [\[SOLVED\] Filebeat to Logstash best practice](https://discuss.elastic.co/t/solved-filebeat-to-logstash-best-practice/89728)
**Author:** [@appno\_matt](https://discuss.elastic.co/u/appno_matt)\ **Replies:** 11\ **Last updated:** [June 19, 2017, 7:34am UTC](https://discuss.elastic.co/t/solved-filebeat-to-logstash-best-practice/89728 "2017-06-19T07:34:12Z")
Hi folks, I'm currently looking over a Filebeat config used to ship Nginx data to Logstash. Filebeat config is: filebeat: prospectors: - paths: - /var/log/syslog - /var/log/auth.log docum… --- ## [Logstash HTTP Output Stuck/Hanging](https://discuss.elastic.co/t/logstash-http-output-stuck-hanging/89525)
**Author:** [@todd-bowles-console](https://discuss.elastic.co/u/todd-bowles-console)\ **Replies:** 2\ **Last updated:** [June 19, 2017, 6:13am UTC](https://discuss.elastic.co/t/logstash-http-output-stuck-hanging/89525 "2017-06-19T06:13:07Z")
I have the weirdest problem. We use Logstash deployed via Octopus on AWS EC2 Instances to process an assortment of local files (mostly IIS logs, some application logs, performance counters, etc). All of these EC2 insta… --- ## [Chinese characters become question marks on Windows](https://discuss.elastic.co/t/chinese-characters-become-question-marks-on-windows/89878)
**Author:** [@Gary\_Liu](https://discuss.elastic.co/u/Gary_Liu)\ **Replies:** 0\ **Last updated:** [June 19, 2017, 4:02am UTC](https://discuss.elastic.co/t/chinese-characters-become-question-marks-on-windows/89878 "2017-06-19T04:02:12Z")
Hi all, To verify traditional Chinese characters processing with Logstash 5.4.1 on Windows, I did three simple tests on Windows 10 and Oracle JDK 1.8.0\_121. Here are my three tests: https://gist.github.com/desp0916/2… --- ## [Got error when run simple config file first time](https://discuss.elastic.co/t/got-error-when-run-simple-config-file-first-time/89795)
**Author:** [@Tals](https://discuss.elastic.co/u/Tals)\ **Replies:** 4\ **Last updated:** [June 19, 2017, 4:44am UTC](https://discuss.elastic.co/t/got-error-when-run-simple-config-file-first-time/89795 "2017-06-19T04:44:58Z")
Just install local host Logstash 5.4.1 I ceated the simple config file at the Logstash folder and run c:\\logstash-5.4.1\\bin\>logstash -f logstash-simple.conf and got this: Using JAVA\_HOME=C:\\ProgramData\\Oracle\\Java\\javapa… --- ## [Why does logstash send RST rather than FIN on client\_inactivity\_timeout](https://discuss.elastic.co/t/why-does-logstash-send-rst-rather-than-fin-on-client-inactivity-timeout/89877)
**Author:** [@jgoff](https://discuss.elastic.co/u/jgoff)\ **Replies:** 0\ **Last updated:** [June 19, 2017, 3:50am UTC](https://discuss.elastic.co/t/why-does-logstash-send-rst-rather-than-fin-on-client-inactivity-timeout/89877 "2017-06-19T03:50:31Z")
hi all Logstash 5.4.1, filebeat 5.4.1, ubuntu, development machines (no other traffic beyond the below) I have seen a number of threads about this log message in filebeat (port 8998 is the logstash port), sometimes c… --- ## [Why does logstash consume CPU time in idle state?](https://discuss.elastic.co/t/why-does-logstash-consume-cpu-time-in-idle-state/89872)
**Author:** [@ka4ok85](https://discuss.elastic.co/u/ka4ok85)\ **Replies:** 2\ **Last updated:** [June 19, 2017, 3:25am UTC](https://discuss.elastic.co/t/why-does-logstash-consume-cpu-time-in-idle-state/89872 "2017-06-19T03:25:43Z")
I push data to Logstash through Filebeat plugin on hourly basis, nothing else is transferred. My understanding is that during these idle periods Logstash should not consume CPU time. But it does although it is only 1%. I… --- ## [Documented Logstash X-pack config breaks Logstash](https://discuss.elastic.co/t/documented-logstash-x-pack-config-breaks-logstash/89539)
**Author:** [@bjorn](https://discuss.elastic.co/u/bjorn)\ **Replies:** 1\ **Last updated:** [June 19, 2017, 3:07am UTC](https://discuss.elastic.co/t/documented-logstash-x-pack-config-breaks-logstash/89539 "2017-06-19T03:07:54Z")
The official Elasticsearch output plugin documentation (https://www.elastic.co/guide/en/x-pack/current/logstash.html) says that username and password should be added to each of input {}, filter {}, and the Elasticsearch … --- ## [Help with JSON output format](https://discuss.elastic.co/t/help-with-json-output-format/89839)
**Author:** [@aganatra13](https://discuss.elastic.co/u/aganatra13)\ **Replies:** 3\ **Last updated:** [June 18, 2017, 11:59pm UTC](https://discuss.elastic.co/t/help-with-json-output-format/89839 "2017-06-18T23:59:14Z")
Can someone help me with writing correct configuration file such that JSON output 1) Does not contain metadata like source file name, time stamp, host name etc.. 2) Each column in input does not become a JSON document … --- ## [Configure an index pattern on CentOS 7](https://discuss.elastic.co/t/configure-an-index-pattern-on-centos-7/89844)
**Author:** [@hack3rcon](https://discuss.elastic.co/u/hack3rcon)\ **Replies:** 2\ **Last updated:** [June 18, 2017, 12:51pm UTC](https://discuss.elastic.co/t/configure-an-index-pattern-on-centos-7/89844 "2017-06-18T12:51:53Z")
Hello. I installed "Elasticsearch", "Logstash" and "Kibana" on CentOS 7 x64. I want forwarding Windows OS log to my CentOS server and my configurations are: $ cat /etc/syslog-ng/syslog-ng.conf options { flush\_… --- ## [Throttle filter + sleep command to create upload rate](https://discuss.elastic.co/t/throttle-filter-sleep-command-to-create-upload-rate/89825)
**Author:** [@Amir\_Gilad](https://discuss.elastic.co/u/Amir_Gilad)\ **Replies:** 0\ **Last updated:** [June 18, 2017, 6:02am UTC](https://discuss.elastic.co/t/throttle-filter-sleep-command-to-create-upload-rate/89825 "2017-06-18T06:02:52Z")
Hi, I'm trying to creating "Rate-limit" filter and I've seen some advises regarding using throttle+sleep to achieve this goal. However, I failed todo it myself... Can anyone provide an example for (just for the sake of… --- ## [Jsons and logstash](https://discuss.elastic.co/t/jsons-and-logstash/88222)
**Author:** [@coolshiva123](https://discuss.elastic.co/u/coolshiva123)\ **Replies:** 4\ **Last updated:** [June 17, 2017, 7:15am UTC](https://discuss.elastic.co/t/jsons-and-logstash/88222 "2017-06-17T07:15:53Z")
Hi Team, I am having a json like this \[ { item1:value item2:value item3:\[value a,value b\] } \] I am having filebeat sending the logs to logstash (and this is a multiline json having an array as well in key v… --- ## [Bug when attempting to use logstash-output-s3 with encryption](https://discuss.elastic.co/t/bug-when-attempting-to-use-logstash-output-s3-with-encryption/89775)
**Author:** [@Josh\_Harrison](https://discuss.elastic.co/u/Josh_Harrison)\ **Replies:** 1\ **Last updated:** [June 16, 2017, 10:04pm UTC](https://discuss.elastic.co/t/bug-when-attempting-to-use-logstash-output-s3-with-encryption/89775 "2017-06-16T22:04:54Z")
I have an s3 bucket where I have policy set like { "Version": "2012-10-17", "Id": "PutObjPolicy", "Statement": \[ { "Sid": "DenyIncorrectEncryptionHeader", "Effect": "Deny", … --- ## [Unable to load nested datatype using Logstash](https://discuss.elastic.co/t/unable-to-load-nested-datatype-using-logstash/89769)
**Author:** [@Rory](https://discuss.elastic.co/u/Rory)\ **Replies:** 0\ **Last updated:** [June 16, 2017, 7:45pm UTC](https://discuss.elastic.co/t/unable-to-load-nested-datatype-using-logstash/89769 "2017-06-16T19:45:48Z")
I am trying to load data from SQL Server to ES. I have stored in a column in SQL, data in the following format: \[{"addressid":1,"addressvalue":"123 abc st"},{"addressid":2,"addressvalue":"456 pqr st"}\] The name of th… --- ## [Logstash dies because of to high load](https://discuss.elastic.co/t/logstash-dies-because-of-to-high-load/89019)
**Author:** [@disappear89](https://discuss.elastic.co/u/disappear89)\ **Replies:** 3\ **Last updated:** [June 16, 2017, 7:25pm UTC](https://discuss.elastic.co/t/logstash-dies-because-of-to-high-load/89019 "2017-06-16T19:25:16Z")
Hi, we have the following setup: Many Clients with collectors sidecar and filebeat installed, sending their logs to logstash Some Clients that send their openshift logs via fluentd to logstash Two Graylog / Logstash… --- ## [Offline Install Logstash Translate Plugin 5.4](https://discuss.elastic.co/t/offline-install-logstash-translate-plugin-5-4/89711)
**Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\ **Replies:** 2\ **Last updated:** [June 16, 2017, 7:04pm UTC](https://discuss.elastic.co/t/offline-install-logstash-translate-plugin-5-4/89711 "2017-06-16T19:04:16Z")
Where can logstash translate plugin 5.4 be located for download and offline install. I use to download them from ruby gems but is that still the location to get offline installs of community plugins? Thanks [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=961) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=963)