# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=646 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 647 --- ## [If condition in logstash filter](https://discuss.elastic.co/t/if-condition-in-logstash-filter/157369)
**Author:** [@ashok9177](https://discuss.elastic.co/u/ashok9177)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 1:45pm UTC](https://discuss.elastic.co/t/if-condition-in-logstash-filter/157369 "2018-11-19T13:45:06Z")
I have written if the condition for grok filter filter { if "id4444" in \[fields\]\[component\] { grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} (\\\[%{WORD:loglevel}\\\]) %{DATA} - %{DATA:method} processi… --- ## [Input file (json format) through logstash to elasticsearch](https://discuss.elastic.co/t/input-file-json-format-through-logstash-to-elasticsearch/157109)
**Author:** [@shiv94](https://discuss.elastic.co/u/shiv94)\ **Replies:** 5\ **Last updated:** [November 19, 2018, 7:34pm UTC](https://discuss.elastic.co/t/input-file-json-format-through-logstash-to-elasticsearch/157109 "2018-11-19T19:34:04Z")
Hey, logstash.conf input { file { path =\> "/root/file.json" start\_position =\> "beginning" codec =\> json type =\> "state" } } output { if \[type\] == "state" { elasticsearch { hosts =\> \["localhost:9200"\] index … --- ## [How to access the value in the logstash metadata](https://discuss.elastic.co/t/how-to-access-the-value-in-the-logstash-metadata/157200)
**Author:** [@bijay](https://discuss.elastic.co/u/bijay)\ **Replies:** 14\ **Last updated:** [November 19, 2018, 3:42pm UTC](https://discuss.elastic.co/t/how-to-access-the-value-in-the-logstash-metadata/157200 "2018-11-19T15:42:40Z")
Hi, I want to access the value of the dictionary in the metadata dynamically. How can I do that in logstash filter section? e.g. if \[@metadata\]\[key\] == "test" { \ } Thanks Bijay --- ## [Chaining log events](https://discuss.elastic.co/t/chaining-log-events/156855)
**Author:** [@deivkk](https://discuss.elastic.co/u/deivkk)\ **Replies:** 0\ **Last updated:** [November 15, 2018, 12:36pm UTC](https://discuss.elastic.co/t/chaining-log-events/156855 "2018-11-15T12:36:19Z")
I have a requirement to create a fingerprint per log and chain it with the fingerprint of the previous log entry. So far I have been able to do something closer by using the "aggregate filter" and using a common task for… --- ## [Logstash file input W/ multiline cuts text](https://discuss.elastic.co/t/logstash-file-input-w-multiline-cuts-text/157373)
**Author:** [@urirosenberg](https://discuss.elastic.co/u/urirosenberg)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 2:00pm UTC](https://discuss.elastic.co/t/logstash-file-input-w-multiline-cuts-text/157373 "2018-11-19T14:00:57Z")
Hi, We are using logstash for ingesting files from a file system. The problem we started to observe is that the input/content is being cut (so LS is indexing 2 events for certain files). Here's a sample of our input c… --- ## [Remove fields from nested json](https://discuss.elastic.co/t/remove-fields-from-nested-json/155890)
**Author:** [@Pete\_Barnes](https://discuss.elastic.co/u/Pete_Barnes)\ **Replies:** 2\ **Last updated:** [November 19, 2018, 12:47pm UTC](https://discuss.elastic.co/t/remove-fields-from-nested-json/155890 "2018-11-19T12:47:20Z")
We have a heavily nested json document containing server metrcs, the document contains \> 1000 fields some of which are completely irrelevant to us for analytic purposes so i would like to remove them before indexing the … --- ## [Input-gelf-plugin wrong SOURCE\_HOST\_FIELD](https://discuss.elastic.co/t/input-gelf-plugin-wrong-source-host-field/157344)
**Author:** [@aborche](https://discuss.elastic.co/u/aborche)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 12:16pm UTC](https://discuss.elastic.co/t/input-gelf-plugin-wrong-source-host-field/157344 "2018-11-19T12:16:45Z")
When gelf plugin receives a message, SOURCE\_HOST\_FIELD filled with logstash listen address, instead of connected client address. And all incoming log records contains "0.0.0.0" instead real client ip. Value for SOURCE\_H… --- ## [Multiple pipelines feature not applicable for real world logstash installations](https://discuss.elastic.co/t/multiple-pipelines-feature-not-applicable-for-real-world-logstash-installations/157349)
**Author:** [@stefah](https://discuss.elastic.co/u/stefah)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 12:35pm UTC](https://discuss.elastic.co/t/multiple-pipelines-feature-not-applicable-for-real-world-logstash-installations/157349 "2018-11-19T12:35:17Z")
Hi, while trying to use the multiple pipelines feature with Logstash 6.4.1, we ran into multiple problems. It clearly states in the documentation, that any configuration parameter will disable loading the pipelines.yml… --- ## [Logstash Codec and other](https://discuss.elastic.co/t/logstash-codec-and-other/157298)
**Author:** [@Carlos\_Magalhaes](https://discuss.elastic.co/u/Carlos_Magalhaes)\ **Replies:** 1\ **Last updated:** [November 19, 2018, 12:30pm UTC](https://discuss.elastic.co/t/logstash-codec-and-other/157298 "2018-11-19T12:30:05Z")
Hi there, Just upgraded to 6.5.0 - what was upgraded -\> ELK I was checking the bin/logstash-plugin list and noticed some missing plugins which I use: Sflow Multiline Did these get removed in 6.5.0 or did they get … --- ## [How to change logstash default timezone](https://discuss.elastic.co/t/how-to-change-logstash-default-timezone/157304)
**Author:** [@Manoj\_Hettiarachchi](https://discuss.elastic.co/u/Manoj_Hettiarachchi)\ **Replies:** 3\ **Last updated:** [November 19, 2018, 11:47am UTC](https://discuss.elastic.co/t/how-to-change-logstash-default-timezone/157304 "2018-11-19T11:47:24Z")
Hello, In my system, I use filebeat and logstash. The data written to the log files are sent to logstash via filebeat. Some fields in the logs are filtered and saved in csv format using logstash. Problem: When saving… --- ## [Condition does not work (don't drop the event)](https://discuss.elastic.co/t/condition-does-not-work-dont-drop-the-event/157172)
**Author:** [@Jamson\_Atila](https://discuss.elastic.co/u/Jamson_Atila)\ **Replies:** 1\ **Last updated:** [November 19, 2018, 10:11am UTC](https://discuss.elastic.co/t/condition-does-not-work-dont-drop-the-event/157172 "2018-11-19T10:11:34Z")
I have the following log file Encryption: no Accurate: no Volume name(s): FP-Sem\_Bacula\_0129 Volume Session Id: 2 Volume Session Time: 1542369027 Last Volume Bytes: 3… --- ## [How to perform regression testing in Logstash?](https://discuss.elastic.co/t/how-to-perform-regression-testing-in-logstash/157307)
**Author:** [@shivharsh.singh](https://discuss.elastic.co/u/shivharsh.singh)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 9:09am UTC](https://discuss.elastic.co/t/how-to-perform-regression-testing-in-logstash/157307 "2018-11-19T09:09:55Z")
Hi, I have a requirement to perform regression testing in Logstash using its binary. There is not much of documentation available on web for the same. I have been following the instructions mentioned in README.md. As m… --- ## [Create a JSON Field from the existing string fields](https://discuss.elastic.co/t/create-a-json-field-from-the-existing-string-fields/157300)
**Author:** [@patrick007](https://discuss.elastic.co/u/patrick007)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 8:47am UTC](https://discuss.elastic.co/t/create-a-json-field-from-the-existing-string-fields/157300 "2018-11-19T08:47:06Z")
I want to create a JSON object my existing string fields in Logstash for storing in Elasticsearch Example : "id" : "0001" "name: "John", "designation": "Writer", "age": 25 I would like to create a JSON object calle… --- ## [Logstash add tags](https://discuss.elastic.co/t/logstash-add-tags/157115)
**Author:** [@garcia](https://discuss.elastic.co/u/garcia)\ **Replies:** 1\ **Last updated:** [November 19, 2018, 8:14am UTC](https://discuss.elastic.co/t/logstash-add-tags/157115 "2018-11-19T08:14:39Z")
elk is installed, logs from filebeat come from another server logs in format 2018-11-16 17:26:12 \[http-nio-127.0.0.1-8080-exec-6\] DEBUG c.j.m.helpers.EmailHelper - MAIL WAS SENT, manage post Fri Nov 16 17:26:12 UTC 201… --- ## [How can define multi indices in logstash configuration file based on the type of inputs](https://discuss.elastic.co/t/how-can-define-multi-indices-in-logstash-configuration-file-based-on-the-type-of-inputs/157226)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 7\ **Last updated:** [November 19, 2018, 8:11am UTC](https://discuss.elastic.co/t/how-can-define-multi-indices-in-logstash-configuration-file-based-on-the-type-of-inputs/157226 "2018-11-19T08:11:46Z")
Hi all, I am using logstash to get file from filebeats and ship them to two nodes of elasticsearch. logstash has two inputs with different types. I want to define different indices based on the input type, so I write fo… --- ## [Logstash config question](https://discuss.elastic.co/t/logstash-config-question/157258)
**Author:** [@teichae](https://discuss.elastic.co/u/teichae)\ **Replies:** 6\ **Last updated:** [November 19, 2018, 8:08am UTC](https://discuss.elastic.co/t/logstash-config-question/157258 "2018-11-19T08:08:55Z")
Hi, guys. I have some questions about logstash setup. I want to use logstash to read the logs of our Tomcat through Kibana. However, due to misconfiguration, the multi-line is not being displayed. I need your help on… --- ## [Error while starting logstash windows](https://discuss.elastic.co/t/error-while-starting-logstash-windows/157278)
**Author:** [@nano2903](https://discuss.elastic.co/u/nano2903)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 6:46am UTC](https://discuss.elastic.co/t/error-while-starting-logstash-windows/157278 "2018-11-19T06:46:04Z")
Dear Elastic team, I'm suffering the following issue while starting logstash in windows server 2012 \[2018-11-19T09:58:47,485\]\[DEBUG\]\[logstash.instrument.periodicpoller.cgroup\] One or more required cgroup files or dire… --- ## [Setting up Logstash using Docker](https://discuss.elastic.co/t/setting-up-logstash-using-docker/157228)
**Author:** [@Sai\_Srinivas](https://discuss.elastic.co/u/Sai_Srinivas)\ **Replies:** 1\ **Last updated:** [November 19, 2018, 4:52am UTC](https://discuss.elastic.co/t/setting-up-logstash-using-docker/157228 "2018-11-19T04:52:23Z")
Hi Everyone! This is my first time using Logstash. I am trying to setup logstash on Docker using the following command. docker run -h logstash --name logstash\_a --link elasticsearch\_1:elasticsearch -it --rm -v "SPwD":… --- ## [Elapsed plugin not detecting start tag](https://discuss.elastic.co/t/elapsed-plugin-not-detecting-start-tag/156369)
**Author:** [@chickennuggets](https://discuss.elastic.co/u/chickennuggets)\ **Replies:** 3\ **Last updated:** [November 19, 2018, 4:32am UTC](https://discuss.elastic.co/t/elapsed-plugin-not-detecting-start-tag/156369 "2018-11-19T04:32:54Z")
I've noticed that the elapsed filter has not been detecting a large amount of my start/end tagged events. I tend to see the "elapsed\_end\_without\_start" tag added to my end tagged events, as well as "elapsed\_expired\_error… --- ## [Is it needed to use logstash in cluster?](https://discuss.elastic.co/t/is-it-needed-to-use-logstash-in-cluster/157221)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 0\ **Last updated:** [November 18, 2018, 12:11pm UTC](https://discuss.elastic.co/t/is-it-needed-to-use-logstash-in-cluster/157221 "2018-11-18T12:11:12Z")
Hello, I have a two-node cluster of elasticsearch. I am using filebeat to ship log files into these nodes of elasticsearch and everything works fine. Also, the health of cluster is green. Now, i have a question, is it n… --- ## [Logstash can't connect to Elasticseach](https://discuss.elastic.co/t/logstash-cant-connect-to-elasticseach/157212)
**Author:** [@Sanka\_Ryo](https://discuss.elastic.co/u/Sanka_Ryo)\ **Replies:** 0\ **Last updated:** [November 18, 2018, 8:53am UTC](https://discuss.elastic.co/t/logstash-cant-connect-to-elasticseach/157212 "2018-11-18T08:53:31Z")
Hello Now i had build ELK in centos 7 (6.1.2) Kibana, Elasticsearch and Logstash are running. But when i config logstash in ELK like that: input { snmptrap { community =\> "public" type =\> "snmptrap" host =\> "0.0.… --- ## [Is this json format correct for upload to logstash/elasicsearch](https://discuss.elastic.co/t/is-this-json-format-correct-for-upload-to-logstash-elasicsearch/157194)
**Author:** [@gittinfunky](https://discuss.elastic.co/u/gittinfunky)\ **Replies:** 2\ **Last updated:** [November 18, 2018, 7:40am UTC](https://discuss.elastic.co/t/is-this-json-format-correct-for-upload-to-logstash-elasicsearch/157194 "2018-11-18T07:40:24Z")
I have a series of json files created using tshark (see example below) which I would like to visualise eith ELK. Is the current json format ok for upload to logstash or do I need to parse it in some other format \[{"No."… --- ## [Timestamp parsing in CCM logs](https://discuss.elastic.co/t/timestamp-parsing-in-ccm-logs/154355)
**Author:** [@Maekee](https://discuss.elastic.co/u/Maekee)\ **Replies:** 1\ **Last updated:** [November 18, 2018, 7:33am UTC](https://discuss.elastic.co/t/timestamp-parsing-in-ccm-logs/154355 "2018-11-18T07:33:47Z")
Hello, Trying to parse the date in logstash on logs from ccm logs (System Center Configuration Manager). Example row: \\ **Author:** [@showmeall](https://discuss.elastic.co/u/showmeall)\ **Replies:** 1\ **Last updated:** [November 18, 2018, 4:25am UTC](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin-logstash-input-elasicsearch/157207 "2018-11-18T04:25:32Z") we use logstash 6.5.0 for ES reindex , reindex ES 2.4 TO ES 6.2; but always get a problem, the detail info as fllow: \[ERROR\] 2018-11-18 12:15:21.938 \[\[main\]\ **Author:** [@hvarun](https://discuss.elastic.co/u/hvarun)\ **Replies:** 1\ **Last updated:** [November 17, 2018, 12:54pm UTC](https://discuss.elastic.co/t/handle-null-values-from-inuput/156862 "2018-11-17T12:54:01Z") I am using http as my source and as a first operation in filter, renaming the incoming columns as shown below for a column mutate{ add\_field =\> {"AccID" =\> "%{variable}"} } when i insert this into target, i see co… --- ## [Logstash grok filter with regex](https://discuss.elastic.co/t/logstash-grok-filter-with-regex/157148)
**Author:** [@srkrishna](https://discuss.elastic.co/u/srkrishna)\ **Replies:** 0\ **Last updated:** [November 17, 2018, 4:06am UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-regex/157148 "2018-11-17T04:06:50Z")
Hello, I was trying to create fields in logstash grok filter based on the source file name which contains a regex match for hostname. I have the below log lines ######################################################## … --- ## [Making decisions on the basis of message keys](https://discuss.elastic.co/t/making-decisions-on-the-basis-of-message-keys/156784)
**Author:** [@Faraz\_Mateen](https://discuss.elastic.co/u/Faraz_Mateen)\ **Replies:** 2\ **Last updated:** [November 15, 2018, 10:28am UTC](https://discuss.elastic.co/t/making-decisions-on-the-basis-of-message-keys/156784 "2018-11-15T10:28:59Z")
I am reading messages from a kafka topic and pushing them to logstash. It's pretty straight forward for a simple json but now I want to update my packet structure. Update messages have the following structure: { "in… --- ## [Logstash crashing "can't convert Fixnum into String"](https://discuss.elastic.co/t/logstash-crashing-cant-convert-fixnum-into-string/157129)
**Author:** [@joseph4325](https://discuss.elastic.co/u/joseph4325)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 7:59pm UTC](https://discuss.elastic.co/t/logstash-crashing-cant-convert-fixnum-into-string/157129 "2018-11-16T19:59:47Z")
I put in a filter recently which seemed to be working fine at first, but it would crash logstash every couple of hours with the following message: { "level": "FATAL", "logEvent": { "error": { … --- ## [Logstash translate filter for multiple fields](https://discuss.elastic.co/t/logstash-translate-filter-for-multiple-fields/156812)
**Author:** [@bijay](https://discuss.elastic.co/u/bijay)\ **Replies:** 3\ **Last updated:** [November 16, 2018, 7:52pm UTC](https://discuss.elastic.co/t/logstash-translate-filter-for-multiple-fields/156812 "2018-11-16T19:52:21Z")
Hi, I want to drop the logs based on multiple fields. Below is the content of drop.yml logstash : '{"info": "drop"}' e.g. I want to drop the loglines which has contain the "component" : "logstash" and "severity": "in… --- ## [Regex conditional not working](https://discuss.elastic.co/t/regex-conditional-not-working/157094)
**Author:** [@Miguel\_Leite](https://discuss.elastic.co/u/Miguel_Leite)\ **Replies:** 1\ **Last updated:** [November 16, 2018, 7:30pm UTC](https://discuss.elastic.co/t/regex-conditional-not-working/157094 "2018-11-16T19:30:08Z")
Hey there guys! I have a field with the following value: parsed\_alert\_txt =\> "ORA-06000: at "SYS.DBMS\_QOPATCH", line 777" But this conditional doesn't seem to work, and I don't know why: if \[parsed\_alert\_txt\] =~ /ORA… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=645) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=647)