# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=293 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 294 --- ## [Help with Logstash Config - Conditionals in filters](https://discuss.elastic.co/t/help-with-logstash-config-conditionals-in-filters/251623)
**Author:** [@Zorkmid](https://discuss.elastic.co/u/Zorkmid)\ **Replies:** 3\ **Last updated:** [October 13, 2020, 5:57pm UTC](https://discuss.elastic.co/t/help-with-logstash-config-conditionals-in-filters/251623 "2020-10-13T17:57:09Z")
I'm attempting to parse Versa Analytics Events. The log stream from the SD-Wan devices will have several event types in the over all stream. E.g. URL event logs, FW event logs, IDS event logs etc. My config is as follow… --- ## [Logstash-input-salesforce plugin logstash error](https://discuss.elastic.co/t/logstash-input-salesforce-plugin-logstash-error/251811)
**Author:** [@cmay](https://discuss.elastic.co/u/cmay)\ **Replies:** 5\ **Last updated:** [October 13, 2020, 4:55pm UTC](https://discuss.elastic.co/t/logstash-input-salesforce-plugin-logstash-error/251811 "2020-10-13T16:55:49Z")
I've tried reinstalling logstash and the logstash-input-salesforce plugin after changing from Java 11 to 1.8.0. I restart the service. I never see it in Kibana. ERROR RECEIVED: \[2020-10-12T16:05:14,151\]\[ERROR\]\[logstash… --- ## [Logstash - Elastic App Search Error](https://discuss.elastic.co/t/logstash-elastic-app-search-error/251937)
**Author:** [@Tam2](https://discuss.elastic.co/u/Tam2)\ **Replies:** 2\ **Last updated:** [October 13, 2020, 4:46pm UTC](https://discuss.elastic.co/t/logstash-elastic-app-search-error/251937 "2020-10-13T16:46:06Z")
Running logstash with the elastic\_app\_search output throws an error My config file looks like this: input { jdbc { jdbc\_connection\_string =\> "jdbc:sqlserver://XXXXX:1433;databaseName=XXXX;user=XXXX;passwo… --- ## [Custom Logstash snmp plugin pipeline approach](https://discuss.elastic.co/t/custom-logstash-snmp-plugin-pipeline-approach/251901)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 1\ **Last updated:** [October 13, 2020, 4:23pm UTC](https://discuss.elastic.co/t/custom-logstash-snmp-plugin-pipeline-approach/251901 "2020-10-13T16:23:52Z")
Hello, I need to add data get via logstash SNMP plugin to elastisearch. I wanted to ask things I am not sure about how to approach. 1.) How Can I define static fields like interface name description, hostname, etc, so… --- ## [Tagging a document](https://discuss.elastic.co/t/tagging-a-document/251899)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 2\ **Last updated:** [October 13, 2020, 4:13pm UTC](https://discuss.elastic.co/t/tagging-a-document/251899 "2020-10-13T16:13:10Z")
With the logic below, I am getting a field in the document, a\_device\_customer\_code = 'dupont', but the document is tagged with tags = '%{cust\_code\]'. Is there something I am over looking? |t a\_device\_customer\_code |||--… --- ## [Logstash FileOutput Custom Name](https://discuss.elastic.co/t/logstash-fileoutput-custom-name/251838)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 1\ **Last updated:** [October 13, 2020, 3:58pm UTC](https://discuss.elastic.co/t/logstash-fileoutput-custom-name/251838 "2020-10-13T15:58:41Z")
I have a requirement where the file name (in the output) in a specific way. After 11.30pm, It need to take next day date. For example, today is 13th Oct 2020, the file name is ouput\_13102020.log but then once it passes… --- ## [Unable to push data from mongodb to elastic search using logstash in windows](https://discuss.elastic.co/t/unable-to-push-data-from-mongodb-to-elastic-search-using-logstash-in-windows/251722)
**Author:** [@Durga\_AK](https://discuss.elastic.co/u/Durga_AK)\ **Replies:** 3\ **Last updated:** [October 13, 2020, 3:56pm UTC](https://discuss.elastic.co/t/unable-to-push-data-from-mongodb-to-elastic-search-using-logstash-in-windows/251722 "2020-10-13T15:56:43Z")
Im using the below config file to push data from mongodb to elastic search input{ mongodb{ uri=\>\[""\] placeholder\_db\_dir=\>"C:\\Windows\\System32" placeholder\_db\_name=\>"logstash\_sqlite.db" collection=\>"menus" batch\_si… --- ## [Logstash decprecated?](https://discuss.elastic.co/t/logstash-decprecated/251929)
**Author:** [@opouwels](https://discuss.elastic.co/u/opouwels)\ **Replies:** 1\ **Last updated:** [October 13, 2020, 3:47pm UTC](https://discuss.elastic.co/t/logstash-decprecated/251929 "2020-10-13T15:47:00Z")
Hi Application developer wants to use https://quarkus.io/guides/centralized-log-management to send his multiline java logs to our elastic server, as quarkus defines to use logstash which we don't use at the moment I ne… --- ## [ELK stack Installation issue](https://discuss.elastic.co/t/elk-stack-installation-issue/251596)
**Author:** [@Akhil2](https://discuss.elastic.co/u/Akhil2)\ **Replies:** 2\ **Last updated:** [October 13, 2020, 2:41pm UTC](https://discuss.elastic.co/t/elk-stack-installation-issue/251596 "2020-10-13T14:41:02Z")
Hello there, I am trying to install version 7.8.1 for Elasticsearch, Kibana and Logstash but when I try to ingest data from Logstash it gives me an error. Please look at the attached screenshot of my CMD screen for … --- ## [When using KV in logstash, I am getting my key with escaped character, how do i read this value in the Ruby?](https://discuss.elastic.co/t/when-using-kv-in-logstash-i-am-getting-my-key-with-escaped-character-how-do-i-read-this-value-in-the-ruby/251845)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 1\ **Last updated:** [October 13, 2020, 12:12pm UTC](https://discuss.elastic.co/t/when-using-kv-in-logstash-i-am-getting-my-key-with-escaped-character-how-do-i-read-this-value-in-the-ruby/251845 "2020-10-13T12:12:58Z")
I have data coming from Kafka, and the data will look like this: "x-vcap-request-id":"smitshah", "x-cf-applicationid":"test", "x-cf-instanceid":"test2", "sec-ch-ua":"\\"something\\";\\"willwork\\"" It is simili… --- ## [Retryer: send unwait signal to consumer. Logstash overload?](https://discuss.elastic.co/t/retryer-send-unwait-signal-to-consumer-logstash-overload/250366)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 4\ **Last updated:** [October 13, 2020, 11:00am UTC](https://discuss.elastic.co/t/retryer-send-unwait-signal-to-consumer-logstash-overload/250366 "2020-10-13T11:00:04Z")
Hi everybody, I have a 3 cluster node used to collect logs from several services (received using Filebeat clients). On those nodes, I have installed also Logstash (for each node) with the same configuration and pipelin… --- ## [Error In Logstash JSON Parsing](https://discuss.elastic.co/t/error-in-logstash-json-parsing/251880)
**Author:** [@rkAyachitula](https://discuss.elastic.co/u/rkAyachitula)\ **Replies:** 0\ **Last updated:** [October 13, 2020, 9:56am UTC](https://discuss.elastic.co/t/error-in-logstash-json-parsing/251880 "2020-10-13T09:56:34Z")
Hi All, I have noticed a log message which is being pumped to elastic search through Logstash JSON filter combined two different messages to a single message. Both of these messages came from different filebeats. What m… --- ## [Trigger event when file is closed in file-output plugin](https://discuss.elastic.co/t/trigger-event-when-file-is-closed-in-file-output-plugin/251870)
**Author:** [@yogevyuval](https://discuss.elastic.co/u/yogevyuval)\ **Replies:** 0\ **Last updated:** [October 13, 2020, 9:06am UTC](https://discuss.elastic.co/t/trigger-event-when-file-is-closed-in-file-output-plugin/251870 "2020-10-13T09:06:51Z")
Hi, Is it possible to get some kind of an event/trigger/code execution every time the file-output plugin closes a file? The use case is that I want some processing to happen after every file is created, and I was wonde… --- ## [Logstash ruby time difference appends date\_time](https://discuss.elastic.co/t/logstash-ruby-time-difference-appends-date-time/251857)
**Author:** [@sai\_kiran1](https://discuss.elastic.co/u/sai_kiran1)\ **Replies:** 0\ **Last updated:** [October 13, 2020, 7:31am UTC](https://discuss.elastic.co/t/logstash-ruby-time-difference-appends-date-time/251857 "2020-10-13T07:31:01Z")
Hello, Am trying to get the difference between two time fields in my logstash ruby filter and when the difference in ingested to elasticsearch I see date\_time appended to the value of the difference. Any possible way we… --- ## [Logstash Start and shutdown intermittently](https://discuss.elastic.co/t/logstash-start-and-shutdown-intermittently/251836)
**Author:** [@Amit\_K\_Yadav](https://discuss.elastic.co/u/Amit_K_Yadav)\ **Replies:** 1\ **Last updated:** [October 13, 2020, 4:29am UTC](https://discuss.elastic.co/t/logstash-start-and-shutdown-intermittently/251836 "2020-10-13T04:29:30Z")
I am new to ELK world. I installed ELK, each on different server. My Elastic Stack was working fine before x-pack enabled. After enabling x-pack, Elasticsearch and kibana working fine. Logstash start and shutdown repeat… --- ## [Implementing Update by Query in Logstash](https://discuss.elastic.co/t/implementing-update-by-query-in-logstash/251813)
**Author:** [@NomadicCodeGuy](https://discuss.elastic.co/u/NomadicCodeGuy)\ **Replies:** 0\ **Last updated:** [October 12, 2020, 8:47pm UTC](https://discuss.elastic.co/t/implementing-update-by-query-in-logstash/251813 "2020-10-12T20:47:32Z")
I am attempting to use Update by Query with logstash to copy a field \[scenario\] from entries that share the same field \[file\]. I am implementing the solution described by badger in this thread: https://discuss.elastic.co… --- ## [Need help to mask XML elements](https://discuss.elastic.co/t/need-help-to-mask-xml-elements/251354)
**Author:** [@tgkumarmca](https://discuss.elastic.co/u/tgkumarmca)\ **Replies:** 4\ **Last updated:** [October 12, 2020, 8:28pm UTC](https://discuss.elastic.co/t/need-help-to-mask-xml-elements/251354 "2020-10-12T20:28:56Z")
Hello, I'm actually stuck to perform the masking! I would request you to help in this regard I need to perform masking of some sensitive data(like card number, account number, etc) from the input XML message. I am cur… --- ## [Logstash output custom template](https://discuss.elastic.co/t/logstash-output-custom-template/251664)
**Author:** [@venku](https://discuss.elastic.co/u/venku)\ **Replies:** 9\ **Last updated:** [October 12, 2020, 7:20pm UTC](https://discuss.elastic.co/t/logstash-output-custom-template/251664 "2020-10-12T19:20:10Z")
Hi, I am new to ELK stack. This is my set so far for sending application logs : filebeat -\> redis(dockers) -\> logstash(dockers) -\> ES. My Question is what is the the correct way to setup custom index in logstash output? … --- ## [Trying to secure Logstash](https://discuss.elastic.co/t/trying-to-secure-logstash/251602)
**Author:** [@thatch56](https://discuss.elastic.co/u/thatch56)\ **Replies:** 7\ **Last updated:** [October 12, 2020, 5:27pm UTC](https://discuss.elastic.co/t/trying-to-secure-logstash/251602 "2020-10-12T17:27:16Z")
So im trying to secure the communication to logstash in my elk stack. I have a certificate that is working for kibana so i was going to test and see if it would work with logstash. Kibana and Logstash are on the same ser… --- ## [Parsing MongoDB with Logstash](https://discuss.elastic.co/t/parsing-mongodb-with-logstash/251780)
**Author:** [@Hawasli](https://discuss.elastic.co/u/Hawasli)\ **Replies:** 0\ **Last updated:** [October 12, 2020, 2:17pm UTC](https://discuss.elastic.co/t/parsing-mongodb-with-logstash/251780 "2020-10-12T14:17:25Z")
Hi, I am running Logstash 6.8 and trying to parse MongoDB logs which is well- specified here. Is there a more elegant/efficient way to parse MongoDB logs? Here is my filter filter { if \[fields\]\[application… --- ## [Logstash Apache and Spring log files issue](https://discuss.elastic.co/t/logstash-apache-and-spring-log-files-issue/251717)
**Author:** [@Ashish\_Jindal](https://discuss.elastic.co/u/Ashish_Jindal)\ **Replies:** 4\ **Last updated:** [October 12, 2020, 2:09pm UTC](https://discuss.elastic.co/t/logstash-apache-and-spring-log-files-issue/251717 "2020-10-12T14:09:25Z")
Our task was Read both Apache and Springboot log file and add a tag name as given below, to differentiate the logs. Apache logfile -\> apacheLog spring-boot logfile -\> javaLog Print the standard output and write it… --- ## [Multiline codec along with other type of codec](https://discuss.elastic.co/t/multiline-codec-along-with-other-type-of-codec/251706)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 1\ **Last updated:** [October 12, 2020, 1:54pm UTC](https://discuss.elastic.co/t/multiline-codec-along-with-other-type-of-codec/251706 "2020-10-12T13:54:35Z")
Hi all, I have a questions regarding multiline codec in logstash. i tried to parsed multiline xml structure in windows event log, but not only xml log, there are other log as well, so i wanted to ask if when i put the … --- ## [Logstash mssql JDBC connector](https://discuss.elastic.co/t/logstash-mssql-jdbc-connector/251769)
**Author:** [@thomas7467](https://discuss.elastic.co/u/thomas7467)\ **Replies:** 0\ **Last updated:** [October 12, 2020, 1:29pm UTC](https://discuss.elastic.co/t/logstash-mssql-jdbc-connector/251769 "2020-10-12T13:29:40Z")
Hi, I'm trying to ingest data from MSSQL to logstash through the JDBC Driver. Running SQL queries from logstash to MS SQL works fine with SQL account. Now I need to connect through AD account but it is not working, an… --- ## [Splitting fields into new documents](https://discuss.elastic.co/t/splitting-fields-into-new-documents/251459)
**Author:** [@dudek1337](https://discuss.elastic.co/u/dudek1337)\ **Replies:** 2\ **Last updated:** [October 12, 2020, 11:30am UTC](https://discuss.elastic.co/t/splitting-fields-into-new-documents/251459 "2020-10-12T11:30:54Z")
Hello I have document like below {"fsFreeSize.1":7801,"fsUsedRatio.3":23,"fsDevice.4":"dfbw\_var","fsIndex.1":1,"fsUsedSize.3":130,"fsUsedRatio.5":1,"host":"hostname","fsDevice.5":"dfbw\_dev#shm","fsIndex.4":4,"fsIndex.3… --- ## [Data by mqtt to elasticsearch using logstash](https://discuss.elastic.co/t/data-by-mqtt-to-elasticsearch-using-logstash/251724)
**Author:** [@sai92](https://discuss.elastic.co/u/sai92)\ **Replies:** 2\ **Last updated:** [October 12, 2020, 10:05am UTC](https://discuss.elastic.co/t/data-by-mqtt-to-elasticsearch-using-logstash/251724 "2020-10-12T10:05:07Z")
Hello I am using elasticsearch for IoT and now i want to put data from iot sensor to elasticsearch using mqtt how can i do this can anyone please share some tutorials or guide which would help me. thank you in advandce… --- ## [Processing a json file in filebeat](https://discuss.elastic.co/t/processing-a-json-file-in-filebeat/251438)
**Author:** [@humartinez](https://discuss.elastic.co/u/humartinez)\ **Replies:** 2\ **Last updated:** [October 12, 2020, 8:30am UTC](https://discuss.elastic.co/t/processing-a-json-file-in-filebeat/251438 "2020-10-12T08:30:29Z")
Hi there!, I got a filebeat config (see further below) that is currently working, and Its supposed to read a log file written in JSON and then send it, in this case to a kafka topic. The the log message is stored in un… --- ## [Splitting JSON extracted fields for new-line behaves incorrectly vs JSON original message](https://discuss.elastic.co/t/splitting-json-extracted-fields-for-new-line-behaves-incorrectly-vs-json-original-message/251688)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 2\ **Last updated:** [October 12, 2020, 7:46am UTC](https://discuss.elastic.co/t/splitting-json-extracted-fields-for-new-line-behaves-incorrectly-vs-json-original-message/251688 "2020-10-12T07:46:15Z")
I've an example dataset as below { "org": "COMPANY11", "department": { "name": "Human Resources", "id": "HR" }, "http\_request": "Host: www.something.co.uk\\r\\nConnection: keep-alive\\r\\nUser-Agent: Mozilla… --- ## [Setup logstash as system daemon](https://discuss.elastic.co/t/setup-logstash-as-system-daemon/251401)
**Author:** [@Somesh\_ng](https://discuss.elastic.co/u/Somesh_ng)\ **Replies:** 4\ **Last updated:** [October 12, 2020, 6:47am UTC](https://discuss.elastic.co/t/setup-logstash-as-system-daemon/251401 "2020-10-12T06:47:31Z")
Can someone help me in setting up logstash as a system daemon. I have download logstash-7.9.1.tar.gz --- ## [Logstash adding tag issue](https://discuss.elastic.co/t/logstash-adding-tag-issue/250787)
**Author:** [@Ashish\_Jindal](https://discuss.elastic.co/u/Ashish_Jindal)\ **Replies:** 7\ **Last updated:** [October 12, 2020, 4:35am UTC](https://discuss.elastic.co/t/logstash-adding-tag-issue/250787 "2020-10-12T04:35:33Z")
We had given a task as Try adding tag A if the data read is a. a b a c d Ensure that the input data is tagged with type as a test, and write the output to the file output.txt in the path usr/share/logstash . For whi… --- ## [Getting sum of a field in xml](https://discuss.elastic.co/t/getting-sum-of-a-field-in-xml/251657)
**Author:** [@Ameeruddin\_Mohammed](https://discuss.elastic.co/u/Ameeruddin_Mohammed)\ **Replies:** 3\ **Last updated:** [October 11, 2020, 9:19pm UTC](https://discuss.elastic.co/t/getting-sum-of-a-field-in-xml/251657 "2020-10-11T21:19:58Z")
hi, i am new to elk. with a lot of googling and checking responses of @Badger i was able to come up with a working config to load data as i wanted. sample loaded data { "@timestamp" =\> xxx, "host" =\> "xxx", "data" =\>… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=292) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=294)