# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=178 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 179 --- ## [Query OK in JDBC\_Stream But Not In JDBC\_Static?](https://discuss.elastic.co/t/query-ok-in-jdbc-stream-but-not-in-jdbc-static/290362)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 3\ **Last updated:** [November 28, 2021, 5:29pm UTC](https://discuss.elastic.co/t/query-ok-in-jdbc-stream-but-not-in-jdbc-static/290362 "2021-11-28T17:29:40Z")
Hey, I have a query which works fine while using the JDBC steam "statement" but it does not seem to work while using JDBC static. Here's my JDBC\_Stream configuration: filter { jdbc\_streaming { jdbc\_driver\_class =\>… --- ## [Does JDBC Stream Have A Lot Of Cost For The Database?](https://discuss.elastic.co/t/does-jdbc-stream-have-a-lot-of-cost-for-the-database/290361)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 0\ **Last updated:** [November 28, 2021, 12:56pm UTC](https://discuss.elastic.co/t/does-jdbc-stream-have-a-lot-of-cost-for-the-database/290361 "2021-11-28T12:56:17Z")
Hi! I have a query that I cannot convert from JDBC\_stream to JDBC\_static. How much cost does the plugin have for the database? I am parsing logs and I want to know is that okay to use this plugin? Does it have to access … --- ## [Logstash 7.5 GC too frequently to work](https://discuss.elastic.co/t/logstash-7-5-gc-too-frequently-to-work/290331)
**Author:** [@chenchuangc](https://discuss.elastic.co/u/chenchuangc)\ **Replies:** 2\ **Last updated:** [November 28, 2021, 8:44am UTC](https://discuss.elastic.co/t/logstash-7-5-gc-too-frequently-to-work/290331 "2021-11-28T08:44:46Z")
thank you for read my problem , sorry for bother you !!! 1. Env logstash version 7.5.0 java version java -version openjdk version "1.8.0\_312" OpenJDK Runtime Environment (build 1.8.0\_312-b07) OpenJDK 64-Bit Server VM … --- ## [JDBC Limit equivalent?](https://discuss.elastic.co/t/jdbc-limit-equivalent/290346)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 4\ **Last updated:** [November 28, 2021, 6:35am UTC](https://discuss.elastic.co/t/jdbc-limit-equivalent/290346 "2021-11-28T06:35:09Z")
Hi! I was wondering what would be the equivalent of the limit in JDBC static? it does not accept the limit 10 in my query string. --- ## [JDBC - SQL Query Issue](https://discuss.elastic.co/t/jdbc-sql-query-issue/290345)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 0\ **Last updated:** [November 27, 2021, 8:59pm UTC](https://discuss.elastic.co/t/jdbc-sql-query-issue/290345 "2021-11-27T20:59:52Z")
Hi! I have a database in MySQL and I want to fetch some info based on IP addresses I get from my client logs. There are ip\_from(VARCHAR 50), ip\_to(VARCHAR 50), ip\_from\_int(INT 10), and ip\_to\_int(INT 10) plus a name(VARCH… --- ## [Getting a filebeat error when trying to send filebeat logs to elasticsearch](https://discuss.elastic.co/t/getting-a-filebeat-error-when-trying-to-send-filebeat-logs-to-elasticsearch/290319)
**Author:** [@mandude77](https://discuss.elastic.co/u/mandude77)\ **Replies:** 6\ **Last updated:** [November 26, 2021, 10:23pm UTC](https://discuss.elastic.co/t/getting-a-filebeat-error-when-trying-to-send-filebeat-logs-to-elasticsearch/290319 "2021-11-26T22:23:57Z")
Please advise --- ## [Syslog firewall filter](https://discuss.elastic.co/t/syslog-firewall-filter/290220)
**Author:** [@algira37](https://discuss.elastic.co/u/algira37)\ **Replies:** 2\ **Last updated:** [November 26, 2021, 5:04pm UTC](https://discuss.elastic.co/t/syslog-firewall-filter/290220 "2021-11-26T17:04:48Z")
Hello guys, I'm very new in this field, I would like to filter this log from raspberry pi syslog firewall: Nov 25 22:21:22 raspberrypi kernel: \[26172.577441\] DROP UNMATCHED IN-world:IN=eth0 OUT= MAC=01:00:5e:00:00:fb:a… --- ## [Rename recursively field names in nested structure without changing the structure](https://discuss.elastic.co/t/rename-recursively-field-names-in-nested-structure-without-changing-the-structure/290209)
**Author:** [@Isotta\_Blue](https://discuss.elastic.co/u/Isotta_Blue)\ **Replies:** 2\ **Last updated:** [November 26, 2021, 1:03pm UTC](https://discuss.elastic.co/t/rename-recursively-field-names-in-nested-structure-without-changing-the-structure/290209 "2021-11-26T13:03:26Z")
Hello, I have an event which is nested like this: { "test1": null, "test2": null, "test3": { "test31": null, "test32": null, }, "test4":{ "test5": \[ { … --- ## [Logstash complex IF condition](https://discuss.elastic.co/t/logstash-complex-if-condition/290047)
**Author:** [@Jan\_Kabelka](https://discuss.elastic.co/u/Jan_Kabelka)\ **Replies:** 3\ **Last updated:** [November 26, 2021, 4:17pm UTC](https://discuss.elastic.co/t/logstash-complex-if-condition/290047 "2021-11-26T16:17:23Z")
Dear, would you know how to write complex IF condition on Logstash? I would like to add new tag once two fields have different values, except of some combinations of them. What works: if \[event\]\[code\] == "1" and \[proces… --- ## [Auditbeat+logstash+splunk](https://discuss.elastic.co/t/auditbeat-logstash-splunk/290310)
**Author:** [@alejandromariani](https://discuss.elastic.co/u/alejandromariani)\ **Replies:** 0\ **Last updated:** [November 26, 2021, 4:01pm UTC](https://discuss.elastic.co/t/auditbeat-logstash-splunk/290310 "2021-11-26T16:01:53Z")
Good afternoon, I am working on an integration between auditbeat + logstash + splunk I have my auditbeat configuration from one of my nodes, pointing to my logstash cat auditbeat.yml ###################### Auditbeat … --- ## [Extracting a tag from an XML file using RSS input filter](https://discuss.elastic.co/t/extracting-a-tag-from-an-xml-file-using-rss-input-filter/289137)
**Author:** [@Franke2u](https://discuss.elastic.co/u/Franke2u)\ **Replies:** 6\ **Last updated:** [November 26, 2021, 1:30pm UTC](https://discuss.elastic.co/t/extracting-a-tag-from-an-xml-file-using-rss-input-filter/289137 "2021-11-26T13:30:56Z")
On Logstash the RSS input filter is not getting the 'enclosure' or 'subtitle' items below. How do I force those items to be logged? \ Day https://year/day Sat, 13 Nov 2021 Fo… --- ## [When writing to ES and get 400 status code, can you get the original log message?](https://discuss.elastic.co/t/when-writing-to-es-and-get-400-status-code-can-you-get-the-original-log-message/290239)
**Author:** [@zcola](https://discuss.elastic.co/u/zcola)\ **Replies:** 1\ **Last updated:** [November 26, 2021, 10:54am UTC](https://discuss.elastic.co/t/when-writing-to-es-and-get-400-status-code-can-you-get-the-original-log-message/290239 "2021-11-26T10:54:49Z")
{:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"monitor3\_api\_logtail\_write", :\_type=\>"logs", :routing=\>nil}, #\\], :response=\>{"index"=\>{"\_index"=\>"monitor3\_api\_logtail-2021.11.25-00068… --- ## [Logstash file input path options](https://discuss.elastic.co/t/logstash-file-input-path-options/290269)
**Author:** [@mangesh\_shinde](https://discuss.elastic.co/u/mangesh_shinde)\ **Replies:** 0\ **Last updated:** [November 26, 2021, 10:10am UTC](https://discuss.elastic.co/t/logstash-file-input-path-options/290269 "2021-11-26T10:10:07Z")
I have two subfolders under the parent folder naming like platform\* i want to specify path option like that in parent folder whichever subfolder start with platform should be the path for .csv file --- ## [Netflow input and output](https://discuss.elastic.co/t/netflow-input-and-output/289974)
**Author:** [@btjtaylor](https://discuss.elastic.co/u/btjtaylor)\ **Replies:** 2\ **Last updated:** [November 26, 2021, 9:29am UTC](https://discuss.elastic.co/t/netflow-input-and-output/289974 "2021-11-26T09:29:03Z")
Hi all I need to receive netflow data and output it to multiple destinations (just testing with single destination for now) Im running logstash 7.15.2 under windows I made the following logstash config file but it cra… --- ## [Need help to create logstash json.conf file](https://discuss.elastic.co/t/need-help-to-create-logstash-json-conf-file/290225)
**Author:** [@hagaluly](https://discuss.elastic.co/u/hagaluly)\ **Replies:** 3\ **Last updated:** [November 26, 2021, 4:10am UTC](https://discuss.elastic.co/t/need-help-to-create-logstash-json-conf-file/290225 "2021-11-26T04:10:42Z")
i have jenkins job that outputs a json file. i want to know how can i build the logstash conf file in order to export it to Elasticsearch i have tried different configs but always failing on parsing errors. json file … --- ## [How to compare fields of subevents in Logstash](https://discuss.elastic.co/t/how-to-compare-fields-of-subevents-in-logstash/290227)
**Author:** [@Rahul\_Singh1](https://discuss.elastic.co/u/Rahul_Singh1)\ **Replies:** 0\ **Last updated:** [November 26, 2021, 2:39am UTC](https://discuss.elastic.co/t/how-to-compare-fields-of-subevents-in-logstash/290227 "2021-11-26T02:39:35Z")
Hi All, I am new to ELK and i need one help with Logstash. I am getting log from source and I have splitted that into sub events. Events have one field in common and I want to compare current field to previous one. P… --- ## [Logstash-codec-plain is broken dependency in Logstash version 7.15.2](https://discuss.elastic.co/t/logstash-codec-plain-is-broken-dependency-in-logstash-version-7-15-2/290212)
**Author:** [@Saida\_Meftah](https://discuss.elastic.co/u/Saida_Meftah)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 8:23pm UTC](https://discuss.elastic.co/t/logstash-codec-plain-is-broken-dependency-in-logstash-version-7-15-2/290212 "2021-11-25T20:23:08Z")
I am trying to install Logstash-output-loki plugin, the logstash-codec-plain causes a dependency conflict, looks like this plugin is a broken dependency. My Logstash version is 7.15.2, this problem is not happening in L… --- ## [Csv filter : how to deal with missing field?](https://discuss.elastic.co/t/csv-filter-how-to-deal-with-missing-field/290147)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 2\ **Last updated:** [November 25, 2021, 4:09pm UTC](https://discuss.elastic.co/t/csv-filter-how-to-deal-with-missing-field/290147 "2021-11-25T16:09:17Z")
Hello ! I have the following data : \<188\>Nov 25 07:38:31 172.28.192.111 ts-swsan-p11 raslogd: 2021/11/25-07:38:31, \[MAPS-1003\], 28921, WWN 10:00:88:94:71:c4:c0:e0 | FID 128, WARNING, ts-swsan-p11, SW11\_TS\_VPLEX\_P01\_E1… --- ## [Process in stall state while shutdown](https://discuss.elastic.co/t/process-in-stall-state-while-shutdown/290014)
**Author:** [@Vinutha\_A\_H](https://discuss.elastic.co/u/Vinutha_A_H)\ **Replies:** 3\ **Last updated:** [November 25, 2021, 2:03pm UTC](https://discuss.elastic.co/t/process-in-stall-state-while-shutdown/290014 "2021-11-25T14:03:44Z")
We have a script to stop the logstash, which inturn runs 'kill -15 '. This script is being called as part of system init file which in turn stops as part of server reboot. However, recently we see issues in prod while s… --- ## [Replacing \_id field](https://discuss.elastic.co/t/replacing-id-field/290167)
**Author:** [@mangeshs](https://discuss.elastic.co/u/mangeshs)\ **Replies:** 2\ **Last updated:** [November 25, 2021, 1:10pm UTC](https://discuss.elastic.co/t/replacing-id-field/290167 "2021-11-25T13:10:39Z")
there is \_id field auto generated by Elasticsearch which differ the each doc So I have an unique field to set as \_id field I want to that field as \_id for doc in Logstash conf How do to that --- ## [Synchronization of Elasticsearch and relational databases with Logstash and JDBC disconnected EC2](https://discuss.elastic.co/t/synchronization-of-elasticsearch-and-relational-databases-with-logstash-and-jdbc-disconnected-ec2/290140)
**Author:** [@Soongu\_Jeon](https://discuss.elastic.co/u/Soongu_Jeon)\ **Replies:** 2\ **Last updated:** [November 25, 2021, 11:43am UTC](https://discuss.elastic.co/t/synchronization-of-elasticsearch-and-relational-databases-with-logstash-and-jdbc-disconnected-ec2/290140 "2021-11-25T11:43:00Z")
hi. After reading the above site article, I set the schedule for jdbc and mysql every 5 minutes, but the next day I turned on the background and checked, and the server was disconnected. How can I solve this problem? --- ## [Error with Logstash: error=\>"cannot link Java class io.netty.handler.ssl.OpenSsl, probable missing dependency: io.netty.internal.tcnative.SSLContext.setCipherSuite](https://discuss.elastic.co/t/error-with-logstash-error-cannot-link-java-class-io-netty-handler-ssl-openssl-probable-missing-dependency-io-netty-internal-tcnative-sslcontext-setciphersuite/290155)
**Author:** [@sarath.sarepaka](https://discuss.elastic.co/u/sarath.sarepaka)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 10:03am UTC](https://discuss.elastic.co/t/error-with-logstash-error-cannot-link-java-class-io-netty-handler-ssl-openssl-probable-missing-dependency-io-netty-internal-tcnative-sslcontext-setciphersuite/290155 "2021-11-25T10:03:03Z")
We've upgraded our ELK stack from 6.4 version to 6.8.20 version. Elasticsearch and Kibana were upgraded without any issues. But we observed that Logstash service is restarting continuously after the upgradation. Elastic… --- ## [Persistance Queue Logstash](https://discuss.elastic.co/t/persistance-queue-logstash/290146)
**Author:** [@ahmed\_charafouddine](https://discuss.elastic.co/u/ahmed_charafouddine)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 9:07am UTC](https://discuss.elastic.co/t/persistance-queue-logstash/290146 "2021-11-25T09:07:51Z")
Hello, To better understand logstash queue persistence, we would like to know if there would be a ratio of how much (bytes) between theoretical volume (incoming volume) and practical volume (stored volume) in logstash q… --- ## [Logstash to handle multiline input from Filebeat](https://discuss.elastic.co/t/logstash-to-handle-multiline-input-from-filebeat/290116)
**Author:** [@grazia0912](https://discuss.elastic.co/u/grazia0912)\ **Replies:** 4\ **Last updated:** [November 25, 2021, 5:25am UTC](https://discuss.elastic.co/t/logstash-to-handle-multiline-input-from-filebeat/290116 "2021-11-25T05:25:28Z")
Hi, Would just like to ask how can Logstash handle multiline from input Filebeat. My filebeat already handles the multiline however the logstash still outputs only the first line. after the \\n all the rest are ignored … --- ## [Object mapping errors](https://discuss.elastic.co/t/object-mapping-errors/290032)
**Author:** [@Sconic](https://discuss.elastic.co/u/Sconic)\ **Replies:** 6\ **Last updated:** [November 25, 2021, 4:27am UTC](https://discuss.elastic.co/t/object-mapping-errors/290032 "2021-11-25T04:27:44Z")
Hi All, We've run into an issue with Logstash (or could be Filebeat) where it's not sending some log entries to ES. We're currently running version 7.15.2 of all the Elastic Components, although I haven't been able to … --- ## [Near Realtime Threat Intel enrichment using custom external sources stored on enrichment indexes](https://discuss.elastic.co/t/near-realtime-threat-intel-enrichment-using-custom-external-sources-stored-on-enrichment-indexes/289525)
**Author:** [@claudio.rifo](https://discuss.elastic.co/u/claudio.rifo)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 3:05am UTC](https://discuss.elastic.co/t/near-realtime-threat-intel-enrichment-using-custom-external-sources-stored-on-enrichment-indexes/289525 "2021-11-25T03:05:13Z")
Hi There. First Ill try to explain my general idea (that I already have implemented and is working) then the problems I have and a few questions. The general idea is to have a Logstash ingest pipeline, lets call it Pip… --- ## [Unpack a list of hashes into new fields](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107)
**Author:** [@antonisnyc94](https://discuss.elastic.co/u/antonisnyc94)\ **Replies:** 4\ **Last updated:** [November 25, 2021, 2:39am UTC](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107 "2021-11-25T02:39:02Z")
Hello, I am having a field data.aws.httpRequest.headers , and it has a list of hashes as seen below. { "name": "host", "value": "testwebsite.com" }, { "name": "authorization", "value": "token hidden" }, { "na… --- ## [Add multiple host on xpack.monitoring.elasticsearch.hosts with enviroment variable](https://discuss.elastic.co/t/add-multiple-host-on-xpack-monitoring-elasticsearch-hosts-with-enviroment-variable/289514)
**Author:** [@IsraelPerales](https://discuss.elastic.co/u/IsraelPerales)\ **Replies:** 2\ **Last updated:** [November 25, 2021, 1:02am UTC](https://discuss.elastic.co/t/add-multiple-host-on-xpack-monitoring-elasticsearch-hosts-with-enviroment-variable/289514 "2021-11-25T01:02:51Z")
Hi i try monitoring logstash for detect why i have delay on the logs ingest. But when i try put multiple hosts in the property xpack.monitoring.elasticsearch.hosts this fails . The value is injected with the envirome… --- ## [Create index for kube-system and rest of the namespaces another index](https://discuss.elastic.co/t/create-index-for-kube-system-and-rest-of-the-namespaces-another-index/290058)
**Author:** [@Murali\_Thumalapalli](https://discuss.elastic.co/u/Murali_Thumalapalli)\ **Replies:** 0\ **Last updated:** [November 24, 2021, 12:13pm UTC](https://discuss.elastic.co/t/create-index-for-kube-system-and-rest-of-the-namespaces-another-index/290058 "2021-11-24T12:13:02Z")
Hi Team, we are trying to create a new index for Kube-system namespace related logs as one index and rest of the system as another index. as part of that we have below spec read and deployed but we are not seeing any in… --- ## [Logs contains different format of same field name while parsing in logstash](https://discuss.elastic.co/t/logs-contains-different-format-of-same-field-name-while-parsing-in-logstash/290103)
**Author:** [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\ **Replies:** 1\ **Last updated:** [November 24, 2021, 7:00pm UTC](https://discuss.elastic.co/t/logs-contains-different-format-of-same-field-name-while-parsing-in-logstash/290103 "2021-11-24T19:00:16Z")
Hi, I am using logstash to parse logs from txt file.I am having field "mstrGateway" and "MstrGateway" in two different logs, how to parse this field in single "mstrGateway" field. If I make two different fields such as… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=177) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=179)