# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=69 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 70 --- ## [Logstash input S3 module problem](https://discuss.elastic.co/t/logstash-input-s3-module-problem/334662)
**Author:** [@San9](https://discuss.elastic.co/u/San9)\ **Replies:** 7\ **Last updated:** [June 29, 2023, 3:12pm UTC](https://discuss.elastic.co/t/logstash-input-s3-module-problem/334662 "2023-06-29T15:12:08Z")
The problem is in the operation of the S3 module, the module starts for some time, everything works, but after a couple of hours the module is left with an error: Error: Too many open files - Too many open files May 25 … --- ## [Kafka to Logstash](https://discuss.elastic.co/t/kafka-to-logstash/337173)
**Author:** [@Shalinicts](https://discuss.elastic.co/u/Shalinicts)\ **Replies:** 4\ **Last updated:** [June 29, 2023, 12:58pm UTC](https://discuss.elastic.co/t/kafka-to-logstash/337173 "2023-06-29T12:58:33Z")
Hi Team , In Logstash I can see below error for pipeline having input as Kafka (Oracle Cloud) It fails to connect and then discover . Any idea on what needs to be checked. Once it is stable automatically, we gets data… --- ## [Logstash 8 cannot run with JRE](https://discuss.elastic.co/t/logstash-8-cannot-run-with-jre/337171)
**Author:** [@sxwnhxwx](https://discuss.elastic.co/u/sxwnhxwx)\ **Replies:** 1\ **Last updated:** [June 29, 2023, 12:22pm UTC](https://discuss.elastic.co/t/logstash-8-cannot-run-with-jre/337171 "2023-06-29T12:22:36Z")
When I start logstash 8 with jre, it is failed , jdk is ok Logstash 7 works fine with jre logstash version：8.8.1 jre version: 11.0.18 testing configuration： input{ stdin{} } output{ stdout{} } The error message is… --- ## [Mssql server connectivity issue with logstash](https://discuss.elastic.co/t/mssql-server-connectivity-issue-with-logstash/337165)
**Author:** [@Nawab\_Zaidi](https://discuss.elastic.co/u/Nawab_Zaidi)\ **Replies:** 0\ **Last updated:** [June 29, 2023, 7:56am UTC](https://discuss.elastic.co/t/mssql-server-connectivity-issue-with-logstash/337165 "2023-06-29T07:56:22Z")
I am trying to fetch data from MSSQL with the following mssql.conf script in config directory input { jdbc { jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc\_driver\_library =\> "" jdbc\_connect… --- ## [Problem with data field in logstash](https://discuss.elastic.co/t/problem-with-data-field-in-logstash/336875)
**Author:** [@KarlWolf](https://discuss.elastic.co/u/KarlWolf)\ **Replies:** 2\ **Last updated:** [June 28, 2023, 11:19pm UTC](https://discuss.elastic.co/t/problem-with-data-field-in-logstash/336875 "2023-06-28T23:19:01Z")
hi, I have a strange case regarding my Logstash process. I'm having a filebeat which sends file-log to Logstash. That file log is updated from old proxy system in the following manner: logs from 15 minutes are gathere… --- ## [Kafka Codec Avro Plugin Polling Frequency](https://discuss.elastic.co/t/kafka-codec-avro-plugin-polling-frequency/337121)
**Author:** [@fine\_porcupine](https://discuss.elastic.co/u/fine_porcupine)\ **Replies:** 1\ **Last updated:** [June 28, 2023, 10:52pm UTC](https://discuss.elastic.co/t/kafka-codec-avro-plugin-polling-frequency/337121 "2023-06-28T22:52:25Z")
I'm planning on using the Codec Avro Plugin to deserialize incoming events from SQS. However, requests to the Kafka schema registry are rate-limited to 25 queries/second when using an HTTP URI. How frequently does this … --- ## [Data transfer from logstash to kibana](https://discuss.elastic.co/t/data-transfer-from-logstash-to-kibana/337055)
**Author:** [@kazuo](https://discuss.elastic.co/u/kazuo)\ **Replies:** 1\ **Last updated:** [June 28, 2023, 8:49am UTC](https://discuss.elastic.co/t/data-transfer-from-logstash-to-kibana/337055 "2023-06-28T08:49:05Z")
Hello, Output drop occurs in L2SW when transferring data from logstash to kibana. Are there any parameters that control data transfer with logstash? Thank you in advance --- ## [Read a date of a file to add this property in Logstash to send a ElasticSearch](https://discuss.elastic.co/t/read-a-date-of-a-file-to-add-this-property-in-logstash-to-send-a-elasticsearch/335359)
**Author:** [@AlejandroVindel](https://discuss.elastic.co/u/AlejandroVindel)\ **Replies:** 2\ **Last updated:** [June 28, 2023, 8:36am UTC](https://discuss.elastic.co/t/read-a-date-of-a-file-to-add-this-property-in-logstash-to-send-a-elasticsearch/335359 "2023-06-28T08:36:09Z")
HI, I am collecting files with Logstash and sending them to an Elasticsearch database. But I'm running into the problem that I can't pick up the date that file was created or last modified. I'm working on Linux, and wh… --- ## [Which nodes to have logstash send to cluster](https://discuss.elastic.co/t/which-nodes-to-have-logstash-send-to-cluster/337018)
**Author:** [@eh2021-elastic](https://discuss.elastic.co/u/eh2021-elastic)\ **Replies:** 0\ **Last updated:** [June 27, 2023, 6:37pm UTC](https://discuss.elastic.co/t/which-nodes-to-have-logstash-send-to-cluster/337018 "2023-06-27T18:37:00Z")
I have an elastic 7.16 cluster that consist of 4 dedicated masterand 16 data nodes. I have logstash sending syslog data from various network systems, firewalls, etc and just noticed the logstash config on some devices di… --- ## [Pipeline error "pipeline-id" :exception=\>#\ **Author:** [@Gelinski](https://discuss.elastic.co/u/Gelinski)\ **Replies:** 2\ **Last updated:** [June 27, 2023, 6:29pm UTC](https://discuss.elastic.co/t/pipeline-error-pipeline-id-exception-psych-tried-to-load-unspecified-class-time/336786 "2023-06-27T18:29:20Z") Hello folks, I have a logstash pipeline that is using a jdbc input and is configured with a schedule (0/1 \* \* \* \*) and after change its schedule to any other schedule the following error starts to happen: \[2023-06-23T… --- ## [Stopping logstash](https://discuss.elastic.co/t/stopping-logstash/336586)
**Author:** [@Hanni](https://discuss.elastic.co/u/Hanni)\ **Replies:** 13\ **Last updated:** [June 27, 2023, 1:06pm UTC](https://discuss.elastic.co/t/stopping-logstash/336586 "2023-06-27T13:06:11Z")
Hello, i'm currently working on logstash and i have a question. To launch my logtash script, i use this command: sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf When I run it in my terminal, to… --- ## [Using the "docker.elastic.co" container registry behind a firewall](https://discuss.elastic.co/t/using-the-docker-elastic-co-container-registry-behind-a-firewall/336888)
**Author:** [@haseHH](https://discuss.elastic.co/u/haseHH)\ **Replies:** 2\ **Last updated:** [June 27, 2023, 11:42am UTC](https://discuss.elastic.co/t/using-the-docker-elastic-co-container-registry-behind-a-firewall/336888 "2023-06-27T11:42:49Z")
Hi everyone, I am looking for guidance on how to properly access the "docker.elastic.co" CR behind a corporate firewall. This question was already posed back in 2020, but never answered. Here's that original topic. Wha… --- ## [Logstash HTTP filter shows ruby exception NoMethodError strip for nil class](https://discuss.elastic.co/t/logstash-http-filter-shows-ruby-exception-nomethoderror-strip-for-nil-class/336947)
**Author:** [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)\ **Replies:** 0\ **Last updated:** [June 27, 2023, 7:28am UTC](https://discuss.elastic.co/t/logstash-http-filter-shows-ruby-exception-nomethoderror-strip-for-nil-class/336947 "2023-06-27T07:28:30Z")
Hi Team, I am extracting contents of thousands of URLs using http filter. But some urls throws below error which is stopping pipeline. Pipeline worker error, the pipeline will be stopped {:pipeline\_id=\>"new-english-pd… --- ## [Converting filebeat message with logstash](https://discuss.elastic.co/t/converting-filebeat-message-with-logstash/336931)
**Author:** [@fizem](https://discuss.elastic.co/u/fizem)\ **Replies:** 1\ **Last updated:** [June 26, 2023, 9:32pm UTC](https://discuss.elastic.co/t/converting-filebeat-message-with-logstash/336931 "2023-06-26T21:32:37Z")
Hi, I have setup filebeat to parse my API logs and send messages to a centralized logstash cluster. filebeat will collect all the logs with a minimum CPU consumption. logstash can transform the data, define multiple … --- ## [Setup\_ssl error](https://discuss.elastic.co/t/setup-ssl-error/336840)
**Author:** [@Lampros](https://discuss.elastic.co/u/Lampros)\ **Replies:** 7\ **Last updated:** [June 26, 2023, 3:41pm UTC](https://discuss.elastic.co/t/setup-ssl-error/336840 "2023-06-26T15:41:57Z")
Hello, I am a little bit lost on this topic. I have a container which is running logstash. What I am trying to do is to use the output syslog module to send logs to a third party application over ssl. But it fails wi… --- ## [Pull data from Snowflake Database Tables to Elasticsearch using Logstash](https://discuss.elastic.co/t/pull-data-from-snowflake-database-tables-to-elasticsearch-using-logstash/336869)
**Author:** [@Shalinicts](https://discuss.elastic.co/u/Shalinicts)\ **Replies:** 2\ **Last updated:** [June 26, 2023, 11:07am UTC](https://discuss.elastic.co/t/pull-data-from-snowflake-database-tables-to-elasticsearch-using-logstash/336869 "2023-06-26T11:07:46Z")
Hi Team, Can anyone let know how can we pull data from snowflake tables (Saas based solution) to Elasticsearch onpremise using Logstash Thanks in advance. --- ## [Encountered logstash error "Expected one of \[ \\\\t\\\\r\\\\n\], \\"#\\", \\"input\\", \\"filter\\", \\"output\\" at line 1, column 1 (byte 1)""](https://discuss.elastic.co/t/encountered-logstash-error-expected-one-of-t-r-n-input-filter-output-at-line-1-column-1-byte-1/336796)
**Author:** [@pdowma](https://discuss.elastic.co/u/pdowma)\ **Replies:** 1\ **Last updated:** [June 26, 2023, 1:40am UTC](https://discuss.elastic.co/t/encountered-logstash-error-expected-one-of-t-r-n-input-filter-output-at-line-1-column-1-byte-1/336796 "2023-06-26T01:40:23Z")
Problem: When setting up a Docker-based Elastic Stack (Elasticsearch, Logstash, and Kibana) environment. The Logstash service was not able to start correctly and reported the following error message: \[2023-06-23T16:41:… --- ## [Is it possible to create a dynamic table name in statement of jdbc input plugin in Logstash?](https://discuss.elastic.co/t/is-it-possible-to-create-a-dynamic-table-name-in-statement-of-jdbc-input-plugin-in-logstash/336846)
**Author:** [@rabih](https://discuss.elastic.co/u/rabih)\ **Replies:** 1\ **Last updated:** [June 25, 2023, 7:51pm UTC](https://discuss.elastic.co/t/is-it-possible-to-create-a-dynamic-table-name-in-statement-of-jdbc-input-plugin-in-logstash/336846 "2023-06-25T19:51:18Z")
input { jdbc { jdbc\_connection\_string =\> "jdbc:sqlserver://ip\_address:1433;databaseName=database\_name;encrypt=true;trustServerCertificate=true;" jdbc\_user =\> "userxxxx" jdbc\_password =\> "passxxxx" jdbc\_… --- ## [Mutate - add\_field - only shows string not the value](https://discuss.elastic.co/t/mutate-add-field-only-shows-string-not-the-value/336816)
**Author:** [@humblemags](https://discuss.elastic.co/u/humblemags)\ **Replies:** 2\ **Last updated:** [June 24, 2023, 8:37pm UTC](https://discuss.elastic.co/t/mutate-add-field-only-shows-string-not-the-value/336816 "2023-06-24T20:37:34Z")
Hi, I am using Windows 10 with 7.17.6 on localhost install. Filebeat is input being sent to Logstash. Yes, I know the json parser will handle this for me. But I do not understand why "someNewField" does not have the v… --- ## [Logstash date timezone](https://discuss.elastic.co/t/logstash-date-timezone/336803)
**Author:** [@Mahdi\_Davoodi](https://discuss.elastic.co/u/Mahdi_Davoodi)\ **Replies:** 6\ **Last updated:** [June 24, 2023, 3:00pm UTC](https://discuss.elastic.co/t/logstash-date-timezone/336803 "2023-06-24T15:00:14Z")
I want to parse date-time records with logstash date filter. My records have Asia/Tehran time zone. After the recent changes in the time zone in Iran and the removal of DST from it, apparently my date of records does no… --- ## [\[ERROR\]\[logstash.filters.aggregate\]\[main\]Aggregate exception occurred {:error=\>#\](https://discuss.elastic.co/t/error-logstash-filters-aggregate-main-aggregate-exception-occurred-error-nomethoderror-undefined-method-for-nil-nilclass/336741)
**Author:** [@Ceyhun\_Quliyev](https://discuss.elastic.co/u/Ceyhun_Quliyev)\ **Replies:** 2\ **Last updated:** [June 23, 2023, 4:25pm UTC](https://discuss.elastic.co/t/error-logstash-filters-aggregate-main-aggregate-exception-occurred-error-nomethoderror-undefined-method-for-nil-nilclass/336741 "2023-06-23T16:25:51Z")
Dear forum members, We have encountered a problem and cannot solve it. I would be grateful if you could help us with a solution. Below I am providing a link to the configuration file itself and the error logs. --- ## [Access fields from input plugin (cloudwatch\_logs\_importer)](https://discuss.elastic.co/t/access-fields-from-input-plugin-cloudwatch-logs-importer/336585)
**Author:** [@Maarten\_Dekker](https://discuss.elastic.co/u/Maarten_Dekker)\ **Replies:** 19\ **Last updated:** [June 23, 2023, 4:13pm UTC](https://discuss.elastic.co/t/access-fields-from-input-plugin-cloudwatch-logs-importer/336585 "2023-06-23T16:13:27Z")
Hi, I am using the cloudwatch\_logs\_importer plugin to read and grok logs from cloudwatch. It all works fine, but I am facing issues to access a field which is created by the input module itself: \[cloudwatch\_logs\]\[log\_… --- ## [Logstash won't start as deamon](https://discuss.elastic.co/t/logstash-wont-start-as-deamon/336629)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 3\ **Last updated:** [June 23, 2023, 2:51pm UTC](https://discuss.elastic.co/t/logstash-wont-start-as-deamon/336629 "2023-06-23T14:51:41Z")
I know I have seen this issue in past and was fixed by using different ls\_temp dir for logstash. but this time it won't work. here is error message. any idea? this dir /s1/log/logstash is wide open stat /s1/log/logst… --- ## [Multiple tables as jdbc input in logstash pipeline](https://discuss.elastic.co/t/multiple-tables-as-jdbc-input-in-logstash-pipeline/336724)
**Author:** [@PodarcisMuralis](https://discuss.elastic.co/u/PodarcisMuralis)\ **Replies:** 4\ **Last updated:** [June 23, 2023, 1:58pm UTC](https://discuss.elastic.co/t/multiple-tables-as-jdbc-input-in-logstash-pipeline/336724 "2023-06-23T13:58:09Z")
I have 4 tables in oracle database and have a SQL query beginning with CTS (Common Table Expressions) which combines and creates a new single table. I want to create a pipeline using jdbc input and read date once in a … --- ## [Logstash s3 input reads file multiple times](https://discuss.elastic.co/t/logstash-s3-input-reads-file-multiple-times/336751)
**Author:** [@jpchev](https://discuss.elastic.co/u/jpchev)\ **Replies:** 0\ **Last updated:** [June 23, 2023, 8:52am UTC](https://discuss.elastic.co/t/logstash-s3-input-reads-file-multiple-times/336751 "2023-06-23T08:52:56Z")
hello, I have configured the following s3 input in logstash, and it keeps reading the same file from the given S3 bucket. I noticed that the given sincedb file ${DATA\_DIR}/.sincedb\_activities.txt is being used and it co… --- ## [Command line syntax to use the logstash.yml instead of the pipeline (.conf)](https://discuss.elastic.co/t/command-line-syntax-to-use-the-logstash-yml-instead-of-the-pipeline-conf/336630)
**Author:** [@elBilo](https://discuss.elastic.co/u/elBilo)\ **Replies:** 5\ **Last updated:** [June 22, 2023, 9:25pm UTC](https://discuss.elastic.co/t/command-line-syntax-to-use-the-logstash-yml-instead-of-the-pipeline-conf/336630 "2023-06-22T21:25:36Z")
I see plenty of examples with the command line option -f for using a specific pipeline file can I use the same syntax to use a specific logstash.yml file? the current helm file settings I have are command: - logstash … --- ## [Reading containers logs using FileBeat and logstash](https://discuss.elastic.co/t/reading-containers-logs-using-filebeat-and-logstash/336717)
**Author:** [@M\_D](https://discuss.elastic.co/u/M_D)\ **Replies:** 0\ **Last updated:** [June 22, 2023, 5:18pm UTC](https://discuss.elastic.co/t/reading-containers-logs-using-filebeat-and-logstash/336717 "2023-06-22T17:18:58Z")
I am trying to read docker containers logs using Filebeat and logstash. right now, i got multiple outputs like following: { "@version" =\> "1", "event" =\> { "original" =\> "233.61.46.84 - - \[22/Jun… --- ## [Usage of Logstash - Nullifying the logs when not required to be published](https://discuss.elastic.co/t/usage-of-logstash-nullifying-the-logs-when-not-required-to-be-published/334027)
**Author:** [@pavan\_tiriveedhi](https://discuss.elastic.co/u/pavan_tiriveedhi)\ **Replies:** 5\ **Last updated:** [June 22, 2023, 3:40pm UTC](https://discuss.elastic.co/t/usage-of-logstash-nullifying-the-logs-when-not-required-to-be-published/334027 "2023-06-22T15:40:14Z")
Hi, We are using Logstash to push logs from our kubernetes environments to Azure for storing and computing, we have installed the services via help chart - helm-charts/logstash at main · elastic/helm-charts · GitHub. Mo… --- ## [Logstash conditional statement not working](https://discuss.elastic.co/t/logstash-conditional-statement-not-working/336657)
**Author:** [@Shabu](https://discuss.elastic.co/u/Shabu)\ **Replies:** 4\ **Last updated:** [June 22, 2023, 11:18am UTC](https://discuss.elastic.co/t/logstash-conditional-statement-not-working/336657 "2023-06-22T11:18:14Z")
I want to use a conditional statement in my logstash config, so that syslogs are sent to "syslogindex" and other logs are sent to "testindex". I have tried multiple things, but It just does not work. This is my config: … --- ## [Logstash JDBC connection not sending request to Database custom port](https://discuss.elastic.co/t/logstash-jdbc-connection-not-sending-request-to-database-custom-port/336664)
**Author:** [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Replies:** 0\ **Last updated:** [June 22, 2023, 10:11am UTC](https://discuss.elastic.co/t/logstash-jdbc-connection-not-sending-request-to-database-custom-port/336664 "2023-06-22T10:11:05Z")
When I I am trying to retrieve data from a MSSQL database , Logstash JDBC input not connecting to the required custom port 59237. Logstash only trying to connect port 3306. Mentioned the required port 59237 in Connectio… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=68) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=70)