# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=96 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 97 --- ## [Syslog output plugin :number type input parametrization](https://discuss.elastic.co/t/syslog-output-plugin-number-type-input-parametrization/322905)
**Author:** [@arirajamaki](https://discuss.elastic.co/u/arirajamaki)\ **Replies:** 3\ **Last updated:** [January 11, 2023, 12:38pm UTC](https://discuss.elastic.co/t/syslog-output-plugin-number-type-input-parametrization/322905 "2023-01-11T12:38:49Z")
Hello Community, I'm in trouble with my logstash pipeline configuration. I'm trying to use syslog output plugin so that I can dynamically change the destination syslog server port. I'm trying do tcp/udp connection to d… --- ## [Running logstash with user nobody:nogroup is broken in logstash:7.17.8 - Easily reproducible](https://discuss.elastic.co/t/running-logstash-with-user-nobody-nogroup-is-broken-in-logstash-7-17-8-easily-reproducible/322874)
**Author:** [@mikeba](https://discuss.elastic.co/u/mikeba)\ **Replies:** 1\ **Last updated:** [January 11, 2023, 10:42am UTC](https://discuss.elastic.co/t/running-logstash-with-user-nobody-nogroup-is-broken-in-logstash-7-17-8-easily-reproducible/322874 "2023-01-11T10:42:49Z")
Our usage of logstash requires that we run as user nobody:nogroup and we run with an appropriate securityContext in K8s that enforces it + denies PriviledgeEscalation to root securityContext: allowPrivilegeEsc… --- ## [SNMP Traps version 3](https://discuss.elastic.co/t/snmp-traps-version-3/322889)
**Author:** [@m3bgwad](https://discuss.elastic.co/u/m3bgwad)\ **Replies:** 4\ **Last updated:** [January 11, 2023, 10:06am UTC](https://discuss.elastic.co/t/snmp-traps-version-3/322889 "2023-01-11T10:06:39Z")
Hello Everyone, Can the snmptrap input plugin for logstash receive snmp version 3 traps ? If not, are there other solutions ? I would like to collect SNMP v3 traps. Thank You --- ## [Exclude log messages in logstash](https://discuss.elastic.co/t/exclude-log-messages-in-logstash/322829)
**Author:** [@tejal\_kubde](https://discuss.elastic.co/u/tejal_kubde)\ **Replies:** 7\ **Last updated:** [January 11, 2023, 6:31am UTC](https://discuss.elastic.co/t/exclude-log-messages-in-logstash/322829 "2023-01-11T06:31:50Z")
I'm picking up data from log files using filebeat and sending it to Elasticsearch via Logstash. I wanted to exclude few log lines. So can I use an if condition in Logstash. If yes, please share me the format and guide me… --- ## [Logstash configuration to delete input files once processed](https://discuss.elastic.co/t/logstash-configuration-to-delete-input-files-once-processed/322029)
**Author:** [@BhawanaSharma](https://discuss.elastic.co/u/BhawanaSharma)\ **Replies:** 11\ **Last updated:** [January 11, 2023, 6:27am UTC](https://discuss.elastic.co/t/logstash-configuration-to-delete-input-files-once-processed/322029 "2023-01-11T06:27:43Z")
I am trying to delete input files from directory once it was processed by Filebeat. Also wanted to confirm from filebeat that particular file is already processed so it is safe to delete. For that I tried few things on m… --- ## [How does logstash use pipeline.batch.size to execute pipeline?](https://discuss.elastic.co/t/how-does-logstash-use-pipeline-batch-size-to-execute-pipeline/322772)
**Author:** [@rickfish](https://discuss.elastic.co/u/rickfish)\ **Replies:** 7\ **Last updated:** [January 10, 2023, 2:00pm UTC](https://discuss.elastic.co/t/how-does-logstash-use-pipeline-batch-size-to-execute-pipeline/322772 "2023-01-10T14:00:03Z")
I am looking for documentation on how Logstash processes a pipeline when pipeline.batch.size is more than 1. This is my assumption: The input plugin generates several events, not dictated by batch size While there are… --- ## [Data duplication happening when using multiple configuration for logstash with different input ports](https://discuss.elastic.co/t/data-duplication-happening-when-using-multiple-configuration-for-logstash-with-different-input-ports/322799)
**Author:** [@asambasivan](https://discuss.elastic.co/u/asambasivan)\ **Replies:** 2\ **Last updated:** [January 10, 2023, 11:05am UTC](https://discuss.elastic.co/t/data-duplication-happening-when-using-multiple-configuration-for-logstash-with-different-input-ports/322799 "2023-01-10T11:05:32Z")
Hello, We have two logstash configurations with two different input ports 5044 and 5045 for accepting connections, but while the filebeat is sending logs to logstash via port 5045 the data is getting duplicated and we a… --- ## [Can I use pattern in Logstash output plugin](https://discuss.elastic.co/t/can-i-use-pattern-in-logstash-output-plugin/322600)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 12\ **Last updated:** [January 10, 2023, 7:30am UTC](https://discuss.elastic.co/t/can-i-use-pattern-in-logstash-output-plugin/322600 "2023-01-10T07:30:27Z")
Hello, I want to track all the failures log in a file. For Example, if "\_jsonparsefailure" in \[tags\] { file { path =\> "\_jsonparsefailure.txt" } } Can I use pattern like - if "\*failure\*" in \[tags\] so that I… --- ## [Log events not stored in syslog persistent queue](https://discuss.elastic.co/t/log-events-not-stored-in-syslog-persistent-queue/322622)
**Author:** [@ferdose\_shaik](https://discuss.elastic.co/u/ferdose_shaik)\ **Replies:** 3\ **Last updated:** [January 10, 2023, 7:17am UTC](https://discuss.elastic.co/t/log-events-not-stored-in-syslog-persistent-queue/322622 "2023-01-10T07:17:05Z")
Hi, We are using Logstash 8.3.0. We are using syslog output with persistent queue. Please see following configuration. logstash.yml: ---- http.host: "0.0.0.0" http.port: 9600 log.level: "info" pipeline.workers: 2 pipel… --- ## [Logstash regex pattern for a windows path](https://discuss.elastic.co/t/logstash-regex-pattern-for-a-windows-path/322790)
**Author:** [@ShubhamKumarJena](https://discuss.elastic.co/u/ShubhamKumarJena)\ **Replies:** 0\ **Last updated:** [January 10, 2023, 6:22am UTC](https://discuss.elastic.co/t/logstash-regex-pattern-for-a-windows-path/322790 "2023-01-10T06:22:33Z")
Hello Elastic community, I am trying to use a if condition for my logstash filter for a windows path but not sure about the delimiter and correct syntax. Here is my complete windows directory " R:\\I3\\IC\\Logs\\Sabio\_Elk\_… --- ## [Logstash configuration](https://discuss.elastic.co/t/logstash-configuration/322736)
**Author:** [@themainguru](https://discuss.elastic.co/u/themainguru)\ **Replies:** 1\ **Last updated:** [January 9, 2023, 10:59pm UTC](https://discuss.elastic.co/t/logstash-configuration/322736 "2023-01-09T22:59:24Z")
Hello All, I am using a docker image of sebp/elk. I have setup an EC2 Ubuntu machine. This is a vanilla installation. I have a rails app on another server. I want to push logs from rails to logstash. I am not sure wher… --- ## [Indexing multiple csv files into one index with nested fields](https://discuss.elastic.co/t/indexing-multiple-csv-files-into-one-index-with-nested-fields/322687)
**Author:** [@ivandreev1618](https://discuss.elastic.co/u/ivandreev1618)\ **Replies:** 1\ **Last updated:** [January 9, 2023, 10:25pm UTC](https://discuss.elastic.co/t/indexing-multiple-csv-files-into-one-index-with-nested-fields/322687 "2023-01-09T22:25:10Z")
I want to load data from multiple CSV files(Users, Scores, Messages) into one index via logstash. All CSV files have the same "userId" field that connects data in it. My goal is to have User-Index as a result, that has… --- ## [Understanding metrics filters](https://discuss.elastic.co/t/understanding-metrics-filters/322648)
**Author:** [@Dustin527](https://discuss.elastic.co/u/Dustin527)\ **Replies:** 11\ **Last updated:** [January 9, 2023, 10:15pm UTC](https://discuss.elastic.co/t/understanding-metrics-filters/322648 "2023-01-09T22:15:23Z")
Hi I am having trouble understanding the proper way to use metrics filter plugin to count the number of fields of a particular type. Say I have a file formatted with a name and age separated by a space e.g.: dustin 40 e… --- ## [Logstash syntax error](https://discuss.elastic.co/t/logstash-syntax-error/322754)
**Author:** [@cool999](https://discuss.elastic.co/u/cool999)\ **Replies:** 5\ **Last updated:** [January 9, 2023, 8:28pm UTC](https://discuss.elastic.co/t/logstash-syntax-error/322754 "2023-01-09T20:28:20Z")
Hi Team, I have logstash on two servers. It seems its working as index are getting created and logstash service is running from few days but whenever i checked logstash syntax, it shows below error. Is there anything wr… --- ## [Syslog client doesn't send to Logstash](https://discuss.elastic.co/t/syslog-client-doesnt-send-to-logstash/322618)
**Author:** [@diegz](https://discuss.elastic.co/u/diegz)\ **Replies:** 6\ **Last updated:** [January 9, 2023, 4:52pm UTC](https://discuss.elastic.co/t/syslog-client-doesnt-send-to-logstash/322618 "2023-01-09T16:52:03Z")
Hello, Hello, I would like to get some advice for this configuration: Send rsyslog to my logstash. I tested both configurations without success. One moment it worked after restarting the pipeline nothing. $ModLoad i… --- ## [Logstash SSL verification](https://discuss.elastic.co/t/logstash-ssl-verification/322615)
**Author:** [@hanna](https://discuss.elastic.co/u/hanna)\ **Replies:** 7\ **Last updated:** [January 9, 2023, 11:15am UTC](https://discuss.elastic.co/t/logstash-ssl-verification/322615 "2023-01-09T11:15:05Z")
Hello, how can I enable SSL certificate verification in my logstash pipeline output to elasticsearch? I don't find any documentation on which certificates to use here. The cluster version is 8.5.3. This is my logstash… --- ## [Data Pipeline Design Considerations](https://discuss.elastic.co/t/data-pipeline-design-considerations/322717)
**Author:** [@Tiharqa](https://discuss.elastic.co/u/Tiharqa)\ **Replies:** 0\ **Last updated:** [January 9, 2023, 9:47am UTC](https://discuss.elastic.co/t/data-pipeline-design-considerations/322717 "2023-01-09T09:47:21Z")
So I have a confluent kafka with multiple topics per network divided as follows NET1: Data NET1-SYSLG NET1-WIN ====== NET2 : Data NET2-SYSLG NET2-WIN I have 2 logstash servers reading from those topics on Kafka … --- ## [Webhdfs Output plugin does not working with use\_kerberos\_auth](https://discuss.elastic.co/t/webhdfs-output-plugin-does-not-working-with-use-kerberos-auth/322714)
**Author:** [@vincent.ea3](https://discuss.elastic.co/u/vincent.ea3)\ **Replies:** 0\ **Last updated:** [January 9, 2023, 9:20am UTC](https://discuss.elastic.co/t/webhdfs-output-plugin-does-not-working-with-use-kerberos-auth/322714 "2023-01-09T09:20:05Z")
Hello, I am working to deploy logstash in k8s container to output message to hadoop. https://github.com/elastic/helm-charts According to the documentation, webHDFS and Kerberos authentication are supported. https://w… --- ## [Logstash in Docker and Elastic down](https://discuss.elastic.co/t/logstash-in-docker-and-elastic-down/322675)
**Author:** [@hofrichterovak](https://discuss.elastic.co/u/hofrichterovak)\ **Replies:** 2\ **Last updated:** [January 9, 2023, 9:06am UTC](https://discuss.elastic.co/t/logstash-in-docker-and-elastic-down/322675 "2023-01-09T09:06:38Z")
Hello, I'm running Logstash in Docker. Sometimes it happens that Elasticsearch is unavailable and when I restart the Logstash docker container, Logstash removing messages are they are no longer saved to Elasticsearch. I… --- ## [Communications link failure\\n\\nThe last packet successfully received from the server was 50,079 milliseconds ago. The last packet sent successfully to the server was 50,080 milliseconds ago.",](https://discuss.elastic.co/t/communications-link-failure-n-nthe-last-packet-successfully-received-from-the-server-was-50-079-milliseconds-ago-the-last-packet-sent-successfully-to-the-server-was-50-080-milliseconds-ago/322698)
**Author:** [@sohan](https://discuss.elastic.co/u/sohan)\ **Replies:** 0\ **Last updated:** [January 9, 2023, 4:49am UTC](https://discuss.elastic.co/t/communications-link-failure-n-nthe-last-packet-successfully-received-from-the-server-was-50-079-milliseconds-ago-the-last-packet-sent-successfully-to-the-server-was-50-080-milliseconds-ago/322698 "2023-01-09T04:49:53Z")
jdbc\_driver\_library =\> "/opt/bitnami/logstash/logstash-core/lib/jars/mysql-connector-java.jar" jdbc\_driver\_class =\> "com.mysql.cj.jdbc.Driver" jdbc\_connection\_string =\> "jdbc:mysql://db:3306/dbname?zeroDateTimeBehavior… --- ## [Logstash pkg update results log prasing stopped](https://discuss.elastic.co/t/logstash-pkg-update-results-log-prasing-stopped/322542)
**Author:** [@Dilipssn](https://discuss.elastic.co/u/Dilipssn)\ **Replies:** 7\ **Last updated:** [January 9, 2023, 1:57am UTC](https://discuss.elastic.co/t/logstash-pkg-update-results-log-prasing-stopped/322542 "2023-01-09T01:57:50Z")
Hello Team, We have logstash to push the logs from one server to other, where all the traps are collected. Through which we plot graphs in "Grafana". The port number in which logs parsing is "7546". until "logstash-7… --- ## [Using NDJSON for Logstash HTTP Output Plugin](https://discuss.elastic.co/t/using-ndjson-for-logstash-http-output-plugin/322683)
**Author:** [@gs04](https://discuss.elastic.co/u/gs04)\ **Replies:** 0\ **Last updated:** [January 8, 2023, 3:29pm UTC](https://discuss.elastic.co/t/using-ndjson-for-logstash-http-output-plugin/322683 "2023-01-08T15:29:00Z")
I'm looking to use a Logstash http output plugin to send a batch of JSON events where all the events are stored in the HTTP message as new-line delimited JSON events. For example, we'd need the data of the HTTP transmis… --- ## [Kubernetes Logstash statefulset under a Service of type Load Balancer (Amazon EKS) uneven distribution of events between pods](https://discuss.elastic.co/t/kubernetes-logstash-statefulset-under-a-service-of-type-load-balancer-amazon-eks-uneven-distribution-of-events-between-pods/322578)
**Author:** [@vikasp](https://discuss.elastic.co/u/vikasp)\ **Replies:** 3\ **Last updated:** [January 8, 2023, 3:08pm UTC](https://discuss.elastic.co/t/kubernetes-logstash-statefulset-under-a-service-of-type-load-balancer-amazon-eks-uneven-distribution-of-events-between-pods/322578 "2023-01-08T15:08:20Z")
I am working with a self managed elasticsearch cluster hosted in Amazon EKS. The pipeline flow is: filebeat agent is deployed in all ec2 servers seding data to logstash. https://logstash.company.com:5046. Logstash is … --- ## [Json data from Filebeat to Logstash](https://discuss.elastic.co/t/json-data-from-filebeat-to-logstash/322672)
**Author:** [@Hamburglar](https://discuss.elastic.co/u/Hamburglar)\ **Replies:** 4\ **Last updated:** [January 8, 2023, 1:54pm UTC](https://discuss.elastic.co/t/json-data-from-filebeat-to-logstash/322672 "2023-01-08T13:54:35Z")
Json data to be ingested: /var/log/file.json {"event\_type":"temp","time":"2023-01-07 23:30:12","temp":64.0,"fan":2291} {"event\_type":"temp","time":"2023-01-07 23:30:22","temp":63.0,"fan":2308} {"event\_type":"temp","time… --- ## [Jdbc mongo plugin](https://discuss.elastic.co/t/jdbc-mongo-plugin/322282)
**Author:** [@reza\_sabz](https://discuss.elastic.co/u/reza_sabz)\ **Replies:** 2\ **Last updated:** [January 8, 2023, 8:58am UTC](https://discuss.elastic.co/t/jdbc-mongo-plugin/322282 "2023-01-08T08:58:46Z")
Hello every one I want to connect mongodb to elastic via logstash. I did a lot of research on this. I checked and tested many methods. Finally, I reached jdbc. I have a problem in configuring this plugin and somehow I i… --- ## [S3 output: Is there a way to have every event in its own file?](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663)
**Author:** [@rickfish](https://discuss.elastic.co/u/rickfish)\ **Replies:** 4\ **Last updated:** [January 7, 2023, 6:57pm UTC](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663 "2023-01-07T18:57:43Z")
I have an unusual requirement to put every event in a separate file in an s3 bucket. I have tried to use this: rotation\_strategy =\> "size" size\_file =\> 1 but that is just an estimate and it still puts multiple events i… --- ## [Constant logstash s3 input plugin restart due to failing to open TCP connection to the target s3 bucket](https://discuss.elastic.co/t/constant-logstash-s3-input-plugin-restart-due-to-failing-to-open-tcp-connection-to-the-target-s3-bucket/322637)
**Author:** [@maltesersabc](https://discuss.elastic.co/u/maltesersabc)\ **Replies:** 2\ **Last updated:** [January 6, 2023, 8:03pm UTC](https://discuss.elastic.co/t/constant-logstash-s3-input-plugin-restart-due-to-failing-to-open-tcp-connection-to-the-target-s3-bucket/322637 "2023-01-06T20:03:30Z")
Background: We are using logstash s3 input plugin to ingest the logs from s3 bucket in AWS, however, we observed in the logstash plain logs, there are constant plugin errors that lead to the restart of the plugin. And t… --- ## [How to sync data between Azure SQL Database with Elasticsearch through Logstash](https://discuss.elastic.co/t/how-to-sync-data-between-azure-sql-database-with-elasticsearch-through-logstash/322560)
**Author:** [@seerj29](https://discuss.elastic.co/u/seerj29)\ **Replies:** 2\ **Last updated:** [January 6, 2023, 2:36pm UTC](https://discuss.elastic.co/t/how-to-sync-data-between-azure-sql-database-with-elasticsearch-through-logstash/322560 "2023-01-06T14:36:37Z")
Hi everyone, I'm trying to find a way to sync data between Azure SQL Database with Elasticsearch using Logstash. I found an article about doing this with the JBDC plugin, but if I understood correctly this is only for … --- ## [Grok Filter not working with File beat Log Message](https://discuss.elastic.co/t/grok-filter-not-working-with-file-beat-log-message/322605)
**Author:** [@Prakash111](https://discuss.elastic.co/u/Prakash111)\ **Replies:** 0\ **Last updated:** [January 6, 2023, 8:58am UTC](https://discuss.elastic.co/t/grok-filter-not-working-with-file-beat-log-message/322605 "2023-01-06T08:58:03Z")
I'm using file beat for as log collector. my log data : 2023-01-05T11:57:48.179Z \[ERROR\] ABC Company {"Pod": "7d45bf43lbr", "Service": "liveX", "failed to create patch:": "invalid JSON Document"} I supposed to get Key… --- ## [How to properly connect a standalone Logstash to an Elasticsearch cluster](https://discuss.elastic.co/t/how-to-properly-connect-a-standalone-logstash-to-an-elasticsearch-cluster/322520)
**Author:** [@Francesco1966](https://discuss.elastic.co/u/Francesco1966)\ **Replies:** 2\ **Last updated:** [January 5, 2023, 9:24pm UTC](https://discuss.elastic.co/t/how-to-properly-connect-a-standalone-logstash-to-an-elasticsearch-cluster/322520 "2023-01-05T21:24:03Z")
Hello, I have successfully created an Elasticsearch Cluster with two servers Ubuntu each one running an Elasticsearch node (one ES-Node1 acting as master node and ES-Node2 acting as data node). I than created a Standal… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=95) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=97)