# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=45 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 46 --- ## [Plugin logstash.inputs.tcp debug logging showing many "initialized channel" messages](https://discuss.elastic.co/t/plugin-logstash-inputs-tcp-debug-logging-showing-many-initialized-channel-messages/350990)
**Author:** [@bbenne821](https://discuss.elastic.co/u/bbenne821)\ **Replies:** 0\ **Last updated:** [January 12, 2024, 9:30pm UTC](https://discuss.elastic.co/t/plugin-logstash-inputs-tcp-debug-logging-showing-many-initialized-channel-messages/350990 "2024-01-12T21:30:53Z")
Running OSS logstash 2.8.2, bundled JDK, on CentOS 7 Linux plugin tcp input specifying "tcp\_keep\_alive=true". Experiencing recurring "closing due: java.net.SocketException: Connection reset" errors for this pipeline (var… --- ## [Unable to start Logstash as a service. Errors with: Unable to locate required config /etc/logstash/logstash.conf](https://discuss.elastic.co/t/unable-to-start-logstash-as-a-service-errors-with-unable-to-locate-required-config-etc-logstash-logstash-conf/350938)
**Author:** [@Maiky](https://discuss.elastic.co/u/Maiky)\ **Replies:** 3\ **Last updated:** [January 12, 2024, 6:33pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-as-a-service-errors-with-unable-to-locate-required-config-etc-logstash-logstash-conf/350938 "2024-01-12T18:33:40Z")
Hi, On RHEL7 I'm able to run logstash v 7.17 directly as root like so: logstash -f /home/maiky/first-pipeline.conf --config.reload.automatic However when trying to run it as a service, I get the following error: Job … --- ## [Invalid UTF-8](https://discuss.elastic.co/t/invalid-utf-8/350978)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 7\ **Last updated:** [January 12, 2024, 5:10pm UTC](https://discuss.elastic.co/t/invalid-utf-8/350978 "2024-01-12T17:10:44Z")
I've been using ruby to decode hex to ascii if (\[field\]) { mutate { gsub =\> \[ "\[field\]", ":", "" \] } ruby { code =\> 'event.set("\[field\]", \[event.get("\[field\]")\].pack("H\*"))' } } but I'v… --- ## [Jenkins logstash plugin don't send build log](https://discuss.elastic.co/t/jenkins-logstash-plugin-dont-send-build-log/350967)
**Author:** [@khergner](https://discuss.elastic.co/u/khergner)\ **Replies:** 0\ **Last updated:** [January 12, 2024, 2:09pm UTC](https://discuss.elastic.co/t/jenkins-logstash-plugin-dont-send-build-log/350967 "2024-01-12T14:09:22Z")
Hi everyone I have a problem. ı want to use logstash plugin with jenkins. İt isn't send build log elasticsearch but ı have bellow error. I don't upgrade logstash latest plugin because many plugin must upgrade from jenk… --- ## [Custom logs from Logstash to Cloudwatch](https://discuss.elastic.co/t/custom-logs-from-logstash-to-cloudwatch/350955)
**Author:** [@Vadsgator](https://discuss.elastic.co/u/Vadsgator)\ **Replies:** 1\ **Last updated:** [January 12, 2024, 1:55pm UTC](https://discuss.elastic.co/t/custom-logs-from-logstash-to-cloudwatch/350955 "2024-01-12T13:55:28Z")
Hiya, Currently there is no actual support to send custom logs from Logstash to Cloudwatch. (There is a way to send metrics data using the Cloudwatch Output Plugin) and there was some support for a plugin called logstas… --- ## [Logstash input with beats function is not work good by OCP platform in ingress](https://discuss.elastic.co/t/logstash-input-with-beats-function-is-not-work-good-by-ocp-platform-in-ingress/350915)
**Author:** [@bigwind123](https://discuss.elastic.co/u/bigwind123)\ **Replies:** 0\ **Last updated:** [January 12, 2024, 7:47am UTC](https://discuss.elastic.co/t/logstash-input-with-beats-function-is-not-work-good-by-ocp-platform-in-ingress/350915 "2024-01-12T07:47:36Z")
I am currently facing a problem. I'm planning to set up an ELK service on a redhat ocp platform and install metricbeat on the VM to send the data to a logstash pod in ocp, I'm currently doing the following. a pod -\> e… --- ## [How to pause the logstash output temporarily](https://discuss.elastic.co/t/how-to-pause-the-logstash-output-temporarily/350841)
**Author:** [@jevonsnotes](https://discuss.elastic.co/u/jevonsnotes)\ **Replies:** 3\ **Last updated:** [January 12, 2024, 5:36am UTC](https://discuss.elastic.co/t/how-to-pause-the-logstash-output-temporarily/350841 "2024-01-12T05:36:49Z")
how to keep the logstash accept the input data but pause the output temporarily? --- ## [Logstash TCP Input Codecs](https://discuss.elastic.co/t/logstash-tcp-input-codecs/350517)
**Author:** [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\ **Replies:** 10\ **Last updated:** [January 11, 2024, 11:09pm UTC](https://discuss.elastic.co/t/logstash-tcp-input-codecs/350517 "2024-01-11T23:09:15Z")
Is there a place to undestand exactly what format each of the TCP input codecs are meant to cover? (line vs json vs plain vs cef). I have a situation of a new log source (Sophos firewall). Must use TLS so syslog input i… --- ## [Logstash date parse issue with date filter using csv file input plugin](https://discuss.elastic.co/t/logstash-date-parse-issue-with-date-filter-using-csv-file-input-plugin/350850)
**Author:** [@jgregory\_tc](https://discuss.elastic.co/u/jgregory_tc)\ **Replies:** 0\ **Last updated:** [January 11, 2024, 11:23am UTC](https://discuss.elastic.co/t/logstash-date-parse-issue-with-date-filter-using-csv-file-input-plugin/350850 "2024-01-11T11:23:30Z")
Hoping someone can assist me with my issue below: I have Logstash conf setup to use the csv input plugin. The data inputs a date field with value like follows… 2024-01-09 22:21:04 I then have this logic in the filter … --- ## [I have error in logstash](https://discuss.elastic.co/t/i-have-error-in-logstash/350790)
**Author:** [@ahmedtamawe](https://discuss.elastic.co/u/ahmedtamawe)\ **Replies:** 1\ **Last updated:** [January 10, 2024, 8:50pm UTC](https://discuss.elastic.co/t/i-have-error-in-logstash/350790 "2024-01-10T20:50:34Z")
i have this error and want to solve it --- ## [Logs received from panorama](https://discuss.elastic.co/t/logs-received-from-panorama/350785)
**Author:** [@juancamiloll](https://discuss.elastic.co/u/juancamiloll)\ **Replies:** 2\ **Last updated:** [January 10, 2024, 8:41pm UTC](https://discuss.elastic.co/t/logs-received-from-panorama/350785 "2024-01-10T20:41:25Z")
Hello everyone, Normally when I do the ELK installation I ask the firewall administrators to send the logs via port 514 TPC to the server I administer. In the server what I do is that I modify the rsyslog.com file to o… --- ## [How to extract the time stamp from](https://discuss.elastic.co/t/how-to-extract-the-time-stamp-from/348741)
**Author:** [@pero](https://discuss.elastic.co/u/pero)\ **Replies:** 20\ **Last updated:** [January 10, 2024, 5:24pm UTC](https://discuss.elastic.co/t/how-to-extract-the-time-stamp-from/348741 "2024-01-10T17:24:43Z")
Hi All, Please I need help on how to extract the timestamp from "type=SYSCALL msg=audit(1701877882.123:5786893): " in the below code using grok filter { "\_index": "auditbeat-2023.12.06", "\_type": "\_doc", "\_id": "… --- ## [How Could I send my logs from One EC2 instance to Other EC2 instance](https://discuss.elastic.co/t/how-could-i-send-my-logs-from-one-ec2-instance-to-other-ec2-instance/350724)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 0\ **Last updated:** [January 10, 2024, 9:59am UTC](https://discuss.elastic.co/t/how-could-i-send-my-logs-from-one-ec2-instance-to-other-ec2-instance/350724 "2024-01-10T09:59:55Z")
I configured two EC2 instance. In one instance I have Logstash and from other instance I want to send audit logs to Logstash. I am using SYSLOG input plugin to collect the events. input { syslog { port =\> 5… --- ## [Updating Elasticsearch Indices conditionally when referring to 2 database table](https://discuss.elastic.co/t/updating-elasticsearch-indices-conditionally-when-referring-to-2-database-table/350663)
**Author:** [@jainesh\_singh](https://discuss.elastic.co/u/jainesh_singh)\ **Replies:** 1\ **Last updated:** [January 10, 2024, 6:45am UTC](https://discuss.elastic.co/t/updating-elasticsearch-indices-conditionally-when-referring-to-2-database-table/350663 "2024-01-10T06:45:37Z")
Description: We have two SQL tables: FileDetail for storing file details and FileUserActivity for file activities. Using Logstash, we're indexing data into Elasticsearch with a flat index approach, combining file detail… --- ## [Remove Parent fields in logstash filter](https://discuss.elastic.co/t/remove-parent-fields-in-logstash-filter/350646)
**Author:** [@Priyanka\_chauhan](https://discuss.elastic.co/u/Priyanka_chauhan)\ **Replies:** 5\ **Last updated:** [January 10, 2024, 6:45am UTC](https://discuss.elastic.co/t/remove-parent-fields-in-logstash-filter/350646 "2024-01-10T06:45:26Z")
I have large log json message which I have to parse to visualize at kibana. I have used json filter first to parse message but there are generated lots of parent and dynamic fields. Due to dynamic fields in each log me… --- ## [GeoIP Filter in ECS-Compatiblity mode requires a \`target\` when \`source\` is not an \`ip\` sub-field, eg. \[client\]\[ip\]](https://discuss.elastic.co/t/geoip-filter-in-ecs-compatiblity-mode-requires-a-target-when-source-is-not-an-ip-sub-field-eg-client-ip/350343)
**Author:** [@e-ferrari](https://discuss.elastic.co/u/e-ferrari)\ **Replies:** 3\ **Last updated:** [January 9, 2024, 5:19pm UTC](https://discuss.elastic.co/t/geoip-filter-in-ecs-compatiblity-mode-requires-a-target-when-source-is-not-an-ip-sub-field-eg-client-ip/350343 "2024-01-09T17:19:26Z")
Hi, i'm trying to setup Parsing Logs with Logstash | Logstash Reference \[8.11\] | Elastic. But i get errors: \[2024-01-04T00:06:45,246\]\[ERROR\]\[logstash.javapipeline \]\[main\] Pipeline error {:pipeline\_id=\>"main", :exc… --- ## [Output based on grok message](https://discuss.elastic.co/t/output-based-on-grok-message/350670)
**Author:** [@Brandon\_Kauffman](https://discuss.elastic.co/u/Brandon_Kauffman)\ **Replies:** 3\ **Last updated:** [January 9, 2024, 5:17pm UTC](https://discuss.elastic.co/t/output-based-on-grok-message/350670 "2024-01-09T17:17:40Z")
I am trying to output based on different sysloghosts input { udp { port =\> 5010 type =\> "obs\_test\_udp" } } filter { grok { match =\> {"message" =\> "\<%{POSINT:syslog\_priority}\>%{POSINT:syslog\_version} %{T… --- ## [I want to Install Logstash in EC2 Linux UBUNTU 22.04 and want to run Syslog input Configuration but facing ERROR](https://discuss.elastic.co/t/i-want-to-install-logstash-in-ec2-linux-ubuntu-22-04-and-want-to-run-syslog-input-configuration-but-facing-error/350571)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 3\ **Last updated:** [January 9, 2024, 12:41pm UTC](https://discuss.elastic.co/t/i-want-to-install-logstash-in-ec2-linux-ubuntu-22-04-and-want-to-run-syslog-input-configuration-but-facing-error/350571 "2024-01-09T12:41:04Z")
I Followed this URL: Installing Logstash | Logstash Reference \[7.14\] | Elastic APT one I followed. Then after the Installation I set the path of bin in Environment variable using below command. 1- Location of logstas… --- ## [Is it possible to do Load balancing using Kafka Input Plugin](https://discuss.elastic.co/t/is-it-possible-to-do-load-balancing-using-kafka-input-plugin/350447)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 5\ **Last updated:** [January 9, 2024, 12:22pm UTC](https://discuss.elastic.co/t/is-it-possible-to-do-load-balancing-using-kafka-input-plugin/350447 "2024-01-09T12:22:26Z")
This is Configuartion I am Using. input { kafka{ #Insert any one string from the kafka\_brokers\_sasl from the service credential in the Event stream. bootstrap\_servers =\> "\" #Insert the … --- ## [Logstash is not printing the whole exception log in a single message](https://discuss.elastic.co/t/logstash-is-not-printing-the-whole-exception-log-in-a-single-message/350556)
**Author:** [@sudhir\_singh](https://discuss.elastic.co/u/sudhir_singh)\ **Replies:** 4\ **Last updated:** [January 8, 2024, 6:50pm UTC](https://discuss.elastic.co/t/logstash-is-not-printing-the-whole-exception-log-in-a-single-message/350556 "2024-01-08T18:50:46Z")
Below is my exception log which I'm sending to logstash through filebeat but it only prints the single line of it. It is not considering the whole message: Failed to complete request: org.springframework.web.multipart.M… --- ## [I am trying to Install SYSLOG input plugin in Logstash which I installed in EC2 Ubuntu Linux but nto able to do this](https://discuss.elastic.co/t/i-am-trying-to-install-syslog-input-plugin-in-logstash-which-i-installed-in-ec2-ubuntu-linux-but-nto-able-to-do-this/350443)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 3\ **Last updated:** [January 8, 2024, 12:35pm UTC](https://discuss.elastic.co/t/i-am-trying-to-install-syslog-input-plugin-in-logstash-which-i-installed-in-ec2-ubuntu-linux-but-nto-able-to-do-this/350443 "2024-01-08T12:35:55Z")
Setup I followed for Installation: Step to install : 1- Go to root : sudo su - 2- Download and install the Public Signing Key: wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /… --- ## [LogStash filter for matching timestamp example: \[2024-01-04 23:00:00,931\]](https://discuss.elastic.co/t/logstash-filter-for-matching-timestamp-example-2024-01-04-2300-931/350549)
**Author:** [@criss79](https://discuss.elastic.co/u/criss79)\ **Replies:** 2\ **Last updated:** [January 8, 2024, 10:07am UTC](https://discuss.elastic.co/t/logstash-filter-for-matching-timestamp-example-2024-01-04-2300-931/350549 "2024-01-08T10:07:02Z")
Hi guys, I am having difficulties to match this timestamp format for a log entry that looks like this: \[timestamp\] \[Loglevel\] message Log entry example: \[2024-01-04 23:00:00,931\] \[INFO\] Multi\_Language.UserInfoContain… --- ## [In Docker can we use 2 Logstash push log in to one elastic seach?](https://discuss.elastic.co/t/in-docker-can-we-use-2-logstash-push-log-in-to-one-elastic-seach/350542)
**Author:** [@2\_3\_0\_8](https://discuss.elastic.co/u/2_3_0_8)\ **Replies:** 0\ **Last updated:** [January 8, 2024, 3:34am UTC](https://discuss.elastic.co/t/in-docker-can-we-use-2-logstash-push-log-in-to-one-elastic-seach/350542 "2024-01-08T03:34:35Z")
Hi I want to know it can make it ? In Docker can we use 2 or more Logstash push log in to one elastic seach in one VM? if i have many gateway api (1VM for 1 gateway) for my plane i need to use 1 VM to create many logst… --- ## [The s3 input for creating the index does not work with preffix and csv files](https://discuss.elastic.co/t/the-s3-input-for-creating-the-index-does-not-work-with-preffix-and-csv-files/350496)
**Author:** [@Marcos\_Daniel\_Santos](https://discuss.elastic.co/u/Marcos_Daniel_Santos)\ **Replies:** 3\ **Last updated:** [January 6, 2024, 12:33pm UTC](https://discuss.elastic.co/t/the-s3-input-for-creating-the-index-does-not-work-with-preffix-and-csv-files/350496 "2024-01-06T12:33:04Z")
Hi everyone, I have a bucekt s3 with 2 csv files, one that is a securityhub repot and the other a guardduty report, both AWS services and security. I'm using the preffix to get my object, using the logstash -f file.conf… --- ## [Logstash tcp input plugin connection reset error](https://discuss.elastic.co/t/logstash-tcp-input-plugin-connection-reset-error/350441)
**Author:** [@fmelk65](https://discuss.elastic.co/u/fmelk65)\ **Replies:** 0\ **Last updated:** [January 5, 2024, 9:10am UTC](https://discuss.elastic.co/t/logstash-tcp-input-plugin-connection-reset-error/350441 "2024-01-05T09:10:16Z")
Hello, I have 25 Kubernetes clusters forward their logs to logstash VM (k8s fluentd ---\> logstash). I'm getting a lot of connection reset errors. It's been discussed here before, can @true64gurus specifically help? \[… --- ## [How to set volume map for logstash.conf file in docker compose file](https://discuss.elastic.co/t/how-to-set-volume-map-for-logstash-conf-file-in-docker-compose-file/350422)
**Author:** [@Sunny84](https://discuss.elastic.co/u/Sunny84)\ **Replies:** 1\ **Last updated:** [January 5, 2024, 8:46am UTC](https://discuss.elastic.co/t/how-to-set-volume-map-for-logstash-conf-file-in-docker-compose-file/350422 "2024-01-05T08:46:06Z")
Hello, I am trying to setup a docker compose file for setting up ELK stack to be able to use logging on local machine. Below is the error shown in the terminal window, my stack is windows 11 home, Docker Engine v24.0.7,… --- ## [Geo.location as object and not as geo\_point](https://discuss.elastic.co/t/geo-location-as-object-and-not-as-geo-point/349230)
**Author:** [@helldunkel](https://discuss.elastic.co/u/helldunkel)\ **Replies:** 3\ **Last updated:** [January 4, 2024, 11:48am UTC](https://discuss.elastic.co/t/geo-location-as-object-and-not-as-geo-point/349230 "2024-01-04T11:48:44Z")
Hi, I created a component template for a custon sensor. everything works, has the right datatypes. Only destination.geo.location is set to object and not to geo\_point: "destination": { "type": "object", … --- ## [Logstash output adding ES cluster exception](https://discuss.elastic.co/t/logstash-output-adding-es-cluster-exception/350345)
**Author:** [@kinho](https://discuss.elastic.co/u/kinho)\ **Replies:** 2\ **Last updated:** [January 4, 2024, 9:30am UTC](https://discuss.elastic.co/t/logstash-output-adding-es-cluster-exception/350345 "2024-01-04T09:30:52Z")
Logstash docker I plan to use Docker Compose to deploy ELK in Docker, where ES is a cluster The version is 8.11.3 .env: ELASTIC\_PASSWORD=123456 KIBANA\_PASSWORD=123456 STACK\_VERSION=8.11.3 CLUSTER\_NAME=docker-clu… --- ## [Logstash metrics using modules in Metricbeat](https://discuss.elastic.co/t/logstash-metrics-using-modules-in-metricbeat/350279)
**Author:** [@maadhav](https://discuss.elastic.co/u/maadhav)\ **Replies:** 4\ **Last updated:** [January 4, 2024, 5:41am UTC](https://discuss.elastic.co/t/logstash-metrics-using-modules-in-metricbeat/350279 "2024-01-04T05:41:29Z")
Hi Team There are 2 modules in Metricbeat to collect Logstash metrics logstash logstash-xpack Which module should be used ? Regards --- ## [How to read GZIP and encoding with UTF-8 logs from kafka topic through logstash pipeline](https://discuss.elastic.co/t/how-to-read-gzip-and-encoding-with-utf-8-logs-from-kafka-topic-through-logstash-pipeline/349775)
**Author:** [@upreddy](https://discuss.elastic.co/u/upreddy)\ **Replies:** 5\ **Last updated:** [January 3, 2024, 4:53pm UTC](https://discuss.elastic.co/t/how-to-read-gzip-and-encoding-with-utf-8-logs-from-kafka-topic-through-logstash-pipeline/349775 "2024-01-03T16:53:39Z")
Hi All, Application team doing "GZIP and encoding with UTF-8" and sending their logs to kafka topics. Now i want to read those logs through logstash pipeline. Could you please guide me on this? Thanks [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=44) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=46)