# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=88 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 89 --- ## [Query on Logstash S3 input](https://discuss.elastic.co/t/query-on-logstash-s3-input/325964)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 3\ **Last updated:** [February 27, 2023, 9:13pm UTC](https://discuss.elastic.co/t/query-on-logstash-s3-input/325964 "2023-02-27T21:13:39Z")
Hi Team, I wanted some clarification on the Logstash S3 input plugin behaviour. There is an option of "sincedb\_path" where as per documentation, it defines where to write the since database (keeps track of the date the… --- ## [After parsing in logstash got error](https://discuss.elastic.co/t/after-parsing-in-logstash-got-error/326560)
**Author:** [@neeldey](https://discuss.elastic.co/u/neeldey)\ **Replies:** 1\ **Last updated:** [February 27, 2023, 5:09pm UTC](https://discuss.elastic.co/t/after-parsing-in-logstash-got-error/326560 "2023-02-27T17:09:49Z")
Hi all, i getting error, while to start logstash. logstash log is as bellow. \[2023-02-27T15:18:50,526\]\[FATAL\]\[org.logstash.Logstash \] Logstash stopped processing because of an error: (SystemExit) exit org.jruby.ex… --- ## [Correct parsing Syslog message to json](https://discuss.elastic.co/t/correct-parsing-syslog-message-to-json/326564)
**Author:** [@poky](https://discuss.elastic.co/u/poky)\ **Replies:** 1\ **Last updated:** [February 27, 2023, 5:07pm UTC](https://discuss.elastic.co/t/correct-parsing-syslog-message-to-json/326564 "2023-02-27T17:07:34Z")
Hi Folks! I'm trying to parse the following message from mcafee proxy syslog inside my logstash pipeline: \<30\>Feb 24 9:33:45 mwg-n3 mwg-n3: x-message="{"DateTime":"2023-02-22 14:03:44.927","MWG\_Source":"mwg-n3.foo.de",… --- ## [How to Ingest MultiLine Json file into ElasticSearch using Logstash Pipeline](https://discuss.elastic.co/t/how-to-ingest-multiline-json-file-into-elasticsearch-using-logstash-pipeline/326580)
**Author:** [@prabhakar\_kamath](https://discuss.elastic.co/u/prabhakar_kamath)\ **Replies:** 0\ **Last updated:** [February 27, 2023, 12:15pm UTC](https://discuss.elastic.co/t/how-to-ingest-multiline-json-file-into-elasticsearch-using-logstash-pipeline/326580 "2023-02-27T12:15:29Z")
I have a json file similar to following: { "Key1": "value1", "Key2": "value2" ....... } I want to ingest it as it is into logstash, The Keys should be fields and values should be values to the field, value can be a… --- ## [Logstash input elasticsearch](https://discuss.elastic.co/t/logstash-input-elasticsearch/326561)
**Author:** [@almteref](https://discuss.elastic.co/u/almteref)\ **Replies:** 0\ **Last updated:** [February 27, 2023, 9:59am UTC](https://discuss.elastic.co/t/logstash-input-elasticsearch/326561 "2023-02-27T09:59:55Z")
Hi all I have question about the logstash in elasticsearch input plugin can I use the search template ID that I created in my cluster ? rether than pass query ? --- ## [Add a csv input for every batch](https://discuss.elastic.co/t/add-a-csv-input-for-every-batch/326553)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 0\ **Last updated:** [February 27, 2023, 8:23am UTC](https://discuss.elastic.co/t/add-a-csv-input-for-every-batch/326553 "2023-02-27T08:23:19Z")
Hello, I am using http\_poller input plugin and elasticsearch output plugin.. I want to use CSV output plugin also for logging logstash success in a CSV file..But in my CSV all the events are noted but not only once. I … --- ## [How can I add field by a same field when across event](https://discuss.elastic.co/t/how-can-i-add-field-by-a-same-field-when-across-event/326551)
**Author:** [@OICAn](https://discuss.elastic.co/u/OICAn)\ **Replies:** 0\ **Last updated:** [February 27, 2023, 8:09am UTC](https://discuss.elastic.co/t/how-can-i-add-field-by-a-same-field-when-across-event/326551 "2023-02-27T08:09:02Z")
A user access our system will trigger many transcations. A transaction will generate some log, which are serval event in es and they have a same field called "globalNo". One event will log the name of the transaction and… --- ## [Logstash not shipping data to Elasticsearch](https://discuss.elastic.co/t/logstash-not-shipping-data-to-elasticsearch/326376)
**Author:** [@Technolust](https://discuss.elastic.co/u/Technolust)\ **Replies:** 2\ **Last updated:** [February 25, 2023, 7:50pm UTC](https://discuss.elastic.co/t/logstash-not-shipping-data-to-elasticsearch/326376 "2023-02-25T19:50:43Z")
How do I get logstash to ship data to Elasticsearch? I'm not sure what to change in the logstash.yml file or what section I should change for that matter. My pipelines.yml points to /etc/logstash/conf.d/syslog.conf... Th… --- ## [How load big data from database using Logstash in elasticsearch index?](https://discuss.elastic.co/t/how-load-big-data-from-database-using-logstash-in-elasticsearch-index/326489)
**Author:** [@boliwe](https://discuss.elastic.co/u/boliwe)\ **Replies:** 0\ **Last updated:** [February 25, 2023, 8:06am UTC](https://discuss.elastic.co/t/how-load-big-data-from-database-using-logstash-in-elasticsearch-index/326489 "2023-02-25T08:06:52Z")
I want to learn how to load big data from database to elasticsearch using logstash jdbc input plugin. I could not find my answer from other forums. I have 1billion data in databse. Logstash settings has 8 workers, 15000… --- ## [Logstash.licensechecker.licensereader: elasticsearch: Name or service not known](https://discuss.elastic.co/t/logstash-licensechecker-licensereader-elasticsearch-name-or-service-not-known/326462)
**Author:** [@pocketcolin](https://discuss.elastic.co/u/pocketcolin)\ **Replies:** 2\ **Last updated:** [February 24, 2023, 6:53pm UTC](https://discuss.elastic.co/t/logstash-licensechecker-licensereader-elasticsearch-name-or-service-not-known/326462 "2023-02-24T18:53:33Z")
Apologies for all of the questions recently and huge thanks to everyone who has responded and helped me get this far. At this point I have logs being passed from a Render web server to a private service running Logstash … --- ## [Logstash is failing with file not found - 'cat JDK\_VERSION' when "LS\_JAVA\_HOME" is set](https://discuss.elastic.co/t/logstash-is-failing-with-file-not-found-cat-jdk-version-when-ls-java-home-is-set/326407)
**Author:** [@skumarp7](https://discuss.elastic.co/u/skumarp7)\ **Replies:** 1\ **Last updated:** [February 24, 2023, 5:23pm UTC](https://discuss.elastic.co/t/logstash-is-failing-with-file-not-found-cat-jdk-version-when-ls-java-home-is-set/326407 "2023-02-24T17:23:47Z")
Hi, Logstash RPM used: 8.6.1 We have built a docker image with the logstash rpm and running the container in our kubernetes cluster We are exporting LS\_JAVA\_HOME env in the container to the jdk already installed as a … --- ## [Unable to see trace id or transaction info in logs](https://discuss.elastic.co/t/unable-to-see-trace-id-or-transaction-info-in-logs/326375)
**Author:** [@pocketcolin](https://discuss.elastic.co/u/pocketcolin)\ **Replies:** 6\ **Last updated:** [February 24, 2023, 4:01pm UTC](https://discuss.elastic.co/t/unable-to-see-trace-id-or-transaction-info-in-logs/326375 "2023-02-24T16:01:44Z")
I currently have a NodeJS server passing logs to a Logstash server with a TCP tunnel using a Winston transport. Logs make it through all the way to my Elastic Cloud hosted Elasticsearch and pretty much everything looks g… --- ## [Configure Elasticsearch on Django App](https://discuss.elastic.co/t/configure-elasticsearch-on-django-app/326320)
**Author:** [@ekane3](https://discuss.elastic.co/u/ekane3)\ **Replies:** 8\ **Last updated:** [February 24, 2023, 1:31pm UTC](https://discuss.elastic.co/t/configure-elasticsearch-on-django-app/326320 "2023-02-24T13:31:52Z")
:wave: Hello everyone, I have been trying for weeks now to configure Elasticsearch/logstash on my Django app. But, all the tutorials i found are dealing with local elasticsearch meanwhile the one i’m dealing with is a… --- ## [Not able to connect to Elastic search from logstash](https://discuss.elastic.co/t/not-able-to-connect-to-elastic-search-from-logstash/326267)
**Author:** [@vijay78](https://discuss.elastic.co/u/vijay78)\ **Replies:** 4\ **Last updated:** [February 24, 2023, 1:11pm UTC](https://discuss.elastic.co/t/not-able-to-connect-to-elastic-search-from-logstash/326267 "2023-02-24T13:11:51Z")
iam getting below error after running logstash pod Using bundled JDK: /usr/share/logstash/jdk Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties \[2023-02-23T06:44:35,912\]\[I… --- ## [Logstash plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/logstash-plugin-had-an-unrecoverable-error-will-restart-this-plugin/326220)
**Author:** [@Vinicius\_Carmo](https://discuss.elastic.co/u/Vinicius_Carmo)\ **Replies:** 6\ **Last updated:** [February 24, 2023, 12:39pm UTC](https://discuss.elastic.co/t/logstash-plugin-had-an-unrecoverable-error-will-restart-this-plugin/326220 "2023-02-24T12:39:56Z")
Hello everyone, I need help I tried using the Microsoft-sentinel plugin output logstash. but I get an error: A plugin had an unrecoverable error. Will restart this plugin Error: address already in use I used two outp… --- ## [Historic tomcat access logs transform historic timestamp to @timestamp](https://discuss.elastic.co/t/historic-tomcat-access-logs-transform-historic-timestamp-to-timestamp/325862)
**Author:** [@flomickl](https://discuss.elastic.co/u/flomickl)\ **Replies:** 6\ **Last updated:** [February 21, 2023, 11:57pm UTC](https://discuss.elastic.co/t/historic-tomcat-access-logs-transform-historic-timestamp-to-timestamp/325862 "2023-02-21T23:57:14Z")
Hi, I have some historic tomcat log files like 172.x.x.xx - - \[19/Dec/2022:23:59:58 +0100\] "POST /url/text/json HTTP/1.1" 200 348 I have already a Logstash grok pattern but I have a problem with the correct timestamp. … --- ## [Tag events based in words in different fields](https://discuss.elastic.co/t/tag-events-based-in-words-in-different-fields/326382)
**Author:** [@xalmer](https://discuss.elastic.co/u/xalmer)\ **Replies:** 0\ **Last updated:** [February 23, 2023, 11:35pm UTC](https://discuss.elastic.co/t/tag-events-based-in-words-in-different-fields/326382 "2023-02-23T23:35:04Z")
Hello everybody, I want to make a better code, but i try a lot of thing and nothing works. I need to tag the input based in many words in 4 different fields. Im doing like this, but need to replicate all the filter fo… --- ## [Logstash is taking high cpu](https://discuss.elastic.co/t/logstash-is-taking-high-cpu/326316)
**Author:** [@divya.m](https://discuss.elastic.co/u/divya.m)\ **Replies:** 1\ **Last updated:** [February 23, 2023, 2:00pm UTC](https://discuss.elastic.co/t/logstash-is-taking-high-cpu/326316 "2023-02-23T14:00:24Z")
Without any load consistently logstash is using 46% of CPU, after performance load testing logstash cpu is high root@::~ $ docker logs XXXXX | grep -i "2023-02-23 10:57" | wc 86 1238 18546 --- ## [Problems with cloudwatch input plugin - namespace AWS/EC2 NameError](https://discuss.elastic.co/t/problems-with-cloudwatch-input-plugin-namespace-aws-ec2-nameerror/326297)
**Author:** [@wodnd6646](https://discuss.elastic.co/u/wodnd6646)\ **Replies:** 0\ **Last updated:** [February 23, 2023, 10:27am UTC](https://discuss.elastic.co/t/problems-with-cloudwatch-input-plugin-namespace-aws-ec2-nameerror/326297 "2023-02-23T10:27:27Z")
Hello All, problem summary: logstash can't recognize 'AWS/EC2' I am setting up a logstash configuration file to get cloudwatch(EC2, RDS) data. I have written input plugin code as below, and RDS is working as I expecte… --- ## [Transform a log fields integer value received with snmp](https://discuss.elastic.co/t/transform-a-log-fields-integer-value-received-with-snmp/326292)
**Author:** [@Christer\_Palmen](https://discuss.elastic.co/u/Christer_Palmen)\ **Replies:** 0\ **Last updated:** [February 23, 2023, 9:52am UTC](https://discuss.elastic.co/t/transform-a-log-fields-integer-value-received-with-snmp/326292 "2023-02-23T09:52:56Z")
Blockquote Hello. I am setting up a logstash pipeline to monitor the environment of my customers server cabinets with the snmp plugin. Everything looks good except for the value of the temperature, which is shown in… --- ## [How to get current/existing pipelines from ELK](https://discuss.elastic.co/t/how-to-get-current-existing-pipelines-from-elk/326217)
**Author:** [@tymercer](https://discuss.elastic.co/u/tymercer)\ **Replies:** 4\ **Last updated:** [February 22, 2023, 10:51pm UTC](https://discuss.elastic.co/t/how-to-get-current-existing-pipelines-from-elk/326217 "2023-02-22T22:51:50Z")
I just became the owner of a very old ELK cluster and need to get all of the configs pulled from it to migrate to a cloud hosted instance. The system shows there are several pipelines in Kibana Management, Logstash, Pip… --- ## [Config Map condition based on the log event on child element](https://discuss.elastic.co/t/config-map-condition-based-on-the-log-event-on-child-element/326213)
**Author:** [@rravitech](https://discuss.elastic.co/u/rravitech)\ **Replies:** 4\ **Last updated:** [February 22, 2023, 10:09pm UTC](https://discuss.elastic.co/t/config-map-condition-based-on-the-log-event-on-child-element/326213 "2023-02-22T22:09:24Z")
Here is what i am trying to achieve. Below is my log event { "version" : "1.0.0", "message" : "noisemaker draftsmanship's soundproofing grads werewolf's", "@version" : "1", "logplane"… --- ## [Logstash logs filling up disk. How to configure log4j?](https://discuss.elastic.co/t/logstash-logs-filling-up-disk-how-to-configure-log4j/326207)
**Author:** [@Thijsvdp](https://discuss.elastic.co/u/Thijsvdp)\ **Replies:** 0\ **Last updated:** [February 22, 2023, 5:29pm UTC](https://discuss.elastic.co/t/logstash-logs-filling-up-disk-how-to-configure-log4j/326207 "2023-02-22T17:29:13Z")
Hi all, We are currently facing an issue where Logstash fills up disk space on some of the nodes on our K8s cluster by outputting entire events to stdout. I am aware that I can change the loglevel of Logstash as a whole… --- ## [Error Events with "\>"](https://discuss.elastic.co/t/error-events-with/326192)
**Author:** [@akrog79](https://discuss.elastic.co/u/akrog79)\ **Replies:** 1\ **Last updated:** [February 22, 2023, 3:43pm UTC](https://discuss.elastic.co/t/error-events-with/326192 "2023-02-22T15:43:21Z")
Hello people! I have a trouble with the ingestion of a anomaly events in fortigate. The raw event is: \<185\>logver=702032456 timestamp=1676305141 devname="FG200-E" devid="FG200ETK189243" vd="root" date=2023-02-13 time=… --- ## [Question about KEMP LoadManager](https://discuss.elastic.co/t/question-about-kemp-loadmanager/326188)
**Author:** [@MKirby](https://discuss.elastic.co/u/MKirby)\ **Replies:** 1\ **Last updated:** [February 22, 2023, 3:02pm UTC](https://discuss.elastic.co/t/question-about-kemp-loadmanager/326188 "2023-02-22T15:02:27Z")
In my home lab i am testing the collection of syslog from a Kemp LoadManager. Has anyone every worked with this product to ingest or have the syslogs captured? I was able to find the folowing page and have gone throug… --- ## [Rabbitmq output plugin with 'x-delayed-message' exchange](https://discuss.elastic.co/t/rabbitmq-output-plugin-with-x-delayed-message-exchange/326182)
**Author:** [@yilmazbuhar](https://discuss.elastic.co/u/yilmazbuhar)\ **Replies:** 0\ **Last updated:** [February 22, 2023, 1:04pm UTC](https://discuss.elastic.co/t/rabbitmq-output-plugin-with-x-delayed-message-exchange/326182 "2023-02-22T13:04:07Z")
Hi all, Can i send a message to "x-delayed-message" exchange with logstash. When i try this, getting error like this output { rabbitmq { # This setting must be a \["fanout", "direct", "topic", "x-consistent-… --- ## [Logstash setup on Azure kubernetes services](https://discuss.elastic.co/t/logstash-setup-on-azure-kubernetes-services/326159)
**Author:** [@vijay78](https://discuss.elastic.co/u/vijay78)\ **Replies:** 0\ **Last updated:** [February 22, 2023, 10:38am UTC](https://discuss.elastic.co/t/logstash-setup-on-azure-kubernetes-services/326159 "2023-02-22T10:38:13Z")
i am running Elasticsearch and kibana as container on azure kubernetes services iam able to run as expected both Elasticsearch and kibana and load some sample logs from a file Now iam trying to load kubernetes sample lo… --- ## [Kafka increasing consumer lag](https://discuss.elastic.co/t/kafka-increasing-consumer-lag/326125)
**Author:** [@Sophia](https://discuss.elastic.co/u/Sophia)\ **Replies:** 0\ **Last updated:** [February 22, 2023, 5:53am UTC](https://discuss.elastic.co/t/kafka-increasing-consumer-lag/326125 "2023-02-22T05:53:12Z")
Hello! I have ELK stack installed with Kafka on Docker containers. There is lag between incoming logs and consuming logs that is increasing, and offset reading speed is not enough. Kafka have only one topic. I tried dif… --- ## [Trying to bring up filebeat + logstash + Elasticsea + Kibana](https://discuss.elastic.co/t/trying-to-bring-up-filebeat-logstash-elasticsea-kibana/326095)
**Author:** [@vassiliy.vins](https://discuss.elastic.co/u/vassiliy.vins)\ **Replies:** 3\ **Last updated:** [February 22, 2023, 1:05am UTC](https://discuss.elastic.co/t/trying-to-bring-up-filebeat-logstash-elasticsea-kibana/326095 "2023-02-22T01:05:58Z")
Hi! Followed Elasticsearch docs while installing elasticsearch + kibana + logstash + filebeat default set up. For the moment filebeat configured to send events using logstash-tutorial.log.gz (extracted to logstash-tu… --- ## [No logstash output on windows](https://discuss.elastic.co/t/no-logstash-output-on-windows/326077)
**Author:** [@jeanette](https://discuss.elastic.co/u/jeanette)\ **Replies:** 6\ **Last updated:** [February 21, 2023, 11:50pm UTC](https://discuss.elastic.co/t/no-logstash-output-on-windows/326077 "2023-02-21T23:50:09Z")
Hello, I have been troubleshooting my logstash for some time now. I was following the "Parsing Logs with Logstash" tutorial for Windows and have not been able to see any output with the basic pipeline. Below is my first-… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=87) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=89)