# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=186 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 187 --- ## [Inputting XML from File (Windows)](https://discuss.elastic.co/t/inputting-xml-from-file-windows/287513)
**Author:** [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\ **Replies:** 3\ **Last updated:** [October 24, 2021, 6:39pm UTC](https://discuss.elastic.co/t/inputting-xml-from-file-windows/287513 "2021-10-24T18:39:33Z")
I am attempting to use the File input plugin to ingest XML into Logstash running on a Windows host. At this point, I just want to verify that the input section of the pipeline is working. Here is the input section of my … --- ## [Can't replace character with Backslash with gsub](https://discuss.elastic.co/t/cant-replace-character-with-backslash-with-gsub/287501)
**Author:** [@Bryan\_Hamilton](https://discuss.elastic.co/u/Bryan_Hamilton)\ **Replies:** 2\ **Last updated:** [October 24, 2021, 10:35am UTC](https://discuss.elastic.co/t/cant-replace-character-with-backslash-with-gsub/287501 "2021-10-24T10:35:09Z")
Hi all. I am trying to replace the pound character in a field value with the backslash but am truggling to get it to work. The field looks like this: PoundedUNC =\> "##Network#Share#Name$" and I would like to turn it int… --- ## [After using pipeline to pipeline comunication in pipelines.yml, the other pipelines are not working](https://discuss.elastic.co/t/after-using-pipeline-to-pipeline-comunication-in-pipelines-yml-the-other-pipelines-are-not-working/287455)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 6\ **Last updated:** [October 23, 2021, 10:02pm UTC](https://discuss.elastic.co/t/after-using-pipeline-to-pipeline-comunication-in-pipelines-yml-the-other-pipelines-are-not-working/287455 "2021-10-23T22:02:56Z")
Hi, when I use pipeline to pipeline comunication in pipelines.yml the other pipelines in the file stop working. if I comment those lines the other pipelines start to work. this is my pipelines.yml, the first pipeline … --- ## [Logstash pass through Index Templates](https://discuss.elastic.co/t/logstash-pass-through-index-templates/287487)
**Author:** [@gose](https://discuss.elastic.co/u/gose)\ **Replies:** 0\ **Last updated:** [October 23, 2021, 12:21pm UTC](https://discuss.elastic.co/t/logstash-pass-through-index-templates/287487 "2021-10-23T12:21:08Z")
I send all my data through Logstash. Each time there's a new Elastic release, I have to export the Index Templates for each Beat I'm using (Filebeat, Metricbeat, Heartbeat, & APM Server) and then hand-load them into Ela… --- ## [Uri is not valid, Host is not specified](https://discuss.elastic.co/t/uri-is-not-valid-host-is-not-specified/287385)
**Author:** [@DeepakStile](https://discuss.elastic.co/u/DeepakStile)\ **Replies:** 1\ **Last updated:** [October 22, 2021, 3:44pm UTC](https://discuss.elastic.co/t/uri-is-not-valid-host-is-not-specified/287385 "2021-10-22T15:44:46Z")
Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:XXXXX, :exception=\>"Java::JavaLang::IllegalStateException", :message=\>"Unable to configure plugins: (ArgumentError) URI is not valid - host … --- ## [Pass access token received from http\_poller call to http filter](https://discuss.elastic.co/t/pass-access-token-received-from-http-poller-call-to-http-filter/287225)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 2\ **Last updated:** [October 22, 2021, 1:25pm UTC](https://discuss.elastic.co/t/pass-access-token-received-from-http-poller-call-to-http-filter/287225 "2021-10-22T13:25:48Z")
I am able to successfully get an access token from the http\_poller plugin but now I need to pass that token on to the http filter in order to make my REST call. I'm assuming that I need to do this through the headers fi… --- ## [How to parase xml in logstash](https://discuss.elastic.co/t/how-to-parase-xml-in-logstash/285214)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 16\ **Last updated:** [October 14, 2021, 4:33pm UTC](https://discuss.elastic.co/t/how-to-parase-xml-in-logstash/285214 "2021-10-14T16:33:08Z")
Hi I'm wondering how can I can parse below structure of XML in logstash as XML in the single event. Maybe in that case would be use fluentd.??? This logs will be upload by filebeat. here it is worth mentioning that the… --- ## [How to add field "id"](https://discuss.elastic.co/t/how-to-add-field-id/287336)
**Author:** [@khouloud](https://discuss.elastic.co/u/khouloud)\ **Replies:** 2\ **Last updated:** [October 22, 2021, 10:29am UTC](https://discuss.elastic.co/t/how-to-add-field-id/287336 "2021-10-22T10:29:16Z")
Hello, I wanna add a unique id identifier per file log in logstash. Thank you --- ## [Unable to connect to elasticsearch when Xpack is configured](https://discuss.elastic.co/t/unable-to-connect-to-elasticsearch-when-xpack-is-configured/287279)
**Author:** [@Kadhem](https://discuss.elastic.co/u/Kadhem)\ **Replies:** 3\ **Last updated:** [October 22, 2021, 8:34am UTC](https://discuss.elastic.co/t/unable-to-connect-to-elasticsearch-when-xpack-is-configured/287279 "2021-10-22T08:34:25Z")
Hi, i've configured Xpack to get users and roles. it's all working fine but when i want to run logstash file.conf to push data i get an error saying: \[2021-10-21T09:34:55,359\]\[WARN \]\[logstash.outputs.elasticsearch\]\[main… --- ## [Save all message event to array in ruby filter](https://discuss.elastic.co/t/save-all-message-event-to-array-in-ruby-filter/287381)
**Author:** [@Hertz\_Drive](https://discuss.elastic.co/u/Hertz_Drive)\ **Replies:** 0\ **Last updated:** [October 22, 2021, 3:51am UTC](https://discuss.elastic.co/t/save-all-message-event-to-array-in-ruby-filter/287381 "2021-10-22T03:51:44Z")
I need save event to array. print (event) %{host} %{message} I need print message in event. Thank you. --- ## [Add a new field with a specifically formatted timestamp from @timestamp](https://discuss.elastic.co/t/add-a-new-field-with-a-specifically-formatted-timestamp-from-timestamp/287373)
**Author:** [@trwillis](https://discuss.elastic.co/u/trwillis)\ **Replies:** 1\ **Last updated:** [October 22, 2021, 12:25am UTC](https://discuss.elastic.co/t/add-a-new-field-with-a-specifically-formatted-timestamp-from-timestamp/287373 "2021-10-22T00:25:17Z")
I want to take the value from @timestamp, format it like "yyyy MM dd HH:mm:ss" and prepend it to the message field. Input data is syslog in json format. --- ## [RSpec: Testing Aggregate Filters](https://discuss.elastic.co/t/rspec-testing-aggregate-filters/287345)
**Author:** [@UXabre](https://discuss.elastic.co/u/UXabre)\ **Replies:** 0\ **Last updated:** [October 21, 2021, 6:02pm UTC](https://discuss.elastic.co/t/rspec-testing-aggregate-filters/287345 "2021-10-21T18:02:56Z")
I'm trying to unittest a filter that utilizes aggregate filter. In my sample, I'm dropping all messages and wait for the timeout to occur instead. I'd like to retrieve this async message somehow and verify if the conten… --- ## [Jdbc\_streaming and mutate](https://discuss.elastic.co/t/jdbc-streaming-and-mutate/287319)
**Author:** [@Roberto\_B](https://discuss.elastic.co/u/Roberto_B)\ **Replies:** 4\ **Last updated:** [October 21, 2021, 4:21pm UTC](https://discuss.elastic.co/t/jdbc-streaming-and-mutate/287319 "2021-10-21T16:21:24Z")
Hi Friends, I use a jdbc\_streaming filter in order to enrich data. This filter produces an array with only 1 document inside. How can I pop out this document in another field? I tried without success with ruby code. J… --- ## [SHA256 or MD5 in Logstash cipher filter plugin algorithm](https://discuss.elastic.co/t/sha256-or-md5-in-logstash-cipher-filter-plugin-algorithm/287328)
**Author:** [@Askia\_Mohamed\_Kadri](https://discuss.elastic.co/u/Askia_Mohamed_Kadri)\ **Replies:** 1\ **Last updated:** [October 21, 2021, 4:13pm UTC](https://discuss.elastic.co/t/sha256-or-md5-in-logstash-cipher-filter-plugin-algorithm/287328 "2021-10-21T16:13:27Z")
Hi! I'm using cipher filter plugin in my Logstash configuration and the only example I saw until now for the algorithm is aes-256-cbc and I'd like to use md5 or sha256. How is it possible? Here is my configuration belo… --- ## [Use converted timestamp from csv like index sufix](https://discuss.elastic.co/t/use-converted-timestamp-from-csv-like-index-sufix/287308)
**Author:** [@Alexey\_Nikolaev](https://discuss.elastic.co/u/Alexey_Nikolaev)\ **Replies:** 4\ **Last updated:** [October 21, 2021, 12:13pm UTC](https://discuss.elastic.co/t/use-converted-timestamp-from-csv-like-index-sufix/287308 "2021-10-21T12:13:14Z")
Hello! I have a lot of CISCO CDR files, about 500k. Its logs from clustef of CISCO CUCM, and I want to put it to indexes with name CUCM-CDR-{mm-yyyy}. Date I want to get from field named "dateTimeDisconnect". Can you he… --- ## [Is Logstash beats input with multiline codec allowed or not?](https://discuss.elastic.co/t/is-logstash-beats-input-with-multiline-codec-allowed-or-not/287230)
**Author:** [@Rodrigo\_Jimenez](https://discuss.elastic.co/u/Rodrigo_Jimenez)\ **Replies:** 2\ **Last updated:** [October 21, 2021, 11:55am UTC](https://discuss.elastic.co/t/is-logstash-beats-input-with-multiline-codec-allowed-or-not/287230 "2021-10-21T11:55:33Z")
I know some of this might have been asked here before but Documentation and logs express differently. I want to fetch logs from AWS Cloudwatch. For that, i'm using filebeat's input. This input is not doing any kind of m… --- ## [Logstash doesn't open files](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100)
**Author:** [@mrdiogon](https://discuss.elastic.co/u/mrdiogon)\ **Replies:** 8\ **Last updated:** [October 21, 2021, 11:32am UTC](https://discuss.elastic.co/t/logstash-doesnt-open-files/287100 "2021-10-21T11:32:04Z")
Hi everyone, I tried to install filebeat on a windows server which send logs to a logstash server on docker. But no indexes are created and it seems logstash doesn't get any input from logstash. Here's my configuration … --- ## [Logstash Handling Filter Errors](https://discuss.elastic.co/t/logstash-handling-filter-errors/287309)
**Author:** [@maltewhiite](https://discuss.elastic.co/u/maltewhiite)\ **Replies:** 2\ **Last updated:** [October 21, 2021, 11:31am UTC](https://discuss.elastic.co/t/logstash-handling-filter-errors/287309 "2021-10-21T11:31:20Z")
I want to handle errors in my Logstash Filter. How do I do that? And more importantly, where is the documentation? I can see there is Documentation here for Elastic Pipeline Error Handling: Is there something similar… --- ## [Logstash-output-elasticsearch load balancing not working when one of the nodes is down](https://discuss.elastic.co/t/logstash-output-elasticsearch-load-balancing-not-working-when-one-of-the-nodes-is-down/287300)
**Author:** [@preetish\_P](https://discuss.elastic.co/u/preetish_P)\ **Replies:** 0\ **Last updated:** [October 21, 2021, 10:09am UTC](https://discuss.elastic.co/t/logstash-output-elasticsearch-load-balancing-not-working-when-one-of-the-nodes-is-down/287300 "2021-10-21T10:09:22Z")
Hi Team, Currently we are working on negative testing of Elasticsearch multi-node clustering. Out current setup is: node 1: Elasticsearch,logstash,kibana node 2: Elasticsearch node 3: Elasticsearch the logstash on n… --- ## [Webhdfs output plugin: store\_interval\_in\_secs not used](https://discuss.elastic.co/t/webhdfs-output-plugin-store-interval-in-secs-not-used/285974)
**Author:** [@dkhwangbo](https://discuss.elastic.co/u/dkhwangbo)\ **Replies:** 2\ **Last updated:** [October 21, 2021, 7:29am UTC](https://discuss.elastic.co/t/webhdfs-output-plugin-store-interval-in-secs-not-used/285974 "2021-10-21T07:29:23Z")
store\_interval\_in\_secs is introduced in this link. but cannot find this variable in any code base of logstash and webhdfs output plugin. is it correct? --- ## [Timestamp filter logstash](https://discuss.elastic.co/t/timestamp-filter-logstash/287199)
**Author:** [@Paveltest](https://discuss.elastic.co/u/Paveltest)\ **Replies:** 4\ **Last updated:** [October 21, 2021, 6:07am UTC](https://discuss.elastic.co/t/timestamp-filter-logstash/287199 "2021-10-21T06:07:10Z")
I am trying to display the time from the message log, but the load time from logstash is coming out. The message looks like this: 9.17.20.121 - - \[11/Oct/2021:00:00:24 +0300\] 0.474 0.072 "POST /api/?AppType=1&AppVersi… --- ## [Does http\_poller input plugin respond to backpressure from Logstash?](https://discuss.elastic.co/t/does-http-poller-input-plugin-respond-to-backpressure-from-logstash/287241)
**Author:** [@jhettich](https://discuss.elastic.co/u/jhettich)\ **Replies:** 0\ **Last updated:** [October 21, 2021, 2:10am UTC](https://discuss.elastic.co/t/does-http-poller-input-plugin-respond-to-backpressure-from-logstash/287241 "2021-10-21T02:10:54Z")
When using Logstash with a persistent queue, will the http\_poller input plugin pause polling for new data if the Logstash queue is blocked? --- ## [Dynamic fields name within grok match](https://discuss.elastic.co/t/dynamic-fields-name-within-grok-match/287150)
**Author:** [@paska](https://discuss.elastic.co/u/paska)\ **Replies:** 2\ **Last updated:** [October 21, 2021, 2:52am UTC](https://discuss.elastic.co/t/dynamic-fields-name-within-grok-match/287150 "2021-10-21T02:52:41Z")
Hello. I'm wanted to use dynamic fields name within grok match. But I couldn't find any information about it and about correct syntax. First of all I'm getting field, which depends on field log.file.path grok { mat… --- ## [Problem using http\_poller to get access token from AppDynamics OAUTH API](https://discuss.elastic.co/t/problem-using-http-poller-to-get-access-token-from-appdynamics-oauth-api/287006)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 1\ **Last updated:** [October 20, 2021, 7:58pm UTC](https://discuss.elastic.co/t/problem-using-http-poller-to-get-access-token-from-appdynamics-oauth-api/287006 "2021-10-20T19:58:17Z")
I am using the http\_poller input filter to get an outh access token from AppDynamics that I will then use to pass to the http filter plugin to retrieve AppDynamics REST API metric data to insert into Elasticsearch. My l… --- ## [Error while loading \`logstash-core-plugin-api.gemspec\`: load error: psych -- java.lang.RuntimeException: BUG: we can not copy embedded jar to temp directory](https://discuss.elastic.co/t/error-while-loading-logstash-core-plugin-api-gemspec-load-error-psych-java-lang-runtimeexception-bug-we-can-not-copy-embedded-jar-to-temp-directory/287232)
**Author:** [@rajdevworks](https://discuss.elastic.co/u/rajdevworks)\ **Replies:** 0\ **Last updated:** [October 20, 2021, 7:32pm UTC](https://discuss.elastic.co/t/error-while-loading-logstash-core-plugin-api-gemspec-load-error-psych-java-lang-runtimeexception-bug-we-can-not-copy-embedded-jar-to-temp-directory/287232 "2021-10-20T19:32:28Z")
When I try to ingest with logstash and if the given directory contains tar.gz files, it fails with following error. When the same logs are ingested with logstash and the given log directory doesn't contain tar.gz files,… --- ## [Logstash - unable to install amazon\_es - NoMethodError](https://discuss.elastic.co/t/logstash-unable-to-install-amazon-es-nomethoderror/287224)
**Author:** [@GaneshKannan](https://discuss.elastic.co/u/GaneshKannan)\ **Replies:** 0\ **Last updated:** [October 20, 2021, 4:23pm UTC](https://discuss.elastic.co/t/logstash-unable-to-install-amazon-es-nomethoderror/287224 "2021-10-20T16:23:37Z")
I'm trying to install amazon\_es in an AWS EC2 linux instance (x64). But getting an error called Resolving mixin dependencies NoMethodError: undefined method \`error' for #\ And when I … --- ## [Check if a pipeline is running with the monitoring APIs?](https://discuss.elastic.co/t/check-if-a-pipeline-is-running-with-the-monitoring-apis/286022)
**Author:** [@nico127](https://discuss.elastic.co/u/nico127)\ **Replies:** 3\ **Last updated:** [October 20, 2021, 1:38pm UTC](https://discuss.elastic.co/t/check-if-a-pipeline-is-running-with-the-monitoring-apis/286022 "2021-10-20T13:38:11Z")
Is there a way to know if a pipeline has failed with the monitoring APIs - or any other API? My issue is that the file output plugin "freezes" when the target volume is full. Hence, I want my log rotator to check if a p… --- ## [Logstash GELF input Json ParserError](https://discuss.elastic.co/t/logstash-gelf-input-json-parsererror/287159)
**Author:** [@Stian\_Vale](https://discuss.elastic.co/u/Stian_Vale)\ **Replies:** 2\ **Last updated:** [October 20, 2021, 1:33pm UTC](https://discuss.elastic.co/t/logstash-gelf-input-json-parsererror/287159 "2021-10-20T13:33:51Z")
Hi! I'm experiencing an issue with the Logstash GELF input related to JSON parsing. It seems that there's some messages from Graylog that causes these errors: \[2021-10-20T05:21:59,443\]\[ERROR\]\[logstash.inputs.gelf \]… --- ## [Find source of events causing Logstash errors](https://discuss.elastic.co/t/find-source-of-events-causing-logstash-errors/287173)
**Author:** [@Napsty](https://discuss.elastic.co/u/Napsty)\ **Replies:** 0\ **Last updated:** [October 20, 2021, 9:24am UTC](https://discuss.elastic.co/t/find-source-of-events-causing-logstash-errors/287173 "2021-10-20T09:24:23Z")
A central Logstash receives events from many clients, all using Filebeat as "sender" application. Since a couple of weeks, Logstash loggs a lot of JSON parsing errors, for example: Oct 20 11:17:46 inf-elkloge01-p logsta… --- ## [Grok parsing timestamp with 2 fields](https://discuss.elastic.co/t/grok-parsing-timestamp-with-2-fields/286312)
**Author:** [@alon\_carmelly](https://discuss.elastic.co/u/alon_carmelly)\ **Replies:** 8\ **Last updated:** [October 20, 2021, 8:36am UTC](https://discuss.elastic.co/t/grok-parsing-timestamp-with-2-fields/286312 "2021-10-20T08:36:11Z")
Hi, I got this log which has 2 fields of time stamp. How would I go about parsing it, couldn't find any examples online ! {"type": "GreatLog", \*\*"date": "10/3/2021", "time": "6:21:35 AM"\*\*, "message": "Take a Measureme… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=185) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=187)