# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=26 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 27 --- ## [\[Resolved\] LS pipeline.id not being honoured in logstash.yml](https://discuss.elastic.co/t/resolved-ls-pipeline-id-not-being-honoured-in-logstash-yml/362760)
**Author:** [@blueren](https://discuss.elastic.co/u/blueren)\ **Replies:** 9\ **Last updated:** [July 9, 2024, 3:38pm UTC](https://discuss.elastic.co/t/resolved-ls-pipeline-id-not-being-honoured-in-logstash-yml/362760 "2024-07-09T15:38:32Z")
I have a logstash instance running as a docker continer. Since it is using only a single pipeline, I've not bothered configuring the pipeline.yml. Instead, I've defined the pipeline params in the logstash.yml file itself… --- ## [Need help in appending SevOne logs to logstash](https://discuss.elastic.co/t/need-help-in-appending-sevone-logs-to-logstash/362787)
**Author:** [@keerthi12](https://discuss.elastic.co/u/keerthi12)\ **Replies:** 0\ **Last updated:** [July 9, 2024, 9:57am UTC](https://discuss.elastic.co/t/need-help-in-appending-sevone-logs-to-logstash/362787 "2024-07-09T09:57:01Z")
Hi Team We have installed ELK on premises, and we were trying to append Sevone(An IBM network performance monitoring tool) logs to logstash, does anyone have done the same, we need procedure to append sevone logs to Log… --- ## [Plugin in the list but I can't use in the pipeline \[ERROR\]](https://discuss.elastic.co/t/plugin-in-the-list-but-i-cant-use-in-the-pipeline-error/362683)
**Author:** [@Leo23](https://discuss.elastic.co/u/Leo23)\ **Replies:** 0\ **Last updated:** [July 8, 2024, 9:06am UTC](https://discuss.elastic.co/t/plugin-in-the-list-but-i-cant-use-in-the-pipeline-error/362683 "2024-07-08T09:06:03Z")
Hello, I have installed a filter-plugin that I have created in a machine A, I have pack this plugin in a zip file to install it on a machine B that don't have access to Internet. When I install thix plugin with zip I don… --- ## [Convert fields from text into date format using in Logstash config file](https://discuss.elastic.co/t/convert-fields-from-text-into-date-format-using-in-logstash-config-file/362621)
**Author:** [@Manoilayans](https://discuss.elastic.co/u/Manoilayans)\ **Replies:** 2\ **Last updated:** [July 8, 2024, 10:09am UTC](https://discuss.elastic.co/t/convert-fields-from-text-into-date-format-using-in-logstash-config-file/362621 "2024-07-08T10:09:07Z")
Logstash conf file: filter { json { source =\> "message" } } output { elasticsearch { hosts =\>"http://xx.xx.xx.xx.com" index =\> "sample\_data\_%{+YYYY.MM.dd}" user =\> "elastic" password =\> "xxxxxxxxx" } } Inde… --- ## [MIB to YAML Conversion issue](https://discuss.elastic.co/t/mib-to-yaml-conversion-issue/362652)
**Author:** [@vijjigani](https://discuss.elastic.co/u/vijjigani)\ **Replies:** 5\ **Last updated:** [July 7, 2024, 2:57pm UTC](https://discuss.elastic.co/t/mib-to-yaml-conversion-issue/362652 "2024-07-07T14:57:10Z")
Hello Experts, When i am converting the MIB to YAML files , i am getting the below error,Can you please help me on this . --- ## [Export elastic logs to an external syslog](https://discuss.elastic.co/t/export-elastic-logs-to-an-external-syslog/362629)
**Author:** [@Gabriel\_Cunha](https://discuss.elastic.co/u/Gabriel_Cunha)\ **Replies:** 4\ **Last updated:** [July 5, 2024, 6:35pm UTC](https://discuss.elastic.co/t/export-elastic-logs-to-an-external-syslog/362629 "2024-07-05T18:35:54Z")
Hello, I have a setup with elasticsearch + kibana + elasticagent and logstash. I am ingesting CloudFlare logs and I need to sent this logs through syslog for another internal host I have. I have installed the logstash … --- ## [Get Max Aggregate value for top N hits from elasticsearch](https://discuss.elastic.co/t/get-max-aggregate-value-for-top-n-hits-from-elasticsearch/362485)
**Author:** [@Het\_Test](https://discuss.elastic.co/u/Het_Test)\ **Replies:** 3\ **Last updated:** [July 5, 2024, 5:41pm UTC](https://discuss.elastic.co/t/get-max-aggregate-value-for-top-n-hits-from-elasticsearch/362485 "2024-07-05T17:41:12Z")
Hey guys, I have an Elasticsearch field called activity\_time, and I need to get the maximum activity\_time from a subset of my data. For Ex, I want to get the max value of activity\_time from the first 100 records when s… --- ## [Malformed csv error while reading csv file with Logstash](https://discuss.elastic.co/t/malformed-csv-error-while-reading-csv-file-with-logstash/362601)
**Author:** [@Rick\_V](https://discuss.elastic.co/u/Rick_V)\ **Replies:** 1\ **Last updated:** [July 5, 2024, 12:37pm UTC](https://discuss.elastic.co/t/malformed-csv-error-while-reading-csv-file-with-logstash/362601 "2024-07-05T12:37:41Z")
Hi all, I try to read a .csv file using logstash, it works like a charm but for just a few lines I get the error "(MalformedCSVError) Illegal quoting in line 1." The error occurs whenever a field contains a value with … --- ## [Logstash Http-input-plugin not closed properly FD process](https://discuss.elastic.co/t/logstash-http-input-plugin-not-closed-properly-fd-process/362511)
**Author:** [@pradeep-logstashuser](https://discuss.elastic.co/u/pradeep-logstashuser)\ **Replies:** 4\ **Last updated:** [July 5, 2024, 3:51am UTC](https://discuss.elastic.co/t/logstash-http-input-plugin-not-closed-properly-fd-process/362511 "2024-07-05T03:51:59Z")
Hi, I am using Logstash 8.10.0 with the HTTP input plugin and having 2 outputs, one using the HTTP output plugin and the other using the file output. The file rotation will occur every 1 hour, and my system-level ulimit… --- ## [Duplicate Fields in JSON Response payload when querying Kibana Dashboard Using Discover Tab](https://discuss.elastic.co/t/duplicate-fields-in-json-response-payload-when-querying-kibana-dashboard-using-discover-tab/362571)
**Author:** [@kaushalshriyan](https://discuss.elastic.co/u/kaushalshriyan)\ **Replies:** 3\ **Last updated:** [July 4, 2024, 7:34pm UTC](https://discuss.elastic.co/t/duplicate-fields-in-json-response-payload-when-querying-kibana-dashboard-using-discover-tab/362571 "2024-07-04T19:34:18Z")
Hi, I am running the latest version of Elastic Stack Elasticsearch version 8.14.2 | Elasticsearch Guide \[8.14\] | Elastic on Red Hat Enterprise Linux OS version 8.10 (8.10 Release Notes | Red Hat Product Documentation). … --- ## [Logstash snmptrap](https://discuss.elastic.co/t/logstash-snmptrap/361403)
**Author:** [@Viktor\_Movita](https://discuss.elastic.co/u/Viktor_Movita)\ **Replies:** 13\ **Last updated:** [July 4, 2024, 1:07pm UTC](https://discuss.elastic.co/t/logstash-snmptrap/361403 "2024-07-04T13:07:03Z")
Hello everyone, I enabled receiving SNMP traps as input on my Logstash server with the following addition to the configuration: input { snmptrap { id =\> "snmptrap" } } I am sending an SNMP trap to the server. … --- ## [Filtering and Processing XML Logs in Logstash for JSON Conversion](https://discuss.elastic.co/t/filtering-and-processing-xml-logs-in-logstash-for-json-conversion/362515)
**Author:** [@ayushyadav685](https://discuss.elastic.co/u/ayushyadav685)\ **Replies:** 1\ **Last updated:** [July 4, 2024, 11:43am UTC](https://discuss.elastic.co/t/filtering-and-processing-xml-logs-in-logstash-for-json-conversion/362515 "2024-07-04T11:43:19Z")
Hi All, I want to use only a few parameterized XML tags to convert into JSON and remove all other parameterized tags before processing. My XML log looks like this: \ \ \ **Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 8\ **Last updated:** [July 4, 2024, 5:59am UTC](https://discuss.elastic.co/t/grokparsefailure/362450 "2024-07-04T05:59:35Z") Hi there, i'm trying to use grok pattern here and as you can see my pattern is already match the log but when I see in kibana, idk why the tags always show \_grokparsefailure this is my code in pipeline if \[kuberne… --- ## [\_geoip\_lookup\_failure](https://discuss.elastic.co/t/geoip-lookup-failure/362491)
**Author:** [@juancamiloll](https://discuss.elastic.co/u/juancamiloll)\ **Replies:** 3\ **Last updated:** [July 3, 2024, 10:07pm UTC](https://discuss.elastic.co/t/geoip-lookup-failure/362491 "2024-07-03T22:07:15Z")
Hello, I am trying to use geoip but I get the error message "\_geoip\_lookup\_failure". I have looked at several forum threads, I don't know if I am missing something or if I need to install something. input { tcp { … --- ## [XML parsing is failing with REXML::ParseException: Missing end tag](https://discuss.elastic.co/t/xml-parsing-is-failing-with-rexml-missing-end-tag/362462)
**Author:** [@ayushyadav685](https://discuss.elastic.co/u/ayushyadav685)\ **Replies:** 3\ **Last updated:** [July 3, 2024, 4:16pm UTC](https://discuss.elastic.co/t/xml-parsing-is-failing-with-rexml-missing-end-tag/362462 "2024-07-03T16:16:18Z")
Hi All, I am trying to migrate the XML logs to JSON with Logstash, but the parsing fails due to some unnecessary XML logs. The \ tag is not needed and I attempted to remove it using remove\_tag, but it is not w… --- ## [Adding a field to a metrics event](https://discuss.elastic.co/t/adding-a-field-to-a-metrics-event/362203)
**Author:** [@Jose\_E](https://discuss.elastic.co/u/Jose_E)\ **Replies:** 5\ **Last updated:** [July 3, 2024, 10:27am UTC](https://discuss.elastic.co/t/adding-a-field-to-a-metrics-event/362203 "2024-07-03T10:27:26Z")
Hi there, I am having a big issue with a really simple situation. I am trying to add a custom field to the event generated by the metrics filter plugin, however, it seems that the metrics plugin first creates the event a… --- ## [Logstash HTTP API port throwing errors when metricbeat is run](https://discuss.elastic.co/t/logstash-http-api-port-throwing-errors-when-metricbeat-is-run/362378)
**Author:** [@akasegaonkar](https://discuss.elastic.co/u/akasegaonkar)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 5:31pm UTC](https://discuss.elastic.co/t/logstash-http-api-port-throwing-errors-when-metricbeat-is-run/362378 "2024-07-02T17:31:01Z")
Hi! I hope this is the right forum to ask this question. I have a metricbeat -\> logstash -\> elasticsearch pipeline. My logstash setup consists of multiple pipelines. When I run metricbeat (when logstash/elasticsearch a… --- ## [Logstash HTTP output NDJSON with json\_batch](https://discuss.elastic.co/t/logstash-http-output-ndjson-with-json-batch/359779)
**Author:** [@gunlomboy](https://discuss.elastic.co/u/gunlomboy)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 1:14pm UTC](https://discuss.elastic.co/t/logstash-http-output-ndjson-with-json-batch/359779 "2024-07-02T13:14:44Z")
Hi, I am trying to output batched newline delimited JSON using the logstash http output. Can the json\_batch format be used together with the json\_lines codec to achieve this? A similar query was posted here, but nobod… --- ## [java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 80 During pipeline creation](https://discuss.elastic.co/t/java-lang-arrayindexoutofboundsexception-index-1-out-of-bounds-for-length-80-during-pipeline-creation/362379)
**Author:** [@sasikiranvaddi](https://discuss.elastic.co/u/sasikiranvaddi)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 12:55pm UTC](https://discuss.elastic.co/t/java-lang-arrayindexoutofboundsexception-index-1-out-of-bounds-for-length-80-during-pipeline-creation/362379 "2024-07-02T12:55:49Z")
We observe at times ArrayIndexOutofBoundsException during creating of pipeline. Here we use log4j2 pattern as PatternLayout for processing the logs. From the logs it is unclear, is it something with log event not match… --- ## [Logstash Ruby filter not saving when fields change](https://discuss.elastic.co/t/logstash-ruby-filter-not-saving-when-fields-change/362167)
**Author:** [@emas](https://discuss.elastic.co/u/emas)\ **Replies:** 3\ **Last updated:** [July 2, 2024, 11:55am UTC](https://discuss.elastic.co/t/logstash-ruby-filter-not-saving-when-fields-change/362167 "2024-07-02T11:55:58Z")
I am currently trying to parse some XML with the XML plugin, and then getting the attributes with a Ruby script. When it works, the XML I am parsing looks like this \ \ **Author:** [@Viktor\_Movita](https://discuss.elastic.co/u/Viktor_Movita)\ **Replies:** 7\ **Last updated:** [July 2, 2024, 9:59am UTC](https://discuss.elastic.co/t/logstash-snmp-plugin/360825 "2024-07-02T09:59:15Z") Hello everyone, Can I install a plugin on my Logstash server to receive SNMP messages when my Logstash server is completely disconnected from the internet? --- ## [Logstash grok divides the log message into segments](https://discuss.elastic.co/t/logstash-grok-divides-the-log-message-into-segments/362170)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 1\ **Last updated:** [June 27, 2024, 5:32pm UTC](https://discuss.elastic.co/t/logstash-grok-divides-the-log-message-into-segments/362170 "2024-06-27T17:32:24Z")
The message is chopped from the log file and grok fails for few of the events. The below is chopped one al:9092/alp.incident.raw/group0/0\\\\\\",\\\\\\"cen://kafka-china-qa-1a-0.kafka-china-qa-1a-headless.china-qa-posdatabus… --- ## [Logstash skips string in the beginning of the event and ends up with json parse error](https://discuss.elastic.co/t/logstash-skips-string-in-the-beginning-of-the-event-and-ends-up-with-json-parse-error/362303)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 0\ **Last updated:** [July 1, 2024, 3:30pm UTC](https://discuss.elastic.co/t/logstash-skips-string-in-the-beginning-of-the-event-and-ends-up-with-json-parse-error/362303 "2024-07-01T15:30:02Z")
\[2024-06-27T07:38:58,131\]\[ERROR\]\[logstash.codecs.json \]\[main\]\[f606102f50879b5c431b527abd5f18d638386c850d0f511237fefcb5bd81e725\] JSON parse error, original data now in message field {:message=\>"incompatible json objec… --- ## [Logstash.yml overwritten by Logstash and deleting comments?](https://discuss.elastic.co/t/logstash-yml-overwritten-by-logstash-and-deleting-comments/362186)
**Author:** [@fti](https://discuss.elastic.co/u/fti)\ **Replies:** 0\ **Last updated:** [June 28, 2024, 6:56am UTC](https://discuss.elastic.co/t/logstash-yml-overwritten-by-logstash-and-deleting-comments/362186 "2024-06-28T06:56:13Z")
Greetings. I'm using the Logstash Docker image and I noticed something strange. I created a bind mount in Docker on the logstash.yml file so that I could easily edit the configuration. I also enabled automatic configur… --- ## [Logstash doesn’t seem to open new port for new pipeline. Filebeat error “no connection could be made because the target machine actively refused it”](https://discuss.elastic.co/t/logstash-doesn-t-seem-to-open-new-port-for-new-pipeline-filebeat-error-no-connection-could-be-made-because-the-target-machine-actively-refused-it/362140)
**Author:** [@Mhvrke](https://discuss.elastic.co/u/Mhvrke)\ **Replies:** 5\ **Last updated:** [June 27, 2024, 5:44pm UTC](https://discuss.elastic.co/t/logstash-doesn-t-seem-to-open-new-port-for-new-pipeline-filebeat-error-no-connection-could-be-made-because-the-target-machine-actively-refused-it/362140 "2024-06-27T17:44:05Z")
Hi community, I’ve been looking for some help regarding to the following scenario. The ELK in use it’s running over RKE in Red Hat. Logstash doesn’t seem to open a new port for new pipeline. Filebeat error “no connect… --- ## [Logstash in Alpine Linux](https://discuss.elastic.co/t/logstash-in-alpine-linux/362043)
**Author:** [@Madhab\_Chandra\_Pal](https://discuss.elastic.co/u/Madhab_Chandra_Pal)\ **Replies:** 3\ **Last updated:** [June 27, 2024, 1:20pm UTC](https://discuss.elastic.co/t/logstash-in-alpine-linux/362043 "2024-06-27T13:20:27Z")
Hi, Due to blocker from CVE I posted in the following topic, I am trying to build alternative Logstash using Alpine Linux. Is there any alternative release of Logstash using Alpine Linux? --- ## [Logstash fails to aggregate documents with similar timestamps](https://discuss.elastic.co/t/logstash-fails-to-aggregate-documents-with-similar-timestamps/361943)
**Author:** [@goncalobsantos](https://discuss.elastic.co/u/goncalobsantos)\ **Replies:** 3\ **Last updated:** [June 27, 2024, 11:03am UTC](https://discuss.elastic.co/t/logstash-fails-to-aggregate-documents-with-similar-timestamps/361943 "2024-06-27T11:03:43Z")
I have Logstash running on a server, receiving its input from Filebeat and sending its output to Elasticsearch. The inputs are of two types: "data arrival" and "sent data". Each input of type "data arrival" should be pai… --- ## [What is the easiest way to create indexes for different nginx logs in logstash](https://discuss.elastic.co/t/what-is-the-easiest-way-to-create-indexes-for-different-nginx-logs-in-logstash/362031)
**Author:** [@ffamous](https://discuss.elastic.co/u/ffamous)\ **Replies:** 5\ **Last updated:** [June 27, 2024, 11:03am UTC](https://discuss.elastic.co/t/what-is-the-easiest-way-to-create-indexes-for-different-nginx-logs-in-logstash/362031 "2024-06-27T11:03:30Z")
Hello. I have more than one nginx logs in folder which filebeat send to logstash and want to make index for each of them depends of the log name. My logstash.conf looks like this: input { beats { port =\> 5044 … --- ## [Can't prepare offline plugin](https://discuss.elastic.co/t/cant-prepare-offline-plugin/362063)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 7\ **Last updated:** [June 27, 2024, 7:33am UTC](https://discuss.elastic.co/t/cant-prepare-offline-plugin/362063 "2024-06-27T07:33:22Z")
hello everyone, i have a problem about logstash right now. i'm trying to create offline plugin here but when i run the command, i got this error Gem::LoadError: You have already activated ffi 1.16.3, but your Gemfile r… --- ## [Cluster Block Exception retrying failed action with response code: 403](https://discuss.elastic.co/t/cluster-block-exception-retrying-failed-action-with-response-code-403/362015)
**Author:** [@Simon\_Prinz](https://discuss.elastic.co/u/Simon_Prinz)\ **Replies:** 5\ **Last updated:** [June 26, 2024, 4:24pm UTC](https://discuss.elastic.co/t/cluster-block-exception-retrying-failed-action-with-response-code-403/362015 "2024-06-26T16:24:56Z")
Hello Dear Community, since a Couple of days out of the Blue i get more and more: retrying failed action with response code: 403 ({"type"=\>"cluster\_block\_exception", "reason"=\>"index \[Logs\] blocked by: \[FORBIDDEN/8/ind… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=25) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=27)