# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=151 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 152 --- ## [\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or co mmand line options are specified](https://discuss.elastic.co/t/logstash-config-source-multilocal-ignoring-the-pipelines-yml-file-because-modules-or-co-mmand-line-options-are-specified/300852)
**Author:** [@lug-gh](https://discuss.elastic.co/u/lug-gh)\ **Replies:** 6\ **Last updated:** [March 29, 2022, 1:23pm UTC](https://discuss.elastic.co/t/logstash-config-source-multilocal-ignoring-the-pipelines-yml-file-because-modules-or-co-mmand-line-options-are-specified/300852 "2022-03-29T13:23:34Z")
Hi, I wan't to use multiple pipelines, but after some testing I noticed the pipelines.yml won't be loaded. I searched the web and found it's ignored when you specify -f or -e (Multiple Pipelines | Logstash Reference \[8.1… --- ## [Logstash redundancy and availability in ELK cluster](https://discuss.elastic.co/t/logstash-redundancy-and-availability-in-elk-cluster/300963)
**Author:** [@ELK\_Dummy](https://discuss.elastic.co/u/ELK_Dummy)\ **Replies:** 2\ **Last updated:** [March 29, 2022, 1:15pm UTC](https://discuss.elastic.co/t/logstash-redundancy-and-availability-in-elk-cluster/300963 "2022-03-29T13:15:44Z")
Hi all, The question is about Logstash redundancy and availability. We have an Elasticsearch Cluster running fine with 3 CentOS nodes. One of this nodes hosts Losgstash. Logstash is set to ingest log files sent by SFT… --- ## [Logstash service fails to start due to Ruby Error](https://discuss.elastic.co/t/logstash-service-fails-to-start-due-to-ruby-error/300902)
**Author:** [@ASA01](https://discuss.elastic.co/u/ASA01)\ **Replies:** 2\ **Last updated:** [March 29, 2022, 12:45pm UTC](https://discuss.elastic.co/t/logstash-service-fails-to-start-due-to-ruby-error/300902 "2022-03-29T12:45:25Z")
I cannot get logstash to start and keep running. I have verified all my logstash confs work individually with test\_and\_exit, but when I try to start it as a service I get the following error. \[2022-03-29T01:37:10,010\]\[… --- ## [How to add Available Fields on my Discover menu for visualization purposes on my dashboards?](https://discuss.elastic.co/t/how-to-add-available-fields-on-my-discover-menu-for-visualization-purposes-on-my-dashboards/300954)
**Author:** [@renatoa12](https://discuss.elastic.co/u/renatoa12)\ **Replies:** 0\ **Last updated:** [March 29, 2022, 11:42am UTC](https://discuss.elastic.co/t/how-to-add-available-fields-on-my-discover-menu-for-visualization-purposes-on-my-dashboards/300954 "2022-03-29T11:42:06Z")
Hi everyone, Does anyone here know how can i add specific fields for my index for visualization purposes on my dashboards?? --- ## [Attempted to resurrect connection to dead ES instance](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance/300480)
**Author:** [@Bavaria](https://discuss.elastic.co/u/Bavaria)\ **Replies:** 1\ **Last updated:** [March 29, 2022, 9:21am UTC](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance/300480 "2022-03-29T09:21:30Z")
Am I missing something? I am trying to send in a basic tryout some logs from a Server with Filebeat, via Logstash to Elasticsearch. I am getting the data from Beats in Logstash. But Logstash is giving me following messag… --- ## [Logstash "Marking url as dead" issue and http client logging log4j2](https://discuss.elastic.co/t/logstash-marking-url-as-dead-issue-and-http-client-logging-log4j2/300937)
**Author:** [@can.ozdemir](https://discuss.elastic.co/u/can.ozdemir)\ **Replies:** 0\ **Last updated:** [March 29, 2022, 9:10am UTC](https://discuss.elastic.co/t/logstash-marking-url-as-dead-issue-and-http-client-logging-log4j2/300937 "2022-03-29T09:10:52Z")
Hello there, We are experiencing "marking url as dead" problem on our logstash machines, I know this is a known issue and there are many solutions people are discussing. I tried some of them they are not the solution t… --- ## [Form the syslog\_pri field](https://discuss.elastic.co/t/form-the-syslog-pri-field/300814)
**Author:** [@anushka1203](https://discuss.elastic.co/u/anushka1203)\ **Replies:** 2\ **Last updated:** [March 29, 2022, 3:25am UTC](https://discuss.elastic.co/t/form-the-syslog-pri-field/300814 "2022-03-29T03:25:56Z")
I have the following syslog messages - Oct 18 16:11:54 1aen ai\_controller\[419\]: \[16:11:54.479933\] ERROR \[ntp\_client.cpp:113 fetch\_time()\] Can't get NTP response Oct 18 16:11:55 1aen ntpd\[442\]: bind(21) AF\_INET6 fe80… --- ## [Missing Events in Logstash input SNMP](https://discuss.elastic.co/t/missing-events-in-logstash-input-snmp/300900)
**Author:** [@Ygal\_Mizrachi](https://discuss.elastic.co/u/Ygal_Mizrachi)\ **Replies:** 0\ **Last updated:** [March 29, 2022, 1:39am UTC](https://discuss.elastic.co/t/missing-events-in-logstash-input-snmp/300900 "2022-03-29T01:39:27Z")
Hello im trying to get snmp documents but some events are missing and i am looking for help. thanks in advance snmp2.conf input{ snmp { tables =\> \[{ "name" =\> "ifTable" "columns" =\> \[ "1.3.6.1.2.1.2.2.1.1", "1.3.… --- ## [How to parse apache error.log](https://discuss.elastic.co/t/how-to-parse-apache-error-log/300822)
**Author:** [@Roccof97](https://discuss.elastic.co/u/Roccof97)\ **Replies:** 2\ **Last updated:** [March 28, 2022, 2:48pm UTC](https://discuss.elastic.co/t/how-to-parse-apache-error-log/300822 "2022-03-28T14:48:27Z")
Hi, help! i can't parse this timestamp: \[Mon Mar 28 09:02:28.627528 2022\] \[mpm\_winnt:notice\] \[pid 0000:tid 000\] 0000000: Apache/00.00.0.0 (Win00) OpenSSL/0.0.0h configured -- resuming normal operations suggestions? T… --- ## [Logstash cannot connect to Elasticsearch on kubernetes](https://discuss.elastic.co/t/logstash-cannot-connect-to-elasticsearch-on-kubernetes/300496)
**Author:** [@skander\_khalfet](https://discuss.elastic.co/u/skander_khalfet)\ **Replies:** 3\ **Last updated:** [March 28, 2022, 1:59pm UTC](https://discuss.elastic.co/t/logstash-cannot-connect-to-elasticsearch-on-kubernetes/300496 "2022-03-28T13:59:52Z")
Hi everyone, i'm having trouble concerning Logstash. So i installed ELK stack on kubernetes using helm. every pod is is running and ready. Elasticsearch and kibana are perfectly fine. the problem is with logstash as yo… --- ## [pfSense Logs integration not working - What am I missing?](https://discuss.elastic.co/t/pfsense-logs-integration-not-working-what-am-i-missing/300824)
**Author:** [@EvertM](https://discuss.elastic.co/u/EvertM)\ **Replies:** 0\ **Last updated:** [March 28, 2022, 10:12am UTC](https://discuss.elastic.co/t/pfsense-logs-integration-not-working-what-am-i-missing/300824 "2022-03-28T10:12:34Z")
Hi all, I've added the pfSense Logs integration, but it doesn't seem to receive any data. The pfSense box is sending, and it is arriving on on the Elastic-box (verified with nc -l -u 10.10.10.1 -p 9001). Are there any … --- ## [Logstash 8.1.1 with file input plugin hangs](https://discuss.elastic.co/t/logstash-8-1-1-with-file-input-plugin-hangs/300789)
**Author:** [@geena.rollins](https://discuss.elastic.co/u/geena.rollins)\ **Replies:** 4\ **Last updated:** [March 28, 2022, 12:38am UTC](https://discuss.elastic.co/t/logstash-8-1-1-with-file-input-plugin-hangs/300789 "2022-03-28T00:38:43Z")
On Amazon Linux 2 AMI (HVM) - Kernel 5.10, I installed Java-17, Elasticsearch 8.1.1, and Logstash 8.1.1. When I use the file input plugin, logstash hangs. logstash.yml is node.name: tester file.conf is input { fi… --- ## [Can Logstash 7.x work with elastic 8.x](https://discuss.elastic.co/t/can-logstash-7-x-work-with-elastic-8-x/300512)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 1\ **Last updated:** [March 27, 2022, 11:56pm UTC](https://discuss.elastic.co/t/can-logstash-7-x-work-with-elastic-8-x/300512 "2022-03-27T23:56:26Z")
Hi all I have a question before upgrading elastic to 8.x Do i have to upgrade all logstash and filebeat node to version 8.x or it can be keep at version 7.17 Since there is a lot of logstash and filebeat node in my sy… --- ## [Logstash output to Prometheus](https://discuss.elastic.co/t/logstash-output-to-prometheus/300686)
**Author:** [@Stephy\_Jacob](https://discuss.elastic.co/u/Stephy_Jacob)\ **Replies:** 3\ **Last updated:** [March 27, 2022, 4:15pm UTC](https://discuss.elastic.co/t/logstash-output-to-prometheus/300686 "2022-03-27T16:15:15Z")
Hi, Could you please confirm whether we can send output of Logstash to Prometheus --- ## [S3 output plugin exception exception=\>#\ **Author:** [@Juda\_Barnes](https://discuss.elastic.co/u/Juda_Barnes)\ **Replies:** 0\ **Last updated:** [March 27, 2022, 7:19am UTC](https://discuss.elastic.co/t/s3-output-plugin-exception-exception-errno-no-such-file-or-directory-no-such-file-or-directory/300775 "2022-03-27T07:19:09Z") Getting the following exception and then pipeline being terminated, :error=\>"(IOError) closed stream", :exception=\>Java::OrgJrubyExceptions::IOError, :backtrace=\>\["org.jruby.RubyIO.write(org/jruby/RubyIO.java:1467)", "o… --- ## [How to convert all field names starting with string to integer](https://discuss.elastic.co/t/how-to-convert-all-field-names-starting-with-string-to-integer/300728)
**Author:** [@hagaluly](https://discuss.elastic.co/u/hagaluly)\ **Replies:** 4\ **Last updated:** [March 26, 2022, 7:40pm UTC](https://discuss.elastic.co/t/how-to-convert-all-field-names-starting-with-string-to-integer/300728 "2022-03-26T19:40:36Z")
i have tried this block without success i haven't find a proper way how to use wildcards filter { if \[fieldname\] =~ /^STAGED.\*/ { mutate { convert =\> {"\[fieldname\]" =\> "integer"} } }… --- ## [Share sql\_last\_value between two pods](https://discuss.elastic.co/t/share-sql-last-value-between-two-pods/300753)
**Author:** [@Vipul\_Agarwal](https://discuss.elastic.co/u/Vipul_Agarwal)\ **Replies:** 2\ **Last updated:** [March 26, 2022, 5:08pm UTC](https://discuss.elastic.co/t/share-sql-last-value-between-two-pods/300753 "2022-03-26T17:08:41Z")
Hi All, I am trying to run logstash in a multi-pod setup. My sample config is : input { jdbc { jdbc\_driver\_library =\> "${HOME}/postgres/postgresql-42.3.2.jar" jdbc\_driver\_class =\> "org.postgresql.Driver" … --- ## [Logstash ingesting from S3, affecting OTHER indices?](https://discuss.elastic.co/t/logstash-ingesting-from-s3-affecting-other-indices/300726)
**Author:** [@pritster5](https://discuss.elastic.co/u/pritster5)\ **Replies:** 6\ **Last updated:** [March 25, 2022, 8:36pm UTC](https://discuss.elastic.co/t/logstash-ingesting-from-s3-affecting-other-indices/300726 "2022-03-25T20:36:41Z")
I'm having a strange issue. I'm using the pipelines feature of Logstash via 2 config files that look like this: input { s3 { bucket =\> "\" region =\> "us-east-1" codec =\> "json" … --- ## [How to parse timestamp apache access.log](https://discuss.elastic.co/t/how-to-parse-timestamp-apache-access-log/300709)
**Author:** [@Roccof97](https://discuss.elastic.co/u/Roccof97)\ **Replies:** 2\ **Last updated:** [March 25, 2022, 3:39pm UTC](https://discuss.elastic.co/t/how-to-parse-timestamp-apache-access-log/300709 "2022-03-25T15:39:30Z")
Hi, how can i parse this timestamp using grok filter?: x.x.x.x - - \[24/Mar/2022:00:00:04 +0000\] do you have any advice? Thanks --- ## [Could not connect to a compatible version of Elasticsearch (elasticsearch 7.8.0 with logstash 7.17.1)](https://discuss.elastic.co/t/could-not-connect-to-a-compatible-version-of-elasticsearch-elasticsearch-7-8-0-with-logstash-7-17-1/300688)
**Author:** [@splash](https://discuss.elastic.co/u/splash)\ **Replies:** 2\ **Last updated:** [March 25, 2022, 3:35pm UTC](https://discuss.elastic.co/t/could-not-connect-to-a-compatible-version-of-elasticsearch-elasticsearch-7-8-0-with-logstash-7-17-1/300688 "2022-03-25T15:35:47Z")
Hi everyone, we are running a Robotic Process Automation software called "Automation Anywhere" which uses Elasticsearch 7.8.0: "version" : { "number" : "7.8.0", "build\_flavor" : "oss", "build\_type" : "zip",… --- ## [Concatenate / join values of tags into one field](https://discuss.elastic.co/t/concatenate-join-values-of-tags-into-one-field/300606)
**Author:** [@S-elk](https://discuss.elastic.co/u/S-elk)\ **Replies:** 2\ **Last updated:** [March 25, 2022, 10:29am UTC](https://discuss.elastic.co/t/concatenate-join-values-of-tags-into-one-field/300606 "2022-03-25T10:29:01Z")
hello! i want to join all the tags field values into one new field? can i do that with join mutate filter ? if not, how can i do that ? example : tags = \["1","2","3"\] new\_field = "123" thanks in advanced! p.d. is… --- ## [Need help: amazon\_es is installed but still getting error that the plugin is not installed](https://discuss.elastic.co/t/need-help-amazon-es-is-installed-but-still-getting-error-that-the-plugin-is-not-installed/300648)
**Author:** [@dontusk1980](https://discuss.elastic.co/u/dontusk1980)\ **Replies:** 0\ **Last updated:** [March 24, 2022, 11:53pm UTC](https://discuss.elastic.co/t/need-help-amazon-es-is-installed-but-still-getting-error-that-the-plugin-is-not-installed/300648 "2022-03-24T23:53:11Z")
Hi team, installed logstash-plugin install logstash-output-amazon\_es, but once restarted the logstash, log says not installed, detailed logs are as below \[root@ip-10-33-89-199 bin\]# ./logstash --version Using JAVA\_HOM… --- ## [Logstash xml parse error](https://discuss.elastic.co/t/logstash-xml-parse-error/300624)
**Author:** [@elwdipath](https://discuss.elastic.co/u/elwdipath)\ **Replies:** 0\ **Last updated:** [March 24, 2022, 7:14pm UTC](https://discuss.elastic.co/t/logstash-xml-parse-error/300624 "2022-03-24T19:14:35Z")
Hey all, new to logstash. I'm running into an error when trying to parse an empty xml element. failed to parse field \[xml.record.person.agedata.dob\] of type \[text\] in document with id 'DA9jvH8BhCQVGJZR4sKN'. Preview of… --- ## [Filter load monitoring](https://discuss.elastic.co/t/filter-load-monitoring/300461)
**Author:** [@Dargod](https://discuss.elastic.co/u/Dargod)\ **Replies:** 2\ **Last updated:** [March 24, 2022, 5:55pm UTC](https://discuss.elastic.co/t/filter-load-monitoring/300461 "2022-03-24T17:55:53Z")
I have a lot of filters in logstash that deal with the processing of plain text logs and json. With the linear growth of connected logs, the load increased and there was an increase in the queue. In this connection, th… --- ## [Problem using input snmp](https://discuss.elastic.co/t/problem-using-input-snmp/300611)
**Author:** [@Ygal\_Mizrachi](https://discuss.elastic.co/u/Ygal_Mizrachi)\ **Replies:** 0\ **Last updated:** [March 24, 2022, 5:16pm UTC](https://discuss.elastic.co/t/problem-using-input-snmp/300611 "2022-03-24T17:16:19Z")
Hello im using this ultra simple conf file: input { snmp { hosts =\> \[{host =\> "udp:192.168.24.1/161" community =\> "public" version =\> "2c" retries =\> 2 timeout =\> 1000}\] tables =\> \[ {"name" =\> "interfaces" "columns" =\> … --- ## [COMBINEDAPACHELOG grok definition not creating the expected fields](https://discuss.elastic.co/t/combinedapachelog-grok-definition-not-creating-the-expected-fields/300576)
**Author:** [@Patrick\_Lepage](https://discuss.elastic.co/u/Patrick_Lepage)\ **Replies:** 2\ **Last updated:** [March 24, 2022, 5:04pm UTC](https://discuss.elastic.co/t/combinedapachelog-grok-definition-not-creating-the-expected-fields/300576 "2022-03-24T17:04:46Z")
Complete beginner to logstash here... I'm following the "Parsing Logs with Logstash" tutorial. Input comes from filebeat reading a static log file provided with the tutorial; see example of a sample line below. 83.149.… --- ## [Strange bad certificate](https://discuss.elastic.co/t/strange-bad-certificate/300552)
**Author:** [@labate](https://discuss.elastic.co/u/labate)\ **Replies:** 13\ **Last updated:** [March 24, 2022, 4:55pm UTC](https://discuss.elastic.co/t/strange-bad-certificate/300552 "2022-03-24T16:55:01Z")
Dear all, I'm facing to a serious problem. I get this following message: filebeat\[29206\]: 2022-03-24T10:30:37.927+0100 ERROR \[publisher\_pipeline\_output\] pipeline/output.go:154 Failed to conn… --- ## [Xml Parsing in logstash using xml & split plugins](https://discuss.elastic.co/t/xml-parsing-in-logstash-using-xml-split-plugins/300515)
**Author:** [@manikandanid](https://discuss.elastic.co/u/manikandanid)\ **Replies:** 1\ **Last updated:** [March 24, 2022, 4:47pm UTC](https://discuss.elastic.co/t/xml-parsing-in-logstash-using-xml-split-plugins/300515 "2022-03-24T16:47:59Z")
I have a xml which looks like \\\India\\+91\\Asia\\\123129\\7890890\\\ **Author:** [@nandatibco](https://discuss.elastic.co/u/nandatibco)\ **Replies:** 0\ **Last updated:** [March 24, 2022, 2:18pm UTC](https://discuss.elastic.co/t/logstash-8-1-0-no-appropriate-protocol-protocol-is-disabled-or-cipher-suites-are-inappropriate/300582 "2022-03-24T14:18:14Z") Hello team, I'm using Logstash 8.1.0 with plugin: logstash-input-websocket, when I start Logstash getting below error: \[WARN \]\[logstash.inputs.websocket\]\[main\]\[a0cd44e8c7bfa2705bff3d19091209cb115076d1199fdf41b74beb730f… --- ## [Get sequential relation of keys in a json file in ruby logstash](https://discuss.elastic.co/t/get-sequential-relation-of-keys-in-a-json-file-in-ruby-logstash/300566)
**Author:** [@zubair\_aftab](https://discuss.elastic.co/u/zubair_aftab)\ **Replies:** 0\ **Last updated:** [March 24, 2022, 11:44am UTC](https://discuss.elastic.co/t/get-sequential-relation-of-keys-in-a-json-file-in-ruby-logstash/300566 "2022-03-24T11:44:38Z")
I have a json file of a packet capture. It has multiple layers in 1 message as shown in below example. block 1: mtp3 = 1 map = 1 block 2: mtp3 =2 bicc =2 When i process the file in logstash it combines the similar… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=150) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=152)