# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=289 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 290 --- ## [Testing Ruby For Logstash Event Object](https://discuss.elastic.co/t/testing-ruby-for-logstash-event-object/253196)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 0\ **Last updated:** [October 24, 2020, 6:35am UTC](https://discuss.elastic.co/t/testing-ruby-for-logstash-event-object/253196 "2020-10-24T06:35:12Z")
I am planning on writing some unit test for a ruby code i have written. Here is the snippet: require 'minitest/autorun' describe "Test Event Object" do before do @event = new Event() end it "sho… --- ## [Is there a K8s Container min for Logstash?](https://discuss.elastic.co/t/is-there-a-k8s-container-min-for-logstash/253195)
**Author:** [@fallenreaper](https://discuss.elastic.co/u/fallenreaper)\ **Replies:** 0\ **Last updated:** [October 24, 2020, 5:50am UTC](https://discuss.elastic.co/t/is-there-a-k8s-container-min-for-logstash/253195 "2020-10-24T05:50:16Z")
I was using k8s and was fiddling with the idea of a logstash container. Without putting any requests or limits in place though, Logstash, as well as Elasticsearch and Kibana can be a bit greedy when it comes to some res… --- ## [Multiple logstash](https://discuss.elastic.co/t/multiple-logstash/253009)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 3\ **Last updated:** [October 24, 2020, 1:32am UTC](https://discuss.elastic.co/t/multiple-logstash/253009 "2020-10-24T01:32:30Z")
I want to add 2 logstash in my application for scalability.From availablity point... I have a question does 2 logstash works simultaneously or they works as a active-passive (one stop other takes) or they automatically m… --- ## [Logstash Docker on GCP Cloud Run: Failed to find a usable hardware address from the network interfaces](https://discuss.elastic.co/t/logstash-docker-on-gcp-cloud-run-failed-to-find-a-usable-hardware-address-from-the-network-interfaces/253181)
**Author:** [@NickyW1995](https://discuss.elastic.co/u/NickyW1995)\ **Replies:** 0\ **Last updated:** [October 23, 2020, 9:41pm UTC](https://discuss.elastic.co/t/logstash-docker-on-gcp-cloud-run-failed-to-find-a-usable-hardware-address-from-the-network-interfaces/253181 "2020-10-23T21:41:00Z")
Hi all, I'm coupling PubSub with a Logstash custom image running on GCP Cloud Run, this image will output messages to an Elastic Cloud configuration. I'm currently hitting some errors on Cloud Run which I can't seem to… --- ## [Ingest data from salesforce to elasticsearch using Logstash](https://discuss.elastic.co/t/ingest-data-from-salesforce-to-elasticsearch-using-logstash/252921)
**Author:** [@SUMANT\_MISHRA](https://discuss.elastic.co/u/SUMANT_MISHRA)\ **Replies:** 6\ **Last updated:** [October 23, 2020, 7:45pm UTC](https://discuss.elastic.co/t/ingest-data-from-salesforce-to-elasticsearch-using-logstash/252921 "2020-10-23T19:45:12Z")
Hi, I am trying to ingest data from salesforce to elasticsearch using logstash in Macbook Pro (macOS Mojave Version 10.14.2). But it's not working and gives the below error in my terminal: \*\[2020-10-22T08:37:06,130\]\[IN… --- ## [Parsing underscore](https://discuss.elastic.co/t/parsing-underscore/253104)
**Author:** [@rbeg](https://discuss.elastic.co/u/rbeg)\ **Replies:** 2\ **Last updated:** [October 23, 2020, 1:58pm UTC](https://discuss.elastic.co/t/parsing-underscore/253104 "2020-10-23T13:58:48Z")
Hello, Can you help me please? I need to parse this field in logstash filter : word1\_word2\_word3\_word4 And I want to put word3 into a new field named my\_field. How to do this please? I try with grok and regular exp… --- ## [JDBC\_static lookup timeout](https://discuss.elastic.co/t/jdbc-static-lookup-timeout/253134)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 0\ **Last updated:** [October 23, 2020, 1:29pm UTC](https://discuss.elastic.co/t/jdbc-static-lookup-timeout/253134 "2020-10-23T13:29:13Z")
I'm running a static lookup to enhance my syslog data and I'm seeing this WARN in the log. Is there some configuration I can hit to help this timeout problem? \[2020-10-23T13:25:35,278\]\[WARN \]\[logstash.filters.jdbc.looku… --- ## [Logstash Error after enabling xpack security](https://discuss.elastic.co/t/logstash-error-after-enabling-xpack-security/253123)
**Author:** [@Craig2188](https://discuss.elastic.co/u/Craig2188)\ **Replies:** 3\ **Last updated:** [October 23, 2020, 12:11pm UTC](https://discuss.elastic.co/t/logstash-error-after-enabling-xpack-security/253123 "2020-10-23T12:11:25Z")
Hi, I enabled xpack security on Elastic, then set the logstash account as per the below link: Add user information in Logstash | Elasticsearch Reference \[6.8\] | Elastic I used the keystore option. However I am getting… --- ## [Logstash results coming from behind](https://discuss.elastic.co/t/logstash-results-coming-from-behind/253115)
**Author:** [@bnmtl](https://discuss.elastic.co/u/bnmtl)\ **Replies:** 0\ **Last updated:** [October 23, 2020, 10:14am UTC](https://discuss.elastic.co/t/logstash-results-coming-from-behind/253115 "2020-10-23T10:14:19Z")
Hello, I have incremental json input in my logstash configuration, and using aggregation and some mutation in my filter and output is rabbit I see that logstash outputs are coming from behind about 2-3 hour. I mean tha… --- ## [Logstash can't reach elasticsearch](https://discuss.elastic.co/t/logstash-cant-reach-elasticsearch/253101)
**Author:** [@mw\_two](https://discuss.elastic.co/u/mw_two)\ **Replies:** 0\ **Last updated:** [October 23, 2020, 8:46am UTC](https://discuss.elastic.co/t/logstash-cant-reach-elasticsearch/253101 "2020-10-23T08:46:25Z")
Hello, we've following setup: 3x elasticsearch 2x logstash 2x kibana Since two weeks the second logstash-server cannot reach the elasticsearch servers. Following errors can be found in logs: \[2020-10-22T13:17:28,84… --- ## [Does elasticsearch recommend to use any special character in user password for user](https://discuss.elastic.co/t/does-elasticsearch-recommend-to-use-any-special-character-in-user-password-for-user/252974)
**Author:** [@sreerama.naga](https://discuss.elastic.co/u/sreerama.naga)\ **Replies:** 3\ **Last updated:** [October 23, 2020, 4:23am UTC](https://discuss.elastic.co/t/does-elasticsearch-recommend-to-use-any-special-character-in-user-password-for-user/252974 "2020-10-23T04:23:59Z")
Hi, Here I am trying to get the details, does elasticsearch recommend to use any special character for user password? I tried doing some search in those lines but didn't succeed in getting any proper pages with recommen… --- ## [Logstash generating two indices with the same content](https://discuss.elastic.co/t/logstash-generating-two-indices-with-the-same-content/253072)
**Author:** [@CezarOliveira](https://discuss.elastic.co/u/CezarOliveira)\ **Replies:** 0\ **Last updated:** [October 22, 2020, 11:40pm UTC](https://discuss.elastic.co/t/logstash-generating-two-indices-with-the-same-content/253072 "2020-10-22T23:40:30Z")
I have Elasticsearch, Logstash and Kibana running on Docker. First I run Elasticsearch and Kibana and confirm that no exists index. docker-compose-elastic-kibana.ymlversion: '3' services: elasticsearch: i… --- ## [Mutate rename, remove does not work on fields in list](https://discuss.elastic.co/t/mutate-rename-remove-does-not-work-on-fields-in-list/253069)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 2\ **Last updated:** [October 22, 2020, 11:18pm UTC](https://discuss.elastic.co/t/mutate-rename-remove-does-not-work-on-fields-in-list/253069 "2020-10-22T23:18:06Z")
Hello, I want to use logstash snmp walk input for fetching data from my network devices. This is the config I use. input { snmp { hosts =\> \[ {host =\> "udp:ip/port" community =\> "secret"} … --- ## [Logic Issue....Or Maybe Gsub](https://discuss.elastic.co/t/logic-issue-or-maybe-gsub/253047)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 1\ **Last updated:** [October 22, 2020, 9:37pm UTC](https://discuss.elastic.co/t/logic-issue-or-maybe-gsub/253047 "2020-10-22T21:37:54Z")
I am capturing logs that primarily look like this: \[2020-10-21 22:12:44,067\]\[INFO\]\[audit\]{"JSON formatted stuff"} However, I will occasionally get lines that look like this: \[2020-10-21 22:12:44,067\]\[INFO\]\[audit\]text … --- ## [Pipeline To Pipeline Communication inside Rancher](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-inside-rancher/252743)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 1\ **Last updated:** [October 22, 2020, 3:14pm UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-inside-rancher/252743 "2020-10-22T15:14:29Z")
Hi, I need some guidance on setting up pipeline-to-pipeline communication inside of Rancher. What config files do I need at a minimum? I'm confused as to whether or not I need both logstash.yml and pipelines.yml configur… --- ## [Logstash output to elasticsearch - open distro for elasticsearch](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-open-distro-for-elasticsearch/252961)
**Author:** [@jafri6](https://discuss.elastic.co/u/jafri6)\ **Replies:** 2\ **Last updated:** [October 22, 2020, 2:28pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-open-distro-for-elasticsearch/252961 "2020-10-22T14:28:39Z")
I have been working on this project, and my logstash configuration file works fine. Now I am trying to implement the same thing on a system that is running opendistro/elk stack. I have tried to fix most of the errors, b… --- ## [.logstash\_jdbc\_last\_run file is read only on logstash start up](https://discuss.elastic.co/t/logstash-jdbc-last-run-file-is-read-only-on-logstash-start-up/252993)
**Author:** [@milosh3411](https://discuss.elastic.co/u/milosh3411)\ **Replies:** 3\ **Last updated:** [October 22, 2020, 2:27pm UTC](https://discuss.elastic.co/t/logstash-jdbc-last-run-file-is-read-only-on-logstash-start-up/252993 "2020-10-22T14:27:28Z")
Hi, we have 2 logstash servers with same configurations, connecting to one database server through their jdbc input plugins. The idea was to have a highly available logstash. Both LS servers are using the same .logsta… --- ## [Help me on logstash conf for Json input file](https://discuss.elastic.co/t/help-me-on-logstash-conf-for-json-input-file/253015)
**Author:** [@Uday\_kumar\_Chunduri](https://discuss.elastic.co/u/Uday_kumar_Chunduri)\ **Replies:** 0\ **Last updated:** [October 22, 2020, 2:19pm UTC](https://discuss.elastic.co/t/help-me-on-logstash-conf-for-json-input-file/253015 "2020-10-22T14:19:30Z")
Hello, I have json file adding below I am trying generate fields from the json data using logstash conf file. please suggest me how to write config create a individual fields in elastic search. input json "count": 15, … --- ## [Sending logs from Syslog-ng to Logstash with and without TLS](https://discuss.elastic.co/t/sending-logs-from-syslog-ng-to-logstash-with-and-without-tls/252939)
**Author:** [@Vishnuprasad](https://discuss.elastic.co/u/Vishnuprasad)\ **Replies:** 0\ **Last updated:** [October 22, 2020, 6:25am UTC](https://discuss.elastic.co/t/sending-logs-from-syslog-ng-to-logstash-with-and-without-tls/252939 "2020-10-22T06:25:10Z")
Hello Explorers :slightly\_smiling\_face: Hope I could help you if you are looking to set up a Syslog-ng Logstash configuration to transfer logs from a Client server to Master server. Advantage of this setup: If you ha… --- ## [Add Value to New Field](https://discuss.elastic.co/t/add-value-to-new-field/252985)
**Author:** [@achmad0126](https://discuss.elastic.co/u/achmad0126)\ **Replies:** 0\ **Last updated:** [October 22, 2020, 10:39am UTC](https://discuss.elastic.co/t/add-value-to-new-field/252985 "2020-10-22T10:39:13Z")
i have value from filed name like this. how to make the value become new field? --- ## [Find and replace to a unicode string](https://discuss.elastic.co/t/find-and-replace-to-a-unicode-string/252226)
**Author:** [@Andrew\_Harris](https://discuss.elastic.co/u/Andrew_Harris)\ **Replies:** 2\ **Last updated:** [October 22, 2020, 10:36am UTC](https://discuss.elastic.co/t/find-and-replace-to-a-unicode-string/252226 "2020-10-22T10:36:41Z")
Hi, I am trying to do a find and replace using mutate to replace | (vertical bar) with Unicode u0001 (start of header) anywhere in the source message before being further parsed. I have tried various combinations of fo… --- ## [Inode remaing same - reading old logs. on restart gets new inode on sincedb and get new data](https://discuss.elastic.co/t/inode-remaing-same-reading-old-logs-on-restart-gets-new-inode-on-sincedb-and-get-new-data/252942)
**Author:** [@chandramouli\_sriniva](https://discuss.elastic.co/u/chandramouli_sriniva)\ **Replies:** 0\ **Last updated:** [October 22, 2020, 6:38am UTC](https://discuss.elastic.co/t/inode-remaing-same-reading-old-logs-on-restart-gets-new-inode-on-sincedb-and-get-new-data/252942 "2020-10-22T06:38:59Z")
Hi, on log rotation, inode doesnt change and it doesnt put any new entry in sincedb file. If I stop and start logstash, then new entry with new inode gets created for current file(xx.log) and new data is being sent. S… --- ## [Can logstash be limited by subdirectories?](https://discuss.elastic.co/t/can-logstash-be-limited-by-subdirectories/252917)
**Author:** [@GetYourSh1tTogether](https://discuss.elastic.co/u/GetYourSh1tTogether)\ **Replies:** 0\ **Last updated:** [October 22, 2020, 1:06am UTC](https://discuss.elastic.co/t/can-logstash-be-limited-by-subdirectories/252917 "2020-10-22T01:06:36Z")
I have a list of subdirectories that are consistent in each directory. Is it possible to limit logstash to only look for those subdirectories and pull data from those? Would something like this work? filter { if \[type… --- ## [Multiple dissect in logstash](https://discuss.elastic.co/t/multiple-dissect-in-logstash/252898)
**Author:** [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\ **Replies:** 4\ **Last updated:** [October 21, 2020, 10:36pm UTC](https://discuss.elastic.co/t/multiple-dissect-in-logstash/252898 "2020-10-21T22:36:39Z")
Hi, I would like somehow to have multiple dissect or grok one for Errorlog : dissect { mapping =\> { "message" =\> "\[%logdate}\] %{exception}.%{CHttpException}.%{logtype}.%{loglevel}: %{errormessage}" } } and second one… --- ## [Cisco IOS netflow dhcp/mac](https://discuss.elastic.co/t/cisco-ios-netflow-dhcp-mac/252745)
**Author:** [@PublicName](https://discuss.elastic.co/u/PublicName)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 10:50pm UTC](https://discuss.elastic.co/t/cisco-ios-netflow-dhcp-mac/252745 "2020-10-20T22:50:38Z")
So quick and easy question. How to visualize Cisco IOS netflow/syslog. What I'm after which I haven't been able to figure out yet is a MAC address to a port. Does anyone know of a way to show the MAC and IP assigned to … --- ## [How to check no of events pass in a second?](https://discuss.elastic.co/t/how-to-check-no-of-events-pass-in-a-second/252674)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 1\ **Last updated:** [October 21, 2020, 4:46pm UTC](https://discuss.elastic.co/t/how-to-check-no-of-events-pass-in-a-second/252674 "2020-10-21T16:46:06Z")
When someone say i get 1200 event per second how they determine? --- ## [CI/CD Pipeline for Logstash Conf file](https://discuss.elastic.co/t/ci-cd-pipeline-for-logstash-conf-file/252866)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 0\ **Last updated:** [October 21, 2020, 3:47pm UTC](https://discuss.elastic.co/t/ci-cd-pipeline-for-logstash-conf-file/252866 "2020-10-21T15:47:32Z")
Currently I have a logstash conf file, and around 20 ruby scripts doing some jobs. Now the problem is we have dev, uat, and prod environment. Daily we are making changes to the uat and fixing this, those changes are curr… --- ## [Elasticsearch filter plugin and pipeline workers race condition](https://discuss.elastic.co/t/elasticsearch-filter-plugin-and-pipeline-workers-race-condition/252862)
**Author:** [@rfferrao](https://discuss.elastic.co/u/rfferrao)\ **Replies:** 0\ **Last updated:** [October 21, 2020, 3:22pm UTC](https://discuss.elastic.co/t/elasticsearch-filter-plugin-and-pipeline-workers-race-condition/252862 "2020-10-21T15:22:38Z")
I'm considering migrating the aggregate filter towards a query through the elasticsearch filter plugin in favor of managing parent/child relationships between sets of data. Since I'll be backtracking in search of the st… --- ## [Write only errors log level on logstash-plain.log](https://discuss.elastic.co/t/write-only-errors-log-level-on-logstash-plain-log/252810)
**Author:** [@dbviz](https://discuss.elastic.co/u/dbviz)\ **Replies:** 1\ **Last updated:** [October 21, 2020, 2:55pm UTC](https://discuss.elastic.co/t/write-only-errors-log-level-on-logstash-plain-log/252810 "2020-10-21T14:55:14Z")
Hi guys, I have lot of informations in my logstash-plain and the size can be increased to xx Gb. I thought about two solutions : First, I noticed that my logstash-plain.log contains x days informations, the solution… --- ## [Logstash to multiple ingest node pipelines](https://discuss.elastic.co/t/logstash-to-multiple-ingest-node-pipelines/252850)
**Author:** [@victor\_ide](https://discuss.elastic.co/u/victor_ide)\ **Replies:** 0\ **Last updated:** [October 21, 2020, 2:15pm UTC](https://discuss.elastic.co/t/logstash-to-multiple-ingest-node-pipelines/252850 "2020-10-21T14:15:43Z")
I was looking into the forum on how to setup logstash to sent data using more than one ingest node pipelines but could not find nothing related, my objective is to split the load into elastic cloud and my logstash instan… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=288) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=290)