# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=286 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 287 --- ## [Logstash and ILM - rollover not working](https://discuss.elastic.co/t/logstash-and-ilm-rollover-not-working/253530)
**Author:** [@BBQigniter](https://discuss.elastic.co/u/BBQigniter)\ **Replies:** 6\ **Last updated:** [November 3, 2020, 12:33pm UTC](https://discuss.elastic.co/t/logstash-and-ilm-rollover-not-working/253530 "2020-11-03T12:33:39Z")
I please need help to understand ILM in combination with Logstash. My current setup somehow seems not to work as expected. Using ELK 7.9.1 - My configuration looks like: ILM policy - logstash-syslog-compliance { - "l… --- ## [Not seeing any pfsense logs](https://discuss.elastic.co/t/not-seeing-any-pfsense-logs/254145)
**Author:** [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\ **Replies:** 4\ **Last updated:** [November 3, 2020, 11:48am UTC](https://discuss.elastic.co/t/not-seeing-any-pfsense-logs/254145 "2020-11-03T11:48:14Z")
I only need to display the Ips that are coming from a remote PFSENSE firewall on kibana but I couldn't do it, can anyone help me? input { beats { port =\> 5044 } } --- ## [Logstash manticore worker error](https://discuss.elastic.co/t/logstash-manticore-worker-error/254137)
**Author:** [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)\ **Replies:** 0\ **Last updated:** [November 3, 2020, 10:46am UTC](https://discuss.elastic.co/t/logstash-manticore-worker-error/254137 "2020-11-03T10:46:11Z")
I have installed logstash and am trying to run the following configuration, input { file { path =\>"C:\\Test\\CSV\\dev.csv" start\_position =\>"beginning" } } filter { csv { columns =\>\["Plugin","… --- ## [Unable to access secured Elasticsearch using logstash](https://discuss.elastic.co/t/unable-to-access-secured-elasticsearch-using-logstash/253093)
**Author:** [@aakash075](https://discuss.elastic.co/u/aakash075)\ **Replies:** 5\ **Last updated:** [November 3, 2020, 7:07am UTC](https://discuss.elastic.co/t/unable-to-access-secured-elasticsearch-using-logstash/253093 "2020-11-03T07:07:51Z")
Hello Team, I am using Openshift environment and i deployed EFK (Elasticsearch, fluentd and Kibana) stack in it using some logging operators . and then due to some requirement i deployed Logstash as docker container in… --- ## [Source a file from inside a logstash config pipeline?](https://discuss.elastic.co/t/source-a-file-from-inside-a-logstash-config-pipeline/254103)
**Author:** [@lamp123432](https://discuss.elastic.co/u/lamp123432)\ **Replies:** 0\ **Last updated:** [November 3, 2020, 5:22am UTC](https://discuss.elastic.co/t/source-a-file-from-inside-a-logstash-config-pipeline/254103 "2020-11-03T05:22:38Z")
Here's my logstash input config: input { snmp { tables =\> \[ { "name" =\> "interfaces" "columns" =\> \[ ".1.3.6.1.2.1.31.1.1.1.1" … --- ## [When I created logstash offline pack, it doesn't contain the change](https://discuss.elastic.co/t/when-i-created-logstash-offline-pack-it-doesnt-contain-the-change/253983)
**Author:** [@hyon](https://discuss.elastic.co/u/hyon)\ **Replies:** 1\ **Last updated:** [November 3, 2020, 4:52am UTC](https://discuss.elastic.co/t/when-i-created-logstash-offline-pack-it-doesnt-contain-the-change/253983 "2020-11-03T04:52:08Z")
I want to modify the logstash integration rabbitmq plugin and use it (https://github.com/logstash-plugins/logstash-integration-rabbitmq.git) I modified the inputs/rabbitmq.rb and confirmed that it works properly on my l… --- ## [Memcached filter plugin issues](https://discuss.elastic.co/t/memcached-filter-plugin-issues/254095)
**Author:** [@aleksf](https://discuss.elastic.co/u/aleksf)\ **Replies:** 0\ **Last updated:** [November 3, 2020, 3:48am UTC](https://discuss.elastic.co/t/memcached-filter-plugin-issues/254095 "2020-11-03T03:48:06Z")
I was hoping this was fixed in https://github.com/logstash-plugins/logstash-filter-memcached/pull/25 with the plugin release v1.1.0 Related issue was already raised in https://discuss.elastic.co/t/catch-memcached-filter… --- ## [XML Filter Output to Ruby Code](https://discuss.elastic.co/t/xml-filter-output-to-ruby-code/253870)
**Author:** [@chayelastic](https://discuss.elastic.co/u/chayelastic)\ **Replies:** 7\ **Last updated:** [November 3, 2020, 2:22am UTC](https://discuss.elastic.co/t/xml-filter-output-to-ruby-code/253870 "2020-11-03T02:22:40Z")
Hello, In my Logstash config, I have an XML filter with xpath that allows me to read the XML tags and store in a variable/field. It works well. Next step, I had to use a Ruby Filter for I need to transpose a couple o… --- ## [Filtering all Fields/Events](https://discuss.elastic.co/t/filtering-all-fields-events/254088)
**Author:** [@faridNR](https://discuss.elastic.co/u/faridNR)\ **Replies:** 1\ **Last updated:** [November 3, 2020, 2:21am UTC](https://discuss.elastic.co/t/filtering-all-fields-events/254088 "2020-11-03T02:21:55Z")
Hello Having a centralized log-stream using Logstash with more than hundreds of fields/events types. I'm trying to apply filter to all events/fields and replacing a regex in this case email address. (removing PII) wha… --- ## [Logstash out knesis](https://discuss.elastic.co/t/logstash-out-knesis/253923)
**Author:** [@111387](https://discuss.elastic.co/u/111387)\ **Replies:** 0\ **Last updated:** [November 1, 2020, 4:55am UTC](https://discuss.elastic.co/t/logstash-out-knesis/253923 "2020-11-01T04:55:43Z")
In logstash, input plugin has knesis, but output plugin is not. There is https://github.com/samcday/logstash-output-kinesis, but it doesn't seem to be being updated anymore. Is there no knesis output method in logstash… --- ## [Logstash Failed to parse with all enclosed parsers](https://discuss.elastic.co/t/logstash-failed-to-parse-with-all-enclosed-parsers/254054)
**Author:** [@Yuy\_Heero](https://discuss.elastic.co/u/Yuy_Heero)\ **Replies:** 1\ **Last updated:** [November 2, 2020, 5:56pm UTC](https://discuss.elastic.co/t/logstash-failed-to-parse-with-all-enclosed-parsers/254054 "2020-11-02T17:56:16Z")
I want to use winlogbeat to monitor win10. Logs transfer via logstash to elasticsearch. winlogbeat(7.9.3)-\>logstash(7.9.3)-\>elasticsearch(7.8.0) After I started winlogbeat and logstash, I got no error from winlogbe… --- ## [Logstash replace existing fields & file input advice](https://discuss.elastic.co/t/logstash-replace-existing-fields-file-input-advice/254047)
**Author:** [@iccMe](https://discuss.elastic.co/u/iccMe)\ **Replies:** 3\ **Last updated:** [November 2, 2020, 5:36pm UTC](https://discuss.elastic.co/t/logstash-replace-existing-fields-file-input-advice/254047 "2020-11-02T17:36:29Z")
Hi, So this is a 2 part question. Below is an example of a log that is being read from a log file: \[09/10/2020 12:16:00\] xxx-server - HTTP Connections Spiking Ok 3.00 Perf Counter test (Current Connections) 4857 T… --- ## [Negative regrex \[field\] comparison in Logstash](https://discuss.elastic.co/t/negative-regrex-field-comparison-in-logstash/254060)
**Author:** [@Orest\_Gulman](https://discuss.elastic.co/u/Orest_Gulman)\ **Replies:** 2\ **Last updated:** [November 2, 2020, 5:19pm UTC](https://discuss.elastic.co/t/negative-regrex-field-comparison-in-logstash/254060 "2020-11-02T17:19:24Z")
Hi, I'm trying to achieve simple \[username\] field character cheks. If \[username\] field NOT consists of \[A-Za-z\] then add new field. For example, if \[username\] equals foo.foo then write it into separate field. I can't … --- ## [Replace value with another value](https://discuss.elastic.co/t/replace-value-with-another-value/253741)
**Author:** [@errupeshmca](https://discuss.elastic.co/u/errupeshmca)\ **Replies:** 2\ **Last updated:** [November 2, 2020, 4:55pm UTC](https://discuss.elastic.co/t/replace-value-with-another-value/253741 "2020-11-02T16:55:11Z")
Hi, Below is my code to replace one value with another value, i have True and false value in field Order completed ? , and i want to replace true with Total order shipped and false with total order Pending. But when y… --- ## [Grok succeeds in debugger and through input{stdin. but with this config file I'm getting \_grokparsefailure](https://discuss.elastic.co/t/grok-succeeds-in-debugger-and-through-input-stdin-but-with-this-config-file-im-getting-grokparsefailure/253556)
**Author:** [@Vita\_Rosenberg](https://discuss.elastic.co/u/Vita_Rosenberg)\ **Replies:** 2\ **Last updated:** [November 2, 2020, 2:26pm UTC](https://discuss.elastic.co/t/grok-succeeds-in-debugger-and-through-input-stdin-but-with-this-config-file-im-getting-grokparsefailure/253556 "2020-11-02T14:26:18Z")
path =\> "C:/ELK/LocalLogs/\*" start\_position =\> beginning } } filter { grok{ match =\> { "message" =\> \["%{GREEDYDATA:first}%{DATE\_US:date\_}-%{TIME:time\_}%{GREEDYDATA:last}"\]} } da… --- ## [Logstash-Ruby is using which Testing Framework?](https://discuss.elastic.co/t/logstash-ruby-is-using-which-testing-framework/254041)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 0\ **Last updated:** [November 2, 2020, 2:19pm UTC](https://discuss.elastic.co/t/logstash-ruby-is-using-which-testing-framework/254041 "2020-11-02T14:19:52Z")
I am using the test sample shown in this page: But it seems this assert works events.size == 0 Because the debugging it very hard with this, if i have multiple assert, it doesnt tell me which assert failed... --- ## [Logstash Error while configuring the JDBC connection](https://discuss.elastic.co/t/logstash-error-while-configuring-the-jdbc-connection/254036)
**Author:** [@Kannan\_Rajendran](https://discuss.elastic.co/u/Kannan_Rajendran)\ **Replies:** 0\ **Last updated:** [November 2, 2020, 1:57pm UTC](https://discuss.elastic.co/t/logstash-error-while-configuring-the-jdbc-connection/254036 "2020-11-02T13:57:32Z")
Hi , When i try to connect the JDBC connection and test the logstash conf file i'm getting the below error. i have tried the some forum solution but nothing worked. \[logstash.runner \] The given configuration is… --- ## [Ruby syntax error](https://discuss.elastic.co/t/ruby-syntax-error/252197)
**Author:** [@Jan\_Kabelka](https://discuss.elastic.co/u/Jan_Kabelka)\ **Replies:** 3\ **Last updated:** [November 2, 2020, 10:40am UTC](https://discuss.elastic.co/t/ruby-syntax-error/252197 "2020-11-02T10:40:21Z")
Hi, I did not find anything what would work therefore please let me ask... I need to create new field with length of DNS domain. Also I would like to have another field with entropy value for that DNS domain. What field… --- ## [Logstash error](https://discuss.elastic.co/t/logstash-error/253664)
**Author:** [@punithjigali](https://discuss.elastic.co/u/punithjigali)\ **Replies:** 2\ **Last updated:** [November 2, 2020, 9:05am UTC](https://discuss.elastic.co/t/logstash-error/253664 "2020-11-02T09:05:48Z")
Hi team, I have installed jdk 15. I am getting this error when I run below command... C:\\Elastic stack\\logstash-7.9.3\\logstash-7.9.3\\bin\>logstash -f C:\\Elastic stack\\logstash.conf Unrecognized VM option 'UseConcMarkSw… --- ## [REST return Json object - How to push it to Logstash C#?](https://discuss.elastic.co/t/rest-return-json-object-how-to-push-it-to-logstash-c/253533)
**Author:** [@yaharin\_ben\_ayon](https://discuss.elastic.co/u/yaharin_ben_ayon)\ **Replies:** 3\ **Last updated:** [November 2, 2020, 5:35am UTC](https://discuss.elastic.co/t/rest-return-json-object-how-to-push-it-to-logstash-c/253533 "2020-11-02T05:35:28Z")
Hey I have a json object returning from a rest call - how can i push it to logstash? The code is in C#. Thanks! --- ## [Geoip filter](https://discuss.elastic.co/t/geoip-filter/253960)
**Author:** [@Ronnie\_Raraihuru](https://discuss.elastic.co/u/Ronnie_Raraihuru)\ **Replies:** 3\ **Last updated:** [November 2, 2020, 3:01am UTC](https://discuss.elastic.co/t/geoip-filter/253960 "2020-11-02T03:01:46Z")
My sample log contents #Fields: date-time,connector-id,session-id,sequence-number,local-endpoint,remote-endpoint,event,data,context 2020-10-22T23:59:53.533Z,SIG-EXCH13-01\\Default Frontend SIG-EXCH13-01,08D8724E3CB78EDF… --- ## [Could not execute action: PipelineAction::Create\ after xpack.security enabled](https://discuss.elastic.co/t/could-not-execute-action-pipelineaction-create-main-after-xpack-security-enabled/253861)
**Author:** [@ASA01](https://discuss.elastic.co/u/ASA01)\ **Replies:** 5\ **Last updated:** [November 2, 2020, 3:06am UTC](https://discuss.elastic.co/t/could-not-execute-action-pipelineaction-create-main-after-xpack-security-enabled/253861 "2020-11-02T03:06:12Z")
Logstash works fine for me until I enable xpack security in elasticsearch.yml. When I do I get the following error from any conf that creates indices. \[ERROR\] 2020-10-30 15:52:06.741 \[Converge PipelineAction::Create\] a… --- ## [Two inputs Logstash](https://discuss.elastic.co/t/two-inputs-logstash/253958)
**Author:** [@yaharin\_ben\_ayon](https://discuss.elastic.co/u/yaharin_ben_ayon)\ **Replies:** 1\ **Last updated:** [November 1, 2020, 7:58pm UTC](https://discuss.elastic.co/t/two-inputs-logstash/253958 "2020-11-01T19:58:17Z")
Can i use two inputs in the config file? HTTP and File? --- ## [\ Use StaticDate and Time for Dev Env and CurrentDateTime for Prod Env](https://discuss.elastic.co/t/ruby-use-staticdate-and-time-for-dev-env-and-currentdatetime-for-prod-env/253925)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 1\ **Last updated:** [November 1, 2020, 3:29pm UTC](https://discuss.elastic.co/t/ruby-use-staticdate-and-time-for-dev-env-and-currentdatetime-for-prod-env/253925 "2020-11-01T15:29:01Z")
I have a requirement where I need to use specific date/time for a testing at Ruby scripts and not at Logstash level. Is there a way i could stub those value? Currently I am using the enviorment variable to check if it i… --- ## [Externalization of hosts settings in output](https://discuss.elastic.co/t/externalization-of-hosts-settings-in-output/253680)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 5\ **Last updated:** [October 31, 2020, 6:33am UTC](https://discuss.elastic.co/t/externalization-of-hosts-settings-in-output/253680 "2020-10-31T06:33:28Z")
Hi, I have a logstash indexing to 3 elasticsearch nodes. The output filter is as below: elasticsearch { hosts =\> \[ "https://myHost9a.vsi.uat.dbs.com:9202", "https://myHost10a.vsi.uat.dbs.com:9202", "https://myHost11… --- ## [Custom the logstash java-Filter meet problem(cannot create task "gem")](https://discuss.elastic.co/t/custom-the-logstash-java-filter-meet-problem-cannot-create-task-gem/253774)
**Author:** [@xiaoping1993](https://discuss.elastic.co/u/xiaoping1993)\ **Replies:** 1\ **Last updated:** [October 30, 2020, 11:45pm UTC](https://discuss.elastic.co/t/custom-the-logstash-java-filter-meet-problem-cannot-create-task-gem/253774 "2020-10-30T23:45:56Z")
I want to custom the logstash java-Filter follow the url:https://www.elastic.co/guide/en/logstash/7.6/java-filter-plugin.html but I meet a problem when I run "gradlew gem" the error is below --- ## [Can not decode Citrix Netscaler IPFIX messages](https://discuss.elastic.co/t/can-not-decode-citrix-netscaler-ipfix-messages/244654)
**Author:** [@tkuronen](https://discuss.elastic.co/u/tkuronen)\ **Replies:** 7\ **Last updated:** [October 30, 2020, 7:17pm UTC](https://discuss.elastic.co/t/can-not-decode-citrix-netscaler-ipfix-messages/244654 "2020-10-30T19:17:17Z")
Is anyone here successfully decoded a IPFIX data send by Citrix Netscaler ? Version: Logstash (7.7.1) / logstash-codec-netflow (4.2.1) Operating System: RHEL 7.8 Logstash Input config: input { udp { id =\> … --- ## [TLS config issues?](https://discuss.elastic.co/t/tls-config-issues/253602)
**Author:** [@jcor](https://discuss.elastic.co/u/jcor)\ **Replies:** 10\ **Last updated:** [October 30, 2020, 5:41pm UTC](https://discuss.elastic.co/t/tls-config-issues/253602 "2020-10-30T17:41:03Z")
I'm trying to setup tls on ELK. I have tls and https working between elastic and kibana but no luck with logstash. Below are the logstash configs. I've followed this guide almost to the T till step 7 besides swapping dns… --- ## [Pipeline-to-Pipeline Configuration Does Not Work with TLS Cluster](https://discuss.elastic.co/t/pipeline-to-pipeline-configuration-does-not-work-with-tls-cluster/253692)
**Author:** [@sivanov](https://discuss.elastic.co/u/sivanov)\ **Replies:** 3\ **Last updated:** [October 30, 2020, 4:19pm UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-configuration-does-not-work-with-tls-cluster/253692 "2020-10-30T16:19:25Z")
Hi guys, I have an ECK cluster that receives logs from Logstash. Logstash is deployed and running as a k8s /deployment/ pod next to that cluster. Filebeat (deployed elsewhere) is used to read logs and ship them to Logst… --- ## [How logstash create indices based on date - wrong date](https://discuss.elastic.co/t/how-logstash-create-indices-based-on-date-wrong-date/253826)
**Author:** [@dorinand](https://discuss.elastic.co/u/dorinand)\ **Replies:** 2\ **Last updated:** [October 30, 2020, 2:59pm UTC](https://discuss.elastic.co/t/how-logstash-create-indices-based-on-date-wrong-date/253826 "2020-10-30T14:59:49Z")
I am running ELK with filebeat in kubernetes. Filebeat is harvesting logs and sending it to logstash. This is my logstash filter: if \[kubernetes\]\[annotations\]\[elastic\_index\] { mutate { add\_field … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=285) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=287)