# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=89 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 90 --- ## [Ruby script for auditd EXECVE logs](https://discuss.elastic.co/t/ruby-script-for-auditd-execve-logs/326098)
**Author:** [@JCW](https://discuss.elastic.co/u/JCW)\ **Replies:** 2\ **Last updated:** [February 21, 2023, 10:53pm UTC](https://discuss.elastic.co/t/ruby-script-for-auditd-execve-logs/326098 "2023-02-21T22:53:09Z")
I want to make a ruby script that makes an extra field for "command" that it parses out of the message that auditd creates, however it does not work and I am unable to figure out why. example log: type=EXECVE msg=audit… --- ## [Compare 2 fields and drop the matching](https://discuss.elastic.co/t/compare-2-fields-and-drop-the-matching/326032)
**Author:** [@Dr.Dark92](https://discuss.elastic.co/u/Dr.Dark92)\ **Replies:** 1\ **Last updated:** [February 21, 2023, 5:27pm UTC](https://discuss.elastic.co/t/compare-2-fields-and-drop-the-matching/326032 "2023-02-21T17:27:45Z")
Hi, I am trying to make a Dashbaord for Bind9 engine, in brief i have 3 indexes, first index for Blacklist logs second index for whitelist logs third index for resolver logs. the goal is : to drop any blacklisted d… --- ## [Snmptrap to logstash](https://discuss.elastic.co/t/snmptrap-to-logstash/326092)
**Author:** [@diegz](https://discuss.elastic.co/u/diegz)\ **Replies:** 0\ **Last updated:** [February 21, 2023, 3:35pm UTC](https://discuss.elastic.co/t/snmptrap-to-logstash/326092 "2023-02-21T15:35:45Z")
Hello, I would like to send traps retrieved via the snmptrap service on RHEL 8 to logstash to store them on Elasticsearch. As the snmptrap input module does not support v3, I installed snmptrapd on the logstash server … --- ## [Logstash Grok Path "\\" character](https://discuss.elastic.co/t/logstash-grok-path-character/326083)
**Author:** [@GinkoLucas](https://discuss.elastic.co/u/GinkoLucas)\ **Replies:** 1\ **Last updated:** [February 21, 2023, 3:03pm UTC](https://discuss.elastic.co/t/logstash-grok-path-character/326083 "2023-02-21T15:03:42Z")
Hello, I have a problem that seems simple to solve, but I'm having trouble solving it. I have to grok a path, here is a path similar to mine: D:\\MyFiles\\allmylogs.log I have a problem with the "\\". How can I inclu… --- ## [Logstash webhook when parsing is done](https://discuss.elastic.co/t/logstash-webhook-when-parsing-is-done/326050)
**Author:** [@Jirka\_Liska](https://discuss.elastic.co/u/Jirka_Liska)\ **Replies:** 1\ **Last updated:** [February 21, 2023, 12:59pm UTC](https://discuss.elastic.co/t/logstash-webhook-when-parsing-is-done/326050 "2023-02-21T12:59:33Z")
Hello community! I'm currently working on integration elastic stack with my custom application. I'm trying to find out if possible for logstash to create webook call when processing is done? The workload is: File upload… --- ## [How to run a ruby function that updates and event and clears the empty fields recursively](https://discuss.elastic.co/t/how-to-run-a-ruby-function-that-updates-and-event-and-clears-the-empty-fields-recursively/326057)
**Author:** [@vilman](https://discuss.elastic.co/u/vilman)\ **Replies:** 2\ **Last updated:** [February 21, 2023, 11:12am UTC](https://discuss.elastic.co/t/how-to-run-a-ruby-function-that-updates-and-event-and-clears-the-empty-fields-recursively/326057 "2023-02-21T11:12:30Z")
I got an event which sometimes contain empty fields. I would like to delete those fields which are null and those which are empty. --- ## [How di I map Timestamp to event Timestamp from filebeat thru logstash](https://discuss.elastic.co/t/how-di-i-map-timestamp-to-event-timestamp-from-filebeat-thru-logstash/325577)
**Author:** [@jkingstone](https://discuss.elastic.co/u/jkingstone)\ **Replies:** 9\ **Last updated:** [February 21, 2023, 6:23am UTC](https://discuss.elastic.co/t/how-di-i-map-timestamp-to-event-timestamp-from-filebeat-thru-logstash/325577 "2023-02-21T06:23:25Z")
Hi, I want to change the @timestamp to the timestamp out of event.original or message. I don't know what I do wrong. right now the @timestamp is the time where the filebeat logfile ist importet thru logstash into elast… --- ## [Nginx access log using grok filter](https://discuss.elastic.co/t/nginx-access-log-using-grok-filter/324877)
**Author:** [@yc99](https://discuss.elastic.co/u/yc99)\ **Replies:** 1\ **Last updated:** [February 21, 2023, 12:36am UTC](https://discuss.elastic.co/t/nginx-access-log-using-grok-filter/324877 "2023-02-21T00:36:14Z")
My nginx access log format as below, there certain access log without the "$request\_time" "$http\_x\_forwarded\_for" $http\_host ' field, therefore, for certain access log, the grok filter not working, is there anyway to a… --- ## [Matching ip address](https://discuss.elastic.co/t/matching-ip-address/325992)
**Author:** [@Lalii](https://discuss.elastic.co/u/Lalii)\ **Replies:** 4\ **Last updated:** [February 20, 2023, 4:55pm UTC](https://discuss.elastic.co/t/matching-ip-address/325992 "2023-02-20T16:55:01Z")
Hello everyone, I'm trying to do a condition on IP regex. Trying to match every IPs if \[destination.XXX\] =~ /^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}$/ { mutate { ... } } I tried the solution from that post but doesnt wor… --- ## [Issues enabling Logstash logs integration for ELK stack](https://discuss.elastic.co/t/issues-enabling-logstash-logs-integration-for-elk-stack/325993)
**Author:** [@Joshua\_Sheathelm](https://discuss.elastic.co/u/Joshua_Sheathelm)\ **Replies:** 0\ **Last updated:** [February 20, 2023, 4:13pm UTC](https://discuss.elastic.co/t/issues-enabling-logstash-logs-integration-for-elk-stack/325993 "2023-02-20T16:13:23Z")
I am currently configuring an ELK stack with three separate hosts for each service (Elastic search on one host, Logstash on another, and Kibana on the last) I have verified that Elasticsearch and Kibana are accessible fr… --- ## [Not an int hash when using Murmur3](https://discuss.elastic.co/t/not-an-int-hash-when-using-murmur3/324902)
**Author:** [@divadpoc](https://discuss.elastic.co/u/divadpoc)\ **Replies:** 3\ **Last updated:** [February 20, 2023, 3:09pm UTC](https://discuss.elastic.co/t/not-an-int-hash-when-using-murmur3/324902 "2023-02-20T15:09:12Z")
I've tried the Fingerprint filter plugin w/ the MURMUR3 method. If set to MURMUR3 or MURMUR3\_128 the non-cryptographic MurmurHash function (either the 32-bit or 128-bit implementation, respectively) will be used. So … --- ## [Logstash masking logs syntax](https://discuss.elastic.co/t/logstash-masking-logs-syntax/325699)
**Author:** [@furkano](https://discuss.elastic.co/u/furkano)\ **Replies:** 3\ **Last updated:** [February 20, 2023, 5:32am UTC](https://discuss.elastic.co/t/logstash-masking-logs-syntax/325699 "2023-02-20T05:32:58Z")
Hi, I want to mask some logs in spesific fields, for example if end point ends with api or token i want to remove userKey messages from field ResponseMessage But not whole field that i want to remove or mask, only the … --- ## [Is cloudfront codec ever works before?](https://discuss.elastic.co/t/is-cloudfront-codec-ever-works-before/325879)
**Author:** [@stwang](https://discuss.elastic.co/u/stwang)\ **Replies:** 1\ **Last updated:** [February 18, 2023, 5:15am UTC](https://discuss.elastic.co/t/is-cloudfront-codec-ever-works-before/325879 "2023-02-18T05:15:56Z")
Hi There, I am try to parse cloudfront log in logstash, and I found there has a cloudfront codec can be used. but I never make it works. Can someone pointing me a vaild config, or this codec never works before? Cheers… --- ## [Large-scale setup on AWS](https://discuss.elastic.co/t/large-scale-setup-on-aws/325818)
**Author:** [@eof](https://discuss.elastic.co/u/eof)\ **Replies:** 0\ **Last updated:** [February 17, 2023, 7:20am UTC](https://discuss.elastic.co/t/large-scale-setup-on-aws/325818 "2023-02-17T07:20:12Z")
I'm currently running our infrastructure on ECS Fargate with logs going into CloudWatch. I've been less than happy with CloudWatch as a log tool and have used an ELK stack previously for smaller setups. We have a set of … --- ## [ELK setup on kubernetes](https://discuss.elastic.co/t/elk-setup-on-kubernetes/325811)
**Author:** [@vijay78](https://discuss.elastic.co/u/vijay78)\ **Replies:** 0\ **Last updated:** [February 17, 2023, 6:25am UTC](https://discuss.elastic.co/t/elk-setup-on-kubernetes/325811 "2023-02-17T06:25:04Z")
Iam trying to setup ELK on Azure kubernetes services (AKS) iam finding difficulties as not able to run Elasticsearch,logstash,kibana as a containers kindly help me on this as i don't want to go with ECK let me know if we… --- ## [Logstash to load input file based on time change](https://discuss.elastic.co/t/logstash-to-load-input-file-based-on-time-change/325806)
**Author:** [@dhiyaneshwaran](https://discuss.elastic.co/u/dhiyaneshwaran)\ **Replies:** 0\ **Last updated:** [February 17, 2023, 3:31am UTC](https://discuss.elastic.co/t/logstash-to-load-input-file-based-on-time-change/325806 "2023-02-17T03:31:13Z")
I'm using Logstash 7.17.0, in that i'm trying to load file using pipeline. It is taking file based on size or checksum changes, but i wanted to pick the file even if the size same but change in file timings. For exampl… --- ## [How to get a single field from one beats event and add it to another beats event?](https://discuss.elastic.co/t/how-to-get-a-single-field-from-one-beats-event-and-add-it-to-another-beats-event/325778)
**Author:** [@scantron](https://discuss.elastic.co/u/scantron)\ **Replies:** 1\ **Last updated:** [February 16, 2023, 9:10pm UTC](https://discuss.elastic.co/t/how-to-get-a-single-field-from-one-beats-event-and-add-it-to-another-beats-event/325778 "2023-02-16T21:10:01Z")
Hello, Essentially, I am looking to monitor when my server is getting full, using metricbeat to log the space left on the filesystem, and also using a filebeat cronjob to monitor more specifically where all of the stora… --- ## [Migrating from self hosted to elastic service, how to change our ingest flow from kafka/logstash?](https://discuss.elastic.co/t/migrating-from-self-hosted-to-elastic-service-how-to-change-our-ingest-flow-from-kafka-logstash/325766)
**Author:** [@tymercer](https://discuss.elastic.co/u/tymercer)\ **Replies:** 7\ **Last updated:** [February 16, 2023, 7:14pm UTC](https://discuss.elastic.co/t/migrating-from-self-hosted-to-elastic-service-how-to-change-our-ingest-flow-from-kafka-logstash/325766 "2023-02-16T19:14:48Z")
We are in the process of migrating our self hosted ELK stack to the hosted Elastic Services in Azure. Currently we have our servers running filebeat configured to push their logs to Kafka/zookeeper which then pushes to … --- ## [Convert values from string to int](https://discuss.elastic.co/t/convert-values-from-string-to-int/325164)
**Author:** [@Law\_Rence](https://discuss.elastic.co/u/Law_Rence)\ **Replies:** 6\ **Last updated:** [February 16, 2023, 2:31pm UTC](https://discuss.elastic.co/t/convert-values-from-string-to-int/325164 "2023-02-16T14:31:23Z")
How can I convert all the keys that have numbers from strings to int using ruby? example: "x": "hello", "a": "1", "b": "2", "c": "3", "d": "bye" to: "x": "hello", "a": 1, "b": 2, "c": 3, e.t.c here's my ruby… --- ## [When i use snmp . why value in key:value is missing](https://discuss.elastic.co/t/when-i-use-snmp-why-value-in-key-value-is-missing/325684)
**Author:** [@sirichai\_phungsuntho](https://discuss.elastic.co/u/sirichai_phungsuntho)\ **Replies:** 0\ **Last updated:** [February 16, 2023, 4:37am UTC](https://discuss.elastic.co/t/when-i-use-snmp-why-value-in-key-value-is-missing/325684 "2023-02-16T04:37:06Z")
When i use input snmp and selct more than 10 columns in function tables i will receive missing value like this how can i fix it? --- ## [Logstash Kafka consumer count](https://discuss.elastic.co/t/logstash-kafka-consumer-count/325671)
**Author:** [@rsk0](https://discuss.elastic.co/u/rsk0)\ **Replies:** 1\ **Last updated:** [February 16, 2023, 5:47am UTC](https://discuss.elastic.co/t/logstash-kafka-consumer-count/325671 "2023-02-16T05:47:58Z")
According to the Logstash guide: "How many partitions should I use per topic?" At least the number of Logstash nodes multiplied by consumer threads per node. Better yet, use a multiple of the above number. Increasing… --- ## [Multiple Logstash Containers](https://discuss.elastic.co/t/multiple-logstash-containers/325319)
**Author:** [@Vaibhav\_Saxena1](https://discuss.elastic.co/u/Vaibhav_Saxena1)\ **Replies:** 0\ **Last updated:** [February 11, 2023, 5:32pm UTC](https://discuss.elastic.co/t/multiple-logstash-containers/325319 "2023-02-11T17:32:58Z")
Hello, We have following containers setup on our environment: 1- logstash ( Stomp) 2- logstash ( Filebeat) port: 5044 3- Kibana 4- Elasticsearch But by mistake i created the logstash(filebeat) to read only one "lo… --- ## [Logstash pipeline index question](https://discuss.elastic.co/t/logstash-pipeline-index-question/325273)
**Author:** [@MKirby](https://discuss.elastic.co/u/MKirby)\ **Replies:** 12\ **Last updated:** [February 15, 2023, 9:28pm UTC](https://discuss.elastic.co/t/logstash-pipeline-index-question/325273 "2023-02-15T21:28:45Z")
AS many of you know and have been following, my syslog collectors keep stopping due to running out of shards. I have made some improvements and they now run for about 3 weeks before I have to "close" the index. Better … --- ## [Logstash rename json fields](https://discuss.elastic.co/t/logstash-rename-json-fields/325399)
**Author:** [@yilmazbuhar](https://discuss.elastic.co/u/yilmazbuhar)\ **Replies:** 6\ **Last updated:** [February 15, 2023, 9:24pm UTC](https://discuss.elastic.co/t/logstash-rename-json-fields/325399 "2023-02-15T21:24:05Z")
Hi community, We have a json log as below { "Timestamp": "2023-02-09T17:41:54.5320239+03:00", "Level": "", "MessageTemplate": "", "Properties": { "responsetime": 4758, "SourceContext": "", "Username… --- ## [【Logstash】The output configuration of logstash cannot connect to the elasticsearch](https://discuss.elastic.co/t/logstash-the-output-configuration-of-logstash-cannot-connect-to-the-elasticsearch/325523)
**Author:** [@Roy176](https://discuss.elastic.co/u/Roy176)\ **Replies:** 3\ **Last updated:** [February 15, 2023, 8:47pm UTC](https://discuss.elastic.co/t/logstash-the-output-configuration-of-logstash-cannot-connect-to-the-elasticsearch/325523 "2023-02-15T20:47:53Z")
I build a single-node of elasticsearch on GCP and a logstash on the local side. I want to connect the output configuration of logstash to elasticsearch. Info: Elasticsearch、Kibana、Logstash: 8.6.1. I set up an extenal … --- ## [Logstash - Could not connect to a compatible version of Elasticsearch](https://discuss.elastic.co/t/logstash-could-not-connect-to-a-compatible-version-of-elasticsearch/325629)
**Author:** [@hnclientes\_HN](https://discuss.elastic.co/u/hnclientes_HN)\ **Replies:** 1\ **Last updated:** [February 15, 2023, 3:20pm UTC](https://discuss.elastic.co/t/logstash-could-not-connect-to-a-compatible-version-of-elasticsearch/325629 "2023-02-15T15:20:58Z")
I'm trying to upload a .csv file via logstash to a test version on Cloud V 8.6.1 and I get an error when trying (I'm using logstash version 8.6.1 anyway) and I get the following error: ´\`\`\` \[2023-02-14T23:21:15,274\]\[ER… --- ## [Multiline filter is not working even after installing the plugin](https://discuss.elastic.co/t/multiline-filter-is-not-working-even-after-installing-the-plugin/325611)
**Author:** [@Balaguru\_Maruthamuth](https://discuss.elastic.co/u/Balaguru_Maruthamuth)\ **Replies:** 2\ **Last updated:** [February 15, 2023, 12:44pm UTC](https://discuss.elastic.co/t/multiline-filter-is-not-working-even-after-installing-the-plugin/325611 "2023-02-15T12:44:36Z")
Warning: Manual override - there are filters that might not work with multiple worker threads {:pipeline\_id=\>"exterro", :worker\_threads=\>3, :filters=\>\["multiline", "multiline", "multiline", "multiline", "multiline", "mul… --- ## [Change time zone using date filter](https://discuss.elastic.co/t/change-time-zone-using-date-filter/325461)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 5\ **Last updated:** [February 15, 2023, 12:37pm UTC](https://discuss.elastic.co/t/change-time-zone-using-date-filter/325461 "2023-02-15T12:37:36Z")
Hi there, i have a problem with timezone in date filter. so this is the situation: i have a field contain an epoch timestamp like this i try to convert it using date filter like this but it didn't work mutate{ … --- ## [Custom plugin and custom entries in /etc/default/logstash gets deleted after each update on ubuntu](https://discuss.elastic.co/t/custom-plugin-and-custom-entries-in-etc-default-logstash-gets-deleted-after-each-update-on-ubuntu/325549)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 0\ **Last updated:** [February 15, 2023, 6:44am UTC](https://discuss.elastic.co/t/custom-plugin-and-custom-entries-in-etc-default-logstash-gets-deleted-after-each-update-on-ubuntu/325549 "2023-02-15T06:44:03Z")
Hello All, I wanted a to ask if anybody knows why after almost each apt-get update/upgrate on my server two output plugins always gets deleted and I need to reinstall them along with all custom Logstash entries in /etc/… --- ## [Restarting logstash container sends events again to elastic, despite sincedb](https://discuss.elastic.co/t/restarting-logstash-container-sends-events-again-to-elastic-despite-sincedb/324675)
**Author:** [@paul\_chrlt](https://discuss.elastic.co/u/paul_chrlt)\ **Replies:** 7\ **Last updated:** [February 14, 2023, 6:03pm UTC](https://discuss.elastic.co/t/restarting-logstash-container-sends-events-again-to-elastic-despite-sincedb/324675 "2023-02-14T18:03:09Z")
Hi all, Can you help us ? We use elk 7.17.7 in docker containers hosted on a server with persistent shared volumes for path (read only), file\_completed\_log\_path, sincedb\_path. Each time we stop and start our logstash … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=88) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=90)