# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=248 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 249 --- ## [How to filter a json field in an event?](https://discuss.elastic.co/t/how-to-filter-a-json-field-in-an-event/267007)
**Author:** [@hackercat](https://discuss.elastic.co/u/hackercat)\ **Replies:** 2\ **Last updated:** [March 11, 2021, 11:57pm UTC](https://discuss.elastic.co/t/how-to-filter-a-json-field-in-an-event/267007 "2021-03-11T23:57:26Z")
Hi, I am really new to logstash and was spending quite a few hours working on this but couldn't make any progress so hopefully, someone can point me out in some correct direction. The event eventually in the kabana is l… --- ## [Translate filter dictionary update](https://discuss.elastic.co/t/translate-filter-dictionary-update/267000)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 1\ **Last updated:** [March 11, 2021, 8:55pm UTC](https://discuss.elastic.co/t/translate-filter-dictionary-update/267000 "2021-03-11T20:55:16Z")
I am using logstash pipeline. one of the job has translate filter. if I change translate dictionary file key-value. do I have to restart logstash? or will it read the dictionary yml file everytime it runs pipeline? --- ## [Help with filtering message based on regex](https://discuss.elastic.co/t/help-with-filtering-message-based-on-regex/266582)
**Author:** [@judge](https://discuss.elastic.co/u/judge)\ **Replies:** 2\ **Last updated:** [March 11, 2021, 7:46pm UTC](https://discuss.elastic.co/t/help-with-filtering-message-based-on-regex/266582 "2021-03-11T19:46:12Z")
Hi, I really need some help with logstash. I a trying to filter messages based on content and send them to an alternative server for example: {"type":"syslog","host":"1.1.1.1","@timestamp":"2021-03-08T13:18:55.722Z","m… --- ## [Is there a way to configure filebeat to send certain logs as quick as possible?](https://discuss.elastic.co/t/is-there-a-way-to-configure-filebeat-to-send-certain-logs-as-quick-as-possible/266979)
**Author:** [@dmitryyankowski](https://discuss.elastic.co/u/dmitryyankowski)\ **Replies:** 4\ **Last updated:** [March 11, 2021, 6:45pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-configure-filebeat-to-send-certain-logs-as-quick-as-possible/266979 "2021-03-11T18:45:02Z")
I'm wondering if there is a way to get Filebeat to report specific logs as soon as possible.. rather than waiting for the minimum amount of events to send in a batch. For example: I have a security detection setup, t… --- ## [Logstash read configuration values from properties file](https://discuss.elastic.co/t/logstash-read-configuration-values-from-properties-file/266894)
**Author:** [@adityaPsl](https://discuss.elastic.co/u/adityaPsl)\ **Replies:** 2\ **Last updated:** [March 11, 2021, 6:12pm UTC](https://discuss.elastic.co/t/logstash-read-configuration-values-from-properties-file/266894 "2021-03-11T18:12:30Z")
Hello Team, Could you please help me understand is there an option to keep configuration variables outside of pipeline. for example if i have 4 pipeline with elasticsearch as input and output for this most of the confi… --- ## [Output syslog : limited output size?](https://discuss.elastic.co/t/output-syslog-limited-output-size/266986)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 1\ **Last updated:** [March 11, 2021, 6:00pm UTC](https://discuss.elastic.co/t/output-syslog-limited-output-size/266986 "2021-03-11T18:00:37Z")
Hello ! I have one kafka input to grab winlogbeat logs. Regarding output, I have one elasticsearch output and one syslog output : output { syslog { host =\> "1.2.3.4" port =\> 777 protocol =\> "tcp" codec =\> json … --- ## [Xml filter on nested element](https://discuss.elastic.co/t/xml-filter-on-nested-element/266619)
**Author:** [@StashLogs](https://discuss.elastic.co/u/StashLogs)\ **Replies:** 11\ **Last updated:** [March 11, 2021, 2:06pm UTC](https://discuss.elastic.co/t/xml-filter-on-nested-element/266619 "2021-03-11T14:06:12Z")
Hello, I am using the jdbc streaming plugin to add a new field to a record. This new field is an array of objects ( history ) as seen below: record: { id: "1", someXMLField: \, hi… --- ## [Why kafka consumer always Re-joining group?](https://discuss.elastic.co/t/why-kafka-consumer-always-re-joining-group/266944)
**Author:** [@KeithTt](https://discuss.elastic.co/u/KeithTt)\ **Replies:** 0\ **Last updated:** [March 11, 2021, 12:36pm UTC](https://discuss.elastic.co/t/why-kafka-consumer-always-re-joining-group/266944 "2021-03-11T12:36:11Z")
Logstash version: 6.3.0 Here is the log, and this info is always repeating: \[2021-03-11T20:25:47,359\]\[INFO \]\[org.apache.kafka.clients.consumer.internals.AbstractCoordinator\] \[Consumer clientId=logstash-0, groupId=nginx… --- ## [I need to leave a string field in the date field. I've done several things but I'm not sure how to do it. Follow my files](https://discuss.elastic.co/t/i-need-to-leave-a-string-field-in-the-date-field-ive-done-several-things-but-im-not-sure-how-to-do-it-follow-my-files/266804)
**Author:** [@francieliton\_araujo](https://discuss.elastic.co/u/francieliton_araujo)\ **Replies:** 4\ **Last updated:** [March 11, 2021, 11:43am UTC](https://discuss.elastic.co/t/i-need-to-leave-a-string-field-in-the-date-field-ive-done-several-things-but-im-not-sure-how-to-do-it-follow-my-files/266804 "2021-03-11T11:43:31Z")
I need to leave a string field in the date field. I've done several things but I'm not sure how to do it. Follow my files. input{ file{ path =\> "/etc/logstash/unico/FM099-Registro\\ de\\ Indisponibilidade\\ -\\ 2021.xl… --- ## [How to make solr clod secure](https://discuss.elastic.co/t/how-to-make-solr-clod-secure/266931)
**Author:** [@Arpan\_Jain](https://discuss.elastic.co/u/Arpan_Jain)\ **Replies:** 1\ **Last updated:** [March 11, 2021, 11:28am UTC](https://discuss.elastic.co/t/how-to-make-solr-clod-secure/266931 "2021-03-11T11:28:49Z")
I am using logstash to save events in elastic search and solr cloud. I want to know is there any plugin or something else through which i can do solr basic authentication. --- ## [Distinguishing between log files when sending multiple log files from filebeat to logstash](https://discuss.elastic.co/t/distinguishing-between-log-files-when-sending-multiple-log-files-from-filebeat-to-logstash/266043)
**Author:** [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Replies:** 17\ **Last updated:** [March 11, 2021, 11:27am UTC](https://discuss.elastic.co/t/distinguishing-between-log-files-when-sending-multiple-log-files-from-filebeat-to-logstash/266043 "2021-03-11T11:27:39Z")
I'm trying to collect multiple logs from filebeat and send them to logstash. I would like to use logstash's filter to process each log file individually, but I'm having trouble. For example, first I want to rename the … --- ## [Logstash - Nested fields?](https://discuss.elastic.co/t/logstash-nested-fields/266851)
**Author:** [@SaraC](https://discuss.elastic.co/u/SaraC)\ **Replies:** 6\ **Last updated:** [March 11, 2021, 10:51am UTC](https://discuss.elastic.co/t/logstash-nested-fields/266851 "2021-03-11T10:51:20Z")
Hi, I have the following sample of log: { "@timestamp": "2021-03-10T17:30:58.899Z", "@version": "1", "Actor": \[ { "ID": "Microsoft Intune", "Type": 1 }, { … --- ## [New template doesn't works Geoip](https://discuss.elastic.co/t/new-template-doesnt-works-geoip/266801)
**Author:** [@hoff](https://discuss.elastic.co/u/hoff)\ **Replies:** 5\ **Last updated:** [March 11, 2021, 9:25am UTC](https://discuss.elastic.co/t/new-template-doesnt-works-geoip/266801 "2021-03-11T09:25:04Z")
Hi, I need to create geoip map but in my mapping field location is float type. I have some issue with create new template. I want to change field type from float to geo\_point. When I try to put new template and delete o… --- ## [Can't find Logstash /bin directory](https://discuss.elastic.co/t/cant-find-logstash-bin-directory/266902)
**Author:** [@Muhammad\_Suleman](https://discuss.elastic.co/u/Muhammad_Suleman)\ **Replies:** 0\ **Last updated:** [March 11, 2021, 8:06am UTC](https://discuss.elastic.co/t/cant-find-logstash-bin-directory/266902 "2021-03-11T08:06:22Z")
I did install my ElK stack with the help of docker compose on my bare metal window server 2019. I can't find my logstash bin directory for running my logstash. Thanks, --- ## [Raw message output](https://discuss.elastic.co/t/raw-message-output/266895)
**Author:** [@st1988](https://discuss.elastic.co/u/st1988)\ **Replies:** 0\ **Last updated:** [March 11, 2021, 7:21am UTC](https://discuss.elastic.co/t/raw-message-output/266895 "2021-03-11T07:21:22Z")
Hi, I currently have an issue where I am sending Syslog to Logstash and then forwarding to ELK and via syslog to a SIEM. The SIEM however is struggling to understand the log as think it is being sent in a format its not … --- ## [Installing logstash-output-kusto plugin thru logstash container deployment](https://discuss.elastic.co/t/installing-logstash-output-kusto-plugin-thru-logstash-container-deployment/266684)
**Author:** [@omkarg81](https://discuss.elastic.co/u/omkarg81)\ **Replies:** 1\ **Last updated:** [March 11, 2021, 3:47am UTC](https://discuss.elastic.co/t/installing-logstash-output-kusto-plugin-thru-logstash-container-deployment/266684 "2021-03-11T03:47:26Z")
Could someone please provide guidance on how we can install additional plugins like logstash-output-kusto using a command like bin/logstash-plugin install logstash-output-kusto inside a logstash container? Any example c… --- ## [\[publisher\_pipeline\_output\] pipeline /output.go:154 Failed to connect to backoff(async(tcp://logstash-xxx-xxx.apps.-xxx.xxx:443)): EOF](https://discuss.elastic.co/t/publisher-pipeline-output-pipeline-output-go-154-failed-to-connect-to-backoff-async-tcp-logstash-xxx-xxx-apps-xxx-xxx-443-eof/266451)
**Author:** [@Rohan-boogeyman](https://discuss.elastic.co/u/Rohan-boogeyman)\ **Replies:** 13\ **Last updated:** [March 10, 2021, 7:42pm UTC](https://discuss.elastic.co/t/publisher-pipeline-output-pipeline-output-go-154-failed-to-connect-to-backoff-async-tcp-logstash-xxx-xxx-apps-xxx-xxx-443-eof/266451 "2021-03-10T19:42:50Z")
Hello Guys, Thanks in advance for taking your time to look into the issue i am experiencing. Error in File beat at the time of Debug \[publisher\_pipeline\_output\] pipeline /output.go:154 Failed to connect to backof… --- ## [Create field form file name](https://discuss.elastic.co/t/create-field-form-file-name/266846)
**Author:** [@ManuelF](https://discuss.elastic.co/u/ManuelF)\ **Replies:** 2\ **Last updated:** [March 10, 2021, 7:01pm UTC](https://discuss.elastic.co/t/create-field-form-file-name/266846 "2021-03-10T19:01:47Z")
Hi, Note: Running ELK 7.10.1 I am trying to generate a new field using Grok, by extracting data from the file name. File name comes from "path". path =\> "/opt/app\_logs/\*.log" Example: ip\_block-hgTest.Dev1test-2021-… --- ## [2 different time](https://discuss.elastic.co/t/2-different-time/266640)
**Author:** [@111435](https://discuss.elastic.co/u/111435)\ **Replies:** 4\ **Last updated:** [March 10, 2021, 4:34pm UTC](https://discuss.elastic.co/t/2-different-time/266640 "2021-03-10T16:34:33Z")
Hello guys! Please, I need your advise. From one client server to logstash comming 2 different logs with different time template: \[2021-02-24 00:00:06,335\] \[\[ACTIVE\] ExecuteThread: '121' for queue: 'weblogic.kernel.… --- ## [How to change logstash default timezone](https://discuss.elastic.co/t/how-to-change-logstash-default-timezone/266771)
**Author:** [@sreedhar1](https://discuss.elastic.co/u/sreedhar1)\ **Replies:** 1\ **Last updated:** [March 10, 2021, 3:53pm UTC](https://discuss.elastic.co/t/how-to-change-logstash-default-timezone/266771 "2021-03-10T15:53:04Z")
I am using Logstash to populate data to Elasticsearch server. By default Elasticsearch using UTC format to populate the data. How to change UTC to local time zone? --- ## [Logstash not using bundled JDK](https://discuss.elastic.co/t/logstash-not-using-bundled-jdk/266814)
**Author:** [@Daniel\_Carmel](https://discuss.elastic.co/u/Daniel_Carmel)\ **Replies:** 1\ **Last updated:** [March 10, 2021, 3:47pm UTC](https://discuss.elastic.co/t/logstash-not-using-bundled-jdk/266814 "2021-03-10T15:47:59Z")
Hey Elastic team, im using logstash 7.11.0 and ubuntu 14.04 Whenever i run it manually like this sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash logstash running as expected and using bundled JDK,… --- ## [Logstash file input reading partial lines with docker setup](https://discuss.elastic.co/t/logstash-file-input-reading-partial-lines-with-docker-setup/266798)
**Author:** [@jong99](https://discuss.elastic.co/u/jong99)\ **Replies:** 0\ **Last updated:** [March 10, 2021, 10:47am UTC](https://discuss.elastic.co/t/logstash-file-input-reading-partial-lines-with-docker-setup/266798 "2021-03-10T10:47:46Z")
I've been having difficulty with the logstash input filter sometimes picking up partial lines with my setup. I think it may be related to how I am using docker but I'm running out of ideas. I have logstash in a docker … --- ## [Cannot link Java class io.netty.handler.ssl.OpenSsl](https://discuss.elastic.co/t/cannot-link-java-class-io-netty-handler-ssl-openssl/266790)
**Author:** [@zarak](https://discuss.elastic.co/u/zarak)\ **Replies:** 0\ **Last updated:** [March 10, 2021, 10:01am UTC](https://discuss.elastic.co/t/cannot-link-java-class-io-netty-handler-ssl-openssl/266790 "2021-03-10T10:01:28Z")
Hello ! I'm running Logstash 6.8.14 on debian. It's running fine, until I try to send a SIHGUP to reload the pipelines and config. I now have this in the logs, restarting it doesn't solve the issue: Mar 10 10:51:39 hos… --- ## [Every line of Logs is not passing through](https://discuss.elastic.co/t/every-line-of-logs-is-not-passing-through/266698)
**Author:** [@Saravana37](https://discuss.elastic.co/u/Saravana37)\ **Replies:** 2\ **Last updated:** [March 10, 2021, 5:25am UTC](https://discuss.elastic.co/t/every-line-of-logs-is-not-passing-through/266698 "2021-03-10T05:25:29Z")
Hello All , I have below sample line in from the log file which is not passing through logstash. I am not sure where the problem is , I have the below log line 2021-02-24 10:46:29,827 INFO \[5b5021c6-6af1-4e70-9cfb-f0… --- ## [/\_template OR /\_index\_template?](https://discuss.elastic.co/t/template-or-index-template/266567)
**Author:** [@thesn](https://discuss.elastic.co/u/thesn)\ **Replies:** 4\ **Last updated:** [March 10, 2021, 4:20am UTC](https://discuss.elastic.co/t/template-or-index-template/266567 "2021-03-10T04:20:09Z")
Hi, I am using ES 7.11.1 and Logstash 7.11.1 in Docker container. here is my logstash.conf: input { file { codec =\> "json" path =\> "/usr/share/logstash/config/sample.log" } } filter { } output { el… --- ## [KV Filter Regex](https://discuss.elastic.co/t/kv-filter-regex/266730)
**Author:** [@lumi](https://discuss.elastic.co/u/lumi)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 6:13pm UTC](https://discuss.elastic.co/t/kv-filter-regex/266730 "2021-03-09T18:13:28Z")
Hello I have the following message getting from Qradar: ( I have changed some things like ID, logon etc...) A network share object was accessed. Subject: Security ID: ELK\\\\test-test$ Account Name: test-test$ Acc… --- ## [Convert field containing a timestamp from string to a date / time or timestamp?](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594)
**Author:** [@riahc3](https://discuss.elastic.co/u/riahc3)\ **Replies:** 47\ **Last updated:** [March 9, 2021, 5:34pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594 "2021-03-09T17:34:55Z")
Hello As you can observe here, I have a two fields containing date and time: Id like to change it so Elastic stores it as a date/time and maybe even change the order (yyyy/mm/dd hh/mm ) How can I change this? My lo… --- ## [Logstash ternary condition when a winlogbeat field doesn't exist](https://discuss.elastic.co/t/logstash-ternary-condition-when-a-winlogbeat-field-doesnt-exist/266721)
**Author:** [@mguttula](https://discuss.elastic.co/u/mguttula)\ **Replies:** 4\ **Last updated:** [March 9, 2021, 5:17pm UTC](https://discuss.elastic.co/t/logstash-ternary-condition-when-a-winlogbeat-field-doesnt-exist/266721 "2021-03-09T17:17:04Z")
Logstash is sending winlogbeat fields to Spunk and when the fields don't exist it is send something like %{\[winlog\]\[event\_data\]\[AuthenticationPackageName\]} I am trying to find a way to either not send the field if it do… --- ## [Sync data from mongodb](https://discuss.elastic.co/t/sync-data-from-mongodb/266719)
**Author:** [@Gharsa\_Khouloud](https://discuss.elastic.co/u/Gharsa_Khouloud)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 4:24pm UTC](https://discuss.elastic.co/t/sync-data-from-mongodb/266719 "2021-03-09T16:24:43Z")
I just want to sync the data from mongodb to elastic using logstash. Its working good, when any new record comes in mongodb, logstash pushes into elastic. But when I update any record in mongodb then it does not change i… --- ## [How to make difference between 2 source of logs in Logstash input](https://discuss.elastic.co/t/how-to-make-difference-between-2-source-of-logs-in-logstash-input/266709)
**Author:** [@Abdelhalim](https://discuss.elastic.co/u/Abdelhalim)\ **Replies:** 1\ **Last updated:** [March 9, 2021, 3:35pm UTC](https://discuss.elastic.co/t/how-to-make-difference-between-2-source-of-logs-in-logstash-input/266709 "2021-03-09T15:35:50Z")
Hello everyone, I want to send log of the "auth.log" file using system module of filebeat, and I have another file that I want to parse using logstash (as there is no module for that type of log). And as I know, there … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=247) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=249)