# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=259 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 260 --- ## [Create arrays in Logstash](https://discuss.elastic.co/t/create-arrays-in-logstash/262260)
**Author:** [@gneves](https://discuss.elastic.co/u/gneves)\ **Replies:** 4\ **Last updated:** [February 1, 2021, 12:43pm UTC](https://discuss.elastic.co/t/create-arrays-in-logstash/262260 "2021-02-01T12:43:27Z")
Hello guys. Dont know if it is possible, but i would like to create an array in my logstash to use in an if statement. ill give an example for you to understand me. Giving a random "log to be processed" as example: ra… --- ## [JMX Input Plugin Config metrics on sub path (host:port/jmx)](https://discuss.elastic.co/t/jmx-input-plugin-config-metrics-on-sub-path-host-port-jmx/262830)
**Author:** [@Andy\_Barber1](https://discuss.elastic.co/u/Andy_Barber1)\ **Replies:** 0\ **Last updated:** [February 1, 2021, 12:24pm UTC](https://discuss.elastic.co/t/jmx-input-plugin-config-metrics-on-sub-path-host-port-jmx/262830 "2021-02-01T12:24:19Z")
Hi I am looking to do some basic JMX connection using logstash and the JMX-input plugin. basic platform stuff (elastic kibana and logstash) are working ok. the Input plugin configuration had parameters for host and port… --- ## [If-in statement does not work if array has one object \[SOLVED\]](https://discuss.elastic.co/t/if-in-statement-does-not-work-if-array-has-one-object-solved/262818)
**Author:** [@dorinand](https://discuss.elastic.co/u/dorinand)\ **Replies:** 0\ **Last updated:** [February 1, 2021, 10:30am UTC](https://discuss.elastic.co/t/if-in-statement-does-not-work-if-array-has-one-object-solved/262818 "2021-02-01T10:30:16Z")
Hi, I prepare this question and also find solution after few hours, so I decide to upload question and answer, maybe it will help somebody: QUESTION: I have Filebeat in k8s that sends logs to Logstash. I have namespace… --- ## [Logstash not reading log file but reads text file](https://discuss.elastic.co/t/logstash-not-reading-log-file-but-reads-text-file/262817)
**Author:** [@vighnesh\_mahadik](https://discuss.elastic.co/u/vighnesh_mahadik)\ **Replies:** 0\ **Last updated:** [February 1, 2021, 10:11am UTC](https://discuss.elastic.co/t/logstash-not-reading-log-file-but-reads-text-file/262817 "2021-02-01T10:11:22Z")
The contents of my log file is same but it does not read the file in .log extension but is able to read it in .txt extension. reads .txt file and shows on console as well as kibana Config for reading .txt file :- input… --- ## [Logstash file output plugin writing to the same file from multiple Logstash processes?!](https://discuss.elastic.co/t/logstash-file-output-plugin-writing-to-the-same-file-from-multiple-logstash-processes/261750)
**Author:** [@bjosve](https://discuss.elastic.co/u/bjosve)\ **Replies:** 2\ **Last updated:** [February 1, 2021, 9:18am UTC](https://discuss.elastic.co/t/logstash-file-output-plugin-writing-to-the-same-file-from-multiple-logstash-processes/261750 "2021-02-01T09:18:27Z")
Hi, In the following post, https://discuss.elastic.co/t/multiple-logstash-nodes-use-file-output-plugin-to-output-messages-to-one-file-in-a-shared-file-system/94275/9 @magnusbaeck writes, If Logstash is opening the o… --- ## [Logstash ‘77⚠constant::Fixnum is deprecated’ and not work](https://discuss.elastic.co/t/logstash-77constant-fixnum-is-deprecated-and-not-work/262792)
**Author:** [@plumy0902](https://discuss.elastic.co/u/plumy0902)\ **Replies:** 0\ **Last updated:** [February 1, 2021, 6:49am UTC](https://discuss.elastic.co/t/logstash-77constant-fixnum-is-deprecated-and-not-work/262792 "2021-02-01T06:49:15Z")
Hi I was trying to send data to elasticsearch using scheduler. I want it to run once a day and update new records. But it gets stuck after these outputs. Please suggest any solution, thanks. (ps : logstash version is 7.… --- ## [Unable to listen on port 5044 for logstash on unbuntu 20.04](https://discuss.elastic.co/t/unable-to-listen-on-port-5044-for-logstash-on-unbuntu-20-04/262664)
**Author:** [@Paul\_Fleming](https://discuss.elastic.co/u/Paul_Fleming)\ **Replies:** 1\ **Last updated:** [February 1, 2021, 12:48am UTC](https://discuss.elastic.co/t/unable-to-listen-on-port-5044-for-logstash-on-unbuntu-20-04/262664 "2021-02-01T00:48:18Z")
Using Ubuntu 20.04 I have confirmed OpenSSL is running and then I have installed logstash by running the following commands. \` openssl version -a apt install logstash -y Edit the etc/hosts file and add the following … --- ## [Location type geo\_point](https://discuss.elastic.co/t/location-type-geo-point/262059)
**Author:** [@Elves\_Ribeiro](https://discuss.elastic.co/u/Elves_Ribeiro)\ **Replies:** 2\ **Last updated:** [January 31, 2021, 10:48pm UTC](https://discuss.elastic.co/t/location-type-geo-point/262059 "2021-01-31T22:48:12Z")
Hi, I am trying to create a map on my environment, but I am not able to create the field location type geo\_point. I am using version 7.10.2 My logstash conf geoip { source =\> "ip\_client" } mutate { add\_field =\> … --- ## [Logstash 7.2.0 install error](https://discuss.elastic.co/t/logstash-7-2-0-install-error/188356)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 7\ **Last updated:** [January 29, 2021, 11:33pm UTC](https://discuss.elastic.co/t/logstash-7-2-0-install-error/188356 "2021-01-29T23:33:41Z")
I was upgrading whole stack to 7.2.0 and got following error Updating / installing... 1:logstash-1:7.2.0-1 ################################# \[ 17%\] Using provided startup.options file: /etc/logstash/star… --- ## [JSON Parsing Of Filtered Syslog JSON Objects Fails](https://discuss.elastic.co/t/json-parsing-of-filtered-syslog-json-objects-fails/261959)
**Author:** [@InfosecAtom](https://discuss.elastic.co/u/InfosecAtom)\ **Replies:** 2\ **Last updated:** [January 29, 2021, 10:42pm UTC](https://discuss.elastic.co/t/json-parsing-of-filtered-syslog-json-objects-fails/261959 "2021-01-29T22:42:45Z")
I am using the following config and seeing no output. Non-JSON object containing Syslog entries, as well as JSON object containing Syslog entries, are both being dropped. If I comment out the JSON parse then the JSON obj… --- ## [Logstash is not ingesting data in real time](https://discuss.elastic.co/t/logstash-is-not-ingesting-data-in-real-time/262642)
**Author:** [@shoaib\_akram](https://discuss.elastic.co/u/shoaib_akram)\ **Replies:** 1\ **Last updated:** [January 29, 2021, 8:23pm UTC](https://discuss.elastic.co/t/logstash-is-not-ingesting-data-in-real-time/262642 "2021-01-29T20:23:11Z")
Hi Everyone, I am ingesting data from thousands of file with logstash into elasticsearch and files are parallely updating and updating records are not ingesting in real time. Please help me out what i need to change se… --- ## [Redis in elk](https://discuss.elastic.co/t/redis-in-elk/262076)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 7\ **Last updated:** [January 29, 2021, 3:22pm UTC](https://discuss.elastic.co/t/redis-in-elk/262076 "2021-01-29T15:22:21Z")
Hi, I have this elk stack running, 8 datanodes, 4 ingestnodes, 4 logstashes en a shitload of applications that send data from multiple servers via filebeat. The size of these applications vary from 4 servers to 24 on … --- ## [Split field and add new fields](https://discuss.elastic.co/t/split-field-and-add-new-fields/262571)
**Author:** [@jtillman2020](https://discuss.elastic.co/u/jtillman2020)\ **Replies:** 3\ **Last updated:** [January 29, 2021, 5:01pm UTC](https://discuss.elastic.co/t/split-field-and-add-new-fields/262571 "2021-01-29T17:01:51Z")
I have a string field named "One". \<165\>Original Address=1.1.1.1 1 2021-01-28T15:08:03.136-05:00 DeviceName - - - - fpc3 DHCP Packet Drop: Packet src ip/mac 192.168.123.1/54:48:10:db:86:b6 I use mutate { split =\> … --- ## [Grok not parsing any pattern](https://discuss.elastic.co/t/grok-not-parsing-any-pattern/262669)
**Author:** [@werther158](https://discuss.elastic.co/u/werther158)\ **Replies:** 0\ **Last updated:** [January 29, 2021, 4:14pm UTC](https://discuss.elastic.co/t/grok-not-parsing-any-pattern/262669 "2021-01-29T16:14:02Z")
Hi, I'm trying to parse some apache logs like this: 46.105.14.53 - - \[20/May/2015:21:05:15 +0000\] "GET /blog/tags/puppet?flav=rss20 HTTP/1.1" 200 14872 "-" "UniversalFeedParser/4.2-pre-314-svn +http://feedparser.org/" F… --- ## [Grok pattern to parse to multiple values](https://discuss.elastic.co/t/grok-pattern-to-parse-to-multiple-values/262651)
**Author:** [@Deny7](https://discuss.elastic.co/u/Deny7)\ **Replies:** 5\ **Last updated:** [January 29, 2021, 3:49pm UTC](https://discuss.elastic.co/t/grok-pattern-to-parse-to-multiple-values/262651 "2021-01-29T15:49:29Z")
Hi, I have log line like this: ",session":"kred06@gmail.com" Grok Pattern: (,"session":"(%{DATA:name}@%{DATA:company})?")? and I want to split the email to values name and company and also save email as whole to s… --- ## [Masking logic is not working](https://discuss.elastic.co/t/masking-logic-is-not-working/261827)
**Author:** [@shree2](https://discuss.elastic.co/u/shree2)\ **Replies:** 10\ **Last updated:** [January 29, 2021, 3:31pm UTC](https://discuss.elastic.co/t/masking-logic-is-not-working/261827 "2021-01-29T15:31:49Z")
Hi, I want to mask the few fields input is coming from json . Please find the configuration below. input { file { path =\> "xxx/sample.log" } } filter { mutate { gsub =\> \["message", "PASSWORD:((?=.\[a-z\])(?=.\[… --- ## [Persistent queue and kafka](https://discuss.elastic.co/t/persistent-queue-and-kafka/262662)
**Author:** [@bovy89](https://discuss.elastic.co/u/bovy89)\ **Replies:** 0\ **Last updated:** [January 29, 2021, 3:31pm UTC](https://discuss.elastic.co/t/persistent-queue-and-kafka/262662 "2021-01-29T15:31:35Z")
Hi, we are using logstash with persistent queue feature enabled and the following setup: filebeat ---\> logstash (with PQ) ---\> elasticsearch Now we are moving to a kafka based setup like that: filebeat ---\> kafka ---… --- ## [Not parsing apache access and error logs](https://discuss.elastic.co/t/not-parsing-apache-access-and-error-logs/262535)
**Author:** [@kevintts](https://discuss.elastic.co/u/kevintts)\ **Replies:** 2\ **Last updated:** [January 29, 2021, 2:58pm UTC](https://discuss.elastic.co/t/not-parsing-apache-access-and-error-logs/262535 "2021-01-29T14:58:49Z")
I am using filebeat to send apache logs to logstash on my elastic server. I can see the log entries with event.dataset set to apache.access , but they aren't being parsed, so no info in dashboard apache screens. Also, wh… --- ## [Many or single pipeline performance with querys](https://discuss.elastic.co/t/many-or-single-pipeline-performance-with-querys/262650)
**Author:** [@pereyrdi](https://discuss.elastic.co/u/pereyrdi)\ **Replies:** 0\ **Last updated:** [January 29, 2021, 1:18pm UTC](https://discuss.elastic.co/t/many-or-single-pipeline-performance-with-querys/262650 "2021-01-29T13:18:21Z")
Hello everyone, Im working updating configurations of logstash and Im wondering with method is more performance. For example Multiple pipelines going to a single pipepile to do some checks (sql querys) before ingest i… --- ## [Streaming pipeline](https://discuss.elastic.co/t/streaming-pipeline/262628)
**Author:** [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)\ **Replies:** 0\ **Last updated:** [January 29, 2021, 10:25am UTC](https://discuss.elastic.co/t/streaming-pipeline/262628 "2021-01-29T10:25:35Z")
Hi experts! I made a data pipeline using Filebeat and Logstash. I was expecting to see the data streamed from the log file into Elasticsearch but it's not the case. When new lines are inserted into the log file, I some… --- ## [Reg csv file import into Elasticsearch using Logstash](https://discuss.elastic.co/t/reg-csv-file-import-into-elasticsearch-using-logstash/261449)
**Author:** [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Replies:** 12\ **Last updated:** [January 29, 2021, 9:06am UTC](https://discuss.elastic.co/t/reg-csv-file-import-into-elasticsearch-using-logstash/261449 "2021-01-29T09:06:04Z")
Hello Support team, I have query regarding csv file upload into elasticsearch using logstash. With this fourm https://www.bmc.com/blogs/elasticsearch-load-csv-logstash/ Able to import our csv files into elasticsearch … --- ## [How to specify subpath to logstash host in filebeat.yml](https://discuss.elastic.co/t/how-to-specify-subpath-to-logstash-host-in-filebeat-yml/262501)
**Author:** [@eeijlar](https://discuss.elastic.co/u/eeijlar)\ **Replies:** 2\ **Last updated:** [January 28, 2021, 4:24pm UTC](https://discuss.elastic.co/t/how-to-specify-subpath-to-logstash-host-in-filebeat-yml/262501 "2021-01-28T16:24:44Z")
I am trying to ship logs to logstash from filebeat. With reference to: In the filebeat.yml, it says to use this for the host: output.logstash: hosts: \["127.0.0.1:5044"\] My logstash URL has /logstash appending o… --- ## [Multiple indices in kibana dashboard](https://discuss.elastic.co/t/multiple-indices-in-kibana-dashboard/261751)
**Author:** [@Srikanth\_Kesireddy](https://discuss.elastic.co/u/Srikanth_Kesireddy)\ **Replies:** 3\ **Last updated:** [January 29, 2021, 5:49am UTC](https://discuss.elastic.co/t/multiple-indices-in-kibana-dashboard/261751 "2021-01-29T05:49:56Z")
I have 10 target servers installed Filebeat on all machines... Logstash , ElasticSearch , Kibana these 3 are installed on main server. How can I create 10 indices for 10 target machines --- ## [How to escape sprintf (to compare against an uninterpolated value)](https://discuss.elastic.co/t/how-to-escape-sprintf-to-compare-against-an-uninterpolated-value/262581)
**Author:** [@cknz](https://discuss.elastic.co/u/cknz)\ **Replies:** 2\ **Last updated:** [January 29, 2021, 3:59am UTC](https://discuss.elastic.co/t/how-to-escape-sprintf-to-compare-against-an-uninterpolated-value/262581 "2021-01-29T03:59:43Z")
Hi all, I'm on Elastic 7.10.2, and I noticed a problem with my processing. Some incoming data has tried to set a field using a sprintf string, but the sprintf expression did not result in a replacement, so I'm left with … --- ## [Filebeat Error: connect: connection refused](https://discuss.elastic.co/t/filebeat-error-connect-connection-refused/260866)
**Author:** [@haithem899](https://discuss.elastic.co/u/haithem899)\ **Replies:** 4\ **Last updated:** [January 28, 2021, 10:48pm UTC](https://discuss.elastic.co/t/filebeat-error-connect-connection-refused/260866 "2021-01-28T22:48:21Z")
Hello, i've setup ELK stack with three nodes ( kibana-elasticsearch-logstash) separating however i can't recieve logs and with checking filebeat log i see this error: "connect: connection refused". Even i increased clie… --- ## [Logstash, Date plugin not working as expected](https://discuss.elastic.co/t/logstash-date-plugin-not-working-as-expected/262525)
**Author:** [@sguerrero](https://discuss.elastic.co/u/sguerrero)\ **Replies:** 4\ **Last updated:** [January 28, 2021, 6:14pm UTC](https://discuss.elastic.co/t/logstash-date-plugin-not-working-as-expected/262525 "2021-01-28T18:14:38Z")
Logstash Version: 7.10.1 OS: Debian 10 Buster Architecture: x86-64 Kernel: Linux 4.19.0-11-amd64 I'm trying to parse this date: 20210128 94501065 The pattern is: date { match =\> \["timestamp", "yyyyM… --- ## [Logstash grok](https://discuss.elastic.co/t/logstash-grok/262498)
**Author:** [@igormarqs](https://discuss.elastic.co/u/igormarqs)\ **Replies:** 1\ **Last updated:** [January 28, 2021, 5:23pm UTC](https://discuss.elastic.co/t/logstash-grok/262498 "2021-01-28T17:23:20Z")
hey all; I have this log line: TID: \[\] \[\] \[2021-01-28 12:25:50,298\] INFO {org.wso2.carbon.databridge.core.DataBridge} - user admin connected {org.wso2.carbon.databridge.core.DataBridge} and i use this grok: TID: \\\[\] … --- ## [Optional grok pattern field doesnt match](https://discuss.elastic.co/t/optional-grok-pattern-field-doesnt-match/262483)
**Author:** [@Deny7](https://discuss.elastic.co/u/Deny7)\ **Replies:** 1\ **Last updated:** [January 28, 2021, 5:20pm UTC](https://discuss.elastic.co/t/optional-grok-pattern-field-doesnt-match/262483 "2021-01-28T17:20:48Z")
Hi, my log looks like this: "rootModel":"modelR","providedByRM":false,"rm\_user":"100001935398@rm.com","publishers":\["Unknown"\] the part: ,"rm\_user":"100001935398@rm.com" is optional so i marked it with ()?, but it do… --- ## [Logstash Error](https://discuss.elastic.co/t/logstash-error/262392)
**Author:** [@PriyankaS](https://discuss.elastic.co/u/PriyankaS)\ **Replies:** 3\ **Last updated:** [January 28, 2021, 5:18pm UTC](https://discuss.elastic.co/t/logstash-error/262392 "2021-01-28T17:18:43Z")
I am trying run the logstash with the below pattern: .\* PulseSecure: .\*%{IP:\[additionalinfo\]\[client\_public\_ip\]}.\*\\\\%{USERNAME:\[additionalinfo\]\[suser\]}.\*%{IP:\[additionalinfo\]\[client\_ip\]} for the below sample log: \<182\>1… --- ## [Logstash Json remove nested properties that have empty field name](https://discuss.elastic.co/t/logstash-json-remove-nested-properties-that-have-empty-field-name/262516)
**Author:** [@david.preston](https://discuss.elastic.co/u/david.preston)\ **Replies:** 1\ **Last updated:** [January 28, 2021, 4:40pm UTC](https://discuss.elastic.co/t/logstash-json-remove-nested-properties-that-have-empty-field-name/262516 "2021-01-28T16:40:08Z")
Hi, Are there any "out of the box" logstash functions for removing json properties that have empty field names. e.g. { "event": { "data": { "" : "some value" } } } I'm not a… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=258) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=260)