# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=582 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 583 --- ## [Logstash Geo IP address wrong country code](https://discuss.elastic.co/t/logstash-geo-ip-address-wrong-country-code/172841)
**Author:** [@naftalin\_n](https://discuss.elastic.co/u/naftalin_n)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 5:57pm UTC](https://discuss.elastic.co/t/logstash-geo-ip-address-wrong-country-code/172841 "2019-03-18T17:57:29Z")
hi all, i tried to check the geoip filter with VPN, my remote address is 77.111.246.62 which is an American ip. the problem is that the country code im getting from that filter is NO (Norway) "geoip" : { "i… --- ## [Remove an array field that has empty objects inside](https://discuss.elastic.co/t/remove-an-array-field-that-has-empty-objects-inside/172828)
**Author:** [@bigster](https://discuss.elastic.co/u/bigster)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 4:32pm UTC](https://discuss.elastic.co/t/remove-an-array-field-that-has-empty-objects-inside/172828 "2019-03-18T16:32:01Z")
Hi all, I've got a field that is an array of objects, but the object is null: And when this situation happens i want to remove the field. After several tries with ruby filter, conditional ifs, i can't figure out how… --- ## [Logstash authentication in rabbitMQ: PKIX path building failed](https://discuss.elastic.co/t/logstash-authentication-in-rabbitmq-pkix-path-building-failed/172816)
**Author:** [@wertu\_oz](https://discuss.elastic.co/u/wertu_oz)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 4:09pm UTC](https://discuss.elastic.co/t/logstash-authentication-in-rabbitmq-pkix-path-building-failed/172816 "2019-03-18T16:09:19Z")
Hi guys, Help me set up logstash authentication in rabbitMQ using a certificate. Input config: rabbitmq { id =\> "Inspector.IncomingEvents" host =\> 'server1' vhost =\> "TEST" queue =\> "Inspector.IncomingEvents" dur… --- ## [Flush pattern in multiline](https://discuss.elastic.co/t/flush-pattern-in-multiline/172786)
**Author:** [@Sanriya\_Krimzn](https://discuss.elastic.co/u/Sanriya_Krimzn)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 1:49pm UTC](https://discuss.elastic.co/t/flush-pattern-in-multiline/172786 "2019-03-18T13:49:02Z")
Hi I am Doing Data Transformation in Logstash. I need to parse a block of pattern in Logstash. I am using multiine given below multiline { pattern =\> "^\[" #Start of Block what =\> "previous" negate=\> true } Now I n… --- ## [Doubt Regarding Data Extraction From Same Line - Logstash](https://discuss.elastic.co/t/doubt-regarding-data-extraction-from-same-line-logstash/172139)
**Author:** [@Sanriya\_Krimzn](https://discuss.elastic.co/u/Sanriya_Krimzn)\ **Replies:** 8\ **Last updated:** [March 18, 2019, 1:17pm UTC](https://discuss.elastic.co/t/doubt-regarding-data-extraction-from-same-line-logstash/172139 "2019-03-18T13:17:30Z")
Hi , I am extracting information from log and creating fields. Sample Log Line: Keepalive set (10 sec) aliveset (9 sec) Now how can I parse the above line to get two rows like { "parameter" =\> "Keepalive set", "du… --- ## [Logstash file output crashes when writing to tamper proof storage](https://discuss.elastic.co/t/logstash-file-output-crashes-when-writing-to-tamper-proof-storage/172780)
**Author:** [@bjosve](https://discuss.elastic.co/u/bjosve)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 1:19pm UTC](https://discuss.elastic.co/t/logstash-file-output-crashes-when-writing-to-tamper-proof-storage/172780 "2019-03-18T13:19:34Z")
Hi. We are using NetApp NFS mounts on RedHat 7.6. The NetApp storage is setup as tamper proof storage with SnapLock Volume Append Mode (VAM) activated. The Logstash file output can write for a short while, then crashes … --- ## [Parsing Windows Event Logs with nested xml](https://discuss.elastic.co/t/parsing-windows-event-logs-with-nested-xml/172500)
**Author:** [@jmohar](https://discuss.elastic.co/u/jmohar)\ **Replies:** 4\ **Last updated:** [March 18, 2019, 1:18pm UTC](https://discuss.elastic.co/t/parsing-windows-event-logs-with-nested-xml/172500 "2019-03-18T13:18:09Z")
Dear community. After spending more than a week with logstash and winlogbeat to parse Windows\_Event\_Logs..but have no clue how to parse the nested xml in the security windows eventlogs (vistaeventlogs). Using Logstash … --- ## [Logstash error : Pipelines.yml ignored on logstash?](https://discuss.elastic.co/t/logstash-error-pipelines-yml-ignored-on-logstash/172153)
**Author:** [@Danyal\_Tarik](https://discuss.elastic.co/u/Danyal_Tarik)\ **Replies:** 5\ **Last updated:** [March 18, 2019, 1:02pm UTC](https://discuss.elastic.co/t/logstash-error-pipelines-yml-ignored-on-logstash/172153 "2019-03-18T13:02:32Z")
I have tried solution like commenting out path.config in my logstash.yml, but its still not working. Kindly tell if there is any solution to this issue --- ## [How does logstash identify files?](https://discuss.elastic.co/t/how-does-logstash-identify-files/172688)
**Author:** [@GiteshKhanna](https://discuss.elastic.co/u/GiteshKhanna)\ **Replies:** 3\ **Last updated:** [March 18, 2019, 12:14pm UTC](https://discuss.elastic.co/t/how-does-logstash-identify-files/172688 "2019-03-18T12:14:27Z")
How does logstash identify files? By using inode or just the filename? --- ## [Help with grok pattern and regex](https://discuss.elastic.co/t/help-with-grok-pattern-and-regex/172749)
**Author:** [@rfcgraciano](https://discuss.elastic.co/u/rfcgraciano)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 11:00am UTC](https://discuss.elastic.co/t/help-with-grok-pattern-and-regex/172749 "2019-03-18T11:00:43Z")
Hello, I'm looking for some help with my grok patterns, since I'm not being able to parse one of the messages that I need: The message is: "Variable: instrucoes | Value: some text here\\r\\n | Path: \["POST","json","para… --- ## [Unable to read logs from S3 storage](https://discuss.elastic.co/t/unable-to-read-logs-from-s3-storage/172746)
**Author:** [@Meiyappan\_Kannappa](https://discuss.elastic.co/u/Meiyappan_Kannappa)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 10:51am UTC](https://discuss.elastic.co/t/unable-to-read-logs-from-s3-storage/172746 "2019-03-18T10:51:04Z")
Hi, I am trying to read logs from internal S3 storage using logstash. Below is the config. But When i try it always gives no files found in bucket. Through explorer i could see the files. Can someone help me on this. I … --- ## [Logstash won't initialize (Security Onion)](https://discuss.elastic.co/t/logstash-wont-initialize-security-onion/172737)
**Author:** [@Svenson10](https://discuss.elastic.co/u/Svenson10)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 9:48am UTC](https://discuss.elastic.co/t/logstash-wont-initialize-security-onion/172737 "2019-03-18T09:48:32Z")
Hi there Logstash on my storage node is in a WARN state, but I dont know why. sudo so-status output is below. Additional information: - installation from Security Onion 16.04.5.6 ISO image - Distributed deployment (M… --- ## [\[Need Help \]Filebeat and Logstash in multiple instances](https://discuss.elastic.co/t/need-help-filebeat-and-logstash-in-multiple-instances/172288)
**Author:** [@avj1986](https://discuss.elastic.co/u/avj1986)\ **Replies:** 2\ **Last updated:** [March 18, 2019, 6:33am UTC](https://discuss.elastic.co/t/need-help-filebeat-and-logstash-in-multiple-instances/172288 "2019-03-18T06:33:10Z")
Hello there, I am new to ELK stack. We are designing a system wherein we will be checking the performance of our product via product logs on a weekly basis and draw some visualizations on Kibana dashboard. For this re… --- ## [Logstash stops processing the logs after sometime for tcp input](https://discuss.elastic.co/t/logstash-stops-processing-the-logs-after-sometime-for-tcp-input/169818)
**Author:** [@vsskrishna](https://discuss.elastic.co/u/vsskrishna)\ **Replies:** 4\ **Last updated:** [March 18, 2019, 6:23am UTC](https://discuss.elastic.co/t/logstash-stops-processing-the-logs-after-sometime-for-tcp-input/169818 "2019-03-18T06:23:51Z")
Hi, In my ELK stack(5.6.1), I have configured my logstash to listen on both TCP(5000) and filebeat(5044) input. Logs coming from filebeat is working fine but logs coming from TCP stops after some time without any error… --- ## [When does the PULSAR plugin can be developed?](https://discuss.elastic.co/t/when-does-the-pulsar-plugin-can-be-developed/171660)
**Author:** [@z11220857](https://discuss.elastic.co/u/z11220857)\ **Replies:** 2\ **Last updated:** [March 18, 2019, 2:51am UTC](https://discuss.elastic.co/t/when-does-the-pulsar-plugin-can-be-developed/171660 "2019-03-18T02:51:13Z")
when does the PULSAR plugin can be developed? And how could I develop the plugin ?If I can develop it by myself,how to commit it ? --- ## [Haproxy format changes when sending directly](https://discuss.elastic.co/t/haproxy-format-changes-when-sending-directly/172681)
**Author:** [@10acc](https://discuss.elastic.co/u/10acc)\ **Replies:** 1\ **Last updated:** [March 17, 2019, 5:34pm UTC](https://discuss.elastic.co/t/haproxy-format-changes-when-sending-directly/172681 "2019-03-17T17:34:39Z")
Hello, I am using logstash to ship haproxy logs to my elk stack. MY haproxy configuration - global log 127.0.0.1:10514 local0 chroot /var/lib/haproxy user haproxy group haproxy maxconn 555… --- ## [Where are the Logstash Logs?](https://discuss.elastic.co/t/where-are-the-logstash-logs/172645)
**Author:** [@GiteshKhanna](https://discuss.elastic.co/u/GiteshKhanna)\ **Replies:** 3\ **Last updated:** [March 17, 2019, 3:41pm UTC](https://discuss.elastic.co/t/where-are-the-logstash-logs/172645 "2019-03-17T15:41:00Z")
I was wondering where can I find the logs of Logstash itself? I need to check if logstash is working as expected. Is there any default place where logstash stores its logs? --- ## [Logrotate , Logstash Syncing](https://discuss.elastic.co/t/logrotate-logstash-syncing/172661)
**Author:** [@GiteshKhanna](https://discuss.elastic.co/u/GiteshKhanna)\ **Replies:** 0\ **Last updated:** [March 17, 2019, 8:41am UTC](https://discuss.elastic.co/t/logrotate-logstash-syncing/172661 "2019-03-17T08:41:10Z")
I am trying to configure logstash to support logrotate. I am only giving the path of the log file being truncated after each logrotate. So whenever a new file is generated after copying(by logrotate), I am assuming log… --- ## [CSV filter in logstash question](https://discuss.elastic.co/t/csv-filter-in-logstash-question/172649)
**Author:** [@johnfowles](https://discuss.elastic.co/u/johnfowles)\ **Replies:** 4\ **Last updated:** [March 16, 2019, 8:08pm UTC](https://discuss.elastic.co/t/csv-filter-in-logstash-question/172649 "2019-03-16T20:08:20Z")
How would you add a tag to a column that has been parsed by the CSV plugin? For instance, I have the following and it gets all the columns indexed, but it does not add tags to the matching \[sourcetype\] regex. filter { … --- ## [Cannot be changed from type \[long\] to \[float\] exception in logstash jdbc plugin using](https://discuss.elastic.co/t/cannot-be-changed-from-type-long-to-float-exception-in-logstash-jdbc-plugin-using/172260)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 2\ **Last updated:** [March 16, 2019, 8:51am UTC](https://discuss.elastic.co/t/cannot-be-changed-from-type-long-to-float-exception-in-logstash-jdbc-plugin-using/172260 "2019-03-16T08:51:43Z")
Hi all, I am using jdbc plugin to load data from database and merge it with a an created index. my logstash configuration is as following: input { jdbc { jdbc\_driver\_library =\> "sqljdbc42.jar" jdbc\_driver\_class… --- ## [Plugin directory](https://discuss.elastic.co/t/plugin-directory/172626)
**Author:** [@Michael.Swiss](https://discuss.elastic.co/u/Michael.Swiss)\ **Replies:** 0\ **Last updated:** [March 16, 2019, 7:09am UTC](https://discuss.elastic.co/t/plugin-directory/172626 "2019-03-16T07:09:05Z")
Dear colleagues Is there a directory with eg filters for logstash plugins developed by the community? We have some unusual requirements, that is data enrichement by calling APIs with social media (twitter exists) and sc… --- ## [Consume data from sharded rabbitmq queue #2](https://discuss.elastic.co/t/consume-data-from-sharded-rabbitmq-queue-2/172178)
**Author:** [@lsdev14](https://discuss.elastic.co/u/lsdev14)\ **Replies:** 5\ **Last updated:** [March 16, 2019, 1:27am UTC](https://discuss.elastic.co/t/consume-data-from-sharded-rabbitmq-queue-2/172178 "2019-03-16T01:27:44Z")
I'm testing the rabbitmq-sharding plugin https://github.com/rabbitmq/rabbitmq-sharding I created a test scenario based in this question https://discuss.elastic.co/t/consume-data-from-sharded-rabbitmq-queue/130248 In re… --- ## [Logstash how to output the source log path in my message](https://discuss.elastic.co/t/logstash-how-to-output-the-source-log-path-in-my-message/167945)
**Author:** [@fvegaucr1](https://discuss.elastic.co/u/fvegaucr1)\ **Replies:** 16\ **Last updated:** [March 15, 2019, 11:53pm UTC](https://discuss.elastic.co/t/logstash-how-to-output-the-source-log-path-in-my-message/167945 "2019-03-15T23:53:39Z")
Im trying to get the number of times an ERROR|Error|error or Warn loglevel shows up in my log file and depending of the amount of times it repeats send a email/slack to the service owner with a summary like that CDP-er… --- ## [Compare fields with an external file](https://discuss.elastic.co/t/compare-fields-with-an-external-file/172492)
**Author:** [@theo1991](https://discuss.elastic.co/u/theo1991)\ **Replies:** 6\ **Last updated:** [March 15, 2019, 9:52pm UTC](https://discuss.elastic.co/t/compare-fields-with-an-external-file/172492 "2019-03-15T21:52:58Z")
Hi everybody, I'm dealing with a problem since a few days. First, I'm unable to delete the field message in kibana. I try with mutate, or directly in the grok filter, but there's always the field in kibana after a refr… --- ## [Unable to retrieve license information from license server](https://discuss.elastic.co/t/unable-to-retrieve-license-information-from-license-server/172583)
**Author:** [@drivera](https://discuss.elastic.co/u/drivera)\ **Replies:** 0\ **Last updated:** [March 15, 2019, 5:20pm UTC](https://discuss.elastic.co/t/unable-to-retrieve-license-information-from-license-server/172583 "2019-03-15T17:20:20Z")
Here is my exact error: \[2019-03-15T10:09:50,524\]\[ERROR\]\[logstash.licensechecker.licensemanager\] Unable to retrieve license information from license server {:message=\>"Elasticsearch Unreachable: \[http://logstash\_system:… --- ## [Strange kafka output plugin UNKNOWN\_TOPIC\_OR\_PARTITION](https://discuss.elastic.co/t/strange-kafka-output-plugin-unknown-topic-or-partition/172586)
**Author:** [@lmalossi](https://discuss.elastic.co/u/lmalossi)\ **Replies:** 1\ **Last updated:** [March 15, 2019, 6:53pm UTC](https://discuss.elastic.co/t/strange-kafka-output-plugin-unknown-topic-or-partition/172586 "2019-03-15T18:53:34Z")
Hi All, I've logstash configured to publish to 3 kafka topics metrics and apache and weblogic logs. If I use 2 of the three topics everything is well.. As soon as I start to use the third one I get: \[2019-03-15T18:22:… --- ## [GROK FILTER NOT WORKING WITH MULTILINE UNSTRUCTURED MESSAGE](https://discuss.elastic.co/t/grok-filter-not-working-with-multiline-unstructured-message/172386)
**Author:** [@Sharma3007](https://discuss.elastic.co/u/Sharma3007)\ **Replies:** 23\ **Last updated:** [March 15, 2019, 6:09pm UTC](https://discuss.elastic.co/t/grok-filter-not-working-with-multiline-unstructured-message/172386 "2019-03-15T18:09:39Z")
Hello, I am new to ELK stack and currently I am having issues with GROK filter. My grok filter is working fine with the grok debugger http://grokdebug.herokuapp.com/. But when I try to use it on logstash then it is n… --- ## [How to use script upsert in logstash for updating document](https://discuss.elastic.co/t/how-to-use-script-upsert-in-logstash-for-updating-document/172588)
**Author:** [@Raunak\_Kumar](https://discuss.elastic.co/u/Raunak_Kumar)\ **Replies:** 0\ **Last updated:** [March 15, 2019, 5:35pm UTC](https://discuss.elastic.co/t/how-to-use-script-upsert-in-logstash-for-updating-document/172588 "2019-03-15T17:35:18Z")
I am using below output block to upsert the document and incrementing the counter (partial updates) for an existing document with matching ID. Currently, post first doc entry into elasticsearch , "script" has not impact… --- ## [Why is my Logstash startup so slow?](https://discuss.elastic.co/t/why-is-my-logstash-startup-so-slow/172572)
**Author:** [@G82EtHAjnY4U](https://discuss.elastic.co/u/G82EtHAjnY4U)\ **Replies:** 3\ **Last updated:** [March 15, 2019, 5:02pm UTC](https://discuss.elastic.co/t/why-is-my-logstash-startup-so-slow/172572 "2019-03-15T17:02:27Z")
I'm running Logstash 6.6.1 in a Docker container in a cloud computing environment (Heroku). There's a limit on app boot time on Heroku, which is problematic for me because Logstash sometimes takes up to 2 minutes to star… --- ## [Cgroup issue logstash-6.6.2 and RedHat7](https://discuss.elastic.co/t/cgroup-issue-logstash-6-6-2-and-redhat7/172569)
**Author:** [@kentatoi](https://discuss.elastic.co/u/kentatoi)\ **Replies:** 1\ **Last updated:** [March 15, 2019, 4:46pm UTC](https://discuss.elastic.co/t/cgroup-issue-logstash-6-6-2-and-redhat7/172569 "2019-03-15T16:46:24Z")
Hi there, When I am running my logstash-6.6.2 rpm pipeline under Redhat 7, I have got this message in debug mode. \[DEBUG\] 2019-03-15 12:01:11.199 \[Ruby-0-Thread-9: :1\] pipeline - Pushing flush onto pipeline {:pipeline\_… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=581) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=583)