# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=261 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 262 --- ## [Logstash throws \[message=\>“undefined method \`update' for nil:NilClass”\] when Elasticsearch index template defined in logstash](https://discuss.elastic.co/t/logstash-throws-message-undefined-method-update-for-nil-nilclass-when-elasticsearch-index-template-defined-in-logstash/262135)
**Author:** [@nadyahmed123](https://discuss.elastic.co/u/nadyahmed123)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 4:19pm UTC](https://discuss.elastic.co/t/logstash-throws-message-undefined-method-update-for-nil-nilclass-when-elasticsearch-index-template-defined-in-logstash/262135 "2021-01-25T16:19:52Z")
I am trying to load data from sql server to Elasticsearch using logstash. For index creation i am using index template that is specified in the config file of logstash. Output field of logstash.config : output{ st… --- ## [How to push a mobile logs to ELK](https://discuss.elastic.co/t/how-to-push-a-mobile-logs-to-elk/262130)
**Author:** [@fio](https://discuss.elastic.co/u/fio)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 3:59pm UTC](https://discuss.elastic.co/t/how-to-push-a-mobile-logs-to-elk/262130 "2021-01-25T15:59:33Z")
Hello, Is it possible to send the logs from my mobile app (Kotlin, Swift) to Kibana? How can I do it? There is an API or something like that --- ## [Attempted to send event to pipeline](https://discuss.elastic.co/t/attempted-to-send-event-to-pipeline/262124)
**Author:** [@gose](https://discuss.elastic.co/u/gose)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 3:13pm UTC](https://discuss.elastic.co/t/attempted-to-send-event-to-pipeline/262124 "2021-01-25T15:13:08Z")
I'm using Centralized Pipeline Management. I updated a pipeline (pipeline-1) that started sending events to another pipeline (pipeline-2). I didn't have pipeline-2 in my logstash.yml file, so Logstash didn't know about… --- ## [Usage of Cisco module's "asa" fileset usage in Logstash pipeline bypassing filebeat](https://discuss.elastic.co/t/usage-of-cisco-modules-asa-fileset-usage-in-logstash-pipeline-bypassing-filebeat/261945)
**Author:** [@mguttula](https://discuss.elastic.co/u/mguttula)\ **Replies:** 1\ **Last updated:** [January 25, 2021, 3:11pm UTC](https://discuss.elastic.co/t/usage-of-cisco-modules-asa-fileset-usage-in-logstash-pipeline-bypassing-filebeat/261945 "2021-01-25T15:11:59Z")
The current architecture of my set up is as follows: Cisco VPN --\> Logstash pipeline ---\> Elasticsearch I would like to use Cisco module asa fileset directly in the pipeline somehow so that I don't have to write the l… --- ## [Logstash to Logstash communication issue with domain based SSL signed certification](https://discuss.elastic.co/t/logstash-to-logstash-communication-issue-with-domain-based-ssl-signed-certification/260796)
**Author:** [@alexbennet](https://discuss.elastic.co/u/alexbennet)\ **Replies:** 1\ **Last updated:** [January 25, 2021, 3:09pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-communication-issue-with-domain-based-ssl-signed-certification/260796 "2021-01-25T15:09:16Z")
We are implementing application log monitoring on ELK flow from filebeat (logstash output) -\> on-prem logstash server(beats input & lumberjack output) -\> aws logstash server (beats input & elastic output) -\> elasticear… --- ## [Doc\_as\_upsert along with script in Elasticsearch output plugin](https://discuss.elastic.co/t/doc-as-upsert-along-with-script-in-elasticsearch-output-plugin/262106)
**Author:** [@Ahmed\_Sharara](https://discuss.elastic.co/u/Ahmed_Sharara)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 12:55pm UTC](https://discuss.elastic.co/t/doc-as-upsert-along-with-script-in-elasticsearch-output-plugin/262106 "2021-01-25T12:55:57Z")
Hello, I'm using logstash to index data from Kafka to Elasticsearch. There are multiple events with the same id. I'm using doc\_as\_upsert in logstash to update existing documents with the new values based on the \_id. sa… --- ## [Logstash Upgrade](https://discuss.elastic.co/t/logstash-upgrade/262100)
**Author:** [@Rithesh\_Subramanian](https://discuss.elastic.co/u/Rithesh_Subramanian)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 11:56am UTC](https://discuss.elastic.co/t/logstash-upgrade/262100 "2021-01-25T11:56:44Z")
Hi, I had used logstash-6.2.2 and now i hadupgraded to logstash-7.10.1. I am using logstash package for linux. I am using filebeat as a log shipper. Both the filebeat version and logstash version are same. But after up… --- ## [SSL certificate reload](https://discuss.elastic.co/t/ssl-certificate-reload/262090)
**Author:** [@raivis.krumins](https://discuss.elastic.co/u/raivis.krumins)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 11:34am UTC](https://discuss.elastic.co/t/ssl-certificate-reload/262090 "2021-01-25T11:34:22Z")
I want to use hashicorp vault to manage logstash ssl certs on kubernetes. I am using logstash helm chart --version 7.10.1 These pod annotations work as expected: podAnnotations: vault.hashicorp.com/agent-inject: "tr… --- ## [Can not to delete child document](https://discuss.elastic.co/t/can-not-to-delete-child-document/262064)
**Author:** [@Mamo](https://discuss.elastic.co/u/Mamo)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 8:00am UTC](https://discuss.elastic.co/t/can-not-to-delete-child-document/262064 "2021-01-25T08:00:32Z")
logstash version: 6.4.2 elasticsearch version: 6.4.2 output { elasticsearch { host =\> \[...\] action =\> "delete" index =\> "test" document\_id =\> "%{childId}" routing =\> "%{parentId}" } } or outpu… --- ## [Intercept incoming data to logstash pipeline](https://discuss.elastic.co/t/intercept-incoming-data-to-logstash-pipeline/262028)
**Author:** [@g2h2o](https://discuss.elastic.co/u/g2h2o)\ **Replies:** 2\ **Last updated:** [January 25, 2021, 7:02am UTC](https://discuss.elastic.co/t/intercept-incoming-data-to-logstash-pipeline/262028 "2021-01-25T07:02:52Z")
Actually we are getting jenkins build logs result into elasticsearch through filebeats and logstash. We are receiving the expected data in logstash but for an unknown reason is getting repeated constantly. Seems that t… --- ## [Logstash error: Could not process event: no implicit conversion of NilClass into String](https://discuss.elastic.co/t/logstash-error-could-not-process-event-no-implicit-conversion-of-nilclass-into-string/262056)
**Author:** [@Subhas\_Patil](https://discuss.elastic.co/u/Subhas_Patil)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 5:51am UTC](https://discuss.elastic.co/t/logstash-error-could-not-process-event-no-implicit-conversion-of-nilclass-into-string/262056 "2021-01-25T05:51:14Z")
Iam getting the error Could not process event: no implicit conversion of NilClass into String in logstash ruby filter. Here is the ruby file which iam using in the logstash filter: def filter(event) require 'json' … --- ## [How Logstash Process data](https://discuss.elastic.co/t/how-logstash-process-data/261216)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 2\ **Last updated:** [January 25, 2021, 2:49am UTC](https://discuss.elastic.co/t/how-logstash-process-data/261216 "2021-01-25T02:49:33Z")
Hello Team I am in requirement of pushing data from Oracle Table to Elasticsearch Index. My Oracle Table is of 300GB in Size. I would like to understand , How Logstash process the data when he pulls from Oracle { inpu… --- ## [Throttle filter: Notify on throttle](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861)
**Author:** [@holobolo0815](https://discuss.elastic.co/u/holobolo0815)\ **Replies:** 8\ **Last updated:** [January 24, 2021, 4:51pm UTC](https://discuss.elastic.co/t/throttle-filter-notify-on-throttle/261861 "2021-01-24T16:51:15Z")
When using the throttle filter, how would I go about notifying one event once the throttle has triggered? Meaning if I allow 5 events per minute, I'd like to know somehow if there has been a 6th event and send a mail or… --- ## [How to index nanoseconds precision events with Logstash (7.10) and type date\_nanos](https://discuss.elastic.co/t/how-to-index-nanoseconds-precision-events-with-logstash-7-10-and-type-date-nanos/262029)
**Author:** [@juan.domenech](https://discuss.elastic.co/u/juan.domenech)\ **Replies:** 0\ **Last updated:** [January 24, 2021, 11:14am UTC](https://discuss.elastic.co/t/how-to-index-nanoseconds-precision-events-with-logstash-7-10-and-type-date-nanos/262029 "2021-01-24T11:14:24Z")
Problem statement Elasticsearch 7.10 supports timestamp with nanoseconds precision (type date\_nanos) Kibana 7.10 visualises these timestamps but Logstash can't handle them yet (precision above milliseconds is lost whe… --- ## [Accessing individual values in a logstash](https://discuss.elastic.co/t/accessing-individual-values-in-a-logstash/262027)
**Author:** [@learner1](https://discuss.elastic.co/u/learner1)\ **Replies:** 0\ **Last updated:** [January 24, 2021, 9:53am UTC](https://discuss.elastic.co/t/accessing-individual-values-in-a-logstash/262027 "2021-01-24T09:53:06Z")
I'm trying to mask some values in the logs but looks like it's not working because of the log format. Here is the format: "\_source": { "log": "2021-01-24 07:15:37 +0000 \[warn\]: #0 dump an error event: error\_cl… --- ## [Mapper\_parsing\_exception](https://discuss.elastic.co/t/mapper-parsing-exception/261871)
**Author:** [@adityak248](https://discuss.elastic.co/u/adityak248)\ **Replies:** 3\ **Last updated:** [January 22, 2021, 10:05pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception/261871 "2021-01-22T22:05:01Z")
Hello Team, I have been getting this issue no matter what. I have applied mutate (convert,rename) json but still get the following issue. "reason"=\>"Could not dynamically add mapping for field \[app.kubernetes.io/name\].… --- ## [Logstash ruby filter math on very large numbers](https://discuss.elastic.co/t/logstash-ruby-filter-math-on-very-large-numbers/261950)
**Author:** [@Andrew22](https://discuss.elastic.co/u/Andrew22)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 6:26pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-math-on-very-large-numbers/261950 "2021-01-22T18:26:18Z")
Hello, I am trying to do some math on some very large numbers in logstash and it appears that the .to\_f is not large enough as its giving me very inaccurate information. is there anything larger than a float I can use? … --- ## [Geolocation configuration in logstash](https://discuss.elastic.co/t/geolocation-configuration-in-logstash/261937)
**Author:** [@Falikou1](https://discuss.elastic.co/u/Falikou1)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 3:56pm UTC](https://discuss.elastic.co/t/geolocation-configuration-in-logstash/261937 "2021-01-22T15:56:33Z")
I configured my logstash to parse the logs. I now want to configure geolocation. Is there a unique configuration for geolocation? If so, I need help. Here is my logstash setup below. How to complete with the geolocati… --- ## [Dissector mapping, pattern not found error](https://discuss.elastic.co/t/dissector-mapping-pattern-not-found-error/261904)
**Author:** [@Shubha](https://discuss.elastic.co/u/Shubha)\ **Replies:** 1\ **Last updated:** [January 22, 2021, 1:25pm UTC](https://discuss.elastic.co/t/dissector-mapping-pattern-not-found-error/261904 "2021-01-22T13:25:40Z")
Hi All, I am having the ELK version 7.10 on ec2 instances. facing the below error in logstash-plain.log. Can anyone help me on this. \[2021-01-22T10:29:00,423\]\[WARN \]\[org.logstash.dissect.Dissector\]\[main\]\[1cbe1dfc036a2… --- ## [Logstash need server restart](https://discuss.elastic.co/t/logstash-need-server-restart/261913)
**Author:** [@Melvin\_Shaju](https://discuss.elastic.co/u/Melvin_Shaju)\ **Replies:** 1\ **Last updated:** [January 22, 2021, 1:03pm UTC](https://discuss.elastic.co/t/logstash-need-server-restart/261913 "2021-01-22T13:03:26Z")
I'm using logstash feeds to take data from json file then filter it and pass it to elastic search. Each time to run logstash feeds I need to restart the whole server, otherwise logstash is causing errors. Is there any so… --- ## [Passing custom regex inside grok filter](https://discuss.elastic.co/t/passing-custom-regex-inside-grok-filter/261907)
**Author:** [@Flavio1](https://discuss.elastic.co/u/Flavio1)\ **Replies:** 1\ **Last updated:** [January 22, 2021, 11:49am UTC](https://discuss.elastic.co/t/passing-custom-regex-inside-grok-filter/261907 "2021-01-22T11:49:24Z")
Hi everybody, During these days i'm trying to implement a custom regex system configuration in order to have a single point, outside pipelines, in which i can make crud operations on regexes. I tried this solution: mut… --- ## [:exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of \[ \\\\t\\\\r\\\\n\], \\"#\\", \[A-Za-z0-9\_-\], '\\"', \\"'\\", \[A-Za-z\_\], \\"-\\", \[0-9\], \\"\[\\", \\"{\\", \\"\]\\"](https://discuss.elastic.co/t/exception-logstash-configurationerror-message-expected-one-of-t-r-n-a-za-z0-9-a-za-z-0-9/261903)
**Author:** [@ashishkpal](https://discuss.elastic.co/u/ashishkpal)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 10:44am UTC](https://discuss.elastic.co/t/exception-logstash-configurationerror-message-expected-one-of-t-r-n-a-za-z0-9-a-za-z-0-9/261903 "2021-01-22T10:44:23Z")
hi, i want to push the ELB logs from s3 to ELK for the same have i have written the logstash.conf file like this input { s3 { access\_key\_id =\> "..." secret\_access\_key =\> "..." bucket =\> "..." region =\> "eu-central… --- ## [Listen to multiple topics for google-pub sub plugin Logstash](https://discuss.elastic.co/t/listen-to-multiple-topics-for-google-pub-sub-plugin-logstash/261894)
**Author:** [@curiousmind](https://discuss.elastic.co/u/curiousmind)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 9:51am UTC](https://discuss.elastic.co/t/listen-to-multiple-topics-for-google-pub-sub-plugin-logstash/261894 "2021-01-22T09:51:41Z")
My current configuration is like below: google\_pubsub { project\_id =\> "svg-nsg-log-51mx7" topic =\> "audit\_logs" subscription =\> "logstash-sub-audit\_logs" include\_metadata =\> true codec =\> "json" … --- ## [How to solve ELK bottlenecks](https://discuss.elastic.co/t/how-to-solve-elk-bottlenecks/261879)
**Author:** [@jang](https://discuss.elastic.co/u/jang)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 7:41am UTC](https://discuss.elastic.co/t/how-to-solve-elk-bottlenecks/261879 "2021-01-22T07:41:41Z")
I am using pipeline that uses filebeat, kafka, logstash, and Elastic service. kafka and logstash consist of one server.(4core, 4GB ram) Elastic service uses one master node and three data nodes.(2core, 2GB ram) Recent… --- ## [Credentials with Private gem Repository for Logstash](https://discuss.elastic.co/t/credentials-with-private-gem-repository-for-logstash/261856)
**Author:** [@jason\_0](https://discuss.elastic.co/u/jason_0)\ **Replies:** 0\ **Last updated:** [January 21, 2021, 10:02pm UTC](https://discuss.elastic.co/t/credentials-with-private-gem-repository-for-logstash/261856 "2021-01-21T22:02:38Z")
Hi All, Our Logstash servers don't have access to the Internet and instead we use a private repository available at https://repo.domain.com etc. This repo requires a username and password to access. Can we do this/how d… --- ## [Having problems parsing data](https://discuss.elastic.co/t/having-problems-parsing-data/261817)
**Author:** [@BeMoore](https://discuss.elastic.co/u/BeMoore)\ **Replies:** 3\ **Last updated:** [January 21, 2021, 5:30pm UTC](https://discuss.elastic.co/t/having-problems-parsing-data/261817 "2021-01-21T17:30:45Z")
Hi there, Im having some issues filtering this section of a log file, 2021-01-19T13:32:25.263Z localhost {reason=user\_approved, txid=cbcf50e8-e05e-4ee8-9c6d-125d78b6ff9e, ood\_software=null, isotimestamp=2021-01-19T13:2… --- ## [Custom Template for Logstash 7.10 output](https://discuss.elastic.co/t/custom-template-for-logstash-7-10-output/261822)
**Author:** [@GrandOurs35](https://discuss.elastic.co/u/GrandOurs35)\ **Replies:** 0\ **Last updated:** [January 21, 2021, 4:22pm UTC](https://discuss.elastic.co/t/custom-template-for-logstash-7-10-output/261822 "2021-01-21T16:22:20Z")
Hello, i'm new to Elastic Stack and my setup is with Docker and i'm using 7.10.2 version. For now, i'm trying to analyze my nginx logs in Kibana. I can see my logs in Discover but not in "terms" in the dashboard part. … --- ## [Scripted upsert issues](https://discuss.elastic.co/t/scripted-upsert-issues/261808)
**Author:** [@Dede\_Pessu](https://discuss.elastic.co/u/Dede_Pessu)\ **Replies:** 0\ **Last updated:** [January 21, 2021, 3:05pm UTC](https://discuss.elastic.co/t/scripted-upsert-issues/261808 "2021-01-21T15:05:12Z")
I am currently trying to do calculations based on correlating field values from different documents and decided achieve this by using scripted\_upsert method to update and create the documents based on the document id but… --- ## [Logstash Persistent Queues](https://discuss.elastic.co/t/logstash-persistent-queues/261745)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [January 21, 2021, 8:32am UTC](https://discuss.elastic.co/t/logstash-persistent-queues/261745 "2021-01-21T08:32:18Z")
Hi, I'm seeing some weird behaviour where logstash appears to be bottlenecking throughput from filebeat yet persistent queues are not filling. Is this possible? Note: Logstash appears to be the bottleneck as throughput… --- ## [Filebeat/Logstash Interface](https://discuss.elastic.co/t/filebeat-logstash-interface/261736)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [January 21, 2021, 7:51am UTC](https://discuss.elastic.co/t/filebeat-logstash-interface/261736 "2021-01-21T07:51:08Z")
Hi, What is the impact on the beat-input of logstash if each filebeat is creating a large number of connections, for example 128? Thx D [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=260) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=262)