# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=266 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 267 --- ## [Jdbc postgres nested object solution](https://discuss.elastic.co/t/jdbc-postgres-nested-object-solution/258875)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 1\ **Last updated:** [January 6, 2021, 3:57pm UTC](https://discuss.elastic.co/t/jdbc-postgres-nested-object-solution/258875 "2021-01-06T15:57:34Z")
Does someone have this working? I am getting an Exception using a row\_to\_json sql statement. "Exception when executing JDBC query {:exception=\>"Java::OrgLogstash::MissingConverterException: Missing Converter handling f… --- ## [If in json nested field](https://discuss.elastic.co/t/if-in-json-nested-field/260343)
**Author:** [@probson](https://discuss.elastic.co/u/probson)\ **Replies:** 5\ **Last updated:** [January 6, 2021, 3:22pm UTC](https://discuss.elastic.co/t/if-in-json-nested-field/260343 "2021-01-06T15:22:08Z")
Hi, I am trying to use logstash to create a field of dns.type based on the types within dns.answers. This is from event.code 22 from sysmon. i have tried: if "AAAA" in \[dns\]\[answers\] { mutate { … --- ## [So many variables in logstash filters](https://discuss.elastic.co/t/so-many-variables-in-logstash-filters/260325)
**Author:** [@nyquillus](https://discuss.elastic.co/u/nyquillus)\ **Replies:** 0\ **Last updated:** [January 6, 2021, 10:35am UTC](https://discuss.elastic.co/t/so-many-variables-in-logstash-filters/260325 "2021-01-06T10:35:57Z")
Hi, I'm planning to convert to a new logstash config to remodel my indexing and i added an if/if else/else filter to logstash. Right now the cluster has around 85 active indices which roll daily. Almost half of it will b… --- ## [Enrichment file for threat feeds in logstash](https://discuss.elastic.co/t/enrichment-file-for-threat-feeds-in-logstash/260258)
**Author:** [@Falikou1](https://discuss.elastic.co/u/Falikou1)\ **Replies:** 2\ **Last updated:** [January 6, 2021, 10:07am UTC](https://discuss.elastic.co/t/enrichment-file-for-threat-feeds-in-logstash/260258 "2021-01-06T10:07:36Z")
Hello, I'm sending threat feeds to elasticsearch from a linux machine. I created a patern feeds index in kibana. Logstash receives Fortinet logs continuously. I created a patern fortinet index in kibana. My goal is … --- ## [\[ERROR\]\[logstash.config.sourceloader\] No configuration found in the configured sources](https://discuss.elastic.co/t/error-logstash-config-sourceloader-no-configuration-found-in-the-configured-sources/260238)
**Author:** [@aandrewsllp](https://discuss.elastic.co/u/aandrewsllp)\ **Replies:** 2\ **Last updated:** [January 6, 2021, 2:35am UTC](https://discuss.elastic.co/t/error-logstash-config-sourceloader-no-configuration-found-in-the-configured-sources/260238 "2021-01-06T02:35:34Z")
help me!!!! --- ## [Take input from elk index and need to send notification over email against invalid inputs](https://discuss.elastic.co/t/take-input-from-elk-index-and-need-to-send-notification-over-email-against-invalid-inputs/260223)
**Author:** [@Maahi\_Gupta](https://discuss.elastic.co/u/Maahi_Gupta)\ **Replies:** 3\ **Last updated:** [January 6, 2021, 2:25am UTC](https://discuss.elastic.co/t/take-input-from-elk-index-and-need-to-send-notification-over-email-against-invalid-inputs/260223 "2021-01-06T02:25:56Z")
Hi, I m a newbie in elk and i have been trying the below code from quite sometime. This is the configuration file i have input { elasticsearch { hosts =\> 'localhost' index =\> 'filebeat-\*' } } output… --- ## [Send a notification mail for invalid inputs made to the index (filebeat)](https://discuss.elastic.co/t/send-a-notification-mail-for-invalid-inputs-made-to-the-index-filebeat/260227)
**Author:** [@nottyheadedboss](https://discuss.elastic.co/u/nottyheadedboss)\ **Replies:** 1\ **Last updated:** [January 6, 2021, 2:22am UTC](https://discuss.elastic.co/t/send-a-notification-mail-for-invalid-inputs-made-to-the-index-filebeat/260227 "2021-01-06T02:22:58Z")
HI, I have been trying to work on the code to send an email to the support team for any invalid inputs such as (Invalid Credentials) into the system. A case being, when I try to use Putty and invalid credentials are ent… --- ## [Cannot do command to debug in logstash](https://discuss.elastic.co/t/cannot-do-command-to-debug-in-logstash/260259)
**Author:** [@Falikou1](https://discuss.elastic.co/u/Falikou1)\ **Replies:** 0\ **Last updated:** [January 5, 2021, 9:39pm UTC](https://discuss.elastic.co/t/cannot-do-command-to-debug-in-logstash/260259 "2021-01-05T21:39:39Z")
I can no longer make the following command: tail -f /var/log/logstash/logstash-plain.log I checked in / var / log / logstash / logstash-plain.log file no longer exists How can I fix this problem? --- ## [Add Geopoint based off of parsed value to logstash config](https://discuss.elastic.co/t/add-geopoint-based-off-of-parsed-value-to-logstash-config/26580)
**Author:** [@russK](https://discuss.elastic.co/u/russK)\ **Replies:** 13\ **Last updated:** [January 5, 2021, 9:03pm UTC](https://discuss.elastic.co/t/add-geopoint-based-off-of-parsed-value-to-logstash-config/26580 "2021-01-05T21:03:45Z")
Hello, Part of my matched message returns the fields %{NUMBER:XCent} %{NUMBER:YCent} which are lat, long points. I'm attempting to add a location pin but keep getting a config failure when i use the --debug flag on… --- ## [Infinite loop of retryable error code=\>400](https://discuss.elastic.co/t/infinite-loop-of-retryable-error-code-400/260234)
**Author:** [@maxonage](https://discuss.elastic.co/u/maxonage)\ **Replies:** 2\ **Last updated:** [January 5, 2021, 6:09pm UTC](https://discuss.elastic.co/t/infinite-loop-of-retryable-error-code-400/260234 "2021-01-05T18:09:05Z")
Hello, We're using filebeat that sends data to logstash, suddenly we started receiving Encountered a retryable error. Will Retry with exponential backoff {:code=\>400, :url=\>"http://localhost:9200/\_bulk"} From filebea… --- ## [Logstash Persistent Queue Buffer time](https://discuss.elastic.co/t/logstash-persistent-queue-buffer-time/260211)
**Author:** [@ksaha](https://discuss.elastic.co/u/ksaha)\ **Replies:** 2\ **Last updated:** [January 5, 2021, 2:29pm UTC](https://discuss.elastic.co/t/logstash-persistent-queue-buffer-time/260211 "2021-01-05T14:29:04Z")
Hi, I am exploring on Logstash resiliency part and come to know about the logstash Persistent Queue. I am searching what is the maximum buffer time of this Persistent Queue? Our requirement is the buffer should hold the… --- ## [AWS EC2 Logstash input with external IPs](https://discuss.elastic.co/t/aws-ec2-logstash-input-with-external-ips/260202)
**Author:** [@Sunflower](https://discuss.elastic.co/u/Sunflower)\ **Replies:** 0\ **Last updated:** [January 5, 2021, 12:16pm UTC](https://discuss.elastic.co/t/aws-ec2-logstash-input-with-external-ips/260202 "2021-01-05T12:16:20Z")
(topic withdrawn by author, will be automatically deleted in 24 hours unless flagged) --- ## [How to save large data send from logstash to Redis with list type?](https://discuss.elastic.co/t/how-to-save-large-data-send-from-logstash-to-redis-with-list-type/260210)
**Author:** [@iooi](https://discuss.elastic.co/u/iooi)\ **Replies:** 0\ **Last updated:** [January 5, 2021, 1:30pm UTC](https://discuss.elastic.co/t/how-to-save-large-data-send-from-logstash-to-redis-with-list-type/260210 "2021-01-05T13:30:24Z")
I'm using logstash to send data to Redis as output { redis { host =\> \["${REDIS\_URL}"\] data\_type =\> "list" key =\> "ID" codec =\> line { format =\> "%{id}"} } } When I check data in Redis, it created an… --- ## [LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError: Got response code '401' contacting Elasticsearc](https://discuss.elastic.co/t/logstash-got-response-code-401-contacting-elasticsearc/260209)
**Author:** [@gitusersybchina](https://discuss.elastic.co/u/gitusersybchina)\ **Replies:** 0\ **Last updated:** [January 5, 2021, 1:27pm UTC](https://discuss.elastic.co/t/logstash-got-response-code-401-contacting-elasticsearc/260209 "2021-01-05T13:27:31Z")
when I deploy logstash in k8s and config out to es，the pod has occur this error log: LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError: Got response code '401' contacting Elasticsearch Failed to … --- ## [Logs correlation](https://discuss.elastic.co/t/logs-correlation/260110)
**Author:** [@sm\_loguser](https://discuss.elastic.co/u/sm_loguser)\ **Replies:** 2\ **Last updated:** [January 5, 2021, 12:22pm UTC](https://discuss.elastic.co/t/logs-correlation/260110 "2021-01-05T12:22:47Z")
I have two log files. I am correlating these two files by timestamp using aggregate filter. Where there is timestamp match between two files, correlation is working fine. But there are scenarios where either of the two… --- ## [Ingesting Oracle's Audit Trail](https://discuss.elastic.co/t/ingesting-oracles-audit-trail/259897)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 2\ **Last updated:** [January 5, 2021, 6:29am UTC](https://discuss.elastic.co/t/ingesting-oracles-audit-trail/259897 "2021-01-05T06:29:17Z")
Hi, I'm ingesting the content from sys.dba\_audit\_trail into ES using the jdbc input plugin. Everything goes ok with the exception of the field transactionid which is of type (at db level of RAW(8)) Here an example 000… --- ## [File paths must be absolute, relative path specified](https://discuss.elastic.co/t/file-paths-must-be-absolute-relative-path-specified/260016)
**Author:** [@Cyphy](https://discuss.elastic.co/u/Cyphy)\ **Replies:** 9\ **Last updated:** [January 5, 2021, 4:09am UTC](https://discuss.elastic.co/t/file-paths-must-be-absolute-relative-path-specified/260016 "2021-01-05T04:09:27Z")
hey there, hope you can help me :slight\_smile: I started my ELK setup with docker-compose a few days ago. It looked fine as all 3 containers where running without any data. Today i wanted to import some xml files with … --- ## [Logstash best practice for filters order](https://discuss.elastic.co/t/logstash-best-practice-for-filters-order/259775)
**Author:** [@Mohammad\_Mousavi](https://discuss.elastic.co/u/Mohammad_Mousavi)\ **Replies:** 2\ **Last updated:** [January 5, 2021, 3:18am UTC](https://discuss.elastic.co/t/logstash-best-practice-for-filters-order/259775 "2021-01-05T03:18:49Z")
Hi, I have a lot of input and filters on my logstash and I've been thinking are they optimize and how can I measure logstash performance. I only monitor "heap used" and it's fine. I came up with a question. There are 2 t… --- ## [Show data in kibana, used snmp plugin](https://discuss.elastic.co/t/show-data-in-kibana-used-snmp-plugin/260125)
**Author:** [@aandrewsllp](https://discuss.elastic.co/u/aandrewsllp)\ **Replies:** 1\ **Last updated:** [January 5, 2021, 2:53am UTC](https://discuss.elastic.co/t/show-data-in-kibana-used-snmp-plugin/260125 "2021-01-05T02:53:38Z")
i need show data in kibana, used snmp plugin --- ## [JDBC\_Streaming plugin issue](https://discuss.elastic.co/t/jdbc-streaming-plugin-issue/260149)
**Author:** [@Ashok\_Botcha](https://discuss.elastic.co/u/Ashok_Botcha)\ **Replies:** 0\ **Last updated:** [January 5, 2021, 2:29am UTC](https://discuss.elastic.co/t/jdbc-streaming-plugin-issue/260149 "2021-01-05T02:29:26Z")
Hi, I am trying below sample logstash config, Output also pasted below. data related to table C not coming inside B, Can you suggest if anything wrong with config ? CONFIG: \`input { jdbc { ....... statement =\> "SELE… --- ## [Can logstash's Google Cloud Storage output plugin do persistent transmission after restart machine?](https://discuss.elastic.co/t/can-logstashs-google-cloud-storage-output-plugin-do-persistent-transmission-after-restart-machine/260139)
**Author:** [@iooi](https://discuss.elastic.co/u/iooi)\ **Replies:** 0\ **Last updated:** [January 4, 2021, 11:45pm UTC](https://discuss.elastic.co/t/can-logstashs-google-cloud-storage-output-plugin-do-persistent-transmission-after-restart-machine/260139 "2021-01-04T23:45:41Z")
Using this config for logstash's output. It's using /tmp/logstash-gcs as a local folder. Send to GCS when file become 1024 kbytes. input { beats { port =\> 5044 } } filter {} output { google\_cloud\_storage { … --- ## [Mutate gsub regex pattern help](https://discuss.elastic.co/t/mutate-gsub-regex-pattern-help/260122)
**Author:** [@S3l3ct3d](https://discuss.elastic.co/u/S3l3ct3d)\ **Replies:** 6\ **Last updated:** [January 4, 2021, 10:27pm UTC](https://discuss.elastic.co/t/mutate-gsub-regex-pattern-help/260122 "2021-01-04T22:27:12Z")
I am having an issue with the regex pattern in mutate gsub function. My current filter is below filter { # Log types are "SIEM\_EVENTS", "TTP\_EVENTS", "AUDIT\_EVENTS". mutate { gsub =\> \[ "\[message\]", "^\<\\d+\>", "" \] … --- ## [Logstash multiline pattern](https://discuss.elastic.co/t/logstash-multiline-pattern/259997)
**Author:** [@Muhammad\_Faisal](https://discuss.elastic.co/u/Muhammad_Faisal)\ **Replies:** 3\ **Last updated:** [January 4, 2021, 6:19pm UTC](https://discuss.elastic.co/t/logstash-multiline-pattern/259997 "2021-01-04T18:19:49Z")
hi guys, i have a log file which consists of below pattern , i need to treat all lines of this log file as 1 aggregate event.file will close after 30 minutes and then new log will start..which pattern will work here...… --- ## [Logstash do not import file](https://discuss.elastic.co/t/logstash-do-not-import-file/260114)
**Author:** [@Sad\_Creem](https://discuss.elastic.co/u/Sad_Creem)\ **Replies:** 2\ **Last updated:** [January 4, 2021, 6:15pm UTC](https://discuss.elastic.co/t/logstash-do-not-import-file/260114 "2021-01-04T18:15:30Z")
Is Logstash have a limit for max line characters leight? I have 12 mb json file wich content single line json with multi objects. Logstash do not work with this file i'am dont know why no error no debug string just sta… --- ## [Multipe host output using keystore passwords](https://discuss.elastic.co/t/multipe-host-output-using-keystore-passwords/260090)
**Author:** [@mcosta](https://discuss.elastic.co/u/mcosta)\ **Replies:** 4\ **Last updated:** [January 4, 2021, 4:29pm UTC](https://discuss.elastic.co/t/multipe-host-output-using-keystore-passwords/260090 "2021-01-04T16:29:29Z")
Hi All, I need to configure logstash (ver 7.7) to write to two elastisearch. I have two separate output blocks, one for each elasticsearch but how do I configure password retrieval from keystore, since documentation ref… --- ## [Logstash plugin is installed and listed but not found by logstash](https://discuss.elastic.co/t/logstash-plugin-is-installed-and-listed-but-not-found-by-logstash/259895)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 3\ **Last updated:** [January 4, 2021, 3:43pm UTC](https://discuss.elastic.co/t/logstash-plugin-is-installed-and-listed-but-not-found-by-logstash/259895 "2021-01-04T15:43:34Z")
Hello Elasticians ! I installed syslog output plug in (logstash-output-syslog-3.0.5.gem) using offline method as I don't have wan access on target server. Plug in has been successfully installed and is listed with lo… --- ## [Will my "%{GREEDYDATA:\[log\_message\]}" overwrides all my other grok filters?](https://discuss.elastic.co/t/will-my-greedydata-log-message-overwrides-all-my-other-grok-filters/260063)
**Author:** [@yulaika](https://discuss.elastic.co/u/yulaika)\ **Replies:** 1\ **Last updated:** [January 4, 2021, 2:43pm UTC](https://discuss.elastic.co/t/will-my-greedydata-log-message-overwrides-all-my-other-grok-filters/260063 "2021-01-04T14:43:13Z")
grok { match =\> { "message" =\> \[ "(?\<\[log\]\[timestamp\]\>%{YEAR}-%{MONTHNUM}-%{MONTHDAY}%{SPACE}%{TIME}) \\\[%{NOTSPACE:\[LOG\]\[LEVEL\]}\\\] %{GREEDYDATA:\[log\_message\]}" , "(?\<\[log\]\[timestamp\]\>%{YEA… --- ## [Multiple resources on conf](https://discuss.elastic.co/t/multiple-resources-on-conf/260079)
**Author:** [@Sunflower](https://discuss.elastic.co/u/Sunflower)\ **Replies:** 6\ **Last updated:** [January 4, 2021, 11:40am UTC](https://discuss.elastic.co/t/multiple-resources-on-conf/260079 "2021-01-04T11:40:59Z")
Hi, I have multiple resources that I want to configure to send Syslog to Logstash and then send it to S3. I'd appreciate hearing your thoughts regarding the two following options: Multiple resources on the input, eac… --- ## [Postgres with Elastic search issue](https://discuss.elastic.co/t/postgres-with-elastic-search-issue/259448)
**Author:** [@vasimsaiyad2000](https://discuss.elastic.co/u/vasimsaiyad2000)\ **Replies:** 3\ **Last updated:** [January 4, 2021, 11:12am UTC](https://discuss.elastic.co/t/postgres-with-elastic-search-issue/259448 "2021-01-04T11:12:33Z")
I have implemented logstash with postgres to send my restaurant and menu items to Elasticsearch. I have applied paging to fetch the data from postgres and also have jdbc streaming. If i run the logstash with jdbc stream… --- ## [Logstash Filter - Grok pattern not working as expected](https://discuss.elastic.co/t/logstash-filter-grok-pattern-not-working-as-expected/259271)
**Author:** [@pavank](https://discuss.elastic.co/u/pavank)\ **Replies:** 2\ **Last updated:** [January 4, 2021, 7:15am UTC](https://discuss.elastic.co/t/logstash-filter-grok-pattern-not-working-as-expected/259271 "2021-01-04T07:15:11Z")
Hi All, We are trying to apply Grok filter patterns for our application logs. The logs are applied successfully on the grok debugger online as well as Kibana Dev Tool debugger, but is not getting applied properly in the… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=265) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=267)