# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=400 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 401 --- ## [Getting wrong dates even when i declare a date filter](https://discuss.elastic.co/t/getting-wrong-dates-even-when-i-declare-a-date-filter/220673)
**Author:** [@samargh](https://discuss.elastic.co/u/samargh)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 2:17pm UTC](https://discuss.elastic.co/t/getting-wrong-dates-even-when-i-declare-a-date-filter/220673 "2020-02-24T14:17:41Z")
i'm getting wrong dates even when i declare a date filter date { match =\> \[ "Temporary Service Class Expiry Date", "YYYY-MM-dd" \] target =\> "Temporary Service Class Expiry Date" } shows May 9, 2014 @ 23:00:00.000 … --- ## [Logstash redis output plugin does not update record it always insert new record in the list of redis](https://discuss.elastic.co/t/logstash-redis-output-plugin-does-not-update-record-it-always-insert-new-record-in-the-list-of-redis/220525)
**Author:** [@bharat\_goswami2](https://discuss.elastic.co/u/bharat_goswami2)\ **Replies:** 1\ **Last updated:** [February 24, 2020, 2:12pm UTC](https://discuss.elastic.co/t/logstash-redis-output-plugin-does-not-update-record-it-always-insert-new-record-in-the-list-of-redis/220525 "2020-02-24T14:12:39Z")
Hello Guys, I am migrating data from mysql to redis using logstash output pluging, but logstash does not update the record if it is already exist , it always insert new one into the list. Please see the below code. in… --- ## [JDBC Input - parameter with multiple values](https://discuss.elastic.co/t/jdbc-input-parameter-with-multiple-values/220657)
**Author:** [@saif3r](https://discuss.elastic.co/u/saif3r)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 1:12pm UTC](https://discuss.elastic.co/t/jdbc-input-parameter-with-multiple-values/220657 "2020-02-24T13:12:18Z")
Hello, I have a SQL Server query that that has one parameter with one or more values. In raw SQL it looks like this: select top 1 \* from a inner join b on b.ID=a.ID and b.Resp in (1, 2) So Logstash statement looks lik… --- ## [Logs from remote server](https://discuss.elastic.co/t/logs-from-remote-server/220631)
**Author:** [@PerseiX1](https://discuss.elastic.co/u/PerseiX1)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 10:55am UTC](https://discuss.elastic.co/t/logs-from-remote-server/220631 "2020-02-24T10:55:27Z")
Hello, I have 2 servers, one with my ELK (each of those in separated container) and second server with my application (Centos). I would like to read logs by Logstash from my server with the application. I don't want to … --- ## [Syslog input to elastic using logstah](https://discuss.elastic.co/t/syslog-input-to-elastic-using-logstah/220342)
**Author:** [@mvijayrkr](https://discuss.elastic.co/u/mvijayrkr)\ **Replies:** 2\ **Last updated:** [February 24, 2020, 10:54am UTC](https://discuss.elastic.co/t/syslog-input-to-elastic-using-logstah/220342 "2020-02-24T10:54:21Z")
Hi Guys, Im pretty much new to ELK stack, im currently fetching the syslog logs to logstash through syslog plugin and trying to push it to elastic, as we need to provide the json as an input to elastic, im trying to con… --- ## [Problem creating dynamic index from filename (filebeat)](https://discuss.elastic.co/t/problem-creating-dynamic-index-from-filename-filebeat/220315)
**Author:** [@Amine\_Maalfi](https://discuss.elastic.co/u/Amine_Maalfi)\ **Replies:** 6\ **Last updated:** [February 24, 2020, 9:31am UTC](https://discuss.elastic.co/t/problem-creating-dynamic-index-from-filename-filebeat/220315 "2020-02-24T09:31:12Z")
Hello, i am trying to create index name same as filename from source but it doesn't seem to work, here's my logstash config file : input { beats{ port=\> 5044 } } filter { grok { … --- ## [\_jsonparsefailure with special characters?](https://discuss.elastic.co/t/jsonparsefailure-with-special-characters/220585)
**Author:** [@barryb](https://discuss.elastic.co/u/barryb)\ **Replies:** 0\ **Last updated:** [February 24, 2020, 7:53am UTC](https://discuss.elastic.co/t/jsonparsefailure-with-special-characters/220585 "2020-02-24T07:53:34Z")
We are getting \_jsonparsefailure for some records being input from a client in France We have a copy of their Database and when importing with the same logstash.config file we succeed in parsing all the records correctl… --- ## [Logstash shutdown after executing command for MS SQL writing to elastic search](https://discuss.elastic.co/t/logstash-shutdown-after-executing-command-for-ms-sql-writing-to-elastic-search/220547)
**Author:** [@yashjain](https://discuss.elastic.co/u/yashjain)\ **Replies:** 2\ **Last updated:** [February 24, 2020, 5:20am UTC](https://discuss.elastic.co/t/logstash-shutdown-after-executing-command-for-ms-sql-writing-to-elastic-search/220547 "2020-02-24T05:20:50Z")
The logstash runner shutsdown after executing the index once for MS SQL writing data to elastic search db. The conf file: input { jdbc { jdbc\_driver\_library =\> "C:\\MicrosoftSQLJDBCDriver\\sqljdbc\_4.2\\enu\\jre8\\sqljdb… --- ## [Remove part of message string BEFORE and AFTER config](https://discuss.elastic.co/t/remove-part-of-message-string-before-and-after-config/220470)
**Author:** [@isarib](https://discuss.elastic.co/u/isarib)\ **Replies:** 2\ **Last updated:** [February 24, 2020, 1:58am UTC](https://discuss.elastic.co/t/remove-part-of-message-string-before-and-after-config/220470 "2020-02-24T01:58:54Z")
Hi, Message: /merlion/dpa2/cn133ta/j4\_sryuo/j4\_02\_sv\_ip I am trying to find a way to remove the strings BEFORE and AFTER a defined string in a message. In this message, the constant value is dpa2, and I want to discar… --- ## [Parse numbers from Storage units in logs](https://discuss.elastic.co/t/parse-numbers-from-storage-units-in-logs/218400)
**Author:** [@nmoham](https://discuss.elastic.co/u/nmoham)\ **Replies:** 9\ **Last updated:** [February 23, 2020, 9:23pm UTC](https://discuss.elastic.co/t/parse-numbers-from-storage-units-in-logs/218400 "2020-02-23T21:23:44Z")
Extract the NUMBER from logs with storage sizes. Sample Logs - physical.memory.total=31.4G, physical.memory.free=18.8G, swap.space.total=7.8G, swap.space.free=7.8G, heap.memory.used=4.5G, heap.memory.free=536.5M, heap.… --- ## [Connection performance Logstash =\> Logstash (Lumberjack protocol) - solution performance](https://discuss.elastic.co/t/connection-performance-logstash-logstash-lumberjack-protocol-solution-performance/220566)
**Author:** [@Rysiu](https://discuss.elastic.co/u/Rysiu)\ **Replies:** 0\ **Last updated:** [February 23, 2020, 7:02pm UTC](https://discuss.elastic.co/t/connection-performance-logstash-logstash-lumberjack-protocol-solution-performance/220566 "2020-02-23T19:02:11Z")
Connection performance Logstash =\> Logstash (Lumberjack protocol) - solution performance I have a question about Logstash =\> Logstash connection (data flow between two different Logstashes on different physical machines… --- ## [Elasticsearch Unreachable: hosted in aws from logstash](https://discuss.elastic.co/t/elasticsearch-unreachable-hosted-in-aws-from-logstash/220558)
**Author:** [@Ravikumar\_K](https://discuss.elastic.co/u/Ravikumar_K)\ **Replies:** 0\ **Last updated:** [February 23, 2020, 2:47pm UTC](https://discuss.elastic.co/t/elasticsearch-unreachable-hosted-in-aws-from-logstash/220558 "2020-02-23T14:47:57Z")
Hi, I am trying to connect to Elastic search hosted in aws from an EC2 instance via logstash getting below error. \[2020-02-23T14:46:08,685\]\[WARN \]\[logstash.outputs.elasticsearch\]\[main\] Attempted to resurrect connection… --- ## [Logstash errors about no available connections in the connection pool with elastic in the same server](https://discuss.elastic.co/t/logstash-errors-about-no-available-connections-in-the-connection-pool-with-elastic-in-the-same-server/220554)
**Author:** [@dimitris\_sb](https://discuss.elastic.co/u/dimitris_sb)\ **Replies:** 0\ **Last updated:** [February 23, 2020, 2:09pm UTC](https://discuss.elastic.co/t/logstash-errors-about-no-available-connections-in-the-connection-pool-with-elastic-in-the-same-server/220554 "2020-02-23T14:09:13Z")
Hi All, I noticed that one pipeline was falling behind and I checked the logs where I found the following entries, which should mean that there are no available connections or routes to elastic but both instances run in… --- ## [How to attach an attachment in the logstash output email](https://discuss.elastic.co/t/how-to-attach-an-attachment-in-the-logstash-output-email/220530)
**Author:** [@kc55](https://discuss.elastic.co/u/kc55)\ **Replies:** 0\ **Last updated:** [February 23, 2020, 5:37am UTC](https://discuss.elastic.co/t/how-to-attach-an-attachment-in-the-logstash-output-email/220530 "2020-02-23T05:37:21Z")
Hi all, In the guide format , they said the attachment is using array. However after the format and it seems the email has been sent but no attachment in it. output { if "shouldmail" in \[tags\] { email { to =\>… --- ## [Transforming/filtering json logs](https://discuss.elastic.co/t/transforming-filtering-json-logs/220523)
**Author:** [@swisscheese](https://discuss.elastic.co/u/swisscheese)\ **Replies:** 0\ **Last updated:** [February 23, 2020, 2:34am UTC](https://discuss.elastic.co/t/transforming-filtering-json-logs/220523 "2020-02-23T02:34:18Z")
I have some WAF logs (from AWS) that are already in json format. I want to modify them with logstash in a few specific ways, and it’s not clear to me from the docs what the best way to do this is. Hoping someone can poin… --- ## [Logstash startup error in docker container](https://discuss.elastic.co/t/logstash-startup-error-in-docker-container/220479)
**Author:** [@sahar](https://discuss.elastic.co/u/sahar)\ **Replies:** 1\ **Last updated:** [February 22, 2020, 2:40pm UTC](https://discuss.elastic.co/t/logstash-startup-error-in-docker-container/220479 "2020-02-22T14:40:06Z")
Hello, I'm trying to run ELK for the first time inside a docker-container using the image sebp/elk and it seems to be failing to start with a clean installation on logstash in particular. These are the startup logs: e… --- ## [Logstash grok parse failure, while kibana grok debugger works fine](https://discuss.elastic.co/t/logstash-grok-parse-failure-while-kibana-grok-debugger-works-fine/220485)
**Author:** [@sahar](https://discuss.elastic.co/u/sahar)\ **Replies:** 1\ **Last updated:** [February 22, 2020, 2:38pm UTC](https://discuss.elastic.co/t/logstash-grok-parse-failure-while-kibana-grok-debugger-works-fine/220485 "2020-02-22T14:38:47Z")
Hello, I tested a certain pattern in kibana debugger: As you can see, it works fine, but when sending the same log to logstash I always end up getting grok parse failure, how can I check what the issue is...? This i… --- ## [Ogstash.javapipeline \]\[main\] Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\ **Author:** [@saidjon](https://discuss.elastic.co/u/saidjon)\ **Replies:** 6\ **Last updated:** [February 22, 2020, 12:21pm UTC](https://discuss.elastic.co/t/ogstash-javapipeline-main-pipeline-aborted-due-to-error-pipeline-id-main-exception-logstash-unexpected-character-code-60-expected-a-valid-value-number-string-array-object-true-false-or-null/220465 "2020-02-22T12:21:26Z") hi there . im trying to import data to elastic cloud but getting this error . my conf file looks like this \ input{ file{ path =\> "/home/saidjon/Downloads/big\_data/urbandict-word-defs.csv" start\_position =\>… --- ## [Ruby exception occurred: undefined method \`/' for nil:NilClass](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-for-nil-nilclass/220338)
**Author:** [@Huy\_Le1](https://discuss.elastic.co/u/Huy_Le1)\ **Replies:** 2\ **Last updated:** [February 22, 2020, 7:38am UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-for-nil-nilclass/220338 "2020-02-22T07:38:36Z")
I configured my logstash exactly the same as filter { if "modsecurity" in \[tags\] { ruby { code =\> " if !event.get('message').nil? modSecSectionData = event.get('message').split(/(?:--… --- ## [Logstash date filter match pattern](https://discuss.elastic.co/t/logstash-date-filter-match-pattern/220458)
**Author:** [@vikasp](https://discuss.elastic.co/u/vikasp)\ **Replies:** 3\ **Last updated:** [February 22, 2020, 12:40am UTC](https://discuss.elastic.co/t/logstash-date-filter-match-pattern/220458 "2020-02-22T00:40:23Z")
Hi, I have a logstash config that pulls from jdbc and pushes it to elasticsearch based on the select statement where my tracking column is an attribute mod\_dttm (Example: 23-JAN-20 12.23.28.771000000 PM). So, I created… --- ## [Csv filter match date field](https://discuss.elastic.co/t/csv-filter-match-date-field/220428)
**Author:** [@david.derosa](https://discuss.elastic.co/u/david.derosa)\ **Replies:** 4\ **Last updated:** [February 21, 2020, 11:04pm UTC](https://discuss.elastic.co/t/csv-filter-match-date-field/220428 "2020-02-21T23:04:05Z")
I am trying to parse e csv. All works fine except for date field. This is the format: 2020-02-21 18:25:37.254774 and this is my date code: date { match =\> \["date", "yyyy-MM-dd HH:mm:ss.SSS"\] target =\> "@times… --- ## [Question around field mappings for siem / audit](https://discuss.elastic.co/t/question-around-field-mappings-for-siem-audit/220462)
**Author:** [@runtman](https://discuss.elastic.co/u/runtman)\ **Replies:** 0\ **Last updated:** [February 21, 2020, 10:56pm UTC](https://discuss.elastic.co/t/question-around-field-mappings-for-siem-audit/220462 "2020-02-21T22:56:50Z")
Hello, first of all I don't really know if this should go into beats or logstash so I have added it here for now. We use logstash for the majority of our logs, mostly because we use a lot of log types that are not popul… --- ## [Indexing double nested relation using logstash](https://discuss.elastic.co/t/indexing-double-nested-relation-using-logstash/220447)
**Author:** [@Louie\_Sota](https://discuss.elastic.co/u/Louie_Sota)\ **Replies:** 0\ **Last updated:** [February 21, 2020, 7:54pm UTC](https://discuss.elastic.co/t/indexing-double-nested-relation-using-logstash/220447 "2020-02-21T19:54:09Z")
Hello there, I'm configuring to use logstash to index all the data I've got in the database, and I'm currently having problems trying to find a solution for what I'm doing. I'm trying to join 3 tables together, to crea… --- ## [Error with manage\_template on multiple elasticsearch outputs](https://discuss.elastic.co/t/error-with-manage-template-on-multiple-elasticsearch-outputs/220435)
**Author:** [@tmccoy](https://discuss.elastic.co/u/tmccoy)\ **Replies:** 0\ **Last updated:** [February 21, 2020, 6:17pm UTC](https://discuss.elastic.co/t/error-with-manage-template-on-multiple-elasticsearch-outputs/220435 "2020-02-21T18:17:19Z")
Hi, thanks in advance for your assistance! Description: I have an output configuration where I am using an if statement to select an output to elasticsearch, essentially I want to index documents into an index dependin… --- ## [Logstash syslog output ignores message](https://discuss.elastic.co/t/logstash-syslog-output-ignores-message/220434)
**Author:** [@mtudisco](https://discuss.elastic.co/u/mtudisco)\ **Replies:** 0\ **Last updated:** [February 21, 2020, 6:16pm UTC](https://discuss.elastic.co/t/logstash-syslog-output-ignores-message/220434 "2020-02-21T18:16:10Z")
Hi, logstash syslog output is ignoring the message y configure and adds something "%{host}" in the message, dont know what may be wrong. I have 2 servers, one with filebeat 1.3.1 reading apache logs and shipping them to… --- ## [Filter text file use logstash](https://discuss.elastic.co/t/filter-text-file-use-logstash/220427)
**Author:** [@Achmad\_Arifin](https://discuss.elastic.co/u/Achmad_Arifin)\ **Replies:** 0\ **Last updated:** [February 21, 2020, 5:36pm UTC](https://discuss.elastic.co/t/filter-text-file-use-logstash/220427 "2020-02-21T17:36:47Z")
how do I display in the logstash all the contents in a .txt file? this is the screenshot of my txt and this is my logstash config input { file { #type =\> "doc" #path =\> "/home/wsagiva/Desktop/logs/log.txt" path … --- ## [Convert IP Address fro Hexidecimal to Normal Format](https://discuss.elastic.co/t/convert-ip-address-fro-hexidecimal-to-normal-format/220419)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 0\ **Last updated:** [February 21, 2020, 4:44pm UTC](https://discuss.elastic.co/t/convert-ip-address-fro-hexidecimal-to-normal-format/220419 "2020-02-21T16:44:04Z")
Hello, I have records/logs messages that have fields with a single IP address in them, however the IP addresses are in hexidecimal format. How can I convert those to regular IP address format? For example: AP\_TTRIP =… --- ## [Logstash-Logback Encoder: "IllegalAccessError: failed to access class \[...\]"](https://discuss.elastic.co/t/logstash-logback-encoder-illegalaccesserror-failed-to-access-class/219216)
**Author:** [@Ruhrpottler](https://discuss.elastic.co/u/Ruhrpottler)\ **Replies:** 2\ **Last updated:** [February 21, 2020, 4:17pm UTC](https://discuss.elastic.co/t/logstash-logback-encoder-illegalaccesserror-failed-to-access-class/219216 "2020-02-21T16:17:52Z")
I try to send logs from a java application with the Logstash tcp socket appender to logstash. The java app. already works with logback 1.1.9 (slf4j) and other appenders. Now I added the following lines to the logback-te… --- ## [RabbitMQ input module - connection error, proxy](https://discuss.elastic.co/t/rabbitmq-input-module-connection-error-proxy/220252)
**Author:** [@wedkarz014](https://discuss.elastic.co/u/wedkarz014)\ **Replies:** 1\ **Last updated:** [February 21, 2020, 4:08pm UTC](https://discuss.elastic.co/t/rabbitmq-input-module-connection-error-proxy/220252 "2020-02-21T16:08:16Z")
Hi everyone, I have container with latest logstash image so its v7.5.2, I chose this one because in past there was a problem with default host and port. In the latest version of logstash it looks that it works good but … --- ## [Unix\_ms time to date](https://discuss.elastic.co/t/unix-ms-time-to-date/220262)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [February 21, 2020, 3:36pm UTC](https://discuss.elastic.co/t/unix-ms-time-to-date/220262 "2020-02-21T15:36:11Z")
I have date\_time = 1582225804228 which is Thursday, February 20, 2020 1:10:04.228 when I use following filter it convert that to 2020-02-20T19:10:04.228Z ( which is UTC) time format But I also want another field creat… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=399) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=401)