# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=241 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 242 --- ## [A multilne message - how to deal with?](https://discuss.elastic.co/t/a-multilne-message-how-to-deal-with/269375)
**Author:** [@awer1967](https://discuss.elastic.co/u/awer1967)\ **Replies:** 2\ **Last updated:** [April 7, 2021, 3:53am UTC](https://discuss.elastic.co/t/a-multilne-message-how-to-deal-with/269375 "2021-04-07T03:53:14Z")
Hello ! I am in complete stuck in some dealing with Logstash. There is a non-structured information gotten from one source , divided by \\n char I cut it out in input section into set of messages with the multiline c… --- ## [java.lang.OutOfMemoryError: Java heap space with proc statement](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-with-proc-statement/269230)
**Author:** [@schmitt\_kevin](https://discuss.elastic.co/u/schmitt_kevin)\ **Replies:** 3\ **Last updated:** [April 6, 2021, 9:32pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-with-proc-statement/269230 "2021-04-06T21:32:11Z")
Hello, I use logstash with sql procedure statement to insert json in elastic, no problem here. But when i use server with less RAM i have error java.lang.OutOfMemoryError: Java heap space with proc statement CALL get\_… --- ## [Logstash Post Install of plugins](https://discuss.elastic.co/t/logstash-post-install-of-plugins/269348)
**Author:** [@Dallas\_Toth](https://discuss.elastic.co/u/Dallas_Toth)\ **Replies:** 1\ **Last updated:** [April 6, 2021, 3:47pm UTC](https://discuss.elastic.co/t/logstash-post-install-of-plugins/269348 "2021-04-06T15:47:46Z")
For those who are struggeling with the install of plugins and having them install on spinning up a container I have wrote this mount the file from your volumn outside the container to Container Path: /usr/share/logstas… --- ## [I am getting this error when running logstash configuration pipeline through logstash.yml file](https://discuss.elastic.co/t/i-am-getting-this-error-when-running-logstash-configuration-pipeline-through-logstash-yml-file/269318)
**Author:** [@Husnain](https://discuss.elastic.co/u/Husnain)\ **Replies:** 2\ **Last updated:** [April 6, 2021, 3:40pm UTC](https://discuss.elastic.co/t/i-am-getting-this-error-when-running-logstash-configuration-pipeline-through-logstash-yml-file/269318 "2021-04-06T15:40:23Z")
I have written a logstash configuration pipeline that receive events from syslog and stores that events in elasticsearch.When I run this pipeline through command line everything works fine,but when I run this pipeline th… --- ## [Grok works (no grok parse failure) but doesnt create the fields](https://discuss.elastic.co/t/grok-works-no-grok-parse-failure-but-doesnt-create-the-fields/269203)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 7\ **Last updated:** [April 6, 2021, 3:15pm UTC](https://discuss.elastic.co/t/grok-works-no-grok-parse-failure-but-doesnt-create-the-fields/269203 "2021-04-06T15:15:45Z")
Hi I have a field called "if\_speed\_in\_out", this field contains strings like this "100 Mbps:100 Mbps" i have tested the grok on kibana devs tools and works, but never create the fields defined in the grok, just get the o… --- ## [Logstash translate filter and converting integers to strings](https://discuss.elastic.co/t/logstash-translate-filter-and-converting-integers-to-strings/268803)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 2\ **Last updated:** [April 6, 2021, 2:48pm UTC](https://discuss.elastic.co/t/logstash-translate-filter-and-converting-integers-to-strings/268803 "2021-04-06T14:48:13Z")
I am on logstash 7.11. I need a translate from an integer code to real speed value. mutate { convert =\> { "ifspeed" =\> "string" } } translate { field =\> "\[ifspeed\]" destination =\> "\[inf\_speed\]" dictionary =\> … --- ## [Logstash workers dying randomly](https://discuss.elastic.co/t/logstash-workers-dying-randomly/269342)
**Author:** [@Fran\_Raknic](https://discuss.elastic.co/u/Fran_Raknic)\ **Replies:** 0\ **Last updated:** [April 6, 2021, 2:20pm UTC](https://discuss.elastic.co/t/logstash-workers-dying-randomly/269342 "2021-04-06T14:20:33Z")
Hello, we use Logstash to collect syslog messages from network devices. The instance has one pipeline listening on UDP port 7514 with 16 workers, the filter part consists of a couple of grok patterns, mutate fileds, DNS… --- ## [Initial JDBC input for a tracking column type of timestamp](https://discuss.elastic.co/t/initial-jdbc-input-for-a-tracking-column-type-of-timestamp/269163)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 2\ **Last updated:** [April 6, 2021, 12:25pm UTC](https://discuss.elastic.co/t/initial-jdbc-input-for-a-tracking-column-type-of-timestamp/269163 "2021-04-06T12:25:11Z")
I am trying to pull data from an MS SQL Server database. The data is refreshed on the hour and I want to pull at 15 minutes after the top of the hour. The input looks like this: input { jdbc { jdbc\_driver\_library … --- ## [Grok filter](https://discuss.elastic.co/t/grok-filter/269194)
**Author:** [@Dinesh\_Sharma](https://discuss.elastic.co/u/Dinesh_Sharma)\ **Replies:** 10\ **Last updated:** [April 6, 2021, 10:54am UTC](https://discuss.elastic.co/t/grok-filter/269194 "2021-04-06T10:54:51Z")
Hi, I have a csv file which has currently three rows in it which are as follow: name,age,ip A,12,10.11.1.12 B,13,10.11.1 Now I want to check during the data ingestion via this csv that whether the IP is in proper fo… --- ## [How to configure logstash file to make nested object inside another nested filed from sql query?](https://discuss.elastic.co/t/how-to-configure-logstash-file-to-make-nested-object-inside-another-nested-filed-from-sql-query/268077)
**Author:** [@My-project-repositor](https://discuss.elastic.co/u/My-project-repositor)\ **Replies:** 2\ **Last updated:** [April 6, 2021, 10:28am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-file-to-make-nested-object-inside-another-nested-filed-from-sql-query/268077 "2021-04-06T10:28:12Z")
I am new in Elasticsearch and logstash and I have an issue with nested objects. So what I want to ask is how can I create index with multiple nested fields inside another nested fields. How can I build the data inside l… --- ## [Nested fields ElasticSearch and Logstash](https://discuss.elastic.co/t/nested-fields-elasticsearch-and-logstash/269306)
**Author:** [@My-project-repositor](https://discuss.elastic.co/u/My-project-repositor)\ **Replies:** 0\ **Last updated:** [April 6, 2021, 9:04am UTC](https://discuss.elastic.co/t/nested-fields-elasticsearch-and-logstash/269306 "2021-04-06T09:04:39Z")
Hello, I am new in ES and logstash so I need some help I need to perform an indexation of data from postgres in ES My problem is that there are many relationships in the database, and I don't know how to perform many n… --- ## [Logstash email output using linux sendmail](https://discuss.elastic.co/t/logstash-email-output-using-linux-sendmail/269295)
**Author:** [@crazy\_coder](https://discuss.elastic.co/u/crazy_coder)\ **Replies:** 0\ **Last updated:** [April 6, 2021, 7:53am UTC](https://discuss.elastic.co/t/logstash-email-output-using-linux-sendmail/269295 "2021-04-06T07:53:02Z")
Hi i want to use Linux sendmail service to send email using Logstash email output plugin. how can i do it. email { to =\> "crazy@noreply.com" codec =\> "plain" body =\> "hi how are you" via =\> "sendmail" } Is these… --- ## [Add new field with value from grok filter](https://discuss.elastic.co/t/add-new-field-with-value-from-grok-filter/269253)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 2\ **Last updated:** [April 5, 2021, 7:51pm UTC](https://discuss.elastic.co/t/add-new-field-with-value-from-grok-filter/269253 "2021-04-05T19:51:13Z")
Hi all, I need to add a value to a field from a variable within a grok filter. filter { grok { match =\> \[ "message", "(?\(.\*?))\\t(?\(.\*?))" \] } mutate { add\_field =\> { "container\_id" =\> "%{u… --- ## [Timeout executing grok](https://discuss.elastic.co/t/timeout-executing-grok/269184)
**Author:** [@Andrey\_RF](https://discuss.elastic.co/u/Andrey_RF)\ **Replies:** 8\ **Last updated:** [April 5, 2021, 2:27pm UTC](https://discuss.elastic.co/t/timeout-executing-grok/269184 "2021-04-05T14:27:19Z")
I have a quite long regex to pars my log message. But it drops with Timeout executing grok . Does have any mechanisms to speed up regex work? Also I have a multiple regexs and if one is failed does it other works ? --- ## [Parsing quoted and escaped json events](https://discuss.elastic.co/t/parsing-quoted-and-escaped-json-events/269233)
**Author:** [@LeonT123](https://discuss.elastic.co/u/LeonT123)\ **Replies:** 0\ **Last updated:** [April 5, 2021, 12:38pm UTC](https://discuss.elastic.co/t/parsing-quoted-and-escaped-json-events/269233 "2021-04-05T12:38:23Z")
Hi I'm trying to find a way to parse a JSON encoded message like this. I do not know if filebeat is a good choice here or logstash is better one. the events are look like this and I'm looking for a way to extract the "… --- ## [GROK query generating .keyword fields in Elasticsearch](https://discuss.elastic.co/t/grok-query-generating-keyword-fields-in-elasticsearch/269152)
**Author:** [@finbarr996](https://discuss.elastic.co/u/finbarr996)\ **Replies:** 2\ **Last updated:** [April 5, 2021, 10:24am UTC](https://discuss.elastic.co/t/grok-query-generating-keyword-fields-in-elasticsearch/269152 "2021-04-05T10:24:57Z")
Hi there, I'm happily parsing a bunch of syslog files in Logstash, but notice that an entry like %{DATA:Something} creates a Something field and additionally a Something.keyword field in Elasticsearch. Is there a way t… --- ## [Encountered a retryable error. will retry with exponential backoff (code:503)](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-503/269217)
**Author:** [@lilyyy](https://discuss.elastic.co/u/lilyyy)\ **Replies:** 0\ **Last updated:** [April 5, 2021, 9:09am UTC](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-503/269217 "2021-04-05T09:09:41Z")
Hello. I have a problem when i use logstash. My ELK version is 6.8 and when i start logstash, it was going well at first but a few times later i got this error. I've searched about this error on google but i was not abl… --- ## [Geoip - Apache logs](https://discuss.elastic.co/t/geoip-apache-logs/269166)
**Author:** [@gloupe](https://discuss.elastic.co/u/gloupe)\ **Replies:** 5\ **Last updated:** [April 4, 2021, 7:07pm UTC](https://discuss.elastic.co/t/geoip-apache-logs/269166 "2021-04-04T19:07:12Z")
Hi, I'm trying to get geoip info from ipclient from my access logs generated by a apache2 server. I tried this filter in logstash: filter { if \[tags\]\[path\] =~ "/var/log/apache2/xxx-access.log" { … --- ## [Using grok filter to parse log file](https://discuss.elastic.co/t/using-grok-filter-to-parse-log-file/269138)
**Author:** [@bab](https://discuss.elastic.co/u/bab)\ **Replies:** 3\ **Last updated:** [April 4, 2021, 1:20pm UTC](https://discuss.elastic.co/t/using-grok-filter-to-parse-log-file/269138 "2021-04-04T13:20:02Z")
Hi guys, i'm trying to parse the following log files, i succeed to filter all fields unless the last one, can some one help pls, thank you. Log file: 1;2;06-19-15;start inbound processing;50034744;1;ok;2021-03-18 16:1… --- ## [Give path as an argument](https://discuss.elastic.co/t/give-path-as-an-argument/269173)
**Author:** [@vincedt09](https://discuss.elastic.co/u/vincedt09)\ **Replies:** 1\ **Last updated:** [April 4, 2021, 12:50pm UTC](https://discuss.elastic.co/t/give-path-as-an-argument/269173 "2021-04-04T12:50:52Z")
Hi! Is there a way to give path as an argument? I was thinking at something like this: input { file { path =\> argv sincedb\_path =\> "NUL" start\_position =\> "beginning" } } logsta… --- ## ["Expected one of \[ \\\\t\\\\r\\\\n\], \\"#\\", \\"{\\", \\"}\\"](https://discuss.elastic.co/t/expected-one-of-t-r-n/269151)
**Author:** [@tmrgbox](https://discuss.elastic.co/u/tmrgbox)\ **Replies:** 2\ **Last updated:** [April 4, 2021, 8:14am UTC](https://discuss.elastic.co/t/expected-one-of-t-r-n/269151 "2021-04-04T08:14:35Z")
Hi, I'm trying to parse a log using the following config: input { udp { port =\> 5555 type =\> "esetlog" add\_field =\> { "etl\_input\_port" =\> 5555 "etl\_input\_protocol" =\> "udp" } } } filter { if \[type\] == "eset… --- ## [Don't save in ES logs that have in tag \_grokparsefailure or \_dateparsefailure](https://discuss.elastic.co/t/dont-save-in-es-logs-that-have-in-tag-grokparsefailure-or-dateparsefailure/269141)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 2\ **Last updated:** [April 4, 2021, 5:20am UTC](https://discuss.elastic.co/t/dont-save-in-es-logs-that-have-in-tag-grokparsefailure-or-dateparsefailure/269141 "2021-04-04T05:20:42Z")
Hi, How can i filter the logs that are saved within ES in order to have only valid data that has no grok parse failure or date parse error? My approach is bellow output { if ("\_grokparsefailure" not in \[tags\]) or ("\_… --- ## [Logstash pipeline schedule](https://discuss.elastic.co/t/logstash-pipeline-schedule/269139)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 3\ **Last updated:** [April 4, 2021, 5:18am UTC](https://discuss.elastic.co/t/logstash-pipeline-schedule/269139 "2021-04-04T05:18:45Z")
Hi there I\`ve faced with a problem... I have a running Logstash with multiple pipelines. I just want to run one of these pipelines from 00:00 am until 08:00 am everyday According to Logstash documentation, pipeline co… --- ## [Date filter does not work](https://discuss.elastic.co/t/date-filter-does-not-work/269148)
**Author:** [@Andrey\_RF](https://discuss.elastic.co/u/Andrey_RF)\ **Replies:** 3\ **Last updated:** [April 3, 2021, 6:32pm UTC](https://discuss.elastic.co/t/date-filter-does-not-work/269148 "2021-04-03T18:32:34Z")
I use grok and date filter to set correct @timestamp. It works in simple case. But when I added if condition @timestamp no longer installed from the message. filter { if 'backend.info.log' in \[log\]\[file\]\[path\] { g… --- ## [Logstash is not processing logs](https://discuss.elastic.co/t/logstash-is-not-processing-logs/269084)
**Author:** [@James\_P](https://discuss.elastic.co/u/James_P)\ **Replies:** 1\ **Last updated:** [April 3, 2021, 7:49am UTC](https://discuss.elastic.co/t/logstash-is-not-processing-logs/269084 "2021-04-03T07:49:10Z")
Hi, Logstash was installed and was receiving logs from filebeat and syslog file stream. Currently logstash starts up successfully but no logs are being processed. \[2021-04-02T08:48:29,269\]\[INFO \]\[logstash.runner … --- ## [Help me writing grok filter for the pattern](https://discuss.elastic.co/t/help-me-writing-grok-filter-for-the-pattern/268794)
**Author:** [@abhishek\_s1](https://discuss.elastic.co/u/abhishek_s1)\ **Replies:** 8\ **Last updated:** [April 2, 2021, 3:39pm UTC](https://discuss.elastic.co/t/help-me-writing-grok-filter-for-the-pattern/268794 "2021-04-02T15:39:21Z")
I've http log of the form 10.255.255.255 - jira \[11/Mar/2021:10:00:03 -0800\] "GET /svn/repos/branches/feature-IPv6-TWLP-3.2/ZProxyHealthManager.cpp HTTP/1.1" 200 29110 & my pattern is: %{IPORHOST:client\_ip} %{HTTPDUS… --- ## [Best way to maintain total count for rollup documents?](https://discuss.elastic.co/t/best-way-to-maintain-total-count-for-rollup-documents/269109)
**Author:** [@hsalim](https://discuss.elastic.co/u/hsalim)\ **Replies:** 0\ **Last updated:** [April 2, 2021, 1:37pm UTC](https://discuss.elastic.co/t/best-way-to-maintain-total-count-for-rollup-documents/269109 "2021-04-02T13:37:54Z")
Hello, I am performing Elastic rollups on an index with the following streaming documents and fields. Rollup is on Field 1: Field1 Field2 doc1 1200 a doc2 1200 b doc3 1200 c doc4 … --- ## [Sending data from logstash to ElasticSearch (ECK): Certificate issue](https://discuss.elastic.co/t/sending-data-from-logstash-to-elasticsearch-eck-certificate-issue/269106)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 0\ **Last updated:** [April 2, 2021, 1:25pm UTC](https://discuss.elastic.co/t/sending-data-from-logstash-to-elasticsearch-eck-certificate-issue/269106 "2021-04-02T13:25:11Z")
have setup operator based elasticsearch & kibana.. but since logstash is not within ECK, running logstash as standalone. Unfortunately, i'm trying it hard to find to connect to Elasticsearch due to certificate issue. ou… --- ## [Set timestamp from message](https://discuss.elastic.co/t/set-timestamp-from-message/269073)
**Author:** [@Andrey\_RF](https://discuss.elastic.co/u/Andrey_RF)\ **Replies:** 4\ **Last updated:** [April 2, 2021, 11:12am UTC](https://discuss.elastic.co/t/set-timestamp-from-message/269073 "2021-04-02T11:12:38Z")
I have a pipeline to receive logs from different programs with different timestamp formats. input { beats { port =\> 5000 } } filter { date { match =\> \["message", "yyyy-MM-d… --- ## [Logstash pipeline order](https://discuss.elastic.co/t/logstash-pipeline-order/269057)
**Author:** [@Narayan\_Banik](https://discuss.elastic.co/u/Narayan_Banik)\ **Replies:** 3\ **Last updated:** [April 2, 2021, 8:34am UTC](https://discuss.elastic.co/t/logstash-pipeline-order/269057 "2021-04-02T08:34:54Z")
Hello All, Is there any order of logstash pipelines config: I've below pipelines: \[root@gzp-p-qv-logst-3 pipeline\]# ls -larth total 60K drwxr-xr-x. 3 logstash logstash 42 Oct 11 2019 .. -rw-r--r--. 1 root root … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=240) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=242)