# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=258 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 259 --- ## [Ruby code to add entry into an array?](https://discuss.elastic.co/t/ruby-code-to-add-entry-into-an-array/263046)
**Author:** [@aidanoc15](https://discuss.elastic.co/u/aidanoc15)\ **Replies:** 3\ **Last updated:** [February 3, 2021, 5:37pm UTC](https://discuss.elastic.co/t/ruby-code-to-add-entry-into-an-array/263046 "2021-02-03T17:37:10Z")
Hi, I'm using logstash as a method posting tickets into our ticketing system using data from an elasticsearch index. The API for out ticketing system expects all of the custom fields for our ticket to come through as an … --- ## [Use NGINX with Logstash](https://discuss.elastic.co/t/use-nginx-with-logstash/263126)
**Author:** [@carmine.fabrizio](https://discuss.elastic.co/u/carmine.fabrizio)\ **Replies:** 0\ **Last updated:** [February 3, 2021, 1:25pm UTC](https://discuss.elastic.co/t/use-nginx-with-logstash/263126 "2021-02-03T13:25:37Z")
Do we have any kind of benefit or improvement if I configure an NGINX on top of LS? The only one that I can see is that I'll just configure one entry in my output.logstash and having a load balancer with a few helthchec… --- ## [Logstash errors](https://discuss.elastic.co/t/logstash-errors/263116)
**Author:** [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)\ **Replies:** 0\ **Last updated:** [February 3, 2021, 12:02pm UTC](https://discuss.elastic.co/t/logstash-errors/263116 "2021-02-03T12:02:08Z")
Hi! Since last week Im having a lot of errors in logstash like these: \[ERROR\] 2021-02-03 11:59:46.340 \[nioEventLoopGroup-2-1\] tcp - Error in Netty pipeline: java.io.IOException: Connection reset by peer \[ERROR\] 2021-0… --- ## [Logstash error. Expected symbols](https://discuss.elastic.co/t/logstash-error-expected-symbols/262752)
**Author:** [@111435](https://discuss.elastic.co/u/111435)\ **Replies:** 5\ **Last updated:** [February 3, 2021, 11:38am UTC](https://discuss.elastic.co/t/logstash-error-expected-symbols/262752 "2021-02-03T11:38:34Z")
Hello, I am new at logstash, so please could you help me. I have this log: 2021-01-27 00:20:10 GET /ads/servlet/dbRequest?BNumber=353&ANumber=485739274 200 0.013 There are tabs between "date", "time", "GET" and "… --- ## [@timestamp in date\_nanos](https://discuss.elastic.co/t/timestamp-in-date-nanos/263112)
**Author:** [@ritusingh](https://discuss.elastic.co/u/ritusingh)\ **Replies:** 0\ **Last updated:** [February 3, 2021, 11:17am UTC](https://discuss.elastic.co/t/timestamp-in-date-nanos/263112 "2021-02-03T11:17:19Z")
Hi , I am using ELK 7.7.0 and want to know that how can I change @timestamp to date\_nanos while sending logs from logstash to elasticsearch? I know that we can change format of @timestamp to date\_nanos , but it simply … --- ## [Syslog pipeline =\> \[host\] error in ElasticSearch](https://discuss.elastic.co/t/syslog-pipeline-host-error-in-elasticsearch/262657)
**Author:** [@expressionlibre](https://discuss.elastic.co/u/expressionlibre)\ **Replies:** 11\ **Last updated:** [February 3, 2021, 10:32am UTC](https://discuss.elastic.co/t/syslog-pipeline-host-error-in-elasticsearch/262657 "2021-02-03T10:32:07Z")
Hi, Sorry may be a newbie question but, I read lots of treads and examples on the net to setup syslog pipeline but I still cannot make my logstash pipeline to function in ES. platform : CentOS 8.3. Epel versions for … --- ## [An we add log retentions for elasticsearch, logstash, kibana and metric beat](https://discuss.elastic.co/t/an-we-add-log-retentions-for-elasticsearch-logstash-kibana-and-metric-beat/263096)
**Author:** [@Somesh\_ng](https://discuss.elastic.co/u/Somesh_ng)\ **Replies:** 0\ **Last updated:** [February 3, 2021, 8:37am UTC](https://discuss.elastic.co/t/an-we-add-log-retentions-for-elasticsearch-logstash-kibana-and-metric-beat/263096 "2021-02-03T08:37:59Z")
Can we add log retentions for elasticsearch, logstash, kibana and metric beat like below in .yml file logging.level: debug logging.to\_files: true logging.to\_syslog: false logging.files: path: /log… --- ## [Error while indexing data to App Search using Logstash](https://discuss.elastic.co/t/error-while-indexing-data-to-app-search-using-logstash/263094)
**Author:** [@sanju1323](https://discuss.elastic.co/u/sanju1323)\ **Replies:** 0\ **Last updated:** [February 3, 2021, 8:34am UTC](https://discuss.elastic.co/t/error-while-indexing-data-to-app-search-using-logstash/263094 "2021-02-03T08:34:19Z")
Hi Team, I have been trying to index the data from local Elasticsearch cluster to App Search engine using logstash. But I'm getting the below error. Copy to clipboard \[2021-02-01T13:45:30,097\]\[DEBUG\]\[logstash.javapipe… --- ## [Connection refused logstash to elasticsearch](https://discuss.elastic.co/t/connection-refused-logstash-to-elasticsearch/262970)
**Author:** [@vikram\_singh](https://discuss.elastic.co/u/vikram_singh)\ **Replies:** 8\ **Last updated:** [February 3, 2021, 8:17am UTC](https://discuss.elastic.co/t/connection-refused-logstash-to-elasticsearch/262970 "2021-02-03T08:17:20Z")
Hi, I enable trial licence for my 3 node elasticsearch cluster, on docker. And now elasticsearch and kibana is running on https but logstash start giving problem. Earlier ELK was running on http correctly. My all setup … --- ## [Remove ^M from message](https://discuss.elastic.co/t/remove-m-from-message/263058)
**Author:** [@daemon](https://discuss.elastic.co/u/daemon)\ **Replies:** 2\ **Last updated:** [February 3, 2021, 3:56am UTC](https://discuss.elastic.co/t/remove-m-from-message/263058 "2021-02-03T03:56:46Z")
Hi, I am trying to ingest Java exceptions into Elasticsearch via logstash. In Kibana, they show up like this, \[2021-01-29 16:00:36.875\] \[139642182200000\] \[INF\] : \[58.122.202.198:57076\] ==\> RTSP OPTIONS Resp: RTSP/1.0… --- ## [Create multiple Index for different steams of data](https://discuss.elastic.co/t/create-multiple-index-for-different-steams-of-data/261203)
**Author:** [@leinad](https://discuss.elastic.co/u/leinad)\ **Replies:** 9\ **Last updated:** [February 3, 2021, 3:45am UTC](https://discuss.elastic.co/t/create-multiple-index-for-different-steams-of-data/261203 "2021-02-03T03:45:05Z")
Hi , I am very new on setting up log stash, but have been using Kibana on user level of createing dashboard and analysis using basic templates My input logs are all JSON types, this is the logstash config which works p… --- ## [Logstash kafka output plugin - What happens if Kafka goes down](https://discuss.elastic.co/t/logstash-kafka-output-plugin-what-happens-if-kafka-goes-down/263012)
**Author:** [@vigneshr35](https://discuss.elastic.co/u/vigneshr35)\ **Replies:** 4\ **Last updated:** [February 3, 2021, 3:28am UTC](https://discuss.elastic.co/t/logstash-kafka-output-plugin-what-happens-if-kafka-goes-down/263012 "2021-02-03T03:28:30Z")
Hi All, I am looking to use the Logstash Kafka output plugin to post events to a Kafka topic. While logstash is pushing logs what happens if Kafka server goes down suddenly for some reason? By the time we bring back Ka… --- ## [Fail to install logstash plugin, version 7.6.2](https://discuss.elastic.co/t/fail-to-install-logstash-plugin-version-7-6-2/262443)
**Author:** [@wink](https://discuss.elastic.co/u/wink)\ **Replies:** 7\ **Last updated:** [February 3, 2021, 2:59am UTC](https://discuss.elastic.co/t/fail-to-install-logstash-plugin-version-7-6-2/262443 "2021-02-03T02:59:01Z")
Hi, I just do everything like on https://www.elastic.co/guide/en/logstash/current/java-filter-plugin.html#\_copy\_the\_example\_repo\_3. I did not change any code. But It comes out some error when I try to install the plug… --- ## [Monitor state transistion of ongoing application](https://discuss.elastic.co/t/monitor-state-transistion-of-ongoing-application/263027)
**Author:** [@schuessa](https://discuss.elastic.co/u/schuessa)\ **Replies:** 1\ **Last updated:** [February 3, 2021, 2:20am UTC](https://discuss.elastic.co/t/monitor-state-transistion-of-ongoing-application/263027 "2021-02-03T02:20:13Z")
I'm trying to set up the ability have some application log jobs such as 2021-02-02T12:55:51-0500 \[JOB:444\] \[START\] My external job has started 2021-02-02T12:55:51-0500 \[JOB:444\] \[MY\_EVENT\] Some update to the job… --- ## [Parsing Mapping and finding the right ECS fields for logs in general](https://discuss.elastic.co/t/parsing-mapping-and-finding-the-right-ecs-fields-for-logs-in-general/262451)
**Author:** [@cgekoski](https://discuss.elastic.co/u/cgekoski)\ **Replies:** 1\ **Last updated:** [February 2, 2021, 11:43pm UTC](https://discuss.elastic.co/t/parsing-mapping-and-finding-the-right-ecs-fields-for-logs-in-general/262451 "2021-02-02T23:43:52Z")
Hello everyone. I'm going to start with saying sorry TLDR but hope someone can shine the light in the right direction. Pretty new to logstash/elastic and trying to come to grasp with the ECS field guide and want to teach… --- ## [Parsing multiline stacktrace](https://discuss.elastic.co/t/parsing-multiline-stacktrace/263022)
**Author:** [@RFeiten](https://discuss.elastic.co/u/RFeiten)\ **Replies:** 3\ **Last updated:** [February 2, 2021, 7:36pm UTC](https://discuss.elastic.co/t/parsing-multiline-stacktrace/263022 "2021-02-02T19:36:53Z")
Having a hard time here to index my logs containing multiple stack trace lines, as following Here are a log sample 2021-01-04T00:47:39.6082940+00:00 0HM5E5E2861GQ:00000005 \[ERR\] Something went wrong:(Value cannot be nu… --- ## [Processing events in json format and do correlation to find the duration based on correlation id](https://discuss.elastic.co/t/processing-events-in-json-format-and-do-correlation-to-find-the-duration-based-on-correlation-id/261320)
**Author:** [@sdingria](https://discuss.elastic.co/u/sdingria)\ **Replies:** 5\ **Last updated:** [February 2, 2021, 6:20pm UTC](https://discuss.elastic.co/t/processing-events-in-json-format-and-do-correlation-to-find-the-duration-based-on-correlation-id/261320 "2021-02-02T18:20:35Z")
Newbie in logstash, 24 hours, so you might find the problem is trivial. input is multiple records in json format. PFB. Below is log stash config file. while executing, getting \_jsonparsefailure. input { stdin{} } … --- ## [Unable to parse date in grok and filters](https://discuss.elastic.co/t/unable-to-parse-date-in-grok-and-filters/262988)
**Author:** [@RFeiten](https://discuss.elastic.co/u/RFeiten)\ **Replies:** 4\ **Last updated:** [February 2, 2021, 5:53pm UTC](https://discuss.elastic.co/t/unable-to-parse-date-in-grok-and-filters/262988 "2021-02-02T17:53:38Z")
I have the following message 2021-01-04T00:04:00.8033345+00:00 0HM5E5E28610F:00000001 \[INF\] MyMessage What I'm trying to do is to extract each of those fields into a separate column for my index. This is what I have b… --- ## [Need to fetch only selected data from Grok filter](https://discuss.elastic.co/t/need-to-fetch-only-selected-data-from-grok-filter/262972)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 2\ **Last updated:** [February 2, 2021, 5:43pm UTC](https://discuss.elastic.co/t/need-to-fetch-only-selected-data-from-grok-filter/262972 "2021-02-02T17:43:30Z")
Sample Log : 2020-12-16T04:43:43Z Bitcoin Core version v0.20.1.0-g7ff64311bee570874c4f0dfa18f518552188df08 (release build) I want to get the only version(v0.20.1.0) from the above log, Note : version string is a whole … --- ## [Logstash is not sending the data to elasticsearch](https://discuss.elastic.co/t/logstash-is-not-sending-the-data-to-elasticsearch/262365)
**Author:** [@sauravsuman689](https://discuss.elastic.co/u/sauravsuman689)\ **Replies:** 2\ **Last updated:** [February 2, 2021, 3:37pm UTC](https://discuss.elastic.co/t/logstash-is-not-sending-the-data-to-elasticsearch/262365 "2021-02-02T15:37:49Z")
Hi Team, My dataflow is as below : filebeat -\> kafka -\> logstash -\> es -\> kibana Messages are coming to Kafka topic but I don't see logstash is processing anything and sending to es cluster. Below is the warning mess… --- ## [Converting date to TIMESTAMP\_ISO8601 format](https://discuss.elastic.co/t/converting-date-to-timestamp-iso8601-format/262952)
**Author:** [@gloupe](https://discuss.elastic.co/u/gloupe)\ **Replies:** 1\ **Last updated:** [February 2, 2021, 3:24pm UTC](https://discuss.elastic.co/t/converting-date-to-timestamp-iso8601-format/262952 "2021-02-02T15:24:56Z")
Hi, I'm actually receiving logs with timestamp to this format : Mon Feb 1 13:29:48 2021 From Telegraf who's parsing the log with this Grok pattern : SOURCELOG %{DAY} %{MONTH} (?: %{POSINT:MONTHDAY}) %{TIME} %{YEAR} … --- ## [How to create logstash for different network device over same udp port](https://discuss.elastic.co/t/how-to-create-logstash-for-different-network-device-over-same-udp-port/262879)
**Author:** [@sajiby3k](https://discuss.elastic.co/u/sajiby3k)\ **Replies:** 3\ **Last updated:** [February 2, 2021, 8:29am UTC](https://discuss.elastic.co/t/how-to-create-logstash-for-different-network-device-over-same-udp-port/262879 "2021-02-02T08:29:54Z")
Hi, I am very new to elk stack. Trying to use logstash for Cisco routers and Fortigate firewalls. Both of these device types will use default udp port 514 to logstash to send log files. Most basic configuration looks… --- ## [Unable to install logstash-devutils locally for offline-packaging](https://discuss.elastic.co/t/unable-to-install-logstash-devutils-locally-for-offline-packaging/262936)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 0\ **Last updated:** [February 2, 2021, 7:36am UTC](https://discuss.elastic.co/t/unable-to-install-logstash-devutils-locally-for-offline-packaging/262936 "2021-02-02T07:36:45Z")
I am trying build a dev-env for my work-stations where i do not have access to the internet. I was hoping to install basic gem offline and transport it over to my work-station however while trying the below command: gem… --- ## [How to stop logstash schedule? and Keep the schedule function? (jdbc input)](https://discuss.elastic.co/t/how-to-stop-logstash-schedule-and-keep-the-schedule-function-jdbc-input/262919)
**Author:** [@plumy0902](https://discuss.elastic.co/u/plumy0902)\ **Replies:** 1\ **Last updated:** [February 2, 2021, 3:48am UTC](https://discuss.elastic.co/t/how-to-stop-logstash-schedule-and-keep-the-schedule-function-jdbc-input/262919 "2021-02-02T03:48:40Z")
I confirmed that the schedule is working normally. (I use the jdbc input) In the test environment, I stopped the job by using ctrl+c. How do I stop the schedule when I turn off the window that started logstash? After … --- ## [How to ship specific from date to end date from logstash](https://discuss.elastic.co/t/how-to-ship-specific-from-date-to-end-date-from-logstash/262898)
**Author:** [@Vamsi\_Krishna1](https://discuss.elastic.co/u/Vamsi_Krishna1)\ **Replies:** 1\ **Last updated:** [February 2, 2021, 2:58am UTC](https://discuss.elastic.co/t/how-to-ship-specific-from-date-to-end-date-from-logstash/262898 "2021-02-02T02:58:36Z")
Hi I need to ship data from 2020 dec to till date from one es index to another es index is there any way to do this, if any one know pls help me --- ## [Mysql query speed slow, cause offset](https://discuss.elastic.co/t/mysql-query-speed-slow-cause-offset/262868)
**Author:** [@star\_gold](https://discuss.elastic.co/u/star_gold)\ **Replies:** 0\ **Last updated:** [February 1, 2021, 4:58pm UTC](https://discuss.elastic.co/t/mysql-query-speed-slow-cause-offset/262868 "2021-02-01T16:58:58Z")
this is my config file: \` input { jdbc { jdbc\_driver\_library =\> "/tmp/logstash/logstash-7.9.3/mysql/mysql-connector-java-8.0.16.jar" jdbc\_driver\_class =\> "com.mysql.cj.jdbc.Driver" jdbc\_page\_size =\> 3 tracking\_col… --- ## [Logstash error](https://discuss.elastic.co/t/logstash-error/262558)
**Author:** [@suraj.ghorp](https://discuss.elastic.co/u/suraj.ghorp)\ **Replies:** 2\ **Last updated:** [February 1, 2021, 4:57pm UTC](https://discuss.elastic.co/t/logstash-error/262558 "2021-02-01T16:57:49Z")
Hi Masters, I am pulling logs from cloud based WAF through logstash. i am getting an below error. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.jruby.ext.openss… --- ## [Grok filters are not getting applied](https://discuss.elastic.co/t/grok-filters-are-not-getting-applied/262851)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 4\ **Last updated:** [February 1, 2021, 4:07pm UTC](https://discuss.elastic.co/t/grok-filters-are-not-getting-applied/262851 "2021-02-01T16:07:16Z")
grok patterns are not getting applied. Please apply. grok debugger shows no issue. please find the filebeat.yml filebeat.inputs: - type: log paths: - C:/Program Files (x86)/dox/onion.log fields: tags: appl… --- ## [How to use grok to match filenames as indexes in logstash](https://discuss.elastic.co/t/how-to-use-grok-to-match-filenames-as-indexes-in-logstash/262840)
**Author:** [@ThePreMan](https://discuss.elastic.co/u/ThePreMan)\ **Replies:** 0\ **Last updated:** [February 1, 2021, 1:14pm UTC](https://discuss.elastic.co/t/how-to-use-grok-to-match-filenames-as-indexes-in-logstash/262840 "2021-02-01T13:14:55Z")
We try to use developer names as indexes in logstash. Therefore our filenames are something like: developer1.log We have tried to use grok to match a certain part of the path as our filename. Nothing seems to be workin… --- ## [Logstash multilevel aggregation #logstash](https://discuss.elastic.co/t/logstash-multilevel-aggregation-logstash/262329)
**Author:** [@onkarborade](https://discuss.elastic.co/u/onkarborade)\ **Replies:** 1\ **Last updated:** [February 1, 2021, 1:13pm UTC](https://discuss.elastic.co/t/logstash-multilevel-aggregation-logstash/262329 "2021-02-01T13:13:59Z")
Hi, I am trying do a aggregation using plugins logstash-filter-aggregate (2.8.0). I extract data from relation database using plugin-jdbc. Where a teachers has and multiple contact\_details. I would like this result: … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=257) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=259)