# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=404 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 405 --- ## [Logstash server specification for heavy load](https://discuss.elastic.co/t/logstash-server-specification-for-heavy-load/219458)
**Author:** [@kulwantmor](https://discuss.elastic.co/u/kulwantmor)\ **Replies:** 3\ **Last updated:** [February 16, 2020, 1:45pm UTC](https://discuss.elastic.co/t/logstash-server-specification-for-heavy-load/219458 "2020-02-16T13:45:57Z")
Hi, I am having requirement where multiple hosts ( count would be approx 120) sending data via filebeat. Amount of data would be more during business hours and less during night. Total avg data ingested is 35 GB per day… --- ## [Is there any way to get application logged in user id in Logstash input statement configuration](https://discuss.elastic.co/t/is-there-any-way-to-get-application-logged-in-user-id-in-logstash-input-statement-configuration/219513)
**Author:** [@riya1995](https://discuss.elastic.co/u/riya1995)\ **Replies:** 1\ **Last updated:** [February 16, 2020, 1:42pm UTC](https://discuss.elastic.co/t/is-there-any-way-to-get-application-logged-in-user-id-in-logstash-input-statement-configuration/219513 "2020-02-16T13:42:48Z")
Hi All, My application works based on user roles.The elastic search should return data based on person role.This can be achieved by getting logged in person id in the logstash input statement.Can you guys help me to sor… --- ## [Grok DATA as MONTH-NUMBER](https://discuss.elastic.co/t/grok-data-as-month-number/219243)
**Author:** [@ldistria79](https://discuss.elastic.co/u/ldistria79)\ **Replies:** 1\ **Last updated:** [February 16, 2020, 10:45am UTC](https://discuss.elastic.co/t/grok-data-as-month-number/219243 "2020-02-16T10:45:35Z")
Hi All, i have two date types: "2020-02-12 09:37:19,334" and "2020Feb07-10:42:22", I'd like to import a unique format into kibana. Currently I worked on the second one, using grok I was able to read "2020Feb07-10:42:2… --- ## [Send many alert mail according motif error define in file error in my fied message](https://discuss.elastic.co/t/send-many-alert-mail-according-motif-error-define-in-file-error-in-my-fied-message/219491)
**Author:** [@reneroy](https://discuss.elastic.co/u/reneroy)\ **Replies:** 1\ **Last updated:** [February 15, 2020, 11:58pm UTC](https://discuss.elastic.co/t/send-many-alert-mail-according-motif-error-define-in-file-error-in-my-fied-message/219491 "2020-02-15T23:58:02Z")
Hello ; please ; i have a basic licence ; is not have option alerting (Watcher); i want to configure my logtash to send many mails alerts to users according the file error where i am registre my differents error ; i… --- ## [Filter Error logs only](https://discuss.elastic.co/t/filter-error-logs-only/218701)
**Author:** [@Adam\_Songambele](https://discuss.elastic.co/u/Adam_Songambele)\ **Replies:** 14\ **Last updated:** [February 15, 2020, 10:20pm UTC](https://discuss.elastic.co/t/filter-error-logs-only/218701 "2020-02-15T22:20:26Z")
Hi, i am still very new to the ELK stack and i am wondering how can i filter only error logs through logstash? i am currently feeding application, security and system logs into Logstash. below is my configuration for l… --- ## [Configuring Kafka Input with SASL PLAINTEXT auth](https://discuss.elastic.co/t/configuring-kafka-input-with-sasl-plaintext-auth/219485)
**Author:** [@chadskidmore](https://discuss.elastic.co/u/chadskidmore)\ **Replies:** 0\ **Last updated:** [February 15, 2020, 4:51pm UTC](https://discuss.elastic.co/t/configuring-kafka-input-with-sasl-plaintext-auth/219485 "2020-02-15T16:51:05Z")
Having some struggles getting kafka-input setup in logstash with SASL PLAINTEXT. Documentation seems to be largely non-existent. I did seem to get Beats sending data in to Kafka with this auth scheme. The input is prett… --- ## [Syntax in filter Error log](https://discuss.elastic.co/t/syntax-in-filter-error-log/219359)
**Author:** [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\ **Replies:** 5\ **Last updated:** [February 15, 2020, 3:23pm UTC](https://discuss.elastic.co/t/syntax-in-filter-error-log/219359 "2020-02-15T15:23:44Z")
Hi :slight\_smile: I am a bit struggling with the syntax in logstash. I would like to filter Log Level -Error. In my .log I have something like timestamp applicationtest.Error messsage. Is it possible that i my e… --- ## [Passive deployment for log ingestion behind a TAP/SPAN/diode](https://discuss.elastic.co/t/passive-deployment-for-log-ingestion-behind-a-tap-span-diode/219481)
**Author:** [@m4rkus](https://discuss.elastic.co/u/m4rkus)\ **Replies:** 0\ **Last updated:** [February 15, 2020, 2:42pm UTC](https://discuss.elastic.co/t/passive-deployment-for-log-ingestion-behind-a-tap-span-diode/219481 "2020-02-15T14:42:50Z")
Hi, I would like to setup an ELK stack for passive log ingestion behind a TAP/SPAN/diode, from an existing log stream. Unfortunately I haven't been able to find any information about passive deployment. Can Logstash be… --- ## [Most Efficient Way to Add filebeats to Logstash](https://discuss.elastic.co/t/most-efficient-way-to-add-filebeats-to-logstash/219410)
**Author:** [@Archie\_Crawford](https://discuss.elastic.co/u/Archie_Crawford)\ **Replies:** 5\ **Last updated:** [February 14, 2020, 9:00pm UTC](https://discuss.elastic.co/t/most-efficient-way-to-add-filebeats-to-logstash/219410 "2020-02-14T21:00:00Z")
Currently I have added beats to my logstash with there own port numbers and created new configs in logstash for each one of them. Is this the most efficient way to add filebeats to logstash when creating my stack. Ex: F… --- ## [Logstash directory folder deletion](https://discuss.elastic.co/t/logstash-directory-folder-deletion/219435)
**Author:** [@cowensel](https://discuss.elastic.co/u/cowensel)\ **Replies:** 0\ **Last updated:** [February 14, 2020, 6:51pm UTC](https://discuss.elastic.co/t/logstash-directory-folder-deletion/219435 "2020-02-14T18:51:10Z")
Has anyone come across a instance where logstash has lost the config folder in the logstash directory if there is an issue with disk space. We have had a issue where the config folder has been removed --- ## [Logstash Error when starting up - EACCES - Permission denied - NUL](https://discuss.elastic.co/t/logstash-error-when-starting-up-eacces-permission-denied-nul/219424)
**Author:** [@peter.tu](https://discuss.elastic.co/u/peter.tu)\ **Replies:** 0\ **Last updated:** [February 14, 2020, 5:22pm UTC](https://discuss.elastic.co/t/logstash-error-when-starting-up-eacces-permission-denied-nul/219424 "2020-02-14T17:22:31Z")
When I start logstash, I get the following error: Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (EACCES) Permission denied - NUL It was working fine a couple of days ago,… --- ## [Date format, how to change it?](https://discuss.elastic.co/t/date-format-how-to-change-it/219259)
**Author:** [@Mbt\_Mbt](https://discuss.elastic.co/u/Mbt_Mbt)\ **Replies:** 3\ **Last updated:** [February 14, 2020, 4:45pm UTC](https://discuss.elastic.co/t/date-format-how-to-change-it/219259 "2020-02-14T16:45:38Z")
Hello good afternoon, I am trying to change the way of displaying date type data but I am not able. Can someone help me on how to visualize the dates with the format: 2020-13-02T10:01:01.000Z Currently the field has t… --- ## [Optimizing process time](https://discuss.elastic.co/t/optimizing-process-time/218568)
**Author:** [@not\_found](https://discuss.elastic.co/u/not_found)\ **Replies:** 1\ **Last updated:** [February 14, 2020, 4:36pm UTC](https://discuss.elastic.co/t/optimizing-process-time/218568 "2020-02-14T16:36:34Z")
Hello everyone, I'm pretty new in the elastic ecosystem. For the work, I have to analyze logstash process, understand it and improve it if it's possible. So I'm wondering a lot of questions. Firstly, a little bit con… --- ## [How to remove the last byte from an IP address?](https://discuss.elastic.co/t/how-to-remove-the-last-byte-from-an-ip-address/219241)
**Author:** [@Aleksandra\_Minko](https://discuss.elastic.co/u/Aleksandra_Minko)\ **Replies:** 1\ **Last updated:** [February 14, 2020, 2:21pm UTC](https://discuss.elastic.co/t/how-to-remove-the-last-byte-from-an-ip-address/219241 "2020-02-14T14:21:31Z")
Hello, I have a NetFlow field named \[flow\]\[addr\] which represents an IP address. For example \[flow\]\[addr\] looks like: 10.96.31.41 . I would like to remove the last Byte of my field and keep only 10.96.31. I also would… --- ## [Logstash not sending ACK to Filebeat thereby causing duplicate events](https://discuss.elastic.co/t/logstash-not-sending-ack-to-filebeat-thereby-causing-duplicate-events/218686)
**Author:** [@stratmike](https://discuss.elastic.co/u/stratmike)\ **Replies:** 2\ **Last updated:** [February 14, 2020, 2:08pm UTC](https://discuss.elastic.co/t/logstash-not-sending-ack-to-filebeat-thereby-causing-duplicate-events/218686 "2020-02-14T14:08:17Z")
I have configured the following on a Windows 10 machine running on VMWare: Filebeat-\>Logstash-\>Elasticsearch All of these are running on the same machine. The problem is that Logstash is not sending an ACK to Filebeat… --- ## [Capture a substring from a windows path](https://discuss.elastic.co/t/capture-a-substring-from-a-windows-path/218533)
**Author:** [@Radha\_Krishna](https://discuss.elastic.co/u/Radha_Krishna)\ **Replies:** 7\ **Last updated:** [February 14, 2020, 8:36am UTC](https://discuss.elastic.co/t/capture-a-substring-from-a-windows-path/218533 "2020-02-14T08:36:01Z")
I need to capture services.exe from the path C:\\Windows\\System32\\services.exe, but for every event the path changes dynamically. What would be a generic way to capture whatever is there after the last \\. Thanks in advan… --- ## [Logstash always take file opened to read](https://discuss.elastic.co/t/logstash-always-take-file-opened-to-read/219333)
**Author:** [@alexnar](https://discuss.elastic.co/u/alexnar)\ **Replies:** 0\ **Last updated:** [February 14, 2020, 8:59am UTC](https://discuss.elastic.co/t/logstash-always-take-file-opened-to-read/219333 "2020-02-14T08:59:55Z")
Problem description We have the following problem with file input: Logstash read customer system logs Customer system rotate file every 3 hours. it is working the following way: -- At first File is renamed -- At sec… --- ## [Parsing message using kv filter](https://discuss.elastic.co/t/parsing-message-using-kv-filter/219267)
**Author:** [@toms130](https://discuss.elastic.co/u/toms130)\ **Replies:** 2\ **Last updated:** [February 14, 2020, 8:04am UTC](https://discuss.elastic.co/t/parsing-message-using-kv-filter/219267 "2020-02-14T08:04:12Z")
Hi all, I'd like to parse some part of a message with VKv filter, and keep end of the message into another field. Message is : \[MarketplaceGroup=data\]\[MarketplaceId=10\]\[AccountId=1234\] updating 926 items My filter is… --- ## [How to Mongodb logs to elasticsearch through logstash](https://discuss.elastic.co/t/how-to-mongodb-logs-to-elasticsearch-through-logstash/217181)
**Author:** [@jskishor](https://discuss.elastic.co/u/jskishor)\ **Replies:** 4\ **Last updated:** [February 14, 2020, 7:38am UTC](https://discuss.elastic.co/t/how-to-mongodb-logs-to-elasticsearch-through-logstash/217181 "2020-02-14T07:38:26Z")
Hi There, How do I send mongodb logs into elasticsearch. While MongoDB is on a VM and logstash on other different server, what is the best way to ingest logs into Elasticsearch Thanks Kishore --- ## [System setting for logstash](https://discuss.elastic.co/t/system-setting-for-logstash/219309)
**Author:** [@SNJY](https://discuss.elastic.co/u/SNJY)\ **Replies:** 0\ **Last updated:** [February 14, 2020, 5:05am UTC](https://discuss.elastic.co/t/system-setting-for-logstash/219309 "2020-02-14T05:05:45Z")
Below document suggests system settings for elasticsearch -- https://www.elastic.co/guide/en/elasticsearch/reference/6.2/system-config.html Are above recommendations valid for logstash as well ? Is there any such docu… --- ## [Filter a field that has different formats](https://discuss.elastic.co/t/filter-a-field-that-has-different-formats/219232)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 1\ **Last updated:** [February 13, 2020, 7:32pm UTC](https://discuss.elastic.co/t/filter-a-field-that-has-different-formats/219232 "2020-02-13T19:32:45Z")
Hi, I am currently filtering logs that normaly return this kind of field (among others): vfs.fs.size\[D:,pused\] and my grok for this field is the following: vfs.fs.size\\\[%{DATA:disco},pused\\\] but for certain machines … --- ## [Filter syslog messages via priority](https://discuss.elastic.co/t/filter-syslog-messages-via-priority/219190)
**Author:** [@ptselios](https://discuss.elastic.co/u/ptselios)\ **Replies:** 3\ **Last updated:** [February 13, 2020, 3:59pm UTC](https://discuss.elastic.co/t/filter-syslog-messages-via-priority/219190 "2020-02-13T15:59:13Z")
Hello, Currently I have logstash as a centralized syslog server. I want to send to Elasticsearch only syslog messages with priority warn and above. So, I create the following configuration file: input { tcp { p… --- ## [ISO8601 to epoch seconds](https://discuss.elastic.co/t/iso8601-to-epoch-seconds/219033)
**Author:** [@fim](https://discuss.elastic.co/u/fim)\ **Replies:** 10\ **Last updated:** [February 13, 2020, 2:47pm UTC](https://discuss.elastic.co/t/iso8601-to-epoch-seconds/219033 "2020-02-13T14:47:23Z")
Hi I ingest data from a mysql database and in a specific column there are date/time in ISO8601 format stored, which I would like to convert as @timestamp in elasticsearch. I followed the question here: https://discuss.… --- ## [Logstash prune nested fields](https://discuss.elastic.co/t/logstash-prune-nested-fields/218984)
**Author:** [@gangireddy\_l](https://discuss.elastic.co/u/gangireddy_l)\ **Replies:** 3\ **Last updated:** [February 13, 2020, 2:22pm UTC](https://discuss.elastic.co/t/logstash-prune-nested-fields/218984 "2020-02-13T14:22:32Z")
HI TEAM, I am using ELK version 7.0. Facing an issue with prune filter. I need to blacklist exact nested fields coming from filebeat. When i tried with below format, prune is unable to blacklist host.architecture field… --- ## [Split Filter Parse Failure - Only String and Array types are splittable](https://discuss.elastic.co/t/split-filter-parse-failure-only-string-and-array-types-are-splittable/219204)
**Author:** [@mohuss](https://discuss.elastic.co/u/mohuss)\ **Replies:** 4\ **Last updated:** [February 13, 2020, 1:54pm UTC](https://discuss.elastic.co/t/split-filter-parse-failure-only-string-and-array-types-are-splittable/219204 "2020-02-13T13:54:41Z")
I have this error with split filter in logstash . "Only String and Array types are splittable. field:\[log\] is of type = NilClass" I am trying to create events for each object in an array of objects but cannot figure ou… --- ## [How to connect SSL enable MYSQL from LOGSTASH](https://discuss.elastic.co/t/how-to-connect-ssl-enable-mysql-from-logstash/219212)
**Author:** [@santojitg](https://discuss.elastic.co/u/santojitg)\ **Replies:** 0\ **Last updated:** [February 13, 2020, 1:39pm UTC](https://discuss.elastic.co/t/how-to-connect-ssl-enable-mysql-from-logstash/219212 "2020-02-13T13:39:50Z")
Hello I would like to sync my data from MYSQL to ElasticSearch using Logstash. I can do that easily from a standard MYSQL database. But facing problem with a SSL enable MYSQL database, It's unable to connect to my jks … --- ## [Logstash JSON Parser is just parsing fields if no target is defined](https://discuss.elastic.co/t/logstash-json-parser-is-just-parsing-fields-if-no-target-is-defined/219205)
**Author:** [@Simon\_Becker](https://discuss.elastic.co/u/Simon_Becker)\ **Replies:** 0\ **Last updated:** [February 13, 2020, 1:08pm UTC](https://discuss.elastic.co/t/logstash-json-parser-is-just-parsing-fields-if-no-target-is-defined/219205 "2020-02-13T13:08:17Z")
Hello, I am currently trying to parse a JSON message through the JSON filter. Without defining any target it works perfectly. When adding a target field it is not parsing any more throwing this error: response=\>{"index… --- ## [EPS count on ELK](https://discuss.elastic.co/t/eps-count-on-elk/219200)
**Author:** [@DavidLevi](https://discuss.elastic.co/u/DavidLevi)\ **Replies:** 0\ **Last updated:** [February 13, 2020, 12:42pm UTC](https://discuss.elastic.co/t/eps-count-on-elk/219200 "2020-02-13T12:42:15Z")
Hello, How can I see the EPS count on my ELK system? ELK version is 6.4. --- ## [Unable to update the ES using http-output-plugin](https://discuss.elastic.co/t/unable-to-update-the-es-using-http-output-plugin/219179)
**Author:** [@psiva017](https://discuss.elastic.co/u/psiva017)\ **Replies:** 0\ **Last updated:** [February 13, 2020, 9:54am UTC](https://discuss.elastic.co/t/unable-to-update-the-es-using-http-output-plugin/219179 "2020-02-13T09:54:12Z")
http { url =\> "http://localhost:9200/test\_index/\_update\_by\_query" format =\> "message" content\_type =\> "application/json" message =\> '{"query": {"match":{ "userId": %{userId}}},"script": "ctx.\_source.recentStatus = %{… --- ## [Migration data from MariaDB to Elasticsearch via Logstash](https://discuss.elastic.co/t/migration-data-from-mariadb-to-elasticsearch-via-logstash/218225)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 1\ **Last updated:** [February 13, 2020, 9:42am UTC](https://discuss.elastic.co/t/migration-data-from-mariadb-to-elasticsearch-via-logstash/218225 "2020-02-13T09:42:36Z")
I'm trying to migrate data from MariaDB to Elasticsearch by using Logstash. I never worked with jdbc before so I installed jdbc with sudo apt-get install libmysql-java command and created the following mysql.conf file… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=403) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=405)