# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=586 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 587 --- ## [Possbile to store date timezone into elasticsearch via logstash output plugin..?](https://discuss.elastic.co/t/possbile-to-store-date-timezone-into-elasticsearch-via-logstash-output-plugin/171799)
**Author:** [@raulk89](https://discuss.elastic.co/u/raulk89)\ **Replies:** 1\ **Last updated:** [March 11, 2019, 4:12pm UTC](https://discuss.elastic.co/t/possbile-to-store-date-timezone-into-elasticsearch-via-logstash-output-plugin/171799 "2019-03-11T16:12:08Z")
Hi Elastic, kibana, logtash - 6.6.1 Centos 7.6 Have I understood correctly, that all the date column data that are stored to elasticsearch, these are stored as UTC time..? (Meaning, when elasticsearch has stored a va… --- ## [Logstash conf file if else condition is not working](https://discuss.elastic.co/t/logstash-conf-file-if-else-condition-is-not-working/171111)
**Author:** [@Suresh\_Pal](https://discuss.elastic.co/u/Suresh_Pal)\ **Replies:** 2\ **Last updated:** [March 11, 2019, 3:17pm UTC](https://discuss.elastic.co/t/logstash-conf-file-if-else-condition-is-not-working/171111 "2019-03-11T15:17:27Z")
Team, Vesrion 6.2 In my config file i'm trying to use different grok pattern for different prospector. filter { if \[type\] == "db\_log" { grok { match =\> \[ "message", "%{TIMESTAMP\_ISO8601:timetamp}%{SPACE}%{NOTSPACE}… --- ## [Logstash non-heap memory](https://discuss.elastic.co/t/logstash-non-heap-memory/171787)
**Author:** [@cciaccio](https://discuss.elastic.co/u/cciaccio)\ **Replies:** 2\ **Last updated:** [March 11, 2019, 3:12pm UTC](https://discuss.elastic.co/t/logstash-non-heap-memory/171787 "2019-03-11T15:12:14Z")
I run Logstash in a docker container (using the official image). I assigned 10GB of memory to the container. How much of these 10GB should be heap and how much non-heap? 50%-50% like Elasticsearch? 90%-10%? --- ## [How to let Logstash read all my Input texts Files automatically](https://discuss.elastic.co/t/how-to-let-logstash-read-all-my-input-texts-files-automatically/171724)
**Author:** [@samyo](https://discuss.elastic.co/u/samyo)\ **Replies:** 10\ **Last updated:** [March 11, 2019, 2:54pm UTC](https://discuss.elastic.co/t/how-to-let-logstash-read-all-my-input-texts-files-automatically/171724 "2019-03-11T14:54:34Z")
Hello Folks, i need some help to know if its Possible, to let Logstash read my 1000 Textfiles separately and automatically from the Input. I have about 1000 textfiles , each textfile has his name and different text ins… --- ## [Multiple aggregate filter with same task\_id pattern doesn't share map](https://discuss.elastic.co/t/multiple-aggregate-filter-with-same-task-id-pattern-doesnt-share-map/171713)
**Author:** [@Andrey\_Tomilin](https://discuss.elastic.co/u/Andrey_Tomilin)\ **Replies:** 4\ **Last updated:** [March 11, 2019, 2:02pm UTC](https://discuss.elastic.co/t/multiple-aggregate-filter-with-same-task-id-pattern-doesnt-share-map/171713 "2019-03-11T14:02:53Z")
Here are events: Id: 0, Source: a, Status: 0 Id: 1, Source: a, Status: 1 Id: 2, Source: a, Status: 0 Id: 3, Source: a, Status: 1 Id: 4, Source: a, Status: 1 Id: 5, Source: a, Status: 0 Id: 6, Source: a, Status: 1 … --- ## [Logstash Error while running in docker](https://discuss.elastic.co/t/logstash-error-while-running-in-docker/171759)
**Author:** [@Vignesh\_Kumar](https://discuss.elastic.co/u/Vignesh_Kumar)\ **Replies:** 1\ **Last updated:** [March 11, 2019, 2:01pm UTC](https://discuss.elastic.co/t/logstash-error-while-running-in-docker/171759 "2019-03-11T14:01:58Z")
Have the below error please help to solve it. \[2019-03-11T12:15:39,040\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>#\, :backtrace=\>\["/usr… --- ## [Couldn't find any Elasticsearch data on kibana?](https://discuss.elastic.co/t/couldnt-find-any-elasticsearch-data-on-kibana/171671)
**Author:** [@pramodchoudhari](https://discuss.elastic.co/u/pramodchoudhari)\ **Replies:** 3\ **Last updated:** [March 11, 2019, 1:56pm UTC](https://discuss.elastic.co/t/couldnt-find-any-elasticsearch-data-on-kibana/171671 "2019-03-11T13:56:30Z")
this is my logstash.conf input { beats { port =\> 5044 } } filter { if \[fields\]\[log\_type\] == "syslog" { grok { match =\> { "message" =\> "{WORD:name},%{GREEDYDATA:date},%{NUMBER:CLNo},%{WORD:status}" }… --- ## [Problems to run Logstash](https://discuss.elastic.co/t/problems-to-run-logstash/171779)
**Author:** [@Verdugo\_Gonzalo](https://discuss.elastic.co/u/Verdugo_Gonzalo)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 1:49pm UTC](https://discuss.elastic.co/t/problems-to-run-logstash/171779 "2019-03-11T13:49:53Z")
Hello, I am trying to send SNMP data to my ELK Cloud. Follow the instructions on this page SNMP Plugins and this Guide Cloud I have created a new file called logstash-snmp.conf and the content is as follows: input { … --- ## [Can't use filebeat \[beat\]\[timezone\] with logstash date filter plugin?](https://discuss.elastic.co/t/cant-use-filebeat-beat-timezone-with-logstash-date-filter-plugin/171661)
**Author:** [@PSM](https://discuss.elastic.co/u/PSM)\ **Replies:** 3\ **Last updated:** [March 11, 2019, 1:47pm UTC](https://discuss.elastic.co/t/cant-use-filebeat-beat-timezone-with-logstash-date-filter-plugin/171661 "2019-03-11T13:47:11Z")
I'm sending a log file to logstash using filebeat that does not encode timezone in the timestamp: 2019-03-08 17:48:49,766 \[3\] INFO Corp.FO.DataLoaders.BaseDataFile \[DOMAIN\\username\] - log message detail Until today, wh… --- ## [Logstash update by query not working](https://discuss.elastic.co/t/logstash-update-by-query-not-working/171476)
**Author:** [@Chelambarasan\_CR](https://discuss.elastic.co/u/Chelambarasan_CR)\ **Replies:** 0\ **Last updated:** [March 8, 2019, 11:08am UTC](https://discuss.elastic.co/t/logstash-update-by-query-not-working/171476 "2019-03-08T11:08:37Z")
output { stdout { codec => rubydebug } http { \`\`\` url =\> "http://localhost:9200/test1/\_update\_by\_query?q=id:1" http\_method =\> "post" format =\> "json" \`\`\` } } Tried this for update by query in logstash output. Go… --- ## [Failed to format Suricata json file](https://discuss.elastic.co/t/failed-to-format-suricata-json-file/171752)
**Author:** [@thoarau](https://discuss.elastic.co/u/thoarau)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 12:08pm UTC](https://discuss.elastic.co/t/failed-to-format-suricata-json-file/171752 "2019-03-11T12:08:18Z")
I'm trying to send Suricata event to the ELK stack . I have the ELK stack on a server and on an other server I have Suricata and Filebeat. I activated the filebeat suricata module and Filebeat send event from eve.json fi… --- ## [Jdbc plugin and multiple logstash issue](https://discuss.elastic.co/t/jdbc-plugin-and-multiple-logstash-issue/171734)
**Author:** [@annk](https://discuss.elastic.co/u/annk)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 11:12am UTC](https://discuss.elastic.co/t/jdbc-plugin-and-multiple-logstash-issue/171734 "2019-03-11T11:12:35Z")
Hi , We have 3 logstash in our cluster and several pipelines managed using Centralised pipeline management on kibana. I am getting duplicate records (3 records for each line read from db) when using jdbc plugin , I cant… --- ## [Migrating from 5.6 to 6.6 and issue with persistent queue](https://discuss.elastic.co/t/migrating-from-5-6-to-6-6-and-issue-with-persistent-queue/171507)
**Author:** [@mw\_jko](https://discuss.elastic.co/u/mw_jko)\ **Replies:** 1\ **Last updated:** [March 11, 2019, 10:31am UTC](https://discuss.elastic.co/t/migrating-from-5-6-to-6-6-and-issue-with-persistent-queue/171507 "2019-03-11T10:31:53Z")
Hi there, we are using a very simple one node ELK docker-compose based stack running on one server. Elasticsearch, Kibana, Logstash are all running version 5.6.3. We use the persistent queue feature. I'm aware of the l… --- ## [Input HTTP\_Poller Body with field issue](https://discuss.elastic.co/t/input-http-poller-body-with-field-issue/170002)
**Author:** [@romariol](https://discuss.elastic.co/u/romariol)\ **Replies:** 5\ **Last updated:** [March 11, 2019, 10:02am UTC](https://discuss.elastic.co/t/input-http-poller-body-with-field-issue/170002 "2019-03-11T10:02:52Z")
Hi, I'm using the http\_poller plugin as input where I perform a SOAP Call to an external system. It works perfectly fine while using a fixed body. However, the body contains date fields which must be the date of today. … --- ## [Reading CSV and applying indexing from logstash taking too much time](https://discuss.elastic.co/t/reading-csv-and-applying-indexing-from-logstash-taking-too-much-time/171684)
**Author:** [@pk96375](https://discuss.elastic.co/u/pk96375)\ **Replies:** 3\ **Last updated:** [March 11, 2019, 9:55am UTC](https://discuss.elastic.co/t/reading-csv-and-applying-indexing-from-logstash-taking-too-much-time/171684 "2019-03-11T09:55:44Z")
I am using below configuration for indexing using logstash File -\> Logstash -\> Elasticsearch pipeline. input { file { path =\> "C:/Users/temp/100\_CC\_Records.csv" start\_position=\>"beginning" } } filter { csv{ sepa… --- ## [Logstash 6.5.1 stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/logstash-6-5-1-stopped-processing-because-of-an-error-systemexit-exit/171712)
**Author:** [@vkeswarlal](https://discuss.elastic.co/u/vkeswarlal)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 9:27am UTC](https://discuss.elastic.co/t/logstash-6-5-1-stopped-processing-because-of-an-error-systemexit-exit/171712 "2019-03-11T09:27:40Z")
Logstash 6.5.1 stops processing immediately after throwing the following error and becomes unresponsive till it is restarted, \[2019-02-27T19:32:19,054\]\[WARN \]\[logstash.filters.dns \] DNS: timeout on resolving address… --- ## [Logstash mysql/mariadb jdbc connection error](https://discuss.elastic.co/t/logstash-mysql-mariadb-jdbc-connection-error/171709)
**Author:** [@Marcin\_Bartoszek](https://discuss.elastic.co/u/Marcin_Bartoszek)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 9:24am UTC](https://discuss.elastic.co/t/logstash-mysql-mariadb-jdbc-connection-error/171709 "2019-03-11T09:24:29Z")
Hi, I'm struggling with connection to mariadb DB from logstash. Database is hosting on AWS. When connecting manualy there is no problem, but when using logstash I'm getting : "\[2019-03-11T09:17:00,029\]\[ERROR\]\[logstas… --- ## [Logstash does not read new pipeline after incorrect config has been deployed once](https://discuss.elastic.co/t/logstash-does-not-read-new-pipeline-after-incorrect-config-has-been-deployed-once/171686)
**Author:** [@JohanRask](https://discuss.elastic.co/u/JohanRask)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 7:38am UTC](https://discuss.elastic.co/t/logstash-does-not-read-new-pipeline-after-incorrect-config-has-been-deployed-once/171686 "2019-03-11T07:38:12Z")
We use central pipeline management for logstash. After pushing incorrect config (should be string, but was long), Logstash enters a loop where it tries to restart the plugin Logstash does not download the new config a… --- ## [Need to define environment variable to calculate the previous date](https://discuss.elastic.co/t/need-to-define-environment-variable-to-calculate-the-previous-date/171233)
**Author:** [@vsingh](https://discuss.elastic.co/u/vsingh)\ **Replies:** 4\ **Last updated:** [March 11, 2019, 5:41am UTC](https://discuss.elastic.co/t/need-to-define-environment-variable-to-calculate-the-previous-date/171233 "2019-03-11T05:41:00Z")
Hello there, I need to import the prior date data to create the index in elastic from mysql. this is script: input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/testdb?zeroDateTimeBehavior=convertT… --- ## [DLQ with mapping error](https://discuss.elastic.co/t/dlq-with-mapping-error/171647)
**Author:** [@jasoneng96](https://discuss.elastic.co/u/jasoneng96)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 12:26am UTC](https://discuss.elastic.co/t/dlq-with-mapping-error/171647 "2019-03-11T00:26:15Z")
Hi guys, I have enabled the DLQ setting to true in logstash.yml. I can see there are logs in the DLQ data path. But, I can tell that LogStash still stop collecting logs whenever it encounters mapping error. Any insights… --- ## [Generator input is unordered](https://discuss.elastic.co/t/generator-input-is-unordered/171628)
**Author:** [@Andrey\_Tomilin](https://discuss.elastic.co/u/Andrey_Tomilin)\ **Replies:** 2\ **Last updated:** [March 10, 2019, 12:55pm UTC](https://discuss.elastic.co/t/generator-input-is-unordered/171628 "2019-03-10T12:55:50Z")
According to this documentation and logic in general I assume this input should process specified lines one by one in the specified order, not random or anything else. But it it's output is in random order. Also the te… --- ## [Logstash.yml configuration](https://discuss.elastic.co/t/logstash-yml-configuration/171612)
**Author:** [@Mysterious\_Ze](https://discuss.elastic.co/u/Mysterious_Ze)\ **Replies:** 1\ **Last updated:** [March 10, 2019, 10:40am UTC](https://discuss.elastic.co/t/logstash-yml-configuration/171612 "2019-03-10T10:40:14Z")
Hello I am having 2 issues with logstash: i have installed winlogbeat on my windows servers. I see that redis is getting all the logs and while looking into logstash-plain.log, i saw errors of type invalid index name.… --- ## [Can't see Suricata events in Kibana](https://discuss.elastic.co/t/cant-see-suricata-events-in-kibana/171516)
**Author:** [@thoarau](https://discuss.elastic.co/u/thoarau)\ **Replies:** 1\ **Last updated:** [March 10, 2019, 2:54am UTC](https://discuss.elastic.co/t/cant-see-suricata-events-in-kibana/171516 "2019-03-10T02:54:57Z")
I'm trying to send Suricata event to the ELK stack . I have the ELK stack on a server and on an other server I have Suricata and Filebeat. I activated the filebeat suricata module and Filebeat send event from eve.json fi… --- ## [Sending azure\_event\_hub messages to Elasticsearch, Logstash failing](https://discuss.elastic.co/t/sending-azure-event-hub-messages-to-elasticsearch-logstash-failing/171448)
**Author:** [@Abereham](https://discuss.elastic.co/u/Abereham)\ **Replies:** 0\ **Last updated:** [March 8, 2019, 8:00am UTC](https://discuss.elastic.co/t/sending-azure-event-hub-messages-to-elasticsearch-logstash-failing/171448 "2019-03-08T08:00:31Z")
I am new to Azure and Elasticsearch. While I am trying to send event hub messages to elastic search, nothing is showing up in kibana and my console shows me exception which I do no how to solve it. I followed this doc \[e… --- ## [Issue with extracting timestamp value from log file](https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580)
**Author:** [@raghunayak](https://discuss.elastic.co/u/raghunayak)\ **Replies:** 3\ **Last updated:** [March 9, 2019, 5:10pm UTC](https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580 "2019-03-09T17:10:49Z")
Log file 1 format 2019-02-26 03:00:50.000 -05:00 172.11.0.1 LOCAL:DEBUG some message Log file 2 format 2019-01-12T05:03:04.956-05:00 172.11.0.1 LOCAL:DEBUG some message I have created a common GROK format for the abo… --- ## [Logstash not seeing updates in logs](https://discuss.elastic.co/t/logstash-not-seeing-updates-in-logs/171054)
**Author:** [@Jozziej](https://discuss.elastic.co/u/Jozziej)\ **Replies:** 3\ **Last updated:** [March 9, 2019, 10:20am UTC](https://discuss.elastic.co/t/logstash-not-seeing-updates-in-logs/171054 "2019-03-09T10:20:46Z")
When I use Logstash with the file plugin it reads the log till the point when Logstash discovered the file. Any updates in the Logs are not tailed and showed A little explanation from my side. I use rsync to synchroni… --- ## [Using existing INDEX](https://discuss.elastic.co/t/using-existing-index/171586)
**Author:** [@behnam](https://discuss.elastic.co/u/behnam)\ **Replies:** 0\ **Last updated:** [March 9, 2019, 8:33am UTC](https://discuss.elastic.co/t/using-existing-index/171586 "2019-03-09T08:33:47Z")
HI THERE ! I'M new in elastic and dependent modules my question is that how can i import data into an existing INDEX while it corrupted by running again logstash.bat to import in existing logstash ! when i running it … --- ## [Beats and logstash](https://discuss.elastic.co/t/beats-and-logstash/171402)
**Author:** [@Farhan\_Umer](https://discuss.elastic.co/u/Farhan_Umer)\ **Replies:** 7\ **Last updated:** [March 9, 2019, 2:16am UTC](https://discuss.elastic.co/t/beats-and-logstash/171402 "2019-03-09T02:16:19Z")
Hi, I have 2 logstash files i want to create indexes with specific names such as IIS-Log-\ and windows-eventlogs-\. I am using logstash with filebeat and winlogbeat. Right now everything is goi… --- ## [Multiple beats 1 input port with multiple index output](https://discuss.elastic.co/t/multiple-beats-1-input-port-with-multiple-index-output/171504)
**Author:** [@Pedro\_Cabral](https://discuss.elastic.co/u/Pedro_Cabral)\ **Replies:** 7\ **Last updated:** [March 8, 2019, 11:20pm UTC](https://discuss.elastic.co/t/multiple-beats-1-input-port-with-multiple-index-output/171504 "2019-03-08T23:20:04Z")
Hello does anyone knows if is it possible to have multiple beats pointing to the same input port and then having multiple index output. Basically I need to identify which beat arrives and then add a tag to then have a pr… --- ## [Using "merge" to append a string literal to an array field?](https://discuss.elastic.co/t/using-merge-to-append-a-string-literal-to-an-array-field/171353)
**Author:** [@tlacuache](https://discuss.elastic.co/u/tlacuache)\ **Replies:** 4\ **Last updated:** [March 8, 2019, 7:39pm UTC](https://discuss.elastic.co/t/using-merge-to-append-a-string-literal-to-an-array-field/171353 "2019-03-08T19:39:46Z")
this is probably a stupid question, but how can I append a literal string to an array in logstash? i tried merge: if \[my\_ip\] and (\[my\_ip\] == "127.0.0.1") { mutate { merge =\> { "\[my\_hostname\]" =\> "loopback" } } … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=585) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=587)