# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=375 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 376 --- ## [Multiple Output by @timestamp filter in Logstash](https://discuss.elastic.co/t/multiple-output-by-timestamp-filter-in-logstash/227842)
**Author:** [@newbie001](https://discuss.elastic.co/u/newbie001)\ **Replies:** 0\ **Last updated:** [April 14, 2020, 3:00am UTC](https://discuss.elastic.co/t/multiple-output-by-timestamp-filter-in-logstash/227842 "2020-04-14T03:00:09Z")
Hello - Is there a way I can output the message in different files by @timestamp in logstash? These are all outputted in JSON Files. I am very new to logstash and I am not quite sure on how to output it in multiple file… --- ## [Load CSV data into ElasticSearch using Logstash](https://discuss.elastic.co/t/load-csv-data-into-elasticsearch-using-logstash/227820)
**Author:** [@inchirah](https://discuss.elastic.co/u/inchirah)\ **Replies:** 4\ **Last updated:** [April 13, 2020, 11:04pm UTC](https://discuss.elastic.co/t/load-csv-data-into-elasticsearch-using-logstash/227820 "2020-04-13T23:04:09Z")
Hello, I'm trying to import a csv file into ES, when I run the config file I get this : Sending Logstash logs to C:/elastic\_stack/logstash-7.6.2/logs which is now configured via log4j2.properties \[2020-04-13T20:17:24,4… --- ## [Reuse db connection,driverclass, for jdbc inputs in logstash](https://discuss.elastic.co/t/reuse-db-connection-driverclass-for-jdbc-inputs-in-logstash/227808)
**Author:** [@David\_Samm](https://discuss.elastic.co/u/David_Samm)\ **Replies:** 3\ **Last updated:** [April 13, 2020, 10:41pm UTC](https://discuss.elastic.co/t/reuse-db-connection-driverclass-for-jdbc-inputs-in-logstash/227808 "2020-04-13T22:41:28Z")
input { jdbc { jdbc\_connection\_string =\> "jdbc:oracle:thin:@\*\*\*:1521/\*\*\*" jdbc\_user =\> "\*\*\*" jdbc\_password =\> "\*\*\*" jdbc\_validate\_connection =\> true jdbc\_driver\_library =\> "/opt/logstash-5.5.0/ojdbc… --- ## [How send the log to different index in Kibana](https://discuss.elastic.co/t/how-send-the-log-to-different-index-in-kibana/227817)
**Author:** [@Prabhu\_Chinnasamy](https://discuss.elastic.co/u/Prabhu_Chinnasamy)\ **Replies:** 5\ **Last updated:** [April 13, 2020, 9:20pm UTC](https://discuss.elastic.co/t/how-send-the-log-to-different-index-in-kibana/227817 "2020-04-13T21:20:05Z")
I am new to Elastic stack and trying to see the different logs under different Kiban indexes. for an example, I am trying to place the logs from my dhcp server under a index called "DHCP". My logstash config is as below,… --- ## [Unable to send json to elastic](https://discuss.elastic.co/t/unable-to-send-json-to-elastic/227639)
**Author:** [@Magnuss](https://discuss.elastic.co/u/Magnuss)\ **Replies:** 15\ **Last updated:** [April 13, 2020, 7:25pm UTC](https://discuss.elastic.co/t/unable-to-send-json-to-elastic/227639 "2020-04-13T19:25:45Z")
Hi, I have some issues sending a json file to elastic. Logstash can connect to elastic and also in the logging are no errors. I may think it has to do with the json file or maybe the format.(or do i need to use the filt… --- ## [Issue pulling data from elasticsearch with date parameter](https://discuss.elastic.co/t/issue-pulling-data-from-elasticsearch-with-date-parameter/227793)
**Author:** [@bheierle](https://discuss.elastic.co/u/bheierle)\ **Replies:** 0\ **Last updated:** [April 13, 2020, 3:40pm UTC](https://discuss.elastic.co/t/issue-pulling-data-from-elasticsearch-with-date-parameter/227793 "2020-04-13T15:40:36Z")
Yes I am pulling data from elastic search. The below source conf file does work. My issue is the syntax does not seem to match that of native elastic search when trying to use a filter. I would like to simply return a… --- ## [Unable to process attachments using logstash 7.6.0 IMAP Input plugin](https://discuss.elastic.co/t/unable-to-process-attachments-using-logstash-7-6-0-imap-input-plugin/227416)
**Author:** [@Diptee](https://discuss.elastic.co/u/Diptee)\ **Replies:** 2\ **Last updated:** [April 13, 2020, 2:13pm UTC](https://discuss.elastic.co/t/unable-to-process-attachments-using-logstash-7-6-0-imap-input-plugin/227416 "2020-04-13T14:13:56Z")
I have configured logstash input to process emails from Gmail/Outlook using IMAP Input plugin. I am getting error when the email contains attachments or images. Sample config below- input { imap { host =\> "xyz.com" … --- ## [How to parse multiple json using logstash which can occure anywhere in the log](https://discuss.elastic.co/t/how-to-parse-multiple-json-using-logstash-which-can-occure-anywhere-in-the-log/227781)
**Author:** [@Preyas\_Mistry](https://discuss.elastic.co/u/Preyas_Mistry)\ **Replies:** 0\ **Last updated:** [April 13, 2020, 1:53pm UTC](https://discuss.elastic.co/t/how-to-parse-multiple-json-using-logstash-which-can-occure-anywhere-in-the-log/227781 "2020-04-13T13:53:25Z")
I want to parse this log into ES which has multiple JSON string in it, the position of JSON is not fixed! 2020-03-30 17:42:15,672 INFO \[DefaultMessageListenerContainer-4\] (MeetingServiceImpl.getMeetingParticipants:270)… --- ## [\_dateparsefailure - Failing to parse timestamp field into @timestamp](https://discuss.elastic.co/t/dateparsefailure-failing-to-parse-timestamp-field-into-timestamp/227764)
**Author:** [@rsustic](https://discuss.elastic.co/u/rsustic)\ **Replies:** 4\ **Last updated:** [April 13, 2020, 1:48pm UTC](https://discuss.elastic.co/t/dateparsefailure-failing-to-parse-timestamp-field-into-timestamp/227764 "2020-04-13T13:48:14Z")
Hello, I'm dealing with this problem for quite some time now and after reading all topics related to this, I still wasn't able to find the solution. I am trying to replace the @timestamp field with my log timestamp havi… --- ## [RabbitMQ: Configure read interval for rmq as input plugin](https://discuss.elastic.co/t/rabbitmq-configure-read-interval-for-rmq-as-input-plugin/226705)
**Author:** [@notricky](https://discuss.elastic.co/u/notricky)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 1:28pm UTC](https://discuss.elastic.co/t/rabbitmq-configure-read-interval-for-rmq-as-input-plugin/226705 "2020-04-13T13:28:06Z")
HI! Im new here. Please dont get mad. :slight\_smile: I would like to know how to set an interval, that fires Logstash to check the RabbitMQ (rmq) for new messages? I was studying the manual to find out which argument … --- ## [After adding Prune filter - logs are not going to Elastic search](https://discuss.elastic.co/t/after-adding-prune-filter-logs-are-not-going-to-elastic-search/227614)
**Author:** [@Vignesh2912](https://discuss.elastic.co/u/Vignesh2912)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 10:37am UTC](https://discuss.elastic.co/t/after-adding-prune-filter-logs-are-not-going-to-elastic-search/227614 "2020-04-13T10:37:14Z")
Hi Team, I am learning ELK and trying to do as a POC for my project. I am applying KV filter for the sample integration logs from my project and i could see lot of extra fields are coming as a result so i have tried to … --- ## [Sql\_last\_value not working properly](https://discuss.elastic.co/t/sql-last-value-not-working-properly/227244)
**Author:** [@chelseia](https://discuss.elastic.co/u/chelseia)\ **Replies:** 2\ **Last updated:** [April 13, 2020, 9:30am UTC](https://discuss.elastic.co/t/sql-last-value-not-working-properly/227244 "2020-04-13T09:30:10Z")
i am trying to ingest the records with following timestamps 01.03.38.75894600 01.03.38.88577300 01.03.39.08149800 .but before that I have successfully ingested 100 rec which having same timestamp.so no w i have my sq… --- ## [Sql\_last\_value in logstash http\_pooler or http\_input](https://discuss.elastic.co/t/sql-last-value-in-logstash-http-pooler-or-http-input/227744)
**Author:** [@sandy\_rizqi](https://discuss.elastic.co/u/sandy_rizqi)\ **Replies:** 0\ **Last updated:** [April 13, 2020, 8:26am UTC](https://discuss.elastic.co/t/sql-last-value-in-logstash-http-pooler-or-http-input/227744 "2020-04-13T08:26:43Z")
Hi there, May someone give me short explanation about http\_pooler or http input plugin , from what variable i can get tracking data column ? i.e in jdbc input plugin we can use sql\_last\_value , so how in http or http… --- ## [World-writeable permission](https://discuss.elastic.co/t/world-writeable-permission/227721)
**Author:** [@kenahack](https://discuss.elastic.co/u/kenahack)\ **Replies:** 0\ **Last updated:** [April 13, 2020, 3:16am UTC](https://discuss.elastic.co/t/world-writeable-permission/227721 "2020-04-13T03:16:22Z")
Hi, I have found world-writeable permission (777) on the following directory and after: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/snmp-1.3.1/\* My question is that does Logstash require such permission? The aff… --- ## [I don't see recent logs in Kibana](https://discuss.elastic.co/t/i-dont-see-recent-logs-in-kibana/227583)
**Author:** [@manoj123](https://discuss.elastic.co/u/manoj123)\ **Replies:** 1\ **Last updated:** [April 12, 2020, 8:27am UTC](https://discuss.elastic.co/t/i-dont-see-recent-logs-in-kibana/227583 "2020-04-12T08:27:27Z")
Hi, I am new to Elasticsearch. I am using a 5.6.8 version of Elasticsearch and data comes from Mesos to Elasticsearch. Everything went well in setting up ELK stack but somehow I am not able to see recent logs in Kibana.… --- ## [Logstash java class error](https://discuss.elastic.co/t/logstash-java-class-error/227644)
**Author:** [@Krish\_Na](https://discuss.elastic.co/u/Krish_Na)\ **Replies:** 1\ **Last updated:** [April 12, 2020, 8:22am UTC](https://discuss.elastic.co/t/logstash-java-class-error/227644 "2020-04-12T08:22:06Z")
Hi Gurus, Its been crazy Easter days for me, I tried installing ELK and JDK 14. Elasticsearch is working fine however when I try to run logbash.bat I am getting below error Java HotSpot(TM) 64-Bit Server VM warning: Ign… --- ## [Please help - date parse error (a very simple dd-mm-yyyy) csv to logstash](https://discuss.elastic.co/t/please-help-date-parse-error-a-very-simple-dd-mm-yyyy-csv-to-logstash/227599)
**Author:** [@Venkatesh\_Murthy](https://discuss.elastic.co/u/Venkatesh_Murthy)\ **Replies:** 2\ **Last updated:** [April 12, 2020, 5:27am UTC](https://discuss.elastic.co/t/please-help-date-parse-error-a-very-simple-dd-mm-yyyy-csv-to-logstash/227599 "2020-04-12T05:27:07Z")
my CSV has the following: gymid,gymdate,sets,exercise,worktime,rest,reps,weight,muclegroup 3013,20-03-2020,1,Barbell Deadlift,01.42,03.02,11,47,back my config has: input { file{ path=\>"D:/gym1.csv" start\_po… --- ## [Grok pattern DATA vs GREEDYDATA mismatch](https://discuss.elastic.co/t/grok-pattern-data-vs-greedydata-mismatch/225300)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 3\ **Last updated:** [April 11, 2020, 4:45pm UTC](https://discuss.elastic.co/t/grok-pattern-data-vs-greedydata-mismatch/225300 "2020-04-11T16:45:52Z")
In one log file, I have two different formats of log lines as below. Why does %{GREEDYDATA:loglevel} and %{DATA:loglevel} make a huge difference in loglevel output? 2020-03-26 11:31:10,324 \[Thread-40\] INFO … --- ## [Aggregate plugin sends all events to ES](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445)
**Author:** [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Replies:** 7\ **Last updated:** [April 11, 2020, 2:28pm UTC](https://discuss.elastic.co/t/aggregate-plugin-sends-all-events-to-es/227445 "2020-04-11T14:28:17Z")
Hi , Need to use multiple events and build single aggregated event based on session id to send to ES Here is how data looks like: session open for user abc (session id :1) session open for user bbc (session id :2) .… --- ## [Extract a integer from an array based in field values](https://discuss.elastic.co/t/extract-a-integer-from-an-array-based-in-field-values/227498)
**Author:** [@krille.com](https://discuss.elastic.co/u/krille.com)\ **Replies:** 3\ **Last updated:** [April 11, 2020, 11:45am UTC](https://discuss.elastic.co/t/extract-a-integer-from-an-array-based-in-field-values/227498 "2020-04-11T11:45:23Z")
Hello... I am trying to extract a forex rate from an array based on the year and month fields. I have a array which is like this: // forexRate =\> // 2017;7;0.0001105093, 2017;8;0.0001116904, 2017;9;0.0001114438, 2017;1… --- ## [How to make many conditions for one field?](https://discuss.elastic.co/t/how-to-make-many-conditions-for-one-field/227594)
**Author:** [@Emna1](https://discuss.elastic.co/u/Emna1)\ **Replies:** 0\ **Last updated:** [April 11, 2020, 10:23am UTC](https://discuss.elastic.co/t/how-to-make-many-conditions-for-one-field/227594 "2020-04-11T10:23:03Z")
Hi, i want to make many conditions associate to one field, itry this code if \[Duration\] \>= 336 { mutate { add\_field =\> { "price" =\> "50$" } } } else if \[Duration\] == 15{ mutate { add\_field =\> { "price" =\> "10$" } … --- ## [Type error no implicit conversion of NilClass into String with null date](https://discuss.elastic.co/t/type-error-no-implicit-conversion-of-nilclass-into-string-with-null-date/227547)
**Author:** [@mattiacareddu](https://discuss.elastic.co/u/mattiacareddu)\ **Replies:** 2\ **Last updated:** [April 11, 2020, 10:00am UTC](https://discuss.elastic.co/t/type-error-no-implicit-conversion-of-nilclass-into-string-with-null-date/227547 "2020-04-11T10:00:22Z")
I'm having the following error when logstash try to index a null value in date field: Type error no implicit conversion of NilClass into String This is the complete stacktrace 2458 rufus-scheduler intercepted an er… --- ## [Log line with multiple JSON Objects](https://discuss.elastic.co/t/log-line-with-multiple-json-objects/227562)
**Author:** [@ccabral](https://discuss.elastic.co/u/ccabral)\ **Replies:** 2\ **Last updated:** [April 10, 2020, 11:52pm UTC](https://discuss.elastic.co/t/log-line-with-multiple-json-objects/227562 "2020-04-10T23:52:13Z")
Hello, apologies if this has already been asked and answered but I can't seem to find a way to achieve what I'm looking for. I'm using logstash to break down log lines and I have everything separated but I run into a str… --- ## [ELK deploy no longer pulling in data. Logstash logs error, stops immediately](https://discuss.elastic.co/t/elk-deploy-no-longer-pulling-in-data-logstash-logs-error-stops-immediately/227503)
**Author:** [@EICAR-STANDARD-ANTIV](https://discuss.elastic.co/u/EICAR-STANDARD-ANTIV)\ **Replies:** 7\ **Last updated:** [April 10, 2020, 9:47pm UTC](https://discuss.elastic.co/t/elk-deploy-no-longer-pulling-in-data-logstash-logs-error-stops-immediately/227503 "2020-04-10T21:47:36Z")
Hi, I'm trying to run a log aggregation system based on ELK called LME \[1\]. I've pulled most of the text of this post out of the GitHub issue I created \[2\]. I was hoping maybe the community here would be able to e.g. t… --- ## [Logstash pipeline port is not visible outside Openshift/Kubernetes](https://discuss.elastic.co/t/logstash-pipeline-port-is-not-visible-outside-openshift-kubernetes/227372)
**Author:** [@nila](https://discuss.elastic.co/u/nila)\ **Replies:** 5\ **Last updated:** [April 10, 2020, 7:37pm UTC](https://discuss.elastic.co/t/logstash-pipeline-port-is-not-visible-outside-openshift-kubernetes/227372 "2020-04-10T19:37:48Z")
Hello all - I have setup ELK cluster on Openshift/Kubernetes with TLS. All components work fine. But when I configure filebeat to my logstash with pipeline port(created service with a nodeport), it is unable to find t… --- ## [Unable to see Logstash monitoring info in Kibana UI](https://discuss.elastic.co/t/unable-to-see-logstash-monitoring-info-in-kibana-ui/227420)
**Author:** [@sandeepkanabar](https://discuss.elastic.co/u/sandeepkanabar)\ **Replies:** 2\ **Last updated:** [April 10, 2020, 7:10pm UTC](https://discuss.elastic.co/t/unable-to-see-logstash-monitoring-info-in-kibana-ui/227420 "2020-04-10T19:10:01Z")
As per the compatibility matrix, Logstash 5.6.16 is supported with ES and Kibana 6.8.6. However, in my case, while Logstash 5.6.16 is able to push monitoring data to ES 6.8.6, Logstash monitoring details are NOT visible … --- ## [Date Filter on Unix Epoch Failing](https://discuss.elastic.co/t/date-filter-on-unix-epoch-failing/227422)
**Author:** [@ben-10](https://discuss.elastic.co/u/ben-10)\ **Replies:** 8\ **Last updated:** [April 10, 2020, 4:09pm UTC](https://discuss.elastic.co/t/date-filter-on-unix-epoch-failing/227422 "2020-04-10T16:09:43Z")
The date filter fails when attempting to match a unix epoch integer field and I can't figure out why. The data ingested is from xml using the following logstash config: input { file { path =\> \[ "/home/ben/d… --- ## [How to Create Different index for different Logpaths](https://discuss.elastic.co/t/how-to-create-different-index-for-different-logpaths/227438)
**Author:** [@Pacha\_Gopi](https://discuss.elastic.co/u/Pacha_Gopi)\ **Replies:** 3\ **Last updated:** [April 10, 2020, 3:58pm UTC](https://discuss.elastic.co/t/how-to-create-different-index-for-different-logpaths/227438 "2020-04-10T15:58:44Z")
Hi I am getting logs from filebeat to my ELK server.One filebeat is sending two diiferent log files.My file beat is sending Nginx logs and Laraverl logs,my requirement is that i have to create Different index for differ… --- ## [How change the format of duration between two dates in dateTime?](https://discuss.elastic.co/t/how-change-the-format-of-duration-between-two-dates-in-datetime/227488)
**Author:** [@Emna1](https://discuss.elastic.co/u/Emna1)\ **Replies:** 3\ **Last updated:** [April 10, 2020, 3:50pm UTC](https://discuss.elastic.co/t/how-change-the-format-of-duration-between-two-dates-in-datetime/227488 "2020-04-10T15:50:18Z")
I calculate the difference between two dates with this code. ruby{ code =\> "event.set('Duration', (event.get('Release\_date') - event.get('Date\_of\_entry')))" } the output that i have : field duration with type number … --- ## [Clear any inflight events in memory before starting Logstash after restart](https://discuss.elastic.co/t/clear-any-inflight-events-in-memory-before-starting-logstash-after-restart/224135)
**Author:** [@nmoham](https://discuss.elastic.co/u/nmoham)\ **Replies:** 1\ **Last updated:** [April 10, 2020, 2:28pm UTC](https://discuss.elastic.co/t/clear-any-inflight-events-in-memory-before-starting-logstash-after-restart/224135 "2020-04-10T14:28:53Z")
We're sending logs using filebeats to Logstash (Parsing with configs) --\> LumberJack --\> ES. In some situations while collecting newer data types, we want to re-index data, so we do the following - stop filebeats stop… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=374) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=376)