# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=675 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 676 --- ## [Last\_value for elasticsearch input plugin](https://discuss.elastic.co/t/last-value-for-elasticsearch-input-plugin/150972)
**Author:** [@MarkusT](https://discuss.elastic.co/u/MarkusT)\ **Replies:** 1\ **Last updated:** [October 4, 2018, 7:31pm UTC](https://discuss.elastic.co/t/last-value-for-elasticsearch-input-plugin/150972 "2018-10-04T19:31:56Z")
Hey there, is there something like the sql\_last\_value of jdbc input plugin for the elasticsearch input plugin? What I want to do is to execute a timed search on elastic, sorted by time values (and enrich raw from sql i… --- ## [Sample stdout not working](https://discuss.elastic.co/t/sample-stdout-not-working/150747)
**Author:** [@Barath95](https://discuss.elastic.co/u/Barath95)\ **Replies:** 6\ **Last updated:** [October 4, 2018, 7:22pm UTC](https://discuss.elastic.co/t/sample-stdout-not-working/150747 "2018-10-04T19:22:00Z")
Hi, I am using logstash 6.4.1 and a beginner. I am trying to execute the sample tutorial in the introduction blog but I dont get any results and the process doesn't end forcing me to do a ctrl^c to abort. Below is my co… --- ## [Time converted to EST, when Unixtime is already EST](https://discuss.elastic.co/t/time-converted-to-est-when-unixtime-is-already-est/150734)
**Author:** [@Nich240](https://discuss.elastic.co/u/Nich240)\ **Replies:** 3\ **Last updated:** [October 4, 2018, 7:21pm UTC](https://discuss.elastic.co/t/time-converted-to-est-when-unixtime-is-already-est/150734 "2018-10-04T19:21:19Z")
The date passed from logstash is 4 hours before the time the event occurs. I am receiving time as UNIX\_MS, but this is in EST. Therefore, the match will convert it to EST (taking off 4 hours) which it doesn't need to d… --- ## [Multiple kafka clusters?](https://discuss.elastic.co/t/multiple-kafka-clusters/150626)
**Author:** [@Bernie\_Hsiung](https://discuss.elastic.co/u/Bernie_Hsiung)\ **Replies:** 1\ **Last updated:** [October 4, 2018, 6:03pm UTC](https://discuss.elastic.co/t/multiple-kafka-clusters/150626 "2018-10-04T18:03:54Z")
Is it possible to have a single logstash instance listen to multiple kafka clusters? I'm uncertain if: input { kafka { bootstrap\_servers =\> "kafka\_cluster\_A-01:9092,kafka\_cluster\_A-02:9092" } } input { kafka { b… --- ## [Creating an output plugin -- guide not working](https://discuss.elastic.co/t/creating-an-output-plugin-guide-not-working/150432)
**Author:** [@mjnorman](https://discuss.elastic.co/u/mjnorman)\ **Replies:** 2\ **Last updated:** [October 4, 2018, 4:56pm UTC](https://discuss.elastic.co/t/creating-an-output-plugin-guide-not-working/150432 "2018-10-04T16:56:23Z")
Hello all. I am attempting to update an older plugin with the assistance of the original developer. We are both fairly new to Ruby, so keep that in mind while reading the below. At first I attempted to work with the e… --- ## [Queue.drain: true not being respected](https://discuss.elastic.co/t/queue-drain-true-not-being-respected/151073)
**Author:** [@cmendelson](https://discuss.elastic.co/u/cmendelson)\ **Replies:** 0\ **Last updated:** [October 4, 2018, 3:55pm UTC](https://discuss.elastic.co/t/queue-drain-true-not-being-respected/151073 "2018-10-04T15:55:47Z")
I've enabled the queue.drain: true setting on one of my logstash instances which has a persistent queue. The behavior I would expect is that when I stop the logstash process (service logstash stop) the page files would … --- ## [Elastic 6.4.1 on CentOS 7 x64 - Pipelines.yml Question](https://discuss.elastic.co/t/elastic-6-4-1-on-centos-7-x64-pipelines-yml-question/150773)
**Author:** [@mreloysanchez](https://discuss.elastic.co/u/mreloysanchez)\ **Replies:** 2\ **Last updated:** [October 4, 2018, 1:33pm UTC](https://discuss.elastic.co/t/elastic-6-4-1-on-centos-7-x64-pipelines-yml-question/150773 "2018-10-04T13:33:24Z")
Hello, Our installation is progressing and I'm setting up logstash. I've got it to run with the noted test config file here: https://www.elastic.co/guide/en/logstash/current/configuration.html However, I am receiving a… --- ## [Problem using jdbc-input plugin with Oracle PKI/Cert Authentication](https://discuss.elastic.co/t/problem-using-jdbc-input-plugin-with-oracle-pki-cert-authentication/150931)
**Author:** [@jmstein](https://discuss.elastic.co/u/jmstein)\ **Replies:** 1\ **Last updated:** [October 4, 2018, 1:18pm UTC](https://discuss.elastic.co/t/problem-using-jdbc-input-plugin-with-oracle-pki-cert-authentication/150931 "2018-10-04T13:18:01Z")
I have one Oracle database using username/password authentication, and another Oracle database using PKI authentication. I've successfully gotten the Logstash jdbc-input plugin working with the username/password authent… --- ## [Problem to start logstash : java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/problem-to-start-logstash-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/151017)
**Author:** [@morihend](https://discuss.elastic.co/u/morihend)\ **Replies:** 2\ **Last updated:** [October 4, 2018, 1:02pm UTC](https://discuss.elastic.co/t/problem-to-start-logstash-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/151017 "2018-10-04T13:02:37Z")
Hello, I use windows logstash 6.4.0 version. My logstash service worked untill i install the input-jdbc plugin. Now, when i tape in my terminal window this : bin\\logstash.bat to start logstash i obtained this mistack … --- ## [Multiline pattern for logstash rsyslog](https://discuss.elastic.co/t/multiline-pattern-for-logstash-rsyslog/150654)
**Author:** [@7ee7d25eea152135bf4d](https://discuss.elastic.co/u/7ee7d25eea152135bf4d)\ **Replies:** 7\ **Last updated:** [October 4, 2018, 12:56pm UTC](https://discuss.elastic.co/t/multiline-pattern-for-logstash-rsyslog/150654 "2018-10-04T12:56:16Z")
--- ## [Logstasch pipe line error](https://discuss.elastic.co/t/logstasch-pipe-line-error/150656)
**Author:** [@Mohamed.Yousry](https://discuss.elastic.co/u/Mohamed.Yousry)\ **Replies:** 6\ **Last updated:** [October 4, 2018, 12:35pm UTC](https://discuss.elastic.co/t/logstasch-pipe-line-error/150656 "2018-10-04T12:35:28Z")
Hi There I am facing a problem each time I start logstash , below are the errors, Can you support Sending Logstash logs to C:/ELK\_STACK/logstash-6.4.0/logs which is now configured via log4j2.properties \[2018-10-02T09:… --- ## [Output file to create based on timestamp](https://discuss.elastic.co/t/output-file-to-create-based-on-timestamp/150880)
**Author:** [@pkshara](https://discuss.elastic.co/u/pkshara)\ **Replies:** 1\ **Last updated:** [October 4, 2018, 12:02pm UTC](https://discuss.elastic.co/t/output-file-to-create-based-on-timestamp/150880 "2018-10-04T12:02:19Z")
Hi Team, Please suggest for the below query, input { } filter { } output { file { path =\> "tmp/path/output-%{+YYYY-MM-dd}" } } This would create an output file based on the day wise / everyday new text file w… --- ## [Unable to create logstash pattern using grok for below](https://discuss.elastic.co/t/unable-to-create-logstash-pattern-using-grok-for-below/150997)
**Author:** [@Hanit\_grover](https://discuss.elastic.co/u/Hanit_grover)\ **Replies:** 1\ **Last updated:** [October 4, 2018, 11:54am UTC](https://discuss.elastic.co/t/unable-to-create-logstash-pattern-using-grok-for-below/150997 "2018-10-04T11:54:37Z")
INFO 2018-09-26 18:55:10,802 \[\[logging\_framework\].HTTP\_Listener\_Configuration.worker.01\] com.temp.analytics.Logger: {"flowLevel":"subflow","apiName":"test\_MainFlow","transactionId":"df782540-c169-11e8-8c83-40b89aa3f91a"… --- ## [Error code when importing csv file with logstash](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762)
**Author:** [@drr](https://discuss.elastic.co/u/drr)\ **Replies:** 9\ **Last updated:** [October 4, 2018, 10:50am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762 "2018-10-04T10:50:17Z")
hi logstash team I am trying to import csv file andI cant figure out why I am getting below error, your help would be greatly appreciated ./logstash -f ../conf/logstash.yml Sending Logstash's logs to /home/d/Documents/… --- ## [Logstash 6.4.1 Ignoring Netflow 10](https://discuss.elastic.co/t/logstash-6-4-1-ignoring-netflow-10/149488)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 6\ **Last updated:** [October 4, 2018, 10:09am UTC](https://discuss.elastic.co/t/logstash-6-4-1-ignoring-netflow-10/149488 "2018-10-04T10:09:57Z")
Hi, I have installed Logstash 6.4.1 and I have enabled the netflow module but I'm getting "Ignoring Netflow version v10". As far as I understand, netflow 10 is supported in this version. Am I correct? Here is the error … --- ## [Replace same substring in multiple places in a text](https://discuss.elastic.co/t/replace-same-substring-in-multiple-places-in-a-text/151012)
**Author:** [@rajaboini](https://discuss.elastic.co/u/rajaboini)\ **Replies:** 0\ **Last updated:** [October 4, 2018, 9:45am UTC](https://discuss.elastic.co/t/replace-same-substring-in-multiple-places-in-a-text/151012 "2018-10-04T09:45:36Z")
Hi All, My name Raja, I am new to elasticsearch I am using logstash to pull data from my database and push to elasticsearch. I have text content like below " "content": "attachment:37:gfggfg.csv:application/octet-st… --- ## [Multiple Pipeline Recommendation](https://discuss.elastic.co/t/multiple-pipeline-recommendation/151006)
**Author:** [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\ **Replies:** 0\ **Last updated:** [October 4, 2018, 9:25am UTC](https://discuss.elastic.co/t/multiple-pipeline-recommendation/151006 "2018-10-04T09:25:41Z")
Hi There, We have purchased an elastic cloud instance and currently running a single pipeline for a specific application. We have multiple clients, with different applications and would like to know if it is better to … --- ## [Elapsed and aggregate filter with multiple workers](https://discuss.elastic.co/t/elapsed-and-aggregate-filter-with-multiple-workers/150647)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 5\ **Last updated:** [October 4, 2018, 8:43am UTC](https://discuss.elastic.co/t/elapsed-and-aggregate-filter-with-multiple-workers/150647 "2018-10-04T08:43:20Z")
Elapsed and aggregate filters will not work properly when multiple logstash workers are used. In that case, how do we solve the same problem? Thank you. same question with this https://stackoverflow.com/questions/373533… --- ## [Lumberjack (beats) output not supporting SNI TLS enpoints?](https://discuss.elastic.co/t/lumberjack-beats-output-not-supporting-sni-tls-enpoints/150566)
**Author:** [@USP\_rol](https://discuss.elastic.co/u/USP_rol)\ **Replies:** 1\ **Last updated:** [October 4, 2018, 7:10am UTC](https://discuss.elastic.co/t/lumberjack-beats-output-not-supporting-sni-tls-enpoints/150566 "2018-10-04T07:10:56Z")
Hi all We're testing with logstash/beats/elasticsearch and are having a data flow (like beats =\> logstash =\> elasticsearch) over different networks. this works fine as long as there is no cloud (i.e. openshift) involve… --- ## [Plugin to move log files to Processed directory on successful load and to Rejected folder on failure](https://discuss.elastic.co/t/plugin-to-move-log-files-to-processed-directory-on-successful-load-and-to-rejected-folder-on-failure/149770)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 2\ **Last updated:** [October 4, 2018, 7:03am UTC](https://discuss.elastic.co/t/plugin-to-move-log-files-to-processed-directory-on-successful-load-and-to-rejected-folder-on-failure/149770 "2018-10-04T07:03:09Z")
I am loading new log files from an AWS S3 bucket every hour. I want to maintain a system which will load and process only the new uploads and move them to Processed folder on successful load, and move to Rejected folder … --- ## [Hardware recommendation for fb + Logstash](https://discuss.elastic.co/t/hardware-recommendation-for-fb-logstash/150555)
**Author:** [@ratman](https://discuss.elastic.co/u/ratman)\ **Replies:** 2\ **Last updated:** [October 4, 2018, 6:42am UTC](https://discuss.elastic.co/t/hardware-recommendation-for-fb-logstash/150555 "2018-10-04T06:42:48Z")
Hello! I am mounting a server to run Filebeat + Logstahs which will send the output info to our Elastic Cloud Service. Currently, we are daily receiving around 500-600mb of logs and the processing they would go through … --- ## [URIHOST failing with trailing numbers on 2LD in Visualization](https://discuss.elastic.co/t/urihost-failing-with-trailing-numbers-on-2ld-in-visualization/150956)
**Author:** [@Dave\_Evenden](https://discuss.elastic.co/u/Dave_Evenden)\ **Replies:** 0\ **Last updated:** [October 4, 2018, 3:07am UTC](https://discuss.elastic.co/t/urihost-failing-with-trailing-numbers-on-2ld-in-visualization/150956 "2018-10-04T03:07:45Z")
when we use this grok: match =\> { "message" =\> "^%{URIHOST:domain}" } It successfully matches domains like so: domain: mydomain.net however when creating a visualization any 2LD's with a trailing number is separated … --- ## [Check service is running or not on remote machine before run Logstash exec input plugin](https://discuss.elastic.co/t/check-service-is-running-or-not-on-remote-machine-before-run-logstash-exec-input-plugin/150252)
**Author:** [@Gireesh\_Deepak\_R](https://discuss.elastic.co/u/Gireesh_Deepak_R)\ **Replies:** 6\ **Last updated:** [October 3, 2018, 9:07pm UTC](https://discuss.elastic.co/t/check-service-is-running-or-not-on-remote-machine-before-run-logstash-exec-input-plugin/150252 "2018-10-03T21:07:23Z")
Hello Team, I need help to find a work around for my use case. I am using logstash exec input plugin to execute some bash scripts, to process data and index them into ES. And, exec was scheduled to run daily once at 12… --- ## [Grok filter with file input](https://discuss.elastic.co/t/grok-filter-with-file-input/150348)
**Author:** [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)\ **Replies:** 4\ **Last updated:** [October 3, 2018, 8:17pm UTC](https://discuss.elastic.co/t/grok-filter-with-file-input/150348 "2018-10-03T20:17:59Z")
Hi I have set of logs that are generated to .log files, and I want to index them in Elasticsearch using Logstash. The log file contains many lines each line looks like the following: 2018-09-26 17:05:34,060 INFO: Proc… --- ## [Logstash 6.2.3 convert json values to GeoPoint Issue](https://discuss.elastic.co/t/logstash-6-2-3-convert-json-values-to-geopoint-issue/150418)
**Author:** [@Sajith\_Vijesekara](https://discuss.elastic.co/u/Sajith_Vijesekara)\ **Replies:** 5\ **Last updated:** [October 3, 2018, 6:44pm UTC](https://discuss.elastic.co/t/logstash-6-2-3-convert-json-values-to-geopoint-issue/150418 "2018-10-03T18:44:35Z")
Hi I am new to ELK so first I want to setup custom json file read using logstash. I want to show the customer locations in kibana map . This is the json format I want to convert. {"id":1,"first\_name":"Freeman","last\_nam… --- ## [How do you install the syslog input plugin?](https://discuss.elastic.co/t/how-do-you-install-the-syslog-input-plugin/150490)
**Author:** [@meatwad](https://discuss.elastic.co/u/meatwad)\ **Replies:** 1\ **Last updated:** [October 3, 2018, 6:01pm UTC](https://discuss.elastic.co/t/how-do-you-install-the-syslog-input-plugin/150490 "2018-10-03T18:01:01Z")
Hi, I got Elastic configured last night and have Logstash listening on port 5000: $ netstat -aptn |grep LIST |grep 5000 tcp 0 0 0.0.0.0:5000 0.0.0.0:\* LISTEN 10743/java I have rsyslog on a different forwarding its … --- ## [Issues with DATESTAMP pattern](https://discuss.elastic.co/t/issues-with-datestamp-pattern/150624)
**Author:** [@Samvid\_Kulkarni](https://discuss.elastic.co/u/Samvid_Kulkarni)\ **Replies:** 1\ **Last updated:** [October 3, 2018, 5:57pm UTC](https://discuss.elastic.co/t/issues-with-datestamp-pattern/150624 "2018-10-03T17:57:44Z")
I am trying to parse the date which has YYYY/MM/DD HH:MM:SS format. For this I am using DATESTAMP pattern. If I just take the date from log 2018/09/26 17:00:04 and use %{DATESTAMP:test} then it gives proper output in gro… --- ## [Kafka to Elastic search through Logstash for add, delete, update actions](https://discuss.elastic.co/t/kafka-to-elastic-search-through-logstash-for-add-delete-update-actions/150406)
**Author:** [@vipindwivedi](https://discuss.elastic.co/u/vipindwivedi)\ **Replies:** 2\ **Last updated:** [October 3, 2018, 5:51pm UTC](https://discuss.elastic.co/t/kafka-to-elastic-search-through-logstash-for-add-delete-update-actions/150406 "2018-10-03T17:51:23Z")
Hi, I am looking a solution for loading data in form of messages with Add, Update, Delete actions from Kafka stream to Elastic search, some of the options I have explored and found simple way is through Logstash but hav… --- ## [Http\_poller "error": "Read timed out"](https://discuss.elastic.co/t/http-poller-error-read-timed-out/150899)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 1\ **Last updated:** [October 3, 2018, 3:42pm UTC](https://discuss.elastic.co/t/http-poller-error-read-timed-out/150899 "2018-10-03T15:42:41Z")
Below works only when limiting return of snapshots to a small number. Curl on the full set of snapshots works fine. It appear request timeout is not helping either. What am I missing? input { http\_poller { urls =\> … --- ## [Logstash Data Enrichment with URLHAUS](https://discuss.elastic.co/t/logstash-data-enrichment-with-urlhaus/150756)
**Author:** [@martb](https://discuss.elastic.co/u/martb)\ **Replies:** 7\ **Last updated:** [October 3, 2018, 3:18pm UTC](https://discuss.elastic.co/t/logstash-data-enrichment-with-urlhaus/150756 "2018-10-03T15:18:25Z")
Hi, After watching the recent elastic threat hunting Webex which mentioned URLHAUS, so I wanted to try to add a lookup to the haus csv data source. Currently in my logstash conf Im connecting to a database which contai… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=674) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=676)