# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=884 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 885 --- ## [Run SQL statement](https://discuss.elastic.co/t/run-sql-statement/105651)
**Author:** [@Sumit\_Gupta1](https://discuss.elastic.co/u/Sumit_Gupta1)\ **Replies:** 3\ **Last updated:** [October 30, 2017, 6:53am UTC](https://discuss.elastic.co/t/run-sql-statement/105651 "2017-10-30T06:53:31Z")
Hi, I want to run following SQL statment: input { jdbc { jdbc\_driver\_library =\> "/app/logstash-5.6.2/lib/ojdbc7.jar" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver" jdbc\_connection\_string =\> "jdbc:orac… --- ## [GEO Grok Question For Auth Log](https://discuss.elastic.co/t/geo-grok-question-for-auth-log/105596)
**Author:** [@RichardLaing](https://discuss.elastic.co/u/RichardLaing)\ **Replies:** 1\ **Last updated:** [October 30, 2017, 6:44am UTC](https://discuss.elastic.co/t/geo-grok-question-for-auth-log/105596 "2017-10-30T06:44:03Z")
Hi there I am looking for some help with getting GEOIP working with logstash. At present I have a working grok for my auth log however I keep getting "\_geoip\_lookup\_failure" whenever I try to use the following, my ques… --- ## [Json mapping causes logs to be lost](https://discuss.elastic.co/t/json-mapping-causes-logs-to-be-lost/105468)
**Author:** [@beckonite](https://discuss.elastic.co/u/beckonite)\ **Replies:** 4\ **Last updated:** [October 30, 2017, 6:33am UTC](https://discuss.elastic.co/t/json-mapping-causes-logs-to-be-lost/105468 "2017-10-30T06:33:44Z")
When a JSON object is logged as a string and subsequently as an object (or visa versa), it causes Elasticsearch to throw a stack trace and not record the element, but there is no recovery form Logstash, which causes the … --- ## [Logstash reads beyond "message" field when using GREEDYDATA](https://discuss.elastic.co/t/logstash-reads-beyond-message-field-when-using-greedydata/105628)
**Author:** [@sidhant](https://discuss.elastic.co/u/sidhant)\ **Replies:** 2\ **Last updated:** [October 30, 2017, 4:26am UTC](https://discuss.elastic.co/t/logstash-reads-beyond-message-field-when-using-greedydata/105628 "2017-10-30T04:26:21Z")
Hi, I recently stumbled upon this issue with GREEDYDATA in Logstash grok filter, and I'm not sure if I'm doing something wrong or if it's a general bug. It appears that when a grok match pattern ends with %{GREEDYDATA},… --- ## [Does Logstash support Windows 64?](https://discuss.elastic.co/t/does-logstash-support-windows-64/105661)
**Author:** [@yyqiang](https://discuss.elastic.co/u/yyqiang)\ **Replies:** 5\ **Last updated:** [October 30, 2017, 1:34am UTC](https://discuss.elastic.co/t/does-logstash-support-windows-64/105661 "2017-10-30T01:34:21Z")
When starting logstash-5.6.3 service, error occurred like that, LoadError: Unsupported platform: x86\_64-windows --- ## [Best practice for adding user profile information to event stream](https://discuss.elastic.co/t/best-practice-for-adding-user-profile-information-to-event-stream/105689)
**Author:** [@danielmcquillen](https://discuss.elastic.co/u/danielmcquillen)\ **Replies:** 1\ **Last updated:** [October 30, 2017, 12:24am UTC](https://discuss.elastic.co/t/best-practice-for-adding-user-profile-information-to-event-stream/105689 "2017-10-30T00:24:58Z")
Is there a best practice in logstash for adding extra user information to incoming realtime log events? I'm wondering what the best way is to lookup, for example, MySQL user profile data by the incoming user\_id and add … --- ## [@timestamp field cannot change even date filter is correct: default to YYYY.01.01](https://discuss.elastic.co/t/timestamp-field-cannot-change-even-date-filter-is-correct-default-to-yyyy-01-01/104085)
**Author:** [@pope843](https://discuss.elastic.co/u/pope843)\ **Replies:** 1\ **Last updated:** [October 29, 2017, 11:54pm UTC](https://discuss.elastic.co/t/timestamp-field-cannot-change-even-date-filter-is-correct-default-to-yyyy-01-01/104085 "2017-10-29T23:54:05Z")
Hello, Im having hard time to know why i received the logs in may ES with a @timestamp": "2017-01-01T09:32:25.532Z" which should not be the case. as you can see from below ES data, field "received\_at": "2017-10-16T09:34… --- ## [Grok timestamp help](https://discuss.elastic.co/t/grok-timestamp-help/103723)
**Author:** [@pope843](https://discuss.elastic.co/u/pope843)\ **Replies:** 16\ **Last updated:** [October 29, 2017, 11:53pm UTC](https://discuss.elastic.co/t/grok-timestamp-help/103723 "2017-10-29T23:53:05Z")
Hello, Need help pls, whats the grok format for this kind of timestamp? also i would like to add the current date(e.g YYYY-MM-DD). to it 04:38:30.793UTC My logs is like this. sample log is like this: 05:08:33.351UTC… --- ## [Parsing JSON along with double Quotes](https://discuss.elastic.co/t/parsing-json-along-with-double-quotes/105645)
**Author:** [@ranjith](https://discuss.elastic.co/u/ranjith)\ **Replies:** 1\ **Last updated:** [October 29, 2017, 8:37pm UTC](https://discuss.elastic.co/t/parsing-json-along-with-double-quotes/105645 "2017-10-29T20:37:59Z")
Hi all, I am trying to parse the following json which contains spec field. {"multiplier":1,"name":"bullet","spec":"Deluxe Sink, two compartment, 87"W x 30"D x 41"H, 16/300 stainless steel, 24" x 24" x 11-1/2" deep com… --- ## [Bug: Logstash grok with empty match is impossible to debug](https://discuss.elastic.co/t/bug-logstash-grok-with-empty-match-is-impossible-to-debug/105442)
**Author:** [@kub1x](https://discuss.elastic.co/u/kub1x)\ **Replies:** 2\ **Last updated:** [October 29, 2017, 1:11pm UTC](https://discuss.elastic.co/t/bug-logstash-grok-with-empty-match-is-impossible-to-debug/105442 "2017-10-29T13:11:55Z")
Hey, our Logstash pipeline config files are generated. There was a bug in our generation system which led to following grog pattern. The pattern was one of MANY patterns and contained override and other clauses as well,… --- ## [ArcSight Index Pattern Missing Fields](https://discuss.elastic.co/t/arcsight-index-pattern-missing-fields/105622)
**Author:** [@Luis\_Arriaga](https://discuss.elastic.co/u/Luis_Arriaga)\ **Replies:** 0\ **Last updated:** [October 28, 2017, 12:18am UTC](https://discuss.elastic.co/t/arcsight-index-pattern-missing-fields/105622 "2017-10-28T00:18:55Z")
I installed the ArcSight module and am getting data from the smart connector but it seems like there are some fields missing from the arcsigh-\* index. I am getting the following errors. Could someone please help me resol… --- ## [Logstash-filter-rest throws client\_error](https://discuss.elastic.co/t/logstash-filter-rest-throws-client-error/105307)
**Author:** [@mahsa\_b](https://discuss.elastic.co/u/mahsa_b)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 5:51pm UTC](https://discuss.elastic.co/t/logstash-filter-rest-throws-client-error/105307 "2017-10-27T17:51:17Z")
Hi , I installed the logstash-filter-rest plugin and tried to run this config: logstash --debug --path.data=/tmp/mahsa-test -e 'input { stdin {} } filter { rest { request =\> { url =\> "http://www.google.se/" } target =\>… --- ## [Conditional error when choosing jdbc\_streaming statement](https://discuss.elastic.co/t/conditional-error-when-choosing-jdbc-streaming-statement/105590)
**Author:** [@LuigiAG](https://discuss.elastic.co/u/LuigiAG)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 5:26pm UTC](https://discuss.elastic.co/t/conditional-error-when-choosing-jdbc-streaming-statement/105590 "2017-10-27T17:26:25Z")
Hello, I am having an issue in my logstash 5.6.3 and I don't know why (tss those beginners :slight\_smile: ) Here is my code (some dummy code in here for my safety XD) jdbc\_streaming { jdbc\_driver\_library =\> "C:\\Prog… --- ## [\_dateparsefailure when field is fine](https://discuss.elastic.co/t/dateparsefailure-when-field-is-fine/105597)
**Author:** [@JeremyinNC](https://discuss.elastic.co/u/JeremyinNC)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 5:25pm UTC](https://discuss.elastic.co/t/dateparsefailure-when-field-is-fine/105597 "2017-10-27T17:25:30Z")
I am having trouble getting the correct dates in my logstash config. I have the following as my "date" filter: date { locale =\> en match =\> \[ "msr\_open\_date\_time" , "yyyy-MM-dd'T'HH:mm:ss.SSS", "dd/MMM/yyyy:HH:m… --- ## [Error logs for nginx and apache not working](https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494)
**Author:** [@joperr](https://discuss.elastic.co/u/joperr)\ **Replies:** 3\ **Last updated:** [October 27, 2017, 5:23pm UTC](https://discuss.elastic.co/t/error-logs-for-nginx-and-apache-not-working/105494 "2017-10-27T17:23:22Z")
Hi there! I'm adding some conf files to my logstash based on https://www.elastic.co/guide/en/logstash/current/logstash-config-for-filebeat-modules.html So I have two config files working great: file1- input { beats… --- ## [Logstash mysql input credentials error](https://discuss.elastic.co/t/logstash-mysql-input-credentials-error/105588)
**Author:** [@Tim\_Rourke](https://discuss.elastic.co/u/Tim_Rourke)\ **Replies:** 2\ **Last updated:** [October 27, 2017, 4:10pm UTC](https://discuss.elastic.co/t/logstash-mysql-input-credentials-error/105588 "2017-10-27T16:10:02Z")
Hello! I'm trying to migrate some MySQL data from my local dev Vagrant box into Elasticsearch using the jdbc MySQL connector, and I'm running into a MySQL credentials problem. I'm not sure if the problem is the .conf fi… --- ## [Logstash not indexing data into elasticsearch](https://discuss.elastic.co/t/logstash-not-indexing-data-into-elasticsearch/105290)
**Author:** [@cm47](https://discuss.elastic.co/u/cm47)\ **Replies:** 4\ **Last updated:** [October 27, 2017, 2:39pm UTC](https://discuss.elastic.co/t/logstash-not-indexing-data-into-elasticsearch/105290 "2017-10-27T14:39:52Z")
Hi there, I am trying to get custom log information indexed into elasticsearch using a grok filter. Logstash configuration file: input { file { path =\> "/etc/logstash/customlog/testfile.log" codec =\> line start\_pos… --- ## [Custom GROK Pattern for XML Text file](https://discuss.elastic.co/t/custom-grok-pattern-for-xml-text-file/105576)
**Author:** [@ranjith](https://discuss.elastic.co/u/ranjith)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 2:03pm UTC](https://discuss.elastic.co/t/custom-grok-pattern-for-xml-text-file/105576 "2017-10-27T14:03:23Z")
Hi all, I am new to grok in logstash. I am trying to grok the following text file . ac92ec61-5a5c-df11-beff-001ec95274b6,8b2135d0-ba04-4473-8398-3e0e1045c7c4Warranty754dcbf6-af93-47be-a6d3-8468fcf9ef8c1160WARJACK08M00… --- ## [Grok Pattern for text file](https://discuss.elastic.co/t/grok-pattern-for-text-file/105567)
**Author:** [@ranjith](https://discuss.elastic.co/u/ranjith)\ **Replies:** 2\ **Last updated:** [October 27, 2017, 1:11pm UTC](https://discuss.elastic.co/t/grok-pattern-for-text-file/105567 "2017-10-27T13:11:48Z")
Hi all, I'm trying to parse an text file and am not able to grok the PKey in these text. Please find the sample text file below. ac92ec61-5a5c-df11-beff-001ec95274b6,8b2135d0-ba04-4473-8398-3e0e1045c7c4 I am trying i… --- ## [Unable to access fields in metrics filter event](https://discuss.elastic.co/t/unable-to-access-fields-in-metrics-filter-event/105554)
**Author:** [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\ **Replies:** 0\ **Last updated:** [October 27, 2017, 10:53am UTC](https://discuss.elastic.co/t/unable-to-access-fields-in-metrics-filter-event/105554 "2017-10-27T10:53:47Z")
i'm unable to get metrics count in the metrics event , my conf. file as shown below, kindly help here. filter{ \*\*using grok to extract file\_name \*\* metrics { meter =\> \["%{file\_name}"\] add\_tag =\> "metrics" id =\> "fi… --- ## [Parsing Issue for XML File](https://discuss.elastic.co/t/parsing-issue-for-xml-file/105243)
**Author:** [@sanju1323](https://discuss.elastic.co/u/sanju1323)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 9:18am UTC](https://discuss.elastic.co/t/parsing-issue-for-xml-file/105243 "2017-10-27T09:18:45Z")
Hi, I'm trying to index an XML file and am not able to index the entire log. Please find the sample log below. \ \ \ \ \ **Author:** [@Fopa\_Leon\_Constantin](https://discuss.elastic.co/u/Fopa_Leon_Constantin)\ **Replies:** 2\ **Last updated:** [October 27, 2017, 8:59am UTC](https://discuss.elastic.co/t/is-it-necessary-to-restart-logstash-after-modifying-a-dictionary-path-file/105537 "2017-10-27T08:59:25Z") I used the following translate filter in logstash translate { field =\> "countries" destination =\> "cities" dictionary\_path =\> "/home/rrr/cities.yml" } And I started logstash this way /usr/share/logstash/bin/logst… --- ## [Configuration setup documentation for ELK and ELK is installed as service](https://discuss.elastic.co/t/configuration-setup-documentation-for-elk-and-elk-is-installed-as-service/105527)
**Author:** [@Rajeshwari](https://discuss.elastic.co/u/Rajeshwari)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 8:52am UTC](https://discuss.elastic.co/t/configuration-setup-documentation-for-elk-and-elk-is-installed-as-service/105527 "2017-10-27T08:52:15Z")
Hi, Configuration set up documentation for logstash and filebeat to restart the those services. (ELK installed as a service since not able to see the bin folder to restart the logstash service). Added file in conf.d in… --- ## [Logstash configuration java error](https://discuss.elastic.co/t/logstash-configuration-java-error/105528)
**Author:** [@Aayush\_Arora](https://discuss.elastic.co/u/Aayush_Arora)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 8:48am UTC](https://discuss.elastic.co/t/logstash-configuration-java-error/105528 "2017-10-27T08:48:49Z")
\[error\] I am trying to link elasticsearch and logstash together (transferring mysql database to elasticsearch ) facing this error while running the logstash.config file Please tell what the error is and how to correct… --- ## [Logstash not sending data to elastic search](https://discuss.elastic.co/t/logstash-not-sending-data-to-elastic-search/105515)
**Author:** [@Navinder](https://discuss.elastic.co/u/Navinder)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 6:55am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-elastic-search/105515 "2017-10-27T06:55:23Z")
Hi my config file is executed without any error. It's below input { file { type =\> "jetorders" path =\> "/home/app/jetorders\*.csv" start\_position =\> "beginning" sincedb\_path =\> "/home/app/inputorders.log" } file { ty… --- ## [Loadbalance Logstash without double ES-data?](https://discuss.elastic.co/t/loadbalance-logstash-without-double-es-data/105523)
**Author:** [@Maekee](https://discuss.elastic.co/u/Maekee)\ **Replies:** 3\ **Last updated:** [October 27, 2017, 6:54am UTC](https://discuss.elastic.co/t/loadbalance-logstash-without-double-es-data/105523 "2017-10-27T06:54:58Z")
Hi, The architecture is not done yet, but i want to ask so i know how to Think. Lets say i have: 10 clients i want to collect logs from, which will have beatagents. 2 Logstash servers that the beatagents will send to … --- ## [Stop file ingestion without stopping Logstash](https://discuss.elastic.co/t/stop-file-ingestion-without-stopping-logstash/105449)
**Author:** [@udaykirankona](https://discuss.elastic.co/u/udaykirankona)\ **Replies:** 3\ **Last updated:** [October 27, 2017, 6:53am UTC](https://discuss.elastic.co/t/stop-file-ingestion-without-stopping-logstash/105449 "2017-10-27T06:53:55Z")
Hi Guys, Is there any way to stop particular file ingestion without stopping logstash service. Thanks, Uday.K --- ## [Skip empty fields](https://discuss.elastic.co/t/skip-empty-fields/104694)
**Author:** [@Maekee](https://discuss.elastic.co/u/Maekee)\ **Replies:** 2\ **Last updated:** [October 27, 2017, 6:47am UTC](https://discuss.elastic.co/t/skip-empty-fields/104694 "2017-10-27T06:47:57Z")
In the csv filter plugin there is a configuration option called skip\_empty\_columns. I would like to use the same functionality with Windows Eventlogs. That it, remove fields that are empty. Right now they only say "-… --- ## [Issue with mertics filter plugin](https://discuss.elastic.co/t/issue-with-mertics-filter-plugin/105439)
**Author:** [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\ **Replies:** 1\ **Last updated:** [October 27, 2017, 6:43am UTC](https://discuss.elastic.co/t/issue-with-mertics-filter-plugin/105439 "2017-10-27T06:43:07Z")
HI Team , i'm using metrics filter plugin for different files with their respective count. but i'm unable to get the metrics results. here is my logstash conf input{ reading log file here } filter{ \*\*using grok t… --- ## [XML encoded logs from Mcafee EPO logs to logstash](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380)
**Author:** [@junaid](https://discuss.elastic.co/u/junaid)\ **Replies:** 9\ **Last updated:** [October 27, 2017, 5:02am UTC](https://discuss.elastic.co/t/xml-encoded-logs-from-mcafee-epo-logs-to-logstash/104380 "2017-10-27T05:02:40Z")
Hi my scenario is we need to collect logs from Mcafee EPO and send to our third party cloud logging platform.We have logstash server in between and its receiving logs from EPO and forwarding to logging platform.However m… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=883) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=885)