# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=888 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 889 --- ## [Which Data type to be used for time field](https://discuss.elastic.co/t/which-data-type-to-be-used-for-time-field/104676)
**Author:** [@Joeltstephen](https://discuss.elastic.co/u/Joeltstephen)\ **Replies:** 1\ **Last updated:** [October 23, 2017, 5:18am UTC](https://discuss.elastic.co/t/which-data-type-to-be-used-for-time-field/104676 "2017-10-23T05:18:05Z")
Hi Team, My log file contains time field which has data like 0.3354353 but using data type float in my configuration file, it is parsing only last 2 digit after dot. What data type should to parse the total data in t… --- ## [When i using the user-defined pattern it doesn't work](https://discuss.elastic.co/t/when-i-using-the-user-defined-pattern-it-doesnt-work/104375)
**Author:** [@antony.y](https://discuss.elastic.co/u/antony.y)\ **Replies:** 6\ **Last updated:** [October 23, 2017, 3:27am UTC](https://discuss.elastic.co/t/when-i-using-the-user-defined-pattern-it-doesnt-work/104375 "2017-10-23T03:27:32Z")
i am now using the pattern as below \<%{NUMBER:message\_type\_id}\>%{NUMBER:internal\_id}:%{SPACE}%{SYSLOGHOST:origin\_hostname}:%{unp:hahha} and i defined unp as below unp ^\\s{1}\]$ When i input: \<187\>11497: PH18L1-CS29S… --- ## [Logstash not running using systemd due to permission errors](https://discuss.elastic.co/t/logstash-not-running-using-systemd-due-to-permission-errors/104757)
**Author:** [@lemongrab](https://discuss.elastic.co/u/lemongrab)\ **Replies:** 0\ **Last updated:** [October 21, 2017, 2:24am UTC](https://discuss.elastic.co/t/logstash-not-running-using-systemd-due-to-permission-errors/104757 "2017-10-21T02:24:14Z")
Hello. I installed logstash on archlinux and I'm having trouble with running it using systemd. I have kibana and elasticsearch up and running just fine as a service, however. The bug is reported here as well: https://b… --- ## [Grokking the Linux authorization logs](https://discuss.elastic.co/t/grokking-the-linux-authorization-logs/104467)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 3\ **Last updated:** [October 22, 2017, 1:59am UTC](https://discuss.elastic.co/t/grokking-the-linux-authorization-logs/104467 "2017-10-22T01:59:09Z")
Attempting to use a syslog pattern to parse ubuntu logs for building auth reports but having issues with syslog-filter.conf file. Below are the config files and error message: Error message: \[2017-10-18T20:34:18,093\]\[… --- ## [Stop config file of logstash](https://discuss.elastic.co/t/stop-config-file-of-logstash/104665)
**Author:** [@parsa](https://discuss.elastic.co/u/parsa)\ **Replies:** 1\ **Last updated:** [October 21, 2017, 8:26pm UTC](https://discuss.elastic.co/t/stop-config-file-of-logstash/104665 "2017-10-21T20:26:44Z")
How can I stop all instances of the conf file that ran by logstash ? --- ## [Measure logstash pipeline throughput performance - which counters and how to use them](https://discuss.elastic.co/t/measure-logstash-pipeline-throughput-performance-which-counters-and-how-to-use-them/104761)
**Author:** [@trondhindenes](https://discuss.elastic.co/u/trondhindenes)\ **Replies:** 0\ **Last updated:** [October 21, 2017, 11:25am UTC](https://discuss.elastic.co/t/measure-logstash-pipeline-throughput-performance-which-counters-and-how-to-use-them/104761 "2017-10-21T11:25:54Z")
Hi, We're implementing Logstash 5.6.3 (upgrading from ELK 2.x) and now that we have access to logstash's rest endpoints I'd like some rudimentary monitoring of how our logstash nodes perform. Theres a ton of stuff to c… --- ## [Parse path with file input in logstash](https://discuss.elastic.co/t/parse-path-with-file-input-in-logstash/104735)
**Author:** [@bm1391](https://discuss.elastic.co/u/bm1391)\ **Replies:** 2\ **Last updated:** [October 21, 2017, 9:19am UTC](https://discuss.elastic.co/t/parse-path-with-file-input-in-logstash/104735 "2017-10-21T09:19:48Z")
Im throwing in some suricata logs through logstash and this works fine, but I'm wondering if there is a better way to separate and tag the log files... \> input { \> file { \> path =\> "/home/user/nfs/suricata/bronn/e… --- ## [Logstash output plugin kafka is not pushing data into kafka](https://discuss.elastic.co/t/logstash-output-plugin-kafka-is-not-pushing-data-into-kafka/104383)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 3\ **Last updated:** [October 21, 2017, 6:42am UTC](https://discuss.elastic.co/t/logstash-output-plugin-kafka-is-not-pushing-data-into-kafka/104383 "2017-10-21T06:42:20Z")
Hi all here i am trying to push Elasticsearch data into kafka input { elasticsearch { hosts =\> "192.168.1.59:9233" index =\> "titanic1" # user =\> "elastic" # password =\> "changeme" # query =\> '{… --- ## [Gzip\_lines codec not working](https://discuss.elastic.co/t/gzip-lines-codec-not-working/104752)
**Author:** [@nairu](https://discuss.elastic.co/u/nairu)\ **Replies:** 0\ **Last updated:** [October 20, 2017, 8:39pm UTC](https://discuss.elastic.co/t/gzip-lines-codec-not-working/104752 "2017-10-20T20:39:44Z")
I'm using logstash(5.6.0) to read .gz file(has a list of .dat files) by using gzip\_lines codec. It appears the compressed file is not being read but I do not see any errors in the logstash logs and indexes in ElasticSear… --- ## [Http\_poller fails on subsets](https://discuss.elastic.co/t/http-poller-fails-on-subsets/103562)
**Author:** [@Callahan](https://discuss.elastic.co/u/Callahan)\ **Replies:** 3\ **Last updated:** [October 20, 2017, 6:55pm UTC](https://discuss.elastic.co/t/http-poller-fails-on-subsets/103562 "2017-10-20T18:55:01Z")
Hi, So I'm trying to pull a json response from a site. The response has now changed as the API has been upgraded. Whereas before, the tickets would be returned as the sample shown below: \[ { "account\_id": 1… --- ## [\_grokparsefailure in tags. Not able to change the format of the log](https://discuss.elastic.co/t/grokparsefailure-in-tags-not-able-to-change-the-format-of-the-log/104731)
**Author:** [@abhinav](https://discuss.elastic.co/u/abhinav)\ **Replies:** 1\ **Last updated:** [October 20, 2017, 6:54pm UTC](https://discuss.elastic.co/t/grokparsefailure-in-tags-not-able-to-change-the-format-of-the-log/104731 "2017-10-20T18:54:16Z")
HI team, Your help is greatly appreciated. I am stuck with this issue from two days. I am dumping the logs through a cloud floundary on port 5000 and my log stash is listening to 5000. I have a simple filter in my conf… --- ## [Logstash generating exception in pipeline worker](https://discuss.elastic.co/t/logstash-generating-exception-in-pipeline-worker/104617)
**Author:** [@kenny123](https://discuss.elastic.co/u/kenny123)\ **Replies:** 2\ **Last updated:** [October 20, 2017, 6:40pm UTC](https://discuss.elastic.co/t/logstash-generating-exception-in-pipeline-worker/104617 "2017-10-20T18:40:09Z")
Hi , when i run the logstash in the debug mode, it is generating error as the below.Can i please know whether the error is due to the wrong filter or not using of persistent queues. Exception in pipelineworker, the pi… --- ## [Grok multiline parse failures](https://discuss.elastic.co/t/grok-multiline-parse-failures/104607)
**Author:** [@Jordan160](https://discuss.elastic.co/u/Jordan160)\ **Replies:** 6\ **Last updated:** [October 20, 2017, 3:58pm UTC](https://discuss.elastic.co/t/grok-multiline-parse-failures/104607 "2017-10-20T15:58:39Z")
Hello, I am fairly new to ELK stack and currently am having issues with GROK filter. My pattern matches when using http://grokconstructor.appspot.com, but it still seems to be breaking on a particular log pattern that … --- ## [Removing fields from logstash output](https://discuss.elastic.co/t/removing-fields-from-logstash-output/103903)
**Author:** [@s.essa](https://discuss.elastic.co/u/s.essa)\ **Replies:** 3\ **Last updated:** [October 20, 2017, 3:28pm UTC](https://discuss.elastic.co/t/removing-fields-from-logstash-output/103903 "2017-10-20T15:28:26Z")
Hello everyone, I'm new to LogStash and so I'm running into complications. I'm trying to use logstash to forward logs but the logs that are sent are completely different from the logs I expect. When I use the syslog out… --- ## [Convert lambert II to wgs84](https://discuss.elastic.co/t/convert-lambert-ii-to-wgs84/104707)
**Author:** [@Ala\_t](https://discuss.elastic.co/u/Ala_t)\ **Replies:** 0\ **Last updated:** [October 20, 2017, 1:33pm UTC](https://discuss.elastic.co/t/convert-lambert-ii-to-wgs84/104707 "2017-10-20T13:33:13Z")
Hi, how can i use ruby code to convert lambertII to wgs84 ?, in fact i installed this library https://github.com/yageek/lambert-ruby#usage and I verified that this library is installed because I found it under the folde… --- ## [Logstash.err](https://discuss.elastic.co/t/logstash-err/104695)
**Author:** [@wandile](https://discuss.elastic.co/u/wandile)\ **Replies:** 0\ **Last updated:** [October 20, 2017, 12:07pm UTC](https://discuss.elastic.co/t/logstash-err/104695 "2017-10-20T12:07:56Z")
Hi i tried to less the logstash.err file after installing the logstash in the ELK server and i receive and end response doesn't show if it has loaded or not and the security port doesn't show on netstart -NTLP --- ## [Cannot restart logstash](https://discuss.elastic.co/t/cannot-restart-logstash/104508)
**Author:** [@neal1991](https://discuss.elastic.co/u/neal1991)\ **Replies:** 4\ **Last updated:** [October 20, 2017, 11:55am UTC](https://discuss.elastic.co/t/cannot-restart-logstash/104508 "2017-10-20T11:55:51Z")
I am config a logstash to obtains logs from a client. But now there is a problem with logstah. There is error information in /var/log/logstash: logstash.agent cannot create pipeline{reson:"Expected one of #, input, filt… --- ## [Split up indices based on? tags?](https://discuss.elastic.co/t/split-up-indices-based-on-tags/104667)
**Author:** [@Maekee](https://discuss.elastic.co/u/Maekee)\ **Replies:** 1\ **Last updated:** [October 20, 2017, 11:43am UTC](https://discuss.elastic.co/t/split-up-indices-based-on-tags/104667 "2017-10-20T11:43:31Z")
Hello fellow logstash peeps, At the moment i am using winlogbeat agents to collect logs/documents to Logstash which sends it to ElasticSearch. My logstash config looks like this: input { beats { port =\> 5044 type =\>… --- ## [Logstash unable to parse fully big csv file](https://discuss.elastic.co/t/logstash-unable-to-parse-fully-big-csv-file/104684)
**Author:** [@surajiiitm](https://discuss.elastic.co/u/surajiiitm)\ **Replies:** 0\ **Last updated:** [October 20, 2017, 10:39am UTC](https://discuss.elastic.co/t/logstash-unable-to-parse-fully-big-csv-file/104684 "2017-10-20T10:39:29Z")
Hi, I am parsing the csv data using logstash to elasticsearch but it parse only 340 lines out of 1500 lines and logstash shows the parsed lines in output but then logstash remain idle and nothing is being displayed and … --- ## [An unknown error occurred sending a bulk request](https://discuss.elastic.co/t/an-unknown-error-occurred-sending-a-bulk-request/104677)
**Author:** [@lmalossi](https://discuss.elastic.co/u/lmalossi)\ **Replies:** 0\ **Last updated:** [October 20, 2017, 9:02am UTC](https://discuss.elastic.co/t/an-unknown-error-occurred-sending-a-bulk-request/104677 "2017-10-20T09:02:10Z")
Hi, after upgrading logstash to 5.6.3 I'm getting the following error. I'm using JDBC Plugin \[2017-10-20T10:43:10,868\]\[ERROR\]\[logstash.outputs.elasticsearch\] An unknown error occurred sending a bulk request to Elastic… --- ## [Logstash convert Nil values into a number](https://discuss.elastic.co/t/logstash-convert-nil-values-into-a-number/104271)
**Author:** [@Pinheiro](https://discuss.elastic.co/u/Pinheiro)\ **Replies:** 10\ **Last updated:** [October 20, 2017, 8:46am UTC](https://discuss.elastic.co/t/logstash-convert-nil-values-into-a-number/104271 "2017-10-20T08:46:11Z")
Hello everyone, I'm trying to find the fields that are empty and change them for a value. I want to do something like the next expression, but Logstash can not find the nil value. Does anyone have a suggestion for find… --- ## [XDELETE and now unable to create index pattern](https://discuss.elastic.co/t/xdelete-and-now-unable-to-create-index-pattern/104601)
**Author:** [@Slop](https://discuss.elastic.co/u/Slop)\ **Replies:** 4\ **Last updated:** [October 20, 2017, 6:41am UTC](https://discuss.elastic.co/t/xdelete-and-now-unable-to-create-index-pattern/104601 "2017-10-20T06:41:49Z")
Hello, I'm here because after having developped and clear all my grok filter on logstash, to have something very clear .. I've done a : curl -XDELETE 'localhost:9200/\_all' Then a lot of errors appeared and I've decide… --- ## [Logstash add\_field not working](https://discuss.elastic.co/t/logstash-add-field-not-working/104510)
**Author:** [@VidyVideni](https://discuss.elastic.co/u/VidyVideni)\ **Replies:** 2\ **Last updated:** [October 20, 2017, 3:23am UTC](https://discuss.elastic.co/t/logstash-add-field-not-working/104510 "2017-10-20T03:23:45Z")
I add a field named request using following filter, however it doesn't use url\_path and 'query\_string' values, I tried to change logstash config as "request" =\> "%{\[uri\_path\]}%{\[query\_string\]}" why the add\_f… --- ## [Logstash plugins rubygems ssl errors](https://discuss.elastic.co/t/logstash-plugins-rubygems-ssl-errors/104635)
**Author:** [@mathurin68](https://discuss.elastic.co/u/mathurin68)\ **Replies:** 2\ **Last updated:** [October 20, 2017, 3:03am UTC](https://discuss.elastic.co/t/logstash-plugins-rubygems-ssl-errors/104635 "2017-10-20T03:03:14Z")
Anyone else had this problem? /usr/share/logstash/bin/logstash-plugin install logstash-filter-geoip Validating logstash-filter-geoip Unable to download data from https://rubygems.org - certificate verify failed (https… --- ## [NoMethodError: "undefined method 'to\_hash'" with CSV input in Logstash 5.6.2](https://discuss.elastic.co/t/nomethoderror-undefined-method-to-hash-with-csv-input-in-logstash-5-6-2/103545)
**Author:** [@thechauffeur](https://discuss.elastic.co/u/thechauffeur)\ **Replies:** 11\ **Last updated:** [October 20, 2017, 1:17am UTC](https://discuss.elastic.co/t/nomethoderror-undefined-method-to-hash-with-csv-input-in-logstash-5-6-2/103545 "2017-10-20T01:17:41Z")
I'm trying to send data from a .csv file containing JMeter results to an online Elasticsearch instance. My .conf file looks this way: input { file { path =\> \["\\path\\to\\somefile.csv"\] start\_position =\> "beginning" … --- ## [Keep newline formating with multiline](https://discuss.elastic.co/t/keep-newline-formating-with-multiline/104335)
**Author:** [@pluto](https://discuss.elastic.co/u/pluto)\ **Replies:** 3\ **Last updated:** [October 19, 2017, 11:48pm UTC](https://discuss.elastic.co/t/keep-newline-formating-with-multiline/104335 "2017-10-19T23:48:26Z")
Here is my config file. I am processing java logs. The errors do come into Kibana but are all one long chunk of text. Is there a way to have the entry come in as one event but keep the formating so it is more readable? i… --- ## [Logstash to Elasticsearch over SSL Configuration](https://discuss.elastic.co/t/logstash-to-elasticsearch-over-ssl-configuration/103950)
**Author:** [@cm47](https://discuss.elastic.co/u/cm47)\ **Replies:** 2\ **Last updated:** [October 19, 2017, 9:49pm UTC](https://discuss.elastic.co/t/logstash-to-elasticsearch-over-ssl-configuration/103950 "2017-10-19T21:49:43Z")
Hi, I am trying to achieve what is described in the title. I believe I have taken all steps for proper configuration but am receiving an error that seems like a connection is not being initiated through https. Logstash … --- ## [Avro plugin logstash issue](https://discuss.elastic.co/t/avro-plugin-logstash-issue/104615)
**Author:** [@venkat\_venkat](https://discuss.elastic.co/u/venkat_venkat)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 8:20pm UTC](https://discuss.elastic.co/t/avro-plugin-logstash-issue/104615 "2017-10-19T20:20:31Z")
I have installed logstash avro plugin "bin/logstash-plugin install logstash-codec-avro" and I am using below config input { kafka { topics =\> 'cdn' bootstrap\_servers =\> 'XXXXXXXX:9092,XXXXXXXXXXX:9092,XXXXXXXXXX:9092… --- ## [Help needed with OpenLDAP Logstash filter](https://discuss.elastic.co/t/help-needed-with-openldap-logstash-filter/104605)
**Author:** [@pdna](https://discuss.elastic.co/u/pdna)\ **Replies:** 1\ **Last updated:** [October 19, 2017, 7:12pm UTC](https://discuss.elastic.co/t/help-needed-with-openldap-logstash-filter/104605 "2017-10-19T19:12:31Z")
I have the following logstash.conf file that I have taken from github input { file { path =\> "/var/log/openldap.log" start\_position =\> beginning type =\> "openldap\_log\_file" } } filter { grok { … --- ## [Help with tomcat grok pattern](https://discuss.elastic.co/t/help-with-tomcat-grok-pattern/104551)
**Author:** [@sbampa](https://discuss.elastic.co/u/sbampa)\ **Replies:** 3\ **Last updated:** [October 19, 2017, 6:49pm UTC](https://discuss.elastic.co/t/help-with-tomcat-grok-pattern/104551 "2017-10-19T18:49:25Z")
Hi all, i have the following tomcat logs: 04-Oct-2017 14:27:51.457 INFO \[main\] org.apache.catalina.startup.VersionLoggerListener.log Server number: 8.5.8.0 04-Oct-2017 14:27:51.457 INFO \[main\] org.apache.catalina.star… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=887) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=889)