# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=831 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 832 --- ## [How to install an optional gem for a bundled plugin](https://discuss.elastic.co/t/how-to-install-an-optional-gem-for-a-bundled-plugin/117803)
**Author:** [@Dheeraj\_Gupta](https://discuss.elastic.co/u/Dheeraj_Gupta)\ **Replies:** 0\ **Last updated:** [January 31, 2018, 12:41pm UTC](https://discuss.elastic.co/t/how-to-install-an-optional-gem-for-a-bundled-plugin/117803 "2018-01-31T12:41:46Z")
Hi, I am trying to get logstash-output-webhdfs to work with kerberos. The documentation states that I need to install the gssapi gem for using the kerberos feature (The plugin works fine without gssapi for authenticatio… --- ## [Logstash doesn't send logs to elasticsearch](https://discuss.elastic.co/t/logstash-doesnt-send-logs-to-elasticsearch/117604)
**Author:** [@User\_2\_Guest\_Incomin](https://discuss.elastic.co/u/User_2_Guest_Incomin)\ **Replies:** 0\ **Last updated:** [January 30, 2018, 11:43am UTC](https://discuss.elastic.co/t/logstash-doesnt-send-logs-to-elasticsearch/117604 "2018-01-30T11:43:11Z")
Hello there, I'm getting troubles with logstash trying to send logs to one of my index, the error seems to be popular but I can't solve it. I have this index with this type: Index: \[imagen\] Type: \[imagen\] And when… --- ## [Errors in while starting Logstash](https://discuss.elastic.co/t/errors-in-while-starting-logstash/117797)
**Author:** [@script007](https://discuss.elastic.co/u/script007)\ **Replies:** 4\ **Last updated:** [January 31, 2018, 12:04pm UTC](https://discuss.elastic.co/t/errors-in-while-starting-logstash/117797 "2018-01-31T12:04:36Z")
I feel something is not right in my conf file. Ant help. I am not able to start this conf file. Conf File: input { file { path =\> "/data/es/snmp/stage/snmp.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/nu… --- ## [Unable to Start NetFlow](https://discuss.elastic.co/t/unable-to-start-netflow/117691)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 1\ **Last updated:** [January 31, 2018, 11:38am UTC](https://discuss.elastic.co/t/unable-to-start-netflow/117691 "2018-01-31T11:38:32Z")
ELK Stack 6.1.1 on Server 2012 VMWare guest, all applications on a single host. All applications are configured to run as services. (Used NSSM for LogStash and Kibana) LogStash heap is set to 8GB ElasticSearch heap is… --- ## [Issue With Index templates](https://discuss.elastic.co/t/issue-with-index-templates/116171)
**Author:** [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Replies:** 9\ **Last updated:** [January 31, 2018, 9:52am UTC](https://discuss.elastic.co/t/issue-with-index-templates/116171 "2018-01-31T09:52:16Z")
Hi All, I have a working pipeline with static ES Index as output . But when i am trying to create index dynamically with index template , then indexes are not created . Below are my configuration , output { stdout {… --- ## [Ruby code calculate problem](https://discuss.elastic.co/t/ruby-code-calculate-problem/117763)
**Author:** [@hsiuming](https://discuss.elastic.co/u/hsiuming)\ **Replies:** 2\ **Last updated:** [January 31, 2018, 8:31am UTC](https://discuss.elastic.co/t/ruby-code-calculate-problem/117763 "2018-01-31T08:31:01Z")
Hello,we now want to do Arithmetic in logstash ruby code. The ruby code is: ruby { init =\> "@@count = -1" code =\> " event.set('\[test\]', @@count) event.set('\[test2\]', event.get('\[layers\]\[frame\_time\_relative\]').to\_f … --- ## [How to provide default values in case of grok parse failure?](https://discuss.elastic.co/t/how-to-provide-default-values-in-case-of-grok-parse-failure/117737)
**Author:** [@jainbhavya53](https://discuss.elastic.co/u/jainbhavya53)\ **Replies:** 5\ **Last updated:** [January 31, 2018, 8:18am UTC](https://discuss.elastic.co/t/how-to-provide-default-values-in-case-of-grok-parse-failure/117737 "2018-01-31T08:18:44Z")
I am using grok for parsing logs.My requirement is to set default value for any missing string field as null and for any integer field as 0. Sample logs are as follows:-- Jan 23 12:16:52 \[10.10.10.10\] \<13\> 10.11.12.13 … --- ## [Lgstash is not sending data](https://discuss.elastic.co/t/lgstash-is-not-sending-data/117739)
**Author:** [@Vigneshprasanna](https://discuss.elastic.co/u/Vigneshprasanna)\ **Replies:** 5\ **Last updated:** [January 31, 2018, 8:01am UTC](https://discuss.elastic.co/t/lgstash-is-not-sending-data/117739 "2018-01-31T08:01:39Z")
Sending Logstash's logs to D:/Personal/logstash-6.1.2/logs which is now configured via log4j2.properties \[2018-01-31T10:24:36,576\]\[INFO \]\[logstash.modules.scaffold\] Initializing module{:module\_name=\>"fb\_apache",:directo… --- ## [Configuration file to read xls](https://discuss.elastic.co/t/configuration-file-to-read-xls/117715)
**Author:** [@Errors](https://discuss.elastic.co/u/Errors)\ **Replies:** 1\ **Last updated:** [January 31, 2018, 7:01am UTC](https://discuss.elastic.co/t/configuration-file-to-read-xls/117715 "2018-01-31T07:01:26Z")
I've downloaded and installed the logstash-input-xls plugin. I'm wondering if there's now some sort of guide/documentation or something to follow which shows how to create a config file to read in xls and push it to elas… --- ## [Logstash not parsing the stacktrace into JSON format](https://discuss.elastic.co/t/logstash-not-parsing-the-stacktrace-into-json-format/117742)
**Author:** [@niks\_23](https://discuss.elastic.co/u/niks_23)\ **Replies:** 0\ **Last updated:** [January 31, 2018, 6:14am UTC](https://discuss.elastic.co/t/logstash-not-parsing-the-stacktrace-into-json-format/117742 "2018-01-31T06:14:02Z")
I have got two questions here. Can we configure such a way that the logs generated by the application are parsed without giving input to log statsh and save as JSON format into the logs. Here we can have only one fil… --- ## ["type" : "index\_not\_found\_exception"](https://discuss.elastic.co/t/type-index-not-found-exception/117582)
**Author:** [@Vigneshprasanna](https://discuss.elastic.co/u/Vigneshprasanna)\ **Replies:** 8\ **Last updated:** [January 31, 2018, 5:27am UTC](https://discuss.elastic.co/t/type-index-not-found-exception/117582 "2018-01-31T05:27:28Z")
Hi, In this I'm trying to test ELK using a jboss log file placed in a folder I'm done with both ES and kibana. the logstash part I'm not able to configer i have placed the below routine inside a .conf file of config fol… --- ## [Ealsticsearch doucment\_type](https://discuss.elastic.co/t/ealsticsearch-doucment-type/117586)
**Author:** [@tishion](https://discuss.elastic.co/u/tishion)\ **Replies:** 5\ **Last updated:** [January 31, 2018, 3:08am UTC](https://discuss.elastic.co/t/ealsticsearch-doucment-type/117586 "2018-01-31T03:08:50Z")
document\_type edit Value type is string There is no default value for this setting. This option is deprecated Note: This option is deprecated due to the removal of types in Logstash 6.0. It will be removed in the nex… --- ## [Addtag based on which match expression was used?](https://discuss.elastic.co/t/addtag-based-on-which-match-expression-was-used/115769)
**Author:** [@nnet](https://discuss.elastic.co/u/nnet)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 6:07pm UTC](https://discuss.elastic.co/t/addtag-based-on-which-match-expression-was-used/115769 "2018-01-16T18:07:37Z")
Hi folks, My match statement has an array of expressions. How can I addtag based on which expression was used? Thanks --- ## [Mapping Issues With GeoIP](https://discuss.elastic.co/t/mapping-issues-with-geoip/117490)
**Author:** [@clannadqs](https://discuss.elastic.co/u/clannadqs)\ **Replies:** 5\ **Last updated:** [January 30, 2018, 5:36pm UTC](https://discuss.elastic.co/t/mapping-issues-with-geoip/117490 "2018-01-30T17:36:42Z")
I'm at a complete loss. I'm using 5.6 and literally nothing I do changes how coordinates get mapped. They seem to be mapping automatically to whatever elastic thinks it is while entirely ignoring my mapping. My mapping … --- ## [Problems to parse "/" character](https://discuss.elastic.co/t/problems-to-parse-character/117615)
**Author:** [@lucasbiccio](https://discuss.elastic.co/u/lucasbiccio)\ **Replies:** 2\ **Last updated:** [January 30, 2018, 5:09pm UTC](https://discuss.elastic.co/t/problems-to-parse-character/117615 "2018-01-30T17:09:33Z")
Hi all, I'm using logstash to parse a FIX string message into a index data tag by tag. Check the following configuration: if "fix" in \[tags\] { grok { match =\> { "message" =\> "%{GREEDYDATA:direction} %{GREEDYDATA:t… --- ## [Filebeats/Kafka vs File Input - Codec Used](https://discuss.elastic.co/t/filebeats-kafka-vs-file-input-codec-used/117668)
**Author:** [@jalcock](https://discuss.elastic.co/u/jalcock)\ **Replies:** 0\ **Last updated:** [January 30, 2018, 4:56pm UTC](https://discuss.elastic.co/t/filebeats-kafka-vs-file-input-codec-used/117668 "2018-01-30T16:56:56Z")
I am attempting to collect messages on Windows servers using Filebeats, send them to Kafka, ingest/parse in Logstash, and then distribute to a third-party destination (Log Insight) using the Logstash output plugin for Lo… --- ## [Logstash - Syslog input and output](https://discuss.elastic.co/t/logstash-syslog-input-and-output/117647)
**Author:** [@drayke\_jackson](https://discuss.elastic.co/u/drayke_jackson)\ **Replies:** 0\ **Last updated:** [January 30, 2018, 3:10pm UTC](https://discuss.elastic.co/t/logstash-syslog-input-and-output/117647 "2018-01-30T15:10:08Z")
Hello everyone, I am trying to send multiple sources to logstash, filter them by hostname and/or source ip and output them via syslog. I need the source ip to be retained. Does anyone have experience with this? So far my… --- ## [Read log from local file](https://discuss.elastic.co/t/read-log-from-local-file/117628)
**Author:** [@Vigneshprasanna](https://discuss.elastic.co/u/Vigneshprasanna)\ **Replies:** 1\ **Last updated:** [January 30, 2018, 2:12pm UTC](https://discuss.elastic.co/t/read-log-from-local-file/117628 "2018-01-30T14:12:07Z")
Im trying to read logs from a local folder in my system. I get the following error D:\\Personal\\logstash-6.1.2\\bin\>logstash.bat -f ..\\config\\logstash.confSending Logstash's logs to D:/Personal/logstash-6.1.2/logs which i… --- ## [Webhdfs output not replacing placeholders for empty value](https://discuss.elastic.co/t/webhdfs-output-not-replacing-placeholders-for-empty-value/117626)
**Author:** [@parthkdas](https://discuss.elastic.co/u/parthkdas)\ **Replies:** 0\ **Last updated:** [January 30, 2018, 1:32pm UTC](https://discuss.elastic.co/t/webhdfs-output-not-replacing-placeholders-for-empty-value/117626 "2018-01-30T13:32:59Z")
my logstash configuration working.But the putput is not placeholders with empty values. sample input : 0216042000018087881|ELECTRONIC WAYS|2017-12-16|8100.00||4812|Telecom||Primary Actual output : 0216042000018087881… --- ## [Input plugin to handle server sent events](https://discuss.elastic.co/t/input-plugin-to-handle-server-sent-events/117111)
**Author:** [@charekc](https://discuss.elastic.co/u/charekc)\ **Replies:** 2\ **Last updated:** [January 30, 2018, 1:06pm UTC](https://discuss.elastic.co/t/input-plugin-to-handle-server-sent-events/117111 "2018-01-30T13:06:13Z")
Hi can the existing http poller be used to receive server sent events? I was looking for a plugin/discussion on this but have not found anything. My goal is to read events from the Marathon event bus (https://mesosphere… --- ## [Add data to aggregate from a log line with no specific task\_id](https://discuss.elastic.co/t/add-data-to-aggregate-from-a-log-line-with-no-specific-task-id/117618)
**Author:** [@coda](https://discuss.elastic.co/u/coda)\ **Replies:** 0\ **Last updated:** [January 30, 2018, 12:39pm UTC](https://discuss.elastic.co/t/add-data-to-aggregate-from-a-log-line-with-no-specific-task-id/117618 "2018-01-30T12:39:06Z")
Excuse me everyone, Given the following events: event1 = { "queueid" =\> "ABC1234", "process" =\> "qmgr", "from" =\> "sender@domain.com" } event2 = { "queue\_id" =\> "ABC1234", "process" =\> "smtp", "to" =\> "rec… --- ## [Error with Cassandra and jdbc\_paging\_enabled =\> true](https://discuss.elastic.co/t/error-with-cassandra-and-jdbc-paging-enabled-true/117616)
**Author:** [@nomedx](https://discuss.elastic.co/u/nomedx)\ **Replies:** 0\ **Last updated:** [January 30, 2018, 12:37pm UTC](https://discuss.elastic.co/t/error-with-cassandra-and-jdbc-paging-enabled-true/117616 "2018-01-30T12:37:05Z")
Hi, I use Logstash with jdbc plugin and Cassandra. Elasticsearch: 6.1.2 logstash-input-jdbc: 4.3.3 Cassandra: 3.11.1 Jdbc driver: dbschema.CassandraJdbcDriver This is my config: input { jdbc { clean\_run =\> "fal… --- ## [Logstash + xpack Authentication failure 401](https://discuss.elastic.co/t/logstash-xpack-authentication-failure-401/117605)
**Author:** [@A.Klos](https://discuss.elastic.co/u/A.Klos)\ **Replies:** 2\ **Last updated:** [January 30, 2018, 11:46am UTC](https://discuss.elastic.co/t/logstash-xpack-authentication-failure-401/117605 "2018-01-30T11:46:48Z")
Hi, I installed x-pack on my 5.6.6 ELK Stack and after logstash has following errors in log: \[2018-01-30T12:26:43,208\]\[WARN \]\[logstash.outputs.elasticsearch\] Attempted to resurrect connection to dead ES instance, but g… --- ## [Can't using Geoip Apache2](https://discuss.elastic.co/t/cant-using-geoip-apache2/117588)
**Author:** [@Vi\_t\_Dung\_Vu](https://discuss.elastic.co/u/Vi_t_Dung_Vu)\ **Replies:** 6\ **Last updated:** [January 30, 2018, 9:58am UTC](https://discuss.elastic.co/t/cant-using-geoip-apache2/117588 "2018-01-30T09:58:38Z")
I can't found apache2.access.geoip.location in Management \>\> Index Patterns \>\> fields This is my filer file \> filter { if \[type\] == "apache\_access" { grok { match =\> { "message" =\> \["%{IPORHOST:\[apa… --- ## [Logstash xml filter with xpath with multiple fields query](https://discuss.elastic.co/t/logstash-xml-filter-with-xpath-with-multiple-fields-query/116185)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 2\ **Last updated:** [January 30, 2018, 9:55am UTC](https://discuss.elastic.co/t/logstash-xml-filter-with-xpath-with-multiple-fields-query/116185 "2018-01-30T09:55:57Z")
I have an xml file, which contains fields like: With xpath, within xml filter, I get the fields returned as below, with each element in its array: "PhaseName" =\> \[ \[ 0\] "CRITICAL", \[ 1\] "TEST", \[ 2\] "OTH… --- ## [Logstash.runner Fatal error TypeError: illegal access on 'getThreadCount'](https://discuss.elastic.co/t/logstash-runner-fatal-error-typeerror-illegal-access-on-getthreadcount/116674)
**Author:** [@sujalkpatel](https://discuss.elastic.co/u/sujalkpatel)\ **Replies:** 3\ **Last updated:** [January 30, 2018, 8:49am UTC](https://discuss.elastic.co/t/logstash-runner-fatal-error-typeerror-illegal-access-on-getthreadcount/116674 "2018-01-30T08:49:29Z")
I am running LogStash for very first time. So I have no idea what's going on. Please help someone. I appreciate it in advance. Thank you. \[2018-01-23T05:04:22,719\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! … --- ## [Multiple pipelines questions](https://discuss.elastic.co/t/multiple-pipelines-questions/117571)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 6\ **Last updated:** [January 30, 2018, 8:14am UTC](https://discuss.elastic.co/t/multiple-pipelines-questions/117571 "2018-01-30T08:14:35Z")
Hi, currently we are only using the main pipeline. Processing of different types of logfiles are separated by conditionals. I have a few questions about using multiple pipelines: Can both pipelines use the same beats… --- ## [Why do messages and Error stack trace do not recognize new line charecters](https://discuss.elastic.co/t/why-do-messages-and-error-stack-trace-do-not-recognize-new-line-charecters/117346)
**Author:** [@SawyerFord](https://discuss.elastic.co/u/SawyerFord)\ **Replies:** 4\ **Last updated:** [January 30, 2018, 7:14am UTC](https://discuss.elastic.co/t/why-do-messages-and-error-stack-trace-do-not-recognize-new-line-charecters/117346 "2018-01-30T07:14:30Z")
I would like to know why new lines (\\n) are available in the strack trace and why are not getting displayed in a new line. What settings I have to make such that the error stack trace appear with new line as traditional … --- ## [Help with parsing binary flat file logs using logstash](https://discuss.elastic.co/t/help-with-parsing-binary-flat-file-logs-using-logstash/117570)
**Author:** [@abylay313](https://discuss.elastic.co/u/abylay313)\ **Replies:** 0\ **Last updated:** [January 30, 2018, 7:14am UTC](https://discuss.elastic.co/t/help-with-parsing-binary-flat-file-logs-using-logstash/117570 "2018-01-30T07:14:12Z")
Hi all, I have binary logs generated from HMI(Human Machine Interface) which I need to index to elasticsearch. However I can't figure out how to set up my config file and whether I need to use a specific plug in. Below… --- ## [How to use ruby script file with parameter?](https://discuss.elastic.co/t/how-to-use-ruby-script-file-with-parameter/117537)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 4\ **Last updated:** [January 30, 2018, 7:14am UTC](https://discuss.elastic.co/t/how-to-use-ruby-script-file-with-parameter/117537 "2018-01-30T07:14:09Z")
Hi, sometimes I used ruby as logstash filter plugin - only the inline functionality until now. Additionally I have to admit, that I am not really fond of ruby. Now I wanna outsource the code to a script file, but I do … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=830) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=832)