# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=762 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 763 --- ## [Grok filtering](https://discuss.elastic.co/t/grok-filtering/131450)
**Author:** [@SteveBaker](https://discuss.elastic.co/u/SteveBaker)\ **Replies:** 0\ **Last updated:** [May 11, 2018, 9:46am UTC](https://discuss.elastic.co/t/grok-filtering/131450 "2018-05-11T09:46:32Z")
Hi all, I'm completely new to Grok and have been tasked with writing a seemingly simple grok filter to identify whether our servers are in sync or not. In short, every day each server runs a database sync check and pump… --- ## [Logstash skips the last line of build.xml file](https://discuss.elastic.co/t/logstash-skips-the-last-line-of-build-xml-file/131099)
**Author:** [@raja\_gopal](https://discuss.elastic.co/u/raja_gopal)\ **Replies:** 2\ **Last updated:** [May 11, 2018, 9:30am UTC](https://discuss.elastic.co/t/logstash-skips-the-last-line-of-build-xml-file/131099 "2018-05-11T09:30:09Z")
Hello All, I am trying to read build.xml file from my jenkins server into elasticsearch through logstash. I have used file input plugin along with multiline codec but what happens is logstash does not read the xml line … --- ## [Nested Field Different Data Types](https://discuss.elastic.co/t/nested-field-different-data-types/131422)
**Author:** [@Johnnycc1](https://discuss.elastic.co/u/Johnnycc1)\ **Replies:** 2\ **Last updated:** [May 11, 2018, 9:03am UTC](https://discuss.elastic.co/t/nested-field-different-data-types/131422 "2018-05-11T09:03:11Z")
Heya, I have a source document that fails to load into elastic search. The reason is that the source document contains text in a nested field that should be of type boolean. I would like to process this log in logstash… --- ## [How to use ruby code to index some variable element from XML node](https://discuss.elastic.co/t/how-to-use-ruby-code-to-index-some-variable-element-from-xml-node/131429)
**Author:** [@terrymu](https://discuss.elastic.co/u/terrymu)\ **Replies:** 1\ **Last updated:** [May 11, 2018, 8:26am UTC](https://discuss.elastic.co/t/how-to-use-ruby-code-to-index-some-variable-element-from-xml-node/131429 "2018-05-11T08:26:56Z")
Hi expert, I have used the logstash to parse some information with XML format, such like this: \ \ \ **Author:** [@Balamurali](https://discuss.elastic.co/u/Balamurali)\ **Replies:** 0\ **Last updated:** [May 11, 2018, 7:48am UTC](https://discuss.elastic.co/t/not-getting-logs-due-to-i-o-error/131426 "2018-05-11T07:48:24Z") Hi I'm currently using Logstash for jboss log management. Configured one filebeat instance in every application server which will send logs to one logstash (for all filebeat) Need to get logs from more than 10 jboss n… --- ## [Netflow index file size is not growing](https://discuss.elastic.co/t/netflow-index-file-size-is-not-growing/131393)
**Author:** [@venki522](https://discuss.elastic.co/u/venki522)\ **Replies:** 0\ **Last updated:** [May 11, 2018, 5:01am UTC](https://discuss.elastic.co/t/netflow-index-file-size-is-not-growing/131393 "2018-05-11T05:01:24Z")
iam running logstash 6.\* and i ran the below command and i started to see the netflow data on kibana and my file size of index in elastic search is growing.But after some time lets say 20 minutes i dont see the data on … --- ## [Using one logstash to monitor log with multiple rules](https://discuss.elastic.co/t/using-one-logstash-to-monitor-log-with-multiple-rules/131382)
**Author:** [@hfismyangel](https://discuss.elastic.co/u/hfismyangel)\ **Replies:** 1\ **Last updated:** [May 11, 2018, 4:56am UTC](https://discuss.elastic.co/t/using-one-logstash-to-monitor-log-with-multiple-rules/131382 "2018-05-11T04:56:48Z")
I want to monitor all log of a web server with one logstash.but this log has multiple rules. Now I use the ruby plug-in to match different log rules and output json.Then the json goes into elasticSearch，because of the j… --- ## [Timezone Problem](https://discuss.elastic.co/t/timezone-problem/131341)
**Author:** [@Paulo\_Montanha](https://discuss.elastic.co/u/Paulo_Montanha)\ **Replies:** 3\ **Last updated:** [May 10, 2018, 10:42pm UTC](https://discuss.elastic.co/t/timezone-problem/131341 "2018-05-10T22:42:00Z")
Hi everyone, I need to send data between elasticsearch(input) and s3 bucket(output) - not so complicated. We are using this configuration in logstash: Scenario: logstash file used to export data from elasticsearch and… --- ## [Another custom log parsing](https://discuss.elastic.co/t/another-custom-log-parsing/131054)
**Author:** [@kondakov](https://discuss.elastic.co/u/kondakov)\ **Replies:** 7\ **Last updated:** [May 10, 2018, 5:23pm UTC](https://discuss.elastic.co/t/another-custom-log-parsing/131054 "2018-05-10T17:23:08Z")
I have slightly modified Apache log - LogFormat "%h %l %u %t %{Host}i "%r" %\>s %b "%{Referer}i" "%{User-Agent}i" "%{nsdr}C"" combined LogFormat "%h %l %u %t "%r" "%{Cookie}i" %\>s %b" common LogFormat "%{Referer}i -\> %… --- ## [How to send Nagios reports to ELK](https://discuss.elastic.co/t/how-to-send-nagios-reports-to-elk/130975)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 2\ **Last updated:** [May 10, 2018, 2:35pm UTC](https://discuss.elastic.co/t/how-to-send-nagios-reports-to-elk/130975 "2018-05-10T14:35:04Z")
Hey guys, I have nagios for monitoring my servers and i'm getting various reports in the nagios console, for example availability report. Is there a way to ingest all those reports into ELK and view it in a real time. … --- ## [Parsing nested JSON](https://discuss.elastic.co/t/parsing-nested-json/131302)
**Author:** [@ashay.dhavale](https://discuss.elastic.co/u/ashay.dhavale)\ **Replies:** 1\ **Last updated:** [May 10, 2018, 1:56pm UTC](https://discuss.elastic.co/t/parsing-nested-json/131302 "2018-05-10T13:56:31Z")
Hi I have a JSON file as shown below. I want to read this file using logstash. Each log file i receive has only 1 JSON structure as shown below and nothing new gets appended to the log file. How can i combine these mult… --- ## [How to create fields containing info from different lines when using the multiline codec?](https://discuss.elastic.co/t/how-to-create-fields-containing-info-from-different-lines-when-using-the-multiline-codec/131250)
**Author:** [@stevefai](https://discuss.elastic.co/u/stevefai)\ **Replies:** 1\ **Last updated:** [May 10, 2018, 1:08pm UTC](https://discuss.elastic.co/t/how-to-create-fields-containing-info-from-different-lines-when-using-the-multiline-codec/131250 "2018-05-10T13:08:09Z")
I'm using Java Tomcat logs as input and I'm trying to save the relevant info for each log into Elasticsearch. A single log might look like this: \[timestamp\] \[log\_level\] - \[error short description\] \[stack trace\] (cont… --- ## [Logstash JDBC not returning data](https://discuss.elastic.co/t/logstash-jdbc-not-returning-data/131306)
**Author:** [@joaoantoniopereira](https://discuss.elastic.co/u/joaoantoniopereira)\ **Replies:** 0\ **Last updated:** [May 10, 2018, 12:18pm UTC](https://discuss.elastic.co/t/logstash-jdbc-not-returning-data/131306 "2018-05-10T12:18:19Z")
Hi, I am running Logstash 5.6.4 with JDBC input with the following config: input { jdbc { jdbc\_driver\_library =\> "/src/logstash-5.6.4/logstash-plugin-jdbc/ojdbc7-12.1.0.2.jar" jdbc\_driver\_class =\> "Java… --- ## [The perennial auditd event correlation problem](https://discuss.elastic.co/t/the-perennial-auditd-event-correlation-problem/131202)
**Author:** [@0xeb0de](https://discuss.elastic.co/u/0xeb0de)\ **Replies:** 2\ **Last updated:** [May 10, 2018, 8:48am UTC](https://discuss.elastic.co/t/the-perennial-auditd-event-correlation-problem/131202 "2018-05-10T08:48:35Z")
Many, many people over the years have tried to find a sensible way of reducing the several events produced by Linux auditd to a single, meaningful one. audisp, for example, will produce syslog messages of type SYSCALL, C… --- ## [WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash](https://discuss.elastic.co/t/warning-could-not-find-logstash-yml-which-is-typically-located-in-ls-home-config-or-etc-logstash/131022)
**Author:** [@sayed\_mohamed](https://discuss.elastic.co/u/sayed_mohamed)\ **Replies:** 23\ **Last updated:** [May 10, 2018, 8:14am UTC](https://discuss.elastic.co/t/warning-could-not-find-logstash-yml-which-is-typically-located-in-ls-home-config-or-etc-logstash/131022 "2018-05-10T08:14:59Z")
i have a problem that i run file as command : sudo bin/logstash --path.data sensor38 -f /home/sayed/logstash/sayed.conf and i have an warning that i set some configuration on it WARNING: Could not find logstash.yml whi… --- ## [Logstash - Startup - Template, Exception in start](https://discuss.elastic.co/t/logstash-startup-template-exception-in-start/130688)
**Author:** [@Karthik\_Ramachandran](https://discuss.elastic.co/u/Karthik_Ramachandran)\ **Replies:** 3\ **Last updated:** [May 10, 2018, 8:13am UTC](https://discuss.elastic.co/t/logstash-startup-template-exception-in-start/130688 "2018-05-10T08:13:55Z")
ES Version 6.2.1 Logstash Version (6.0.1 and 6.2.4). I'm observing multiple logstash templates being created, and was not clear on these. Environment: 2 Clusters Cluster 1 Core Logging Cluster for keeping business … --- ## [Logstash failed to push data (create indices) to elasticsearch](https://discuss.elastic.co/t/logstash-failed-to-push-data-create-indices-to-elasticsearch/131266)
**Author:** [@elsnewbie](https://discuss.elastic.co/u/elsnewbie)\ **Replies:** 0\ **Last updated:** [May 10, 2018, 8:09am UTC](https://discuss.elastic.co/t/logstash-failed-to-push-data-create-indices-to-elasticsearch/131266 "2018-05-10T08:09:07Z")
I am newbie in ELK. I am having a problem to create indices in ES which data coming from logstash (cloudwatch plugin). I use logstash 6.1.4 and ES 6.2.4. The config of logstash likes following //input { // cloudwatc… --- ## [Netflow module](https://discuss.elastic.co/t/netflow-module/131123)
**Author:** [@venki522](https://discuss.elastic.co/u/venki522)\ **Replies:** 10\ **Last updated:** [May 10, 2018, 7:17am UTC](https://discuss.elastic.co/t/netflow-module/131123 "2018-05-10T07:17:36Z")
Hello, I am testing netflow module on elk stack and not getting any where. Can someone let me know how to configure logstash to get the netflow data and send to elastic so that i can see the data on kibana --- ## [What are the default rollover settings? How to identify rollover settings in Logstash?](https://discuss.elastic.co/t/what-are-the-default-rollover-settings-how-to-identify-rollover-settings-in-logstash/131234)
**Author:** [@shankarps](https://discuss.elastic.co/u/shankarps)\ **Replies:** 2\ **Last updated:** [May 10, 2018, 6:21am UTC](https://discuss.elastic.co/t/what-are-the-default-rollover-settings-how-to-identify-rollover-settings-in-logstash/131234 "2018-05-10T06:21:16Z")
I started working on an existing ELK installation. I see that it has rollover option daily. The logs are written to the logstash-\* indices, rolling over daily, like this index name - logstash-2018.05.09. I am trying to … --- ## [After installing x-pack on ELK, Logstash not create index to elasticsearch](https://discuss.elastic.co/t/after-installing-x-pack-on-elk-logstash-not-create-index-to-elasticsearch/131106)
**Author:** [@SK5277-SINGH](https://discuss.elastic.co/u/SK5277-SINGH)\ **Replies:** 15\ **Last updated:** [May 10, 2018, 6:07am UTC](https://discuss.elastic.co/t/after-installing-x-pack-on-elk-logstash-not-create-index-to-elasticsearch/131106 "2018-05-10T06:07:52Z")
Before x-pack installed on ELK, There was no any issue This my logstash.conf file input{ file{ path =\> "E:\\ELK\\data\\actor.csv" start\_position =\> "beginning" } } filter { csv { separator =\> "," columns =\> \["act… --- ## [How to get input to logstash from kafka consumer instead of kafka bootstrap server?](https://discuss.elastic.co/t/how-to-get-input-to-logstash-from-kafka-consumer-instead-of-kafka-bootstrap-server/131253)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 0\ **Last updated:** [May 10, 2018, 6:01am UTC](https://discuss.elastic.co/t/how-to-get-input-to-logstash-from-kafka-consumer-instead-of-kafka-bootstrap-server/131253 "2018-05-10T06:01:50Z")
Hi, my pipeline is producer--\>kafka--\>consumer--\>logstash--\>ES--\>Kibana here is my config used to get input from kafka input { kafka { codec =\> json bootstrap\_servers =\> "10.24.13.13:9092" topics =\> \["topic\_dmes… --- ## [ERROR: Settings 'path.config' (-f) and 'config.string' (-e) can't be used simultaneously](https://discuss.elastic.co/t/error-settings-path-config-f-and-config-string-e-cant-be-used-simultaneously/131240)
**Author:** [@Thoremp](https://discuss.elastic.co/u/Thoremp)\ **Replies:** 0\ **Last updated:** [May 10, 2018, 3:26am UTC](https://discuss.elastic.co/t/error-settings-path-config-f-and-config-string-e-cant-be-used-simultaneously/131240 "2018-05-10T03:26:12Z")
I use this code in powershell(windows): bin/logstash -e "input{stdin{codec=\>line}}output{stdout{codec=\>ru bydebug}}" then ERROR: Settings 'path.config' (-f) and 'config.string' (-e) can't be used simultaneously. I n… --- ## [Template mapping dosen't work](https://discuss.elastic.co/t/template-mapping-dosent-work/131130)
**Author:** [@akml\_kk](https://discuss.elastic.co/u/akml_kk)\ **Replies:** 11\ **Last updated:** [May 9, 2018, 8:57pm UTC](https://discuss.elastic.co/t/template-mapping-dosent-work/131130 "2018-05-09T20:57:32Z")
I'm trying to convert latitude and longitude to geo\_point location in order to use it a map visulaztion. this is my configuration: input{ file{ path=\>"C:\\Users\\Ahmed\\Desktop\\311\_Service\_Requests\_from\_2015.csv" … --- ## [Location dosen't convert to geo\_point](https://discuss.elastic.co/t/location-dosent-convert-to-geo-point/131199)
**Author:** [@akml\_kk](https://discuss.elastic.co/u/akml_kk)\ **Replies:** 3\ **Last updated:** [May 9, 2018, 8:55pm UTC](https://discuss.elastic.co/t/location-dosent-convert-to-geo-point/131199 "2018-05-09T20:55:08Z")
as descriped here i ran this template : put \_template/nyc { "order":0, "index\_patterns":"nyc", "mapping":{ "\_default":{ "properties":{ "location ":{ "type":"geo\_point" } } } } } t… --- ## [S3 output debug](https://discuss.elastic.co/t/s3-output-debug/131141)
**Author:** [@mememe](https://discuss.elastic.co/u/mememe)\ **Replies:** 1\ **Last updated:** [May 9, 2018, 5:54pm UTC](https://discuss.elastic.co/t/s3-output-debug/131141 "2018-05-09T17:54:19Z")
Hi, I am quite new to logstash and do run logstash 5 on AWS Linux using s3 output and elasticsearch output like in below snippet from logstash.conf The problem I face is that data is not put in s3. Still, the s3 test fi… --- ## [Service doesn't process log but command line works](https://discuss.elastic.co/t/service-doesnt-process-log-but-command-line-works/130903)
**Author:** [@Michael3](https://discuss.elastic.co/u/Michael3)\ **Replies:** 9\ **Last updated:** [May 9, 2018, 4:33pm UTC](https://discuss.elastic.co/t/service-doesnt-process-log-but-command-line-works/130903 "2018-05-09T16:33:34Z")
I am hoping someone would have some suggestions on this because so far I've got nothing. I have a pattern that works in logstash 5.6 on centos 7, but not on centos 6 when run as a service. Run from the command line as … --- ## [Logstash.inputs.redis - Redis connection problem](https://discuss.elastic.co/t/logstash-inputs-redis-redis-connection-problem/130207)
**Author:** [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\ **Replies:** 2\ **Last updated:** [May 9, 2018, 3:02pm UTC](https://discuss.elastic.co/t/logstash-inputs-redis-redis-connection-problem/130207 "2018-05-09T15:02:19Z")
Hello I am using redis-server to queue my data which is sent from the logstash shipper. From the redis queue the logstash indexers picks the data. But for some reason after some time again and again there is a connecti… --- ## [How to escape field values for elasticsearch filter query? Take 2](https://discuss.elastic.co/t/how-to-escape-field-values-for-elasticsearch-filter-query-take-2/131180)
**Author:** [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\ **Replies:** 0\ **Last updated:** [May 9, 2018, 2:09pm UTC](https://discuss.elastic.co/t/how-to-escape-field-values-for-elasticsearch-filter-query-take-2/131180 "2018-05-09T14:09:15Z")
Continuing the discussion from How to escape field values for elasticsearch filter query?: Sorry for disappearing on my previous question, I had to drop my work for a while. I do not understand the reply @magnusbaeck ga… --- ## [ServiceNow Incident Table data in ELK](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 13\ **Last updated:** [May 9, 2018, 1:55pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216 "2018-05-09T13:55:35Z")
Hey Guys, I have been trying to get servicenow incident data into ELK through http\_poller method using logstash, i'm getting the data but all the incident records are sitting under a single field. Because of this i'm n… --- ## [Filtre Cisco-Asa](https://discuss.elastic.co/t/filtre-cisco-asa/131169)
**Author:** [@asalma](https://discuss.elastic.co/u/asalma)\ **Replies:** 2\ **Last updated:** [May 9, 2018, 1:32pm UTC](https://discuss.elastic.co/t/filtre-cisco-asa/131169 "2018-05-09T13:32:11Z")
Hi, Please I need the link of Github that contain all the filter of cisco-asa. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=761) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=763)