# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=915 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 916 --- ## [Logstash MSSQL JDBC connection troubleshooting](https://discuss.elastic.co/t/logstash-mssql-jdbc-connection-troubleshooting/97622)
**Author:** [@gh0stid](https://discuss.elastic.co/u/gh0stid)\ **Replies:** 5\ **Last updated:** [September 4, 2017, 9:13am UTC](https://discuss.elastic.co/t/logstash-mssql-jdbc-connection-troubleshooting/97622 "2017-09-04T09:13:53Z")
Hello, sorry to bother you, once again I have a question about Logstash input... I just got back from a 9month medical leave, so im a bit lost i must admit... anyway, heres the config : stripped input and output to ke… --- ## [Logstash kafka output plugin failed](https://discuss.elastic.co/t/logstash-kafka-output-plugin-failed/99246)
**Author:** [@Yim\_ChitGing](https://discuss.elastic.co/u/Yim_ChitGing)\ **Replies:** 0\ **Last updated:** [September 4, 2017, 6:16am UTC](https://discuss.elastic.co/t/logstash-kafka-output-plugin-failed/99246 "2017-09-04T06:16:18Z")
For details : issue --- ## [Mutate under conditional statement is not working. any suggestion please](https://discuss.elastic.co/t/mutate-under-conditional-statement-is-not-working-any-suggestion-please/99225)
**Author:** [@smeka](https://discuss.elastic.co/u/smeka)\ **Replies:** 1\ **Last updated:** [September 4, 2017, 5:30am UTC](https://discuss.elastic.co/t/mutate-under-conditional-statement-is-not-working-any-suggestion-please/99225 "2017-09-04T05:30:58Z")
Hi, Need some help please. path =\> \[ “/var/log/jbossas/domain/server1/server.log”, “/var/log/jbossas/domain/server2/server.log”\] in filter portion i used the below to create a directory to write the log file to centra… --- ## [\[HELP\]How to run multiple logstash instances](https://discuss.elastic.co/t/help-how-to-run-multiple-logstash-instances/99195)
**Author:** [@fsaas](https://discuss.elastic.co/u/fsaas)\ **Replies:** 5\ **Last updated:** [September 4, 2017, 5:25am UTC](https://discuss.elastic.co/t/help-how-to-run-multiple-logstash-instances/99195 "2017-09-04T05:25:37Z")
Hi all. I’m a beginner to logstash and I need to run multiple logstash instances. My logstash version is 5.5 and I followed this link. here and here And the result is grep -v ^# logstash.yml | grep \[a-z\] \[image\] t… --- ## [Day of Week from timestamp using ruby code](https://discuss.elastic.co/t/day-of-week-from-timestamp-using-ruby-code/99086)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 1\ **Last updated:** [September 3, 2017, 6:40pm UTC](https://discuss.elastic.co/t/day-of-week-from-timestamp-using-ruby-code/99086 "2017-09-03T18:40:33Z")
Hi, I am trying to get the day of the week(Sun,Mon…) from the timestamp field in the log with the below code: ruby { code =\> ‘event.set(“day\_of\_week”, event.get(’@timestamp’).time.strftime ‘%a’)’ } But getting " ERR… --- ## [Parse RFC3339Nano Date to @timestamp field](https://discuss.elastic.co/t/parse-rfc3339nano-date-to-timestamp-field/98978)
**Author:** [@rucciva](https://discuss.elastic.co/u/rucciva)\ **Replies:** 5\ **Last updated:** [September 3, 2017, 6:37pm UTC](https://discuss.elastic.co/t/parse-rfc3339nano-date-to-timestamp-field/98978 "2017-09-03T18:37:07Z")
How do you parse time that is formatted using golang’s RFC3339Nano into @timestamp field? --- ## [COMBINEDAPACHELOG And COMMONAPACHELOG query](https://discuss.elastic.co/t/combinedapachelog-and-commonapachelog-query/99187)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 5\ **Last updated:** [September 3, 2017, 9:05am UTC](https://discuss.elastic.co/t/combinedapachelog-and-commonapachelog-query/99187 "2017-09-03T09:05:50Z")
Hi Guys, I have this basic query and I would really appreciate if someone can answer this for me? COMBINEDAPACHELOG can that be used directly in logstash or do I need to create COMBINEDAPACHELOG pattern under patterns … --- ## [Parsing / Indexing strategies for BRO?](https://discuss.elastic.co/t/parsing-indexing-strategies-for-bro/99016)
**Author:** [@lucasjkr](https://discuss.elastic.co/u/lucasjkr)\ **Replies:** 5\ **Last updated:** [September 2, 2017, 8:56pm UTC](https://discuss.elastic.co/t/parsing-indexing-strategies-for-bro/99016 "2017-09-02T20:56:21Z")
This is part logstash, part elastic search, but since the decision is being made at the Logstash level, I thought I would ask here: What are peoples strategies for indexing incoming BRO data? It creates over a dozen log… --- ## [Can someone please help me with nginx logstash filter](https://discuss.elastic.co/t/can-someone-please-help-me-with-nginx-logstash-filter/98377)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 10\ **Last updated:** [September 2, 2017, 1:00pm UTC](https://discuss.elastic.co/t/can-someone-please-help-me-with-nginx-logstash-filter/98377 "2017-09-02T13:00:54Z")
Hi Guys, Can someone please help me to identify the issue with this filter, I am trying to parse the nginx logs with file input and getting below error. can somone please help me to rectify the error? file { { type … --- ## [Logstash pipeline starts and then stops with a warning](https://discuss.elastic.co/t/logstash-pipeline-starts-and-then-stops-with-a-warning/99071)
**Author:** [@elk\_of\_the\_forest](https://discuss.elastic.co/u/elk_of_the_forest)\ **Replies:** 3\ **Last updated:** [September 2, 2017, 2:03am UTC](https://discuss.elastic.co/t/logstash-pipeline-starts-and-then-stops-with-a-warning/99071 "2017-09-02T02:03:45Z")
This is the error im getting when I start logstash. \[2017-08-31T18:59:33,010\]\[INFO \]\[logstash.pipeline \] Starting pipeline {“id”=\>“main”, “pipeline.workers”=\>4, “pipeline.batch.size”=\>125, “pipeline.batch.delay”=\>… --- ## [Index fields concatenated together?](https://discuss.elastic.co/t/index-fields-concatenated-together/98604)
**Author:** [@bulsac](https://discuss.elastic.co/u/bulsac)\ **Replies:** 4\ **Last updated:** [September 1, 2017, 10:31pm UTC](https://discuss.elastic.co/t/index-fields-concatenated-together/98604 "2017-09-01T22:31:00Z")
I recently added 2 new .CONF files to my /etc/logstash/conf.d/ directory to ingest data into 2 new indices, in addition to 1 existing index. When I view the 2 new indices in Kibana, it appears that the 2 new indices cont… --- ## [Unable to parse substring of a message in logstash](https://discuss.elastic.co/t/unable-to-parse-substring-of-a-message-in-logstash/99149)
**Author:** [@Pola](https://discuss.elastic.co/u/Pola)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 3:47pm UTC](https://discuss.elastic.co/t/unable-to-parse-substring-of-a-message-in-logstash/99149 "2017-09-01T15:47:21Z")
Hi, I am kind of new to ELK. I have trouble in splitting up substring of a message into fields and my goal is to get active alarms from each host that would be (no. of alarms set - no of alarms cleared). message field … --- ## [Logstash is not picking config file under /etc/logstash/conf.d/ directory](https://discuss.elastic.co/t/logstash-is-not-picking-config-file-under-etc-logstash-conf-d-directory/98987)
**Author:** [@rajesh](https://discuss.elastic.co/u/rajesh)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 2:44pm UTC](https://discuss.elastic.co/t/logstash-is-not-picking-config-file-under-etc-logstash-conf-d-directory/98987 "2017-09-01T14:44:15Z")
Hi… I am running Logstash as a service on Redhat distribution with the command “sudo initctl start logstash” and when I checked the status, it says Logstash is running. But the problem is it’s not using the config file … --- ## [Failed to parse mapping \[\_default\_\]: A dynamic template must be defined with a name](https://discuss.elastic.co/t/failed-to-parse-mapping--default--a-dynamic-template-must-be-defined-with-a-name/96595)
**Author:** [@Alexander\_Gerasimenk](https://discuss.elastic.co/u/Alexander_Gerasimenk)\ **Replies:** 4\ **Last updated:** [September 1, 2017, 2:26pm UTC](https://discuss.elastic.co/t/failed-to-parse-mapping--default--a-dynamic-template-must-be-defined-with-a-name/96595 "2017-09-01T14:26:59Z")
Hello, I'm trying to make logstash store suricata data to a specific elasticsearch index. When I delete template and elasticsearch index and then restart logstash - it works fine, but when logstash creates new daily ela… --- ## [Json\_encode filter issues](https://discuss.elastic.co/t/json-encode-filter-issues/99018)
**Author:** [@BIitzkrieg](https://discuss.elastic.co/u/BIitzkrieg)\ **Replies:** 4\ **Last updated:** [September 1, 2017, 2:22pm UTC](https://discuss.elastic.co/t/json-encode-filter-issues/99018 "2017-09-01T14:22:34Z")
I am trying to use the json\_encode filter in my logstash config. I have installed the plugin via /opt/logstash/bin/logstash-plugin install logstash-filter-json\_encode It returns " installation successful" My logstash … --- ## [JDBC Sqlserver to elastic search : one input to different output?](https://discuss.elastic.co/t/jdbc-sqlserver-to-elastic-search-one-input-to-different-output/99126)
**Author:** [@Chesters](https://discuss.elastic.co/u/Chesters)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 1:37pm UTC](https://discuss.elastic.co/t/jdbc-sqlserver-to-elastic-search-one-input-to-different-output/99126 "2017-09-01T13:37:36Z")
Hello, I am quite new to logstash configuration, in fact I learned about logstash altogether yesterday. Please feel free to direct me to documentation if my question is excessively obvious, here it goes : I have a sql… --- ## [Data is getting removed from Redis once transfered from redis to elastic using logtash](https://discuss.elastic.co/t/data-is-getting-removed-from-redis-once-transfered-from-redis-to-elastic-using-logtash/98129)
**Author:** [@swaplogstash](https://discuss.elastic.co/u/swaplogstash)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 1:29pm UTC](https://discuss.elastic.co/t/data-is-getting-removed-from-redis-once-transfered-from-redis-to-elastic-using-logtash/98129 "2017-09-01T13:29:24Z")
Data is getting removed from Redis once transfered from redis to elastic using logtash. Is it expected behaviour? --- ## [Gsub not work](https://discuss.elastic.co/t/gsub-not-work/99091)
**Author:** [@Alexander\_Orlenko](https://discuss.elastic.co/u/Alexander_Orlenko)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 6:30am UTC](https://discuss.elastic.co/t/gsub-not-work/99091 "2017-09-01T06:30:12Z")
Hello, We have field, need cut symbol from end, we tru use gsub(regexp), but not cuting mutate {replace =\> {type =\> “ztk\_login”} copy =\> { “host\_name” =\> “host\_name\_cut” } gsub =\> \[“host\_name\_cut”, “-ztk\\d\\d”, " "\] … --- ## [Logstash grok filter for quoted value OR dash](https://discuss.elastic.co/t/logstash-grok-filter-for-quoted-value-or-dash/99094)
**Author:** [@Maekee](https://discuss.elastic.co/u/Maekee)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 8:25am UTC](https://discuss.elastic.co/t/logstash-grok-filter-for-quoted-value-or-dash/99094 "2017-09-01T08:25:42Z")
Hello, I have a blue coat Proxy log that have a field that can either be a quouted value like “Microsoft-CryptoAPI/15.2” or a dash (-). I captured this with %{QUOTEDSTRING:useragent} Before i found out that the value c… --- ## [Condition with regex edge-case - BUG?](https://discuss.elastic.co/t/condition-with-regex-edge-case-bug/99118)
**Author:** [@BennyInc](https://discuss.elastic.co/u/BennyInc)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 11:11am UTC](https://discuss.elastic.co/t/condition-with-regex-edge-case-bug/99118 "2017-09-01T11:11:49Z")
I am parsing our logs, and want to apply a filter only to entries where the “message” field starts with “\[AUDIT\]”, like this: \[AUDIT\] an audit entry My filter I’m testing with looks like this: if \[message\] =~ /^\\\[AUDI… --- ## [How to handle self closed tags in xml parsing](https://discuss.elastic.co/t/how-to-handle-self-closed-tags-in-xml-parsing/98957)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 8:57am UTC](https://discuss.elastic.co/t/how-to-handle-self-closed-tags-in-xml-parsing/98957 "2017-09-01T08:57:08Z")
If I use logstash to parse XML with an self-closing tag (e.g. \), it turns it into an empty object instead of an empty string. And its not creating any index in elastic search. I am suspecting because column 1 v… --- ## [Logstash input UDP netflow codec - BUG?](https://discuss.elastic.co/t/logstash-input-udp-netflow-codec-bug/99098)
**Author:** [@pierre1](https://discuss.elastic.co/u/pierre1)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 8:04am UTC](https://discuss.elastic.co/t/logstash-input-udp-netflow-codec-bug/99098 "2017-09-01T08:04:20Z")
i think there is a bug with UDP / netflow : my conf : input { udp { port =\> 9556 codec =\> netflow { versions =\> \[5, 9\] target =\> "nf" } type =\> "netflow" } } In all case i receive the flow first : when my… --- ## [Logstash-input-cloudwatch plugin errors](https://discuss.elastic.co/t/logstash-input-cloudwatch-plugin-errors/98955)
**Author:** [@blurry](https://discuss.elastic.co/u/blurry)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 8:04am UTC](https://discuss.elastic.co/t/logstash-input-cloudwatch-plugin-errors/98955 "2017-09-01T08:04:08Z")
Hi I am having some issues with the Cloudwatch-Input-plugin for Logstash. I am getting the following error in the logs: \[2017-08-30T13:20:06,433\]\[INFO \]\[logstash.inputs.cloudwatch\] Polling CloudWatch API \[2017-08-30T13… --- ## [Convert ingest node pipeline to logstash](https://discuss.elastic.co/t/convert-ingest-node-pipeline-to-logstash/98222)
**Author:** [@Nadavr](https://discuss.elastic.co/u/Nadavr)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 7:03am UTC](https://discuss.elastic.co/t/convert-ingest-node-pipeline-to-logstash/98222 "2017-09-01T07:03:35Z")
Hi everyone, I’m quite new to logstash. I’m trying to aggregate auditd logs. While Filebeat has a module that does exactly that - it require me to forward it directly to ES as it parses the logs using an ingest node … --- ## [Will logging in DEBUG level cause performance issues in Logstash?](https://discuss.elastic.co/t/will-logging-in-debug-level-cause-performance-issues-in-logstash/98799)
**Author:** [@NandhaKrishna](https://discuss.elastic.co/u/NandhaKrishna)\ **Replies:** 6\ **Last updated:** [September 1, 2017, 6:53am UTC](https://discuss.elastic.co/t/will-logging-in-debug-level-cause-performance-issues-in-logstash/98799 "2017-09-01T06:53:58Z")
Just wanted to know whether there will be any performance issues when the logging level is changed to DEBUG in logstash. --- ## [Grok pattern not being applied in logstash but working on grok debugger](https://discuss.elastic.co/t/grok-pattern-not-being-applied-in-logstash-but-working-on-grok-debugger/97384)
**Author:** [@Swaroop\_Chandre](https://discuss.elastic.co/u/Swaroop_Chandre)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 6:45am UTC](https://discuss.elastic.co/t/grok-pattern-not-being-applied-in-logstash-but-working-on-grok-debugger/97384 "2017-09-01T06:45:44Z")
My grok pattern is working on grok debugger but when reading from filebeat my grok pattern is not applied. My syslog-filter.conf file filter { if \[type\] == “log” { grok { patterns\_dir =\> \["/etc/logstash/conf.d/patter… --- ## [Bluecoat Grok filter](https://discuss.elastic.co/t/bluecoat-grok-filter/98997)
**Author:** [@Maekee](https://discuss.elastic.co/u/Maekee)\ **Replies:** 8\ **Last updated:** [September 1, 2017, 5:56am UTC](https://discuss.elastic.co/t/bluecoat-grok-filter/98997 "2017-09-01T05:56:42Z")
Hello, I am running the ELK-stack on a server and want to parse the Bluecoat Proxy logs. I have created a grok filter and it works when i test it in the Grok Debugger (http://grokdebug.herokuapp.com/) but when i add th… --- ## [Logstash creating more rows than the source table](https://discuss.elastic.co/t/logstash-creating-more-rows-than-the-source-table/99054)
**Author:** [@aliyesami](https://discuss.elastic.co/u/aliyesami)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 5:15am UTC](https://discuss.elastic.co/t/logstash-creating-more-rows-than-the-source-table/99054 "2017-09-01T05:15:22Z")
I have only 24062 rows in my RDBMS table but logstash jdbc connection (conf file shown below) keep adding documents to the Elasticsearch in endless loop . \[root@hadoop1 kibana\]# curl 'hadoop5:9200/\_cat/indices?v' | gre… --- ## [LOGSTASH to connect two UNIX server in a secure network](https://discuss.elastic.co/t/logstash-to-connect-two-unix-server-in-a-secure-network/98209)
**Author:** [@kosigamaani\_thirumoo](https://discuss.elastic.co/u/kosigamaani_thirumoo)\ **Replies:** 4\ **Last updated:** [September 1, 2017, 4:05am UTC](https://discuss.elastic.co/t/logstash-to-connect-two-unix-server-in-a-secure-network/98209 "2017-09-01T04:05:51Z")
Hi, I’ve two UNIX Servers A and B. I’ve installed installed Logstash in Server A and i would like to read logs present in Server B. Kindly suggest suitable input plugin. Note: Server B doesn’t contain Logstash Thanks --- ## [Logstash-plugin install failed](https://discuss.elastic.co/t/logstash-plugin-install-failed/98694)
**Author:** [@arun\_prasath1](https://discuss.elastic.co/u/arun_prasath1)\ **Replies:** 4\ **Last updated:** [September 1, 2017, 3:58am UTC](https://discuss.elastic.co/t/logstash-plugin-install-failed/98694 "2017-09-01T03:58:02Z")
I am trying to install output plugin using below command but it has failed saying there is no connection. But I tried to download and able to download using wget. I have extracted the download .gz file and it is a data f… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=914) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=916)