# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=801 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 802 --- ## [Logstash output Indexname with old Date](https://discuss.elastic.co/t/logstash-output-indexname-with-old-date/123433)
**Author:** [@sumanbehara](https://discuss.elastic.co/u/sumanbehara)\ **Replies:** 5\ **Last updated:** [March 15, 2018, 5:02pm UTC](https://discuss.elastic.co/t/logstash-output-indexname-with-old-date/123433 "2018-03-15T17:02:36Z")
How to add in logstash output Index name as "logstash-old date" , not with timestamp. Current Date is working fine for me. how to convert 2016-05-23T10:42:04.370Z to normal date 2016-05-23 and use this date in logstash o… --- ## [Input S3 with debug mode (BIG POST)](https://discuss.elastic.co/t/input-s3-with-debug-mode-big-post/124127)
**Author:** [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Replies:** 0\ **Last updated:** [March 15, 2018, 3:14pm UTC](https://discuss.elastic.co/t/input-s3-with-debug-mode-big-post/124127 "2018-03-15T15:14:04Z")
Hello , For a couple days I have been working with logstash 6.2 with docker, using the official docker's image for try get S3 file with logs, I'll try describe all the process with lot details, because I can't do get l… --- ## [Reloading Elastic output template](https://discuss.elastic.co/t/reloading-elastic-output-template/124124)
**Author:** [@aclowkey](https://discuss.elastic.co/u/aclowkey)\ **Replies:** 1\ **Last updated:** [March 15, 2018, 3:12pm UTC](https://discuss.elastic.co/t/reloading-elastic-output-template/124124 "2018-03-15T15:12:46Z")
The output installs the template only on start? What if I want to change my template ? Do I have to restart logstash? or do I have to install it manually. The template is handwritten --- ## [Rebooting server with logstash](https://discuss.elastic.co/t/rebooting-server-with-logstash/124054)
**Author:** [@Adren](https://discuss.elastic.co/u/Adren)\ **Replies:** 1\ **Last updated:** [March 15, 2018, 2:56pm UTC](https://discuss.elastic.co/t/rebooting-server-with-logstash/124054 "2018-03-15T14:56:27Z")
Hello, I recently discovered that there was no spaces left in the disk's server so I erased files and reboots Then one Logstash pipeline doesn't work but Idk why, because I receive logs from the other pipeline. Nothing … --- ## [Logstash forwader and indexer](https://discuss.elastic.co/t/logstash-forwader-and-indexer/123844)
**Author:** [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\ **Replies:** 16\ **Last updated:** [March 15, 2018, 2:17pm UTC](https://discuss.elastic.co/t/logstash-forwader-and-indexer/123844 "2018-03-15T14:17:53Z")
Hi Wanted to know if within the logstash output if I want to put another filter is it possible. --- ## [Logstash time](https://discuss.elastic.co/t/logstash-time/124107)
**Author:** [@Jose\_Campos](https://discuss.elastic.co/u/Jose_Campos)\ **Replies:** 2\ **Last updated:** [March 15, 2018, 2:05pm UTC](https://discuss.elastic.co/t/logstash-time/124107 "2018-03-15T14:05:05Z")
Hello, a moment ago I was seeing that the time of my log does not match with the timestamp of my kibana, sending it from a logstash 2.0, but if I send it from another if it matches the timestamp. The difference is two ho… --- ## [Create field depending on array index value](https://discuss.elastic.co/t/create-field-depending-on-array-index-value/124099)
**Author:** [@No\_Executable](https://discuss.elastic.co/u/No_Executable)\ **Replies:** 2\ **Last updated:** [March 15, 2018, 1:23pm UTC](https://discuss.elastic.co/t/create-field-depending-on-array-index-value/124099 "2018-03-15T13:23:13Z")
What I want to do is depending on the ID, array index 0, is to instead put the ID label, array index 1 into the new field Command. My current filter config looks like this: filter { xml { source =\> 'message… --- ## [Logstash improve time performance](https://discuss.elastic.co/t/logstash-improve-time-performance/123741)
**Author:** [@youssef](https://discuss.elastic.co/u/youssef)\ **Replies:** 12\ **Last updated:** [March 15, 2018, 11:18am UTC](https://discuss.elastic.co/t/logstash-improve-time-performance/123741 "2018-03-15T11:18:27Z")
hi all, i have csv file that contains 4 Million lines, each line is more than 100 columns, i process about 30 columns and add another 10 columns, so that in total i upload about 40 columns to the kibana, in average i n… --- ## [Nested array of Json](https://discuss.elastic.co/t/nested-array-of-json/123287)
**Author:** [@gius78](https://discuss.elastic.co/u/gius78)\ **Replies:** 1\ **Last updated:** [March 15, 2018, 10:53am UTC](https://discuss.elastic.co/t/nested-array-of-json/123287 "2018-03-15T10:53:42Z")
Hi all, sorry if topic has already been discussed, but I could not find a fitting example: I've a log with the following structure: message: {PROF\_ID":"P\_CUSTOMER","action":"USERROLE","USR\_COMP":"0xx","USER\_DN":"cn=TE… --- ## [Elastic search filter plugin secured](https://discuss.elastic.co/t/elastic-search-filter-plugin-secured/123911)
**Author:** [@saijana](https://discuss.elastic.co/u/saijana)\ **Replies:** 1\ **Last updated:** [March 15, 2018, 9:26am UTC](https://discuss.elastic.co/t/elastic-search-filter-plugin-secured/123911 "2018-03-15T09:26:23Z")
Will elasticsearch filter plugin works with secured elasticsearch (SSL Enabled) in the latest version ?? --- ## [Beat and Logstash](https://discuss.elastic.co/t/beat-and-logstash/118200)
**Author:** [@sid.sh](https://discuss.elastic.co/u/sid.sh)\ **Replies:** 13\ **Last updated:** [March 15, 2018, 9:06am UTC](https://discuss.elastic.co/t/beat-and-logstash/118200 "2018-03-15T09:06:29Z")
I am new to coding and logstash. I am receiving many files through filebeat on single port in my logstash.conf. How can I differentiate the files basis on their naming conventions and then write a filter for them. Any… --- ## [Replace @timestamp value with SQL DB datetime value](https://discuss.elastic.co/t/replace-timestamp-value-with-sql-db-datetime-value/123748)
**Author:** [@zakkhan](https://discuss.elastic.co/u/zakkhan)\ **Replies:** 13\ **Last updated:** [March 15, 2018, 8:17am UTC](https://discuss.elastic.co/t/replace-timestamp-value-with-sql-db-datetime-value/123748 "2018-03-15T08:17:15Z")
I have tried everything but nothing worked. I need to replace @timestamp with my Datetime column value. Format of both values Datetime: 2017-08-23T13:20:09.000Z @timesatmp: 2018-03-13T12:11:38.080Z input { jdbc { … --- ## [Elastic search stops recording data after reaching a certain count](https://discuss.elastic.co/t/elastic-search-stops-recording-data-after-reaching-a-certain-count/123847)
**Author:** [@sunilkumar](https://discuss.elastic.co/u/sunilkumar)\ **Replies:** 12\ **Last updated:** [March 15, 2018, 8:17am UTC](https://discuss.elastic.co/t/elastic-search-stops-recording-data-after-reaching-a-certain-count/123847 "2018-03-15T08:17:04Z")
In Elasticsearch data is added till a certain count and then it stops processing after that point. --- ## [Taking Logstash input from Excel(.xlsx) file](https://discuss.elastic.co/t/taking-logstash-input-from-excel-xlsx-file/124029)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 1\ **Last updated:** [March 15, 2018, 5:37am UTC](https://discuss.elastic.co/t/taking-logstash-input-from-excel-xlsx-file/124029 "2018-03-15T05:37:40Z")
Hi, I am using ELK 5.5.0. Could anyone please help on how can we take input to Logstash(5.5.0) from an EXCEL file. --- ## [How to remove quotes from value of fieldname](https://discuss.elastic.co/t/how-to-remove-quotes-from-value-of-fieldname/124004)
**Author:** [@micruzz82](https://discuss.elastic.co/u/micruzz82)\ **Replies:** 8\ **Last updated:** [March 15, 2018, 12:40am UTC](https://discuss.elastic.co/t/how-to-remove-quotes-from-value-of-fieldname/124004 "2018-03-15T00:40:37Z")
Hi all. I've been having problems with indexing where in the JSON is returning values in string format but I would like to remove the quotes on all values "dn": "topology/pod-1/node-104/sys/phys-\[eth1/48\]/CDeqptEgrTota… --- ## [Looking for some help correcting my config files](https://discuss.elastic.co/t/looking-for-some-help-correcting-my-config-files/123976)
**Author:** [@William-Robert](https://discuss.elastic.co/u/William-Robert)\ **Replies:** 1\ **Last updated:** [March 14, 2018, 10:15pm UTC](https://discuss.elastic.co/t/looking-for-some-help-correcting-my-config-files/123976 "2018-03-14T22:15:26Z")
Followed this tutorial for a basic install and configure the ELK stack for ingesting syslogs. My goal is to create a system where I can ingest apache logs from wordpress sites that use wpengine. WPengine won't allow you… --- ## [Any tool/plugin in ELK stack to scan IIS logs to generate the source in CSV or desired format to logstash](https://discuss.elastic.co/t/any-tool-plugin-in-elk-stack-to-scan-iis-logs-to-generate-the-source-in-csv-or-desired-format-to-logstash/123933)
**Author:** [@jason\_smith](https://discuss.elastic.co/u/jason_smith)\ **Replies:** 3\ **Last updated:** [March 14, 2018, 8:39pm UTC](https://discuss.elastic.co/t/any-tool-plugin-in-elk-stack-to-scan-iis-logs-to-generate-the-source-in-csv-or-desired-format-to-logstash/123933 "2018-03-14T20:39:49Z")
Hello Good folks, So far the samples i have done are all the CSV files that are sourced into Elastic search thru logstash. If I want to read IIS logs for specific errors is there a plugin in ELK stack or any tool that … --- ## [Snort CSV Parsing](https://discuss.elastic.co/t/snort-csv-parsing/123985)
**Author:** [@Brandonmcgr](https://discuss.elastic.co/u/Brandonmcgr)\ **Replies:** 2\ **Last updated:** [March 14, 2018, 8:17pm UTC](https://discuss.elastic.co/t/snort-csv-parsing/123985 "2018-03-14T20:17:39Z")
Hello, I'm working on sending my Snort logs to ELK. So far, if i send it directly to Elasticsearch. It works, but obviously with poor formatting. However, when sending it to logstash; I recieve some weird issues. Here… --- ## [Logstash doesn't install plugin from local gem](https://discuss.elastic.co/t/logstash-doesnt-install-plugin-from-local-gem/123991)
**Author:** [@rons1404](https://discuss.elastic.co/u/rons1404)\ **Replies:** 0\ **Last updated:** [March 14, 2018, 8:05pm UTC](https://discuss.elastic.co/t/logstash-doesnt-install-plugin-from-local-gem/123991 "2018-03-14T20:05:02Z")
Hello, I need to use logstash-output-jdbc with some modification, so I've done the following steps: I've cloned logstash-output-jdbc plugin. I've done changes to the code in jdbc.rb and built a gem using: gem build l… --- ## [Access information inside an object](https://discuss.elastic.co/t/access-information-inside-an-object/123922)
**Author:** [@bkaderlof](https://discuss.elastic.co/u/bkaderlof)\ **Replies:** 3\ **Last updated:** [March 14, 2018, 7:08pm UTC](https://discuss.elastic.co/t/access-information-inside-an-object/123922 "2018-03-14T19:08:19Z")
i'm having trouble accessing data inside an object, this is the original input { "L4\_DST\_PORT" =\> 10082, "IN\_PKTS" =\> 4115, "SRC\_IP\_LOCATION" =\> \[ \[0\] -84.497803, \[1\] 38.0462 \], "DST\_IP\_LOCATION" =\> \[ \[0\] 0.0, … --- ## [How to use the fields collection in prospector with filter expression in logstash](https://discuss.elastic.co/t/how-to-use-the-fields-collection-in-prospector-with-filter-expression-in-logstash/119322)
**Author:** [@stephenpatten](https://discuss.elastic.co/u/stephenpatten)\ **Replies:** 8\ **Last updated:** [March 14, 2018, 6:45pm UTC](https://discuss.elastic.co/t/how-to-use-the-fields-collection-in-prospector-with-filter-expression-in-logstash/119322 "2018-03-14T18:45:14Z")
Hello, I'm running 6.1.2. Given this prospector: - type: log enabled: true multiline.pattern: '^\[0-9\]{4}-\[0-9\]{2}-\[0-9\]{2}' multiline.negate: true multiline.match: after ignore\_older: 24h fields: envir… --- ## [\_grokparsefailure exception with custom pattern](https://discuss.elastic.co/t/grokparsefailure-exception-with-custom-pattern/123962)
**Author:** [@sehrish\_naeem\_naeem](https://discuss.elastic.co/u/sehrish_naeem_naeem)\ **Replies:** 5\ **Last updated:** [March 14, 2018, 6:25pm UTC](https://discuss.elastic.co/t/grokparsefailure-exception-with-custom-pattern/123962 "2018-03-14T18:25:21Z")
Hi, I am newbie and trying to add a field as "cid" in with my logs based on this pattern 2018-03-14 22:26:57.268 INFO 22996 --- \[nio-8090-exec-2\] com.example.ms1.service.ValidatorImpl : validation completed with ci… --- ## [Logstash just started crashing with no changes to any config files](https://discuss.elastic.co/t/logstash-just-started-crashing-with-no-changes-to-any-config-files/123967)
**Author:** [@lolcatlolcat](https://discuss.elastic.co/u/lolcatlolcat)\ **Replies:** 0\ **Last updated:** [March 14, 2018, 5:48pm UTC](https://discuss.elastic.co/t/logstash-just-started-crashing-with-no-changes-to-any-config-files/123967 "2018-03-14T17:48:20Z")
All of a sudden this morning started getting this error messages in the Logstash logs. Nothing has changed in any of the configs so no idea what's going on..... Any suggestions? \[2018-03-14T13:44:39,467\]\[INFO \]\[logstash… --- ## [Logstash output mongodb plugin error](https://discuss.elastic.co/t/logstash-output-mongodb-plugin-error/123442)
**Author:** [@fafahei](https://discuss.elastic.co/u/fafahei)\ **Replies:** 2\ **Last updated:** [March 14, 2018, 5:18pm UTC](https://discuss.elastic.co/t/logstash-output-mongodb-plugin-error/123442 "2018-03-14T17:18:23Z")
when I use logstash to collect data from syslog to mongodb and filter is dissect or grok, logstash log show error and can't transfer data, below is error message: \[2018-03-12T02:16:59,739\]\[WARN \]\[logstash.outputs.mongod… --- ## [Create index by field value](https://discuss.elastic.co/t/create-index-by-field-value/123943)
**Author:** [@jspeer](https://discuss.elastic.co/u/jspeer)\ **Replies:** 1\ **Last updated:** [March 14, 2018, 4:32pm UTC](https://discuss.elastic.co/t/create-index-by-field-value/123943 "2018-03-14T16:32:01Z")
I've already setup a template for apache-\*, is it possible to have an index created for a field value that comes in from one of the filebeats clients? example: Apache QA servers insert the following field: fields: app… --- ## [Sqlserver database overwhelmed by logstash jdbc input](https://discuss.elastic.co/t/sqlserver-database-overwhelmed-by-logstash-jdbc-input/123951)
**Author:** [@Victor\_Dusautois](https://discuss.elastic.co/u/Victor_Dusautois)\ **Replies:** 0\ **Last updated:** [March 14, 2018, 4:27pm UTC](https://discuss.elastic.co/t/sqlserver-database-overwhelmed-by-logstash-jdbc-input/123951 "2018-03-14T16:27:31Z")
Hi, I am trying to get data from a MSSQL database using logstash jdbc-input. The database seems overwhelmed by the requests made by logstash, creates a lot of fake\_worktable, and access to the database for other users a… --- ## [Warning message](https://discuss.elastic.co/t/warning-message/123646)
**Author:** [@wakame](https://discuss.elastic.co/u/wakame)\ **Replies:** 2\ **Last updated:** [March 14, 2018, 2:21pm UTC](https://discuss.elastic.co/t/warning-message/123646 "2018-03-14T14:21:35Z")
I am getting the following message when trying to send data to ES. \[2018-03-12T21:32:21,710\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch......"error"=\>{"type"=\>"mapper\_parsing\_exception… --- ## [Filter CSV and LF](https://discuss.elastic.co/t/filter-csv-and-lf/123894)
**Author:** [@adolfo.diaz](https://discuss.elastic.co/u/adolfo.diaz)\ **Replies:** 1\ **Last updated:** [March 14, 2018, 2:04pm UTC](https://discuss.elastic.co/t/filter-csv-and-lf/123894 "2018-03-14T14:04:14Z")
Hi friends. We have detected that when we are importing CSV files the last line in the CSV is not loaded in elasticsearch. We have verified that the csv files that arrive us to load do not bring "\\ n" to the end of the… --- ## [Logstash date filter, unable to match with pattern defined in a field](https://discuss.elastic.co/t/logstash-date-filter-unable-to-match-with-pattern-defined-in-a-field/123893)
**Author:** [@janl](https://discuss.elastic.co/u/janl)\ **Replies:** 0\ **Last updated:** [March 14, 2018, 11:44am UTC](https://discuss.elastic.co/t/logstash-date-filter-unable-to-match-with-pattern-defined-in-a-field/123893 "2018-03-14T11:44:35Z")
Version 6.1.2 I'm attempting to parse timestamps from various logsources using the date filter. Due to it being multiple logsources with very similar timestamp formats(think 2018/03/14 vs 2018/14/03) I wanted to defi… --- ## [Logstash Failed to excute action](https://discuss.elastic.co/t/logstash-failed-to-excute-action/123753)
**Author:** [@Invictus](https://discuss.elastic.co/u/Invictus)\ **Replies:** 4\ **Last updated:** [March 14, 2018, 1:08pm UTC](https://discuss.elastic.co/t/logstash-failed-to-excute-action/123753 "2018-03-14T13:08:12Z")
Hi, i am running ELK on docker. Every container is running and up. Exept logstash. 0dbb793596af dockerelk\_elasticsearch "/usr/local/bin/dock…" 28 hours ago Up About an hour … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=800) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=802)