# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=818 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 819 --- ## [Logstash SPLIT plugin discards/strips empty strings at the end](https://discuss.elastic.co/t/logstash-split-plugin-discards-strips-empty-strings-at-the-end/120473)
**Author:** [@BariscanElkama](https://discuss.elastic.co/u/BariscanElkama)\ **Replies:** 1\ **Last updated:** [February 20, 2018, 3:04am UTC](https://discuss.elastic.co/t/logstash-split-plugin-discards-strips-empty-strings-at-the-end/120473 "2018-02-20T03:04:34Z")
Hello, I'm trying to parse a log data where each line is constructed with string fields that are separated with a "/t" delimeter. I'm forwarding logs with Filebeat to Logstash and in logstash, I get each line as a GREE… --- ## [Issue while starting Logstash 6.0 on windows 10](https://discuss.elastic.co/t/issue-while-starting-logstash-6-0-on-windows-10/120362)
**Author:** [@Amol\_Vishwarupe](https://discuss.elastic.co/u/Amol_Vishwarupe)\ **Replies:** 3\ **Last updated:** [February 19, 2018, 11:46pm UTC](https://discuss.elastic.co/t/issue-while-starting-logstash-6-0-on-windows-10/120362 "2018-02-19T23:46:42Z")
I am getting below error which starting logstash agent on window 10 , can someone please help here . C:\\HomeWare\\ES6\\logstash-6.0.0\\logstash-6.0.0\\bin\>logstash.bat agent -f test.config LoadError: Could not load FFI Pro… --- ## [Create es index alias from logstash pipeline conf](https://discuss.elastic.co/t/create-es-index-alias-from-logstash-pipeline-conf/120531)
**Author:** [@Arjun\_subramanyam\_va](https://discuss.elastic.co/u/Arjun_subramanyam_va)\ **Replies:** 0\ **Last updated:** [February 19, 2018, 10:50pm UTC](https://discuss.elastic.co/t/create-es-index-alias-from-logstash-pipeline-conf/120531 "2018-02-19T22:50:19Z")
Hi I want to automate the alias creation for es index, currently I specify index name in elasticsearch output plugin, similarly can i specify index alias in the conf file or template file ? how to do this efficiently, I… --- ## [Http output to ScienceLogic API](https://discuss.elastic.co/t/http-output-to-sciencelogic-api/119501)
**Author:** [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Replies:** 1\ **Last updated:** [February 19, 2018, 10:38pm UTC](https://discuss.elastic.co/t/http-output-to-sciencelogic-api/119501 "2018-02-19T22:38:52Z")
Hi all, Trying to integrate a 3rd party API and Logstash 6.2.1 I can do a simple post and create a message in the system but having trouble replicating this in the logstash http output. This works from the command lin… --- ## [Logstash not starting properly](https://discuss.elastic.co/t/logstash-not-starting-properly/120235)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 12\ **Last updated:** [February 19, 2018, 9:27pm UTC](https://discuss.elastic.co/t/logstash-not-starting-properly/120235 "2018-02-19T21:27:28Z")
Hello there, I am trying to start LogStash 5.6 with /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d but when I do, I get several of these errors \[ERROR\] 2018-02-16 13:50:24.556 \[\[main\]\ **Author:** [@Jason\_Brooks](https://discuss.elastic.co/u/Jason_Brooks)\ **Replies:** 3\ **Last updated:** [February 19, 2018, 8:02pm UTC](https://discuss.elastic.co/t/run-logstash-in-batch-mode-signals/120246 "2018-02-19T20:02:31Z") Hello, I have learned how to do some data conversion with logstash. Eventually, I would like to piggyback tasks on to an existing logstash service, but for now I would like to run logstash on a configuration and exit w… --- ## [Grok Filter Error](https://discuss.elastic.co/t/grok-filter-error/120518)
**Author:** [@MrThecrackerx](https://discuss.elastic.co/u/MrThecrackerx)\ **Replies:** 0\ **Last updated:** [February 19, 2018, 7:29pm UTC](https://discuss.elastic.co/t/grok-filter-error/120518 "2018-02-19T19:29:26Z")
Hi I am a bit newbie here but i am trying to parse one of the syslog message IDs for cisco ASA. the syslog message id is 609001 input { stdin {} } filter { grok { match =\> \[ "messag… --- ## [Logstash batch: write to a csv file and close it, or exit when done](https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245)
**Author:** [@Jason\_Brooks](https://discuss.elastic.co/u/Jason_Brooks)\ **Replies:** 3\ **Last updated:** [February 19, 2018, 6:51pm UTC](https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245 "2018-02-19T18:51:44Z")
Hello, I have used logstash to build csv files, but I noticed i needed to exit logstash to be certain the output file is closed. While I am ok with this, it seems silly to have to shutdown logstash. The enclosed examp… --- ## [Dissect extract to metadata field](https://discuss.elastic.co/t/dissect-extract-to-metadata-field/120508)
**Author:** [@ave19](https://discuss.elastic.co/u/ave19)\ **Replies:** 0\ **Last updated:** [February 19, 2018, 6:34pm UTC](https://discuss.elastic.co/t/dissect-extract-to-metadata-field/120508 "2018-02-19T18:34:15Z")
Can I tell the dissect filter to extract a field to @metadata? dissect { mapping =\> { "message" =\> "%{\[@metadata\]\[foo\]} %{+\[@metadata\]\[foo\]} %{actual-foo} %{message}" } } --- ## [Advice on dynamically configuring logstash 6.2 output plugin](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 8\ **Last updated:** [February 19, 2018, 3:48pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486 "2018-02-19T15:48:47Z")
Hi Team, What is the recommended way to dynamically tell logstash to which elasticsearch hosts logstash should connect to? I have several hosts where I run logstash. However for some hosts I need the data to get to clu… --- ## [Ruby filter in Logstash, how to pass parameters to external Ruby script](https://discuss.elastic.co/t/ruby-filter-in-logstash-how-to-pass-parameters-to-external-ruby-script/120333)
**Author:** [@Heckler\_GlobalOperat](https://discuss.elastic.co/u/Heckler_GlobalOperat)\ **Replies:** 2\ **Last updated:** [February 19, 2018, 3:09pm UTC](https://discuss.elastic.co/t/ruby-filter-in-logstash-how-to-pass-parameters-to-external-ruby-script/120333 "2018-02-19T15:09:04Z")
Hi everyone, I'm preparing a Logstash configuration which use Grok to extract fields from incoming messages and Ruby to process those fields. Below is a simple version of my configuration, I want to extract the Timesta… --- ## [Logstash installed successfully and also service started but read and write functionality is not working](https://discuss.elastic.co/t/logstash-installed-successfully-and-also-service-started-but-read-and-write-functionality-is-not-working/120462)
**Author:** [@Rajkumar\_idsil](https://discuss.elastic.co/u/Rajkumar_idsil)\ **Replies:** 0\ **Last updated:** [February 19, 2018, 1:25pm UTC](https://discuss.elastic.co/t/logstash-installed-successfully-and-also-service-started-but-read-and-write-functionality-is-not-working/120462 "2018-02-19T13:25:13Z")
Hi, I am using window 7, step 1. i have installed elasticsearch-6.2.1 in my system and tested using default port localhost:9200 and it is working perfectly step 2 . after that i have installed logstash-6.2.1 using ns… --- ## [Create new event via ruby filter plugin?](https://discuss.elastic.co/t/create-new-event-via-ruby-filter-plugin/120458)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 0\ **Last updated:** [February 19, 2018, 1:08pm UTC](https://discuss.elastic.co/t/create-new-event-via-ruby-filter-plugin/120458 "2018-02-19T13:08:59Z")
Hi all, is it possible to create a new event via ruby filter plugin? Idea is to set all needed fields in ruby and save them to a new event. I would like to do something similar like clone or split filter. Thanks, Andr… --- ## [Kafka - Commit offset only after batch has been processed](https://discuss.elastic.co/t/kafka-commit-offset-only-after-batch-has-been-processed/120445)
**Author:** [@shaharmor](https://discuss.elastic.co/u/shaharmor)\ **Replies:** 1\ **Last updated:** [February 19, 2018, 12:20pm UTC](https://discuss.elastic.co/t/kafka-commit-offset-only-after-batch-has-been-processed/120445 "2018-02-19T12:20:57Z")
Hey, I understand after reading the code of the logstash-input-kafka plugin, that the offset it being committed after the batch has been transferred to the logstash internal queue, but not actually sent to the output (I… --- ## [Parse JSON input into fields](https://discuss.elastic.co/t/parse-json-input-into-fields/120236)
**Author:** [@exceltior](https://discuss.elastic.co/u/exceltior)\ **Replies:** 4\ **Last updated:** [February 19, 2018, 11:14am UTC](https://discuss.elastic.co/t/parse-json-input-into-fields/120236 "2018-02-19T11:14:12Z")
Hi, I have been trying to turn a JSON blob keys, which I receive from input into data fields but I have been unsuccessful for some hours. JSON Blob fed to input { "timestamp": "\[16/Feb/2018:19:19:03 +0000\]", "@ver… --- ## [Debugging questions + advice request](https://discuss.elastic.co/t/debugging-questions-advice-request/120340)
**Author:** [@aclowkey](https://discuss.elastic.co/u/aclowkey)\ **Replies:** 3\ **Last updated:** [February 19, 2018, 10:37am UTC](https://discuss.elastic.co/t/debugging-questions-advice-request/120340 "2018-02-19T10:37:41Z")
Hi, I've started using Logstash coming from different pipeline products and I find it hard to understand what's wrong with my pipelines when it doesn't behave as expected. For example, I've copied one simple JDBC input… --- ## [Migration of elk on docker](https://discuss.elastic.co/t/migration-of-elk-on-docker/120402)
**Author:** [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\ **Replies:** 9\ **Last updated:** [February 19, 2018, 10:00am UTC](https://discuss.elastic.co/t/migration-of-elk-on-docker/120402 "2018-02-19T10:00:52Z")
Hello I am new to docker also kind of a beginner for elk. Till now I was using windows for elk now was willing to migrate to docker can some one guide me how can I migrate and run my config file of logstash from my loc… --- ## [Unable to test pipeline in logstash](https://discuss.elastic.co/t/unable-to-test-pipeline-in-logstash/120313)
**Author:** [@\_PSPKFan](https://discuss.elastic.co/u/_PSPKFan)\ **Replies:** 1\ **Last updated:** [February 19, 2018, 8:56am UTC](https://discuss.elastic.co/t/unable-to-test-pipeline-in-logstash/120313 "2018-02-19T08:56:24Z")
I'm new to ELK stack. I'm able to install and run the logstash, Elasticsearch, Kibana services individually. But when I'm trying to test pipelining in logstatsh by running below command bin/logstash -f ..LogstashPipelin… --- ## [Log rotate not working](https://discuss.elastic.co/t/log-rotate-not-working/120409)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 0\ **Last updated:** [February 19, 2018, 8:07am UTC](https://discuss.elastic.co/t/log-rotate-not-working/120409 "2018-02-19T08:07:22Z")
Hello everyone. I came with this scenario just recently. Before i install xpack, logstash can log rotate my logs in /var/log/logstash directory. Upon checking just recently, i noticed that my last log file was logstash… --- ## [Processing remote ZIP files with Logstash](https://discuss.elastic.co/t/processing-remote-zip-files-with-logstash/120254)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 3\ **Last updated:** [February 19, 2018, 7:37am UTC](https://discuss.elastic.co/t/processing-remote-zip-files-with-logstash/120254 "2018-02-19T07:37:10Z")
I'd just like to verify I am attempting this correctly. I am trying to start a Logstash pipeline that first downloads some large (~500MB) ZIP files via http, then unzips them, then processes that data. I am having troubl… --- ## [Parse string to date and have difference between 2 dates](https://discuss.elastic.co/t/parse-string-to-date-and-have-difference-between-2-dates/120219)
**Author:** [@Pierre\_What\_s\_Up\_Sca](https://discuss.elastic.co/u/Pierre_What_s_Up_Sca)\ **Replies:** 10\ **Last updated:** [February 19, 2018, 7:25am UTC](https://discuss.elastic.co/t/parse-string-to-date-and-have-difference-between-2-dates/120219 "2018-02-19T07:25:59Z")
Hi ! I have a string with this format : Fri Feb 16 14:30:09 CET 2018 I init a field like this : mutate { add\_field =\> { "dateLog" =\> "%{message}" } } mutate { gsub =\> \[ "dateLog", "INFOS:\\s", "" \] } mutate { g… --- ## [Date filter for logstash](https://discuss.elastic.co/t/date-filter-for-logstash/120203)
**Author:** [@mugil1988](https://discuss.elastic.co/u/mugil1988)\ **Replies:** 4\ **Last updated:** [February 19, 2018, 7:16am UTC](https://discuss.elastic.co/t/date-filter-for-logstash/120203 "2018-02-19T07:16:35Z")
Hi all, We are able to parse the date in the log file using ISO8601 format. But we are using the same format in date filter.. Not able to use the log date as logstash timestamp in kibana. Log date format - 2018-02-16… --- ## [Sql\_last\_value resets to 0 if logstash restart](https://discuss.elastic.co/t/sql-last-value-resets-to-0-if-logstash-restart/120397)
**Author:** [@Timotius\_Pamungkas](https://discuss.elastic.co/u/Timotius_Pamungkas)\ **Replies:** 1\ **Last updated:** [February 19, 2018, 5:55am UTC](https://discuss.elastic.co/t/sql-last-value-resets-to-0-if-logstash-restart/120397 "2018-02-19T05:55:12Z")
I'm collecting data from jdbc (MS SQL) with this configuration: jdbc { jdbc\_driver\_library =\> "C:\\dev\\script\\config\\mssql-jdbc-6.2.2.jre8.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" … --- ## [Csv logststash](https://discuss.elastic.co/t/csv-logststash/120393)
**Author:** [@Pororo](https://discuss.elastic.co/u/Pororo)\ **Replies:** 3\ **Last updated:** [February 19, 2018, 5:02am UTC](https://discuss.elastic.co/t/csv-logststash/120393 "2018-02-19T05:02:50Z")
I was trying to import csv file to ES through logstash. But when I looked through the data in kibana, I just found that the first line of csv file(attributes of data) is accidentally recognized as data instance instead o… --- ## [Installing the syslog input plugin offline](https://discuss.elastic.co/t/installing-the-syslog-input-plugin-offline/120324)
**Author:** [@QuizzyRascal](https://discuss.elastic.co/u/QuizzyRascal)\ **Replies:** 1\ **Last updated:** [February 19, 2018, 3:31am UTC](https://discuss.elastic.co/t/installing-the-syslog-input-plugin-offline/120324 "2018-02-19T03:31:18Z")
Hi, I'm trying to install the syslog input plugin in an environment without internet access. I have downloaded the plugin zip from GitHub and unzipped into a directory on my Linux box. I have the following command line… --- ## [ES gives me 400 for my index template but its seems to be validate](https://discuss.elastic.co/t/es-gives-me-400-for-my-index-template-but-its-seems-to-be-validate/120278)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 3\ **Last updated:** [February 19, 2018, 2:33am UTC](https://discuss.elastic.co/t/es-gives-me-400-for-my-index-template-but-its-seems-to-be-validate/120278 "2018-02-19T02:33:22Z")
Logstash can connect to the cluster fine and I can query http://mycluster/\_template/ so it looks like the 400 is just about something in the template it doesn't like. I get this error: 2018-02-17T00:35:11,571\]\[INFO \]\[l… --- ## [Silly question 102](https://discuss.elastic.co/t/silly-question-102/120380)
**Author:** [@QuizzyRascal](https://discuss.elastic.co/u/QuizzyRascal)\ **Replies:** 5\ **Last updated:** [February 19, 2018, 1:55am UTC](https://discuss.elastic.co/t/silly-question-102/120380 "2018-02-19T01:55:44Z")
I installed an input plugin today by connecting to the internet and running /use/share/logstash/bin/logstash-plugin install logstash-input-syslog But cannot find a conf file to edit... Is there one? If so where is it?… --- ## [DMARC in ElasticStack](https://discuss.elastic.co/t/dmarc-in-elasticstack/120375)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 6\ **Last updated:** [February 19, 2018, 1:48am UTC](https://discuss.elastic.co/t/dmarc-in-elasticstack/120375 "2018-02-19T01:48:45Z")
I've been on a mission the last few weeks to get ElasticStack to ingest DMARC's aggregate reports that come from remote mail domains as compressed xml files. From the onset, one of the goals I had was to build something… --- ## [Combining Field Values Into New Field](https://discuss.elastic.co/t/combining-field-values-into-new-field/120299)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 2\ **Last updated:** [February 18, 2018, 10:57pm UTC](https://discuss.elastic.co/t/combining-field-values-into-new-field/120299 "2018-02-18T22:57:44Z")
Using the below in pipeline: mutate { add\_field =\> { "geoip.location.coordinates" =\> "%{geoip.location.lat},%{geoip.location.lon}" } } Does not result in something like 37.425,-122.00 but rather %{geo… --- ## [Alternative to File Input](https://discuss.elastic.co/t/alternative-to-file-input/120329)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 2\ **Last updated:** [February 18, 2018, 10:56pm UTC](https://discuss.elastic.co/t/alternative-to-file-input/120329 "2018-02-18T22:56:26Z")
I have static XML files that Logstash ingests. My current pipeline utilizes the File Input with the Multiline codec which works but is not perfect. The File input doesn't fully meet my needs because, I assume, it leave… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=817) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=819)