# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=881 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 882 --- ## [Regarding conf file configuration](https://discuss.elastic.co/t/regarding-conf-file-configuration/106075)
**Author:** [@prasad123](https://discuss.elastic.co/u/prasad123)\ **Replies:** 10\ **Last updated:** [November 3, 2017, 6:20am UTC](https://discuss.elastic.co/t/regarding-conf-file-configuration/106075 "2017-11-03T06:20:29Z")
i want to read live logs from server what are the configuration i need to set in logstash conf file --- ## [How to update Mulitple Documents of an index using logstash?](https://discuss.elastic.co/t/how-to-update-mulitple-documents-of-an-index-using-logstash/105613)
**Author:** [@MariumHassan](https://discuss.elastic.co/u/MariumHassan)\ **Replies:** 0\ **Last updated:** [October 27, 2017, 8:20pm UTC](https://discuss.elastic.co/t/how-to-update-mulitple-documents-of-an-index-using-logstash/105613 "2017-10-27T20:20:18Z")
Hi all, I want to update multiple documents when I get a log with the status "FILE\_PROCESSING\_ERROR". I have set document ids. I have also stored some relevant ids. In the output plugin I have done something like this … --- ## [How do I get the first 2 lines of a multiline log using logstash 2.4.0?](https://discuss.elastic.co/t/how-do-i-get-the-first-2-lines-of-a-multiline-log-using-logstash-2-4-0/106260)
**Author:** [@Jerry\_Hoffmeister](https://discuss.elastic.co/u/Jerry_Hoffmeister)\ **Replies:** 0\ **Last updated:** [November 2, 2017, 11:11pm UTC](https://discuss.elastic.co/t/how-do-i-get-the-first-2-lines-of-a-multiline-log-using-logstash-2-4-0/106260 "2017-11-02T23:11:58Z")
Hi... I figured out how to get the first line using this post: How do I get first line from multiline message by using logstash 1.5? using this config: filter { if \[type\] == "java" { grok { # Do multiline… --- ## [File output plugin - index not created](https://discuss.elastic.co/t/file-output-plugin-index-not-created/106255)
**Author:** [@lambomania](https://discuss.elastic.co/u/lambomania)\ **Replies:** 1\ **Last updated:** [November 2, 2017, 10:53pm UTC](https://discuss.elastic.co/t/file-output-plugin-index-not-created/106255 "2017-11-02T22:53:58Z")
Hello, I am an ELK noob. I am running Elastic Search 5.6.2 and Logstash 5.6.2. I am using the file input plug and output to elastic search with no filters, but having no luck getting the output into ES. I started logs… --- ## [Methods in Logstash Ruby](https://discuss.elastic.co/t/methods-in-logstash-ruby/102144)
**Author:** [@fluffierbump](https://discuss.elastic.co/u/fluffierbump)\ **Replies:** 6\ **Last updated:** [November 2, 2017, 9:18pm UTC](https://discuss.elastic.co/t/methods-in-logstash-ruby/102144 "2017-11-02T21:18:11Z")
I'm trying to get methods working in Logstash Ruby and I'm starting with a very simple one def output\_something(value) puts value end Then I call the method and try to create a field with a value from it event.set… --- ## [Using ActionView from rails in custom filter plugin](https://discuss.elastic.co/t/using-actionview-from-rails-in-custom-filter-plugin/106248)
**Author:** [@ryanmcclure4](https://discuss.elastic.co/u/ryanmcclure4)\ **Replies:** 0\ **Last updated:** [November 2, 2017, 8:51pm UTC](https://discuss.elastic.co/t/using-actionview-from-rails-in-custom-filter-plugin/106248 "2017-11-02T20:51:36Z")
I'm trying to write a filter plugin which will strip html tags from a given event property. I'd rather not re-invent the wheel so I was trying to find a gem that already included this function. I found 'sanitizer' and sp… --- ## [Hp 3par syslog](https://discuss.elastic.co/t/hp-3par-syslog/106239)
**Author:** [@cdra](https://discuss.elastic.co/u/cdra)\ **Replies:** 0\ **Last updated:** [November 2, 2017, 6:52pm UTC](https://discuss.elastic.co/t/hp-3par-syslog/106239 "2017-11-02T18:52:23Z")
Our HP 3PAR send syslog messages like: \<14\>Nov 2 15:42:51 3PAR\_1629523 new\_obj sw\_user\_conn:26395:192.168.9.91:3parservice User Connection 26395(192.168.9.91:3parservice) added I know get priority, date, hostname.....b… --- ## [Date Parse failuer](https://discuss.elastic.co/t/date-parse-failuer/106164)
**Author:** [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Replies:** 5\ **Last updated:** [November 2, 2017, 5:59pm UTC](https://discuss.elastic.co/t/date-parse-failuer/106164 "2017-11-02T17:59:26Z")
Hello All, I am tying to create a index below are the config file and sample log. Please let me if anything wrong in my config file. Thanks for your help! Log : 2017-10-02 01:00:02,782 DEBUG \[com.\_360commerce.founda… --- ## [Logstash keeps crashing using the JDBC Plugin](https://discuss.elastic.co/t/logstash-keeps-crashing-using-the-jdbc-plugin/106146)
**Author:** [@bilpor](https://discuss.elastic.co/u/bilpor)\ **Replies:** 1\ **Last updated:** [November 2, 2017, 3:56pm UTC](https://discuss.elastic.co/t/logstash-keeps-crashing-using-the-jdbc-plugin/106146 "2017-11-02T15:56:21Z")
Hi All, I have created the following index: put newsindex { "settings" : { "number\_of\_shards":3, "number\_of\_replicas":2 }, "mappings" : { "news": { "properties": { "data": { "t… --- ## [S3 plugin with path wildcard support](https://discuss.elastic.co/t/s3-plugin-with-path-wildcard-support/103550)
**Author:** [@djoels](https://discuss.elastic.co/u/djoels)\ **Replies:** 2\ **Last updated:** [November 2, 2017, 2:36pm UTC](https://discuss.elastic.co/t/s3-plugin-with-path-wildcard-support/103550 "2017-11-02T14:36:27Z")
The same message has been popping up for a while now: it would be really handy if an existing ELK setup could be moved to the cloud, and part of this is of course porting the file plugin to s3 (or am I missing another op… --- ## [Logstash aggregate plugin problem](https://discuss.elastic.co/t/logstash-aggregate-plugin-problem/106204)
**Author:** [@lkofman](https://discuss.elastic.co/u/lkofman)\ **Replies:** 0\ **Last updated:** [November 2, 2017, 1:49pm UTC](https://discuss.elastic.co/t/logstash-aggregate-plugin-problem/106204 "2017-11-02T13:49:02Z")
I work with ELK 6 after that I have installed aggregate plugin -logstash refuses to work: C:\\DoLense\\logstash\\bin\>gem install --local -f logstash-filter-aggregate-2.6.4.gem Successfully installed logstash-filter-aggre… --- ## [What is the best way to reprocess a processed file through logstash](https://discuss.elastic.co/t/what-is-the-best-way-to-reprocess-a-processed-file-through-logstash/106157)
**Author:** [@Dimitry\_Khan](https://discuss.elastic.co/u/Dimitry_Khan)\ **Replies:** 1\ **Last updated:** [November 2, 2017, 1:02pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-reprocess-a-processed-file-through-logstash/106157 "2017-11-02T13:02:30Z")
Dear All, my case is like this. I am processing a file through logstash. But for some reason I want to delete the old records in ES and want to reprocess that file. As I found - I need to keep the setting as - start\_posi… --- ## [Parsing field to multiple fields and inserting into another field](https://discuss.elastic.co/t/parsing-field-to-multiple-fields-and-inserting-into-another-field/106200)
**Author:** [@GambitK](https://discuss.elastic.co/u/GambitK)\ **Replies:** 0\ **Last updated:** [November 2, 2017, 1:01pm UTC](https://discuss.elastic.co/t/parsing-field-to-multiple-fields-and-inserting-into-another-field/106200 "2017-11-02T13:01:19Z")
I have two fields I want to modify, sql\_bind and sql\_text. The fields contain sql statement from oracle logs: "sql\_bind":" #1(8):25793401" "sql\_text":"delete from table01 where num01 = :b1" "sql\_bind":" #1(8):24834… --- ## [Outputting Metrics while respecting Dynamic Naming Conventions](https://discuss.elastic.co/t/outputting-metrics-while-respecting-dynamic-naming-conventions/106179)
**Author:** [@Ryanbarnes](https://discuss.elastic.co/u/Ryanbarnes)\ **Replies:** 3\ **Last updated:** [November 2, 2017, 12:36pm UTC](https://discuss.elastic.co/t/outputting-metrics-while-respecting-dynamic-naming-conventions/106179 "2017-11-02T12:36:06Z")
Hello, I've been unable to reference a dynamic field name in my output. filter{ metrics { meter =\> \["%{request}"\] add\_tag =\> "requests" flush\_interval =\> "5" clear\_interval =\> "60" } } output { if "requests… --- ## [Sending logs from logstash to syslog-ng](https://discuss.elastic.co/t/sending-logs-from-logstash-to-syslog-ng/104309)
**Author:** [@Lakshman\_Mukkamalla](https://discuss.elastic.co/u/Lakshman_Mukkamalla)\ **Replies:** 4\ **Last updated:** [November 2, 2017, 12:15pm UTC](https://discuss.elastic.co/t/sending-logs-from-logstash-to-syslog-ng/104309 "2017-11-02T12:15:03Z")
Hi, I have a setup where i am able to send the logs from Log Files -\> Filebeat -\> Logstash -\> rsyslog server. But the customer has a syslog-ng as the server, have tried to work with the above pipeline with the logstash… --- ## [Unable to load lat and lon values in map view](https://discuss.elastic.co/t/unable-to-load-lat-and-lon-values-in-map-view/106117)
**Author:** [@arun\_s](https://discuss.elastic.co/u/arun_s)\ **Replies:** 5\ **Last updated:** [November 2, 2017, 11:14am UTC](https://discuss.elastic.co/t/unable-to-load-lat-and-lon-values-in-map-view/106117 "2017-11-02T11:14:38Z")
Hi I am using kibana 5.6.3 and i am facing problem while creating coordinate map after loading index. I am getting error as No Compatible Fields: The "geo4-\*" index pattern does not contain any of the following field t… --- ## [Transfer logstash data to elasticsearch through HA proxy](https://discuss.elastic.co/t/transfer-logstash-data-to-elasticsearch-through-ha-proxy/106151)
**Author:** [@vuxuanlai](https://discuss.elastic.co/u/vuxuanlai)\ **Replies:** 3\ **Last updated:** [November 2, 2017, 11:13am UTC](https://discuss.elastic.co/t/transfer-logstash-data-to-elasticsearch-through-ha-proxy/106151 "2017-11-02T11:13:27Z")
Hi there, I have a logstash server, a elasticsearch server, a HA proxy server. I want to transfer data from LS to ES through HA proxy as flow: LS(10.0.0.2) ------\> HA proxy(172.0.0.2) -------\> ES(192.168.56.2) I just … --- ## [Logstash XML file with multiple patterns](https://discuss.elastic.co/t/logstash-xml-file-with-multiple-patterns/105914)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 4\ **Last updated:** [November 2, 2017, 11:01am UTC](https://discuss.elastic.co/t/logstash-xml-file-with-multiple-patterns/105914 "2017-11-02T11:01:17Z")
I have an xml file as an input in Logstash and ElasticSeach as output. My xml has multiple patterns, where my xml file looks like: \< Header\> \< Value1\>xyz\< /Value1\> \< FileName\>name\< /FileName\> \< System\>test\< /System\> … --- ## [Logstash doesn't recognize type](https://discuss.elastic.co/t/logstash-doesnt-recognize-type/106171)
**Author:** [@lkofman](https://discuss.elastic.co/u/lkofman)\ **Replies:** 0\ **Last updated:** [November 2, 2017, 10:37am UTC](https://discuss.elastic.co/t/logstash-doesnt-recognize-type/106171 "2017-11-02T10:37:11Z")
logstash doesn't recognize type -document\_type in filebeat ',even instead type I set document\_type filebeat: paths: - C:\\data\\XL\\ABPAuditLog\_\*.log input\_type: log document\_type: xlpc exclude\_lines: \["Fi… --- ## [Cant feed data into ES](https://discuss.elastic.co/t/cant-feed-data-into-es/106168)
**Author:** [@RandomDude](https://discuss.elastic.co/u/RandomDude)\ **Replies:** 0\ **Last updated:** [November 2, 2017, 10:15am UTC](https://discuss.elastic.co/t/cant-feed-data-into-es/106168 "2017-11-02T10:15:26Z")
Hello, I have setup a server with ES, kibana and logstash. When I try to feed data to ES using logstash, I get \[2017-11-02T10:38:55,197\]\[INFO \]\[logstash.outputs.elasticsearch\] Running health check to see if an Elastic… --- ## [How to import blob type from mysql to es using logstash](https://discuss.elastic.co/t/how-to-import-blob-type-from-mysql-to-es-using-logstash/106136)
**Author:** [@Canappeco](https://discuss.elastic.co/u/Canappeco)\ **Replies:** 5\ **Last updated:** [November 2, 2017, 10:05am UTC](https://discuss.elastic.co/t/how-to-import-blob-type-from-mysql-to-es-using-logstash/106136 "2017-11-02T10:05:11Z")
Hi all. I am importing data from mysql to es using logstash. I have a blob type in a table in mysql. Because I used to convert the array of double to binary and save as blob. Now I want to import this type of data to es… --- ## [Extract special part from access.log file by Logstash](https://discuss.elastic.co/t/extract-special-part-from-access-log-file-by-logstash/105513)
**Author:** [@feng\_17](https://discuss.elastic.co/u/feng_17)\ **Replies:** 2\ **Last updated:** [November 2, 2017, 9:32am UTC](https://discuss.elastic.co/t/extract-special-part-from-access-log-file-by-logstash/105513 "2017-11-02T09:32:20Z")
Dear Friends： I want to use logstash to extract a special part from access.log file.For instance,to extract all the logs that @timestamp=="2017-10-24T16:41:16+08:00" from the target access.log file to a new file.That is… --- ## [Failure running tests for logstash-filter-geoip plugin - logging configuration not loading](https://discuss.elastic.co/t/failure-running-tests-for-logstash-filter-geoip-plugin-logging-configuration-not-loading/103949)
**Author:** [@nzjess](https://discuss.elastic.co/u/nzjess)\ **Replies:** 10\ **Last updated:** [November 2, 2017, 9:28am UTC](https://discuss.elastic.co/t/failure-running-tests-for-logstash-filter-geoip-plugin-logging-configuration-not-loading/103949 "2017-11-02T09:28:31Z")
I cloned the geoip plugin source from github: https://github.com/logstash-plugins/logstash-filter-geoip/commit/921d1cd Followed the README to install the pre-reqs: bundle install bundle exec rake vendor Then ran the … --- ## [Logstash dissect plugin prase doesn't correct with empty field in content](https://discuss.elastic.co/t/logstash-dissect-plugin-prase-doesnt-correct-with-empty-field-in-content/104992)
**Author:** [@sockaddr\_in](https://discuss.elastic.co/u/sockaddr_in)\ **Replies:** 5\ **Last updated:** [November 2, 2017, 8:55am UTC](https://discuss.elastic.co/t/logstash-dissect-plugin-prase-doesnt-correct-with-empty-field-in-content/104992 "2017-11-02T08:55:55Z")
@zikakou1er @Allwyn @matslats @Kubson @fcza I use the dissect plugin with logstash to parse the log which has empty fileds, but is doesn't match fields correctly.The folowing is my log pattern and logstash config file a… --- ## [Issue starting Logstash](https://discuss.elastic.co/t/issue-starting-logstash/106139)
**Author:** [@akshay](https://discuss.elastic.co/u/akshay)\ **Replies:** 2\ **Last updated:** [November 2, 2017, 8:49am UTC](https://discuss.elastic.co/t/issue-starting-logstash/106139 "2017-11-02T08:49:41Z")
D:\\logstash-5.6.3\>bin\\logstash.bat -f logstash.conf LoadError: load error: com/jrjackson/jr\_jackson -- java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/SerializerProvider require at org/jruby/RubyKernel.j… --- ## [What is the difference between \*:logstash-\* & logstash-\*](https://discuss.elastic.co/t/what-is-the-difference-between-logstash-logstash/106115)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 7\ **Last updated:** [November 2, 2017, 8:05am UTC](https://discuss.elastic.co/t/what-is-the-difference-between-logstash-logstash/106115 "2017-11-02T08:05:57Z")
Hi Guys, Wondering what is the difference between creating index pattern :logstash- and logstash-\* other than index names starting with any:logstatsh? I mean not sure if there is any other significance? --- ## [What happens when logstash can't send data to Elastic](https://discuss.elastic.co/t/what-happens-when-logstash-cant-send-data-to-elastic/106078)
**Author:** [@sbade](https://discuss.elastic.co/u/sbade)\ **Replies:** 1\ **Last updated:** [November 2, 2017, 7:27am UTC](https://discuss.elastic.co/t/what-happens-when-logstash-cant-send-data-to-elastic/106078 "2017-11-02T07:27:13Z")
In a scenario where logstash is sending data, and there is a network problem where it can not send for some period of time, what happens to the data. Is it cached so that when connectivity is restored the data flows? --- ## [Message has not been splitted to fields](https://discuss.elastic.co/t/message-has-not-been-splitted-to-fields/106048)
**Author:** [@lkofman](https://discuss.elastic.co/u/lkofman)\ **Replies:** 3\ **Last updated:** [November 2, 2017, 7:20am UTC](https://discuss.elastic.co/t/message-has-not-been-splitted-to-fields/106048 "2017-11-02T07:20:36Z")
The grok is right has been tested on grok debugger and splits the message to fields. In logstash.conf in filter message passes grok and stays not splitted. On Kibana I see message in quotes and not splitted , even nam… --- ## [Logstash is creating a directory literally called ${sys:ls.logs}](https://discuss.elastic.co/t/logstash-is-creating-a-directory-literally-called-sys-ls-logs/100126)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 6\ **Last updated:** [November 1, 2017, 7:26pm UTC](https://discuss.elastic.co/t/logstash-is-creating-a-directory-literally-called-sys-ls-logs/100126 "2017-11-01T19:26:15Z")
When I run logstash 5.5.1 I am used to getting an error ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Followed by the message Sending Logst… --- ## [Grok Filter Parse a Hyphen as absent](https://discuss.elastic.co/t/grok-filter-parse-a-hyphen-as-absent/106030)
**Author:** [@Ewan](https://discuss.elastic.co/u/Ewan)\ **Replies:** 1\ **Last updated:** [November 1, 2017, 3:36pm UTC](https://discuss.elastic.co/t/grok-filter-parse-a-hyphen-as-absent/106030 "2017-11-01T15:36:59Z")
Hi, I'm trying to parse a log which will insert a hyphen for some fields when there is no value: Is there an easy way with grok to treat the hyphen as if the value is absent? i.e. not add the hyphen as a value, but con… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=880) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=882)