# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=677 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 678 --- ## [Indice names](https://discuss.elastic.co/t/indice-names/150382)
**Author:** [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Replies:** 2\ **Last updated:** [October 1, 2018, 2:05pm UTC](https://discuss.elastic.co/t/indice-names/150382 "2018-10-01T14:05:02Z")
Have a new ELK installation based on online tutorials which is currently scraping a log sent from a SINGLE generic syslog server. I am new so still very green on how many of the pieces fit together. RTFM is not working… --- ## [Filter cross-pollination](https://discuss.elastic.co/t/filter-cross-pollination/150572)
**Author:** [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\ **Replies:** 1\ **Last updated:** [October 1, 2018, 1:47pm UTC](https://discuss.elastic.co/t/filter-cross-pollination/150572 "2018-10-01T13:47:38Z")
Im having issues with logstash ".conf" (modules?) and keeping their actions separated. My logstash host is getting input from various beats and from collectd. Collectd sends its data via port 25826/UDP so my hope was to… --- ## [Logstash-Elapsed Filter](https://discuss.elastic.co/t/logstash-elapsed-filter/150569)
**Author:** [@Sudharsan\_Ganesh](https://discuss.elastic.co/u/Sudharsan_Ganesh)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 1:05pm UTC](https://discuss.elastic.co/t/logstash-elapsed-filter/150569 "2018-10-01T13:05:22Z")
User Name is not coming, its coming as %{usr} when viewed in Kibana. Here George is the Username. Please let me know what should I do. My requirement is to display jobid, elapsedtime and username in kibana. Thanks in adv… --- ## [After upgrading Logstash to 6.4.1 It could not be started there is already another instance using the configured data directory](https://discuss.elastic.co/t/after-upgrading-logstash-to-6-4-1-it-could-not-be-started-there-is-already-another-instance-using-the-configured-data-directory/150013)
**Author:** [@emna](https://discuss.elastic.co/u/emna)\ **Replies:** 7\ **Last updated:** [October 1, 2018, 12:12pm UTC](https://discuss.elastic.co/t/after-upgrading-logstash-to-6-4-1-it-could-not-be-started-there-is-already-another-instance-using-the-configured-data-directory/150013 "2018-10-01T12:12:57Z")
Hi, After upgrading Logstash to version 6.4.1 using the package managers procedure bellow : https://www.elastic.co/guide/en/logstash/current/upgrading-using-package-managers.html Logstash version has changed successf… --- ## [Can I run logstash service with a custom process name?](https://discuss.elastic.co/t/can-i-run-logstash-service-with-a-custom-process-name/150564)
**Author:** [@Gireesh\_Deepak\_R](https://discuss.elastic.co/u/Gireesh_Deepak_R)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 12:01pm UTC](https://discuss.elastic.co/t/can-i-run-logstash-service-with-a-custom-process-name/150564 "2018-10-01T12:01:38Z")
Hello Everyone, I need to run two logstash instances in a single host (logstash multiple pipelines can't be considered for my use case), and need to setup monitoring on only one instance, so if I run two instances with … --- ## [Logstash unable to connect webservice using PKCS12 file](https://discuss.elastic.co/t/logstash-unable-to-connect-webservice-using-pkcs12-file/150557)
**Author:** [@Android\_Games](https://discuss.elastic.co/u/Android_Games)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 11:03am UTC](https://discuss.elastic.co/t/logstash-unable-to-connect-webservice-using-pkcs12-file/150557 "2018-10-01T11:03:50Z")
I'm trying to connect to XML web service using log stash, but its using PEM file credentials to connect web service here's the config file input { http\_poller { urls =\> { myurl =\> "https://abc.com/SWS/incident… --- ## [Help with mutate gsub filter in logstash](https://discuss.elastic.co/t/help-with-mutate-gsub-filter-in-logstash/150047)
**Author:** [@jason27](https://discuss.elastic.co/u/jason27)\ **Replies:** 3\ **Last updated:** [September 27, 2018, 2:22pm UTC](https://discuss.elastic.co/t/help-with-mutate-gsub-filter-in-logstash/150047 "2018-09-27T14:22:41Z")
Hello, I will like to strip off the last character for a field and I donot know why this is not working in logstash. I have a field named "app\_name" which ends with something like "dashboard\_webapp:" and I will like to… --- ## [Logstash 6.1.2 not showing on kibana 6.4.1 anymore](https://discuss.elastic.co/t/logstash-6-1-2-not-showing-on-kibana-6-4-1-anymore/150550)
**Author:** [@emna](https://discuss.elastic.co/u/emna)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 10:21am UTC](https://discuss.elastic.co/t/logstash-6-1-2-not-showing-on-kibana-6-4-1-anymore/150550 "2018-10-01T10:21:22Z")
Hi, I'm facing a problem in detecting logstash on kibana : Logstash 6.1.2 version Kibana 6.4 version logstash is not generating log files anymore , here is the result of tailing the last log file i found : \[2018-0… --- ## [Logstash Adding Filebeat Fields](https://discuss.elastic.co/t/logstash-adding-filebeat-fields/149538)
**Author:** [@eswarloges](https://discuss.elastic.co/u/eswarloges)\ **Replies:** 5\ **Last updated:** [October 1, 2018, 10:17am UTC](https://discuss.elastic.co/t/logstash-adding-filebeat-fields/149538 "2018-10-01T10:17:27Z")
Hi Team, The design we used to read log files - Filebeat -\> Kafka Topic -\> Logstash -\> Elasticsearch When the message is arriving in Logstash it is in the following format { "@timestamp":"2018-09-20T00:23:48.543Z", … --- ## [Log Warning/Error while parsing pipeline](https://discuss.elastic.co/t/log-warning-error-while-parsing-pipeline/150544)
**Author:** [@prometheus1](https://discuss.elastic.co/u/prometheus1)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 9:59am UTC](https://discuss.elastic.co/t/log-warning-error-while-parsing-pipeline/150544 "2018-10-01T09:59:25Z")
I want to achieve logger.error(), logger.info() type functionality without using a ruby filter. Is it possible to do it in logstash. I shall describe my exact usecase to provide additional clarity: I have if - else blo… --- ## [Logstash jms input server issue](https://discuss.elastic.co/t/logstash-jms-input-server-issue/150534)
**Author:** [@pgprb](https://discuss.elastic.co/u/pgprb)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 8:40am UTC](https://discuss.elastic.co/t/logstash-jms-input-server-issue/150534 "2018-10-01T08:40:32Z")
Hi there, I was trying to enable logstash to connect to a server (through environment variables, this is a Docker custom image) that has Tibco JMS message queues. However, I cannot use those environment variables in th… --- ## [Need help on logstash scalability](https://discuss.elastic.co/t/need-help-on-logstash-scalability/150528)
**Author:** [@anra137](https://discuss.elastic.co/u/anra137)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 8:20am UTC](https://discuss.elastic.co/t/need-help-on-logstash-scalability/150528 "2018-10-01T08:20:24Z")
With single logstash, all successfully running. But when I try to add another logstash, it throws error as below. Logstash pipeline started successfully but error on reaching elasticsearch. I have a elasticsearch cluster… --- ## [How to filter json messages in logstash for a specific value to get store to different indexes](https://discuss.elastic.co/t/how-to-filter-json-messages-in-logstash-for-a-specific-value-to-get-store-to-different-indexes/150524)
**Author:** [@kprakash\_77](https://discuss.elastic.co/u/kprakash_77)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 7:51am UTC](https://discuss.elastic.co/t/how-to-filter-json-messages-in-logstash-for-a-specific-value-to-get-store-to-different-indexes/150524 "2018-10-01T07:51:24Z")
I'm new to logstash filter options and I would like to try some thing like filtering json message based on specific value like key=test1 then move that complete message to test1-index similarly if key=test2 then move tha… --- ## [Installation errors on Raspberry Pi 2](https://discuss.elastic.co/t/installation-errors-on-raspberry-pi-2/150468)
**Author:** [@mc0re](https://discuss.elastic.co/u/mc0re)\ **Replies:** 4\ **Last updated:** [October 1, 2018, 7:12am UTC](https://discuss.elastic.co/t/installation-errors-on-raspberry-pi-2/150468 "2018-10-01T07:12:16Z")
I have troubles installing Logstash on my Raspberry Pi. APT Cannot connect to the APT, using echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list it … --- ## [Logstash filter with fields in the log file itself](https://discuss.elastic.co/t/logstash-filter-with-fields-in-the-log-file-itself/150492)
**Author:** [@iamelkuser](https://discuss.elastic.co/u/iamelkuser)\ **Replies:** 1\ **Last updated:** [October 1, 2018, 5:59am UTC](https://discuss.elastic.co/t/logstash-filter-with-fields-in-the-log-file-itself/150492 "2018-10-01T05:59:13Z")
I have Tomcat log file as below. If you see the log file contains Token Id, Request Id. I would like to filter out (filter in .conf file) data from this log file in such a way that i can see all the logs in one shot wher… --- ## [Logstash Filtering Tutorials](https://discuss.elastic.co/t/logstash-filtering-tutorials/150501)
**Author:** [@Esity](https://discuss.elastic.co/u/Esity)\ **Replies:** 2\ **Last updated:** [October 1, 2018, 5:52am UTC](https://discuss.elastic.co/t/logstash-filtering-tutorials/150501 "2018-10-01T05:52:43Z")
Does anyone have a base repo for different standard logstash filtering? I am trying to learn it but really struggling. I was hoping someone could get me started with some copy paste logstash filtering for processing fil… --- ## [Logstash sometimes skipping files on S3](https://discuss.elastic.co/t/logstash-sometimes-skipping-files-on-s3/150511)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 5:34am UTC](https://discuss.elastic.co/t/logstash-sometimes-skipping-files-on-s3/150511 "2018-10-01T05:34:18Z")
Hi, I'm using the S3 input plug-in to read log files from a S3 folder. This is generally working well but from time to time Logstash skips files. I'm using sincedb to keep track of the processes files and move and dele… --- ## [Possible logstash fingerprint filter issue?](https://discuss.elastic.co/t/possible-logstash-fingerprint-filter-issue/150495)
**Author:** [@dlimantoro](https://discuss.elastic.co/u/dlimantoro)\ **Replies:** 1\ **Last updated:** [October 1, 2018, 1:28am UTC](https://discuss.elastic.co/t/possible-logstash-fingerprint-filter-issue/150495 "2018-10-01T01:28:00Z")
Hi everyone, apology for the generic title, but I really can't quite find the best title for this thread (feel free to suggest me some) Quite recently, I checked my Kibana and found out that I have been missing logfile … --- ## [Stop Greedy Data After New Line](https://discuss.elastic.co/t/stop-greedy-data-after-new-line/150493)
**Author:** [@caesarramy](https://discuss.elastic.co/u/caesarramy)\ **Replies:** 0\ **Last updated:** [October 1, 2018, 12:22am UTC](https://discuss.elastic.co/t/stop-greedy-data-after-new-line/150493 "2018-10-01T00:22:44Z")
How to make greedy data stop after new line or tab in multiline? --- ## [Issue sending logs filebeat to logstash?](https://discuss.elastic.co/t/issue-sending-logs-filebeat-to-logstash/150448)
**Author:** [@killmasta93](https://discuss.elastic.co/u/killmasta93)\ **Replies:** 1\ **Last updated:** [September 30, 2018, 8:42pm UTC](https://discuss.elastic.co/t/issue-sending-logs-filebeat-to-logstash/150448 "2018-09-30T20:42:46Z")
Hi, I was wondering if someone could shed some light on the issue im having running logstash (on server) 6.4.1 and filebeat ( on client) 6.4.1. The idea is to send the postgresql information from the client to the serve… --- ## [Limiting index creation - where to set values?](https://discuss.elastic.co/t/limiting-index-creation-where-to-set-values/150485)
**Author:** [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\ **Replies:** 0\ **Last updated:** [September 30, 2018, 7:04pm UTC](https://discuss.elastic.co/t/limiting-index-creation-where-to-set-values/150485 "2018-09-30T19:04:34Z")
One of my indexes is hitting the "1000 fields" limit pretty regularly and I'd like to calm it down a bit. I've seen this information but I can't figure out how to set these values. Oddly enough (to me) when i go to Kib… --- ## [Problem with paresen response code from Websphere logfile](https://discuss.elastic.co/t/problem-with-paresen-response-code-from-websphere-logfile/150456)
**Author:** [@bab](https://discuss.elastic.co/u/bab)\ **Replies:** 0\ **Last updated:** [September 30, 2018, 7:00am UTC](https://discuss.elastic.co/t/problem-with-paresen-response-code-from-websphere-logfile/150456 "2018-09-30T07:00:16Z")
Hi all, I have a problem with my consumer pattern for response code from ibm websphere, below is explained: that is my logstash Config: if \[fields\]\[log\_type\] == "p8\_server\_error" { grok { match =\> \[ "message","%… --- ## [Logstash don't log more than 500 files](https://discuss.elastic.co/t/logstash-dont-log-more-than-500-files/150440)
**Author:** [@zabaala](https://discuss.elastic.co/u/zabaala)\ **Replies:** 0\ **Last updated:** [September 29, 2018, 11:19pm UTC](https://discuss.elastic.co/t/logstash-dont-log-more-than-500-files/150440 "2018-09-29T23:19:20Z")
Hi, I create a pipeline to logstash to read all files (58k) existing on specific directory. Inside of each file has a line of json terminated with a new line. The problems is: the logstash only read (send to my elastic… --- ## [MongoDB Output Plugin Failed when used auth user](https://discuss.elastic.co/t/mongodb-output-plugin-failed-when-used-auth-user/150436)
**Author:** [@Rahmat\_Agung\_W](https://discuss.elastic.co/u/Rahmat_Agung_W)\ **Replies:** 0\ **Last updated:** [September 29, 2018, 5:34pm UTC](https://discuss.elastic.co/t/mongodb-output-plugin-failed-when-used-auth-user/150436 "2018-09-29T17:34:15Z")
I have an logstash config like this: input { file { type =\> "json" path =\> "/home/elk/dionaea.json" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { json { source =\> … --- ## [Failed connect logstash to rabbitmq](https://discuss.elastic.co/t/failed-connect-logstash-to-rabbitmq/150162)
**Author:** [@dangbh](https://discuss.elastic.co/u/dangbh)\ **Replies:** 0\ **Last updated:** [September 27, 2018, 9:39am UTC](https://discuss.elastic.co/t/failed-connect-logstash-to-rabbitmq/150162 "2018-09-27T09:39:32Z")
Hi all, I setup elk with architecture beat --logstash(collect)--rabbitmq--logstash(process)--elasticsearch. File config logstash: https://pastebin.com/VkW3uywD File config rabbitmq: https://pastebin.com/bRR7PHQB File… --- ## [Logstash input Filter path](https://discuss.elastic.co/t/logstash-input-filter-path/150307)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 2\ **Last updated:** [September 29, 2018, 10:45am UTC](https://discuss.elastic.co/t/logstash-input-filter-path/150307 "2018-09-29T10:45:12Z")
I have a logstash.conf file where i'm defining two distinct path for two different type of logs one is for system logs and another is for network logs. However, these logs are being collected on the same directory locati… --- ## [Trying to pull only changed records in elastic](https://discuss.elastic.co/t/trying-to-pull-only-changed-records-in-elastic/150372)
**Author:** [@nitu\_gupta](https://discuss.elastic.co/u/nitu_gupta)\ **Replies:** 1\ **Last updated:** [September 29, 2018, 6:27am UTC](https://discuss.elastic.co/t/trying-to-pull-only-changed-records-in-elastic/150372 "2018-09-29T06:27:43Z")
logstash version:6.4.1 i am trying to pull records from mysql to elastic. the onetime operation is successful.However, i want that afterwards, only changed or new records should be inserted instead of inserting all aga… --- ## [Grok cant match my log](https://discuss.elastic.co/t/grok-cant-match-my-log/149920)
**Author:** [@Haitianisgood](https://discuss.elastic.co/u/Haitianisgood)\ **Replies:** 6\ **Last updated:** [September 29, 2018, 3:49am UTC](https://discuss.elastic.co/t/grok-cant-match-my-log/149920 "2018-09-29T03:49:54Z")
My log: \[23:59:52.743\]2018-08-14 23:59:52 DEBUG \[resin-tcp-connection-\*:8012-59\] /queryup\[sversion=5\]\[model=HUAWEI G7-UL20\]\[from=\_360\]\[lastupdatetime=1532506757000\]\[versioncode=201805140\]\[userId=13994251\]\[mac=8638460274… --- ## [Import data to Mysql](https://discuss.elastic.co/t/import-data-to-mysql/150405)
**Author:** [@jermmy](https://discuss.elastic.co/u/jermmy)\ **Replies:** 1\ **Last updated:** [September 29, 2018, 3:11am UTC](https://discuss.elastic.co/t/import-data-to-mysql/150405 "2018-09-29T03:11:14Z")
\[2018-09-29T10:31:50,353\]\[ERROR\]\[com.zaxxer.hikari.HikariConfig\] Failed to load driver class com.mysql.jdbc.driver from HikariConfig class classloader org.jruby.util.JRubyClassLoader@eac3a26 \[2018-09-29T10:31:50,359\]\[ER… --- ## [Have Logstash take a file containing an array of JSON objects](https://discuss.elastic.co/t/have-logstash-take-a-file-containing-an-array-of-json-objects/150233)
**Author:** [@Jedeu](https://discuss.elastic.co/u/Jedeu)\ **Replies:** 4\ **Last updated:** [September 28, 2018, 9:51pm UTC](https://discuss.elastic.co/t/have-logstash-take-a-file-containing-an-array-of-json-objects/150233 "2018-09-28T21:51:22Z")
Hi all, I've been trying to use various input and filter plugins to take a JSON file containing an array of prettified json objects and spit out an event for each object. I've scoured SO and this forum and found a coupl… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=676) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=678)