# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=764 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 765 --- ## [Custom grok pattern issue](https://discuss.elastic.co/t/custom-grok-pattern-issue/130931)
**Author:** [@AJ\_NOURI](https://discuss.elastic.co/u/AJ_NOURI)\ **Replies:** 4\ **Last updated:** [May 8, 2018, 7:21pm UTC](https://discuss.elastic.co/t/custom-grok-pattern-issue/130931 "2018-05-08T19:21:02Z")
I am testing custom grok patterns. First with the following logstash config: input { stdin{} } filter { grok { match =\> {"message" =\> "%{SYSLOGTIMESTAMP:time} %{GREEDYDATA:other}"} } } output { … --- ## [WARNING: Could not find logstash.yml cannot found](https://discuss.elastic.co/t/warning-could-not-find-logstash-yml-cannot-found/131013)
**Author:** [@sayed\_mohamed](https://discuss.elastic.co/u/sayed_mohamed)\ **Replies:** 3\ **Last updated:** [May 8, 2018, 6:34pm UTC](https://discuss.elastic.co/t/warning-could-not-find-logstash-yml-cannot-found/131013 "2018-05-08T18:34:41Z")
i have a problem that i run file as command : sudo bin/logstash --path.data sensor38 -f /home/sayed/logstash/sayed.conf and i have an warning that i set some configuration on it WARNING: Could not find logstash.yml wh… --- ## [Queue and Dead Letter Queue](https://discuss.elastic.co/t/queue-and-dead-letter-queue/131041)
**Author:** [@squanchydata](https://discuss.elastic.co/u/squanchydata)\ **Replies:** 1\ **Last updated:** [May 8, 2018, 6:26pm UTC](https://discuss.elastic.co/t/queue-and-dead-letter-queue/131041 "2018-05-08T18:26:08Z")
Can anyone explain logstash's queue and dead letter queue in simple terms to me? The queue is the recorded version of the data in logstash before it is sent to Elasticsearch??? --- ## [Bin/logstash-plugin not using private artifactory](https://discuss.elastic.co/t/bin-logstash-plugin-not-using-private-artifactory/131047)
**Author:** [@smbullet](https://discuss.elastic.co/u/smbullet)\ **Replies:** 0\ **Last updated:** [May 8, 2018, 6:20pm UTC](https://discuss.elastic.co/t/bin-logstash-plugin-not-using-private-artifactory/131047 "2018-05-08T18:20:54Z")
$ cat logstash/Gemfile | grep source source "http://artifactory.myserver.com/artifactory/api/gems/ruby-repos/" $ PATH=/home/smbullet/logstash/vendor/jruby/bin:$PATH logstash/bin/logstash-plugin install logstash-filter-my… --- ## [Logstash on Unix can't read Windows created Files](https://discuss.elastic.co/t/logstash-on-unix-cant-read-windows-created-files/130889)
**Author:** [@AloysiusParedes](https://discuss.elastic.co/u/AloysiusParedes)\ **Replies:** 9\ **Last updated:** [May 8, 2018, 5:53pm UTC](https://discuss.elastic.co/t/logstash-on-unix-cant-read-windows-created-files/130889 "2018-05-08T17:53:30Z")
Running Logstash as a service in a Linux Machine. Trying to send files from a mounted folder location (files created in Windows Environment). It seems that Logstash can't parse any files that were created in Windows, but… --- ## [Unavailable\_shards\_exception primary shard is not active](https://discuss.elastic.co/t/unavailable-shards-exception-primary-shard-is-not-active/131029)
**Author:** [@bvoros](https://discuss.elastic.co/u/bvoros)\ **Replies:** 1\ **Last updated:** [May 8, 2018, 4:12pm UTC](https://discuss.elastic.co/t/unavailable-shards-exception-primary-shard-is-not-active/131029 "2018-05-08T16:12:01Z")
One of our logstash servers suddenly essentially sropped processing our logs and started displaying the log entries mentioned in the title. The index in question appears to be ok according to elasticsearch. Both are ve… --- ## [Throughput and backpressure metrics](https://discuss.elastic.co/t/throughput-and-backpressure-metrics/130989)
**Author:** [@fritzhardy](https://discuss.elastic.co/u/fritzhardy)\ **Replies:** 0\ **Last updated:** [May 8, 2018, 11:36am UTC](https://discuss.elastic.co/t/throughput-and-backpressure-metrics/130989 "2018-05-08T11:36:46Z")
Hello, I am on elk stack 6.2.4 (and previously 5.6.2), filebeating logs off of a handful of syslog aggregators, trying to achieve a relatively modest throughput of ~3000 lines/s, 500kiB/s (measurements with pv). In add… --- ## [External Filebeat plus Logstash best approach](https://discuss.elastic.co/t/external-filebeat-plus-logstash-best-approach/130721)
**Author:** [@facumanjj](https://discuss.elastic.co/u/facumanjj)\ **Replies:** 7\ **Last updated:** [May 8, 2018, 1:32pm UTC](https://discuss.elastic.co/t/external-filebeat-plus-logstash-best-approach/130721 "2018-05-08T13:32:33Z")
Hi everyone, our system takes data from thousand of clients. Basically, we develop a client software which each of our clients install and run. That SW connects to our system (in our network). Our idea is to install Fil… --- ## [Creating field names based on entry position](https://discuss.elastic.co/t/creating-field-names-based-on-entry-position/131010)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 0\ **Last updated:** [May 8, 2018, 1:15pm UTC](https://discuss.elastic.co/t/creating-field-names-based-on-entry-position/131010 "2018-05-08T13:15:27Z")
I have some interesting data to work with. I have a plain text file with Windows Events in it which some lines contain a field and value, while other lines only contain a value. The data is in json format and looks like … --- ## [Is logstash taking packets?](https://discuss.elastic.co/t/is-logstash-taking-packets/130955)
**Author:** [@MrNerd](https://discuss.elastic.co/u/MrNerd)\ **Replies:** 9\ **Last updated:** [May 8, 2018, 1:04pm UTC](https://discuss.elastic.co/t/is-logstash-taking-packets/130955 "2018-05-08T13:04:09Z")
Hi, is there any way I can look if my logstash is taking the pakets that arrives at port 5000? Thanks MrNerd --- ## [Increase UDP input plugin performance](https://discuss.elastic.co/t/increase-udp-input-plugin-performance/130798)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 3\ **Last updated:** [May 8, 2018, 12:13pm UTC](https://discuss.elastic.co/t/increase-udp-input-plugin-performance/130798 "2018-05-08T12:13:36Z")
I need to increase UDP input plugin performance only for port 514. I have a machine with 16 CPU and 32 GB RAM I use logstash 2.3.3 version with "-b 500 -w 2" options input { udp { port =\> 514 codec… --- ## [File output plugin add some additional field in output file](https://discuss.elastic.co/t/file-output-plugin-add-some-additional-field-in-output-file/130965)
**Author:** [@adg](https://discuss.elastic.co/u/adg)\ **Replies:** 6\ **Last updated:** [May 8, 2018, 12:13pm UTC](https://discuss.elastic.co/t/file-output-plugin-add-some-additional-field-in-output-file/130965 "2018-05-08T12:13:19Z")
i m having a issue with logstash i m getting logs from multiple remote server via filebeat (INPUT) use csv filter (FILTER) and put it in elasticsearch & file the issue is logs which is stored in file having two ex… --- ## [Multiple database connection using logstash](https://discuss.elastic.co/t/multiple-database-connection-using-logstash/130865)
**Author:** [@Boukhdhira](https://discuss.elastic.co/u/Boukhdhira)\ **Replies:** 5\ **Last updated:** [May 8, 2018, 11:53am UTC](https://discuss.elastic.co/t/multiple-database-connection-using-logstash/130865 "2018-05-08T11:53:13Z")
Is it possible to have many sqlite connection with logstash in the same configuration using JDBC input ? However, databases contains the same tables structures . --- ## [Logstash elasticsearch filter plugin not working with proxy](https://discuss.elastic.co/t/logstash-elasticsearch-filter-plugin-not-working-with-proxy/130994)
**Author:** [@kostja79](https://discuss.elastic.co/u/kostja79)\ **Replies:** 0\ **Last updated:** [May 8, 2018, 11:51am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-plugin-not-working-with-proxy/130994 "2018-05-08T11:51:22Z")
Hi, I'm using the elasticsearch filter plugin to query an Elasticsearch instance installed in the same server: elasticsearch { query\_template =\> "..." hosts =\> \["localhost:9200"\] index =\> "..." } The proxy is conf… --- ## [How to make Translate function non case sensitive](https://discuss.elastic.co/t/how-to-make-translate-function-non-case-sensitive/130930)
**Author:** [@sreejiths](https://discuss.elastic.co/u/sreejiths)\ **Replies:** 1\ **Last updated:** [May 8, 2018, 11:26am UTC](https://discuss.elastic.co/t/how-to-make-translate-function-non-case-sensitive/130930 "2018-05-08T11:26:29Z")
I am using a simple translate plugin to mark a device as access switch or not and is working ..But in the list if the device entry is "mnvt542ges02" but the input hostname comes as "MNVT542GES02" it does not match ..Basi… --- ## [Logstash s3 plugin: Access only few folders inside a bucket](https://discuss.elastic.co/t/logstash-s3-plugin-access-only-few-folders-inside-a-bucket/130983)
**Author:** [@rahulnama](https://discuss.elastic.co/u/rahulnama)\ **Replies:** 2\ **Last updated:** [May 8, 2018, 11:04am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-access-only-few-folders-inside-a-bucket/130983 "2018-05-08T11:04:13Z")
Hello Team, I'm using Logstasg S3 input plugin. I'm stuck up with few questions. Please do suggest I have multiple sub-folders inside a bucket, by default If I give the bucket name which has many sub-folders will the… --- ## [Extracting data from mails](https://discuss.elastic.co/t/extracting-data-from-mails/130982)
**Author:** [@rouchad\_rouchad](https://discuss.elastic.co/u/rouchad_rouchad)\ **Replies:** 0\ **Last updated:** [May 8, 2018, 10:40am UTC](https://discuss.elastic.co/t/extracting-data-from-mails/130982 "2018-05-08T10:40:47Z")
hi everyone i want extract somme informations from the mails , i have used input imap i have transfered the mail inbox into elasticsearch and i want make some filter like grok filter , xml filter but i don't know how to… --- ## [Can I deploy a DIY Logstash appliance on remote site?](https://discuss.elastic.co/t/can-i-deploy-a-diy-logstash-appliance-on-remote-site/130941)
**Author:** [@br14nb0](https://discuss.elastic.co/u/br14nb0)\ **Replies:** 2\ **Last updated:** [May 8, 2018, 9:19am UTC](https://discuss.elastic.co/t/can-i-deploy-a-diy-logstash-appliance-on-remote-site/130941 "2018-05-08T09:19:55Z")
I want to build a box with Logstash installed that collects and parses logs then sends events to Elasticsearch server on other site. Logstash appliance now like USM sensor, responsible for collecting log from devices and… --- ## [Logstash log4j.properties config](https://discuss.elastic.co/t/logstash-log4j-properties-config/130573)
**Author:** [@z377023694](https://discuss.elastic.co/u/z377023694)\ **Replies:** 3\ **Last updated:** [May 8, 2018, 8:53am UTC](https://discuss.elastic.co/t/logstash-log4j-properties-config/130573 "2018-05-08T08:53:33Z")
information： logstash version：5.2.2 system information ： windows 64 bit demand： Implement the logstash self log to print to the file operation process： cmd command：logstash -f E:\\cs\\conf --path.settings /e:/logstas… --- ## [Problem with logstash:name error](https://discuss.elastic.co/t/problem-with-logstash-name-error/129996)
**Author:** [@wiem](https://discuss.elastic.co/u/wiem)\ **Replies:** 4\ **Last updated:** [May 8, 2018, 8:28am UTC](https://discuss.elastic.co/t/problem-with-logstash-name-error/129996 "2018-05-08T08:28:31Z")
Need help SOS! i have been using Logstash for 2 months now .2days ago out of nowhere a problem appeared that i can't find its origin: logstash- java.lang.illegalStateException:org.jruby.exceptions.RaiseException : ca… --- ## [Grok Filter for SNMP Trap](https://discuss.elastic.co/t/grok-filter-for-snmp-trap/130854)
**Author:** [@ysumit](https://discuss.elastic.co/u/ysumit)\ **Replies:** 1\ **Last updated:** [May 8, 2018, 8:19am UTC](https://discuss.elastic.co/t/grok-filter-for-snmp-trap/130854 "2018-05-08T08:19:11Z")
Hi Team I have input from snmp trap from which I would want only few fields to be captured. The SNMP trap is long string. I have attached a complete string below however need to fetch the following part: kpi\_match\_nam… --- ## [Input/Output Plugin](https://discuss.elastic.co/t/input-output-plugin/130949)
**Author:** [@MrNerd](https://discuss.elastic.co/u/MrNerd)\ **Replies:** 3\ **Last updated:** [May 8, 2018, 7:57am UTC](https://discuss.elastic.co/t/input-output-plugin/130949 "2018-05-08T07:57:53Z")
Hi, I have installed into a machine logstash, the problem here is that i want a tcp and udp plugin to take everithing at port 5.000. I'ts better if I use rsyslog for this or the logtash tcp/udp plugins will be fine for … --- ## [ELK Logstash API](https://discuss.elastic.co/t/elk-logstash-api/130939)
**Author:** [@Raja\_Pandian](https://discuss.elastic.co/u/Raja_Pandian)\ **Replies:** 16\ **Last updated:** [May 8, 2018, 7:45am UTC](https://discuss.elastic.co/t/elk-logstash-api/130939 "2018-05-08T07:45:10Z")
Hi, I want to display my .log file into Kibana Dashboard thorugh logstash API(Java).I am able to achieve this through logstash config file.But i am not able to achieve same through logstash api. I need json file to push… --- ## ["Index" output setting](https://discuss.elastic.co/t/index-output-setting/130905)
**Author:** [@ElevenB2003](https://discuss.elastic.co/u/ElevenB2003)\ **Replies:** 1\ **Last updated:** [May 8, 2018, 6:04am UTC](https://discuss.elastic.co/t/index-output-setting/130905 "2018-05-08T06:04:25Z")
Hello, I have multiple configuration files for logstash and each has their own input, filter, and output. However, I'm experiencing something strange: all data is flowing into all indexes even though I'm using the "ind… --- ## [Sequel::AdapterNotFound: mongodb.jdbc.MongoDriver not loaded](https://discuss.elastic.co/t/sequel-adapternotfound-mongodb-jdbc-mongodriver-not-loaded/130928)
**Author:** [@zhujr25](https://discuss.elastic.co/u/zhujr25)\ **Replies:** 0\ **Last updated:** [May 8, 2018, 3:59am UTC](https://discuss.elastic.co/t/sequel-adapternotfound-mongodb-jdbc-mongodriver-not-loaded/130928 "2018-05-08T03:59:03Z")
Hi, I am getting below error while using Mongodb Java Driver to query data from MongoDB. Error: \[2018-05-08T01:10:38,080\]\[ERROR\]\[logstash.agent \] Pipeline aborted due to error {:exception=\># \ **Author:** [@regex99](https://discuss.elastic.co/u/regex99)\ **Replies:** 0\ **Last updated:** [May 8, 2018, 2:43am UTC](https://discuss.elastic.co/t/concatenate-a-word-to-an-email-using-pre-defined-grok-filter/130922 "2018-05-08T02:43:16Z") hi, first of all thank you for reading my question. i have an email address in a log in following format, Apr 24 19:38:51 ip-10-0-1-204 sendmail\[9489\]: w3OJco1s009487: sendid:name@test.co.uk, delay=00:00:01, xdelay=0… --- ## [How should i grok the sentence](https://discuss.elastic.co/t/how-should-i-grok-the-sentence/129808)
**Author:** [@antony.y](https://discuss.elastic.co/u/antony.y)\ **Replies:** 6\ **Last updated:** [May 8, 2018, 2:36am UTC](https://discuss.elastic.co/t/how-should-i-grok-the-sentence/129808 "2018-05-08T02:36:39Z")
I have a syslog like "2018-04-26 15:22:23,200の\[INFO\]の123" and i using "%{TIMESTAMP\_ISO8601:time}の\[%{WORD:FFF}\]の%{INT:DD}" it does work. But when i change the information to "2018-04-26 15:22:23,200の\[INFO\]のtest" and i u… --- ## [Some problem of if and mutate in logstash filter](https://discuss.elastic.co/t/some-problem-of-if-and-mutate-in-logstash-filter/130585)
**Author:** [@zxc654951](https://discuss.elastic.co/u/zxc654951)\ **Replies:** 6\ **Last updated:** [May 8, 2018, 1:24am UTC](https://discuss.elastic.co/t/some-problem-of-if-and-mutate-in-logstash-filter/130585 "2018-05-08T01:24:19Z")
I had json format log and outputto elasticseach. I try to use "if" and "mutate" which copy Message to new field. But the new field can't viewed in elasticsearch and kibana. My Message field like "S", "E", "any string"... … --- ## [The grok error](https://discuss.elastic.co/t/the-grok-error/130430)
**Author:** [@Pinno\_Lin](https://discuss.elastic.co/u/Pinno_Lin)\ **Replies:** 5\ **Last updated:** [May 8, 2018, 12:41am UTC](https://discuss.elastic.co/t/the-grok-error/130430 "2018-05-08T00:41:02Z")
Hi, I have a mail log data in \[mail\]\[suser\] attribute in below, i want to get user account (ABC03245) to new attribute. jamescc\_chen@abc.com.tw;ABC03245|Jamescc Chen(James);abc::abc::User::FIN::PD;abc::abc::User::FIN;… --- ## [Single input with different filter and output](https://discuss.elastic.co/t/single-input-with-different-filter-and-output/129378)
**Author:** [@mayurshah](https://discuss.elastic.co/u/mayurshah)\ **Replies:** 4\ **Last updated:** [May 7, 2018, 10:22pm UTC](https://discuss.elastic.co/t/single-input-with-different-filter-and-output/129378 "2018-05-07T22:22:01Z")
Hi, I am using logstash 5.6.3 and am trying to figureout how to configure multiple filter and output for a single input. My setup is filebeat forwarding to logstash and from logstash I want out put to elasticsearch as w… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=763) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=765)