# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1164 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1165 --- ## [KV filter killing logstash performance](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance/47817)
**Author:** [@bblank](https://discuss.elastic.co/u/bblank)\ **Replies:** 4\ **Last updated:** [April 20, 2016, 4:45pm UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance/47817 "2016-04-20T16:45:05Z")
We are using the kv filter in our logstash.conf file and it is causing performance issues. Changing workers, batch size and heap size don't appear to really make a difference. Our hardware is NOT under powered. I ne… --- ## [Indexing multiple tables from MySQL/MSSQL in one config-file](https://discuss.elastic.co/t/indexing-multiple-tables-from-mysql-mssql-in-one-config-file/47900)
**Author:** [@PascalR](https://discuss.elastic.co/u/PascalR)\ **Replies:** 0\ **Last updated:** [April 20, 2016, 11:17am UTC](https://discuss.elastic.co/t/indexing-multiple-tables-from-mysql-mssql-in-one-config-file/47900 "2016-04-20T11:17:48Z")
Hi everyone, my task is to put our (in this case MySQL) tables into elastics search. I made a config for one table and it worked really well. The problem is, that I have not one or two tables, but over 100. So my idea… --- ## [Logstash Installation on UNIX server](https://discuss.elastic.co/t/logstash-installation-on-unix-server/47903)
**Author:** [@Shrawan\_Bhagwat](https://discuss.elastic.co/u/Shrawan_Bhagwat)\ **Replies:** 1\ **Last updated:** [April 20, 2016, 1:38pm UTC](https://discuss.elastic.co/t/logstash-installation-on-unix-server/47903 "2016-04-20T13:38:54Z")
Hi All, I have setup logstash on one of the UNIX server. When i tried to run configtest using the following command: ./bin/logstash -f /fmsapp/logstash/conf/ --configtest it's giving me an error: No such file or dire… --- ## [LocalJumpError Logstash 2.3.0](https://discuss.elastic.co/t/localjumperror-logstash-2-3-0/46984)
**Author:** [@diwertowski](https://discuss.elastic.co/u/diwertowski)\ **Replies:** 4\ **Last updated:** [April 20, 2016, 12:06pm UTC](https://discuss.elastic.co/t/localjumperror-logstash-2-3-0/46984 "2016-04-20T12:06:34Z")
Logstash 2.3.0 Windows Server 2008 If I start my Logstash-Shipper with this output-configuation... output { redis { host =\> \["redisserver1:6379","redisserver2:6380"\] shuffle\_hosts =\> true … --- ## [Not able to create index with the same name](https://discuss.elastic.co/t/not-able-to-create-index-with-the-same-name/24063)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 10\ **Last updated:** [April 20, 2016, 11:53am UTC](https://discuss.elastic.co/t/not-able-to-create-index-with-the-same-name/24063 "2016-04-20T11:53:01Z")
Hi Experts, I am using ES 1.6.0 and LS 1.5.0 . I am parsing CSV file and store it to ES. Since I am in Dev environment so I am changing my LS conf file regularly, e.g applying different filters , mutate etc. Now every … --- ## [Failed parsing date from field creation\_date (jdbc)](https://discuss.elastic.co/t/failed-parsing-date-from-field-creation-date-jdbc/47244)
**Author:** [@PascalR](https://discuss.elastic.co/u/PascalR)\ **Replies:** 7\ **Last updated:** [April 20, 2016, 11:06am UTC](https://discuss.elastic.co/t/failed-parsing-date-from-field-creation-date-jdbc/47244 "2016-04-20T11:06:14Z")
Hi, I tried to load data from my MySQL database to ES. It worked well, only fields with datetime have a wrong time in it. As you can see in the config-file below, I used a date filter to solve this with several formats… --- ## [How to convert a list of string to a list of date in filter?](https://discuss.elastic.co/t/how-to-convert-a-list-of-string-to-a-list-of-date-in-filter/47886)
**Author:** [@Morriaty](https://discuss.elastic.co/u/Morriaty)\ **Replies:** 3\ **Last updated:** [April 20, 2016, 11:05am UTC](https://discuss.elastic.co/t/how-to-convert-a-list-of-string-to-a-list-of-date-in-filter/47886 "2016-04-20T11:05:41Z")
The input had a field named valid\_dates which is a string column from mysql(jdbc).It formatted like this "2014-09-10 00:00:00,2014-08-21 17:02:02,2014-06-19 00:00:00,2014-08-20 14:29:09" And the filter was filte… --- ## [Logging events lost when Elastic Search connection goes down](https://discuss.elastic.co/t/logging-events-lost-when-elastic-search-connection-goes-down/47801)
**Author:** [@ono](https://discuss.elastic.co/u/ono)\ **Replies:** 2\ **Last updated:** [April 20, 2016, 8:38am UTC](https://discuss.elastic.co/t/logging-events-lost-when-elastic-search-connection-goes-down/47801 "2016-04-20T08:38:18Z")
Guys, We're trying filebeat, forwarding logs to logstash (for JSON parsing) and then onwards into Elastic Search (ES). One of the scenarios we're testing is connection between logstash and ES going down and then the lo… --- ## [Drop GrokParseFailure if message exactly equals "\\r, \\r\\n and \\n"](https://discuss.elastic.co/t/drop-grokparsefailure-if-message-exactly-equals-r-r-n-and-n/47591)
**Author:** [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\ **Replies:** 3\ **Last updated:** [April 19, 2016, 9:46pm UTC](https://discuss.elastic.co/t/drop-grokparsefailure-if-message-exactly-equals-r-r-n-and-n/47591 "2016-04-19T21:46:30Z")
Drop GrokParseFailure if message exactly equals "\\r, \\r\\n and \\n" "message" =\> "\\r", "@timestamp" =\> "2016-04-17T18:40:43.136Z", "Path" =\> "C:/Logs/HttpEventSource 1.txt", "Type" =\> "HttpEventSou… --- ## [Logstash config with cisco asa and filebeat problem](https://discuss.elastic.co/t/logstash-config-with-cisco-asa-and-filebeat-problem/47542)
**Author:** [@navox19](https://discuss.elastic.co/u/navox19)\ **Replies:** 7\ **Last updated:** [April 19, 2016, 8:28pm UTC](https://discuss.elastic.co/t/logstash-config-with-cisco-asa-and-filebeat-problem/47542 "2016-04-19T20:28:41Z")
Hello I need to get a configuration code for the config file of logstash for filebeat and firewall cisco asa at the same time please i need help thanks --- ## [Grokparsefailure migrating from 1.4.1 to 2.3.1](https://discuss.elastic.co/t/grokparsefailure-migrating-from-1-4-1-to-2-3-1/47662)
**Author:** [@GambitK](https://discuss.elastic.co/u/GambitK)\ **Replies:** 4\ **Last updated:** [April 19, 2016, 8:10pm UTC](https://discuss.elastic.co/t/grokparsefailure-migrating-from-1-4-1-to-2-3-1/47662 "2016-04-19T20:10:18Z")
I'm migrating from logstash 1.4.1 to 2.3.1 and grok is giving me troubles, seems like it's not behaving like it used to. Is giving me grokparsefailure on this data: Apr 18 09:25:20 10.0.0.10 date=2016-04-18 time=09:19:… --- ## [Logstash output to external Elastic Search server](https://discuss.elastic.co/t/logstash-output-to-external-elastic-search-server/42913)
**Author:** [@Hans](https://discuss.elastic.co/u/Hans)\ **Replies:** 10\ **Last updated:** [April 19, 2016, 5:51pm UTC](https://discuss.elastic.co/t/logstash-output-to-external-elastic-search-server/42913 "2016-04-19T17:51:28Z")
HI, have been struggling to get the output of one logstash servers output into an elastics search server. When doing a tcpdump on the ES installation I can see the server is connecting, however when looking at the indexe… --- ## [Translate plugin: translate array values/use tag instead of field/use wildcard/regex in field](https://discuss.elastic.co/t/translate-plugin-translate-array-values-use-tag-instead-of-field-use-wildcard-regex-in-field/46725)
**Author:** [@CSUMBmWall](https://discuss.elastic.co/u/CSUMBmWall)\ **Replies:** 2\ **Last updated:** [April 19, 2016, 3:13pm UTC](https://discuss.elastic.co/t/translate-plugin-translate-array-values-use-tag-instead-of-field-use-wildcard-regex-in-field/46725 "2016-04-19T15:13:48Z")
I have a field with an array of values I need translated. Since translate only works on the first value, I renamed all of the fields dynamically to input\_\["value"\]. Now I can't use translate on them because it requi… --- ## [Pushing flush onto Pipeline](https://discuss.elastic.co/t/pushing-flush-onto-pipeline/47802)
**Author:** [@Mag\_Carl](https://discuss.elastic.co/u/Mag_Carl)\ **Replies:** 3\ **Last updated:** [April 19, 2016, 2:49pm UTC](https://discuss.elastic.co/t/pushing-flush-onto-pipeline/47802 "2016-04-19T14:49:31Z")
I am trying to get twitter feed using below config file input { twitter { add your data consumer\_key =\> "\*\*\*\*\*\*\*\*\*\*\*" consumer\_secret =\>"\*\*\*\*\*\*\*\*\*\*\*" oauth\_token =\> "\*\*\*\*\*\*\*\*\*\*\*" oauth\_token\_secret =\> "\*\*\*\*\*\*\*\*\*\*\*" k… --- ## [Several config files for beats in logstash](https://discuss.elastic.co/t/several-config-files-for-beats-in-logstash/47749)
**Author:** [@Adir](https://discuss.elastic.co/u/Adir)\ **Replies:** 4\ **Last updated:** [April 19, 2016, 1:13pm UTC](https://discuss.elastic.co/t/several-config-files-for-beats-in-logstash/47749 "2016-04-19T13:13:03Z")
Hi, We just started to work with logstash recently and i have a few questions about beats config files. We currently have 2 configuration files on our logstash - one for filebeat and another one for winlogbeat . Each c… --- ## [Save a value to variable](https://discuss.elastic.co/t/save-a-value-to-variable/47642)
**Author:** [@Prateek\_Kshtriya](https://discuss.elastic.co/u/Prateek_Kshtriya)\ **Replies:** 4\ **Last updated:** [April 19, 2016, 10:32am UTC](https://discuss.elastic.co/t/save-a-value-to-variable/47642 "2016-04-19T10:32:30Z")
Hi All, Please suggest If it is possible to do a grok & perform a mathematical operation on grok result? Regards, Prateek --- ## [Help with logstash and XML](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501)
**Author:** [@funkisjazz](https://discuss.elastic.co/u/funkisjazz)\ **Replies:** 4\ **Last updated:** [April 19, 2016, 10:17am UTC](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501 "2016-04-19T10:17:13Z")
I've been searching the internet for the last 6 hours trying to find a guide, tutorial, hint or any kind of help to upload the content of some .xml files into ES using logstash. For the life of me I haven't been able t… --- ## [Logstash service started but not working, command line is working](https://discuss.elastic.co/t/logstash-service-started-but-not-working-command-line-is-working/47764)
**Author:** [@Erik\_Parienty](https://discuss.elastic.co/u/Erik_Parienty)\ **Replies:** 4\ **Last updated:** [April 19, 2016, 9:18am UTC](https://discuss.elastic.co/t/logstash-service-started-but-not-working-command-line-is-working/47764 "2016-04-19T09:18:01Z")
when i run logstash in command line all is working but not as a service i get no data no logs conf is ok ps shows: logstash 10120 1 1 08:44 pts/0 00:00:00 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -… --- ## [Not Able to filter two type of events in logstash filter](https://discuss.elastic.co/t/not-able-to-filter-two-type-of-events-in-logstash-filter/47556)
**Author:** [@ashutosh25apr](https://discuss.elastic.co/u/ashutosh25apr)\ **Replies:** 15\ **Last updated:** [April 19, 2016, 8:58am UTC](https://discuss.elastic.co/t/not-able-to-filter-two-type-of-events-in-logstash-filter/47556 "2016-04-19T08:58:12Z")
Hi I am using filebeat to send beats to logstash, Able to see in the logs of filebeat that event is generated after reading the file but ,not able to view the logs of logstash whether event is reaching to to logstash so … --- ## [How to overwrite/rewrite message field after kv and remove field filters?](https://discuss.elastic.co/t/how-to-overwrite-rewrite-message-field-after-kv-and-remove-field-filters/47647)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 3\ **Last updated:** [April 19, 2016, 8:31am UTC](https://discuss.elastic.co/t/how-to-overwrite-rewrite-message-field-after-kv-and-remove-field-filters/47647 "2016-04-19T08:31:27Z")
Hi Experts, I need to overwrite my message field once I am done with kv filter . My flow would be like input{} filter { grok { match=\>\["message",'CEF:%{CEFNUM:cef}\[^\\|\\n\]\*\\|%{CEFBAR:dvcvendor}\\|%{CEFBAR:dvcprodu… --- ## [Logstash Create and Access Array](https://discuss.elastic.co/t/logstash-create-and-access-array/47607)
**Author:** [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Replies:** 3\ **Last updated:** [April 19, 2016, 8:29am UTC](https://discuss.elastic.co/t/logstash-create-and-access-array/47607 "2016-04-19T08:29:00Z")
hi logstash, Is it possible to spilt the log as below based on ',' to array ? "message" =\> "....", "host" =\> "10.10.11.12", "@version" =\> "1", "@timestamp" =\> "2016-04-13T01:52:43.535Z", "DISMAN-EVENT-MIB::sysUpTim… --- ## [Multiple grok match not working](https://discuss.elastic.co/t/multiple-grok-match-not-working/47748)
**Author:** [@komail](https://discuss.elastic.co/u/komail)\ **Replies:** 1\ **Last updated:** [April 19, 2016, 8:26am UTC](https://discuss.elastic.co/t/multiple-grok-match-not-working/47748 "2016-04-19T08:26:11Z")
I have two kinds of logs as shown below in my log file, which I need to match -- url: /display/wDocs/com.abc.downloadmanager+v1.1 | userName: komail badami | referer: https://abcd.com/display/wDocs/com.abc.downloadman… --- ## [How to get rid of new field?](https://discuss.elastic.co/t/how-to-get-rid-of-new-field/47494)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 5\ **Last updated:** [April 19, 2016, 8:11am UTC](https://discuss.elastic.co/t/how-to-get-rid-of-new-field/47494 "2016-04-19T08:11:35Z")
Hi, I have created eventLogTime field in logstash indexer and assign timestamp form the log. I have provided list of timestamp while doing this. Now the problem is when we are adding applications in centralized logging,… --- ## [Snmptrap Input Bug](https://discuss.elastic.co/t/snmptrap-input-bug/47733)
**Author:** [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Replies:** 2\ **Last updated:** [April 19, 2016, 7:34am UTC](https://discuss.elastic.co/t/snmptrap-input-bug/47733 "2016-04-19T07:34:15Z")
hi all, when i start the logstash , i got the warning as follows, # /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash-snmptrap.conf Settings: Default filter workers: 1 Logstash startup completed warning: o… --- ## [Multiple Elasticsearch output errors](https://discuss.elastic.co/t/multiple-elasticsearch-output-errors/47727)
**Author:** [@bstascavage](https://discuss.elastic.co/u/bstascavage)\ **Replies:** 1\ **Last updated:** [April 19, 2016, 3:43am UTC](https://discuss.elastic.co/t/multiple-elasticsearch-output-errors/47727 "2016-04-19T03:43:34Z")
Hi all! I am trying to setup my Logstash instance to output to two separate Elasticsearch clusters (Logstash version: 2.3.1). Logstash indexes to both Elasticsearch clusters just fine in till one cluster becomes unavai… --- ## [Logstash config file and Redis](https://discuss.elastic.co/t/logstash-config-file-and-redis/47530)
**Author:** [@lhorsky1](https://discuss.elastic.co/u/lhorsky1)\ **Replies:** 5\ **Last updated:** [April 19, 2016, 2:59am UTC](https://discuss.elastic.co/t/logstash-config-file-and-redis/47530 "2016-04-19T02:59:55Z")
Previously, we have been running 6 different logstash processes on each Logstash server (5 shippers and 1 indexer) and 1 Redis process. The shippers each had their own configuration file, listening for data incoming on s… --- ## [Logstash input-\>file](https://discuss.elastic.co/t/logstash-input-file/47671)
**Author:** [@birdbiubiubiu](https://discuss.elastic.co/u/birdbiubiubiu)\ **Replies:** 3\ **Last updated:** [April 19, 2016, 2:57am UTC](https://discuss.elastic.co/t/logstash-input-file/47671 "2016-04-19T02:57:52Z")
My log file is generated per day , similar to the file name access-2016-04-17.log、access-2016-04-18.log，How can I write input, read these files can be generic. --- ## [TCP input miss data](https://discuss.elastic.co/t/tcp-input-miss-data/47678)
**Author:** [@kirill.krainov](https://discuss.elastic.co/u/kirill.krainov)\ **Replies:** 1\ **Last updated:** [April 18, 2016, 9:50pm UTC](https://discuss.elastic.co/t/tcp-input-miss-data/47678 "2016-04-18T21:50:53Z")
I'm using tcp input to accept logs in json from other services. There is a bug, if you leave parameter -b default, and try send to socket 40-50k json logs w/o any timeouts, you will get a few invalid jsons and json parse… --- ## [Logstash Pattern](https://discuss.elastic.co/t/logstash-pattern/47715)
**Author:** [@rakesh76](https://discuss.elastic.co/u/rakesh76)\ **Replies:** 2\ **Last updated:** [April 18, 2016, 9:27pm UTC](https://discuss.elastic.co/t/logstash-pattern/47715 "2016-04-18T21:27:59Z")
Here is my message 04/04/2016 16:49:49,956 INFO \[default-dispatcher-18\] WorkResultConsumer - workResult: WorkResult{workId='ID6C00569599FC1:7464AB2', groupId='20160404|DAL|WN', result=complete} I would like to get w… --- ## [How to index nested mysql object into elasticsearch using logstash?](https://discuss.elastic.co/t/how-to-index-nested-mysql-object-into-elasticsearch-using-logstash/47600)
**Author:** [@Yangrui\_Guo](https://discuss.elastic.co/u/Yangrui_Guo)\ **Replies:** 1\ **Last updated:** [April 18, 2016, 9:11pm UTC](https://discuss.elastic.co/t/how-to-index-nested-mysql-object-into-elasticsearch-using-logstash/47600 "2016-04-18T21:11:26Z")
I'm trying to index mysql database with elasticsearch. Consider the example mapping: {"blog": {"properties": {"id": "string"} {"author": "string"} {"time\_created": } {"author\_info": {"author\_n… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1163) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1165)