# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1101 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1102 --- ## [Running multiple logstash from same binary](https://discuss.elastic.co/t/running-multiple-logstash-from-same-binary/60909)
**Author:** [@Mick\_Mahoney](https://discuss.elastic.co/u/Mick_Mahoney)\ **Replies:** 1\ **Last updated:** [September 19, 2016, 8:38pm UTC](https://discuss.elastic.co/t/running-multiple-logstash-from-same-binary/60909 "2016-09-19T20:38:10Z")
Just want to be sure I can run multiple copies of logstash from same codeset, by just duplicating init.d script ==\> logstash2, and changing name, config and log parameters in logstash2 file. Then starting second copy. … --- ## [Wondering if it's possible to parse this type of data](https://discuss.elastic.co/t/wondering-if-its-possible-to-parse-this-type-of-data/60927)
**Author:** [@Mark\_Dolan](https://discuss.elastic.co/u/Mark_Dolan)\ **Replies:** 0\ **Last updated:** [September 19, 2016, 5:29pm UTC](https://discuss.elastic.co/t/wondering-if-its-possible-to-parse-this-type-of-data/60927 "2016-09-19T17:29:29Z")
I have some data from the IBM nmon Linux/AIX monitoring tool. It's a strange hybrid format with several different delimited and headerless fields. For example: ZZZZ,T0002,00:05:06,04-AUG-2016 CPU001,T0002,0.6,0.2,0.0… --- ## [Connection via JDBC fails intermittently](https://discuss.elastic.co/t/connection-via-jdbc-fails-intermittently/60798)
**Author:** [@Dallas\_Toth](https://discuss.elastic.co/u/Dallas_Toth)\ **Replies:** 2\ **Last updated:** [September 19, 2016, 3:44pm UTC](https://discuss.elastic.co/t/connection-via-jdbc-fails-intermittently/60798 "2016-09-19T15:44:52Z")
My Logstash will work fine with the config for a few days but then after just some routine maintenance where I need to restart the logstash service I start to get this error. The I start to mess with the Conf scripts and… --- ## [Logstash gives error while trying to start multiple config files](https://discuss.elastic.co/t/logstash-gives-error-while-trying-to-start-multiple-config-files/60728)
**Author:** [@Sri\_ram](https://discuss.elastic.co/u/Sri_ram)\ **Replies:** 6\ **Last updated:** [September 19, 2016, 2:35pm UTC](https://discuss.elastic.co/t/logstash-gives-error-while-trying-to-start-multiple-config-files/60728 "2016-09-19T14:35:45Z")
Hi, We are collecting the application logs generated on one server, sending them to logstash using Filebeat. There are different format/types of log data, we had individual logstash config file to parse the data( i.e. … --- ## [Logstash multiline fails on heavy load](https://discuss.elastic.co/t/logstash-multiline-fails-on-heavy-load/60896)
**Author:** [@vikas\_J](https://discuss.elastic.co/u/vikas_J)\ **Replies:** 0\ **Last updated:** [September 19, 2016, 1:44pm UTC](https://discuss.elastic.co/t/logstash-multiline-fails-on-heavy-load/60896 "2016-09-19T13:44:58Z")
Hi, I use logstash to parse application logs with multiline being used to split the logs into events based on regex. This works under normal load in logstash. However in certain times when the input from filebeat to lo… --- ## [\[SOLVED\] Auto\_flush\_interval not flushing on UDP input](https://discuss.elastic.co/t/solved-auto-flush-interval-not-flushing-on-udp-input/60387)
**Author:** [@Raul\_Uria](https://discuss.elastic.co/u/Raul_Uria)\ **Replies:** 1\ **Last updated:** [September 19, 2016, 9:27am UTC](https://discuss.elastic.co/t/solved-auto-flush-interval-not-flushing-on-udp-input/60387 "2016-09-19T09:27:37Z")
Hi, I´m having problem with multiline codec, my last event is not processed until new one came along. I know it sounds like I need to configure auto\_flush\_interval to get rid of the last event, but I set auto\_flush\_int… --- ## [Strange behaviour when using both Kafka output and input plugins](https://discuss.elastic.co/t/strange-behaviour-when-using-both-kafka-output-and-input-plugins/59727)
**Author:** [@ryac](https://discuss.elastic.co/u/ryac)\ **Replies:** 4\ **Last updated:** [September 19, 2016, 2:36am UTC](https://discuss.elastic.co/t/strange-behaviour-when-using-both-kafka-output-and-input-plugins/59727 "2016-09-19T02:36:55Z")
Hi all, Am I able to use both, the output and input Kafka plugins on the same Logstash instance? It seems that the configuration for the consumer continuously loops through all messages in Kafka forever. I'm following … --- ## [Parsing repeated patterns](https://discuss.elastic.co/t/parsing-repeated-patterns/29287)
**Author:** [@mparthas](https://discuss.elastic.co/u/mparthas)\ **Replies:** 6\ **Last updated:** [September 18, 2016, 7:07pm UTC](https://discuss.elastic.co/t/parsing-repeated-patterns/29287 "2016-09-18T19:07:31Z")
Hi, I have a log line that has patterns repeating multiple times and the number of times could be varying from line to line. Is there a recipe for this ? For example, abc.com 300 IN CNAME xyz.com xyz.com 300 IN CNA… --- ## [Sync MySQL to Elasticsearch](https://discuss.elastic.co/t/sync-mysql-to-elasticsearch/60705)
**Author:** [@adrianolimit](https://discuss.elastic.co/u/adrianolimit)\ **Replies:** 6\ **Last updated:** [September 18, 2016, 11:25am UTC](https://discuss.elastic.co/t/sync-mysql-to-elasticsearch/60705 "2016-09-18T11:25:37Z")
Hello everyone, In order to synchronise my MySQL database with Elasticsearch I set up the following process : I activated the binary log in MySQL in order to see the different transactions After that, I execute Logst… --- ## [Logstash does not support one to many](https://discuss.elastic.co/t/logstash-does-not-support-one-to-many/60644)
**Author:** [@Syed\_Ather](https://discuss.elastic.co/u/Syed_Ather)\ **Replies:** 5\ **Last updated:** [September 17, 2016, 9:00pm UTC](https://discuss.elastic.co/t/logstash-does-not-support-one-to-many/60644 "2016-09-17T21:00:20Z")
Hi , does logstash support 1-\* relationships ? if yes , does can it not be done without the use of ruby /filters.. --- ## [Large amount of Grok patterns](https://discuss.elastic.co/t/large-amount-of-grok-patterns/60781)
**Author:** [@jaykumar](https://discuss.elastic.co/u/jaykumar)\ **Replies:** 1\ **Last updated:** [September 17, 2016, 5:57pm UTC](https://discuss.elastic.co/t/large-amount-of-grok-patterns/60781 "2016-09-17T17:57:02Z")
Hi All, I am trying to figure out best way to do event classification. In my case, below is sample event: Jul 21 01:19:57.58 172.20.20.100 date=2016-07-20 time=20:12:25 timezone="UTC" device\_name="X2123" device\_id=C0… --- ## [Parse json if json else leave as is](https://discuss.elastic.co/t/parse-json-if-json-else-leave-as-is/60586)
**Author:** [@arpan](https://discuss.elastic.co/u/arpan)\ **Replies:** 1\ **Last updated:** [September 17, 2016, 5:46pm UTC](https://discuss.elastic.co/t/parse-json-if-json-else-leave-as-is/60586 "2016-09-17T17:46:26Z")
I wanted the filter to parse a message as json if it is json, otherwise just let it be in the form it is. There wasn't a lot of documentation on the filter description page so I was not sure if this is the default behavi… --- ## [Parser for multiple line file](https://discuss.elastic.co/t/parser-for-multiple-line-file/60555)
**Author:** [@rodher](https://discuss.elastic.co/u/rodher)\ **Replies:** 5\ **Last updated:** [September 17, 2016, 5:35pm UTC](https://discuss.elastic.co/t/parser-for-multiple-line-file/60555 "2016-09-17T17:35:51Z")
Hello experts! I have the following scenario: I have a logfile with differente and multiple line log. There are at least 3 structures inside the file I have done the grok parser, but when I execute in logstash, It do… --- ## [How Logstash reparse Elasticsearch input](https://discuss.elastic.co/t/how-logstash-reparse-elasticsearch-input/60696)
**Author:** [@Abhishek](https://discuss.elastic.co/u/Abhishek)\ **Replies:** 1\ **Last updated:** [September 17, 2016, 5:50am UTC](https://discuss.elastic.co/t/how-logstash-reparse-elasticsearch-input/60696 "2016-09-17T05:50:28Z")
Hi All, Suppose I have elasticsearch index\_1 with some data. I've created a new index index\_2 and created a logstash pipeline to replicate data from index\_1 to index\_2. After having all data in index\_2, pipeline is clo… --- ## [Elasticsearch Output: Node outages](https://discuss.elastic.co/t/elasticsearch-output-node-outages/60726)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 1\ **Last updated:** [September 17, 2016, 5:28am UTC](https://discuss.elastic.co/t/elasticsearch-output-node-outages/60726 "2016-09-17T05:28:27Z")
Hi, Can anyone tell me how the elasticsearch output handles a subset of it's elasticsearch hosts being down? Should I take the bad hosts out or does logstash handle it seamlessly? Regards, David --- ## [Best way to split json input to multiple events for elasticsearch](https://discuss.elastic.co/t/best-way-to-split-json-input-to-multiple-events-for-elasticsearch/60537)
**Author:** [@Skippy](https://discuss.elastic.co/u/Skippy)\ **Replies:** 2\ **Last updated:** [September 16, 2016, 7:33pm UTC](https://discuss.elastic.co/t/best-way-to-split-json-input-to-multiple-events-for-elasticsearch/60537 "2016-09-16T19:33:40Z")
First of all, my apologies. I am new to ELK and I can find most of my way around with googling what I need. I absolutely love ELK so far, but I could not find any answers to a very simple question (Which leads me to be… --- ## [Logstash - worker dies for no reason](https://discuss.elastic.co/t/logstash-worker-dies-for-no-reason/60564)
**Author:** [@guyisra](https://discuss.elastic.co/u/guyisra)\ **Replies:** 0\ **Last updated:** [September 15, 2016, 5:58am UTC](https://discuss.elastic.co/t/logstash-worker-dies-for-no-reason/60564 "2016-09-15T05:58:15Z")
Logstash version: 2.3.4-1 OS: centos 7 config file: input { kafka { bootstrap\_servers =\> "${KAFKA\_ADDRESS}" topics =\> \["${LOGSTASH\_KAFKA\_TOPIC}"\] } } filter { ruby { code =\> "require 'json' … --- ## [Logstash example -\> Kibana. Field data loading is forbidden on \[timestamp\]](https://discuss.elastic.co/t/logstash-example-kibana-field-data-loading-is-forbidden-on-timestamp/60703)
**Author:** [@jsps](https://discuss.elastic.co/u/jsps)\ **Replies:** 1\ **Last updated:** [September 16, 2016, 12:42pm UTC](https://discuss.elastic.co/t/logstash-example-kibana-field-data-loading-is-forbidden-on-timestamp/60703 "2016-09-16T12:42:03Z")
Hi, Kibana 4.5.0 Logstash 2.4.0 ES 2.3.1 java 1.8.0\_101 I'm following the logstash example documentation, using the sample input and the COMBINEDAPACHELOG filter. This works as expected, the log is consumed and I c… --- ## [Running grok against an HTTP source](https://discuss.elastic.co/t/running-grok-against-an-http-source/60608)
**Author:** [@amit\_oren](https://discuss.elastic.co/u/amit_oren)\ **Replies:** 1\ **Last updated:** [September 16, 2016, 5:41am UTC](https://discuss.elastic.co/t/running-grok-against-an-http-source/60608 "2016-09-16T05:41:07Z")
Hi, I'm trying to recieve HTTP GET requests via the HTTP input and then run grok and a ruby script on the request\_uri in order to extract the paramters and their values. My agent configuration is as follows: input { … --- ## [Removing\\clearing index before processing new CSV file](https://discuss.elastic.co/t/removing-clearing-index-before-processing-new-csv-file/60661)
**Author:** [@Nick\_A](https://discuss.elastic.co/u/Nick_A)\ **Replies:** 1\ **Last updated:** [September 16, 2016, 5:39am UTC](https://discuss.elastic.co/t/removing-clearing-index-before-processing-new-csv-file/60661 "2016-09-16T05:39:36Z")
I receive new CSV file with catalogue data everyday. I need to remove all old indexed documents/data from existing Elasticsearch index before starting processing records from new CSV file. How can I do it? Thanks. --- ## [XPATH for JSON Documents](https://discuss.elastic.co/t/xpath-for-json-documents/60642)
**Author:** [@Johnnycc1](https://discuss.elastic.co/u/Johnnycc1)\ **Replies:** 1\ **Last updated:** [September 16, 2016, 5:32am UTC](https://discuss.elastic.co/t/xpath-for-json-documents/60642 "2016-09-16T05:32:53Z")
Easy questions. Where data comes into logstash as XML, I can use xpath in the XML filter to pull out specific information from the XML structure and store it in a new field. Is there a way to do this when the documen… --- ## [Count field matches](https://discuss.elastic.co/t/count-field-matches/60547)
**Author:** [@Mormaii](https://discuss.elastic.co/u/Mormaii)\ **Replies:** 3\ **Last updated:** [September 15, 2016, 6:21pm UTC](https://discuss.elastic.co/t/count-field-matches/60547 "2016-09-15T18:21:23Z")
Is there a way to count field occurences on an entry? I have this data: "cpus" =\> { "cpu0" =\> { "user" =\> 448515294, "user\_p" =\> 0.3256, "nice" =\> 4644, "system" =\> … --- ## [Send email when some events(start,error,etc) occurs](https://discuss.elastic.co/t/send-email-when-some-events-start-error-etc-occurs/60553)
**Author:** [@Octavius](https://discuss.elastic.co/u/Octavius)\ **Replies:** 3\ **Last updated:** [September 15, 2016, 6:12pm UTC](https://discuss.elastic.co/t/send-email-when-some-events-start-error-etc-occurs/60553 "2016-09-15T18:12:52Z")
Hi, I was testing logstash for a java application and it was looking good but in order to have a better solution i need to know if i can send an email when we have an ERROR or when the server starts(if we deploy our ap… --- ## [Logstash jar dependencies](https://discuss.elastic.co/t/logstash-jar-dependencies/26813)
**Author:** [@dres](https://discuss.elastic.co/u/dres)\ **Replies:** 1\ **Last updated:** [September 15, 2016, 4:02pm UTC](https://discuss.elastic.co/t/logstash-jar-dependencies/26813 "2016-09-15T16:02:52Z")
I'm having some trouble on how to include jar dependencies into my output plugin gem. I have got to the point where it downloads them and puts them into my lib directory but I want them in the vendor directory like the … --- ## [Zabbix and multi\_value key](https://discuss.elastic.co/t/zabbix-and-multi-value-key/60584)
**Author:** [@karnak](https://discuss.elastic.co/u/karnak)\ **Replies:** 2\ **Last updated:** [September 15, 2016, 3:35pm UTC](https://discuss.elastic.co/t/zabbix-and-multi-value-key/60584 "2016-09-15T15:35:43Z")
Hi all, I'm trying to setup the zabbix plugin with the multi\_values key without success. This is the config filter { add\_field =\> { "multivalue" =\> \[ "foo", "bar" \] } } output { zabbix{ zabbix\_server\_host =\> "za… --- ## [\[RESOLVED\] Input plugin Rabbitmq](https://discuss.elastic.co/t/resolved-input-plugin-rabbitmq/60591)
**Author:** [@axelfelix](https://discuss.elastic.co/u/axelfelix)\ **Replies:** 2\ **Last updated:** [September 15, 2016, 1:39pm UTC](https://discuss.elastic.co/t/resolved-input-plugin-rabbitmq/60591 "2016-09-15T13:39:31Z")
Hi all, I have a rabbitmq for queueing. I can see my logs comming in rabbitmq but never go out to my logstash. Logstash log: {:timestamp=\>"2016-09-15T11:10:35.118000+0200", :message=\>"Queue subscription ended! Will… --- ## [Failed action with response of 400 with only one kind of logs](https://discuss.elastic.co/t/failed-action-with-response-of-400-with-only-one-kind-of-logs/28144)
**Author:** [@syunusic](https://discuss.elastic.co/u/syunusic)\ **Replies:** 4\ **Last updated:** [September 15, 2016, 1:17pm UTC](https://discuss.elastic.co/t/failed-action-with-response-of-400-with-only-one-kind-of-logs/28144 "2016-09-15T13:17:27Z")
I'm parsing syslog from an AV server. Every log goes fine from the AV to a syslog server, then to a RMQ, and then to ElasticSearch (1.6) via Logstash (1.5.1). There is only log type that is giving me problems: the one th… --- ## [Logstash speed - virtual vs physical environment](https://discuss.elastic.co/t/logstash-speed-virtual-vs-physical-environment/60358)
**Author:** [@andreas\_at\_work](https://discuss.elastic.co/u/andreas_at_work)\ **Replies:** 9\ **Last updated:** [September 15, 2016, 10:16am UTC](https://discuss.elastic.co/t/logstash-speed-virtual-vs-physical-environment/60358 "2016-09-15T10:16:33Z")
Hey, we are using the ELK stack for some detecting/alerting in incident response cases. Some time ago a college created a python framework to automate the creation and configuration of the complete ELK stack + RabbitMQ… --- ## [message=\>"Failed parsing date from field", :field=\>"timestamp", :value=\>"2016-09-14 15:43:48.258000", :exception=\>"Invalid format: \\"2016-09-14 15:43:48.258000\\"",](https://discuss.elastic.co/t/message-failed-parsing-date-from-field-field-timestamp-value-2016-09-14-15-43-48-258000-exception-invalid-format-2016-09-14-15-43-48-258000/60528)
**Author:** [@gentitope](https://discuss.elastic.co/u/gentitope)\ **Replies:** 2\ **Last updated:** [September 15, 2016, 8:43am UTC](https://discuss.elastic.co/t/message-failed-parsing-date-from-field-field-timestamp-value-2016-09-14-15-43-48-258000-exception-invalid-format-2016-09-14-15-43-48-258000/60528 "2016-09-15T08:43:34Z")
Hello, I have this error, {:timestamp=\>"2016-09-14T16:43:48.937000+0100", :message=\>"Failed parsing date from field", :field=\>"timestamp", :value=\>"2016-09-14 15:43:48.258000", :exception=\>"Invalid format: \\"2016-09-14… --- ## [Logstash loosing data during processing](https://discuss.elastic.co/t/logstash-loosing-data-during-processing/60582)
**Author:** [@Robert\_Vojcik](https://discuss.elastic.co/u/Robert_Vojcik)\ **Replies:** 1\ **Last updated:** [September 15, 2016, 8:27am UTC](https://discuss.elastic.co/t/logstash-loosing-data-during-processing/60582 "2016-09-15T08:27:17Z")
Hi everyone, I'm experiencing strange behavior of logstash when processing data. My setup is central rsyslog which receive all logs from all servers, this rsyslog sends logs to logstash server. Logstash server setup: … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1100) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1102)