# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=770 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 771 --- ## [Logstash 5.6.1 inifinite loop? Encountered a retryable error](https://discuss.elastic.co/t/logstash-5-6-1-inifinite-loop-encountered-a-retryable-error/101753)
**Author:** [@Hush077](https://discuss.elastic.co/u/Hush077)\ **Replies:** 24\ **Last updated:** [April 30, 2018, 2:53pm UTC](https://discuss.elastic.co/t/logstash-5-6-1-inifinite-loop-encountered-a-retryable-error/101753 "2018-04-30T14:53:27Z")
Upgraded to 5.6.1 this morning on all pieces of the ELK stack. Logstash started outputting this in the logs and all messages have thus stopped: \[2017-09-25T19:54:44,850\]\[ERROR\]\[logstash.outputs.elasticsearch\] Encounter… --- ## [Logstash 6.2.4 netflow module no packets no bytes](https://discuss.elastic.co/t/logstash-6-2-4-netflow-module-no-packets-no-bytes/129936)
**Author:** [@dalamar666](https://discuss.elastic.co/u/dalamar666)\ **Replies:** 2\ **Last updated:** [April 30, 2018, 12:34pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-netflow-module-no-packets-no-bytes/129936 "2018-04-30T12:34:23Z")
Hi, I am running an ELK stack 6.2 on Ubuntu 16.04. I have netflow configured on my cisco ASA 5515 to come to the server. I can see from tshark that the packets are coming. When I debug logstash I can see it is gettin… --- ## [Multisplit](https://discuss.elastic.co/t/multisplit/129793)
**Author:** [@laurentbillon](https://discuss.elastic.co/u/laurentbillon)\ **Replies:** 15\ **Last updated:** [April 30, 2018, 12:19pm UTC](https://discuss.elastic.co/t/multisplit/129793 "2018-04-30T12:19:31Z")
Good afternoon, I have log with : and - for separate information. I want to select information between : and - I try with ruby ruby { code =\> "event.set('source', event.get('origine').split(':')\[1\])"; event.set('I… --- ## [Force Line to use perticular grok](https://discuss.elastic.co/t/force-line-to-use-perticular-grok/129974)
**Author:** [@SarojELK](https://discuss.elastic.co/u/SarojELK)\ **Replies:** 6\ **Last updated:** [April 30, 2018, 11:47am UTC](https://discuss.elastic.co/t/force-line-to-use-perticular-grok/129974 "2018-04-30T11:47:26Z")
Hi experts. I have my multineline filter as below in my logstash configuration my sample log lines if \[doc\_type\] =~ "Indexer" { grok { patterns\_dir =\> \["C:/ELK6.2/logstash-6.2.2/vendor/bundle/jruby/2.3.0/gem… --- ## [Can we use regex check in if conditional?](https://discuss.elastic.co/t/can-we-use-regex-check-in-if-conditional/129786)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 8\ **Last updated:** [April 30, 2018, 9:49am UTC](https://discuss.elastic.co/t/can-we-use-regex-check-in-if-conditional/129786 "2018-04-30T09:49:40Z")
Hi, I want to check if regex matched in if conditional then extract the line which is matched and add in ES. if regexMatched { add\_field for matched lines } --- ## [After installing x-pack 5.4 , logstash 5.4 not working](https://discuss.elastic.co/t/after-installing-x-pack-5-4-logstash-5-4-not-working/129940)
**Author:** [@hedieh](https://discuss.elastic.co/u/hedieh)\ **Replies:** 5\ **Last updated:** [April 30, 2018, 7:56am UTC](https://discuss.elastic.co/t/after-installing-x-pack-5-4-logstash-5-4-not-working/129940 "2018-04-30T07:56:37Z")
Hi, after install x-pack on logstash , i get error below \[2018-04-29T11:19:56,973\]\[WARN \]\[logstash.outputs.elasticsearch\] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\ **Author:** [@miki\_haiat](https://discuss.elastic.co/u/miki_haiat)\ **Replies:** 0\ **Last updated:** [April 29, 2018, 3:13pm UTC](https://discuss.elastic.co/t/http-input-tcp-time-wait-issue/129962 "2018-04-29T15:13:08Z") Hi , Im using windows and logsthsh 6.2.4 . We have some Http to Kafka pipeline . For some reason tcp\_time wait are created greatly . Is thire a way to configure logsthsh to get better tcp behavior ? --- ## [Logstash/ElasticSearch mixing data across indexes](https://discuss.elastic.co/t/logstash-elasticsearch-mixing-data-across-indexes/129934)
**Author:** [@enigy](https://discuss.elastic.co/u/enigy)\ **Replies:** 2\ **Last updated:** [April 29, 2018, 11:39am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-mixing-data-across-indexes/129934 "2018-04-29T11:39:21Z")
Hello everyone! I'm a fairly new user to ELK. I've been attempting to setup a single node server to receive two syslog streams in my home lab. I've run into a situation I cannot find a resolution to. Simply put, when… --- ## [Indexing mails to elasticsearch](https://discuss.elastic.co/t/indexing-mails-to-elasticsearch/128482)
**Author:** [@rouchad\_rouchad](https://discuss.elastic.co/u/rouchad_rouchad)\ **Replies:** 6\ **Last updated:** [April 29, 2018, 11:00am UTC](https://discuss.elastic.co/t/indexing-mails-to-elasticsearch/128482 "2018-04-29T11:00:32Z")
please i need ur help i can't index my emails into elastic search C:\\elk\\logstash\\bin\>logstash -f parsemail.conf --config.reload.automatic Sending Logstash's logs to C:/elk/logstash/logs which is now configured via log… --- ## [Pipeline aborted due to error PatternError.pattern not defined](https://discuss.elastic.co/t/pipeline-aborted-due-to-error-patternerror-pattern-not-defined/125876)
**Author:** [@SashaAlex](https://discuss.elastic.co/u/SashaAlex)\ **Replies:** 1\ **Last updated:** [March 31, 2018, 10:01pm UTC](https://discuss.elastic.co/t/pipeline-aborted-due-to-error-patternerror-pattern-not-defined/125876 "2018-03-31T22:01:32Z")
In my project I run logstash as service.But it gives following error. how to correct this \[2018-03-28T07:03:03,652\]\[ERROR\]\[logstash.agent \] Pipeline aborted due to error {:exception=\>#\ **Author:** [@DSM](https://discuss.elastic.co/u/DSM)\ **Replies:** 2\ **Last updated:** [April 28, 2018, 8:07pm UTC](https://discuss.elastic.co/t/only-get-900-netflow-events-sec-from-4-vcpu-ls-node/129928 "2018-04-28T20:07:22Z") Hi, I'm using the netflow module \[ELK 6.2.4\] and struggling to squeeze a decent amount flows/events from my LS node. Hoping someone here can see what i may be missing. My (virtualised) setup: 1 x LS VM Node; 8GB RAM (4… --- ## [Bad field when upload json data into elasticsearch](https://discuss.elastic.co/t/bad-field-when-upload-json-data-into-elasticsearch/129929)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 1\ **Last updated:** [April 28, 2018, 5:34pm UTC](https://discuss.elastic.co/t/bad-field-when-upload-json-data-into-elasticsearch/129929 "2018-04-28T17:34:57Z")
When I upload json data into elasticsearch some fields are dotted. I tried to use mutate plugin to rename dotted field but without success. filter { mutate { rename =\> { "src.ip" =\> "src\_ip" } } } --- ## [Expected \[START\_OBJECT\] but found \[VALUE\_STRING\]](https://discuss.elastic.co/t/expected-start-object-but-found-value-string/129883)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 4\ **Last updated:** [April 28, 2018, 11:55am UTC](https://discuss.elastic.co/t/expected-start-object-but-found-value-string/129883 "2018-04-28T11:55:48Z")
I followed Elasticsearch input plugin | Logstash Reference to implement "Scheduling" of my pipeline: # grep schedule pipeline.cfg schedule =\> '0 \* \* \* \*' # yet logstash produces errors such as this: l… --- ## [Index per logstash instance?](https://discuss.elastic.co/t/index-per-logstash-instance/129874)
**Author:** [@jlixfeld](https://discuss.elastic.co/u/jlixfeld)\ **Replies:** 5\ **Last updated:** [April 28, 2018, 9:12am UTC](https://discuss.elastic.co/t/index-per-logstash-instance/129874 "2018-04-28T09:12:37Z")
Hi there, I've been trying out ELK to collect some Netflow data. I have two LS instances talking to one ES instance, but both LS instances are configured to output to the same ES index name. Needless to say it didn't … --- ## [Date plugin lose the stream to ElasticSearch](https://discuss.elastic.co/t/date-plugin-lose-the-stream-to-elasticsearch/129906)
**Author:** [@liuzhen](https://discuss.elastic.co/u/liuzhen)\ **Replies:** 1\ **Last updated:** [April 28, 2018, 8:12am UTC](https://discuss.elastic.co/t/date-plugin-lose-the-stream-to-elasticsearch/129906 "2018-04-28T08:12:40Z")
Hi, I am setting up one port for all my beats input, including redis & haproxy logs, where haproxy logs work but redis logs do not get into elastic search (after adding Date plugin). if I remove the Data plugin in the … --- ## [RabbitMQ output is missing queue parameter](https://discuss.elastic.co/t/rabbitmq-output-is-missing-queue-parameter/129858)
**Author:** [@smetj](https://discuss.elastic.co/u/smetj)\ **Replies:** 4\ **Last updated:** [April 27, 2018, 10:31pm UTC](https://discuss.elastic.co/t/rabbitmq-output-is-missing-queue-parameter/129858 "2018-04-27T22:31:30Z")
Hi, The RabbitMQ output plugin is missing the queue parameter hence it can't create queues. This makes any serious RabbitMQ usage impossible. When writing to AMQP it should be possible to do 3 things: create exchang… --- ## [CSV \_dateparsefailure](https://discuss.elastic.co/t/csv-dateparsefailure/129745)
**Author:** [@lowdmt](https://discuss.elastic.co/u/lowdmt)\ **Replies:** 6\ **Last updated:** [April 27, 2018, 8:16pm UTC](https://discuss.elastic.co/t/csv-dateparsefailure/129745 "2018-04-27T20:16:55Z")
Hi, I have an issue with \_dateparsefailure #example from CSV FirstNotifiedAt 31/03/2018 23:58:00 filter { date { match =\> \["FirstNotifiedAt", "dd/MM/yyyy HH:mm:ss"\] target =\> "@timestamp" } } The @timestamp in K… --- ## [I write logs into a file](https://discuss.elastic.co/t/i-write-logs-into-a-file/129861)
**Author:** [@Giorgio\_Cerruti](https://discuss.elastic.co/u/Giorgio_Cerruti)\ **Replies:** 0\ **Last updated:** [April 27, 2018, 4:10pm UTC](https://discuss.elastic.co/t/i-write-logs-into-a-file/129861 "2018-04-27T16:10:29Z")
Hello, I've tried a lot of configurations but I'm not able to allow logstash to log onto a file. I'm using this log4j configuration status = error name = LogstashPropertiesConfig appender.console.type = Console append… --- ## [Gsub before CSV parsing](https://discuss.elastic.co/t/gsub-before-csv-parsing/129824)
**Author:** [@SHSauler](https://discuss.elastic.co/u/SHSauler)\ **Replies:** 1\ **Last updated:** [April 27, 2018, 2:25pm UTC](https://discuss.elastic.co/t/gsub-before-csv-parsing/129824 "2018-04-27T14:25:15Z")
Hi guys, I'm trying to work around a logstash-filter-csv issue. The line needs to be cleaned from whitespace and quotes before the CSV parser runs. My current issue is probably that I don't know the "default field name… --- ## [Blindly parse JSON from logs and add it to elasticsearch](https://discuss.elastic.co/t/blindly-parse-json-from-logs-and-add-it-to-elasticsearch/129716)
**Author:** [@Jean-Charle\_Passepon](https://discuss.elastic.co/u/Jean-Charle_Passepon)\ **Replies:** 6\ **Last updated:** [April 27, 2018, 10:38am UTC](https://discuss.elastic.co/t/blindly-parse-json-from-logs-and-add-it-to-elasticsearch/129716 "2018-04-27T10:38:30Z")
Hello, I've started to read quite a few topic about it and either I don't understand it correctly or it's not possible. I'm receiving logs formatted this way: "\[INFO\] JSEXTRA:{\\"user\_id\\": \\"90:70:65:12:56:D5\\", \\"su… --- ## [Can't find out sincedb files](https://discuss.elastic.co/t/cant-find-out-sincedb-files/129785)
**Author:** [@Pinno\_Lin](https://discuss.elastic.co/u/Pinno_Lin)\ **Replies:** 1\ **Last updated:** [April 27, 2018, 9:04am UTC](https://discuss.elastic.co/t/cant-find-out-sincedb-files/129785 "2018-04-27T09:04:57Z")
Hi, I can't find out any sincedb files in $HOME or assign directory . The input configure above. When i restart logstash service. it always reload all record in files to Elasticsearch. but i not see any error about si… --- ## [Pipeline setting in LS 6.2.3](https://discuss.elastic.co/t/pipeline-setting-in-ls-6-2-3/129765)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 2\ **Last updated:** [April 27, 2018, 8:26am UTC](https://discuss.elastic.co/t/pipeline-setting-in-ls-6-2-3/129765 "2018-04-27T08:26:48Z")
Hi, I am facing problem with starting LS 6.2.3. I don't understand pipeline concept. Though I have given 2 diff pipeline ID in both logstash.yml and pipeline.yml to understand from where it is picking up? I created 2… --- ## [How use logstash not installed on appserver](https://discuss.elastic.co/t/how-use-logstash-not-installed-on-appserver/129781)
**Author:** [@jjwang](https://discuss.elastic.co/u/jjwang)\ **Replies:** 2\ **Last updated:** [April 27, 2018, 8:10am UTC](https://discuss.elastic.co/t/how-use-logstash-not-installed-on-appserver/129781 "2018-04-27T08:10:40Z")
I want to use logstash to collect application logs. But it's irregular install logstash on appserver in my company. So if I install logstash to another server, then how to collect application logs? How to configure logst… --- ## [Syslog messages into logstash](https://discuss.elastic.co/t/syslog-messages-into-logstash/129425)
**Author:** [@artem33region](https://discuss.elastic.co/u/artem33region)\ **Replies:** 5\ **Last updated:** [April 27, 2018, 8:08am UTC](https://discuss.elastic.co/t/syslog-messages-into-logstash/129425 "2018-04-27T08:08:46Z")
Hi all! I want to get syslog messages from servers. I have installed ELK stack on CentOS 7 server. There is "no default index pattern" How i understand i need to install and configure "filebeat" on client server side… --- ## [Custom Pattern in grok filter](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413)
**Author:** [@Akash\_Tanwar](https://discuss.elastic.co/u/Akash_Tanwar)\ **Replies:** 7\ **Last updated:** [April 27, 2018, 6:27am UTC](https://discuss.elastic.co/t/custom-pattern-in-grok-filter/129413 "2018-04-27T06:27:08Z")
Hi, I am very new to ELK. I was trying to apply inline custom pattern to my log file but my approach is giving me errors. Can someone help me on this? template of logs: 172.31.29.134 - - \[02/Feb/2018:06:25:05 +0000\] "… --- ## [I got a error while adding custom pattern](https://discuss.elastic.co/t/i-got-a-error-while-adding-custom-pattern/129607)
**Author:** [@Eswar\_Kumar\_Musiboin](https://discuss.elastic.co/u/Eswar_Kumar_Musiboin)\ **Replies:** 9\ **Last updated:** [April 27, 2018, 6:25am UTC](https://discuss.elastic.co/t/i-got-a-error-while-adding-custom-pattern/129607 "2018-04-27T06:25:03Z")
Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#\ **Author:** [@Maheshmadabul](https://discuss.elastic.co/u/Maheshmadabul)\ **Replies:** 3\ **Last updated:** [April 27, 2018, 6:23am UTC](https://discuss.elastic.co/t/index-not-getting-created-in-elasticsearch/129737 "2018-04-27T06:23:07Z") When i start the logstash with my conf file the following log getting printed and no index are getting created in the elastisearch Logstash log Sending Logstash's logs to E:/ELK/logstash-6.2.3/logs which is now configu… --- ## [How should I index the 100 billion log](https://discuss.elastic.co/t/how-should-i-index-the-100-billion-log/129751)
**Author:** [@woon\_minika](https://discuss.elastic.co/u/woon_minika)\ **Replies:** 3\ **Last updated:** [April 27, 2018, 3:48am UTC](https://discuss.elastic.co/t/how-should-i-index-the-100-billion-log/129751 "2018-04-27T03:48:20Z")
Hi, I have multi services which generating at least 2 million of log message per day, I’m using logstash to form a doc and I create new index for every new day log, but I’m feeling it should have better way to handle it… --- ## [Access to elasticsearch server from logstash into private subnet on AWS](https://discuss.elastic.co/t/access-to-elasticsearch-server-from-logstash-into-private-subnet-on-aws/129748)
**Author:** [@Hermann](https://discuss.elastic.co/u/Hermann)\ **Replies:** 4\ **Last updated:** [April 27, 2018, 1:12am UTC](https://discuss.elastic.co/t/access-to-elasticsearch-server-from-logstash-into-private-subnet-on-aws/129748 "2018-04-27T01:12:27Z")
Hi, can someone help me to resolve this problem? I'm new in ELK stack and I deployed it on AWS. Logstash and elasticseach are hosted on different EC2 instances into a private VPC subnet. These EC2 instances are hosted b… --- ## [Stashing Your First Event - Unknown Command '{' error](https://discuss.elastic.co/t/stashing-your-first-event-unknown-command-error/129686)
**Author:** [@yckoh](https://discuss.elastic.co/u/yckoh)\ **Replies:** 2\ **Last updated:** [April 27, 2018, 1:01am UTC](https://discuss.elastic.co/t/stashing-your-first-event-unknown-command-error/129686 "2018-04-27T01:01:01Z")
Hi, sorry I'm new to Logstash but I tried following the guide and got stuck with the following in my cmd prompt (I'm running on Windows). Is there something I missed?: \[image\] [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=769) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=771)