# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=946 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 947 --- ## [Parse docker logs with logstash](https://discuss.elastic.co/t/parse-docker-logs-with-logstash/92972)
**Author:** [@DevOpsRoot](https://discuss.elastic.co/u/DevOpsRoot)\ **Replies:** 4\ **Last updated:** [July 13, 2017, 9:43am UTC](https://discuss.elastic.co/t/parse-docker-logs-with-logstash/92972 "2017-07-13T09:43:58Z")
Hello all, I want to parse logs lines like this one : 2017-07-12T12:50:37.944779015Z \[2017-07-12T12:50:37,944\]\[INFO \]\[logstash.pipeline \] Pipeline main started How can i do that ? with grok ? wich plug-in… --- ## [Errors when connecting Filebeat to logstash](https://discuss.elastic.co/t/errors-when-connecting-filebeat-to-logstash/92835)
**Author:** [@Baco](https://discuss.elastic.co/u/Baco)\ **Replies:** 6\ **Last updated:** [July 13, 2017, 9:43am UTC](https://discuss.elastic.co/t/errors-when-connecting-filebeat-to-logstash/92835 "2017-07-13T09:43:22Z")
Hello, Would you like to help me understand why i have errors below when I try to connect filebeat to logstash. An unexpected error occurred! {:error=\>#\ **Author:** [@Patrik\_Soderstrom](https://discuss.elastic.co/u/Patrik_Soderstrom)\ **Replies:** 1\ **Last updated:** [July 13, 2017, 7:59am UTC](https://discuss.elastic.co/t/logstash-input-rabbitmq/92952 "2017-07-13T07:59:13Z") Hi, I currently have an RabbitMQ input and it works fine. I get the data in to elasticsearch. What I have problem with is using a mutate to remove a field before its sent to elasticsearch output. mutate { … --- ## [Logstash IP to Hostname](https://discuss.elastic.co/t/logstash-ip-to-hostname/92912)
**Author:** [@vvserpent](https://discuss.elastic.co/u/vvserpent)\ **Replies:** 2\ **Last updated:** [July 13, 2017, 6:59am UTC](https://discuss.elastic.co/t/logstash-ip-to-hostname/92912 "2017-07-13T06:59:48Z")
I have configured Logstash to receive the syslog message from the networking devices. The dnsmasq and static host-file are used to translate the IP address to hostname. The system is working for few months. I have … --- ## [Installation of logstash-input-kafka without internet connection](https://discuss.elastic.co/t/installation-of-logstash-input-kafka-without-internet-connection/92625)
**Author:** [@zeenon](https://discuss.elastic.co/u/zeenon)\ **Replies:** 3\ **Last updated:** [July 13, 2017, 6:48am UTC](https://discuss.elastic.co/t/installation-of-logstash-input-kafka-without-internet-connection/92625 "2017-07-13T06:48:41Z")
Hi, I want to install logstash-input-kafka without internet connection on the server. After a lot of search, I have downloaded a gem file: logstash-input-kafka-4.2.0.gem Now, I am running the command # /opt/logstash… --- ## [Changing the datatype of Date Filter in Logstash ,doesn't change the datatype in Kibana](https://discuss.elastic.co/t/changing-the-datatype-of-date-filter-in-logstash-doesnt-change-the-datatype-in-kibana/92819)
**Author:** [@ppramanick](https://discuss.elastic.co/u/ppramanick)\ **Replies:** 17\ **Last updated:** [July 13, 2017, 6:02am UTC](https://discuss.elastic.co/t/changing-the-datatype-of-date-filter-in-logstash-doesnt-change-the-datatype-in-kibana/92819 "2017-07-13T06:02:51Z")
Hi ALL, My issue is I am using logstash to parse a CSV file and then see the visulization in Kibana ,however the date field is changing to string in Kibana,What I want is I will pass Date field as date and get Date in… --- ## [Plugin for cisco nexus 7000](https://discuss.elastic.co/t/plugin-for-cisco-nexus-7000/92926)
**Author:** [@oded17](https://discuss.elastic.co/u/oded17)\ **Replies:** 1\ **Last updated:** [July 13, 2017, 6:01am UTC](https://discuss.elastic.co/t/plugin-for-cisco-nexus-7000/92926 "2017-07-13T06:01:26Z")
Hi i wanted to know if there is plugin for cisco nexus 7000 to get logs from machine. --- ## [Logstash input for HBase](https://discuss.elastic.co/t/logstash-input-for-hbase/92902)
**Author:** [@Hyojoung\_Shin](https://discuss.elastic.co/u/Hyojoung_Shin)\ **Replies:** 2\ **Last updated:** [July 13, 2017, 4:49am UTC](https://discuss.elastic.co/t/logstash-input-for-hbase/92902 "2017-07-13T04:49:36Z")
Hi, Thank you for your amazing works! I am trying to connect HBase with elasticsearch. Does logstash have input plugin for the HBase? I am not able to find it in docs. If you have any suggestion to send data from… --- ## [Appending data from MongoDB into log files being processed by Logstash and parsed into Elasticsearch](https://discuss.elastic.co/t/appending-data-from-mongodb-into-log-files-being-processed-by-logstash-and-parsed-into-elasticsearch/92564)
**Author:** [@kmin93](https://discuss.elastic.co/u/kmin93)\ **Replies:** 2\ **Last updated:** [July 11, 2017, 5:33am UTC](https://discuss.elastic.co/t/appending-data-from-mongodb-into-log-files-being-processed-by-logstash-and-parsed-into-elasticsearch/92564 "2017-07-11T05:33:19Z")
Sorry about the title, my case really couldn't be explained with a single sentence. Here is my situation: I have a large set of log files (around 4GB) that I wish to parse with Logstash to use with Elastic stack (Lo… --- ## [Problem with duplicate data in field](https://discuss.elastic.co/t/problem-with-duplicate-data-in-field/92598)
**Author:** [@ahongyun](https://discuss.elastic.co/u/ahongyun)\ **Replies:** 2\ **Last updated:** [July 13, 2017, 1:10am UTC](https://discuss.elastic.co/t/problem-with-duplicate-data-in-field/92598 "2017-07-13T01:10:45Z")
Hi all I have an issue with the data that is passed into elasticsearch from logstash below is my logstash config. if \[type\] == "coreservices-fs" or "coreservices-net" { grok { match =\> { "message" =\> \[ "^%{TIMESTA… --- ## [Can we add pattern matching in document\_id while updating?](https://discuss.elastic.co/t/can-we-add-pattern-matching-in-document-id-while-updating/92899)
**Author:** [@g\_dev](https://discuss.elastic.co/u/g_dev)\ **Replies:** 0\ **Last updated:** [July 12, 2017, 11:46pm UTC](https://discuss.elastic.co/t/can-we-add-pattern-matching-in-document-id-while-updating/92899 "2017-07-12T23:46:33Z")
I want to add a new field to a document with document\_id like "{id}-{word}" by using logstash. I just know {id} part while updating. Is there a way can I do it either by update\_by\_query in script and how should I define… --- ## [How to create a file link for input file path using logstash filter or any other means to show up the input file as a link of the file in kibana?](https://discuss.elastic.co/t/how-to-create-a-file-link-for-input-file-path-using-logstash-filter-or-any-other-means-to-show-up-the-input-file-as-a-link-of-the-file-in-kibana/92871)
**Author:** [@hara\_acharya](https://discuss.elastic.co/u/hara_acharya)\ **Replies:** 2\ **Last updated:** [July 12, 2017, 10:14pm UTC](https://discuss.elastic.co/t/how-to-create-a-file-link-for-input-file-path-using-logstash-filter-or-any-other-means-to-show-up-the-input-file-as-a-link-of-the-file-in-kibana/92871 "2017-07-12T22:14:25Z")
Hi, I have a logstash filter as below. input { file { path =\> "/home/testserver/Downloads/log\_source/logs/\*/\*\*" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" type =\> "test-logs" } } filt… --- ## [Grok Parse Error even though regex is valid](https://discuss.elastic.co/t/grok-parse-error-even-though-regex-is-valid/92535)
**Author:** [@amarc](https://discuss.elastic.co/u/amarc)\ **Replies:** 3\ **Last updated:** [July 12, 2017, 9:50pm UTC](https://discuss.elastic.co/t/grok-parse-error-even-though-regex-is-valid/92535 "2017-07-12T21:50:47Z")
Grok Patterns: (truncated to matching patterns only) ## Fixing AVC ## Example: ## type=AVC msg=audit(1499716867.329:3287727): avc: denied { execute } for pid=29362 comm=\\"ld-linux-x86-64\\" path=\\"/usr/libexec/hal-sys… --- ## [Grokparsefailure error](https://discuss.elastic.co/t/grokparsefailure-error/92881)
**Author:** [@SandhyaRani](https://discuss.elastic.co/u/SandhyaRani)\ **Replies:** 0\ **Last updated:** [July 12, 2017, 9:34pm UTC](https://discuss.elastic.co/t/grokparsefailure-error/92881 "2017-07-12T21:34:21Z")
Hi there! I'm getting "grokparsefailure" error when I use "multiline" codec in the input, Is the "pattern" I had in my input is fine?? Here is my java exception message 2016-12-16 21:28:05,668 ERROR \[int-http-28\] \[nb… --- ## [Splitting message into fields to create chart based on those fields](https://discuss.elastic.co/t/splitting-message-into-fields-to-create-chart-based-on-those-fields/92291)
**Author:** [@BentCoder](https://discuss.elastic.co/u/BentCoder)\ **Replies:** 6\ **Last updated:** [July 12, 2017, 8:16pm UTC](https://discuss.elastic.co/t/splitting-message-into-fields-to-create-chart-based-on-those-fields/92291 "2017-07-12T20:16:27Z")
I want to be able to split log "message" into three fields (log\_time, log\_level, log\_data) so that I can create Pie Chart based on those fields - see image below. How do I do it? Current Logstash config: input { … --- ## [Keep Logstash from Crashing](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392)
**Author:** [@allenfin](https://discuss.elastic.co/u/allenfin)\ **Replies:** 5\ **Last updated:** [July 12, 2017, 8:03pm UTC](https://discuss.elastic.co/t/keep-logstash-from-crashing/90392 "2017-07-12T20:03:42Z")
Been having a hell of a time with logstash. Right now I setup a few different nginx instances to pump access and error logs into logstash to funnel into elasticsearch, but now I'm getting crashes and I have no idea how t… --- ## [How to count log entries based on a filter](https://discuss.elastic.co/t/how-to-count-log-entries-based-on-a-filter/92592)
**Author:** [@Jonathan\_Kosgei](https://discuss.elastic.co/u/Jonathan_Kosgei)\ **Replies:** 1\ **Last updated:** [July 12, 2017, 7:54pm UTC](https://discuss.elastic.co/t/how-to-count-log-entries-based-on-a-filter/92592 "2017-07-12T19:54:51Z")
I need to be able to count how many requests each user makes to a particular api, each request is logged to logstash and contains the consumerID and the name of the api, I'd like to be able to get a count of how many log… --- ## [Logstach http\_pool input plugin](https://discuss.elastic.co/t/logstach-http-pool-input-plugin/92640)
**Author:** [@huazhang](https://discuss.elastic.co/u/huazhang)\ **Replies:** 1\ **Last updated:** [July 12, 2017, 7:51pm UTC](https://discuss.elastic.co/t/logstach-http-pool-input-plugin/92640 "2017-07-12T19:51:54Z")
Dear experts, logstash 5.4.1 i am trying use the logstach http\_pool plugin to get the http response. as i like use the following command to take the test. it working fine and get the json response. curl -k -H "cont… --- ## [Strange behavior with ELK setup](https://discuss.elastic.co/t/strange-behavior-with-elk-setup/92332)
**Author:** [@rahul.dhammy](https://discuss.elastic.co/u/rahul.dhammy)\ **Replies:** 8\ **Last updated:** [July 12, 2017, 7:26pm UTC](https://discuss.elastic.co/t/strange-behavior-with-elk-setup/92332 "2017-07-12T19:26:41Z")
Here is the setup for my ELK setup. I have about 3 data shippers running on different boxes which ships the log files to central logstash server. The logstash server parses these events and outputs it to Kibana. I have b… --- ## [Elasticsearch input plugin - not pulling new records](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-pulling-new-records/92692)
**Author:** [@mbucknel](https://discuss.elastic.co/u/mbucknel)\ **Replies:** 1\ **Last updated:** [July 12, 2017, 7:10pm UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-pulling-new-records/92692 "2017-07-12T19:10:46Z")
I am using the Elasticsearch input plugin, to grab data from an elasticseach index and then use it for a translation filter, it queries elasticseach, then wtites out a file in YAML format with the translation pairs. This… --- ## [LogStash multiple files in single location](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490)
**Author:** [@ShashiK](https://discuss.elastic.co/u/ShashiK)\ **Replies:** 4\ **Last updated:** [July 12, 2017, 6:01pm UTC](https://discuss.elastic.co/t/logstash-multiple-files-in-single-location/92490 "2017-07-12T18:01:57Z")
Logstash works for me when i have a direct file reference - When I use the single file reference in the logstash config- path =\> "C:\\LogRepository\\transaction\\Web\\HttpUser\_1312.csv" It works for me. However when i put … --- ## [Logstash node memory settings](https://discuss.elastic.co/t/logstash-node-memory-settings/92475)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 4\ **Last updated:** [July 12, 2017, 5:41pm UTC](https://discuss.elastic.co/t/logstash-node-memory-settings/92475 "2017-07-12T17:41:50Z")
Logstash node has 16GB RAM and max and min heap size has been set to 8GB. Seeing the following error when testing the logstash pipeline. What is the recommended setting to avoid this issue? Less than 8GB for min and ma… --- ## [Is there a SysV init script in /etc/init.d/ for logstash?](https://discuss.elastic.co/t/is-there-a-sysv-init-script-in-etc-init-d-for-logstash/92759)
**Author:** [@KeithTt](https://discuss.elastic.co/u/KeithTt)\ **Replies:** 2\ **Last updated:** [July 12, 2017, 5:37pm UTC](https://discuss.elastic.co/t/is-there-a-sysv-init-script-in-etc-init-d-for-logstash/92759 "2017-07-12T17:37:51Z")
logstash version: 5.4.1 OS version: CentOS6.6 Refer to the official guide, it seems that there is a auto-generated configuration file at /etc/init.d/logstash. I installed logstash with RMP package, but there is no a … --- ## [Grok query domain from DNS server log](https://discuss.elastic.co/t/grok-query-domain-from-dns-server-log/92770)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 4\ **Last updated:** [July 12, 2017, 3:35pm UTC](https://discuss.elastic.co/t/grok-query-domain-from-dns-server-log/92770 "2017-07-12T15:35:09Z")
Hi, I'm using Elastic Stack V5.5 and using logstahs for parse log from DNS server log. Something like that (6)mobile(4)pipe(4)aria(9)microsoft(3)com(0) (7)outlook(6)office(3)com(0) (5)nexus(10)officeapps(4)live(3)com… --- ## [Grok parsing using custom pattern](https://discuss.elastic.co/t/grok-parsing-using-custom-pattern/92309)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 2\ **Last updated:** [July 12, 2017, 3:25pm UTC](https://discuss.elastic.co/t/grok-parsing-using-custom-pattern/92309 "2017-07-12T15:25:03Z")
Hi I am trying to parse a log file using grok. Basically trying to write and test grok pattern in https://grokdebug.herokuapp.com/ The patterns for the two words log "\[\*\*\] \[1:2123:5\]" that i came upon is \[\*\*\] \\\\\[%{N… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=945) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=947)