# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1011 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1012 --- ## [Logstash keep restarting and doesn't listen to beats](https://discuss.elastic.co/t/logstash-keep-restarting-and-doesnt-listen-to-beats/80001)
**Author:** [@sahar\_q](https://discuss.elastic.co/u/sahar_q)\ **Replies:** 1\ **Last updated:** [March 27, 2017, 7:21am UTC](https://discuss.elastic.co/t/logstash-keep-restarting-and-doesnt-listen-to-beats/80001 "2017-03-27T07:21:16Z")
Hi guys, i'm using ubuntu server 16.10 headless, i installed ELK and filebeat via apt, configured everything i should've (that i know of) and for some reason, logstash isn't getting the syslogs from the 'filebeat' , and… --- ## [Best way to stream logs from multiple remote servers to single Logstash server](https://discuss.elastic.co/t/best-way-to-stream-logs-from-multiple-remote-servers-to-single-logstash-server/79957)
**Author:** [@netaibiswas](https://discuss.elastic.co/u/netaibiswas)\ **Replies:** 1\ **Last updated:** [March 27, 2017, 7:17am UTC](https://discuss.elastic.co/t/best-way-to-stream-logs-from-multiple-remote-servers-to-single-logstash-server/79957 "2017-03-27T07:17:45Z")
Hi, I am planning to read log files from different servers (different hosts) using single Logstash server. Can you please suggest the best way to achieve it. Note – here all servers are running on Linux. Thanks, N… --- ## [Connection get refused while connecting to elasticsearch](https://discuss.elastic.co/t/connection-get-refused-while-connecting-to-elasticsearch/80075)
**Author:** [@DharaniKumar](https://discuss.elastic.co/u/DharaniKumar)\ **Replies:** 1\ **Last updated:** [March 27, 2017, 7:10am UTC](https://discuss.elastic.co/t/connection-get-refused-while-connecting-to-elasticsearch/80075 "2017-03-27T07:10:06Z")
:~$ /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --verbose starting agent {:level=\>:info} starting pipeline {:id=\>"… --- ## [I got this error when i did sudo service logstash configtest](https://discuss.elastic.co/t/i-got-this-error-when-i-did-sudo-service-logstash-configtest/80069)
**Author:** [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\ **Replies:** 3\ **Last updated:** [March 27, 2017, 6:03am UTC](https://discuss.elastic.co/t/i-got-this-error-when-i-did-sudo-service-logstash-configtest/80069 "2017-03-27T06:03:30Z")
Error: Expected one of #, if, ", ', } at line 49, column 2 (byte 1331) after filter { {:level=\>:error} --- ## [Trying to grok, grok](https://discuss.elastic.co/t/trying-to-grok-grok/80005)
**Author:** [@jmbrown](https://discuss.elastic.co/u/jmbrown)\ **Replies:** 1\ **Last updated:** [March 27, 2017, 5:19am UTC](https://discuss.elastic.co/t/trying-to-grok-grok/80005 "2017-03-27T05:19:10Z")
Hi, I'm a newcomer to ELK. Trying to get a few concepts wrapped in my head. I've got several different devices (cisco, juniper, ubiquiti) pushing syslogs to our logstash. I'm trying to figure out how to parse them base… --- ## [Authentication and Authorization with Logstash](https://discuss.elastic.co/t/authentication-and-authorization-with-logstash/80015)
**Author:** [@shabirmean](https://discuss.elastic.co/u/shabirmean)\ **Replies:** 1\ **Last updated:** [March 27, 2017, 5:17am UTC](https://discuss.elastic.co/t/authentication-and-authorization-with-logstash/80015 "2017-03-27T05:17:59Z")
Hi, I am a completely new user of logstash. I am using log stash simply to parse some files (ideally csv) and publish them to kafka. As of now the the way it works is the file-input plugin checks for new files in a spe… --- ## [Logstash erroring with exception](https://discuss.elastic.co/t/logstash-erroring-with-exception/79958)
**Author:** [@dubul](https://discuss.elastic.co/u/dubul)\ **Replies:** 1\ **Last updated:** [March 27, 2017, 5:13am UTC](https://discuss.elastic.co/t/logstash-erroring-with-exception/79958 "2017-03-27T05:13:47Z")
←\[33mCircuitBreaker::rescuing exceptions {:name=\>"Beats input", :exception=\>LogS tash::Inputs::Beats::InsertingToQueueTakeTooLong, :level=\>:warn}←\[0m ←\[33mBeats input: The circuit breaker has detected a slowdown or stall… --- ## [Grok into a timestamp](https://discuss.elastic.co/t/grok-into-a-timestamp/79946)
**Author:** [@mhooper](https://discuss.elastic.co/u/mhooper)\ **Replies:** 1\ **Last updated:** [March 27, 2017, 5:12am UTC](https://discuss.elastic.co/t/grok-into-a-timestamp/79946 "2017-03-27T05:12:50Z")
What is the cleanest way to match log lines that contain timestamps in this format \[20170324, 00:43:07.637\] into @timestamp? thank you --- ## [Logstash server errors](https://discuss.elastic.co/t/logstash-server-errors/79494)
**Author:** [@rmelton](https://discuss.elastic.co/u/rmelton)\ **Replies:** 9\ **Last updated:** [March 26, 2017, 9:00pm UTC](https://discuss.elastic.co/t/logstash-server-errors/79494 "2017-03-26T21:00:05Z")
installed ELK stack 5.2.2 on ubuntu 14.04 server image and put 5.2.2 filebeat on 5 other servers to send logs up to logstash. I can curl elasticsearch and login to kibana but my new logstash is flooding errors into log… --- ## [Syslog to elasticsearch, can't get it working](https://discuss.elastic.co/t/syslog-to-elasticsearch-cant-get-it-working/80003)
**Author:** [@Jeroen\_Jacobs](https://discuss.elastic.co/u/Jeroen_Jacobs)\ **Replies:** 1\ **Last updated:** [March 26, 2017, 4:38pm UTC](https://discuss.elastic.co/t/syslog-to-elasticsearch-cant-get-it-working/80003 "2017-03-26T16:38:47Z")
set up logstash like this: input { syslog { port =\> 1514 type =\> syslog } } output { elasticsearch { hosts =\> \["http://elasticsearch.service.consul:9200"\] } } However, I'm not receiving any dat… --- ## [Elapsed plugin modify/adding a field not possible](https://discuss.elastic.co/t/elapsed-plugin-modify-adding-a-field-not-possible/80033)
**Author:** [@kumar-m-kiran](https://discuss.elastic.co/u/kumar-m-kiran)\ **Replies:** 1\ **Last updated:** [March 26, 2017, 3:37pm UTC](https://discuss.elastic.co/t/elapsed-plugin-modify-adding-a-field-not-possible/80033 "2017-03-26T15:37:28Z")
Consider the below (watered down version ) of the configuration file Configuration file input { tcp { port =\> 5000 type =\> syslog } udp { port =\> 5000 type =\> syslog } } filter { if \[typ… --- ## [Didnt get the output of cisco router syslog in logstash server](https://discuss.elastic.co/t/didnt-get-the-output-of-cisco-router-syslog-in-logstash-server/58532)
**Author:** [@shamimreza](https://discuss.elastic.co/u/shamimreza)\ **Replies:** 5\ **Last updated:** [March 26, 2017, 9:46am UTC](https://discuss.elastic.co/t/didnt-get-the-output-of-cisco-router-syslog-in-logstash-server/58532 "2017-03-26T09:46:05Z")
OS is CentOS 7.. Packages: logstash-2.2.4-1, elasticsearch-2.3.5-1, kibana-4.4.2-1 Logstash configuration is bellow: input { beats { port =\> 5044 ssl =\> true ssl\_certificate =\> "/etc/pki/tls/certs/log… --- ## [Find delta between successive bunch of mysql input fields](https://discuss.elastic.co/t/find-delta-between-successive-bunch-of-mysql-input-fields/79956)
**Author:** [@pks](https://discuss.elastic.co/u/pks)\ **Replies:** 1\ **Last updated:** [March 25, 2017, 8:37pm UTC](https://discuss.elastic.co/t/find-delta-between-successive-bunch-of-mysql-input-fields/79956 "2017-03-25T20:37:26Z")
I have mysql input via jdbc plugin which has a bunch of incremental values (per minute) for some fields like below 'timestamp' 'id' 'value1' 'value 2' .... 00:00 1 10 14 00:00 … --- ## [Date filter with nano seconds](https://discuss.elastic.co/t/date-filter-with-nano-seconds/80004)
**Author:** [@Toontje](https://discuss.elastic.co/u/Toontje)\ **Replies:** 1\ **Last updated:** [March 25, 2017, 7:23pm UTC](https://discuss.elastic.co/t/date-filter-with-nano-seconds/80004 "2017-03-25T19:23:52Z")
Hi all! I have a date filed in UNIX format, but not up to the microsecond, but up to the nano second. Example: 1448482607735645 For the date filter to get a decent date out of this i need to divide the content of the d… --- ## [Bulk data import from MS SQL Server to Elasticsearch using logstash](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731)
**Author:** [@vtangutoori](https://discuss.elastic.co/u/vtangutoori)\ **Replies:** 4\ **Last updated:** [March 25, 2017, 2:26pm UTC](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731 "2017-03-25T14:26:20Z")
Hi, I am currently using Logstash and SQL Server JDBC to import data from SQL Server to Elasticsearch. The problem i am facing is that the SQL table has 4 million records in it and it takes forever to just upload a 100K … --- ## [Logstash – Both elasticsearch and email outputs](https://discuss.elastic.co/t/logstash-both-elasticsearch-and-email-outputs/25714)
**Author:** [@Ben19](https://discuss.elastic.co/u/Ben19)\ **Replies:** 16\ **Last updated:** [March 24, 2017, 8:38pm UTC](https://discuss.elastic.co/t/logstash-both-elasticsearch-and-email-outputs/25714 "2017-03-24T20:38:00Z")
Hello, Please can some provide an example config that provides two outputs of the same input (syslog)? I’m wanting to pass syslogs both to elasticserarch and email. The below is my current output conf, the e-mail s… --- ## [Performance decrease after upgrading to LS 5](https://discuss.elastic.co/t/performance-decrease-after-upgrading-to-ls-5/79906)
**Author:** [@lfkeitel](https://discuss.elastic.co/u/lfkeitel)\ **Replies:** 2\ **Last updated:** [March 24, 2017, 7:59pm UTC](https://discuss.elastic.co/t/performance-decrease-after-upgrading-to-ls-5/79906 "2017-03-24T19:59:13Z")
We have a system in place that collects, parses, and indexes our firewall logs. We have one server that does the actual Logstash filtering before putting logs in ES. After upgrading to Logstash 5.2 from 2.4, throughput w… --- ## [Read log files from remote location in Linux](https://discuss.elastic.co/t/read-log-files-from-remote-location-in-linux/79935)
**Author:** [@netaibiswas](https://discuss.elastic.co/u/netaibiswas)\ **Replies:** 2\ **Last updated:** [March 24, 2017, 6:49pm UTC](https://discuss.elastic.co/t/read-log-files-from-remote-location-in-linux/79935 "2017-03-24T18:49:05Z")
Hi, I was trying to read log file using Logstash I couldn’t make it working. I have set up Logstash and application server in different Linux box. I have configured something like this, can you please help me out f… --- ## [Cannot add custom headers to http poll input in logstash](https://discuss.elastic.co/t/cannot-add-custom-headers-to-http-poll-input-in-logstash/48870)
**Author:** [@matt617](https://discuss.elastic.co/u/matt617)\ **Replies:** 1\ **Last updated:** [March 24, 2017, 6:19pm UTC](https://discuss.elastic.co/t/cannot-add-custom-headers-to-http-poll-input-in-logstash/48870 "2017-03-24T18:19:05Z")
trying to get data from the uber API to come to stdout (just as an initial test) but cannot seem to get the authorization through the HTTP Header. tried the code below. i think i am close but what am i doing wrong? inp… --- ## [Error using date match filter](https://discuss.elastic.co/t/error-using-date-match-filter/45740)
**Author:** [@Wolls013](https://discuss.elastic.co/u/Wolls013)\ **Replies:** 4\ **Last updated:** [March 24, 2017, 6:10pm UTC](https://discuss.elastic.co/t/error-using-date-match-filter/45740 "2017-03-24T18:10:15Z")
Hi everyone, I keep getting an error while trying to make my timestamp ("time") field from my MS SQL database the @timestamp field using the date match filter in logstash. This is my input i'm using for my timestam… --- ## [Logstash-filter-metrics used 100% CPU?](https://discuss.elastic.co/t/logstash-filter-metrics-used-100-cpu/79765)
**Author:** [@srdegeus](https://discuss.elastic.co/u/srdegeus)\ **Replies:** 2\ **Last updated:** [March 24, 2017, 3:11pm UTC](https://discuss.elastic.co/t/logstash-filter-metrics-used-100-cpu/79765 "2017-03-24T15:11:47Z")
I wanted to get an idication of the ingest speed of logstash and found that the (buildin) plugin logstash-filter-metrics could do this. After adding the example code to my config file, logstash starts printing a rate o… --- ## [Set up Logstash -\> HA Proxy -\> Elasticsearch](https://discuss.elastic.co/t/set-up-logstash-ha-proxy-elasticsearch/79910)
**Author:** [@laangarita](https://discuss.elastic.co/u/laangarita)\ **Replies:** 0\ **Last updated:** [March 24, 2017, 3:07pm UTC](https://discuss.elastic.co/t/set-up-logstash-ha-proxy-elasticsearch/79910 "2017-03-24T15:07:17Z")
Hi, I'm seeing lots of 429 errors in our Elk stack version 2.2. We have 5 clusters set up each with 24 data nodes. So the current data pipeline is: log\_shipper -\> kafka \<- logstash (as kafka consumer) -\> elasticsear… --- ## [API: Get messages per second from input and output](https://discuss.elastic.co/t/api-get-messages-per-second-from-input-and-output/79878)
**Author:** [@AlexW](https://discuss.elastic.co/u/AlexW)\ **Replies:** 2\ **Last updated:** [March 24, 2017, 2:46pm UTC](https://discuss.elastic.co/t/api-get-messages-per-second-from-input-and-output/79878 "2017-03-24T14:46:25Z")
Hi! Is it possible to get the messages digested by Logstash via an API call? I checked the documentation and did not find anything about that. We would like to add that to our monitoring and graph it. Thanks for your h… --- ## [Any way to limit field length?](https://discuss.elastic.co/t/any-way-to-limit-field-length/54734)
**Author:** [@Benoit-DunandLaisin](https://discuss.elastic.co/u/Benoit-DunandLaisin)\ **Replies:** 4\ **Last updated:** [March 24, 2017, 2:22pm UTC](https://discuss.elastic.co/t/any-way-to-limit-field-length/54734 "2017-03-24T14:22:17Z")
Hi, I sometimes receive messages that are huge... (719939 bytes) and indexation to elasticsearch failed with a max\_bytes\_length\_exceeded\_exception (max limit is 32766 bytes). I want to detect those messages in kibana, … --- ## [Grok filter sequence](https://discuss.elastic.co/t/grok-filter-sequence/79881)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [March 24, 2017, 1:08pm UTC](https://discuss.elastic.co/t/grok-filter-sequence/79881 "2017-03-24T13:08:36Z")
Hi, I am using below 3 filters in my test configuration. grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:eventTime}" } tag\_on\_failure =\> \[ "\_grok\_match\_fail\_1" \] } grok { match =\> … --- ## [After configuring the X-PACK log is not coming to logstash](https://discuss.elastic.co/t/after-configuring-the-x-pack-log-is-not-coming-to-logstash/74976)
**Author:** [@shubhrant](https://discuss.elastic.co/u/shubhrant)\ **Replies:** 2\ **Last updated:** [March 24, 2017, 11:42am UTC](https://discuss.elastic.co/t/after-configuring-the-x-pack-log-is-not-coming-to-logstash/74976 "2017-03-24T11:42:29Z")
earlier the log was processing normally but after configuring the X-PACK log is not coming ... my logstash configuration is input { tcp { port =\> 5044 charset =\> "ISO-8859-1" } } filter { if… --- ## [ELK to slow](https://discuss.elastic.co/t/elk-to-slow/79312)
**Author:** [@permyakovsv](https://discuss.elastic.co/u/permyakovsv)\ **Replies:** 14\ **Last updated:** [March 24, 2017, 11:30am UTC](https://discuss.elastic.co/t/elk-to-slow/79312 "2017-03-24T11:30:39Z")
I have elk installation: logshash -\> rmq -\> logstash(2 instances) -\> Elasticsearch Elasticsearch cluster with 3 nodes. elasticsearch-5.2 (8 cores, 32 Gb, SSD) 12Gb HEAP "number\_of\_replicas": 0 cluster.name: logs no… --- ## [Alternate to multiple if..else-if in Logstash .conf file](https://discuss.elastic.co/t/alternate-to-multiple-if-else-if-in-logstash-conf-file/79835)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 3\ **Last updated:** [March 24, 2017, 11:20am UTC](https://discuss.elastic.co/t/alternate-to-multiple-if-else-if-in-logstash-conf-file/79835 "2017-03-24T11:20:02Z")
Hi, In my log file, there is a URI field. Now, I have added another field named 'user\_operation' with a default value and want to assign different values to this 'user\_operation' field depending on the value of the … --- ## [Mapping input and output in configuration](https://discuss.elastic.co/t/mapping-input-and-output-in-configuration/79448)
**Author:** [@waubau](https://discuss.elastic.co/u/waubau)\ **Replies:** 7\ **Last updated:** [March 24, 2017, 7:21am UTC](https://discuss.elastic.co/t/mapping-input-and-output-in-configuration/79448 "2017-03-24T07:21:05Z")
I have bumped into very weird thing. "internet" says that I can't just map input and output of logstash config because of single pipeline or whatever. So I have multiple RabbitMQ queues that I want to read and created di… --- ## [Geo\_point and the logstash-\* index](https://discuss.elastic.co/t/geo-point-and-the-logstash-index/79854)
**Author:** [@mikemi](https://discuss.elastic.co/u/mikemi)\ **Replies:** 1\ **Last updated:** [March 24, 2017, 7:08am UTC](https://discuss.elastic.co/t/geo-point-and-the-logstash-index/79854 "2017-03-24T07:08:06Z")
When I use Logstash's geoip to encode an ip address, I get different field types in Elasticsearch when i use logstash-foo as an index vs logs-foo as an index. logstash-foo (matches the default index naming convention)… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1010) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1012)