# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=766 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 767 --- ## [How can i implement RabbitMQ RPC on Logstash Server config file](https://discuss.elastic.co/t/how-can-i-implement-rabbitmq-rpc-on-logstash-server-config-file/130701)
**Author:** [@yaser](https://discuss.elastic.co/u/yaser)\ **Replies:** 0\ **Last updated:** [May 5, 2018, 6:45am UTC](https://discuss.elastic.co/t/how-can-i-implement-rabbitmq-rpc-on-logstash-server-config-file/130701 "2018-05-05T06:45:30Z")
the main problem is i can not get ack from logstash server . this is my config in logstash server : input { rabbitmq { user =\> "" password =\> "" exchange =\> "" queue =\> "" durable =\> true ack =\> true prefetch\_count =\> … --- ## [Cannot config logstash](https://discuss.elastic.co/t/cannot-config-logstash/130696)
**Author:** [@NIloufarVafaei](https://discuss.elastic.co/u/NIloufarVafaei)\ **Replies:** 2\ **Last updated:** [May 5, 2018, 6:36am UTC](https://discuss.elastic.co/t/cannot-config-logstash/130696 "2018-05-05T06:36:27Z")
i am new to using ELK. i can run my Elasticsearch on 9200 and same for my Kibana on 5601 . but my command on terminal( sudo service logstash configtest ) tack back this line :"logstash: unrecognized service" .how can i… --- ## [Syslog with millisecond pattern not matching](https://discuss.elastic.co/t/syslog-with-millisecond-pattern-not-matching/130678)
**Author:** [@nodesocket](https://discuss.elastic.co/u/nodesocket)\ **Replies:** 13\ **Last updated:** [May 5, 2018, 6:31am UTC](https://discuss.elastic.co/t/syslog-with-millisecond-pattern-not-matching/130678 "2018-05-05T06:31:20Z")
I am using the following: input { tcp { port =\> 5000 type =\> syslog } } filter { if \[type\] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_h… --- ## [Don't send tags to ElasticSearch](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551)
**Author:** [@mrubin](https://discuss.elastic.co/u/mrubin)\ **Replies:** 4\ **Last updated:** [May 4, 2018, 9:41pm UTC](https://discuss.elastic.co/t/dont-send-tags-to-elasticsearch/130551 "2018-05-04T21:41:42Z")
I am processing kafka events and sending them to ES. I tag events so that I can tell which kafka topic they were from. I then use the tag in the output to direct events from different kafka topics to different ES indexes… --- ## [\[FATAL\] 2018-05-04 16:06:38.656 \[LogStash::Runner\] runner - The given configuration is invalid. Reason: Illegal pattern component: T when trying to parse date](https://discuss.elastic.co/t/fatal-2018-05-04-16-06-38-656-logstash-runner-runner-the-given-configuration-is-invalid-reason-illegal-pattern-component-t-when-trying-to-parse-date/130643)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 3\ **Last updated:** [May 4, 2018, 8:29pm UTC](https://discuss.elastic.co/t/fatal-2018-05-04-16-06-38-656-logstash-runner-runner-the-given-configuration-is-invalid-reason-illegal-pattern-component-t-when-trying-to-parse-date/130643 "2018-05-04T20:29:49Z")
Hello everybody, I've got a CSV document which is imported thanks to fileabeat to logstash input, then it is filtered and then goes to ES to be saw in Kabana. I want to parse a date from this CSV file to use it for the… --- ## [Ingest of logs slows down over time](https://discuss.elastic.co/t/ingest-of-logs-slows-down-over-time/129382)
**Author:** [@Forsteri](https://discuss.elastic.co/u/Forsteri)\ **Replies:** 2\ **Last updated:** [May 4, 2018, 8:10pm UTC](https://discuss.elastic.co/t/ingest-of-logs-slows-down-over-time/129382 "2018-05-04T20:10:47Z")
First of all, a big "thanks" to all of the Logstash and ELK Stack developers. Your product is certainly improving my job and a mind-blowing improvement over the tool that we are transitioning away from. We get the logs… --- ## [Sleep filter](https://discuss.elastic.co/t/sleep-filter/130557)
**Author:** [@kabilan](https://discuss.elastic.co/u/kabilan)\ **Replies:** 1\ **Last updated:** [May 4, 2018, 7:59pm UTC](https://discuss.elastic.co/t/sleep-filter/130557 "2018-05-04T19:59:38Z")
hi all, was looking at the Sleep filter, what are the use cases for this? --- ## [Logstash grok is failing to parse greedydata inside conditional statement](https://discuss.elastic.co/t/logstash-grok-is-failing-to-parse-greedydata-inside-conditional-statement/130570)
**Author:** [@Sriram\_Kannan](https://discuss.elastic.co/u/Sriram_Kannan)\ **Replies:** 3\ **Last updated:** [May 4, 2018, 7:44pm UTC](https://discuss.elastic.co/t/logstash-grok-is-failing-to-parse-greedydata-inside-conditional-statement/130570 "2018-05-04T19:44:56Z")
I am having a logs pattern as below 2018-05-04 06:30:00.010 \[http-nio-6080-exec-6\] INFO .Controller - Processing request 16329d7d247 from 10.209.15.10: /uri/64511/5800/ 2018-05-04 06:30:00.007 \[http-nio-6080-exec-7\] IN… --- ## [Knowledge of successful logstash upload](https://discuss.elastic.co/t/knowledge-of-successful-logstash-upload/130149)
**Author:** [@squanchydata](https://discuss.elastic.co/u/squanchydata)\ **Replies:** 9\ **Last updated:** [May 4, 2018, 7:28pm UTC](https://discuss.elastic.co/t/knowledge-of-successful-logstash-upload/130149 "2018-05-04T19:28:08Z")
How does one know if a scheduled logstash upload went through to the index on ES? For example, lets say I have a conf file that is scheduled to upload data from a mysql database to an ElasticSearch index daily. If one d… --- ## [Netflow module ignore packets from virtual cisco](https://discuss.elastic.co/t/netflow-module-ignore-packets-from-virtual-cisco/130623)
**Author:** [@Sergey\_Murashov](https://discuss.elastic.co/u/Sergey_Murashov)\ **Replies:** 1\ **Last updated:** [May 4, 2018, 5:36pm UTC](https://discuss.elastic.co/t/netflow-module-ignore-packets-from-virtual-cisco/130623 "2018-05-04T17:36:28Z")
I use standart NetFlow Module. When I send netfow v9 from hardware Cisco 3850, information is displayed. From virtual cisco netflow v9 ignoring. I see packets in tcpdump, but module keep it. Please help. ELK 6.2.2 --- ## [How to handle multiple beats in Logstash](https://discuss.elastic.co/t/how-to-handle-multiple-beats-in-logstash/130469)
**Author:** [@charan.gandra](https://discuss.elastic.co/u/charan.gandra)\ **Replies:** 3\ **Last updated:** [May 4, 2018, 5:14pm UTC](https://discuss.elastic.co/t/how-to-handle-multiple-beats-in-logstash/130469 "2018-05-04T17:14:52Z")
Hello, I am running winlogbeat for windows event logs and filebeat on Linux for audit logs and syslogs. However I would like to index windows event logs, Linux Audit logs and sys logs separately in Elasticsearch. My lo… --- ## [Changing Logstash Configuration to solve OutofMemoryError](https://discuss.elastic.co/t/changing-logstash-configuration-to-solve-outofmemoryerror/130662)
**Author:** [@rpazhyannur](https://discuss.elastic.co/u/rpazhyannur)\ **Replies:** 1\ **Last updated:** [May 4, 2018, 5:06pm UTC](https://discuss.elastic.co/t/changing-logstash-configuration-to-solve-outofmemoryerror/130662 "2018-05-04T17:06:33Z")
Hello.. I get the following logstash error Specify -J-Xmx####m to increase it (#### = cap size in MB). Specify -w for full OutOfMemoryError stack trace" I am not sure which configuration file I should change. Curren… --- ## [Getting ruby exception](https://discuss.elastic.co/t/getting-ruby-exception/130636)
**Author:** [@Rahulkumar](https://discuss.elastic.co/u/Rahulkumar)\ **Replies:** 2\ **Last updated:** [May 4, 2018, 4:43pm UTC](https://discuss.elastic.co/t/getting-ruby-exception/130636 "2018-05-04T16:43:39Z")
Hi, I am getting error: SyntaxError: (ruby filter code):3: syntax error, unexpected tIDENTIFIER logInfo = event.get('wmcode').split('.').last I am using below filter: ruby {… --- ## [Not able see data in Kibana? Is Logstash pushing to ES](https://discuss.elastic.co/t/not-able-see-data-in-kibana-is-logstash-pushing-to-es/130534)
**Author:** [@kevivek](https://discuss.elastic.co/u/kevivek)\ **Replies:** 7\ **Last updated:** [May 4, 2018, 3:36pm UTC](https://discuss.elastic.co/t/not-able-see-data-in-kibana-is-logstash-pushing-to-es/130534 "2018-05-04T15:36:27Z")
I am new to ELK and working on the whole stack Below are my configs: vivsingh@blr-lvg2e:/etc/logstash/conf.d$ ls filter.conf input.conf output.conf vivsingh@blr-lvg2e:/etc/logstash/conf.d$ cat input.conf output.con… --- ## [Parse Old .gz Log File](https://discuss.elastic.co/t/parse-old-gz-log-file/129628)
**Author:** [@dee78](https://discuss.elastic.co/u/dee78)\ **Replies:** 2\ **Last updated:** [May 4, 2018, 3:32pm UTC](https://discuss.elastic.co/t/parse-old-gz-log-file/129628 "2018-05-04T15:32:58Z")
Hi everybody, hope that someone can help me to solve my issue. I've already read all the related topics on this forum but didn't find anything apt to me. So I use ELK to parse and process the access.log files of my Squ… --- ## [Strange Logstash Running Issue](https://discuss.elastic.co/t/strange-logstash-running-issue/130359)
**Author:** [@AloysiusParedes](https://discuss.elastic.co/u/AloysiusParedes)\ **Replies:** 2\ **Last updated:** [May 4, 2018, 2:43pm UTC](https://discuss.elastic.co/t/strange-logstash-running-issue/130359 "2018-05-04T14:43:12Z")
I have a strange issue on my ELK stack. I have ELK installed as a service that starts up as soon as my RHEL Virtual Machine (via Oracle VirtualBox) server starts. But for some reason, none of my documents are being inges… --- ## [Logstash won't authenticate with elasticsearch](https://discuss.elastic.co/t/logstash-wont-authenticate-with-elasticsearch/129807)
**Author:** [@jarnold](https://discuss.elastic.co/u/jarnold)\ **Replies:** 2\ **Last updated:** [May 4, 2018, 2:29pm UTC](https://discuss.elastic.co/t/logstash-wont-authenticate-with-elasticsearch/129807 "2018-05-04T14:29:54Z")
Hello, I'm setting up a new stack & i'm having some trouble with my logstash authentication. I'm getting this error after startup; Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.… --- ## [Netflow Reduced-size encoding](https://discuss.elastic.co/t/netflow-reduced-size-encoding/130499)
**Author:** [@dalamar666](https://discuss.elastic.co/u/dalamar666)\ **Replies:** 2\ **Last updated:** [May 4, 2018, 1:18pm UTC](https://discuss.elastic.co/t/netflow-reduced-size-encoding/130499 "2018-05-04T13:18:05Z")
Hi, Has anyone seen this warning before? Logstash 6.2 \[2018-05-03T12:57:47,142\]\[WARN \]\[logstash.codecs.netflow \] Reduced-size encoding for uint32 is larger than uint32 {:field=\>\[:uint32, :fwd\_flow\_delta\_bytes\], :leng… --- ## [Logstash service stops every 42 seconds and restarts](https://discuss.elastic.co/t/logstash-service-stops-every-42-seconds-and-restarts/130332)
**Author:** [@dalamar666](https://discuss.elastic.co/u/dalamar666)\ **Replies:** 5\ **Last updated:** [May 4, 2018, 1:17pm UTC](https://discuss.elastic.co/t/logstash-service-stops-every-42-seconds-and-restarts/130332 "2018-05-04T13:17:45Z")
I have all ownership in /etc/logstash/ set to logstash:logstash I have all ownership in /usr/share/logstash set to logstash:logstash The logstash service runs as logstash. When I run logstash from the /usr/share/logst… --- ## [Change filed type from string to number](https://discuss.elastic.co/t/change-filed-type-from-string-to-number/130627)
**Author:** [@Parthgandhi](https://discuss.elastic.co/u/Parthgandhi)\ **Replies:** 1\ **Last updated:** [May 4, 2018, 12:29pm UTC](https://discuss.elastic.co/t/change-filed-type-from-string-to-number/130627 "2018-05-04T12:29:48Z")
Hi, I have below GROK parse used for filtering IIS logs. It seems to be working fine except for certain fields(like SC-Bytes, CS-Bytes, etc) are coming as string and not numbers. How do i change this to number? match =\>… --- ## [Firebird 1.5 + SQL Server 2012](https://discuss.elastic.co/t/firebird-1-5-sql-server-2012/129857)
**Author:** [@TechG](https://discuss.elastic.co/u/TechG)\ **Replies:** 1\ **Last updated:** [May 4, 2018, 12:16pm UTC](https://discuss.elastic.co/t/firebird-1-5-sql-server-2012/129857 "2018-05-04T12:16:03Z")
Hi there. I need to connect 2 DB (1 Firebird 1.5 version) and another (SQL Server 2012). SQL with keep the whole information, and the FB only acess permition. I try to use the Linked server to make this, but doesn´t wo… --- ## [Best option for archiving logs](https://discuss.elastic.co/t/best-option-for-archiving-logs/130453)
**Author:** [@petehaley](https://discuss.elastic.co/u/petehaley)\ **Replies:** 4\ **Last updated:** [May 4, 2018, 12:09pm UTC](https://discuss.elastic.co/t/best-option-for-archiving-logs/130453 "2018-05-04T12:09:03Z")
Hello all, I am in the process of building a log analysis environment based on Elastic Stack in a classic setup: Beats/Syslog -\> Logstash -\> Elasticsearch However, there is a requirement for some of the logs to be sto… --- ## [No connection between Kafka and Logstash](https://discuss.elastic.co/t/no-connection-between-kafka-and-logstash/130260)
**Author:** [@TheAnonymousModeIT](https://discuss.elastic.co/u/TheAnonymousModeIT)\ **Replies:** 3\ **Last updated:** [May 4, 2018, 12:02pm UTC](https://discuss.elastic.co/t/no-connection-between-kafka-and-logstash/130260 "2018-05-04T12:02:22Z")
Hi, I had a working configuration: Logstash Standard Input -\> Output to Elasticsearch -\> Kibana I started Logstash and entered some text (like "test1") in the standard input. Logstash wrote it into Elasticsearch and … --- ## [When does @timestamp get added?](https://discuss.elastic.co/t/when-does-timestamp-get-added/129844)
**Author:** [@aidofitz](https://discuss.elastic.co/u/aidofitz)\ **Replies:** 1\ **Last updated:** [May 4, 2018, 11:52am UTC](https://discuss.elastic.co/t/when-does-timestamp-get-added/129844 "2018-05-04T11:52:22Z")
Hi, I've got a logstash pipeline taking events from a HTTP poller input to an ES output, configured with persistent queue. The incoming events do not contain an @timestamp field, and we don't add one at any point in the… --- ## [Field \_id is defined twice in \[doc\]](https://discuss.elastic.co/t/field-id-is-defined-twice-in-doc/130618)
**Author:** [@JensVanDeynse](https://discuss.elastic.co/u/JensVanDeynse)\ **Replies:** 0\ **Last updated:** [May 4, 2018, 11:49am UTC](https://discuss.elastic.co/t/field-id-is-defined-twice-in-doc/130618 "2018-05-04T11:49:38Z")
Hello there When setting up logstash and elasticsearch I ran into the following issues when indexing the Haproxy logs: \[2018-05-04T13:42:29,144\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticse… --- ## [Logstash Filter for Parsing Mule Log](https://discuss.elastic.co/t/logstash-filter-for-parsing-mule-log/129787)
**Author:** [@Dhanashree](https://discuss.elastic.co/u/Dhanashree)\ **Replies:** 1\ **Last updated:** [May 4, 2018, 11:36am UTC](https://discuss.elastic.co/t/logstash-filter-for-parsing-mule-log/129787 "2018-05-04T11:36:29Z")
Hi, I wish to load data from a mule log file to elasticsearch via logstash. The following is my log data pattern: 2018-04-26 14:35:51,016 \[\[cif\].HTTP\_Listener\_Configuration.worker.01\] INFO org.mule.api.processor.Logg… --- ## [Can't parse multiline logs with grok](https://discuss.elastic.co/t/cant-parse-multiline-logs-with-grok/130600)
**Author:** [@Petersiemens](https://discuss.elastic.co/u/Petersiemens)\ **Replies:** 3\ **Last updated:** [May 4, 2018, 11:22am UTC](https://discuss.elastic.co/t/cant-parse-multiline-logs-with-grok/130600 "2018-05-04T11:22:48Z")
Hi Team, i am currently trying to load Multiline-files with Logstash but for some reason i can't get more than the first line. I know that this topic has been discussed several times but all the solutions won't work in … --- ## [Can we use jdbc plug in for output in logstash?](https://discuss.elastic.co/t/can-we-use-jdbc-plug-in-for-output-in-logstash/130566)
**Author:** [@manjur](https://discuss.elastic.co/u/manjur)\ **Replies:** 1\ **Last updated:** [May 4, 2018, 10:50am UTC](https://discuss.elastic.co/t/can-we-use-jdbc-plug-in-for-output-in-logstash/130566 "2018-05-04T10:50:14Z")
Hi, I want to import data from csv file and put in to a table of sql server. Can i do that using logstash config file? --- ## [Configuring logstash-input-syslog](https://discuss.elastic.co/t/configuring-logstash-input-syslog/130502)
**Author:** [@static](https://discuss.elastic.co/u/static)\ **Replies:** 4\ **Last updated:** [May 4, 2018, 10:36am UTC](https://discuss.elastic.co/t/configuring-logstash-input-syslog/130502 "2018-05-04T10:36:36Z")
Hi guys! I absolutely love the ELK stack - please keep up the great work. Background: We have an Ubuntu 16.04 rsyslog server collecting syslogs from various servers and network equipment. I've been tasked with trying to… --- ## [JDBC input plugin with custom type?](https://discuss.elastic.co/t/jdbc-input-plugin-with-custom-type/130604)
**Author:** [@ansonlam812](https://discuss.elastic.co/u/ansonlam812)\ **Replies:** 0\ **Last updated:** [May 4, 2018, 10:09am UTC](https://discuss.elastic.co/t/jdbc-input-plugin-with-custom-type/130604 "2018-05-04T10:09:34Z")
The mapping type becomes single per index and introduced "custom type" to simulate the mapping type since Elasticsearch version 6.x, then can we create custom type by JDBC input plugin in Logstach? I have to input data … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=765) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=767)