# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=632 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 633 --- ## [PFSense Log Clarification](https://discuss.elastic.co/t/pfsense-log-clarification/159904)
**Author:** [@Booooooo](https://discuss.elastic.co/u/Booooooo)\ **Replies:** 5\ **Last updated:** [December 13, 2018, 1:34pm UTC](https://discuss.elastic.co/t/pfsense-log-clarification/159904 "2018-12-13T13:34:42Z")
Hi there, I am new to this so please bare with me as I need to take small steps to undertand this. I am trying to parse PFSense 2.4.4 logs into the Elastic Stack 6.5 which are sent via syslog. I am getting a bit confuse… --- ## [Mysql to Elasticsearch Sync with Logstash & input JDBC plugin](https://discuss.elastic.co/t/mysql-to-elasticsearch-sync-with-logstash-input-jdbc-plugin/159917)
**Author:** [@XMM](https://discuss.elastic.co/u/XMM)\ **Replies:** 4\ **Last updated:** [December 13, 2018, 12:55pm UTC](https://discuss.elastic.co/t/mysql-to-elasticsearch-sync-with-logstash-input-jdbc-plugin/159917 "2018-12-13T12:55:10Z")
Hello, i am trying to synchronize my MySql database to ElasticSearch with usage of Logstash and JDBC input plugin. As far I have working "dummy" prototype to sync entire Sql table row to single document. My challenge co… --- ## [Why \_grokparsefailure?](https://discuss.elastic.co/t/why-grokparsefailure/160113)
**Author:** [@trelo](https://discuss.elastic.co/u/trelo)\ **Replies:** 4\ **Last updated:** [December 13, 2018, 12:24pm UTC](https://discuss.elastic.co/t/why-grokparsefailure/160113 "2018-12-13T12:24:20Z")
Hi, here is my config: input { stdin {} file { codec =\> multiline { pattern =\> "^{%{DATESTAMP\_EVENTLOG}" what =\> "previous" negate =\> true multiline\_tag =\> "" } path =\> "/tmp/test… --- ## [Logstash-output-zabbix (3.0.5) Zabbix Version 4 ready?](https://discuss.elastic.co/t/logstash-output-zabbix-3-0-5-zabbix-version-4-ready/160704)
**Author:** [@irvined](https://discuss.elastic.co/u/irvined)\ **Replies:** 1\ **Last updated:** [December 13, 2018, 11:57am UTC](https://discuss.elastic.co/t/logstash-output-zabbix-3-0-5-zabbix-version-4-ready/160704 "2018-12-13T11:57:44Z")
Hi, Does 3.0.5 of the logstash-output-zabbix plugin work with V4 of Zabbix? I have the zabbix-sender working and correctly sends data to the zabbix server. When I send via logstash I get: \[2018-12-13T09:18:02,308\]\[WA… --- ## [Need help to understand one template](https://discuss.elastic.co/t/need-help-to-understand-one-template/160708)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 2\ **Last updated:** [December 13, 2018, 11:55am UTC](https://discuss.elastic.co/t/need-help-to-understand-one-template/160708 "2018-12-13T11:55:48Z")
i have this template for indexing data. It was written before. { "data\_tamplate": { "mappings": { "dynamic": "false", "properties": { "rollnumber": { "type": "text", "fields… --- ## [Continuous Data from SQL Server into Logstash](https://discuss.elastic.co/t/continuous-data-from-sql-server-into-logstash/159929)
**Author:** [@mubarak\_shaik](https://discuss.elastic.co/u/mubarak_shaik)\ **Replies:** 14\ **Last updated:** [December 13, 2018, 10:08am UTC](https://discuss.elastic.co/t/continuous-data-from-sql-server-into-logstash/159929 "2018-12-13T10:08:14Z")
I am using logstash to retrieve data from SQL into Elastic Search. I run query data on logstash and index is created and able to visualize data in Kibana. My concern is how to stream data always from SQL to logstash , … --- ## [Delimiter error when using Logstash File Input Plugin. Files not ingested fully](https://discuss.elastic.co/t/delimiter-error-when-using-logstash-file-input-plugin-files-not-ingested-fully/160683)
**Author:** [@dx123](https://discuss.elastic.co/u/dx123)\ **Replies:** 0\ **Last updated:** [December 13, 2018, 7:52am UTC](https://discuss.elastic.co/t/delimiter-error-when-using-logstash-file-input-plugin-files-not-ingested-fully/160683 "2018-12-13T07:52:08Z")
I'm using the Logstash File Input Plugin to ingest a a file but I am receiving this error: \[\[main\]\ **Author:** [@David\_Moreno](https://discuss.elastic.co/u/David_Moreno)\ **Replies:** 3\ **Last updated:** [December 13, 2018, 6:54am UTC](https://discuss.elastic.co/t/syslog-filter-configuration-in-logstash/159816 "2018-12-13T06:54:52Z") Hello, I'm trying to format my raw syslog data into elk using logstash but getting in error. Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationErr… --- ## [Logstash apt package version number differs from ES, Kibana](https://discuss.elastic.co/t/logstash-apt-package-version-number-differs-from-es-kibana/160673)
**Author:** [@peksa](https://discuss.elastic.co/u/peksa)\ **Replies:** 0\ **Last updated:** [December 13, 2018, 6:47am UTC](https://discuss.elastic.co/t/logstash-apt-package-version-number-differs-from-es-kibana/160673 "2018-12-13T06:47:34Z")
Hi I need to define apt package versions of Elasticsearch, Kibana and Logstash for installation scripts. What I noticed is that while for Elasticsearch and Kibana the latest apt version is 6.5.3 (as in "apt-get install … --- ## [Logstash loops](https://discuss.elastic.co/t/logstash-loops/158750)
**Author:** [@dinesh1](https://discuss.elastic.co/u/dinesh1)\ **Replies:** 13\ **Last updated:** [December 13, 2018, 6:21am UTC](https://discuss.elastic.co/t/logstash-loops/158750 "2018-12-13T06:21:29Z")
Hi, how to use loops for output in logstash. output { if \[my\_index\] == "order"{ elasticsearch { hosts =\> \[ "172.160.0.234" \] index =\> "order-%{+… --- ## [About logstash date time zone questions](https://discuss.elastic.co/t/about-logstash-date-time-zone-questions/159631)
**Author:** [@yuxiao](https://discuss.elastic.co/u/yuxiao)\ **Replies:** 13\ **Last updated:** [December 13, 2018, 4:53am UTC](https://discuss.elastic.co/t/about-logstash-date-time-zone-questions/159631 "2018-12-13T04:53:41Z")
I use logstash-input-jdbc sync data from mysql to elasticsearch, in mysql have time field, the type is timestamp,locale time zone is +8:00, but sync to elasticsearch,the time value less 8 hours, who knows the answer,plea… --- ## [Remove](https://discuss.elastic.co/t/remove/160637)
**Author:** [@ciphee](https://discuss.elastic.co/u/ciphee)\ **Replies:** 3\ **Last updated:** [December 13, 2018, 4:47am UTC](https://discuss.elastic.co/t/remove/160637 "2018-12-13T04:47:35Z")
removed--figured it out. --- ## [Elapsed\_expired\_error event is not generated for same repeated logs](https://discuss.elastic.co/t/elapsed-expired-error-event-is-not-generated-for-same-repeated-logs/160316)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 8:00am UTC](https://discuss.elastic.co/t/elapsed-expired-error-event-is-not-generated-for-same-repeated-logs/160316 "2018-12-11T08:00:28Z")
hi there i use logstash elapsed filter to get latency between req and resp logs logstash elapsed help me a lot. but for repeated req (same unique\_id\_field) logs it doesn't show correct count. here i test elapsed confi… --- ## [Investigating why all logs aren't visible in Kibana](https://discuss.elastic.co/t/investigating-why-all-logs-arent-visible-in-kibana/160004)
**Author:** [@duncOps](https://discuss.elastic.co/u/duncOps)\ **Replies:** 0\ **Last updated:** [December 8, 2018, 11:21am UTC](https://discuss.elastic.co/t/investigating-why-all-logs-arent-visible-in-kibana/160004 "2018-12-08T11:21:22Z")
Hi all, I've put "Kibana" as the category for the post, but realistically it could be any part of the Elastic stack that I've misconfigured. Elastic stack structure Azure-hosted VMs running the following; part of the … --- ## [High Number of TCP Resets from Logstash Beats Plugin](https://discuss.elastic.co/t/high-number-of-tcp-resets-from-logstash-beats-plugin/160632)
**Author:** [@sparky005](https://discuss.elastic.co/u/sparky005)\ **Replies:** 0\ **Last updated:** [December 12, 2018, 10:18pm UTC](https://discuss.elastic.co/t/high-number-of-tcp-resets-from-logstash-beats-plugin/160632 "2018-12-12T22:18:51Z")
We have a couple ec2 instances running Logstash configured behind an AWS Network Load Balancer. The Network load balancer periodically connects to each logstash listener in order to verify that it is up and running. Afte… --- ## [Snmptrap input not decoding MAC addresses correctly](https://discuss.elastic.co/t/snmptrap-input-not-decoding-mac-addresses-correctly/160629)
**Author:** [@mj563807](https://discuss.elastic.co/u/mj563807)\ **Replies:** 0\ **Last updated:** [December 12, 2018, 10:14pm UTC](https://discuss.elastic.co/t/snmptrap-input-not-decoding-mac-addresses-correctly/160629 "2018-12-12T22:14:35Z")
I've started using the snmptrap input but I've noticed that for some fields containing a mac address, the value is not decoded correctly and is just some garbled up characters. For other fields containing mac addresses,… --- ## [Logstash failed to execute action :exception=\>"Java::JavaLang::IllegalArgumentException", :message=\>"Illegal pattern component: O"](https://discuss.elastic.co/t/logstash-failed-to-execute-action-exception-java-illegalargumentexception-message-illegal-pattern-component-o/160616)
**Author:** [@Max\_Shen](https://discuss.elastic.co/u/Max_Shen)\ **Replies:** 1\ **Last updated:** [December 12, 2018, 9:46pm UTC](https://discuss.elastic.co/t/logstash-failed-to-execute-action-exception-java-illegalargumentexception-message-illegal-pattern-component-o/160616 "2018-12-12T21:46:34Z")
seems to be related to the date conversion, but couldn't figure out why it complains about token 0 My Config # Sample Logstash configuration for creating a simple # Beats -\> Logstash -\> Elasticsearch pipeline. inpu… --- ## [Validation Failed: 1: no monitoring documents added](https://discuss.elastic.co/t/validation-failed-1-no-monitoring-documents-added/160621)
**Author:** [@kb2295](https://discuss.elastic.co/u/kb2295)\ **Replies:** 0\ **Last updated:** [December 12, 2018, 9:06pm UTC](https://discuss.elastic.co/t/validation-failed-1-no-monitoring-documents-added/160621 "2018-12-12T21:06:16Z")
Every time I try to start a new logstash instance with this config file, I get the following error. I'm not sure what it means nor do I know how to go about fixing it. \[2018-12-12T15:38:26,252\]\[ERROR\]\[logstash.outputs.… --- ## [Help with logstash csv and beats conf file](https://discuss.elastic.co/t/help-with-logstash-csv-and-beats-conf-file/160035)
**Author:** [@lowdmt](https://discuss.elastic.co/u/lowdmt)\ **Replies:** 1\ **Last updated:** [December 12, 2018, 8:11pm UTC](https://discuss.elastic.co/t/help-with-logstash-csv-and-beats-conf-file/160035 "2018-12-12T20:11:16Z")
Hi, I have an issue with my config file - any help appreciated \`input { stdin { type =\> "stdin-type" } file { path =\> "/etc/logstash/data/test.csv" start\_position =\> beginning # to read from the beginning of fi… --- ## [How to compare to log line and add additional fields](https://discuss.elastic.co/t/how-to-compare-to-log-line-and-add-additional-fields/160285)
**Author:** [@Sanjeev7](https://discuss.elastic.co/u/Sanjeev7)\ **Replies:** 2\ **Last updated:** [December 12, 2018, 6:10pm UTC](https://discuss.elastic.co/t/how-to-compare-to-log-line-and-add-additional-fields/160285 "2018-12-12T18:10:22Z")
Hello Sample Log - 2018-12-09 22:25:11,959162184: INFO : \[serviceAd : 1\] service: transaction started, transactionnum : 12345 2018-12-09 22:26:14,330574541: INFO : \[serviceAd : 1\] service: transaction ended, transac… --- ## [AMQP 1.0 support for logstash](https://discuss.elastic.co/t/amqp-1-0-support-for-logstash/160596)
**Author:** [@sawandas](https://discuss.elastic.co/u/sawandas)\ **Replies:** 0\ **Last updated:** [December 12, 2018, 5:59pm UTC](https://discuss.elastic.co/t/amqp-1-0-support-for-logstash/160596 "2018-12-12T17:59:42Z")
Hi, I am new to logstash. I want to receive inputs from ActiveMQ and store it to elasticsearch. Now my ActiveMQ broker is configured with AMQP messages. RabbitMQ plugin is only supporting AMQP version 0.9.0. I have trie… --- ## [JDBC plugin: Doesn't work when the table name contains a hyphen](https://discuss.elastic.co/t/jdbc-plugin-doesnt-work-when-the-table-name-contains-a-hyphen/160278)
**Author:** [@prodrg](https://discuss.elastic.co/u/prodrg)\ **Replies:** 5\ **Last updated:** [December 12, 2018, 4:57pm UTC](https://discuss.elastic.co/t/jdbc-plugin-doesnt-work-when-the-table-name-contains-a-hyphen/160278 "2018-12-12T16:57:37Z")
Hi guys, I am using the JDBC plugin and I need to extract info from a Postgresql database. The issue I'm having is that the table I want to grab info has a hyphen on the name, and this is causing a conflict. This is my… --- ## [2 types of input](https://discuss.elastic.co/t/2-types-of-input/160402)
**Author:** [@fooc](https://discuss.elastic.co/u/fooc)\ **Replies:** 3\ **Last updated:** [December 12, 2018, 4:56pm UTC](https://discuss.elastic.co/t/2-types-of-input/160402 "2018-12-12T16:56:03Z")
Hi, I'm using a logstash-utm.conf file to process syslog data from a firewall, this is working fine. I also want to send data from winlogbeat on a windows server to logstash, is it best practice to add new input and ou… --- ## [Price concept](https://discuss.elastic.co/t/price-concept/160580)
**Author:** [@ivan.melnik](https://discuss.elastic.co/u/ivan.melnik)\ **Replies:** 0\ **Last updated:** [December 12, 2018, 4:06pm UTC](https://discuss.elastic.co/t/price-concept/160580 "2018-12-12T16:06:00Z")
Hi, maybe someones can help me to understand the Elastic Price concept. I'm thinking about to integrate Lostash in my company. So if I want to host server with Loghash by Elastic I need to pay for that, but if I gonna… --- ## [Can't fix \_grokparsefailure](https://discuss.elastic.co/t/cant-fix-grokparsefailure/160523)
**Author:** [@thiesens](https://discuss.elastic.co/u/thiesens)\ **Replies:** 5\ **Last updated:** [December 12, 2018, 12:39pm UTC](https://discuss.elastic.co/t/cant-fix-grokparsefailure/160523 "2018-12-12T12:39:49Z")
Hi I can't seem to get my grok filter to work. I want to use filebeat to parse a log file, send it to Logstash, parse it and send it to Elasticsearch. My filebeat.yml: filebeat: prospectors: - type: log paths: - … --- ## [Logstash and Jboss](https://discuss.elastic.co/t/logstash-and-jboss/51507)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 15\ **Last updated:** [December 12, 2018, 12:16pm UTC](https://discuss.elastic.co/t/logstash-and-jboss/51507 "2018-12-12T12:16:32Z")
I have newly setup a JBOSS JVM server and would like to use ELK to visualize the Server Host Usage Metrics. Though I can see there are server logs but seem are not showing the Host Usage Metrics, i.g. cpu usage, disk usa… --- ## [Parsing data from multiple lines](https://discuss.elastic.co/t/parsing-data-from-multiple-lines/160514)
**Author:** [@shivabharat](https://discuss.elastic.co/u/shivabharat)\ **Replies:** 4\ **Last updated:** [December 12, 2018, 12:01pm UTC](https://discuss.elastic.co/t/parsing-data-from-multiple-lines/160514 "2018-12-12T12:01:27Z")
Hi All, I have a log file which has a big chunk of log like below. Using the GREEDYDATA pattern I am able to parse only till 00000000-00000000 6260 0C100005 CnHttpRequest.cpp:5268 Sending Header I am looking to grab… --- ## [Logstash keystore password access issue](https://discuss.elastic.co/t/logstash-keystore-password-access-issue/159168)
**Author:** [@sachintanpure85](https://discuss.elastic.co/u/sachintanpure85)\ **Replies:** 2\ **Last updated:** [December 12, 2018, 11:15am UTC](https://discuss.elastic.co/t/logstash-keystore-password-access-issue/159168 "2018-12-12T11:15:06Z")
Hi , I was using below code to check the Logstash/elasticsearch node status ElasticResponse() { wget -S --no-check-certificate --spider --user {USER} --password {PASSWORD} https://${LOGSTASH\_HOST} 2\>&1 | tee -a $LO… --- ## [Not a valid Logstash keystore error](https://discuss.elastic.co/t/not-a-valid-logstash-keystore-error/159945)
**Author:** [@sachintanpure85](https://discuss.elastic.co/u/sachintanpure85)\ **Replies:** 2\ **Last updated:** [December 12, 2018, 11:32am UTC](https://discuss.elastic.co/t/not-a-valid-logstash-keystore-error/159945 "2018-12-12T11:32:34Z")
Hi, I am using ELK 6.3.2 version and I have used below configuration steps for logstash keystore set +o history export LOGSTASH\_KEYSTORE\_PASS=mypassword set -o history logstash-keystore create logstash-keystore add… --- ## [Grok Patterns - Quick question](https://discuss.elastic.co/t/grok-patterns-quick-question/160520)
**Author:** [@Carlos\_Magalhaes](https://discuss.elastic.co/u/Carlos_Magalhaes)\ **Replies:** 0\ **Last updated:** [December 12, 2018, 10:45am UTC](https://discuss.elastic.co/t/grok-patterns-quick-question/160520 "2018-12-12T10:45:20Z")
Hi all, I think this is a quick clarifying question. With a Grok pattern: Does the entire string that has been filtered need to be matched with your grok pattern or does the grok pattern pull out what it needs and dis… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=631) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=633)