# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=997 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 998 --- ## [Azure Logstash plugin](https://discuss.elastic.co/t/azure-logstash-plugin/82869)
**Author:** [@mackah666](https://discuss.elastic.co/u/mackah666)\ **Replies:** 2\ **Last updated:** [April 20, 2017, 1:06pm UTC](https://discuss.elastic.co/t/azure-logstash-plugin/82869 "2017-04-20T13:06:14Z")
I am using Umbraco Azure Appender to store log4net entries in an azure table. I would like to Use Azure Logstash plugin to push these entries into elasticsearch. Currently, logs entries appear in azure table, but logstas… --- ## [\[CLOSED / USER ERROR\] Filebeat -\> Logstash multi-line](https://discuss.elastic.co/t/closed-user-error-filebeat-logstash-multi-line/82941)
**Author:** [@gtorrance](https://discuss.elastic.co/u/gtorrance)\ **Replies:** 3\ **Last updated:** [April 20, 2017, 12:58pm UTC](https://discuss.elastic.co/t/closed-user-error-filebeat-logstash-multi-line/82941 "2017-04-20T12:58:40Z")
Hi all, I've searched high and low to get an answer to this, but no success so far. Hope someone can help. I'm trying to get multi-line processing of Java Exceptions to work correctly. My Filebeat configuration conta… --- ## [Logstash-IBM DB2-Reporting](https://discuss.elastic.co/t/logstash-ibm-db2-reporting/82961)
**Author:** [@senthilyes](https://discuss.elastic.co/u/senthilyes)\ **Replies:** 5\ **Last updated:** [April 20, 2017, 12:05pm UTC](https://discuss.elastic.co/t/logstash-ibm-db2-reporting/82961 "2017-04-20T12:05:24Z")
Hi We got a usecase for message queue reporting requirements. Out application systems use IBM MQ for messaging services and data is stored in IBM DB2 database. We need to generate reports out of DB2 database eg. 4. We… --- ## [Logstash plugin installation](https://discuss.elastic.co/t/logstash-plugin-installation/82896)
**Author:** [@Nikparab](https://discuss.elastic.co/u/Nikparab)\ **Replies:** 3\ **Last updated:** [April 20, 2017, 12:00pm UTC](https://discuss.elastic.co/t/logstash-plugin-installation/82896 "2017-04-20T12:00:12Z")
Hi I am trying to install logstash-plugins in centos 7 by using following command /bin\>./logstash-plugin install logstash-input-mongodb But it gives error as following Could not find any executable java binary. Pl… --- ## [Kibana: Creating Selected Fields](https://discuss.elastic.co/t/kibana-creating-selected-fields/81311)
**Author:** [@fabiansc](https://discuss.elastic.co/u/fabiansc)\ **Replies:** 7\ **Last updated:** [April 18, 2017, 8:43am UTC](https://discuss.elastic.co/t/kibana-creating-selected-fields/81311 "2017-04-18T08:43:48Z")
Hi Everyone, I just found out about ELK + Filebeat and setup an own stack. First I player around with default systemlog files, which was working fine. Afterwards I started pushing my own application log file to it to f… --- ## [How to extract xml data?](https://discuss.elastic.co/t/how-to-extract-xml-data/82557)
**Author:** [@laptop\_data](https://discuss.elastic.co/u/laptop_data)\ **Replies:** 7\ **Last updated:** [April 20, 2017, 11:06am UTC](https://discuss.elastic.co/t/how-to-extract-xml-data/82557 "2017-04-20T11:06:41Z")
Hi, I have XML data shown below, how to extract it. Using XML filter. \ \ \ \ **Author:** [@shravan\_kumar](https://discuss.elastic.co/u/shravan_kumar)\ **Replies:** 4\ **Last updated:** [April 20, 2017, 10:58am UTC](https://discuss.elastic.co/t/logstash-pipeline-is-not-strating-when-i-run-it-for-second-time/82745 "2017-04-20T10:58:02Z") Hi, When I run logstash config file, I can see the index created in elastic search. But when I run it again with few changes( like changing index name) I can see logstash pipeline has started but the index is not create… --- ## [Is there a way to generate an Increment and Unique ID in logstash?](https://discuss.elastic.co/t/is-there-a-way-to-generate-an-increment-and-unique-id-in-logstash/83008)
**Author:** [@elastic\_johnson\_yi](https://discuss.elastic.co/u/elastic_johnson_yi)\ **Replies:** 2\ **Last updated:** [April 20, 2017, 8:58am UTC](https://discuss.elastic.co/t/is-there-a-way-to-generate-an-increment-and-unique-id-in-logstash/83008 "2017-04-20T08:58:36Z")
What I really want is a unique ID, but not UUID. Because UUID is too long, and I don't need the ID to be universally unique. Just being unique in my own server will be fine enough. The UUID plugin is quite close to wha… --- ## [Want to run multiple logstash instances in single machine](https://discuss.elastic.co/t/want-to-run-multiple-logstash-instances-in-single-machine/82805)
**Author:** [@Raja1](https://discuss.elastic.co/u/Raja1)\ **Replies:** 3\ **Last updated:** [April 20, 2017, 8:51am UTC](https://discuss.elastic.co/t/want-to-run-multiple-logstash-instances-in-single-machine/82805 "2017-04-20T08:51:38Z")
Hi, I am able to integrate kafka with logstash and then able to see the logs in kafka.Now i want to parse the logs from kafka to elasticsearch thru logstash by using same machine. Can someone please guide me how to u… --- ## [\[Solved\] Output kafka netflow codec](https://discuss.elastic.co/t/solved-output-kafka-netflow-codec/83005)
**Author:** [@sangrho](https://discuss.elastic.co/u/sangrho)\ **Replies:** 0\ **Last updated:** [April 20, 2017, 8:07am UTC](https://discuss.elastic.co/t/solved-output-kafka-netflow-codec/83005 "2017-04-20T08:07:22Z")
Hello. I have been tried integrate logstash and kafka with netflow data. Before this try, I did integrate logstash, nifi and kafka like below, input{ file{ path =\>"\*.pcap" codec =\> netflow { versions =\> \[5\]}… --- ## [Delete fields in events](https://discuss.elastic.co/t/delete-fields-in-events/82888)
**Author:** [@jagadeeswar\_Reddy](https://discuss.elastic.co/u/jagadeeswar_Reddy)\ **Replies:** 2\ **Last updated:** [April 20, 2017, 7:59am UTC](https://discuss.elastic.co/t/delete-fields-in-events/82888 "2017-04-20T07:59:01Z")
Hello All, could you please help us to get the desire out put my logs usign the logstash groke filter My Logformat id :"Apr 17 15:20:01 wiki-jira CROND\[8769\]: (root) CMD (/usr/lib64/sa/sa1 1 1)" I want get the out … --- ## [WARN No appenders could be found for logger (org.apache.kafka.clients.producer.ProducerConfig)](https://discuss.elastic.co/t/warn-no-appenders-could-be-found-for-logger-org-apache-kafka-clients-producer-producerconfig/82515)
**Author:** [@rahiminasab](https://discuss.elastic.co/u/rahiminasab)\ **Replies:** 2\ **Last updated:** [April 20, 2017, 7:26am UTC](https://discuss.elastic.co/t/warn-no-appenders-could-be-found-for-logger-org-apache-kafka-clients-producer-producerconfig/82515 "2017-04-20T07:26:18Z")
Hi, I am using Logstash 5.3.0. When I use Kafka output plugin in my config file as follows: output { kafka { codec =\> plain { format =\> "%{message}" … --- ## [Error when using separate config files](https://discuss.elastic.co/t/error-when-using-separate-config-files/82874)
**Author:** [@dreadlocked1](https://discuss.elastic.co/u/dreadlocked1)\ **Replies:** 1\ **Last updated:** [April 20, 2017, 5:16am UTC](https://discuss.elastic.co/t/error-when-using-separate-config-files/82874 "2017-04-20T05:16:01Z")
Hi, I'm using the following command line to execute logstash with multiple config files: $ /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/{1\_inputs.conf,2\_filters.conf,3\_outputs.conf} --path.settings /etc/logs… --- ## [Statistical problem](https://discuss.elastic.co/t/statistical-problem/82670)
**Author:** [@Sam67000](https://discuss.elastic.co/u/Sam67000)\ **Replies:** 5\ **Last updated:** [April 20, 2017, 5:02am UTC](https://discuss.elastic.co/t/statistical-problem/82670 "2017-04-20T05:02:09Z")
Hello EveryOne, --- ## [Date parsing issue for getting timestamp](https://discuss.elastic.co/t/date-parsing-issue-for-getting-timestamp/82816)
**Author:** [@san\_maj](https://discuss.elastic.co/u/san_maj)\ **Replies:** 3\ **Last updated:** [April 20, 2017, 5:01am UTC](https://discuss.elastic.co/t/date-parsing-issue-for-getting-timestamp/82816 "2017-04-20T05:01:02Z")
I am aware that there have been many questions asked around this and this probably could be a repeat of sorts. But have to request someone guiding me on this. My log file looks like this - \ **Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 5\ **Last updated:** [April 19, 2017, 7:51pm UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728 "2017-04-19T19:51:00Z") Hi, i'm using ES, LS, Kibana 5.3, beat 5.4 and Kafka 0.10.2 I used winlogbeat to send event log to ES Beat -\> Logstash collector -\> Kafka -\> Logstash Processor -\> ElasticSearch This is my config in logstash collec… --- ## [Ssl connection between Filebeat and Logstash](https://discuss.elastic.co/t/ssl-connection-between-filebeat-and-logstash/82928)
**Author:** [@ls4074](https://discuss.elastic.co/u/ls4074)\ **Replies:** 0\ **Last updated:** [April 19, 2017, 5:14pm UTC](https://discuss.elastic.co/t/ssl-connection-between-filebeat-and-logstash/82928 "2017-04-19T17:14:33Z")
Hello, When I am setting up the SSL connection between FIlebeat and Logstash, I got the following error: 2017-04-18T16:44:07,811\]\[ERROR\]\[logstash.inputs.beats \] Looks like you either have an invalid key or your pr… --- ## [Index attachements files of Email using ELK](https://discuss.elastic.co/t/index-attachements-files-of-email-using-elk/82854)
**Author:** [@amine](https://discuss.elastic.co/u/amine)\ **Replies:** 0\ **Last updated:** [April 19, 2017, 9:32am UTC](https://discuss.elastic.co/t/index-attachements-files-of-email-using-elk/82854 "2017-04-19T09:32:29Z")
Hi everyone; I try to index attachement files of Email using logstash and Imap plugin of logstash, but i can get attachement files of emails. imap { host =\> "mydom.mydomain.com" user =\> "test-search@… --- ## [I think my logstash server's nic is over saturated?](https://discuss.elastic.co/t/i-think-my-logstash-servers-nic-is-over-saturated/82910)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 0\ **Last updated:** [April 19, 2017, 2:53pm UTC](https://discuss.elastic.co/t/i-think-my-logstash-servers-nic-is-over-saturated/82910 "2017-04-19T14:53:45Z")
My ELK pipeline is: nxlog on Win servers (about 35-40) \> one logstash server \> ES When I look at the events in ES they are hours old, but I do eventually get them- its just always a few hours behind. No errors or dropp… --- ## [Logstash kafka input retry connection?](https://discuss.elastic.co/t/logstash-kafka-input-retry-connection/82915)
**Author:** [@Maxwell\_Flanders](https://discuss.elastic.co/u/Maxwell_Flanders)\ **Replies:** 0\ **Last updated:** [April 19, 2017, 3:55pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-retry-connection/82915 "2017-04-19T15:55:35Z")
does anyone know if there is retry logic on the logstash-kafka input tool?? We are facing a situation where we may have a logstash instance which has a logstash-kafka input configuration that refers to a kafka cluster th… --- ## [Logstash pipeline crashed, chingCallSite error](https://discuss.elastic.co/t/logstash-pipeline-crashed-chingcallsite-error/82848)
**Author:** [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Replies:** 0\ **Last updated:** [April 19, 2017, 9:09am UTC](https://discuss.elastic.co/t/logstash-pipeline-crashed-chingcallsite-error/82848 "2017-04-19T09:09:01Z")
Hi all, I was getting an error sometimes and i was not worried about it, my pipeline restarted automatically so everything was just fine. But now the error has started to appear more often and the pipeline seems to work… --- ## [Logstash not creating index on elasticsearch](https://discuss.elastic.co/t/logstash-not-creating-index-on-elasticsearch/82146)
**Author:** [@dbElastic](https://discuss.elastic.co/u/dbElastic)\ **Replies:** 4\ **Last updated:** [April 19, 2017, 12:30pm UTC](https://discuss.elastic.co/t/logstash-not-creating-index-on-elasticsearch/82146 "2017-04-19T12:30:42Z")
Hello, I have ElasticSearch and Logstash installed on same machine(Ubuntu). I have the below apache\_logs.conf file contents: input { file { path =\> "/var/log/apache\_logs" type =\> "apache\_log" # a type to id… --- ## [Logstash input http plugin response back](https://discuss.elastic.co/t/logstash-input-http-plugin-response-back/82572)
**Author:** [@Arslan\_Khan](https://discuss.elastic.co/u/Arslan_Khan)\ **Replies:** 4\ **Last updated:** [April 19, 2017, 11:13am UTC](https://discuss.elastic.co/t/logstash-input-http-plugin-response-back/82572 "2017-04-19T11:13:16Z")
Hi all, I am working on logstash to use input http plugin to receive data. I am using CURL command to send data to it and every time it respond in 'ok' but there are errors in parsing data in logstash. My question is t… --- ## [File output plugin of logstash to capture the drop events](https://discuss.elastic.co/t/file-output-plugin-of-logstash-to-capture-the-drop-events/82833)
**Author:** [@santhosh244](https://discuss.elastic.co/u/santhosh244)\ **Replies:** 1\ **Last updated:** [April 19, 2017, 10:47am UTC](https://discuss.elastic.co/t/file-output-plugin-of-logstash-to-capture-the-drop-events/82833 "2017-04-19T10:47:35Z")
Below is the scenario: We are dropping events in filter section if log\_level are DEBUG and that is getting properly dropped and not sent to ES. But we would like to capture those events in a flat file using file outp… --- ## [Failed to parse \[timestamp\]](https://discuss.elastic.co/t/failed-to-parse-timestamp/81966)
**Author:** [@mirnuj\_atom](https://discuss.elastic.co/u/mirnuj_atom)\ **Replies:** 8\ **Last updated:** [April 19, 2017, 8:18am UTC](https://discuss.elastic.co/t/failed-to-parse-timestamp/81966 "2017-04-19T08:18:02Z")
Hey Guys Hope you all are doing great! I am having an issue parsing date in tomcat logs. Elasticsearch throws errors in debug and stops accepting new messages. Here is an example of log lines: 2017-02-27 13:53:14 IN… --- ## [Does logstash work on windows server 2012 R2](https://discuss.elastic.co/t/does-logstash-work-on-windows-server-2012-r2/82834)
**Author:** [@nitinlko](https://discuss.elastic.co/u/nitinlko)\ **Replies:** 0\ **Last updated:** [April 19, 2017, 7:42am UTC](https://discuss.elastic.co/t/does-logstash-work-on-windows-server-2012-r2/82834 "2017-04-19T07:42:45Z")
I am using logstash in my laptop which is having windows 10. it is working fine. same logstash folder i copied to a windows server 2012 R2 machine and then when i try to start logstash it gives below error. i have instal… --- ## [Problems installing logstash-filter-translate plugin](https://discuss.elastic.co/t/problems-installing-logstash-filter-translate-plugin/82187)
**Author:** [@rabbit](https://discuss.elastic.co/u/rabbit)\ **Replies:** 1\ **Last updated:** [April 19, 2017, 7:38am UTC](https://discuss.elastic.co/t/problems-installing-logstash-filter-translate-plugin/82187 "2017-04-19T07:38:36Z")
Guys, I have the following error while installing the logstash-filter-translate plugin. Validating logstash-filter-translate Installing logstash-filter-translate Error Bundler::InstallError, retrying 1/10 An error … --- ## [Changing file and directory permission for /usr/share/logstash?](https://discuss.elastic.co/t/changing-file-and-directory-permission-for-usr-share-logstash/82819)
**Author:** [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\ **Replies:** 0\ **Last updated:** [April 19, 2017, 5:24am UTC](https://discuss.elastic.co/t/changing-file-and-directory-permission-for-usr-share-logstash/82819 "2017-04-19T05:24:34Z")
Hello. Logstash version : 5.3.0 I am trying to startup logstash with customer user via systemd on RHEL 7. I am doing this due to customer's security compliance. I was able to startup logstash by fixing below env… --- ## [Copying Files to Central Remote Server Using Beats/LogStash](https://discuss.elastic.co/t/copying-files-to-central-remote-server-using-beats-logstash/82746)
**Author:** [@arpitwanchoo](https://discuss.elastic.co/u/arpitwanchoo)\ **Replies:** 3\ **Last updated:** [April 19, 2017, 5:33am UTC](https://discuss.elastic.co/t/copying-files-to-central-remote-server-using-beats-logstash/82746 "2017-04-19T05:33:40Z")
Hi I am trying to design the centralized logging structure for our org. I was looking for a following pipeline : RemoteServers(FileBeat) =\> Central Server (LogStash) =\> ElasticSearch I want FileBeat to copy files … --- ## [Help installing unpublished plugin logstash-output-s3 on installed logstash 5.3.0](https://discuss.elastic.co/t/help-installing-unpublished-plugin-logstash-output-s3-on-installed-logstash-5-3-0/82651)
**Author:** [@averi](https://discuss.elastic.co/u/averi)\ **Replies:** 1\ **Last updated:** [April 19, 2017, 3:38am UTC](https://discuss.elastic.co/t/help-installing-unpublished-plugin-logstash-output-s3-on-installed-logstash-5-3-0/82651 "2017-04-19T03:38:44Z")
Hi All, First up thank you for the help. This is my first post on here. I am testing the logstash-output-s3 plugin branch commit id: bc3035b077be465f172377dc9be55d198e037462. The commit allows the plugin to be used with … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=996) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=998)