# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=985 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 986 --- ## [Ignore Text and carriage return characters](https://discuss.elastic.co/t/ignore-text-and-carriage-return-characters/85278)
**Author:** [@harshavardan25000](https://discuss.elastic.co/u/harshavardan25000)\ **Replies:** 5\ **Last updated:** [May 11, 2017, 4:31pm UTC](https://discuss.elastic.co/t/ignore-text-and-carriage-return-characters/85278 "2017-05-11T16:31:04Z")
Hello, How can I manage log lines which have multiple /n and /r characters with Grok? The log line here is a multiline and looking to replace the /r /n with "". My filter looks like this : filter{ grok {match =\> \["mes… --- ## [Logstash Error: Beats input: the pipeline is blocked, temporary refusing new connection](https://discuss.elastic.co/t/logstash-error-beats-input-the-pipeline-is-blocked-temporary-refusing-new-connection/52929)
**Author:** [@Mike\_Wurtz](https://discuss.elastic.co/u/Mike_Wurtz)\ **Replies:** 2\ **Last updated:** [May 11, 2017, 2:37pm UTC](https://discuss.elastic.co/t/logstash-error-beats-input-the-pipeline-is-blocked-temporary-refusing-new-connection/52929 "2017-05-11T14:37:48Z")
After pointing 8 or so filebeat agents to my logstash server, I'm getting this message in the logstash log: Beats input: the pipeline is blocked, temporary refusing new connection. What is the cause of this error? A… --- ## [How to parse different time formats](https://discuss.elastic.co/t/how-to-parse-different-time-formats/85265)
**Author:** [@thorsten1](https://discuss.elastic.co/u/thorsten1)\ **Replies:** 4\ **Last updated:** [May 11, 2017, 1:43pm UTC](https://discuss.elastic.co/t/how-to-parse-different-time-formats/85265 "2017-05-11T13:43:38Z")
Hello, I use a file input with a multiline codec and a pattern that says if the line starts with a date then it is a new line. input { file { ... codec =\> multiline { pattern =\> "^%{MONTH}%{SPACE}%{M… --- ## [Office365 input for logstash](https://discuss.elastic.co/t/office365-input-for-logstash/85409)
**Author:** [@mourlos](https://discuss.elastic.co/u/mourlos)\ **Replies:** 0\ **Last updated:** [May 11, 2017, 1:35pm UTC](https://discuss.elastic.co/t/office365-input-for-logstash/85409 "2017-05-11T13:35:01Z")
Hello, Just wanted to ask if anyone has come across/created an input plugin in order to fetch logs/data from office365. Microsoft allows the creation of an API to fetch different kind of metrics (audit logs are one of … --- ## [Logstash, Mongodb and JDBC](https://discuss.elastic.co/t/logstash-mongodb-and-jdbc/83867)
**Author:** [@AdChamp](https://discuss.elastic.co/u/AdChamp)\ **Replies:** 4\ **Last updated:** [May 11, 2017, 1:31pm UTC](https://discuss.elastic.co/t/logstash-mongodb-and-jdbc/83867 "2017-05-11T13:31:13Z")
Hi, I have a problem configuring logstash. I want to be able to put in input jdbc for mongodb. My config : input{ jdbc{ jdbc\_driver\_library =\> "mongo-java-driver-3.2.2.jar" jdbc\_driver\_class =\> "com.mongodb.Mongo… --- ## [Logstash GREEDYMULTILINE](https://discuss.elastic.co/t/logstash-greedymultiline/85394)
**Author:** [@bogomop](https://discuss.elastic.co/u/bogomop)\ **Replies:** 2\ **Last updated:** [May 11, 2017, 1:24pm UTC](https://discuss.elastic.co/t/logstash-greedymultiline/85394 "2017-05-11T13:24:01Z")
getting this error when trying to start logstash \[2017-05-11T12:35:56,772\]\[ERROR\]\[logstash.filters.grok \] Unknown setting 'pattern\_definitions' for grok \[2017-05-11T12:35:56,795\]\[ERROR\]\[logstash.agent \] Ca… --- ## [Parsing with logstash](https://discuss.elastic.co/t/parsing-with-logstash/85377)
**Author:** [@Sam67000](https://discuss.elastic.co/u/Sam67000)\ **Replies:** 3\ **Last updated:** [May 11, 2017, 12:55pm UTC](https://discuss.elastic.co/t/parsing-with-logstash/85377 "2017-05-11T12:55:15Z")
Hello EveryOne , I need your help to extract information from a document contained in a log. The document is as follows : 2017-05-10T14:28:42.387 Int 22000 ##### EI\_COFBE\_SICRC04\_GestionHO\_SSTR-v8 - 01df028ded513257 … --- ## [Sending mutiple events to single event using multiline codec for email output?](https://discuss.elastic.co/t/sending-mutiple-events-to-single-event-using-multiline-codec-for-email-output/85282)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 8\ **Last updated:** [May 11, 2017, 12:49pm UTC](https://discuss.elastic.co/t/sending-mutiple-events-to-single-event-using-multiline-codec-for-email-output/85282 "2017-05-11T12:49:22Z")
I am using logstash 2.4.0 My config is like this: input { file { path =\> "F:\\logstash-2.4.0\\logstash-2.4.0\\bin\\slowlog.txt" start\_position =\> "beginning" codec =\> multiline { … --- ## [Syslog timestamp format?](https://discuss.elastic.co/t/syslog-timestamp-format/49007)
**Author:** [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Replies:** 7\ **Last updated:** [May 11, 2017, 12:34pm UTC](https://discuss.elastic.co/t/syslog-timestamp-format/49007 "2017-05-11T12:34:07Z")
I am using logs from syslog, and syslog timestamp format doest not have year, and the timestamp format is May 3 09:10:10 how to add the year and convert the format to YYYY-MM-DD hh:mm:ss for the example ,it's will … --- ## [Store incoming data before dealing with it?](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349)
**Author:** [@BrunoC](https://discuss.elastic.co/u/BrunoC)\ **Replies:** 4\ **Last updated:** [May 11, 2017, 11:35am UTC](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349 "2017-05-11T11:35:25Z")
Hey there, I'm getting data to Logstash from tcp and udp but I want to store these data into a file before Logstash process it. Is this possible? --- ## [New Log file not pull up on kibana](https://discuss.elastic.co/t/new-log-file-not-pull-up-on-kibana/85221)
**Author:** [@Saplog](https://discuss.elastic.co/u/Saplog)\ **Replies:** 10\ **Last updated:** [May 11, 2017, 11:04am UTC](https://discuss.elastic.co/t/new-log-file-not-pull-up-on-kibana/85221 "2017-05-11T11:04:32Z")
Hello dears, I have 2 types of logs, pfSense which works perfectly , and I want to add other logs (saplog) in a file. I already create a filter (sapfilter) for these new logs, I try it with logstash-f and that's work. Wh… --- ## [Could not find or load main class logstash](https://discuss.elastic.co/t/could-not-find-or-load-main-class-logstash/85337)
**Author:** [@avdnowhere](https://discuss.elastic.co/u/avdnowhere)\ **Replies:** 12\ **Last updated:** [May 11, 2017, 9:56am UTC](https://discuss.elastic.co/t/could-not-find-or-load-main-class-logstash/85337 "2017-05-11T09:56:18Z")
Dear All, I am trying to load some data into Kibana using Logstash. I searched on the internet about it and I found a sample how to load the data from this link : https://github.com/elastic/examples/tree/master/Elastic… --- ## [Inputting Mongodb events in Elasticsearch \_id is a metadatafield and cannot be added](https://discuss.elastic.co/t/inputting-mongodb-events-in-elasticsearch--id-is-a-metadatafield-and-cannot-be-added/85356)
**Author:** [@lisadeng](https://discuss.elastic.co/u/lisadeng)\ **Replies:** 1\ **Last updated:** [May 11, 2017, 9:42am UTC](https://discuss.elastic.co/t/inputting-mongodb-events-in-elasticsearch--id-is-a-metadatafield-and-cannot-be-added/85356 "2017-05-11T09:42:52Z")
We're outputting mongodb records into filebeat, and then sending those filebeat logs to Elasticsearch, where it will be converted to Json. However, Mongo has an \_id field already, so we get a failed action error from L… --- ## [Logstash filter not working as expected](https://discuss.elastic.co/t/logstash-filter-not-working-as-expected/85190)
**Author:** [@raghuvarma](https://discuss.elastic.co/u/raghuvarma)\ **Replies:** 12\ **Last updated:** [May 11, 2017, 9:13am UTC](https://discuss.elastic.co/t/logstash-filter-not-working-as-expected/85190 "2017-05-11T09:13:32Z")
i am new to ELK and trying to learn it. I have given following filter file for logtash to get log details from the application server. these are the paths where i am taking the logs from /opt/tomcat7/apache-tomcat-7.0.… --- ## [No Output Until Verbose](https://discuss.elastic.co/t/no-output-until-verbose/85272)
**Author:** [@johnslippers](https://discuss.elastic.co/u/johnslippers)\ **Replies:** 2\ **Last updated:** [May 11, 2017, 8:28am UTC](https://discuss.elastic.co/t/no-output-until-verbose/85272 "2017-05-11T08:28:24Z")
Hi guys, This is a very strange one. I am unable to get logstash to output to file by starting logstash as normal: start logstash The strange thing is, when I start logstash in verbose mode it works: /usr/share/l… --- ## [How to handle comma delimited message with comma's existing within quotation marks](https://discuss.elastic.co/t/how-to-handle-comma-delimited-message-with-commas-existing-within-quotation-marks/85345)
**Author:** [@andynz2017](https://discuss.elastic.co/u/andynz2017)\ **Replies:** 0\ **Last updated:** [May 11, 2017, 8:20am UTC](https://discuss.elastic.co/t/how-to-handle-comma-delimited-message-with-commas-existing-within-quotation-marks/85345 "2017-05-11T08:20:00Z")
Hi All, Working on yet another curly parsing issue which I'm seeking community support as I have not readily been able to derive a graceful solution. Issue: Ingested message is in the format "string A,string B,string… --- ## [Nginx proxy logstash](https://discuss.elastic.co/t/nginx-proxy-logstash/85344)
**Author:** [@nigdav007](https://discuss.elastic.co/u/nigdav007)\ **Replies:** 0\ **Last updated:** [May 11, 2017, 8:01am UTC](https://discuss.elastic.co/t/nginx-proxy-logstash/85344 "2017-05-11T08:01:55Z")
Hay all I am setting up and logstash server, but for extra security my manager wants me to put it behind an nginx proxy. I got the logstash config sorted so that it will connect to localhost on an port only and send … --- ## [Is that possible to send logs from wso2 IoT server to elk stack?](https://discuss.elastic.co/t/is-that-possible-to-send-logs-from-wso2-iot-server-to-elk-stack/83821)
**Author:** [@DharaniKumar](https://discuss.elastic.co/u/DharaniKumar)\ **Replies:** 4\ **Last updated:** [May 11, 2017, 4:50am UTC](https://discuss.elastic.co/t/is-that-possible-to-send-logs-from-wso2-iot-server-to-elk-stack/83821 "2017-05-11T04:50:07Z")
If it is possible kindly guide me the steps to send logs from wso2 IoT server to elk stack. Thanking you in advance. --- ## [S3 output plugin does NOT write data into S3 fast enough](https://discuss.elastic.co/t/s3-output-plugin-does-not-write-data-into-s3-fast-enough/85310)
**Author:** [@drwho102003](https://discuss.elastic.co/u/drwho102003)\ **Replies:** 0\ **Last updated:** [May 10, 2017, 9:30pm UTC](https://discuss.elastic.co/t/s3-output-plugin-does-not-write-data-into-s3-fast-enough/85310 "2017-05-10T21:30:11Z")
I'm using the file input logstash plugin to detect new lines in my input files and using the S3 output plugin to have logstash writing data into S3. I'm invoking logstash like this: /usr/share/logstash/bin/logstash -… --- ## [Logstash-filter-grok downgraded from 3.4.0 to 3.3.1 with logstash 5.3 to 5.4 upgrade](https://discuss.elastic.co/t/logstash-filter-grok-downgraded-from-3-4-0-to-3-3-1-with-logstash-5-3-to-5-4-upgrade/84974)
**Author:** [@jeffspahr](https://discuss.elastic.co/u/jeffspahr)\ **Replies:** 1\ **Last updated:** [May 10, 2017, 9:02pm UTC](https://discuss.elastic.co/t/logstash-filter-grok-downgraded-from-3-4-0-to-3-3-1-with-logstash-5-3-to-5-4-upgrade/84974 "2017-05-10T21:02:45Z")
Hello, First off thank you for the 5.4 release! Was the grok filter downgrade intentional or is this a bug? The documentation also reflects the lower version grok plugin in 5.4. https://www.elastic.co/guide/en/log… --- ## [Pulling data from SQL server main pipeline stalls](https://discuss.elastic.co/t/pulling-data-from-sql-server-main-pipeline-stalls/85307)
**Author:** [@cogdellj](https://discuss.elastic.co/u/cogdellj)\ **Replies:** 0\ **Last updated:** [May 10, 2017, 9:01pm UTC](https://discuss.elastic.co/t/pulling-data-from-sql-server-main-pipeline-stalls/85307 "2017-05-10T21:01:32Z")
I wrote a simple sql query to get test data it pulls the top 100 with no problem when I tried to remove the top 100 and pull in the 2500 records in the table logstash shutsdown after a few seconds with the following from… --- ## [How to setup Logstash agent secure SSL communication with RabbitMQ and eventually to ELK server](https://discuss.elastic.co/t/how-to-setup-logstash-agent-secure-ssl-communication-with-rabbitmq-and-eventually-to-elk-server/85248)
**Author:** [@satishkori](https://discuss.elastic.co/u/satishkori)\ **Replies:** 4\ **Last updated:** [May 10, 2017, 8:04pm UTC](https://discuss.elastic.co/t/how-to-setup-logstash-agent-secure-ssl-communication-with-rabbitmq-and-eventually-to-elk-server/85248 "2017-05-10T20:04:08Z")
Hi all, Firstly does Logstash agent supports SSL with RabbitMQ? if so ,could you list the steps to ships logs using logstash agent on one server to RabbitMQ server and finally the ELK server on another machine should… --- ## [Logstash Confluent Unable to config](https://discuss.elastic.co/t/logstash-confluent-unable-to-config/85296)
**Author:** [@gauravprasad](https://discuss.elastic.co/u/gauravprasad)\ **Replies:** 0\ **Last updated:** [May 10, 2017, 7:00pm UTC](https://discuss.elastic.co/t/logstash-confluent-unable-to-config/85296 "2017-05-10T19:00:02Z")
Hi , I m trying to read the Avro message from the confluent kafka topic. Here is the setting in try to provide. input { kafka { bootstrap\_servers =\> "localhost:9092" topics =\> \["AOC\_CE.glob… --- ## [PFsense 2.3.3 and ELK](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928)
**Author:** [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Replies:** 6\ **Last updated:** [May 10, 2017, 6:50pm UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928 "2017-05-10T18:50:09Z")
After I successfully integrated the logs from my Synology. I now go over to my pfsense. There a some tutorials out in the www. Unfortunately none of them is working properly or just with old versions. I'm following the … --- ## [Monitoring csv files in a directory using logstash.conf](https://discuss.elastic.co/t/monitoring-csv-files-in-a-directory-using-logstash-conf/85104)
**Author:** [@sirsyedian](https://discuss.elastic.co/u/sirsyedian)\ **Replies:** 12\ **Last updated:** [May 10, 2017, 3:50pm UTC](https://discuss.elastic.co/t/monitoring-csv-files-in-a-directory-using-logstash-conf/85104 "2017-05-10T15:50:26Z")
Dear All, I am new to ELK stack and have recently installed logstash, Elasticsearch and Kibana (all version 5.4 on CentOS 6.5). Objective is to monitor a directory for new csv files and parse them using logstash and i… --- ## [Cannot get any output](https://discuss.elastic.co/t/cannot-get-any-output/84531)
**Author:** [@jagan](https://discuss.elastic.co/u/jagan)\ **Replies:** 6\ **Last updated:** [May 10, 2017, 3:07pm UTC](https://discuss.elastic.co/t/cannot-get-any-output/84531 "2017-05-10T15:07:12Z")
HI All, i recently upgraded to logstash-5.3.2.when i ran a 'x.conf' file in logstash i get the below message: INFO logstash.agent - Successfully started Logstash API endpoint {:port=\>96000} but i could not see the outpu… --- ## [Sending logs through logstash using email output plugin?](https://discuss.elastic.co/t/sending-logs-through-logstash-using-email-output-plugin/85203)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 9\ **Last updated:** [May 10, 2017, 2:54pm UTC](https://discuss.elastic.co/t/sending-logs-through-logstash-using-email-output-plugin/85203 "2017-05-10T14:54:49Z")
Hi \[2017-01-14 10:48:06,848\]\[WARN \]\[index.search.slowlog.query\] \[yaswanth\] \[bank\]\[0\] took\[27.8ms\], took\_millis\[27\], types\[\], stats\[\], search\_type\[QUERY\_THEN\_FETCH\], total\_shards\[5\], source\[\], extra\_source\[\], \[20… --- ## [Logstash HTTP output plugin not working](https://discuss.elastic.co/t/logstash-http-output-plugin-not-working/85269)
**Author:** [@Dhivya](https://discuss.elastic.co/u/Dhivya)\ **Replies:** 1\ **Last updated:** [May 10, 2017, 2:52pm UTC](https://discuss.elastic.co/t/logstash-http-output-plugin-not-working/85269 "2017-05-10T14:52:42Z")
Hi, I'm trying to integrate Logstash with service-now to create an incident if some error occurs. I'm using http output plugin to call service-now REST API for creating incident. I'm getting the following exception … --- ## [Error After updating to ELK 5](https://discuss.elastic.co/t/error-after-updating-to-elk-5/85259)
**Author:** [@RustuGurkan](https://discuss.elastic.co/u/RustuGurkan)\ **Replies:** 2\ **Last updated:** [May 10, 2017, 2:27pm UTC](https://discuss.elastic.co/t/error-after-updating-to-elk-5/85259 "2017-05-10T14:27:55Z")
Hi, I did a upgrade to ELK 5 but all of the sudden it stopped working I checked my logs and it says ERR Connecting error publishing events (retrying): dial tcp 192.168.154.100:5044: getsockopt: connection refused M… --- ## [Replace tabs by spaces](https://discuss.elastic.co/t/replace-tabs-by-spaces/85222)
**Author:** [@nmane](https://discuss.elastic.co/u/nmane)\ **Replies:** 1\ **Last updated:** [May 10, 2017, 2:30pm UTC](https://discuss.elastic.co/t/replace-tabs-by-spaces/85222 "2017-05-10T14:30:56Z")
I want to replace tabs in my log by spaces. How do I do it? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=984) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=986)