# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=965 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 966 --- ## [Removing specific text from a log mesg](https://discuss.elastic.co/t/removing-specific-text-from-a-log-mesg/89023)
**Author:** [@Raghu19](https://discuss.elastic.co/u/Raghu19)\ **Replies:** 4\ **Last updated:** [June 14, 2017, 12:52pm UTC](https://discuss.elastic.co/t/removing-specific-text-from-a-log-mesg/89023 "2017-06-14T12:52:15Z")
Is there a way to parse the log message and search for a particular string and if that is present, then to ignore/remove that string from that message? Every log message in the log file that i am trying to parse using … --- ## [Filter to verify json fields type](https://discuss.elastic.co/t/filter-to-verify-json-fields-type/89330)
**Author:** [@Amir\_Gilad](https://discuss.elastic.co/u/Amir_Gilad)\ **Replies:** 4\ **Last updated:** [June 14, 2017, 12:48pm UTC](https://discuss.elastic.co/t/filter-to-verify-json-fields-type/89330 "2017-06-14T12:48:44Z")
Hi, We're currently using internal pipeline to store our raw data and want to move to Logstash. The inputs are jsons and the json filter actually works quite well in parsing them. The problem is that currently we have … --- ## [Grok filter cant split error message and java stacktrace into fields](https://discuss.elastic.co/t/grok-filter-cant-split-error-message-and-java-stacktrace-into-fields/88824)
**Author:** [@trousse](https://discuss.elastic.co/u/trousse)\ **Replies:** 1\ **Last updated:** [June 12, 2017, 7:33am UTC](https://discuss.elastic.co/t/grok-filter-cant-split-error-message-and-java-stacktrace-into-fields/88824 "2017-06-12T07:33:09Z")
i'm tryning that parse logs from a tomcat server which write javastacktrace after error like : "\[ERROR - 18:02:00.028, org.springframework.scheduling.support.TaskUtils$LoggingErrorHandler-95\] Unexpected error occurred… --- ## [Monitor multiple Logstash instances running on the same server](https://discuss.elastic.co/t/monitor-multiple-logstash-instances-running-on-the-same-server/89346)
**Author:** [@Kvin](https://discuss.elastic.co/u/Kvin)\ **Replies:** 1\ **Last updated:** [June 14, 2017, 12:17pm UTC](https://discuss.elastic.co/t/monitor-multiple-logstash-instances-running-on-the-same-server/89346 "2017-06-14T12:17:27Z")
We have a use case where we have several Logstash instances running on the same server, however when we would like to monitor them in Kibana, we see Logstashes per node and not per instance. What is the usual way to ma… --- ## [Fix grok pattern](https://discuss.elastic.co/t/fix-grok-pattern/89286)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 4\ **Last updated:** [June 14, 2017, 10:44am UTC](https://discuss.elastic.co/t/fix-grok-pattern/89286 "2017-06-14T10:44:22Z")
Hello, I've a logfile which contains a line like this: 2017-05-01 07:41:25 \[scraper.py\] DEBUG: Scraped from \<200 https://www.diwanegypt.com/\>{'category\_ar': u'biographies'} I want to extract the url into a variable cal… --- ## [Compute difference between 2 timestamps](https://discuss.elastic.co/t/compute-difference-between-2-timestamps/89276)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 2\ **Last updated:** [June 14, 2017, 10:45am UTC](https://discuss.elastic.co/t/compute-difference-between-2-timestamps/89276 "2017-06-14T10:45:02Z")
Hello, I want to calculate time elapsed between the start and end of a logfile, if I extract the timestamp for every line, can I calculate the min and max (or first and last) ones, and get the difference between them? f… --- ## [Multiple Line in Single Event](https://discuss.elastic.co/t/multiple-line-in-single-event/88739)
**Author:** [@sanupam](https://discuss.elastic.co/u/sanupam)\ **Replies:** 5\ **Last updated:** [June 14, 2017, 7:16am UTC](https://discuss.elastic.co/t/multiple-line-in-single-event/88739 "2017-06-14T07:16:20Z")
Hi, Consider the below example: Logfile: START INFO: NOTIFICATION\_ID,NOTIFICATION\_TYPE,NOTIFICATION\_DATE,ORDER\_REFERENCE,US\_ID,SIO\_ID,ORIGINAL\_ORDER\_REFERENCE,ACCESS\_QUOTE\_REF,PHONE,ADDRESS1,ADDRESS2,ADDRESS3,POSTA… --- ## [How to achieve high availability of ELK Stack?](https://discuss.elastic.co/t/how-to-achieve-high-availability-of-elk-stack/89326)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 4\ **Last updated:** [June 14, 2017, 9:55am UTC](https://discuss.elastic.co/t/how-to-achieve-high-availability-of-elk-stack/89326 "2017-06-14T09:55:03Z")
Hi Experts, Sorry if I am asking this question in wrong group. I have Archsight CEF data which I will be ingesting into ES using LS . To achieve HA I have total 8 machines 2-ES, 2-LS(these also have syslog-ng), 2-Kiban… --- ## [Error while setting up enviromental variables](https://discuss.elastic.co/t/error-while-setting-up-enviromental-variables/88532)
**Author:** [@Kvin](https://discuss.elastic.co/u/Kvin)\ **Replies:** 3\ **Last updated:** [June 14, 2017, 9:28am UTC](https://discuss.elastic.co/t/error-while-setting-up-enviromental-variables/88532 "2017-06-14T09:28:34Z")
I have the following line in my Logstash RabbitMQ input: arguments =\> { "x-max-length" =\> 250000 } I want the value to be easily changable, so I would like to store it in an enviromental variable. arguments =\> { "x… --- ## [Syslog + Apache log =\> \_Grokparsefailure](https://discuss.elastic.co/t/syslog-apache-log--grokparsefailure/88841)
**Author:** [@Mehdi\_Khan](https://discuss.elastic.co/u/Mehdi_Khan)\ **Replies:** 6\ **Last updated:** [June 14, 2017, 9:27am UTC](https://discuss.elastic.co/t/syslog-apache-log--grokparsefailure/88841 "2017-06-14T09:27:36Z")
Hi, I'm trying to parse an Apache log that was sent by syslog but I get a \_grokparsefailure error in Kibana and nothing in logstash log. Yet I tested the grok patterns on http://grokconstructor.appspot.com with several… --- ## [Add new field](https://discuss.elastic.co/t/add-new-field/89344)
**Author:** [@Chemse](https://discuss.elastic.co/u/Chemse)\ **Replies:** 0\ **Last updated:** [June 14, 2017, 9:21am UTC](https://discuss.elastic.co/t/add-new-field/89344 "2017-06-14T09:21:46Z")
Hello, I am parsing my data with logstash and I want to add a new field and I want to get it value from the first variable of a hash list called Tps wich I got it from a split of the message. I've tried this line ut … --- ## [Stop logstash in CentOs](https://discuss.elastic.co/t/stop-logstash-in-centos/89342)
**Author:** [@richagautam](https://discuss.elastic.co/u/richagautam)\ **Replies:** 0\ **Last updated:** [June 14, 2017, 9:18am UTC](https://discuss.elastic.co/t/stop-logstash-in-centos/89342 "2017-06-14T09:18:41Z")
Hi, I am new to Logstash and centos. I have installed logstash in centOS7, now everytime i run a config file it gives the messages like "sending ........." then parsed data on cosole... then as logstash behaves it… --- ## [Logstash erroring out](https://discuss.elastic.co/t/logstash-erroring-out/89309)
**Author:** [@manojrai](https://discuss.elastic.co/u/manojrai)\ **Replies:** 1\ **Last updated:** [June 14, 2017, 8:02am UTC](https://discuss.elastic.co/t/logstash-erroring-out/89309 "2017-06-14T08:02:13Z")
Hi Team, We are getting the following error ,and data is losses during the time . \[2017-06-12T20:30:58,612\]\[WARN \]\[org.apache.kafka.clients.consumer.internals.ConsumerCoordinator\] Auto offset commit failed for group … --- ## [Logstash filter: compare time using regex](https://discuss.elastic.co/t/logstash-filter-compare-time-using-regex/89229)
**Author:** [@karmax](https://discuss.elastic.co/u/karmax)\ **Replies:** 2\ **Last updated:** [June 14, 2017, 7:47am UTC](https://discuss.elastic.co/t/logstash-filter-compare-time-using-regex/89229 "2017-06-14T07:47:16Z")
Hello everybody! I use this code in logstash filter to compare time but don't work. if \[timecheck\] =~ /.\*((\\\[0\\\]\\\[0-6\\\]):\\\[0-5\\\]\\\[0-9\\\]:\\\[0-5\\\]\\\[0-9\\\])|((\\\[1\\\]\\\[2-9\\\]|2\\\[0-3\\\]):\\\[0-5\\\]\\\[0-9\\\]:\\\[0-5\\\]\\\[0-9\\\]).\*/ { mut… --- ## [Unable to install plugin on windows](https://discuss.elastic.co/t/unable-to-install-plugin-on-windows/89312)
**Author:** [@BayBay\_Hu](https://discuss.elastic.co/u/BayBay_Hu)\ **Replies:** 0\ **Last updated:** [June 14, 2017, 7:23am UTC](https://discuss.elastic.co/t/unable-to-install-plugin-on-windows/89312 "2017-06-14T07:23:10Z")
Hi,all: I could not install logstash plugin on windows 7. Here are my steps: 1. download logstash zip and unzip it. 2. open cmd and move to \\logstash-5.4.1\\bin 3. logstash-plugin install logstash-input-heroku But it … --- ## [IF condition on the beginning of a log line](https://discuss.elastic.co/t/if-condition-on-the-beginning-of-a-log-line/89230)
**Author:** [@Ksugiu](https://discuss.elastic.co/u/Ksugiu)\ **Replies:** 2\ **Last updated:** [June 14, 2017, 7:04am UTC](https://discuss.elastic.co/t/if-condition-on-the-beginning-of-a-log-line/89230 "2017-06-14T07:04:28Z")
Hi all :), Happy to be new to ELK. ( 2 days :stuck\_out\_tongue: ) I would like to know how to make a condition on the beginnig of a log line. For exemple : This is my log : 2017-05-31 12:09:50,070 10885 INFO opensta… --- ## [Cannot get filebeat 5,4,1 to send logs to logstash 5.4.1](https://discuss.elastic.co/t/cannot-get-filebeat-5-4-1-to-send-logs-to-logstash-5-4-1/89304)
**Author:** [@JayArthur](https://discuss.elastic.co/u/JayArthur)\ **Replies:** 0\ **Last updated:** [June 14, 2017, 6:41am UTC](https://discuss.elastic.co/t/cannot-get-filebeat-5-4-1-to-send-logs-to-logstash-5-4-1/89304 "2017-06-14T06:41:14Z")
I've just installed a fresh ELK stack version 5.4.1. Everything goes well but logstash fails to listen to port 5044. when I run netstat -pnltu, I do not see any port on which its listening. I;ve turned off the firewall. … --- ## [Using wildcard character in logstash input](https://discuss.elastic.co/t/using-wildcard-character-in-logstash-input/88585)
**Author:** [@jigarpatel13533](https://discuss.elastic.co/u/jigarpatel13533)\ **Replies:** 4\ **Last updated:** [June 14, 2017, 6:34am UTC](https://discuss.elastic.co/t/using-wildcard-character-in-logstash-input/88585 "2017-06-14T06:34:33Z")
Hi, I am using logstash 2.1 in Windows Server 2008 R2 OS. (Version Corrected) I have some log files at some another server, and I am taking them as input for my logstash. Shared Path : \\\\MIB030065\\share2\\Statoil-logs… --- ## [Display log in a given format](https://discuss.elastic.co/t/display-log-in-a-given-format/89294)
**Author:** [@deepika](https://discuss.elastic.co/u/deepika)\ **Replies:** 3\ **Last updated:** [June 14, 2017, 6:12am UTC](https://discuss.elastic.co/t/display-log-in-a-given-format/89294 "2017-06-14T06:12:14Z")
Hello, I put the below code to group lines which start with "START" tag: codec =\> multiline { pattern =\> "^START" negate =\> true what =\> previous auto\_flush\_interval =\> 1 } but i want to group line only whi… --- ## [Dateparsefailure while replacing @timestamp](https://discuss.elastic.co/t/dateparsefailure-while-replacing-timestamp/87791)
**Author:** [@pratheek\_k](https://discuss.elastic.co/u/pratheek_k)\ **Replies:** 18\ **Last updated:** [June 14, 2017, 5:34am UTC](https://discuss.elastic.co/t/dateparsefailure-while-replacing-timestamp/87791 "2017-06-14T05:34:43Z")
Problem: Logs not ordered in when viewing through kibana. Goal: Replace timestamp with the time in the log messaage. So that i can order my logs in kibana Error: dataparsefailure. I dont see any other error message i… --- ## [Unable to parse JSON Array object in Logstash](https://discuss.elastic.co/t/unable-to-parse-json-array-object-in-logstash/89247)
**Author:** [@ihsanhaikalz](https://discuss.elastic.co/u/ihsanhaikalz)\ **Replies:** 1\ **Last updated:** [June 13, 2017, 7:02pm UTC](https://discuss.elastic.co/t/unable-to-parse-json-array-object-in-logstash/89247 "2017-06-13T19:02:23Z")
I am trying to parse an example JSON Array into Logstash as follow: { "Records": \[ { "eventVersion": "1.03", "userIdentity": { "type": "IAMUser", "principalId": "11112222… --- ## [Grok filter inquiry](https://discuss.elastic.co/t/grok-filter-inquiry/89064)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 6\ **Last updated:** [June 13, 2017, 8:43pm UTC](https://discuss.elastic.co/t/grok-filter-inquiry/89064 "2017-06-13T20:43:25Z")
Hello, how to extract a specific word out of a string without spaces, for example: 'stock\_status': 'IN\_STOCK','title\_ar': u'Happy Wedding Card' I want to extract the (IN\_STOCK), I tried: grok{ match =\> \[ "message", … --- ## [Filter: error message and string comparison](https://discuss.elastic.co/t/filter-error-message-and-string-comparison/86853)
**Author:** [@pir1981](https://discuss.elastic.co/u/pir1981)\ **Replies:** 2\ **Last updated:** [June 13, 2017, 7:45pm UTC](https://discuss.elastic.co/t/filter-error-message-and-string-comparison/86853 "2017-06-13T19:45:59Z")
Hi I have an error message that I would like to drop. But my filter does not seem to ever trigger. Message 2017-05-23T18:24:47.644Z -- ERROR -- \[main\] ERROR de.komoot.photon.elasticsearch.Server - cannot install pl… --- ## [Nested Avro Schemas](https://discuss.elastic.co/t/nested-avro-schemas/48701)
**Author:** [@jchoinski](https://discuss.elastic.co/u/jchoinski)\ **Replies:** 1\ **Last updated:** [June 13, 2017, 7:12pm UTC](https://discuss.elastic.co/t/nested-avro-schemas/48701 "2017-06-13T19:12:28Z")
Hello, I am using logstash 2.3.2. Is there anyway to specify in the config file or organize the avsc files in order to use nested Avro with logstash? This is my config file. input { kafka { codec =\> avro{ schem… --- ## [Date filter not working for AM/PM](https://discuss.elastic.co/t/date-filter-not-working-for-am-pm/89253)
**Author:** [@wlin](https://discuss.elastic.co/u/wlin)\ **Replies:** 2\ **Last updated:** [June 13, 2017, 6:30pm UTC](https://discuss.elastic.co/t/date-filter-not-working-for-am-pm/89253 "2017-06-13T18:30:11Z")
I have some log file for example this: ... {"timestamp":"Jun 12, 2017 6:19:31 PM","ip":"0:0:0:0:0:0:0:1"} ... As I want to use the timestamp in the log file, I use a data filter to match it input { beats { po… --- ## [Enable SSL certificate in jmx input plugin](https://discuss.elastic.co/t/enable-ssl-certificate-in-jmx-input-plugin/86358)
**Author:** [@sujithattinad](https://discuss.elastic.co/u/sujithattinad)\ **Replies:** 1\ **Last updated:** [June 13, 2017, 5:58pm UTC](https://discuss.elastic.co/t/enable-ssl-certificate-in-jmx-input-plugin/86358 "2017-06-13T17:58:53Z")
How to add SSL certificate to communicate with JMX server host? --- ## [Grok Pattern for below time stamp](https://discuss.elastic.co/t/grok-pattern-for-below-time-stamp/88865)
**Author:** [@Ravikumar\_G](https://discuss.elastic.co/u/Ravikumar_G)\ **Replies:** 3\ **Last updated:** [June 13, 2017, 5:32pm UTC](https://discuss.elastic.co/t/grok-pattern-for-below-time-stamp/88865 "2017-06-13T17:32:19Z")
Log FIle :thumbsdown: 1496975411.858 cqtt=02:30:11 chi=127.0.0.1 cqhm=GET pssc=304 ttms=1 b=0 sssc=304 sscl=0 cfsc=FIN pfsc=FIN crc=TCP\_REFRESH\_HIT phr=DIRECT uas="teakd 1.9.2/2017-05-01/4e528b7-git" xmt="trace-id=c0… --- ## [Syslog\_severity\_code it's always "5" , syslog\_facility it's always "1", syslog\_facility it's always "user-level" and syslog\_severity it's always "notice"](https://discuss.elastic.co/t/syslog-severity-code-its-always-5-syslog-facility-its-always-1-syslog-facility-its-always-user-level-and-syslog-severity-its-always-notice/88892)
**Author:** [@Julian\_Tete](https://discuss.elastic.co/u/Julian_Tete)\ **Replies:** 8\ **Last updated:** [June 13, 2017, 4:46pm UTC](https://discuss.elastic.co/t/syslog-severity-code-its-always-5-syslog-facility-its-always-1-syslog-facility-its-always-user-level-and-syslog-severity-its-always-notice/88892 "2017-06-13T16:46:45Z")
Hello ELK's Friends I am trying to set up an ELK system to replace an Rsyslog + Adiscon LogAnalyzer implementation. The operating system used was CentOS 7.3 The versions of the installed packages were the following: f… --- ## [Finding a pattern multiple times in a multiline event](https://discuss.elastic.co/t/finding-a-pattern-multiple-times-in-a-multiline-event/87779)
**Author:** [@mahsa\_b](https://discuss.elastic.co/u/mahsa_b)\ **Replies:** 3\ **Last updated:** [June 13, 2017, 4:10pm UTC](https://discuss.elastic.co/t/finding-a-pattern-multiple-times-in-a-multiline-event/87779 "2017-06-13T16:10:17Z")
I am reading multiline from input and trying to match all the instances of the pattern. but it only output the first instance and not the rest. I'm not sure where I am making the mistake. this is my conf file content: i… --- ## [Change timestamp to be timestamp from logfile](https://discuss.elastic.co/t/change-timestamp-to-be-timestamp-from-logfile/86993)
**Author:** [@pmohan](https://discuss.elastic.co/u/pmohan)\ **Replies:** 2\ **Last updated:** [June 13, 2017, 2:39pm UTC](https://discuss.elastic.co/t/change-timestamp-to-be-timestamp-from-logfile/86993 "2017-06-13T14:39:55Z")
Hi , Below is one log time i'm trying to parse via logstash pvl-api-ft15.sdsdsdasa 2017-05-24 13:45:06,297 INFO http-/0.0.0.0:8080-56 c.s.s.f.s.r.i.RateLimitingRequestFilter RATE\_LIMIT | path /AccountServices/V1/S… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=964) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=966)