# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1241 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1242 --- ## [Timestamp month name date matching](https://discuss.elastic.co/t/timestamp-month-name-date-matching/25538)
**Author:** [@marcopensio](https://discuss.elastic.co/u/marcopensio)\ **Replies:** 4\ **Last updated:** [August 29, 2015, 3:26pm UTC](https://discuss.elastic.co/t/timestamp-month-name-date-matching/25538 "2015-08-29T15:26:41Z")
Hi everyone, i have a problem with the timestamp month's name matching. I have "29/Apr/2015" in the log file, if i set date locale = en, it works fine. If i set date locale = it, it doesn't match more. I can't find th… --- ## [Syslog and SNMP Trap forwarder](https://discuss.elastic.co/t/syslog-and-snmp-trap-forwarder/28247)
**Author:** [@oil](https://discuss.elastic.co/u/oil)\ **Replies:** 0\ **Last updated:** [August 28, 2015, 2:12pm UTC](https://discuss.elastic.co/t/syslog-and-snmp-trap-forwarder/28247 "2015-08-28T14:12:12Z")
Hi, I'm looking for a Syslog and SNMP Trap forwarder, which will receive Syslog and SNMP trap messages from embedded devices, and send the traffic encrypted, to a Broker - but without needing Java / JVM (which rules ou… --- ## [How to read a huge file, tens of GB](https://discuss.elastic.co/t/how-to-read-a-huge-file-tens-of-gb/28236)
**Author:** [@pemby](https://discuss.elastic.co/u/pemby)\ **Replies:** 1\ **Last updated:** [August 28, 2015, 9:51am UTC](https://discuss.elastic.co/t/how-to-read-a-huge-file-tens-of-gb/28236 "2015-08-28T09:51:28Z")
I am capturing a huge file from Suricata, it is incrementing at multi MB per minute. I just can't read it fast enough with a single server, is it possible to process a single file with more then one server? I have SAN th… --- ## [Web Crawler input for logstash](https://discuss.elastic.co/t/web-crawler-input-for-logstash/24052)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 2\ **Last updated:** [August 28, 2015, 6:57am UTC](https://discuss.elastic.co/t/web-crawler-input-for-logstash/24052 "2015-08-28T06:57:09Z")
Hi All, Is their available plugin for webcrawler. If their any please let me now configuration steps. Thanks, Dan. --- ## [Geoip Issues with Logstash?](https://discuss.elastic.co/t/geoip-issues-with-logstash/28150)
**Author:** [@killmasta93](https://discuss.elastic.co/u/killmasta93)\ **Replies:** 6\ **Last updated:** [August 27, 2015, 5:06pm UTC](https://discuss.elastic.co/t/geoip-issues-with-logstash/28150 "2015-08-27T17:06:51Z")
Hi I was wondering if someone could help me out. I have been at it a while but nothing and whats funny is that sometimes it shows the geoip of some countries but others not. Did i miss something? http://pastebin.com/pE… --- ## [Grok parsing log failed!](https://discuss.elastic.co/t/grok-parsing-log-failed/28179)
**Author:** [@mega\_robo](https://discuss.elastic.co/u/mega_robo)\ **Replies:** 0\ **Last updated:** [August 27, 2015, 12:52pm UTC](https://discuss.elastic.co/t/grok-parsing-log-failed/28179 "2015-08-27T12:52:17Z")
hi everyone :))) i have pfsense 2.2 that send logs to ELK and its work correctly but logstash not pars log and i see just raw data of logs in kibana that can not useful for my dashboard i see log in kibana like that : … --- ## [Logstash-Forwarder stopping registrar of logs from access log after reandom time](https://discuss.elastic.co/t/logstash-forwarder-stopping-registrar-of-logs-from-access-log-after-reandom-time/28170)
**Author:** [@Vineeth\_Zacharia](https://discuss.elastic.co/u/Vineeth_Zacharia)\ **Replies:** 0\ **Last updated:** [August 27, 2015, 11:31am UTC](https://discuss.elastic.co/t/logstash-forwarder-stopping-registrar-of-logs-from-access-log-after-reandom-time/28170 "2015-08-27T11:31:51Z")
Hi, I am trying to do a simple ELK stack configuration with pulling of logs from log-stash forwarder from one of the apache servers. I am running 0.4.0 version of logstash-forwarder. It randomly stops to pull logs from… --- ## [Multiple Segments logs Collection](https://discuss.elastic.co/t/multiple-segments-logs-collection/27962)
**Author:** [@puneet2607](https://discuss.elastic.co/u/puneet2607)\ **Replies:** 4\ **Last updated:** [August 27, 2015, 11:23am UTC](https://discuss.elastic.co/t/multiple-segments-logs-collection/27962 "2015-08-27T11:23:04Z")
Hi, I am a newbie, I am planning to use Logstash for collecting multiple segments logs listening over TCP. Example log format with 23 segments are below: \<181\>Jun 29 14:41:37 server1 CM\_Passed\_Authentications 000000000… --- ## [Help with a grok pattern](https://discuss.elastic.co/t/help-with-a-grok-pattern/27944)
**Author:** [@simonrisberg](https://discuss.elastic.co/u/simonrisberg)\ **Replies:** 5\ **Last updated:** [August 27, 2015, 8:44am UTC](https://discuss.elastic.co/t/help-with-a-grok-pattern/27944 "2015-08-27T08:44:01Z")
I have been working on a grok pattern that extracts the component from a URIPATH. I need some help configuring it so it extracts the piece of information that I want. The message "GET /nexus/content/repositories/jts-… --- ## [Logstash issue with pfSense 2.2.2 (SOLVED)](https://discuss.elastic.co/t/logstash-issue-with-pfsense-2-2-2-solved/27988)
**Author:** [@killmasta93](https://discuss.elastic.co/u/killmasta93)\ **Replies:** 8\ **Last updated:** [August 27, 2015, 2:39am UTC](https://discuss.elastic.co/t/logstash-issue-with-pfsense-2-2-2-solved/27988 "2015-08-27T02:39:51Z")
Hi, I was wondering if someone could guide me or help me with this issue I have been having. I think i know where im wrong i just dont know what do about it. Alright So i have been following 2 guides to get my pfSens… --- ## [Add a new field based on a regex capturing group](https://discuss.elastic.co/t/add-a-new-field-based-on-a-regex-capturing-group/28107)
**Author:** [@DK\_3](https://discuss.elastic.co/u/DK_3)\ **Replies:** 1\ **Last updated:** [August 26, 2015, 1:50pm UTC](https://discuss.elastic.co/t/add-a-new-field-based-on-a-regex-capturing-group/28107 "2015-08-26T13:50:18Z")
The logstash events have path field with a value like: /var/log/my-app\_1-2015-08-26.0.log I want a filter that can take the path field and create a new field with just the my-app\_1 part of the path string My attempt … --- ## [Junk characters!](https://discuss.elastic.co/t/junk-characters/28092)
**Author:** [@sreeram](https://discuss.elastic.co/u/sreeram)\ **Replies:** 2\ **Last updated:** [August 26, 2015, 11:19am UTC](https://discuss.elastic.co/t/junk-characters/28092 "2015-08-26T11:19:54Z")
I'm using ELK for centralized logging, in kibanna I'm seeing junk characters (for first 2 hits) as shown in image Log file encoding format is UCS-2 How to avoid this junk characters?! --- ## [How do I deal with insanely long lines in the Logstash config?](https://discuss.elastic.co/t/how-do-i-deal-with-insanely-long-lines-in-the-logstash-config/28045)
**Author:** [@nekatreven](https://discuss.elastic.co/u/nekatreven)\ **Replies:** 2\ **Last updated:** [August 26, 2015, 8:13am UTC](https://discuss.elastic.co/t/how-do-i-deal-with-insanely-long-lines-in-the-logstash-config/28045 "2015-08-26T08:13:38Z")
I'm working on an email output, which is working, but as I add a header with some help text for the events and some minimal styling, the config line with the 'htmlbody' parameter is going to get very long (400 to 600 cha… --- ## [How to parse live continous logs from iptables using ulog which are in pcap format?](https://discuss.elastic.co/t/how-to-parse-live-continous-logs-from-iptables-using-ulog-which-are-in-pcap-format/28017)
**Author:** [@Akshat\_Kakkar](https://discuss.elastic.co/u/Akshat_Kakkar)\ **Replies:** 1\ **Last updated:** [August 26, 2015, 6:43am UTC](https://discuss.elastic.co/t/how-to-parse-live-continous-logs-from-iptables-using-ulog-which-are-in-pcap-format/28017 "2015-08-26T06:43:47Z")
I am logging from iptables using ulog pcap plugin. I am using pcap mode as this is fastest possible option with ulog. I am looking at traffic rates of the orders of around 15-20K/sec. I want to parse this live pcap outpu… --- ## [Array field, multivalue field, nested object‏](https://discuss.elastic.co/t/array-field-multivalue-field-nested-object/28065)
**Author:** [@shady](https://discuss.elastic.co/u/shady)\ **Replies:** 0\ **Last updated:** [August 26, 2015, 6:20am UTC](https://discuss.elastic.co/t/array-field-multivalue-field-nested-object/28065 "2015-08-26T06:20:21Z")
I need to get one to many relation from oracle database and create an elasticsaerch index with documents with multi value field (array). I have two oracle tables: CarMark (cmk\_id integer, cmk\_name varchar2(100) ); … --- ## [Logstash jdbc input plugin vs jdbc imorter](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-vs-jdbc-imorter/28063)
**Author:** [@poonam](https://discuss.elastic.co/u/poonam)\ **Replies:** 2\ **Last updated:** [August 26, 2015, 6:13am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-vs-jdbc-imorter/28063 "2015-08-26T06:13:35Z")
how do we chose between logstash as jdbc input plugin and jdbc importer. --- ## [Logstash, syslog severity, etc](https://discuss.elastic.co/t/logstash-syslog-severity-etc/28057)
**Author:** [@declick](https://discuss.elastic.co/u/declick)\ **Replies:** 1\ **Last updated:** [August 26, 2015, 6:06am UTC](https://discuss.elastic.co/t/logstash-syslog-severity-etc/28057 "2015-08-26T06:06:48Z")
I am a logstash noob, and have been trying to find the answer for a few hours now, searching here and all over google. Please forgive me if my question has been asked and answered - I'm starting to see double.. I have … --- ## [Logstash JDBC Input Plugin for streaming data](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-for-streaming-data/28033)
**Author:** [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Replies:** 3\ **Last updated:** [August 26, 2015, 5:46am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-for-streaming-data/28033 "2015-08-26T05:46:40Z")
Can we use logstash-jdbc-input plugin for streaming data , ex: if I have data available in my database and I run JDBC input plugin , it will index the data into es, but if after some time more data comes of database , Is… --- ## [Kv filter truncating value string at whitespace within quoted string](https://discuss.elastic.co/t/kv-filter-truncating-value-string-at-whitespace-within-quoted-string/28041)
**Author:** [@robynkoz](https://discuss.elastic.co/u/robynkoz)\ **Replies:** 4\ **Last updated:** [August 25, 2015, 8:07pm UTC](https://discuss.elastic.co/t/kv-filter-truncating-value-string-at-whitespace-within-quoted-string/28041 "2015-08-25T20:07:21Z")
This is a followup to this question: https://discuss.elastic.co/t/kvpairs-where-one-of-the-values-is-a-json-structure/27980 I am using the following filter to interpret my logs: filter { grok { match =\> {"messa… --- ## [Reading environmental values in logstash config file in input/output](https://discuss.elastic.co/t/reading-environmental-values-in-logstash-config-file-in-input-output/27983)
**Author:** [@biswajit86](https://discuss.elastic.co/u/biswajit86)\ **Replies:** 3\ **Last updated:** [August 26, 2015, 12:08am UTC](https://discuss.elastic.co/t/reading-environmental-values-in-logstash-config-file-in-input-output/27983 "2015-08-26T00:08:26Z")
Is there a way to read environmental values in logstash config file . I want to pass in an env value to write test cases for my logstash configs For ex I want to do the following in my logstash config file input { i… --- ## [Configuring dynamic facility with gelf output plugin](https://discuss.elastic.co/t/configuring-dynamic-facility-with-gelf-output-plugin/28043)
**Author:** [@danieljamesscott](https://discuss.elastic.co/u/danieljamesscott)\ **Replies:** 0\ **Last updated:** [August 25, 2015, 5:46pm UTC](https://discuss.elastic.co/t/configuring-dynamic-facility-with-gelf-output-plugin/28043 "2015-08-25T17:46:03Z")
Hi, I'm forwarding logstash messages to graylog using the gelf output plugin. I would like to send the facility to graylog, which I have available as a field called 'application' in logstash. The documentation says t… --- ## [Big gaps in recorded logs](https://discuss.elastic.co/t/big-gaps-in-recorded-logs/27266)
**Author:** [@bschopman](https://discuss.elastic.co/u/bschopman)\ **Replies:** 7\ **Last updated:** [August 25, 2015, 3:27pm UTC](https://discuss.elastic.co/t/big-gaps-in-recorded-logs/27266 "2015-08-25T15:27:19Z")
We're running Logstash on a single server to compare its performance to Heka and decide if we want to use one or the other. We have some high-traffic web servers of which the log lines are sent to a Kafka cluster consist… --- ## [Logstash 1.5.4 service stopping - RHEL 6.7](https://discuss.elastic.co/t/logstash-1-5-4-service-stopping-rhel-6-7/28014)
**Author:** [@mbspence1969](https://discuss.elastic.co/u/mbspence1969)\ **Replies:** 1\ **Last updated:** [August 25, 2015, 2:19pm UTC](https://discuss.elastic.co/t/logstash-1-5-4-service-stopping-rhel-6-7/28014 "2015-08-25T14:19:01Z")
I have installed logstash on a RHEL 6.7 server from RPM. When I start the service it runs for a short time and then stops. Would anyone be able to provide some advice? The error below is logged in the err file: Java Ho… --- ## [Array variable or loop](https://discuss.elastic.co/t/array-variable-or-loop/27959)
**Author:** [@oussama](https://discuss.elastic.co/u/oussama)\ **Replies:** 4\ **Last updated:** [August 25, 2015, 7:26am UTC](https://discuss.elastic.co/t/array-variable-or-loop/27959 "2015-08-25T07:26:49Z")
Hi, I hava many input file in my configuration file, like this: file { path =\> "/root/elasticsearch/test-logs/log-node1.log" type =\> "apache" } file { path =\> "/root/elasticsearch/t… --- ## [Can not access dot (eg foo.bar) fields in logstash](https://discuss.elastic.co/t/can-not-access-dot-eg-foo-bar-fields-in-logstash/27884)
**Author:** [@ankmathur14](https://discuss.elastic.co/u/ankmathur14)\ **Replies:** 2\ **Last updated:** [August 25, 2015, 7:08am UTC](https://discuss.elastic.co/t/can-not-access-dot-eg-foo-bar-fields-in-logstash/27884 "2015-08-25T07:08:07Z")
How to access dot fields in logstash. if \[foo.bar\] == "xyz" { grok { add\_field =\> {"new\_field", "pqr"} } } Field "foo.bar" exists and its value is "xyz" but its not going in the 'if' block I al… --- ## [Kvpairs where one of the values is a JSON structure](https://discuss.elastic.co/t/kvpairs-where-one-of-the-values-is-a-json-structure/27980)
**Author:** [@robynkoz](https://discuss.elastic.co/u/robynkoz)\ **Replies:** 8\ **Last updated:** [August 25, 2015, 6:23am UTC](https://discuss.elastic.co/t/kvpairs-where-one-of-the-values-is-a-json-structure/27980 "2015-08-25T06:23:45Z")
I have a log that I am parsing using grok and kv: filter { grok { match =\> {"message" =\> "msg=%{WORD:action} %{GREEDYDATA:kvpairs}" } } kv { source =\> "kvpairs" remove\_field =\> \["kvpairs"\] } } One … --- ## [Timezone for date filter seems not working](https://discuss.elastic.co/t/timezone-for-date-filter-seems-not-working/27739)
**Author:** [@z\_afshar](https://discuss.elastic.co/u/z_afshar)\ **Replies:** 1\ **Last updated:** [August 25, 2015, 4:05am UTC](https://discuss.elastic.co/t/timezone-for-date-filter-seems-not-working/27739 "2015-08-25T04:05:58Z")
Hello, I am newbie to logstash and have a problem with date filter. I have a historical csv file with information like this in each line: 2015-06-10 16:00:00.017,10.0.0.100,192.168.1.1 the time specified above is in… --- ## [In same cluster, logstash can move data from A index to B index?](https://discuss.elastic.co/t/in-same-cluster-logstash-can-move-data-from-a-index-to-b-index/27183)
**Author:** [@Jungmi\_Yoo](https://discuss.elastic.co/u/Jungmi_Yoo)\ **Replies:** 2\ **Last updated:** [August 25, 2015, 3:54am UTC](https://discuss.elastic.co/t/in-same-cluster-logstash-can-move-data-from-a-index-to-b-index/27183 "2015-08-25T03:54:16Z")
To use Input / Output elasticsearch plugin, Can queried data from A index to B index ? Also query is aggregation query. --- ## [Multiline if events happen in the same .001 Millisecond](https://discuss.elastic.co/t/multiline-if-events-happen-in-the-same-001-millisecond/27975)
**Author:** [@michaellizhou](https://discuss.elastic.co/u/michaellizhou)\ **Replies:** 0\ **Last updated:** [August 24, 2015, 8:26pm UTC](https://discuss.elastic.co/t/multiline-if-events-happen-in-the-same-001-millisecond/27975 "2015-08-24T20:26:46Z")
Is there a method to grouping messages that occur at the same .001 millisecond? So there are some logs that I would like to preserve order and I think the only way is to group messages that occur at the same .001 millise… --- ## [How to multiline large messages?](https://discuss.elastic.co/t/how-to-multiline-large-messages/27971)
**Author:** [@michaellizhou](https://discuss.elastic.co/u/michaellizhou)\ **Replies:** 2\ **Last updated:** [August 24, 2015, 8:15pm UTC](https://discuss.elastic.co/t/how-to-multiline-large-messages/27971 "2015-08-24T20:15:21Z")
After taking a step back I notice a lot of messages are useless. And I want to add a tag on them to easily filter while on Kibana. The message is actually logs that span about 70 lines that when put together describe a h… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1240) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1242)