# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1163 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1164 --- ## [Crash when loading IPv6 GeoIP database](https://discuss.elastic.co/t/crash-when-loading-ipv6-geoip-database/48227)
**Author:** [@icb](https://discuss.elastic.co/u/icb)\ **Replies:** 2\ **Last updated:** [April 23, 2016, 11:05pm UTC](https://discuss.elastic.co/t/crash-when-loading-ipv6-geoip-database/48227 "2016-04-23T23:05:56Z")
I'm wanting to be able to geolocate IPv6 addresses. MaxMind publish a free (beta) IPv6 database along side their free IPv4 database, but when I configure it, logstash crashes during initialization. Has anyone gotten this… --- ## [JOIN different data sources](https://discuss.elastic.co/t/join-different-data-sources/47480)
**Author:** [@veve90](https://discuss.elastic.co/u/veve90)\ **Replies:** 5\ **Last updated:** [April 23, 2016, 10:45pm UTC](https://discuss.elastic.co/t/join-different-data-sources/47480 "2016-04-23T22:45:34Z")
Hello, I wonder, if there is a way to join data that I'm currently receiving (in real time) with existing data (in elasticsearch). If yes, could you point me to how to do it? Thank you! --- ## [Grok patterns for credit card](https://discuss.elastic.co/t/grok-patterns-for-credit-card/47891)
**Author:** [@Gaurav\_Harsola](https://discuss.elastic.co/u/Gaurav_Harsola)\ **Replies:** 7\ **Last updated:** [April 23, 2016, 6:42pm UTC](https://discuss.elastic.co/t/grok-patterns-for-credit-card/47891 "2016-04-23T18:42:19Z")
Can someone suggest grok pattern for 4836-6646-5481 this is credit card number Thanks gaurav --- ## [Simple logstash HTTP poller](https://discuss.elastic.co/t/simple-logstash-http-poller/48052)
**Author:** [@Amit\_Kulkarni](https://discuss.elastic.co/u/Amit_Kulkarni)\ **Replies:** 5\ **Last updated:** [April 23, 2016, 6:41pm UTC](https://discuss.elastic.co/t/simple-logstash-http-poller/48052 "2016-04-23T18:41:42Z")
Hello, Can someone give me a simple HTTP poller config to check the 200 response on my URL. with a filter to check response output. I tried creating config but its not working . Thanks in Advance --- ## [Elasticsearch is down. What happens to output data from Logstash?](https://discuss.elastic.co/t/elasticsearch-is-down-what-happens-to-output-data-from-logstash/23774)
**Author:** [@vilas](https://discuss.elastic.co/u/vilas)\ **Replies:** 7\ **Last updated:** [April 22, 2016, 8:02pm UTC](https://discuss.elastic.co/t/elasticsearch-is-down-what-happens-to-output-data-from-logstash/23774 "2016-04-22T20:02:44Z")
Hi, This probably is a repeated question. But I would like to clarify this. Elasticsearch is down suddenly on central server. But Logstash is running on remote server parsing logs. We started elasticsearch after few … --- ## [Logstash benchmark](https://discuss.elastic.co/t/logstash-benchmark/48196)
**Author:** [@acr](https://discuss.elastic.co/u/acr)\ **Replies:** 0\ **Last updated:** [April 22, 2016, 5:51pm UTC](https://discuss.elastic.co/t/logstash-benchmark/48196 "2016-04-22T17:51:49Z")
Do we have any benchmark , in term of how may log lines can logstash ingest and egress ? I know this depends to a config but is there any ways to measure the rate before hand so that we can tweak things before going to p… --- ## [Logstash date parse failure error](https://discuss.elastic.co/t/logstash-date-parse-failure-error/47566)
**Author:** [@shan](https://discuss.elastic.co/u/shan)\ **Replies:** 4\ **Last updated:** [April 22, 2016, 1:00pm UTC](https://discuss.elastic.co/t/logstash-date-parse-failure-error/47566 "2016-04-22T13:00:17Z")
Hi , I'm getting the date parse failure error., Please help me on this {:timestamp=\>"2016-04-16T15:21:57.639000-0400", :message=\>"Failed parsing date from field", :field=\>"timestamp", :value=\>"2016-04-16 19:21:54", :… --- ## [Every message being throttled](https://discuss.elastic.co/t/every-message-being-throttled/48167)
**Author:** [@kgrant](https://discuss.elastic.co/u/kgrant)\ **Replies:** 0\ **Last updated:** [April 22, 2016, 12:34pm UTC](https://discuss.elastic.co/t/every-message-being-throttled/48167 "2016-04-22T12:34:33Z")
Hi, I'm new at using logstash. I'm trying to use throttling in my filter and every message is getting throttled for some reason. Do I have an error in my config? I was expecting every 2nd, 3rd, and 4th message con… --- ## [Logstash not show any output (again)](https://discuss.elastic.co/t/logstash-not-show-any-output-again/37206)
**Author:** [@Lallartu](https://discuss.elastic.co/u/Lallartu)\ **Replies:** 12\ **Last updated:** [April 22, 2016, 10:54am UTC](https://discuss.elastic.co/t/logstash-not-show-any-output-again/37206 "2016-04-22T10:54:05Z")
I'am trying to use logstash for parsing rest-api logs into file, however the only output is logstash -f ../conf/first-pipeline.conf c:\\ELK\\logstash\\bin\>logstash -f ../conf/first-pipeline.conf io/console not supported; t… --- ## [Offline beats plugin management issue](https://discuss.elastic.co/t/offline-beats-plugin-management-issue/48152)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 6\ **Last updated:** [April 22, 2016, 10:09am UTC](https://discuss.elastic.co/t/offline-beats-plugin-management-issue/48152 "2016-04-22T10:09:42Z")
Hi, I am referring to https://www.elastic.co/guide/en/logstash/current/offline-plugins.html#managing-packs When I execute command /bin/plugin pack on windows, then it says no such subcommand pack. Please let me know If I… --- ## [Appending csv contents to an existing logstash index](https://discuss.elastic.co/t/appending-csv-contents-to-an-existing-logstash-index/48154)
**Author:** [@PKA](https://discuss.elastic.co/u/PKA)\ **Replies:** 2\ **Last updated:** [April 22, 2016, 9:54am UTC](https://discuss.elastic.co/t/appending-csv-contents-to-an-existing-logstash-index/48154 "2016-04-22T09:54:41Z")
I have a folder where a user will keep adding csv files for logstash to pick up. User will add csv files in this folder, on a daily basis. The csv files will have names like yyyymmdd-dailyUsageData.csv, i.e. the file nam… --- ## [Logstash parsing for rich text documents](https://discuss.elastic.co/t/logstash-parsing-for-rich-text-documents/40640)
**Author:** [@Meenal.Luktuke](https://discuss.elastic.co/u/Meenal.Luktuke)\ **Replies:** 2\ **Last updated:** [April 22, 2016, 8:00am UTC](https://discuss.elastic.co/t/logstash-parsing-for-rich-text-documents/40640 "2016-04-22T08:00:38Z")
Hi, Can Logstash parse rich text documents? I have a use-case where the input is not logs, but some documents instead and then build some KPI in Kibana. I have 2 questions: 1) Can logstash do it with some plugin or I… --- ## [Any alternatives to if and mutate?](https://discuss.elastic.co/t/any-alternatives-to-if-and-mutate/48109)
**Author:** [@anhlqn](https://discuss.elastic.co/u/anhlqn)\ **Replies:** 2\ **Last updated:** [April 22, 2016, 6:35am UTC](https://discuss.elastic.co/t/any-alternatives-to-if-and-mutate/48109 "2016-04-22T06:35:50Z")
Hi all, I have a bunch of if + mutate that check dst-port field and then update the protocol field to a correct protocol name ( for custom destination port that cannot be recognized by nProbe) filter { if \[dst-port\]… --- ## [Logstash/Filebeat logs delayed by a day in ES](https://discuss.elastic.co/t/logstash-filebeat-logs-delayed-by-a-day-in-es/48041)
**Author:** [@kartikvelastic](https://discuss.elastic.co/u/kartikvelastic)\ **Replies:** 2\ **Last updated:** [April 22, 2016, 6:27am UTC](https://discuss.elastic.co/t/logstash-filebeat-logs-delayed-by-a-day-in-es/48041 "2016-04-22T06:27:01Z")
Hi, I generated a bunch of logs (the httpd access\_log log file is about 1.5 GB). I am seeing logs come in Kibana after 2 days. Is this normal? Here is a snippet: http://pastebin.com/5rQfVz8g --- ## [Index not getting created for csv import](https://discuss.elastic.co/t/index-not-getting-created-for-csv-import/47877)
**Author:** [@PKA](https://discuss.elastic.co/u/PKA)\ **Replies:** 2\ **Last updated:** [April 22, 2016, 4:40am UTC](https://discuss.elastic.co/t/index-not-getting-created-for-csv-import/47877 "2016-04-22T04:40:50Z")
I want to load csv into ES. I have followed the steps as detailed in this blog: http://blog.webkid.io/visualize-datasets-with-elk/ I am using elasticsearch 2.3.1, kibana-4.5.0-linux-x64, logstash-2.3.1. I have deleted … --- ## [LogStash Imports Numbers as Strings into Elastic](https://discuss.elastic.co/t/logstash-imports-numbers-as-strings-into-elastic/47126)
**Author:** [@wstrater](https://discuss.elastic.co/u/wstrater)\ **Replies:** 2\ **Last updated:** [April 21, 2016, 9:14pm UTC](https://discuss.elastic.co/t/logstash-imports-numbers-as-strings-into-elastic/47126 "2016-04-21T21:14:20Z")
I am struggling with importing fields into Elastic with the correct format. I am getting numbers stored as string and strings stored as IP addresses and wonder if it is my template. I eventually gave up in getting IP add… --- ## [Logstash init script bug](https://discuss.elastic.co/t/logstash-init-script-bug/48081)
**Author:** [@Cheyenne\_Greatorex](https://discuss.elastic.co/u/Cheyenne_Greatorex)\ **Replies:** 1\ **Last updated:** [April 21, 2016, 5:37pm UTC](https://discuss.elastic.co/t/logstash-init-script-bug/48081 "2016-04-21T17:37:08Z")
When logstash doesn't stop in a timely manner, the init script exits with a bug trying to handle the timeout, and when calling it with restart, it ends up starting multiple copies. sudo service logstash restart Killing… --- ## [Rename multiple fields in Logstash?](https://discuss.elastic.co/t/rename-multiple-fields-in-logstash/48069)
**Author:** [@anhlqn](https://discuss.elastic.co/u/anhlqn)\ **Replies:** 4\ **Last updated:** [April 21, 2016, 3:49pm UTC](https://discuss.elastic.co/t/rename-multiple-fields-in-logstash/48069 "2016-04-21T15:49:16Z")
Hi All, I'm using Logstash 2.3.0 to receive and process Netflow data. Average indexing is about 50K events per minute, but as I put the mutate block to rename fields, throughput drops to 15K per minute. ## Rename al… --- ## [Identifying fields using sprintf](https://discuss.elastic.co/t/identifying-fields-using-sprintf/47941)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 4\ **Last updated:** [April 21, 2016, 2:28pm UTC](https://discuss.elastic.co/t/identifying-fields-using-sprintf/47941 "2016-04-21T14:28:49Z")
I have some JSON data coming to me over HTTP. Each record has a field that needs to be converted to an integer but its name will vary, e.g. "fieldA\_count" in one record and "fieldfB\_count" in another. Each record contain… --- ## [Logstash doesn't work with new data](https://discuss.elastic.co/t/logstash-doesnt-work-with-new-data/48049)
**Author:** [@loez1](https://discuss.elastic.co/u/loez1)\ **Replies:** 0\ **Last updated:** [April 21, 2016, 1:38pm UTC](https://discuss.elastic.co/t/logstash-doesnt-work-with-new-data/48049 "2016-04-21T13:38:38Z")
Hi, I made some change in the content of some fields of elasticsearch ! Concretely this corresponds to change the content of some columns in my CSV document and then integrate the new document (in own sense) in the da… --- ## [Logstash 2.3.1 elasticsearch config not working](https://discuss.elastic.co/t/logstash-2-3-1-elasticsearch-config-not-working/47942)
**Author:** [@mnhan](https://discuss.elastic.co/u/mnhan)\ **Replies:** 1\ **Last updated:** [April 21, 2016, 1:29pm UTC](https://discuss.elastic.co/t/logstash-2-3-1-elasticsearch-config-not-working/47942 "2016-04-21T13:29:53Z")
using logstash 2.2.0-1, the config works output { elasticsearch { hosts =\> \["test-es.corp.com/elasticsearch/"\] ssl =\> true cacert =\> "" index =\> "system-%{YYYY.MM.dd}" } stdout { codec =\> rubydebug } } i… --- ## [How can I change the @timestamp match my timestamp?](https://discuss.elastic.co/t/how-can-i-change-the-timestamp-match-my-timestamp/48005)
**Author:** [@mattcrawfoord](https://discuss.elastic.co/u/mattcrawfoord)\ **Replies:** 2\ **Last updated:** [April 21, 2016, 11:28am UTC](https://discuss.elastic.co/t/how-can-i-change-the-timestamp-match-my-timestamp/48005 "2016-04-21T11:28:30Z")
I found that the @timestamp always show UTC time, i tried to using date filter to match my timezone ("Asia/Shanghai") timestamp as follows, "@timestamp" =\> "2016-04-21T08:39:35.884Z" my date filter , date { … --- ## [Grok loglevel catches 'er'](https://discuss.elastic.co/t/grok-loglevel-catches-er/47974)
**Author:** [@Ezer\_Karash](https://discuss.elastic.co/u/Ezer_Karash)\ **Replies:** 4\ **Last updated:** [April 21, 2016, 7:05am UTC](https://discuss.elastic.co/t/grok-loglevel-catches-er/47974 "2016-04-21T07:05:30Z")
the grok lovel pattern catches 'er' (for example if the word 'number' is in the message)' is there a way to fix that --- ## [How to map a (filebeat) numeric field to display a customer name?](https://discuss.elastic.co/t/how-to-map-a-filebeat-numeric-field-to-display-a-customer-name/47837)
**Author:** [@dgans](https://discuss.elastic.co/u/dgans)\ **Replies:** 5\ **Last updated:** [April 21, 2016, 5:37am UTC](https://discuss.elastic.co/t/how-to-map-a-filebeat-numeric-field-to-display-a-customer-name/47837 "2016-04-21T05:37:40Z")
My filebeat client is sending data records with a numeric field name of 'accountID' to my elasticsearch server. When I create a (Kibana) visualization with this data, my graphs show the numberic 'accountID' values on th… --- ## [Set up log file](https://discuss.elastic.co/t/set-up-log-file/47949)
**Author:** [@rvaedex23](https://discuss.elastic.co/u/rvaedex23)\ **Replies:** 4\ **Last updated:** [April 21, 2016, 5:35am UTC](https://discuss.elastic.co/t/set-up-log-file/47949 "2016-04-21T05:35:45Z")
I have ELK presently running in a DEV environment and I was told to set up two directories so that someone will move logs from a Production environment to DEV to view the logs in Kibana. Where would I set up the paths… --- ## [Logstash -Connect strings after filter](https://discuss.elastic.co/t/logstash-connect-strings-after-filter/47859)
**Author:** [@mattcrawfoord](https://discuss.elastic.co/u/mattcrawfoord)\ **Replies:** 7\ **Last updated:** [April 21, 2016, 5:30am UTC](https://discuss.elastic.co/t/logstash-connect-strings-after-filter/47859 "2016-04-21T05:30:21Z")
I am trying to add a new field to concatenate strings from my log as follow, "message" =\> "#@enterprise=\[1.3.6.1.4.1.9.9.187\], @timestamp=#@value=2612151602\>, @varbind\_list=..., @specific\_trap=7, @source\_ip=\\"1.2.3.4\\"… --- ## [Running Perl Script Exec Input - No Output](https://discuss.elastic.co/t/running-perl-script-exec-input-no-output/47939)
**Author:** [@kprime](https://discuss.elastic.co/u/kprime)\ **Replies:** 5\ **Last updated:** [April 20, 2016, 9:39pm UTC](https://discuss.elastic.co/t/running-perl-script-exec-input-no-output/47939 "2016-04-20T21:39:29Z")
Hello, My goal is to take the output of a Perl script and parse it through logstash. I can run the script successfully from shell and it prints the CSV data that I want to parse and export to elastic search. Example… --- ## [Node logstash join to the ELK cluster](https://discuss.elastic.co/t/node-logstash-join-to-the-elk-cluster/47917)
**Author:** [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Replies:** 2\ **Last updated:** [April 20, 2016, 6:54pm UTC](https://discuss.elastic.co/t/node-logstash-join-to-the-elk-cluster/47917 "2016-04-20T18:54:18Z")
Hello Guys, I'm using the last ELK version and the last Logstash version. I'm checking the ELK cluster with Kopf and Elastic-HQ and with both plugins I can see my 4 data ELK nodes. I created … --- ## [How to get part of the path name and add it to the index](https://discuss.elastic.co/t/how-to-get-part-of-the-path-name-and-add-it-to-the-index/47547)
**Author:** [@subbu.nv](https://discuss.elastic.co/u/subbu.nv)\ **Replies:** 6\ **Last updated:** [April 20, 2016, 6:34pm UTC](https://discuss.elastic.co/t/how-to-get-part-of-the-path-name-and-add-it-to-the-index/47547 "2016-04-20T18:34:11Z")
0 down vote favorite I currently have a file name like this. \[SERIALNUMBER\]\[2014\_12\_04\]\[00\_45\_22\]\[141204T014214\]AB\_DEF.log i basically want to extract the year from the file (2014) and add it to the index name in logst… --- ## [Http\_poller response time of the requested URL](https://discuss.elastic.co/t/http-poller-response-time-of-the-requested-url/47873)
**Author:** [@zhmq0124v](https://discuss.elastic.co/u/zhmq0124v)\ **Replies:** 1\ **Last updated:** [April 20, 2016, 5:19pm UTC](https://discuss.elastic.co/t/http-poller-response-time-of-the-requested-url/47873 "2016-04-20T17:19:35Z")
Dear ELK funs, I am playing with http\_poller plugin, and found runtime\_seconds in the returned metadata, does runtime\_seconds stand for the response time(first byte sent and last byte received) of the requested URL? I … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1162) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1164)