# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1252 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1253 --- ## [Add field in logstash config](https://discuss.elastic.co/t/add-field-in-logstash-config/25513)
**Author:** [@Smasell](https://discuss.elastic.co/u/Smasell)\ **Replies:** 5\ **Last updated:** [July 14, 2015, 12:37pm UTC](https://discuss.elastic.co/t/add-field-in-logstash-config/25513 "2015-07-14T12:37:13Z")
Hi!!! I have logs like this: GetLoginGata: login: 'G:59423457'; deviceId: 'FF7567DE-8822-4D0F-9E2E-651A202B6B58' And I want to get fields with grok filter in my config: grok { match =\> \[ "message", " login: %{WORD:… --- ## [How could I build jdbc input plugin?](https://discuss.elastic.co/t/how-could-i-build-jdbc-input-plugin/1099)
**Author:** [@Ducheol\_Kim](https://discuss.elastic.co/u/Ducheol_Kim)\ **Replies:** 3\ **Last updated:** [July 14, 2015, 8:23am UTC](https://discuss.elastic.co/t/how-could-i-build-jdbc-input-plugin/1099 "2015-07-14T08:23:06Z")
Hi All. I try to build jdbc plugin which is mentioned below. I follow the instruction in the https://www.elastic.co/guide/en/logstash/current/\_how\_to\_write\_a\_logstash\_input\_plugin.html and I got below error. $ bundl… --- ## [F5 logs not processing real time](https://discuss.elastic.co/t/f5-logs-not-processing-real-time/25044)
**Author:** [@Crista\_Shawler](https://discuss.elastic.co/u/Crista_Shawler)\ **Replies:** 7\ **Last updated:** [July 14, 2015, 5:40am UTC](https://discuss.elastic.co/t/f5-logs-not-processing-real-time/25044 "2015-07-14T05:40:41Z")
We are shipping all of our F5 logs to Redis and then into Elasticsearch but they seem to be getting backed up. We are seeing a delay of sometimes 4 hours or more. They continue to process and it seems to catchup over n… --- ## [Raw fields missing](https://discuss.elastic.co/t/raw-fields-missing/25408)
**Author:** [@Noodles](https://discuss.elastic.co/u/Noodles)\ **Replies:** 2\ **Last updated:** [July 14, 2015, 5:28am UTC](https://discuss.elastic.co/t/raw-fields-missing/25408 "2015-07-14T05:28:00Z")
I'm trying to use a separate index for my apache logs: elasticsearch { host =\> localhost index =\> "http-%{+YYYY.MM.dd}" } It doesn't look like the .raw fields are being created in ES, but as far as I can see it is usi… --- ## [Output Logs In Original Form](https://discuss.elastic.co/t/output-logs-in-original-form/25349)
**Author:** [@Ernest\_Mueller](https://discuss.elastic.co/u/Ernest_Mueller)\ **Replies:** 2\ **Last updated:** [July 13, 2015, 8:14pm UTC](https://discuss.elastic.co/t/output-logs-in-original-form/25349 "2015-07-13T20:14:36Z")
Hey all, I'm a new logstash user. I'm trying to ease in a client to ELK who is currently analyzing their logs using an existing tool. I wanted to start by just using logstash for log shipping and aggregating them to on… --- ## [Getting error while executing logstash conf file](https://discuss.elastic.co/t/getting-error-while-executing-logstash-conf-file/25422)
**Author:** [@Ashok](https://discuss.elastic.co/u/Ashok)\ **Replies:** 5\ **Last updated:** [July 13, 2015, 10:24am UTC](https://discuss.elastic.co/t/getting-error-while-executing-logstash-conf-file/25422 "2015-07-13T10:24:27Z")
Hi Team, Getting folloowing error while executing logstash conf file. ←\[31mGot error to send bulk of actions: blocked by: \[SERVICE\_UNAVAILABLE/1/state not recovered / initialized\];\[SERVICE\_UNAVAILABLE/2/no master\]; {:lev… --- ## [Will logstash duplicate already indexed data in elasticsearch?](https://discuss.elastic.co/t/will-logstash-duplicate-already-indexed-data-in-elasticsearch/25394)
**Author:** [@bhutchinson](https://discuss.elastic.co/u/bhutchinson)\ **Replies:** 1\ **Last updated:** [July 12, 2015, 7:31pm UTC](https://discuss.elastic.co/t/will-logstash-duplicate-already-indexed-data-in-elasticsearch/25394 "2015-07-12T19:31:47Z")
Versions Windows 7 elasticsearch v1.6.0 logstash v1.5.2 kibana v4.1.0 Background Right now I am just working on a proof of concept locally and am not using a Logstash Forwarder. I am manually running logstash from the… --- ## [Why does logstash not import file into elasticsearch?](https://discuss.elastic.co/t/why-does-logstash-not-import-file-into-elasticsearch/25397)
**Author:** [@bhutchinson](https://discuss.elastic.co/u/bhutchinson)\ **Replies:** 0\ **Last updated:** [July 12, 2015, 5:56pm UTC](https://discuss.elastic.co/t/why-does-logstash-not-import-file-into-elasticsearch/25397 "2015-07-12T17:56:31Z")
Versions Windows 7 elasticsearch v1.6.0 logstash v1.5.2 kibana v4.1.0 Background I am trying to import an Apache Access Log file into elasticsearch. I am finding that sometimes logstash will work as desired, other tim… --- ## [Need help on LogStash encryption](https://discuss.elastic.co/t/need-help-on-logstash-encryption/25341)
**Author:** [@Giri\_Sgs](https://discuss.elastic.co/u/Giri_Sgs)\ **Replies:** 3\ **Last updated:** [July 12, 2015, 8:29am UTC](https://discuss.elastic.co/t/need-help-on-logstash-encryption/25341 "2015-07-12T08:29:43Z")
Hi Friends, Is there any way to read public keys from a keyserver & encrypt messages in logstash. I am not getting how to read key form keyserver.... any suggestions please ...... Thanks --- ## [Grok pattern or stanza for parsing elasticsearch slow logs?](https://discuss.elastic.co/t/grok-pattern-or-stanza-for-parsing-elasticsearch-slow-logs/24638)
**Author:** [@GordonM](https://discuss.elastic.co/u/GordonM)\ **Replies:** 3\ **Last updated:** [July 11, 2015, 5:35pm UTC](https://discuss.elastic.co/t/grok-pattern-or-stanza-for-parsing-elasticsearch-slow-logs/24638 "2015-07-11T17:35:27Z")
Any existing grok patterns (or existing grok stanza) for elasticsearch logs? In particular the slow logs? (index\_indexing and index\_search)? These are the logs located in a typical install in /var/log/elasticsearch T… --- ## [Logstash Kafka Output plugins not working on Windows](https://discuss.elastic.co/t/logstash-kafka-output-plugins-not-working-on-windows/25253)
**Author:** [@zhoufwind](https://discuss.elastic.co/u/zhoufwind)\ **Replies:** 2\ **Last updated:** [July 11, 2015, 6:17am UTC](https://discuss.elastic.co/t/logstash-kafka-output-plugins-not-working-on-windows/25253 "2015-07-11T06:17:00Z")
Hi folks, Kafka output plugin is working perfectly on Linux but it seems not working correctly on Windows, not sure how to debug it. When I make test on Linux, it's working fine: Linux(CentOS6.5/ jdk-8u45/ kafka\_2.10… --- ## [\[Need Help\] Multiple pattern in one log file](https://discuss.elastic.co/t/need-help-multiple-pattern-in-one-log-file/25357)
**Author:** [@SiegeLion](https://discuss.elastic.co/u/SiegeLion)\ **Replies:** 4\ **Last updated:** [July 11, 2015, 5:50am UTC](https://discuss.elastic.co/t/need-help-multiple-pattern-in-one-log-file/25357 "2015-07-11T05:50:36Z")
I am trying to send xtreemfs log files to logstash and parse them into elastic search A problem with xtreemfs log files is that they begin with some thing like ########### Configuration ########### # Stuff # More Stu… --- ## [GC seems to be not working](https://discuss.elastic.co/t/gc-seems-to-be-not-working/25305)
**Author:** [@Fodoj](https://discuss.elastic.co/u/Fodoj)\ **Replies:** 2\ **Last updated:** [July 10, 2015, 9:14pm UTC](https://discuss.elastic.co/t/gc-seems-to-be-not-working/25305 "2015-07-10T21:14:33Z")
We are meeting serious issues with rather simple Logstash setup and averagely low load (20k requests per minute). The problem is that after Logstash instance is started consumption of memory slowly grows over two-three d… --- ## [Only invoke geoip if field is IP](https://discuss.elastic.co/t/only-invoke-geoip-if-field-is-ip/25317)
**Author:** [@Khoa\_Nguyen1](https://discuss.elastic.co/u/Khoa_Nguyen1)\ **Replies:** 3\ **Last updated:** [July 10, 2015, 7:36pm UTC](https://discuss.elastic.co/t/only-invoke-geoip-if-field-is-ip/25317 "2015-07-10T19:36:53Z")
How do I conditionally invoke geoip if the field is a valid IP address? --- ## [Getting Previous field values](https://discuss.elastic.co/t/getting-previous-field-values/2663)
**Author:** [@Siddhant\_Agarwal](https://discuss.elastic.co/u/Siddhant_Agarwal)\ **Replies:** 1\ **Last updated:** [July 10, 2015, 5:48pm UTC](https://discuss.elastic.co/t/getting-previous-field-values/2663 "2015-07-10T17:48:23Z")
Is there a way in which i can access just previous field value(as appeared in log file) and apply arithmetic operation with current field value of same field in any one of log stash, elastic search or kibana? --- ## [Corosync + pacemaker + logstash](https://discuss.elastic.co/t/corosync-pacemaker-logstash/25055)
**Author:** [@saif](https://discuss.elastic.co/u/saif)\ **Replies:** 4\ **Last updated:** [July 10, 2015, 1:00pm UTC](https://discuss.elastic.co/t/corosync-pacemaker-logstash/25055 "2015-07-10T13:00:23Z")
hello I wanted to know if I can use corosync and pacemaker with logstash if not are there any other solution to do a cluster between 2 instance of logstash thank you --- ## ["\_grokparsefailure"](https://discuss.elastic.co/t/-grokparsefailure/24786)
**Author:** [@ABecker](https://discuss.elastic.co/u/ABecker)\ **Replies:** 4\ **Last updated:** [July 10, 2015, 11:09am UTC](https://discuss.elastic.co/t/-grokparsefailure/24786 "2015-07-10T11:09:12Z")
Hello, I have a problem with one of my grok-filters. I know why this problem occur, but I don't know how I can fix it. The grokparsefailure occur because of the smaller "\<" and greater "\>" characters within the mess… --- ## [How to handle german Umlauts or anything thats not UTF-8](https://discuss.elastic.co/t/how-to-handle-german-umlauts-or-anything-thats-not-utf-8/25294)
**Author:** [@roflrox](https://discuss.elastic.co/u/roflrox)\ **Replies:** 3\ **Last updated:** [July 10, 2015, 9:21am UTC](https://discuss.elastic.co/t/how-to-handle-german-umlauts-or-anything-thats-not-utf-8/25294 "2015-07-10T09:21:33Z")
Hallo, i had a little Problem with processing Logmessages that contain some german umlauts. But i found a solution how to handle umlauts (or anything which is not UTF-8 encoded) When you check the logstash.log you mi… --- ## [UTF-8 issue when patterns are loading](https://discuss.elastic.co/t/utf-8-issue-when-patterns-are-loading/25256)
**Author:** [@incogniro](https://discuss.elastic.co/u/incogniro)\ **Replies:** 1\ **Last updated:** [July 10, 2015, 8:34am UTC](https://discuss.elastic.co/t/utf-8-issue-when-patterns-are-loading/25256 "2015-07-10T08:34:37Z")
The pattern files which are read have not been changed in some time. Here is the end of the debug output: {:timestamp=\>"2015-07-09T18:52:30.171000+0100", :message=\>"Adding pattern", "}"=\>"", :level=\>:info, :file=\>"/Pro… --- ## [How to change the data type in Logstash](https://discuss.elastic.co/t/how-to-change-the-data-type-in-logstash/25255)
**Author:** [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Replies:** 7\ **Last updated:** [July 10, 2015, 7:55am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-in-logstash/25255 "2015-07-10T07:55:21Z")
Hi, I am indexing the CSV file using Logstash, default it index every field as string in elasticsearch. I want to change some field to Date type ,how can i do it in logstash ? --- ## [Logs send to Logstash Forwarder in http post request](https://discuss.elastic.co/t/logs-send-to-logstash-forwarder-in-http-post-request/25249)
**Author:** [@marjolaine](https://discuss.elastic.co/u/marjolaine)\ **Replies:** 3\ **Last updated:** [July 10, 2015, 7:15am UTC](https://discuss.elastic.co/t/logs-send-to-logstash-forwarder-in-http-post-request/25249 "2015-07-10T07:15:04Z")
Hello, I use Logstash Forwarder to collect logs in files, but I would like to know if it's possible to collecte log which are sending in http post on the port of Logstash Forwarder ? In fact, I have some logs that I … --- ## [Error Connecting Logstash to Found Elasticsearch](https://discuss.elastic.co/t/error-connecting-logstash-to-found-elasticsearch/25207)
**Author:** [@ScottLawrenceLawson](https://discuss.elastic.co/u/ScottLawrenceLawson)\ **Replies:** 8\ **Last updated:** [July 10, 2015, 5:13am UTC](https://discuss.elastic.co/t/error-connecting-logstash-to-found-elasticsearch/25207 "2015-07-10T05:13:07Z")
I am a total newbie to ELK stack and am following along with this tutorial (https://www.found.no/foundation/analyzing-weblogs-with-elasticsearch/) on my Win7 Laptop. I installed all bits and got LS working with the basic… --- ## [Recommended ES version for Logstash? Docs link broken](https://discuss.elastic.co/t/recommended-es-version-for-logstash-docs-link-broken/25143)
**Author:** [@flup](https://discuss.elastic.co/u/flup)\ **Replies:** 3\ **Last updated:** [July 10, 2015, 4:19am UTC](https://discuss.elastic.co/t/recommended-es-version-for-logstash-docs-link-broken/25143 "2015-07-10T04:19:50Z")
Hi, Logstash used to specify a recommended elasticsearch version to use, but the current docs have a link that's broken. Is there still a recommended ES version for each version of Logstash and, if so, where can I fi… --- ## [Grok tests pass, logstash still borks with "\_grokparsefailure"](https://discuss.elastic.co/t/grok-tests-pass-logstash-still-borks-with--grokparsefailure/25155)
**Author:** [@nseery](https://discuss.elastic.co/u/nseery)\ **Replies:** 6\ **Last updated:** [July 10, 2015, 2:15am UTC](https://discuss.elastic.co/t/grok-tests-pass-logstash-still-borks-with--grokparsefailure/25155 "2015-07-10T02:15:51Z")
Hello all, working on parsing firewall logs and I have it working at http://grokconstructor.appspot.com/ When I put the same pattern in Logstash, it comes back with a lovely "\_grokparsefailure" log line: 2015-07-08T… --- ## [A large amount of Logstash inputs](https://discuss.elastic.co/t/a-large-amount-of-logstash-inputs/25163)
**Author:** [@ChiBiPonD](https://discuss.elastic.co/u/ChiBiPonD)\ **Replies:** 10\ **Last updated:** [July 10, 2015, 1:01am UTC](https://discuss.elastic.co/t/a-large-amount-of-logstash-inputs/25163 "2015-07-10T01:01:47Z")
I would like to achieve 30k inputs per machine to ElasticSearch. I have set up 5 ElasticSearch master nodes. As I want to get inputs from all 20 machines at 600K rate, I need to get 30k inputs per machine. Any suggestio… --- ## [Grok Parse failure but passing configtest](https://discuss.elastic.co/t/grok-parse-failure-but-passing-configtest/25097)
**Author:** [@Paul\_McClure](https://discuss.elastic.co/u/Paul_McClure)\ **Replies:** 5\ **Last updated:** [July 9, 2015, 9:03pm UTC](https://discuss.elastic.co/t/grok-parse-failure-but-passing-configtest/25097 "2015-07-09T21:03:59Z")
Hey first post here! I'm getting a grok parse failure in my tags field: tags netsyslog, \_grokparsefailure I'm really not sure where I've gone wrong though. The config seems fine and passes the configtest. Fields … --- ## [LogStash Stops after a few hours](https://discuss.elastic.co/t/logstash-stops-after-a-few-hours/2249)
**Author:** [@dirtdiver512](https://discuss.elastic.co/u/dirtdiver512)\ **Replies:** 17\ **Last updated:** [July 9, 2015, 6:40pm UTC](https://discuss.elastic.co/t/logstash-stops-after-a-few-hours/2249 "2015-07-09T18:40:10Z")
All, I have a few devices sending logs to ELK. All are sending based on syslog going to specific ports and being indexed based on the port it goes to. All of it so far is just networking gear such as juniper and cisc… --- ## [S3 Output Plugin Codec Issue](https://discuss.elastic.co/t/s3-output-plugin-codec-issue/25237)
**Author:** [@bamundson](https://discuss.elastic.co/u/bamundson)\ **Replies:** 0\ **Last updated:** [July 9, 2015, 1:32pm UTC](https://discuss.elastic.co/t/s3-output-plugin-codec-issue/25237 "2015-07-09T13:32:39Z")
I am experiencing some strange behavior with logstash, using a combination of codecs on a input/file and an output/s3. There seems so be an issue with the output/s3 logstash plugin, as I cannot get any part files to uplo… --- ## [Handling queue ids vs message ids](https://discuss.elastic.co/t/handling-queue-ids-vs-message-ids/24824)
**Author:** [@HoppeT](https://discuss.elastic.co/u/HoppeT)\ **Replies:** 9\ **Last updated:** [July 9, 2015, 12:35pm UTC](https://discuss.elastic.co/t/handling-queue-ids-vs-message-ids/24824 "2015-07-09T12:35:10Z")
hey mates, maybe some of you could help me with a problem i'm acutally trying to solve. we're using logstash in combination with logstash-forwarder and elasticsearch to store the data. we're trying to analyze our … --- ## [Import data from AMAZON](https://discuss.elastic.co/t/import-data-from-amazon/25229)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [July 9, 2015, 12:15pm UTC](https://discuss.elastic.co/t/import-data-from-amazon/25229 "2015-07-09T12:15:36Z")
Hi All, How can i import data from amazon to Elastic search. Please give me step by step procedure.. Thanks, Dan [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1251) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1253)