# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1113 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1114 --- ## [NoMethodError thrown in logstash/pipeline.rb using couchdb\_changes on Alpha 5](https://discuss.elastic.co/t/nomethoderror-thrown-in-logstash-pipeline-rb-using-couchdb-changes-on-alpha-5/58410)
**Author:** [@jarederaj](https://discuss.elastic.co/u/jarederaj)\ **Replies:** 1\ **Last updated:** [August 19, 2016, 3:20pm UTC](https://discuss.elastic.co/t/nomethoderror-thrown-in-logstash-pipeline-rb-using-couchdb-changes-on-alpha-5/58410 "2016-08-19T15:20:59Z")
I'm on: Ubuntu 16.04 ELK Alpha 5 release 5 x-pack is installed CouchDB is not secured SSL is not setup Debug output: {:timestamp=\>"2016-08-18T17:25:07.107000-0600", :message=\>"A plugin had an unrecoverable … --- ## [Beginner issue - testing output to stdout](https://discuss.elastic.co/t/beginner-issue-testing-output-to-stdout/58376)
**Author:** [@Bob\_Metelsky](https://discuss.elastic.co/u/Bob_Metelsky)\ **Replies:** 8\ **Last updated:** [August 19, 2016, 12:32pm UTC](https://discuss.elastic.co/t/beginner-issue-testing-output-to-stdout/58376 "2016-08-19T12:32:36Z")
All im basically trying to parse a json file to pick out 2 fields field 1 is name feild 2 is product my ls.conf input { file { type =\> "mefile" path =\> "/export/home/me/me.json" sincedb\_path =\> "… --- ## [\_grokparsfailure on core patterns](https://discuss.elastic.co/t/-grokparsfailure-on-core-patterns/58263)
**Author:** [@m0n0lith](https://discuss.elastic.co/u/m0n0lith)\ **Replies:** 9\ **Last updated:** [August 19, 2016, 12:31pm UTC](https://discuss.elastic.co/t/-grokparsfailure-on-core-patterns/58263 "2016-08-19T12:31:31Z")
Hello! I'm getting weird grok failures after adding my own patterns. And I have checked my patterns they shouldn't even come close to match the standard once in the core patterns. This is for Cisco ASA firewalls and foll… --- ## [Config isn't generating fields](https://discuss.elastic.co/t/config-isnt-generating-fields/58449)
**Author:** [@ou\_snaaksie](https://discuss.elastic.co/u/ou_snaaksie)\ **Replies:** 2\ **Last updated:** [August 19, 2016, 12:20pm UTC](https://discuss.elastic.co/t/config-isnt-generating-fields/58449 "2016-08-19T12:20:47Z")
First off, let me just say that I am new to the Elasticstack. I have 2 different kind of syslog messages I am receiving: \<30\>ddns\[21535\]: Dynamic DNS update for xxx (Duck DNS) successful \<4\>kernel: DNAT IN=red0 OUT= MA… --- ## [\[Solved\] Logstash high CPU and unresponsive](https://discuss.elastic.co/t/solved-logstash-high-cpu-and-unresponsive/51254)
**Author:** [@michbsd](https://discuss.elastic.co/u/michbsd)\ **Replies:** 8\ **Last updated:** [August 19, 2016, 11:47am UTC](https://discuss.elastic.co/t/solved-logstash-high-cpu-and-unresponsive/51254 "2016-08-19T11:47:47Z")
Hi, My platform looks like this: Varnishlog -\> filebeat -\> redis -\> logstash -\> ES logstash (2.3.2) are running on 3 different hw nodes, and ES (cluster) as well. After running for a few minutes (with a clean … --- ## [Whisper count.wsp files are missing](https://discuss.elastic.co/t/whisper-count-wsp-files-are-missing/58442)
**Author:** [@vishwanathh](https://discuss.elastic.co/u/vishwanathh)\ **Replies:** 0\ **Last updated:** [August 19, 2016, 10:07am UTC](https://discuss.elastic.co/t/whisper-count-wsp-files-are-missing/58442 "2016-08-19T10:07:06Z")
Hello, I have a logstash configuration using metrics of the below form; meter =\> \[ "%{component}.%{host}.%{http\_api}.%{http\_status}" \] For example: component,host,http\_api,http\_status from my dataset will be usermanage… --- ## [Updating logstash plugins takes forever](https://discuss.elastic.co/t/updating-logstash-plugins-takes-forever/58438)
**Author:** [@seidler2547](https://discuss.elastic.co/u/seidler2547)\ **Replies:** 0\ **Last updated:** [August 19, 2016, 9:30am UTC](https://discuss.elastic.co/t/updating-logstash-plugins-takes-forever/58438 "2016-08-19T09:30:16Z")
I'm trying to use logstash-plugin update and it does some stuff (showing the plugins, asking for major version upgrades, downloading something) but then it's just stuck at 100% CPU. Is it supposed to take that long. I've… --- ## [Beats\_input\_codec\_json\_applied in tags](https://discuss.elastic.co/t/beats-input-codec-json-applied-in-tags/58427)
**Author:** [@ShaneC-C-Y](https://discuss.elastic.co/u/ShaneC-C-Y)\ **Replies:** 1\ **Last updated:** [August 19, 2016, 7:12am UTC](https://discuss.elastic.co/t/beats-input-codec-json-applied-in-tags/58427 "2016-08-19T07:12:23Z")
This is the same issue here (https://github.com/logstash-plugins/logstash-input-beats/issues/58 ) Every event will be add a tag beats\_input\_codec\_json\_applied if I specify the codec in input config file. Is there any w… --- ## [Logstash LOG\_OPTION](https://discuss.elastic.co/t/logstash-log-option/58178)
**Author:** [@sundar](https://discuss.elastic.co/u/sundar)\ **Replies:** 3\ **Last updated:** [August 19, 2016, 6:18am UTC](https://discuss.elastic.co/t/logstash-log-option/58178 "2016-08-19T06:18:09Z")
Hello Logstash experts, We are using Logstash 2.3 .1 and find in the documentation that there are only two levels of log (from logstash itself): --verbose and --debug. Is there any other level we can set which is und… --- ## [When running Logstash in foreground it works, but logging file it always empty](https://discuss.elastic.co/t/when-running-logstash-in-foreground-it-works-but-logging-file-it-always-empty/57964)
**Author:** [@jamesl](https://discuss.elastic.co/u/jamesl)\ **Replies:** 5\ **Last updated:** [August 19, 2016, 12:19am UTC](https://discuss.elastic.co/t/when-running-logstash-in-foreground-it-works-but-logging-file-it-always-empty/57964 "2016-08-19T00:19:58Z")
Hi. Just this morning after I have come across an unusual problem with Logstash. When it's running as a service 'ps aux | grep logstash' and 'systemctl status logstash' shows as its running but its not listening on any o… --- ## [Tile map not plotting location](https://discuss.elastic.co/t/tile-map-not-plotting-location/58279)
**Author:** [@bishaka](https://discuss.elastic.co/u/bishaka)\ **Replies:** 3\ **Last updated:** [August 18, 2016, 9:50pm UTC](https://discuss.elastic.co/t/tile-map-not-plotting-location/58279 "2016-08-18T21:50:56Z")
Hi There, I am not sure if it's a logstash, elasticsearch or kibana question..but if you could help me out with this..i would greatly appreciate it. I am having trouble displaying locations in the map. I am using Kiba… --- ## [Logstash output to google pub/sub](https://discuss.elastic.co/t/logstash-output-to-google-pub-sub/58402)
**Author:** [@bennett](https://discuss.elastic.co/u/bennett)\ **Replies:** 0\ **Last updated:** [August 18, 2016, 9:18pm UTC](https://discuss.elastic.co/t/logstash-output-to-google-pub-sub/58402 "2016-08-18T21:18:23Z")
Hi! Is there a way for logstash to output to google pub/sub? I saw there was a plugin that someone made in git for input only. Thanks! --- ## [TileMap, GeoIP for RFC1918 addresses](https://discuss.elastic.co/t/tilemap-geoip-for-rfc1918-addresses/58247)
**Author:** [@bastianhoss](https://discuss.elastic.co/u/bastianhoss)\ **Replies:** 11\ **Last updated:** [August 18, 2016, 8:43pm UTC](https://discuss.elastic.co/t/tilemap-geoip-for-rfc1918-addresses/58247 "2016-08-18T20:43:08Z")
Hi Community, looking for an opportunity to locate rfc1918 addresses to a tile map. Non-rfc1918 works fine via GeoIP Database: geoip { source =\> "client\_address" target =\> "geoip" dat… --- ## [How to create a date field target with date {}?](https://discuss.elastic.co/t/how-to-create-a-date-field-target-with-date/58043)
**Author:** [@agonzalez](https://discuss.elastic.co/u/agonzalez)\ **Replies:** 2\ **Last updated:** [August 18, 2016, 6:50pm UTC](https://discuss.elastic.co/t/how-to-create-a-date-field-target-with-date/58043 "2016-08-18T18:50:03Z")
I am parsing my logs date using this filter: date { match =\> \[ "datetime", "YYYY-MM-dd HH:mm:ss.SSS" \] target =\> "datetime" } but the new "datetime" field is created of type s… --- ## [Unable to rename field name with degree (°)symbol](https://discuss.elastic.co/t/unable-to-rename-field-name-with-degree-symbol/58373)
**Author:** [@saisn](https://discuss.elastic.co/u/saisn)\ **Replies:** 2\ **Last updated:** [August 18, 2016, 6:36pm UTC](https://discuss.elastic.co/t/unable-to-rename-field-name-with-degree-symbol/58373 "2016-08-18T18:36:59Z")
I have below config in brief, where i am trying to rename the field name to have a special character (degree symbol) but using it throws an error. "'The following config files contains non-ascii characters but are not … --- ## [Redis input plugin not taking parameter format="json\_event"?](https://discuss.elastic.co/t/redis-input-plugin-not-taking-parameter-format-json-event/58282)
**Author:** [@ravikumar](https://discuss.elastic.co/u/ravikumar)\ **Replies:** 1\ **Last updated:** [August 18, 2016, 5:41pm UTC](https://discuss.elastic.co/t/redis-input-plugin-not-taking-parameter-format-json-event/58282 "2016-08-18T17:41:42Z")
Hi all, I am facing some issues while installing new KEL on my redhat linux 7. java 8 ,elastic serach-2.3.5 , logstash-Version : 2.3.4,kibana -4.5.4 and redis-2.8.19. 1) logstash keep crashing for this i have v… --- ## [Will Grok fail if the grok field does not exist?](https://discuss.elastic.co/t/will-grok-fail-if-the-grok-field-does-not-exist/58300)
**Author:** [@anhlqn](https://discuss.elastic.co/u/anhlqn)\ **Replies:** 1\ **Last updated:** [August 18, 2016, 5:35pm UTC](https://discuss.elastic.co/t/will-grok-fail-if-the-grok-field-does-not-exist/58300 "2016-08-18T17:35:59Z")
Given that I have the following grok filter { grok { match =\> { "my\_abc" =\> "Duration: %{NUMBER:duration}" } } } Will grok fail if the my\_abc field does not exist in the message? Would it be necessary to check for… --- ## [Filter some personalId from accesslogs](https://discuss.elastic.co/t/filter-some-personalid-from-accesslogs/58333)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [August 18, 2016, 5:26pm UTC](https://discuss.elastic.co/t/filter-some-personalid-from-accesslogs/58333 "2016-08-18T17:26:15Z")
Hello, I have some personalIDs in accesslogs that I want to encrypt or mutate in logstash filter. 127.0.0.1 - - \[12/Aug/2016:17:05:07 +0530\] "GET /myService/v2.0/person/\*\*00123456\*\*/ in above logs, I want to check if som… --- ## [How to update to new logstash-filter-mutate plugin version](https://discuss.elastic.co/t/how-to-update-to-new-logstash-filter-mutate-plugin-version/58366)
**Author:** [@tcaudrey](https://discuss.elastic.co/u/tcaudrey)\ **Replies:** 0\ **Last updated:** [August 18, 2016, 3:32pm UTC](https://discuss.elastic.co/t/how-to-update-to-new-logstash-filter-mutate-plugin-version/58366 "2016-08-18T15:32:05Z")
Hello people, A while ago I reported a bug in the logstash-filter-mutate plugin (specifically with the split method), details can be seen here: Now, there is a new version of this filter that is supposed to fix my… --- ## [Logstash gsub: how to replace with backslash?](https://discuss.elastic.co/t/logstash-gsub-how-to-replace-with-backslash/58364)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 0\ **Last updated:** [August 18, 2016, 3:28pm UTC](https://discuss.elastic.co/t/logstash-gsub-how-to-replace-with-backslash/58364 "2016-08-18T15:28:06Z")
Hi, I have following stuff in a field: "field\_A" =\> "c:\\\\test\\\\test.txt" By some conversion the masking backslash became a valid character. How can I get rid of the double backslashes, so that I have only one (or two… --- ## [Need a complete XML Filter example](https://discuss.elastic.co/t/need-a-complete-xml-filter-example/35398)
**Author:** [@Mohammed\_Abdul\_Khali](https://discuss.elastic.co/u/Mohammed_Abdul_Khali)\ **Replies:** 3\ **Last updated:** [August 18, 2016, 3:09pm UTC](https://discuss.elastic.co/t/need-a-complete-xml-filter-example/35398 "2016-08-18T15:09:49Z")
Hi, I am new to Logstash and I have a requirement where i have to read a xml file from my windows machine and index to ES. The XML file looks like the one in the attached image.... Could someone please give me a ref… --- ## [Xpath to parse XML logs with logstash](https://discuss.elastic.co/t/xpath-to-parse-xml-logs-with-logstash/58361)
**Author:** [@Mr.Nice](https://discuss.elastic.co/u/Mr.Nice)\ **Replies:** 0\ **Last updated:** [August 18, 2016, 2:43pm UTC](https://discuss.elastic.co/t/xpath-to-parse-xml-logs-with-logstash/58361 "2016-08-18T14:43:25Z")
hi here ! I'm actually trying to parse some XML logs for my ELK satck, but i don't understand what i did wrong ... My logstash conf file : \> input { \> udp { \> port =\> 5514 \> type =\> "log\_xml" \> … --- ## [Logstash parent-child event configuration](https://discuss.elastic.co/t/logstash-parent-child-event-configuration/58117)
**Author:** [@aqiao](https://discuss.elastic.co/u/aqiao)\ **Replies:** 1\ **Last updated:** [August 18, 2016, 2:36pm UTC](https://discuss.elastic.co/t/logstash-parent-child-event-configuration/58117 "2016-08-18T14:36:33Z")
2016-08-16 14:00:14,655 ABC BETA P1 DecodeFieldList Refresh \<6\>,TRDPRC\_1,Price, \<15\>,CURRENCY,String,0 \<22\>,BID,Price, \<23\>,BID\_1,Price, \<24\>,BID\_2,Price, \<25\>,ASK,Price, … --- ## [Logstash kubernetes container crashing](https://discuss.elastic.co/t/logstash-kubernetes-container-crashing/58346)
**Author:** [@Gary\_D](https://discuss.elastic.co/u/Gary_D)\ **Replies:** 2\ **Last updated:** [August 18, 2016, 2:50pm UTC](https://discuss.elastic.co/t/logstash-kubernetes-container-crashing/58346 "2016-08-18T14:50:16Z")
Any tips? It starts and goes into a crash loopback, very little info shown The most I can get from a kubectl logs is: {:timestamp=\>"2016-08-18T12:46:14.936000+0000", :message=\>"Pipeline main started"} {:timestamp=\>"20… --- ## [Split string from one field from input to three or mutiple in output](https://discuss.elastic.co/t/split-string-from-one-field-from-input-to-three-or-mutiple-in-output/58343)
**Author:** [@lnx](https://discuss.elastic.co/u/lnx)\ **Replies:** 1\ **Last updated:** [August 18, 2016, 2:08pm UTC](https://discuss.elastic.co/t/split-string-from-one-field-from-input-to-three-or-mutiple-in-output/58343 "2016-08-18T14:08:58Z")
I'm battling to achieve something which in my mind should be simple. I've got a string in a field from the input plugin like this: bigfield:"something - something1 - something2" I need to grok or mutate it or somethi… --- ## [Nginx logs without year and seconds information](https://discuss.elastic.co/t/nginx-logs-without-year-and-seconds-information/57780)
**Author:** [@vvs14](https://discuss.elastic.co/u/vvs14)\ **Replies:** 5\ **Last updated:** [August 18, 2016, 6:39am UTC](https://discuss.elastic.co/t/nginx-logs-without-year-and-seconds-information/57780 "2016-08-18T06:39:51Z")
I have Nginx as loadbalancer which is generating logs without year and second information in timestamp. One of those logs are 08-10 09:28 root ERROR Error connecting to CRS REST API : \[Errno 111\] Connection r… --- ## [Creating geoip database](https://discuss.elastic.co/t/creating-geoip-database/58255)
**Author:** [@James\_Kazie](https://discuss.elastic.co/u/James_Kazie)\ **Replies:** 2\ **Last updated:** [August 18, 2016, 10:24am UTC](https://discuss.elastic.co/t/creating-geoip-database/58255 "2016-08-18T10:24:20Z")
I have a database with IP addresses, names of places and post codes. Is it possible to create my own database to use for the geoip plugin? and if so what format or file types are necessary for it to work. --- ## [How to import data to elasticsearch?](https://discuss.elastic.co/t/how-to-import-data-to-elasticsearch/58100)
**Author:** [@laoyang360](https://discuss.elastic.co/u/laoyang360)\ **Replies:** 7\ **Last updated:** [August 18, 2016, 9:40am UTC](https://discuss.elastic.co/t/how-to-import-data-to-elasticsearch/58100 "2016-08-18T09:40:29Z")
Hi, All I want to know how to import data to elasticsearch? Now I have the follow sence: 1, the data size is very small, eg: the import date size is 10MB/s; 2, the data size is very big, eg: the import data size is 100TB… --- ## [Logstash-filter-elasticsearch plugin - specify index to search](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-specify-index-to-search/58326)
**Author:** [@Shirly\_Radco](https://discuss.elastic.co/u/Shirly_Radco)\ **Replies:** 1\ **Last updated:** [August 18, 2016, 9:35am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-specify-index-to-search/58326 "2016-08-18T09:35:43Z")
Hi, I'm using for the first time the logstash-filter-elasticsearch plugin version 2.1.1. How can I specify which index to query? What is the default for this? searching all indexes or only the current index? --- ## [Json filter question](https://discuss.elastic.co/t/json-filter-question/58249)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 2\ **Last updated:** [August 18, 2016, 9:23am UTC](https://discuss.elastic.co/t/json-filter-question/58249 "2016-08-18T09:23:46Z")
Hi, Apologies if this has been asked before. I am using the file input with no codec specified. I'm just reading some logs. Some of the messages are json but unfortunately also contain some pre and post text wrappin… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1112) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1114)