# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=548 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 549 --- ## [How to parse elasticsearch filter aggregated\_field value (hash type)? \_](https://discuss.elastic.co/t/how-to-parse-elasticsearch-filter-aggregated-field-value-hash-type/180529)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 3\ **Last updated:** [May 15, 2019, 3:09am UTC](https://discuss.elastic.co/t/how-to-parse-elasticsearch-filter-aggregated-field-value-hash-type/180529 "2019-05-15T03:09:17Z")
We have the ff scenario: Goal : Perform join in two es index at indexing level Solution: input { exec { codec =\> json interval =\> 60 comm… --- ## [Work on logstash 6.4.2 but not with 7.0.0](https://discuss.elastic.co/t/work-on-logstash-6-4-2-but-not-with-7-0-0/180916)
**Author:** [@heweixi](https://discuss.elastic.co/u/heweixi)\ **Replies:** 1\ **Last updated:** [May 14, 2019, 1:43pm UTC](https://discuss.elastic.co/t/work-on-logstash-6-4-2-but-not-with-7-0-0/180916 "2019-05-14T13:43:36Z")
on filter ,want to split and get a element from array but is not work on 7.0 and work with 6.4,6.5,code like: mutate { split =\> \["message", " | "\] add\_field =\> { "ip" =\> "%{message\[0\]}" } } data as: elem1 | elem2… --- ## [Grok problem and logstash](https://discuss.elastic.co/t/grok-problem-and-logstash/181081)
**Author:** [@franco.federico](https://discuss.elastic.co/u/franco.federico)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 11:44pm UTC](https://discuss.elastic.co/t/grok-problem-and-logstash/181081 "2019-05-14T23:44:03Z")
I have an old version of logstash exactly1.5.6 I have made a grok filter and I created it by http://grokdebug.herokuapp.com/ For grokdebug it is ok and I have my field. Here the example of grok filter and example of li… --- ## ["Undefined method \`gsub!'" when executes a Ruby filter to change timezone](https://discuss.elastic.co/t/undefined-method-gsub-when-executes-a-ruby-filter-to-change-timezone/180420)
**Author:** [@marcogarfias](https://discuss.elastic.co/u/marcogarfias)\ **Replies:** 4\ **Last updated:** [May 14, 2019, 9:04pm UTC](https://discuss.elastic.co/t/undefined-method-gsub-when-executes-a-ruby-filter-to-change-timezone/180420 "2019-05-14T21:04:54Z")
Hi, I have an issue when I tried to parse a date string to a date fiel with a different timezone, below the info --- Filter configuration input { stdin { } } filter { date { match =\> \["message", "yyyy-MM-dd HH… --- ## [Elasticsearch filter plugin not sorting properly](https://discuss.elastic.co/t/elasticsearch-filter-plugin-not-sorting-properly/181045)
**Author:** [@Kagerou](https://discuss.elastic.co/u/Kagerou)\ **Replies:** 1\ **Last updated:** [May 14, 2019, 6:41pm UTC](https://discuss.elastic.co/t/elasticsearch-filter-plugin-not-sorting-properly/181045 "2019-05-14T18:41:14Z")
I'm currently using the elasticsearch filter plugin to search for documents I have in elasticsearch. a snippet of my config file: query =\> "messageContent:offline, alert AND senderID:%{\[senderID\]}" sort =\> "@timestam… --- ## [Ruby: no implicit conversion of String into Integer](https://discuss.elastic.co/t/ruby-no-implicit-conversion-of-string-into-integer/181054)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [May 14, 2019, 5:58pm UTC](https://discuss.elastic.co/t/ruby-no-implicit-conversion-of-string-into-integer/181054 "2019-05-14T17:58:41Z")
This is my code First I am changing them to integer then if they are nil assign them 0 and then do calculation. mutate { convert =\> { "killo" =\> "integer" } convert =\> { "trace" =\> "integer" } } if !\[… --- ## [Logstash 2.3.x how to load nested objects in elasticsearch index using logstash-jdbc plugin](https://discuss.elastic.co/t/logstash-2-3-x-how-to-load-nested-objects-in-elasticsearch-index-using-logstash-jdbc-plugin/180988)
**Author:** [@vinay\_rawat](https://discuss.elastic.co/u/vinay_rawat)\ **Replies:** 4\ **Last updated:** [May 14, 2019, 5:38pm UTC](https://discuss.elastic.co/t/logstash-2-3-x-how-to-load-nested-objects-in-elasticsearch-index-using-logstash-jdbc-plugin/180988 "2019-05-14T17:38:25Z")
Hi, I am currently using elasticsearch 2.3.4 and logstash 2.3.4 I am trying to load relational data from Oracle db into my elasticsearch index using jdbc plugin. As suggested in various posts, I am using aggregate filt… --- ## [Capture date and time from a sentence](https://discuss.elastic.co/t/capture-date-and-time-from-a-sentence/181017)
**Author:** [@Abhilash\_B](https://discuss.elastic.co/u/Abhilash_B)\ **Replies:** 3\ **Last updated:** [May 14, 2019, 5:35pm UTC](https://discuss.elastic.co/t/capture-date-and-time-from-a-sentence/181017 "2019-05-14T17:35:03Z")
Hi, I have this sentence, The deployment started 3/19/2019 at 2:05 as a "message" to logstash How do I capture the date and time in the above sentence? Also, how should I check to allow this sentence to be parsed and o… --- ## [How to set up mutual SSL authentication between an ELK server (Logstash) and a remote Filebeat server?](https://discuss.elastic.co/t/how-to-set-up-mutual-ssl-authentication-between-an-elk-server-logstash-and-a-remote-filebeat-server/180456)
**Author:** [@zenimagine](https://discuss.elastic.co/u/zenimagine)\ **Replies:** 12\ **Last updated:** [May 14, 2019, 4:14pm UTC](https://discuss.elastic.co/t/how-to-set-up-mutual-ssl-authentication-between-an-elk-server-logstash-and-a-remote-filebeat-server/180456 "2019-05-14T16:14:38Z")
I have 2 servers with Ubuntu 18.04: monitoring.example.com (with ELK on a single server) www.example.com (with Filebeat) Here is the configuration file /etc/logstash/conf.d/logstash.conf on the server monitoring.exa… --- ## [Kafka Output - Node Failover](https://discuss.elastic.co/t/kafka-output-node-failover/181038)
**Author:** [@oats\_gal](https://discuss.elastic.co/u/oats_gal)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 4:13pm UTC](https://discuss.elastic.co/t/kafka-output-node-failover/181038 "2019-05-14T16:13:05Z")
I've seen a few similar notes on this topic, but haven't had any success in finding a solution as of yet. We have Logstash 6.6.0 processes that run on all of our VMs and send their logs to Kafka, specifically a Kafka VI… --- ## [Plugin Offline Install (influxdb-output)](https://discuss.elastic.co/t/plugin-offline-install-influxdb-output/181014)
**Author:** [@Lebedev](https://discuss.elastic.co/u/Lebedev)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 1:52pm UTC](https://discuss.elastic.co/t/plugin-offline-install-influxdb-output/181014 "2019-05-14T13:52:53Z")
Dear, collegues, hello! I try to install (offline) influxdb-otput-plugin. I downloaded it from github (https://github.com/logstash-plugins/logstash-output-influxdb). I started install: logstash-plugin install file:///C… --- ## [Facing issues on translate filter](https://discuss.elastic.co/t/facing-issues-on-translate-filter/180808)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 13\ **Last updated:** [May 14, 2019, 3:05pm UTC](https://discuss.elastic.co/t/facing-issues-on-translate-filter/180808 "2019-05-14T15:05:58Z")
HI Team, I'm using translate filter to match value with yaml, but those translate is doesnt happen, Conf: translate { field =\> "dcn\_id" destination =\> "restricted\_data" override =\> true dictionary\_path =\> "C:\\Gane… --- ## [Error parsing file from filebeat](https://discuss.elastic.co/t/error-parsing-file-from-filebeat/181015)
**Author:** [@hHelen](https://discuss.elastic.co/u/hHelen)\ **Replies:** 1\ **Last updated:** [May 14, 2019, 2:26pm UTC](https://discuss.elastic.co/t/error-parsing-file-from-filebeat/181015 "2019-05-14T14:26:20Z")
I have 2 web servers already running filebeats - everything works as expected, I can see the indices. I now added another web server, making it three. The issue is that I configured the filebeat excatly as the previous s… --- ## [Can we define tags using \`logstash-output-statsd\`?](https://discuss.elastic.co/t/can-we-define-tags-using-logstash-output-statsd/180928)
**Author:** [@YvanZhong](https://discuss.elastic.co/u/YvanZhong)\ **Replies:** 3\ **Last updated:** [May 14, 2019, 2:07pm UTC](https://discuss.elastic.co/t/can-we-define-tags-using-logstash-output-statsd/180928 "2019-05-14T14:07:52Z")
We want to define Datadog tags in statsd output plugin. Just like described in this doc: https://docs.datadoghq.com/tagging/ Can we define a tag such as status:200 --- ## [Custom file delimiter](https://discuss.elastic.co/t/custom-file-delimiter/180957)
**Author:** [@tyb1222](https://discuss.elastic.co/u/tyb1222)\ **Replies:** 1\ **Last updated:** [May 14, 2019, 1:38pm UTC](https://discuss.elastic.co/t/custom-file-delimiter/180957 "2019-05-14T13:38:33Z")
I wrote my conf file like this: input{ file{ path =\> "/data/logs/2" start\_position =\> "beginning" delimiter =\> "\\n" } } but it didn't work .But when I removed 'delimiter ',It workes well. I should not configred … --- ## [Logstash 7.0.1 not work line too large](https://discuss.elastic.co/t/logstash-7-0-1-not-work-line-too-large/180854)
**Author:** [@tavellahh](https://discuss.elastic.co/u/tavellahh)\ **Replies:** 3\ **Last updated:** [May 14, 2019, 1:38pm UTC](https://discuss.elastic.co/t/logstash-7-0-1-not-work-line-too-large/180854 "2019-05-14T13:38:22Z")
Hi, I am having problems when I try to read a log file, which has lines of more than 8000 characters in length each, is it possible that logstash has a limitation about this, or do I have to configure something to work c… --- ## [Undefined method \`each' for nil:NilClass](https://discuss.elastic.co/t/undefined-method-each-for-nil-nilclass/180976)
**Author:** [@souf\_el\_badraoui](https://discuss.elastic.co/u/souf_el_badraoui)\ **Replies:** 1\ **Last updated:** [May 14, 2019, 12:32pm UTC](https://discuss.elastic.co/t/undefined-method-each-for-nil-nilclass/180976 "2019-05-14T12:32:16Z")
Hi, We are trying to flatten a json array with the following ruby code in Logstash ruby { init =\> '@ignore = \[ "@metadata", "@version" , "beat", "type", "tags"\]' code =\> ' event.get("\[EventData\]").each { | k, v… --- ## [EXCEPTION\_ACCESS\_VIOLATION Problematic frame: # C \[racob-x64.dll+0x1aa4\]](https://discuss.elastic.co/t/exception-access-violation-problematic-frame-c-racob-x64-dll-0x1aa4/180971)
**Author:** [@outsidecave](https://discuss.elastic.co/u/outsidecave)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 9:41am UTC](https://discuss.elastic.co/t/exception-access-violation-problematic-frame-c-racob-x64-dll-0x1aa4/180971 "2019-05-14T09:41:39Z")
Hi, I'm not sure where is best to ask this question, please advise if this isn't the place. After swapping from Oracle JDK8 to AdoptOpenJDK 8 I get a fatal error EXCEPTION\_ACCESS\_VIOLATION Problematic frame: C \[raco… --- ## [Extract substring from field](https://discuss.elastic.co/t/extract-substring-from-field/180910)
**Author:** [@Jan\_Kaspar](https://discuss.elastic.co/u/Jan_Kaspar)\ **Replies:** 3\ **Last updated:** [May 14, 2019, 8:40am UTC](https://discuss.elastic.co/t/extract-substring-from-field/180910 "2019-05-14T08:40:13Z")
Hello, I have typical scenatio. I have field that contains distinguished name. I need to extract common name and copy it to new field. I have working regex but i really dont know how to use it in match. Existing field … --- ## [Upgrading Logstash from 6.0.0 to 6.7.2 second type being passed](https://discuss.elastic.co/t/upgrading-logstash-from-6-0-0-to-6-7-2-second-type-being-passed/180677)
**Author:** [@cowensel](https://discuss.elastic.co/u/cowensel)\ **Replies:** 3\ **Last updated:** [May 14, 2019, 6:52am UTC](https://discuss.elastic.co/t/upgrading-logstash-from-6-0-0-to-6-7-2-second-type-being-passed/180677 "2019-05-14T06:52:25Z")
I have upgraded to 6.7.2 from 6.7.2 of logstash where I am now receiving the following error Rejecting mapping update to \[logstash-2019.05.11\] as the final mapping would have more than 1 type: \[log4net, doc\]" the confi… --- ## [Created a new config file, no errors but not loaded in Kibana](https://discuss.elastic.co/t/created-a-new-config-file-no-errors-but-not-loaded-in-kibana/180796)
**Author:** [@kpks](https://discuss.elastic.co/u/kpks)\ **Replies:** 2\ **Last updated:** [May 14, 2019, 6:31am UTC](https://discuss.elastic.co/t/created-a-new-config-file-no-errors-but-not-loaded-in-kibana/180796 "2019-05-14T06:31:42Z")
I have a csv file and I created a new configuration file for Logstash: input { file { path =\> "/Users/naka/Documents/exploringELK/cims1819.csv" type =\> "csv" start\_position =\> "beginning" } } filter { csv { column… --- ## [Logstash syslog \_grokparsefailure errors](https://discuss.elastic.co/t/logstash-syslog-grokparsefailure-errors/180918)
**Author:** [@mkain](https://discuss.elastic.co/u/mkain)\ **Replies:** 1\ **Last updated:** [May 14, 2019, 6:26am UTC](https://discuss.elastic.co/t/logstash-syslog-grokparsefailure-errors/180918 "2019-05-14T06:26:58Z")
Hi, I am using default syslog parser config from elastic site. input { tcp { port =\> 514 type =\> syslog } udp { port =\> 514 type =\> syslog } } filter { if \[type\] == "syslog" { grok { … --- ## [How dockernized logstash knows system time zone?](https://discuss.elastic.co/t/how-dockernized-logstash-knows-system-time-zone/180921)
**Author:** [@codequokka](https://discuss.elastic.co/u/codequokka)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 6:03am UTC](https://discuss.elastic.co/t/how-dockernized-logstash-knows-system-time-zone/180921 "2019-05-14T06:03:51Z")
Hi I'm using docker.elastic.co/logstash/logstash-oss:7.0.1. Docker host(my pc) and logstash container are running in different timezone. Timezone of docker host is JST $ date Tue May 14 14:12:22 JST 2019 Timezone … --- ## [Executing Oracle Stored Procedure failed with ORA-00900: invalid SQL statement](https://discuss.elastic.co/t/executing-oracle-stored-procedure-failed-with-ora-00900-invalid-sql-statement/180775)
**Author:** [@rubylou](https://discuss.elastic.co/u/rubylou)\ **Replies:** 2\ **Last updated:** [May 14, 2019, 1:31am UTC](https://discuss.elastic.co/t/executing-oracle-stored-procedure-failed-with-ora-00900-invalid-sql-statement/180775 "2019-05-14T01:31:51Z")
input { jdbc { jdbc\_connection\_string =\> "###" jdbc\_user =\> "###" jdbc\_password =\> "###" jdbc\_driver\_library =\> "/home/ojdbc6.jar" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver… --- ## [Can't change field type](https://discuss.elastic.co/t/cant-change-field-type/180840)
**Author:** [@Aymen\_Ben\_Moussa](https://discuss.elastic.co/u/Aymen_Ben_Moussa)\ **Replies:** 5\ **Last updated:** [May 13, 2019, 10:51pm UTC](https://discuss.elastic.co/t/cant-change-field-type/180840 "2019-05-13T22:51:07Z")
i have the following two fields on my mongodb collection createdAt : 2019-05-04T09:47:38.767+00:00 updatedAt : 2019-05-04T09:47:41.027+00:00 i want to change both fields type to date, i've tried a lot of things and i … --- ## [Logstash not parsing log4j tcp dump](https://discuss.elastic.co/t/logstash-not-parsing-log4j-tcp-dump/180343)
**Author:** [@shruti1](https://discuss.elastic.co/u/shruti1)\ **Replies:** 1\ **Last updated:** [May 13, 2019, 10:34pm UTC](https://discuss.elastic.co/t/logstash-not-parsing-log4j-tcp-dump/180343 "2019-05-13T22:34:47Z")
I am having log4j.properties with tcp socket appender and I am seeing tcp dump. Also logstash is starting with log4j listener though no entry of data in logstash. --- ## [Aggregate filter - push on event size limit](https://discuss.elastic.co/t/aggregate-filter-push-on-event-size-limit/180875)
**Author:** [@bosand](https://discuss.elastic.co/u/bosand)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 9:23pm UTC](https://discuss.elastic.co/t/aggregate-filter-push-on-event-size-limit/180875 "2019-05-13T21:23:09Z")
Hi there, I would like to aggregate events until the aggregated event reaches a certain size limit (e.g. 256kb) at which point I would like to push the event to the queue. There are no start/end events. Push on timeout… --- ## [Snappy error](https://discuss.elastic.co/t/snappy-error/180891)
**Author:** [@DSPN](https://discuss.elastic.co/u/DSPN)\ **Replies:** 0\ **Last updated:** [May 13, 2019, 9:19pm UTC](https://discuss.elastic.co/t/snappy-error/180891 "2019-05-13T21:19:31Z")
New to logstash. Running it on openshift and getting the below error. Can anyone help please. \[ERROR\]\[org.logstash.Logstash \] java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.7-d45be369-d7ca-4218-9be9-45c405d490ff-lib… --- ## [Xml-file too big for logstash?](https://discuss.elastic.co/t/xml-file-too-big-for-logstash/179016)
**Author:** [@nane96](https://discuss.elastic.co/u/nane96)\ **Replies:** 27\ **Last updated:** [May 13, 2019, 8:35pm UTC](https://discuss.elastic.co/t/xml-file-too-big-for-logstash/179016 "2019-05-13T20:35:11Z")
Hello, I've been trying out a logstash config file for a pretty large xml-file containing log events. Thanks to the help of someon on here it originally worked, but only for a smaller version of the file, with about 400… --- ## [Timeout executing grok '%{PHPRUNTIMEERROR}' against field 'message' with valu](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851)
**Author:** [@Erin](https://discuss.elastic.co/u/Erin)\ **Replies:** 8\ **Last updated:** [May 13, 2019, 6:33pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851 "2019-05-13T18:33:27Z")
Hi, I am having a timeout for my grok filter. The logs are delivered from app1 server to logstash server via rsyslog forwarding, once the timeouts start the logstash server does not recover for several hours and is unav… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=547) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=549)