# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=608 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 609 --- ## [Logstash TCP Input Codec to set utf-16 is not working](https://discuss.elastic.co/t/logstash-tcp-input-codec-to-set-utf-16-is-not-working/166318)
**Author:** [@aliaramli](https://discuss.elastic.co/u/aliaramli)\ **Replies:** 4\ **Last updated:** [January 31, 2019, 3:01pm UTC](https://discuss.elastic.co/t/logstash-tcp-input-codec-to-set-utf-16-is-not-working/166318 "2019-01-31T15:01:14Z")
Hi, I wanted to set the input tcp plugin to have codec charset utf-16, because one of the logs has Chinese characters. \[2019-01-30T09:23:25,617\]\[WARN \]\[logstash.codecs.line \] Received an event that has a different char… --- ## [JSON parsing problem](https://discuss.elastic.co/t/json-parsing-problem/166413)
**Author:** [@Arthur\_Bulakaiev](https://discuss.elastic.co/u/Arthur_Bulakaiev)\ **Replies:** 8\ **Last updated:** [January 31, 2019, 2:51pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413 "2019-01-31T14:51:31Z")
Hi there! My target is very simple (as it seems): I want logstash to receive a json from http, process this json with some scrypt, and bypass it as json next. I've started with json codec, but I've not found any way to… --- ## [Xpack Monitoring Pipeline: You have enabled encryption but DISABLED certificate verification](https://discuss.elastic.co/t/xpack-monitoring-pipeline-you-have-enabled-encryption-but-disabled-certificate-verification/166577)
**Author:** [@Evesy](https://discuss.elastic.co/u/Evesy)\ **Replies:** 0\ **Last updated:** [January 31, 2019, 2:28pm UTC](https://discuss.elastic.co/t/xpack-monitoring-pipeline-you-have-enabled-encryption-but-disabled-certificate-verification/166577 "2019-01-31T14:28:21Z")
I'm seeing the below in Logstash output when it starts up for the monitoring pipeline: \[INFO \] 2019-01-31 14:16:49.047 \[Converge PipelineAction::Create\<.monitoring-logstash\>\] pipeline - Starting pipeline {:pipeline\_id=\>"… --- ## [Same data keeps on adding which results in duplicate data while using jdbc to fetch data from the mysql server](https://discuss.elastic.co/t/same-data-keeps-on-adding-which-results-in-duplicate-data-while-using-jdbc-to-fetch-data-from-the-mysql-server/166574)
**Author:** [@sachin\_augustine](https://discuss.elastic.co/u/sachin_augustine)\ **Replies:** 1\ **Last updated:** [January 31, 2019, 2:27pm UTC](https://discuss.elastic.co/t/same-data-keeps-on-adding-which-results-in-duplicate-data-while-using-jdbc-to-fetch-data-from-the-mysql-server/166574 "2019-01-31T14:27:12Z")
how can i avoid same data to pass and add up to the previous data creating redundancy for example when I add doc A and B, and then later when I add C, then doc A, B again along with C gets added to the elastic search th… --- ## [Logstash 6.6.0 is sending Error: No such file or directory - C:/dev/null Exception: Errno::ENOENT](https://discuss.elastic.co/t/logstash-6-6-0-is-sending-error-no-such-file-or-directory-c-dev-null-exception-errno-enoent/166405)
**Author:** [@Daniel\_Gonzalez1](https://discuss.elastic.co/u/Daniel_Gonzalez1)\ **Replies:** 5\ **Last updated:** [January 31, 2019, 2:20pm UTC](https://discuss.elastic.co/t/logstash-6-6-0-is-sending-error-no-such-file-or-directory-c-dev-null-exception-errno-enoent/166405 "2019-01-31T14:20:24Z")
Hi! I was trying to upgrade Logstash from version 6.2.4 to 6.6.0. I'm using windows 10 Logstash Version 6.2.4 runs fine and send my data to elasticsearch, but trying to do the same thing with version 6.6.0, using the … --- ## [Logstash not processing new beat](https://discuss.elastic.co/t/logstash-not-processing-new-beat/166566)
**Author:** [@Kevin\_Ross](https://discuss.elastic.co/u/Kevin_Ross)\ **Replies:** 1\ **Last updated:** [January 31, 2019, 2:19pm UTC](https://discuss.elastic.co/t/logstash-not-processing-new-beat/166566 "2019-01-31T14:19:10Z")
Hi, I am upgrading my logging so I have reinstalled server with new ELK stack and rebuilt a system to send logs into (bro & suricata). Now in Kibana I can see firewall syslog logs from before and a sensor (to be rebuil… --- ## [Logstash un-usual behaviour](https://discuss.elastic.co/t/logstash-un-usual-behaviour/166026)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 1\ **Last updated:** [January 31, 2019, 1:36pm UTC](https://discuss.elastic.co/t/logstash-un-usual-behaviour/166026 "2019-01-31T13:36:25Z")
Hi, I'm getting following messages into my logstash logs, my logstash version islogstash-6.5.4-1.noarch below id what i'm getting while starting via following command and logstash stops: /usr/share/logstash/bin/logsta… --- ## [Block in Start\_input](https://discuss.elastic.co/t/block-in-start-input/166474)
**Author:** [@fiery\_george](https://discuss.elastic.co/u/fiery_george)\ **Replies:** 1\ **Last updated:** [January 31, 2019, 1:06pm UTC](https://discuss.elastic.co/t/block-in-start-input/166474 "2019-01-31T13:06:13Z")
I am getting the following error: S:/ELK/logstash-6.6.0/logstash-6.6.0/vendor/bundle/jruby/2.3.0/gems/logstash-inp ut-file-4.1.9/lib/filewatch/observing\_base.rb:62:in build\_watch\_and\_dependencie s' S:/ELK/logstash-6.6.… --- ## [Replay files from Minio](https://discuss.elastic.co/t/replay-files-from-minio/166545)
**Author:** [@macdadi112](https://discuss.elastic.co/u/macdadi112)\ **Replies:** 0\ **Last updated:** [January 31, 2019, 12:02pm UTC](https://discuss.elastic.co/t/replay-files-from-minio/166545 "2019-01-31T12:02:43Z")
Hi What I have is files get archived to minio after a certain duration. So how would I play these files again (retrieved by logstash)? Thanks and apologies if I sent this to the wrong group, Mac --- ## [Try Using mutate for timestamp field](https://discuss.elastic.co/t/try-using-mutate-for-timestamp-field/166542)
**Author:** [@adityassa](https://discuss.elastic.co/u/adityassa)\ **Replies:** 0\ **Last updated:** [January 31, 2019, 11:37am UTC](https://discuss.elastic.co/t/try-using-mutate-for-timestamp-field/166542 "2019-01-31T11:37:58Z")
Continuing the discussion from Error to map date field with format yyyy-MM-dd HH:mm:ss.SSS: // mutate { gsub =\> \[ # replace all space with T in timestamp field/column "timestamp", " ", "T" \] // --- ## [Zabbix output plugin not reporting to](https://discuss.elastic.co/t/zabbix-output-plugin-not-reporting-to/166541)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 0\ **Last updated:** [January 31, 2019, 11:31am UTC](https://discuss.elastic.co/t/zabbix-output-plugin-not-reporting-to/166541 "2019-01-31T11:31:22Z")
I want to send simple metric values to a zabbix server. I am creating some metrics via the respective plugin and also adding the fields required by the zabbix plugin: if "paloalto" in \[tags\] { metrics { mete… --- ## [How to query for most recent timestamp and filter query](https://discuss.elastic.co/t/how-to-query-for-most-recent-timestamp-and-filter-query/166539)
**Author:** [@manickalai](https://discuss.elastic.co/u/manickalai)\ **Replies:** 0\ **Last updated:** [January 31, 2019, 11:16am UTC](https://discuss.elastic.co/t/how-to-query-for-most-recent-timestamp-and-filter-query/166539 "2019-01-31T11:16:06Z")
How do I query for the timestamp of the most recent entry for a specific type filter like @applicationName: @HOSTNAME: @appname: @timestamp: sample logs Kindly help me to write right query format? --- ## [Zabbix plugin error about missing field](https://discuss.elastic.co/t/zabbix-plugin-error-about-missing-field/166520)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 0\ **Last updated:** [January 31, 2019, 10:02am UTC](https://discuss.elastic.co/t/zabbix-plugin-error-about-missing-field/166520 "2019-01-31T10:02:09Z")
I am trying to send a simple value to a zabbix server. Here is my configuration: zabbix { zabbix\_host =\> "ip-resolvable-hostname-of-the-host-on-which-logstash-is-running" zabbix\_server\_host =\> "hostname-of-my… --- ## [How to config aruba wireless controller log to ELK](https://discuss.elastic.co/t/how-to-config-aruba-wireless-controller-log-to-elk/166461)
**Author:** [@guoxinabc](https://discuss.elastic.co/u/guoxinabc)\ **Replies:** 1\ **Last updated:** [January 31, 2019, 9:59am UTC](https://discuss.elastic.co/t/how-to-config-aruba-wireless-controller-log-to-elk/166461 "2019-01-31T09:59:54Z")
how to config aruba wireless controller log to ELK? --- ## [How to handle gzip'd gzip'd (twice) files in S3](https://discuss.elastic.co/t/how-to-handle-gzipd-gzipd-twice-files-in-s3/165436)
**Author:** [@markwan](https://discuss.elastic.co/u/markwan)\ **Replies:** 1\ **Last updated:** [January 31, 2019, 9:55am UTC](https://discuss.elastic.co/t/how-to-handle-gzipd-gzipd-twice-files-in-s3/165436 "2019-01-31T09:55:21Z")
A client is providing AWS cloud trail logs via an S3 bucket. The S3 input plugin will read .gz files. However the problem is that the file, eg "cloudtrail\_log\_2019-01-22.gz" is actually gzip'd twice, and should really … --- ## [Adding geopoints to nested objects in an array of parent document](https://discuss.elastic.co/t/adding-geopoints-to-nested-objects-in-an-array-of-parent-document/166326)
**Author:** [@Sagar\_Mishra](https://discuss.elastic.co/u/Sagar_Mishra)\ **Replies:** 2\ **Last updated:** [January 31, 2019, 9:54am UTC](https://discuss.elastic.co/t/adding-geopoints-to-nested-objects-in-an-array-of-parent-document/166326 "2019-01-31T09:54:48Z")
I have a parent and an array of nested objects called children. I am getting lat and lng(longitude) from my sql join query. I want to create geo\_point in nested objects of parents contained in an array using lat and lng… --- ## [Logstash doesn’t process / cgroup files](https://discuss.elastic.co/t/logstash-doesn-t-process-cgroup-files/166494)
**Author:** [@Michael.Swiss](https://discuss.elastic.co/u/Michael.Swiss)\ **Replies:** 1\ **Last updated:** [January 31, 2019, 9:52am UTC](https://discuss.elastic.co/t/logstash-doesn-t-process-cgroup-files/166494 "2019-01-31T09:52:14Z")
Dear all I installed logstash 6.6.0 and ran a first sequence that picks up a csv file and stdouts on the screen. It worked as expected. Then I added and removed a translate filter. Both times logstash didn’t start proce… --- ## [Mutate | Some logs are parsed some not](https://discuss.elastic.co/t/mutate-some-logs-are-parsed-some-not/166007)
**Author:** [@thirty2](https://discuss.elastic.co/u/thirty2)\ **Replies:** 5\ **Last updated:** [January 31, 2019, 9:38am UTC](https://discuss.elastic.co/t/mutate-some-logs-are-parsed-some-not/166007 "2019-01-31T09:38:43Z")
Hi, did you ever observed that some logs are parsed and some not when using MUTATE filter? Thanks --- ## [Logs not reaching from all the servers on ES/Kibana](https://discuss.elastic.co/t/logs-not-reaching-from-all-the-servers-on-es-kibana/166501)
**Author:** [@Harsh\_Sharma](https://discuss.elastic.co/u/Harsh_Sharma)\ **Replies:** 0\ **Last updated:** [January 31, 2019, 8:48am UTC](https://discuss.elastic.co/t/logs-not-reaching-from-all-the-servers-on-es-kibana/166501 "2019-01-31T08:48:36Z")
I'm using ELK 6.5.4 Filebeat = 10 nodes of Nginx Logstash Server = 1 nodes(10.5.202.201) = (8 core, 128GB Ram) || 3 nodes = 1 node of (8 core, 128GB Ram) & 2 nodes of (2 core, 4GB RAM) I'm getting logs from 2 nodes (… --- ## [Convert string to array in logstash](https://discuss.elastic.co/t/convert-string-to-array-in-logstash/166321)
**Author:** [@BlueSpirit](https://discuss.elastic.co/u/BlueSpirit)\ **Replies:** 2\ **Last updated:** [January 31, 2019, 7:02am UTC](https://discuss.elastic.co/t/convert-string-to-array-in-logstash/166321 "2019-01-31T07:02:02Z")
My original data. { message: { data: "\["1,2","3,4","5,6"\]" } } Now I want to convert value of data field to an array. So it should become: { message: { data: \["1,2", "3,4", "5,6"\] } } By using m… --- ## [Pattern Cisco-ASA syslog message : Grokparsefailure](https://discuss.elastic.co/t/pattern-cisco-asa-syslog-message-grokparsefailure/166233)
**Author:** [@uncoveredmind](https://discuss.elastic.co/u/uncoveredmind)\ **Replies:** 3\ **Last updated:** [January 31, 2019, 6:27am UTC](https://discuss.elastic.co/t/pattern-cisco-asa-syslog-message-grokparsefailure/166233 "2019-01-31T06:27:51Z")
I used a cisco-asa pattern to filter my syslog message but i don't know why I keep getting tags:\_grokparsefailure Message: Device chooses cipher ECDHE-RSA-AES256-GCM-SHA384 for the SSL session with client INTERNET:1.2.3… --- ## [Logstash -drupal-plugin](https://discuss.elastic.co/t/logstash-drupal-plugin/166476)
**Author:** [@sundarm](https://discuss.elastic.co/u/sundarm)\ **Replies:** 1\ **Last updated:** [January 31, 2019, 6:25am UTC](https://discuss.elastic.co/t/logstash-drupal-plugin/166476 "2019-01-31T06:25:56Z")
Hi , I am not able to view this page for drupal config. https://www.elastic.co/guide/en/logstash/current/plugins-inputs-drupal\_dblog.html Thanks --- ## [Looking for ideas to mask part of a creditcard](https://discuss.elastic.co/t/looking-for-ideas-to-mask-part-of-a-creditcard/166333)
**Author:** [@Saurav\_Sen](https://discuss.elastic.co/u/Saurav_Sen)\ **Replies:** 2\ **Last updated:** [January 31, 2019, 3:35am UTC](https://discuss.elastic.co/t/looking-for-ideas-to-mask-part-of-a-creditcard/166333 "2019-01-31T03:35:28Z")
Trying to mask out a part (6-10th digit) of the 16 digit credit card. 1234567898763456 to 1234######3456 I can think of using gsub filter, but I can't find a way to match the exact section of numbers. The option would… --- ## [Issue in using xpath in xml filter](https://discuss.elastic.co/t/issue-in-using-xpath-in-xml-filter/166295)
**Author:** [@vivek14](https://discuss.elastic.co/u/vivek14)\ **Replies:** 2\ **Last updated:** [January 31, 2019, 2:09am UTC](https://discuss.elastic.co/t/issue-in-using-xpath-in-xml-filter/166295 "2019-01-31T02:09:14Z")
Hi, I am just beginning to learn ELK, Grok etc. I recently installed Elastic Stack (ELK and Filebeat) to monitor RedHat JBoss Fuse log files. My goal is to create new fields by applying Grok's xml filter and extract i… --- ## [Logstash 6.2.2: Empty square brackets in json field names crash the pipeline](https://discuss.elastic.co/t/logstash-6-2-2-empty-square-brackets-in-json-field-names-crash-the-pipeline/166443)
**Author:** [@vtipparam](https://discuss.elastic.co/u/vtipparam)\ **Replies:** 4\ **Last updated:** [January 31, 2019, 1:44am UTC](https://discuss.elastic.co/t/logstash-6-2-2-empty-square-brackets-in-json-field-names-crash-the-pipeline/166443 "2019-01-31T01:44:42Z")
Empty square brackets in field names causes the Logstash pipeline to crash with following error: Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restar… --- ## [Logstash JSON field from syslog to flat fields?](https://discuss.elastic.co/t/logstash-json-field-from-syslog-to-flat-fields/166421)
**Author:** [@jbrowe](https://discuss.elastic.co/u/jbrowe)\ **Replies:** 2\ **Last updated:** [January 30, 2019, 11:32pm UTC](https://discuss.elastic.co/t/logstash-json-field-from-syslog-to-flat-fields/166421 "2019-01-30T23:32:56Z")
I have a vendor appliance that sends syslog events in JSON format. I am able to receive the message, parse the syslog header with grok and obtain the JSON data in a field from the message portion. The JSON field looks s… --- ## [Unable to install log4j2 plugin into Logstash](https://discuss.elastic.co/t/unable-to-install-log4j2-plugin-into-logstash/166428)
**Author:** [@john.jensen](https://discuss.elastic.co/u/john.jensen)\ **Replies:** 7\ **Last updated:** [January 30, 2019, 10:52pm UTC](https://discuss.elastic.co/t/unable-to-install-log4j2-plugin-into-logstash/166428 "2019-01-30T22:52:26Z")
I'm trying to use log4j2 to send messages to our Logstash instance. I'm running this command: bin\\logstash -r -p "C:\\elasticsearch\_course\\logstash-input-log4j2" -f "C:\\elasticsearch\_course\\logstash\_data\\test.conf" The… --- ## [Logstash config with filebeat fields](https://discuss.elastic.co/t/logstash-config-with-filebeat-fields/166373)
**Author:** [@sundarm](https://discuss.elastic.co/u/sundarm)\ **Replies:** 8\ **Last updated:** [January 30, 2019, 10:49pm UTC](https://discuss.elastic.co/t/logstash-config-with-filebeat-fields/166373 "2019-01-30T22:49:44Z")
HI, I have filebeat installed on all my dev server and qa server in my filebeat config i have added the field as env: dev and env:qa how do i use this in my logstash config output index pattern. i tried this method bu… --- ## [Dynamic JSON -- Can't get text on a START\_OBJECT](https://discuss.elastic.co/t/dynamic-json-cant-get-text-on-a-start-object/165653)
**Author:** [@brandondash](https://discuss.elastic.co/u/brandondash)\ **Replies:** 1\ **Last updated:** [January 30, 2019, 10:18pm UTC](https://discuss.elastic.co/t/dynamic-json-cant-get-text-on-a-start-object/165653 "2019-01-30T22:18:59Z")
I know the error means it is expecting a string but gets an object, but I have no idea why only this field breaks. It looks and behaves like every other field yet consistently errors out. Here is the relevant template I… --- ## [Why newest versions of logstash are not processing any message](https://discuss.elastic.co/t/why-newest-versions-of-logstash-are-not-processing-any-message/166420)
**Author:** [@Daniel\_Gonzalez1](https://discuss.elastic.co/u/Daniel_Gonzalez1)\ **Replies:** 3\ **Last updated:** [January 30, 2019, 9:13pm UTC](https://discuss.elastic.co/t/why-newest-versions-of-logstash-are-not-processing-any-message/166420 "2019-01-30T21:13:07Z")
Hello I've been using logstash version 6.2.4 for some time and today I came with the idea to upgrade to the 6.6.0 Version 6.2.4 Is working fine with any issue. But today when I wanted to do the same stuff of the v 6.2.4… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=607) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=609)