# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=977 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 978 --- ## [Logstash-input-kafka modify non-ascii](https://discuss.elastic.co/t/logstash-input-kafka-modify-non-ascii/87063)
**Author:** [@aimerlee860](https://discuss.elastic.co/u/aimerlee860)\ **Replies:** 1\ **Last updated:** [May 25, 2017, 4:23pm UTC](https://discuss.elastic.co/t/logstash-input-kafka-modify-non-ascii/87063 "2017-05-25T16:23:15Z")
Hi, I meet a problem when using logstash-input-kafka plugin recently. I write a serialized protobuf message to kafka, and then use logstash-input-kafka plugin to consume the messages in logstash, but I find that some … --- ## [Logstash to Elastic search not working for heartbeat](https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694)
**Author:** [@arun\_prasath1](https://discuss.elastic.co/u/arun_prasath1)\ **Replies:** 3\ **Last updated:** [May 25, 2017, 2:42pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694 "2017-05-25T14:42:04Z")
In the ELK pipeline, I have Heartbeat pushing logs to Logstash and then to Elasticsearch. The heartbeat to logstash works fine and from logstash to elasticsearh have issues. I have done my best using the debug options in… --- ## [Reliable Kafka Output Plugin config](https://discuss.elastic.co/t/reliable-kafka-output-plugin-config/86998)
**Author:** [@yuri1969](https://discuss.elastic.co/u/yuri1969)\ **Replies:** 1\ **Last updated:** [May 25, 2017, 1:16pm UTC](https://discuss.elastic.co/t/reliable-kafka-output-plugin-config/86998 "2017-05-25T13:16:22Z")
Hello, I'm wondering about a way how to make output to Kafka reliable. I am new to Kafka, so I'm sorry about any logical errors. I use Logstash 5.4.0 and Kafka 0.10.2.1 with a matching Zookeeper. The Kafka setup is rat… --- ## [Need help getting Logstash index from EC2 instance to Elasticsearch](https://discuss.elastic.co/t/need-help-getting-logstash-index-from-ec2-instance-to-elasticsearch/86726)
**Author:** [@fuel](https://discuss.elastic.co/u/fuel)\ **Replies:** 2\ **Last updated:** [May 25, 2017, 1:13pm UTC](https://discuss.elastic.co/t/need-help-getting-logstash-index-from-ec2-instance-to-elasticsearch/86726 "2017-05-25T13:13:44Z")
I have activated Elasticsearch from the AWS dashboard. My whole goal is to be able to view apache logs from my EC2 instance in Elasticsearch. I have installed Logstash on the EC2 instance. I have been able to start up… --- ## [Teradata JDBC Pagination](https://discuss.elastic.co/t/teradata-jdbc-pagination/87014)
**Author:** [@jennifer.elyse](https://discuss.elastic.co/u/jennifer.elyse)\ **Replies:** 0\ **Last updated:** [May 24, 2017, 4:38pm UTC](https://discuss.elastic.co/t/teradata-jdbc-pagination/87014 "2017-05-24T16:38:02Z")
I am new to the Elastic Stack. I'm using Logstash to load data from Teradata into ElastiSearch. I want to implement jdbc\_paging\_enabled, but Logstash is wrapping my sql statement in a SELECT count(\*) AS "COUNT" FROM () … --- ## [Need help how to index with template in LogStash](https://discuss.elastic.co/t/need-help-how-to-index-with-template-in-logstash/87003)
**Author:** [@marcogoldin](https://discuss.elastic.co/u/marcogoldin)\ **Replies:** 3\ **Last updated:** [May 25, 2017, 12:17pm UTC](https://discuss.elastic.co/t/need-help-how-to-index-with-template-in-logstash/87003 "2017-05-25T12:17:07Z")
Hi, i'm new to LogStash and i'm trying to index from a csv. I really can't understand what is going on, tried many solutions but LogStash keep stopping the pipeline. here's my .conf input { file { path =\>… --- ## [Input json file to logstash issues](https://discuss.elastic.co/t/input-json-file-to-logstash-issues/86859)
**Author:** [@Tal\_Bar\_Or](https://discuss.elastic.co/u/Tal_Bar_Or)\ **Replies:** 1\ **Last updated:** [May 25, 2017, 9:56am UTC](https://discuss.elastic.co/t/input-json-file-to-logstash-issues/86859 "2017-05-25T09:56:16Z")
Hi, I am trying to input an tshark json output and output it "ruby\_debug" for first stage facing issues Tried several ways inout i searched but keep getting errors showed below , any i idea hot input correctly such fil… --- ## [Centralized Server Log](https://discuss.elastic.co/t/centralized-server-log/87010)
**Author:** [@BERUSO](https://discuss.elastic.co/u/BERUSO)\ **Replies:** 1\ **Last updated:** [May 25, 2017, 9:36am UTC](https://discuss.elastic.co/t/centralized-server-log/87010 "2017-05-25T09:36:11Z")
Hi everybody! I need your help, I'm working on a centralized server where logs are read, these logs are delivery from a redis. What I would like to know if it's possible to deliver these logs from the centralized server … --- ## [Logstash syslog output plugin not setting facility](https://discuss.elastic.co/t/logstash-syslog-output-plugin-not-setting-facility/80537)
**Author:** [@sysinit](https://discuss.elastic.co/u/sysinit)\ **Replies:** 5\ **Last updated:** [May 25, 2017, 8:28am UTC](https://discuss.elastic.co/t/logstash-syslog-output-plugin-not-setting-facility/80537 "2017-05-25T08:28:46Z")
Hi, I want to use the logstash syslog output plugin (Syslog output plugin | Logstash Reference \[8.11\] | Elastic) to send specific logs to a remote host. It doesn't work if I configure the output to send to the remote h… --- ## [How can i aggregate this](https://discuss.elastic.co/t/how-can-i-aggregate-this/87061)
**Author:** [@JIELOGAN](https://discuss.elastic.co/u/JIELOGAN)\ **Replies:** 0\ **Last updated:** [May 25, 2017, 2:21am UTC](https://discuss.elastic.co/t/how-can-i-aggregate-this/87061 "2017-05-25T02:21:58Z")
hello, this is my data: because those data are not in a same line , i want one event to hold all data value, just combine all values by logstash aggregate plugin. how can i achieve this？ Thanks a lot --- ## [Is Ruby prerequisite to install Logstash with extra plugins?](https://discuss.elastic.co/t/is-ruby-prerequisite-to-install-logstash-with-extra-plugins/87059)
**Author:** [@techpanga](https://discuss.elastic.co/u/techpanga)\ **Replies:** 0\ **Last updated:** [May 25, 2017, 1:19am UTC](https://discuss.elastic.co/t/is-ruby-prerequisite-to-install-logstash-with-extra-plugins/87059 "2017-05-25T01:19:00Z")
Hi All, Logstash\[5.4.0\] Java \[ 1.8.0\_92\] Is Ruby prerequisite to install Logstash with extra plugins? I am trying to install plugins using below commands /usr/share/logstash\> ./bin/logstash-plugin install logsta… --- ## [Which: no javac for local plugin installation](https://discuss.elastic.co/t/which-no-javac-for-local-plugin-installation/86181)
**Author:** [@lgee](https://discuss.elastic.co/u/lgee)\ **Replies:** 2\ **Last updated:** [May 24, 2017, 9:32pm UTC](https://discuss.elastic.co/t/which-no-javac-for-local-plugin-installation/86181 "2017-05-24T21:32:02Z")
I have a logstash dockerfile which uses the elastic docker 5.3.2 image (docker.elastic.co/logstash/logstash:5.3.2) as a base and installs 3 local gem plugins which aren't in the logstash plugin repository. If I bump the… --- ## [Multiple Onigurama named captures with same name only captures once](https://discuss.elastic.co/t/multiple-onigurama-named-captures-with-same-name-only-captures-once/87009)
**Author:** [@Striata](https://discuss.elastic.co/u/Striata)\ **Replies:** 0\ **Last updated:** [May 24, 2017, 4:11pm UTC](https://discuss.elastic.co/t/multiple-onigurama-named-captures-with-same-name-only-captures-once/87009 "2017-05-24T16:11:40Z")
Hi! The logstash documentation introduces two ways of creating named captures in Logstash: Onigurama regexp syntax:(?\Hello) Grok patterns from pattern file: HELLO Hello and then {%HELLO:a} in the grok pattern S… --- ## [Jdbc output plugin](https://discuss.elastic.co/t/jdbc-output-plugin/86833)
**Author:** [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Replies:** 1\ **Last updated:** [May 24, 2017, 3:38pm UTC](https://discuss.elastic.co/t/jdbc-output-plugin/86833 "2017-05-24T15:38:56Z")
Hi there, As you can imagine reading the title, I'm using the unofficial jdbc output plugin to put logs into postgres. I'm facing 2 problems, and I was wandering somebody else has experienced the same issues. The f… --- ## [Multiline Codec Error?](https://discuss.elastic.co/t/multiline-codec-error/86814)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 1\ **Last updated:** [May 24, 2017, 3:27pm UTC](https://discuss.elastic.co/t/multiline-codec-error/86814 "2017-05-24T15:27:05Z")
When i parsing the logs on TIMESTAMP field of multiline codec it is showing error output for the TIMESTAMP fields present in square brackets. Config: input { file { path =\> "D:\\logstash\\logstash-2.4.0\\bin\\slowl… --- ## [Error in reading data](https://discuss.elastic.co/t/error-in-reading-data/86549)
**Author:** [@Padmavathy](https://discuss.elastic.co/u/Padmavathy)\ **Replies:** 2\ **Last updated:** [May 24, 2017, 2:04pm UTC](https://discuss.elastic.co/t/error-in-reading-data/86549 "2017-05-24T14:04:44Z")
I am using Logstash 5.3. and connecting to Oracle database.I gave a select query with values for 1 year.It worked fine but suddenly when i run logstash it starts overwriting the values.Example:February's data is overwrit… --- ## [Remove fields on windows event log](https://discuss.elastic.co/t/remove-fields-on-windows-event-log/86891)
**Author:** [@tharu85](https://discuss.elastic.co/u/tharu85)\ **Replies:** 5\ **Last updated:** [May 24, 2017, 1:19pm UTC](https://discuss.elastic.co/t/remove-fields-on-windows-event-log/86891 "2017-05-24T13:19:43Z")
{ "message" =\> "The Windows Filtering Platform has permitted a connection......bla bla.....", "@version" =\> "1", "@timestamp" =\> "2017-05-24T03:38:00.775Z", "type" =\> "wineventlog", … --- ## [Logstash "elasticsearch input" problem](https://discuss.elastic.co/t/logstash-elasticsearch-input-problem/86847)
**Author:** [@orhiee](https://discuss.elastic.co/u/orhiee)\ **Replies:** 4\ **Last updated:** [May 24, 2017, 12:39pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input-problem/86847 "2017-05-24T12:39:22Z")
hello, Couldnt find the answer on here or google, so here i am asking.... i want to use logstash to transfer data from one elasticsearch to another, but i cant move all the docs can only move the number spesified in… --- ## [Rename and sub field dont work](https://discuss.elastic.co/t/rename-and-sub-field-dont-work/86940)
**Author:** [@nigdav007](https://discuss.elastic.co/u/nigdav007)\ **Replies:** 3\ **Last updated:** [May 24, 2017, 12:14pm UTC](https://discuss.elastic.co/t/rename-and-sub-field-dont-work/86940 "2017-05-24T12:14:25Z")
Hay All I am trying to rename and field called "@fields.ctxt\_file" to "file" but it does not work the config i am using is json { source =\> "message" } mutate { replace =\> \[ "messag… --- ## [Logstash service multiple configs](https://discuss.elastic.co/t/logstash-service-multiple-configs/86923)
**Author:** [@jjjuurrbbbb](https://discuss.elastic.co/u/jjjuurrbbbb)\ **Replies:** 4\ **Last updated:** [May 24, 2017, 11:11am UTC](https://discuss.elastic.co/t/logstash-service-multiple-configs/86923 "2017-05-24T11:11:15Z")
Hi, Just getting started with Logstash and am trying to get the service started using multiple config files on CentOS. My logstash.service configuration is as follows: \[Unit\] Description=logstash \[Service\] Type=s… --- ## [Can't start logstash after x-pack installation](https://discuss.elastic.co/t/cant-start-logstash-after-x-pack-installation/86824)
**Author:** [@craftytech1](https://discuss.elastic.co/u/craftytech1)\ **Replies:** 10\ **Last updated:** [May 24, 2017, 10:58am UTC](https://discuss.elastic.co/t/cant-start-logstash-after-x-pack-installation/86824 "2017-05-24T10:58:45Z")
using elk 5.4. After installing x-pack for elk stack 5.4, I'm now getting the error: Failed to create monitoring event {:message=\>"For path: events", :error=\>"LogStash::Instrument::MetricStore::MetricNotFound"} kib… --- ## [Recreating an index from LogStash and swapping it via alias](https://discuss.elastic.co/t/recreating-an-index-from-logstash-and-swapping-it-via-alias/86652)
**Author:** [@Xavi\_Ametller](https://discuss.elastic.co/u/Xavi_Ametller)\ **Replies:** 5\ **Last updated:** [May 24, 2017, 10:57am UTC](https://discuss.elastic.co/t/recreating-an-index-from-logstash-and-swapping-it-via-alias/86652 "2017-05-24T10:57:34Z")
Hi there, I would like to periodically generate an index whose data will come from the outcome of a SQL query. Even though I'm aware it is not the traditional use case as the data are not logs nor accumulative, LogStash … --- ## [Jdbc-streaming plugin seems not working](https://discuss.elastic.co/t/jdbc-streaming-plugin-seems-not-working/85530)
**Author:** [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Replies:** 43\ **Last updated:** [May 24, 2017, 10:19am UTC](https://discuss.elastic.co/t/jdbc-streaming-plugin-seems-not-working/85530 "2017-05-24T10:19:08Z")
hi all, I'm using jdbc filter plugin to enrich data from Postgres DB but I don't see the output log enriched. here's the configuration I'm using: input { stdin { } } filter { jdbc\_streaming { jdbc\_driver… --- ## [How to set timezone?](https://discuss.elastic.co/t/how-to-set-timezone/43865)
**Author:** [@ggilat](https://discuss.elastic.co/u/ggilat)\ **Replies:** 5\ **Last updated:** [May 24, 2017, 9:32am UTC](https://discuss.elastic.co/t/how-to-set-timezone/43865 "2017-05-24T09:32:41Z")
Hi, I will be thankful if you'll help me solved this problem: My timezone is UTC + 2 (Red Arrow in the pic) but when i drill down into the event JSON the timestamp is 2 hour erlier (Blue Arrow in the pic): @timestamp"… --- ## [How to raw logs to logstash default index and filtered log to another index](https://discuss.elastic.co/t/how-to-raw-logs-to-logstash-default-index-and-filtered-log-to-another-index/86517)
**Author:** [@shamimgeek](https://discuss.elastic.co/u/shamimgeek)\ **Replies:** 3\ **Last updated:** [May 24, 2017, 9:25am UTC](https://discuss.elastic.co/t/how-to-raw-logs-to-logstash-default-index-and-filtered-log-to-another-index/86517 "2017-05-24T09:25:37Z")
Hi All, How can i configure logstash which can send all logs to default logstash-xxxx index and filtered log { i want only specific fields} will go to analytics index. But I also want to have all raw logs in logstash … --- ## [Log4Net](https://discuss.elastic.co/t/log4net/86476)
**Author:** [@snakpak](https://discuss.elastic.co/u/snakpak)\ **Replies:** 1\ **Last updated:** [May 24, 2017, 9:23am UTC](https://discuss.elastic.co/t/log4net/86476 "2017-05-24T09:23:04Z")
Hi, I just installed LogStash using these steps (https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7) Now, I need to parse my log4net's logs. I want… --- ## [Regarding multiple message field](https://discuss.elastic.co/t/regarding-multiple-message-field/86454)
**Author:** [@amitkumar](https://discuss.elastic.co/u/amitkumar)\ **Replies:** 1\ **Last updated:** [May 24, 2017, 9:21am UTC](https://discuss.elastic.co/t/regarding-multiple-message-field/86454 "2017-05-24T09:21:18Z")
I have use html file path to create index but file content is not coming in single message field.It is breaking in different message fields in hits.how should i get it in single message field. Json for html file is like … --- ## [Logging events to Logstash via REST with custom request and responses](https://discuss.elastic.co/t/logging-events-to-logstash-via-rest-with-custom-request-and-responses/86031)
**Author:** [@dml](https://discuss.elastic.co/u/dml)\ **Replies:** 3\ **Last updated:** [May 24, 2017, 9:17am UTC](https://discuss.elastic.co/t/logging-events-to-logstash-via-rest-with-custom-request-and-responses/86031 "2017-05-24T09:17:47Z")
I would like an application to log events to Logstash using a REST API. The request is JSON-encoded event data, some of which are required, some optional, and some would default to default values. The request would nee… --- ## [Arrays in arrays](https://discuss.elastic.co/t/arrays-in-arrays/86816)
**Author:** [@Toony](https://discuss.elastic.co/u/Toony)\ **Replies:** 1\ **Last updated:** [May 24, 2017, 8:46am UTC](https://discuss.elastic.co/t/arrays-in-arrays/86816 "2017-05-24T08:46:53Z")
Hi everyone, I am trying to index geoshapes from kafka messages ... I just can not use the merge to put arrays in an array. For instance, i want to index an envelope geoshape like this in target elasticsearch json: \[… --- ## [Link lines context in Logstash](https://discuss.elastic.co/t/link-lines-context-in-logstash/86878)
**Author:** [@Leandro\_Sampaio](https://discuss.elastic.co/u/Leandro_Sampaio)\ **Replies:** 1\ **Last updated:** [May 24, 2017, 8:08am UTC](https://discuss.elastic.co/t/link-lines-context-in-logstash/86878 "2017-05-24T08:08:11Z")
Hi My post its to explain a parse that I did by myself because didnt exist anyone reference. I used HashMap reference key value linked using a tag "source" from Filebeat to put a context transaction on my script and… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=976) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=978)