# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1059 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1060 --- ## [Generating timestamp by concatenating date from filename and time from log events](https://discuss.elastic.co/t/generating-timestamp-by-concatenating-date-from-filename-and-time-from-log-events/70196)
**Author:** [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Replies:** 1\ **Last updated:** [December 31, 2016, 1:08pm UTC](https://discuss.elastic.co/t/generating-timestamp-by-concatenating-date-from-filename-and-time-from-log-events/70196 "2016-12-31T13:08:21Z")
Hi, I have a log file as follows:- Loading jar:file:/D:/smart/ProgramFiles/liferay-portal-6.0.5/tomcat-6.0.26/webapps/ROOT/WEB-INF/lib/portal-impl.jar!/system.properties Loading jar:file:/D:/smart/ProgramFiles/liferay-… --- ## [Need to configure Multiple logstash instance on same system](https://discuss.elastic.co/t/need-to-configure-multiple-logstash-instance-on-same-system/70262)
**Author:** [@shakir.mulani](https://discuss.elastic.co/u/shakir.mulani)\ **Replies:** 1\ **Last updated:** [December 31, 2016, 1:04pm UTC](https://discuss.elastic.co/t/need-to-configure-multiple-logstash-instance-on-same-system/70262 "2016-12-31T13:04:56Z")
Hi, I have one logstash instance running on a server in folder /usr/local/logstash I have configured start-elk.sh and stop-elk.sh script in /usr/local/bin/ From where i start the all services like logstash, kibana, … --- ## [How to rename json parsed field](https://discuss.elastic.co/t/how-to-rename-json-parsed-field/70059)
**Author:** [@rajkamalkool6](https://discuss.elastic.co/u/rajkamalkool6)\ **Replies:** 1\ **Last updated:** [December 31, 2016, 12:59pm UTC](https://discuss.elastic.co/t/how-to-rename-json-parsed-field/70059 "2016-12-31T12:59:15Z")
Hi, I am parsing json log file in Logstash. There is a field named @person.name. I tried to rename this field name before sending it to elasticsearch. I also tried to remove the field but I couldn't remove or delete tha… --- ## [Tcp output into statsd plugins](https://discuss.elastic.co/t/tcp-output-into-statsd-plugins/69990)
**Author:** [@gennady](https://discuss.elastic.co/u/gennady)\ **Replies:** 1\ **Last updated:** [December 31, 2016, 10:35am UTC](https://discuss.elastic.co/t/tcp-output-into-statsd-plugins/69990 "2016-12-31T10:35:17Z")
Hi all, I try to configure output statsd plugins to point out tcp protocol, but it does not work. is there any chance to solve it? Thanks in advance BR, Gennady --- ## [Logstash-inputjdbc postgresql json type import to ES problem](https://discuss.elastic.co/t/logstash-inputjdbc-postgresql-json-type-import-to-es-problem/69976)
**Author:** [@maleibo](https://discuss.elastic.co/u/maleibo)\ **Replies:** 1\ **Last updated:** [December 31, 2016, 10:32am UTC](https://discuss.elastic.co/t/logstash-inputjdbc-postgresql-json-type-import-to-es-problem/69976 "2016-12-31T10:32:01Z")
hello ! i have a postgresql database ,some columns of type is json. i used input-jdbc plugin to import these datas,but there is a problem,for example: my json column data: {"type":"Point","coordinates":\[106.1223,29.3467\]… --- ## [Http\_poller on failure add\_field not working](https://discuss.elastic.co/t/http-poller-on-failure-add-field-not-working/70287)
**Author:** [@cgspohn](https://discuss.elastic.co/u/cgspohn)\ **Replies:** 0\ **Last updated:** [December 30, 2016, 5:00pm UTC](https://discuss.elastic.co/t/http-poller-on-failure-add-field-not-working/70287 "2016-12-30T17:00:19Z")
For the http\_poller input I have used the "add\_field" option which works perfect when the request succeeds. But when the request fails (falls into the handle\_failure method) the "add\_field" option is not honored. I wo… --- ## [5.1 : Configtest](https://discuss.elastic.co/t/5-1-configtest/70284)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [December 30, 2016, 4:02pm UTC](https://discuss.elastic.co/t/5-1-configtest/70284 "2016-12-30T16:02:25Z")
Hi, I'm trying to start and troubleshoot logstash on ubuntu xenial. It is proving difficult because the service isn't writing any log output although the service is running: systemctl status logstash.service ● logstas… --- ## [What is wrong with this filter?](https://discuss.elastic.co/t/what-is-wrong-with-this-filter/70226)
**Author:** [@lupolo](https://discuss.elastic.co/u/lupolo)\ **Replies:** 4\ **Last updated:** [December 30, 2016, 2:04pm UTC](https://discuss.elastic.co/t/what-is-wrong-with-this-filter/70226 "2016-12-30T14:04:28Z")
Hi, I have the following filter that passes the configuration test: ./logstash -f ../config/pipeline.conf --config.test\_and\_exit Sending Logstash's logs to /usr/local/rex/logstash-5.0.2/logs which is now configured … --- ## [Logstash 5.1 refuses to "boot up" (wrong configuration)](https://discuss.elastic.co/t/logstash-5-1-refuses-to-boot-up-wrong-configuration/70275)
**Author:** [@moonwhaler](https://discuss.elastic.co/u/moonwhaler)\ **Replies:** 0\ **Last updated:** [December 30, 2016, 1:43pm UTC](https://discuss.elastic.co/t/logstash-5-1-refuses-to-boot-up-wrong-configuration/70275 "2016-12-30T13:43:30Z")
Hello there! I am currently using the latest Debian (Jessie) packages from Logstash 5.1 (in conjunction with ES 5.1 and Kibana 5.1), but Logstash is driving me nuts. (Logstash 2.x was working fine, even testing the co… --- ## [Logstash to read file using ftp](https://discuss.elastic.co/t/logstash-to-read-file-using-ftp/70195)
**Author:** [@hearvishwas](https://discuss.elastic.co/u/hearvishwas)\ **Replies:** 3\ **Last updated:** [December 30, 2016, 9:08am UTC](https://discuss.elastic.co/t/logstash-to-read-file-using-ftp/70195 "2016-12-30T09:08:05Z")
Hi All, I have a remote windows machine from which I have to read log files. But this windows machine has firewall and allow communication using ftp only. Can anyone suggest how to these log files using ftp protocol in l… --- ## [Logstash Security and Performance](https://discuss.elastic.co/t/logstash-security-and-performance/69912)
**Author:** [@JoeLi](https://discuss.elastic.co/u/JoeLi)\ **Replies:** 3\ **Last updated:** [December 30, 2016, 8:30am UTC](https://discuss.elastic.co/t/logstash-security-and-performance/69912 "2016-12-30T08:30:16Z")
I am going to setup ELK on production. So that I have to change to ssl and basic authentication. I upgrade to 5.x, looks greate. With X-Pack, security becomes so easy! Greate job! With x-pack, I have to set roles us… --- ## [Received an event that has a different character encoding than you configured](https://discuss.elastic.co/t/received-an-event-that-has-a-different-character-encoding-than-you-configured/70251)
**Author:** [@sid.sidhant44](https://discuss.elastic.co/u/sid.sidhant44)\ **Replies:** 0\ **Last updated:** [December 30, 2016, 5:05am UTC](https://discuss.elastic.co/t/received-an-event-that-has-a-different-character-encoding-than-you-configured/70251 "2016-12-30T05:05:42Z")
Received an event that has a different character encoding than you configured. {text=\>"log message" :expected\_charset=\>"UTF-8", :level=\>:warn } I have tried setting JAVA\_TOOL\_OPTIONS="-Dfile.encoding=UTF-8". // and "IS… --- ## [Logstash Memory Issue](https://discuss.elastic.co/t/logstash-memory-issue/70240)
**Author:** [@Jimmy\_Kang](https://discuss.elastic.co/u/Jimmy_Kang)\ **Replies:** 0\ **Last updated:** [December 29, 2016, 11:10pm UTC](https://discuss.elastic.co/t/logstash-memory-issue/70240 "2016-12-29T23:10:16Z")
I am using mutate to convert my json tags fields from an array to string so that it can be compared in output. However when I put this in, I am getting some errors involving memory: WARN: kafka.consumer.ConsumerFetch… --- ## [Copy a date from a filed to a second field](https://discuss.elastic.co/t/copy-a-date-from-a-filed-to-a-second-field/69968)
**Author:** [@sbruyere](https://discuss.elastic.co/u/sbruyere)\ **Replies:** 1\ **Last updated:** [December 28, 2016, 11:00pm UTC](https://discuss.elastic.co/t/copy-a-date-from-a-filed-to-a-second-field/69968 "2016-12-28T23:00:33Z")
Hi, I need to pass date from a filed to an other to measure time gap between two log file received via Http\_poller. I use the Memory plugin and Logstash 5.1. Here what I've tried : if \[ColName1\] == "Log Time" { … --- ## [Add\_tag and add\_field not working](https://discuss.elastic.co/t/add-tag-and-add-field-not-working/70238)
**Author:** [@mushi](https://discuss.elastic.co/u/mushi)\ **Replies:** 0\ **Last updated:** [December 29, 2016, 10:26pm UTC](https://discuss.elastic.co/t/add-tag-and-add-field-not-working/70238 "2016-12-29T22:26:02Z")
I'm using add\_tag and add\_field and I don't see any of the fields or tags in the resulting message. in my conf: filter { grok { match =\> { "message" =\> "%{SYSLOG5424LINE}" add\_tag =\> \[ "boo" \] }… --- ## [Logstash kafka - with ES as output](https://discuss.elastic.co/t/logstash-kafka-with-es-as-output/69753)
**Author:** [@taurs](https://discuss.elastic.co/u/taurs)\ **Replies:** 1\ **Last updated:** [December 29, 2016, 9:24pm UTC](https://discuss.elastic.co/t/logstash-kafka-with-es-as-output/69753 "2016-12-29T21:24:17Z")
I am trying to take the data from Kafka and push it to ES index. input { kafka { bootstrap\_servers =\> "XX.XXX.XXX.XXX:9092" topics =\> \["timeseries\_ht"\] } } output { elasticsearch { hosts =\> \["XX.XXX.XXX.XXX:9200"\] inde… --- ## [Logstash s3 input error](https://discuss.elastic.co/t/logstash-s3-input-error/68356)
**Author:** [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\ **Replies:** 12\ **Last updated:** [December 29, 2016, 2:29pm UTC](https://discuss.elastic.co/t/logstash-s3-input-error/68356 "2016-12-29T14:29:56Z")
I am using logstash input plugin to fetch logs from s3. Below is my config. input { s3 { type =\> "redshift-access-log" bucket =\> "xxxxxxxxxxxxx" prefix =\> "xxxxxxxxxxxxx" proxy\_uri =\> "xxxxxxxxxxxxx… --- ## [Filter dissect: mapping on double quotes](https://discuss.elastic.co/t/filter-dissect-mapping-on-double-quotes/67617)
**Author:** [@alex22](https://discuss.elastic.co/u/alex22)\ **Replies:** 2\ **Last updated:** [December 29, 2016, 12:56pm UTC](https://discuss.elastic.co/t/filter-dissect-mapping-on-double-quotes/67617 "2016-12-29T12:56:08Z")
Hi \*, im currently trying to get along with the dissect filter. So i have the following example log which i try to parse with dissect: "a b" 33 c And my dissect mapping looks like this mapping =\> { "message" =\> "… --- ## [Update Script in Elasticsearch Output Parsing Array as String](https://discuss.elastic.co/t/update-script-in-elasticsearch-output-parsing-array-as-string/56910)
**Author:** [@esamudio](https://discuss.elastic.co/u/esamudio)\ **Replies:** 4\ **Last updated:** [December 29, 2016, 11:22am UTC](https://discuss.elastic.co/t/update-script-in-elasticsearch-output-parsing-array-as-string/56910 "2016-12-29T11:22:30Z")
I have a document with the field "name\_servers" and "past\_name\_servers". If I have the following: script\_type =\> "inline" script =\> ' ctx.\_source.name\_servers = "%{name\_servers}"; ctx.\_source.past\_name\_se… --- ## [Grok Parse Failure when dealing with IPv6 addresses](https://discuss.elastic.co/t/grok-parse-failure-when-dealing-with-ipv6-addresses/69629)
**Author:** [@Rebelpyr7](https://discuss.elastic.co/u/Rebelpyr7)\ **Replies:** 1\ **Last updated:** [December 29, 2016, 3:02am UTC](https://discuss.elastic.co/t/grok-parse-failure-when-dealing-with-ipv6-addresses/69629 "2016-12-29T03:02:27Z")
Hello all! I’ve been slowly learning how to use the ELK stack and getting it to work with my PFsense logs and have gotten almost everything to work. Right now the only issue I have seems to be related to IPv6 addresses. … --- ## [Can Logstash jdbc inputs be processed in certain order and how to control that 1 input process has ended?](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856)
**Author:** [@iti](https://discuss.elastic.co/u/iti)\ **Replies:** 5\ **Last updated:** [December 28, 2016, 9:41pm UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856 "2016-12-28T21:41:18Z")
I have the following config: input { jdbc { ... staement =\> "select \* from tab\_1st" type =\> "1st"} jdbc { ... staement =\> "select \* from tab\_2nd" … --- ## [Logstash-codec-cloudtrail support for 5.x?](https://discuss.elastic.co/t/logstash-codec-cloudtrail-support-for-5-x/69314)
**Author:** [@Mike\_Garrison](https://discuss.elastic.co/u/Mike_Garrison)\ **Replies:** 5\ **Last updated:** [December 28, 2016, 8:54pm UTC](https://discuss.elastic.co/t/logstash-codec-cloudtrail-support-for-5-x/69314 "2016-12-28T20:54:35Z")
I'd like to be able to get my cloudtrail logs into ES via Logstash. It seems like this was fixed by the following pull request: Update to logstash 5.0 by patrobinson · Pull Request #11 · logstash-plugins/logstash-codec-c… --- ## [Filters in logstash for sending the logs to elastic search index](https://discuss.elastic.co/t/filters-in-logstash-for-sending-the-logs-to-elastic-search-index/69343)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 22\ **Last updated:** [December 28, 2016, 7:00pm UTC](https://discuss.elastic.co/t/filters-in-logstash-for-sending-the-logs-to-elastic-search-index/69343 "2016-12-28T19:00:41Z")
Hi I have configured syslog-ng which in turn receiving syslogs from different sources/devices like Windows, Linux, firewall devices etc. in port no 514. Syslog-ng is then forwarding all syslogs to Logstash server in… --- ## [Duplicate records when scaling logstash](https://discuss.elastic.co/t/duplicate-records-when-scaling-logstash/43252)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 5\ **Last updated:** [December 28, 2016, 5:03pm UTC](https://discuss.elastic.co/t/duplicate-records-when-scaling-logstash/43252 "2016-12-28T17:03:26Z")
Hi, I am using ELK stack like below LSF==\> logstash-Shipper--\>Redis--\>Indexer--\>nGinx--\>ES I have scaled shipper and indexers as below 1) CASE1 for all logs shipper.conf output { redis { … --- ## [Logstash input TCP with TLS and handshake](https://discuss.elastic.co/t/logstash-input-tcp-with-tls-and-handshake/70126)
**Author:** [@alesnav](https://discuss.elastic.co/u/alesnav)\ **Replies:** 2\ **Last updated:** [December 28, 2016, 12:54pm UTC](https://discuss.elastic.co/t/logstash-input-tcp-with-tls-and-handshake/70126 "2016-12-28T12:54:50Z")
Hello there! I am trying to set up Logstash input TCP plugin with TLS enabled to receive logs from rsyslog using rsyslog-gnutls; and encountered one problem related to SSL handshake. I tried with gnutls and openssl w… --- ## [Has Logstash been deprecated to use in Windows?](https://discuss.elastic.co/t/has-logstash-been-deprecated-to-use-in-windows/70102)
**Author:** [@gupta.yuvraj](https://discuss.elastic.co/u/gupta.yuvraj)\ **Replies:** 3\ **Last updated:** [December 28, 2016, 6:23am UTC](https://discuss.elastic.co/t/has-logstash-been-deprecated-to-use-in-windows/70102 "2016-12-28T06:23:29Z")
Hi I wanted to know whether Logstash has been completely deprecated to use with Windows OS such as Windows 8, Windows 10 etc. As per the documentation, there is no clarity whether Logstash is supported on Windows. Ho… --- ## [Using logstash to automatically generate log files](https://discuss.elastic.co/t/using-logstash-to-automatically-generate-log-files/70104)
**Author:** [@danielwalker](https://discuss.elastic.co/u/danielwalker)\ **Replies:** 1\ **Last updated:** [December 28, 2016, 6:07am UTC](https://discuss.elastic.co/t/using-logstash-to-automatically-generate-log-files/70104 "2016-12-28T06:07:36Z")
I am now using one server for logstash for generating and keeping log files based on yyyy-MM-dd.log format, without sending log data to elasticsearch, as we want to use our customized scripts to analyze the logs files. … --- ## [Logstash filter large message](https://discuss.elastic.co/t/logstash-filter-large-message/69878)
**Author:** [@n3uz](https://discuss.elastic.co/u/n3uz)\ **Replies:** 2\ **Last updated:** [December 28, 2016, 3:24am UTC](https://discuss.elastic.co/t/logstash-filter-large-message/69878 "2016-12-28T03:24:31Z")
I am a newbie Chinese ELK user,When message value is too large ,It costs too much CPU time to handle,I dont know how to fix it,but like below. ELK5.1.1 CentOS7 JDK8 log output INFO 16-12-22 13:34:40.998 - The user\[12… --- ## [Best Practice for building new version of an index and replacing existing using LogStash](https://discuss.elastic.co/t/best-practice-for-building-new-version-of-an-index-and-replacing-existing-using-logstash/69819)
**Author:** [@Airn5475](https://discuss.elastic.co/u/Airn5475)\ **Replies:** 4\ **Last updated:** [December 27, 2016, 8:04pm UTC](https://discuss.elastic.co/t/best-practice-for-building-new-version-of-an-index-and-replacing-existing-using-logstash/69819 "2016-12-27T20:04:58Z")
I am pulling entities out of a database and serializing them to .json using a C# console app. The .json files are then read by Filebeat which sends the data to LogStash, which pushes into Elasticsearch. Every 5 minutes… --- ## [Problem in my Groking](https://discuss.elastic.co/t/problem-in-my-groking/69582)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 4\ **Last updated:** [December 27, 2016, 2:42pm UTC](https://discuss.elastic.co/t/problem-in-my-groking/69582 "2016-12-27T14:42:39Z")
Hi, I have few entries in log that looks like that: I am trying to build a grok that will fits them all, but have a problem where the string is : Exception occurred in API invocation More that, I need to try and c… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1058) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1060)