# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=716 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 717 --- ## [Service Licensing: setting permissions to read access to configuration files](https://discuss.elastic.co/t/service-licensing-setting-permissions-to-read-access-to-configuration-files/141356)
**Author:** [@msaboor35](https://discuss.elastic.co/u/msaboor35)\ **Replies:** 0\ **Last updated:** [July 24, 2018, 10:36am UTC](https://discuss.elastic.co/t/service-licensing-setting-permissions-to-read-access-to-configuration-files/141356 "2018-07-24T10:36:26Z")
0 down vote favorite I am making a software solution using ELK stack. I am setting up a demo for a company but I don't want them to steal my configuration files of elasticsearch, logstash, kibana, filebeat, e.t.c. I a… --- ## [Commands](https://discuss.elastic.co/t/commands/141350)
**Author:** [@Arun\_Murugappan](https://discuss.elastic.co/u/Arun_Murugappan)\ **Replies:** 1\ **Last updated:** [July 24, 2018, 10:02am UTC](https://discuss.elastic.co/t/commands/141350 "2018-07-24T10:02:36Z")
what is the meaning of the following command? bin/logstash -b 2000 -w 8 --- ## [Metrics plugin](https://discuss.elastic.co/t/metrics-plugin/141344)
**Author:** [@Arun\_Murugappan](https://discuss.elastic.co/u/Arun_Murugappan)\ **Replies:** 1\ **Last updated:** [July 24, 2018, 9:59am UTC](https://discuss.elastic.co/t/metrics-plugin/141344 "2018-07-24T09:59:58Z")
what does the metric plugin do? --- ## [Updating data in elasticsearch with logstash](https://discuss.elastic.co/t/updating-data-in-elasticsearch-with-logstash/141346)
**Author:** [@Mohamed\_Sidibe](https://discuss.elastic.co/u/Mohamed_Sidibe)\ **Replies:** 1\ **Last updated:** [July 24, 2018, 9:58am UTC](https://discuss.elastic.co/t/updating-data-in-elasticsearch-with-logstash/141346 "2018-07-24T09:58:08Z")
Hello, I'm new on ELK, please can someone tell me how to update data in elasticsearch with logstash config. I'm working on app that send log data (userId, appVersion, date) to elasticsearch whenever client get logged. … --- ## [Add Logstash host name to JSON formatted logstream](https://discuss.elastic.co/t/add-logstash-host-name-to-json-formatted-logstream/140567)
**Author:** [@Hrast](https://discuss.elastic.co/u/Hrast)\ **Replies:** 3\ **Last updated:** [July 24, 2018, 9:31am UTC](https://discuss.elastic.co/t/add-logstash-host-name-to-json-formatted-logstream/140567 "2018-07-24T09:31:20Z")
I've got a pipeline: JSON formatted log file -\> Filebeat -\> Logstash -\> ES. I'd like to add the Logstash host name to the document before they are sent along to ES. I took a stab at it: default-pipeline.conf input { … --- ## [How to configure different logs from different servers](https://discuss.elastic.co/t/how-to-configure-different-logs-from-different-servers/141294)
**Author:** [@nandan\_bigdata](https://discuss.elastic.co/u/nandan_bigdata)\ **Replies:** 4\ **Last updated:** [July 24, 2018, 9:25am UTC](https://discuss.elastic.co/t/how-to-configure-different-logs-from-different-servers/141294 "2018-07-24T09:25:12Z")
Hello Team, I am having a little bit confusing concept which I want to clear now. I have separate Server for separate operations and tools such as:- IP\_ADDR 1: Apache / nginx IP\_ADDR 2: MySQL IP\_ADDR 3: Cassandra No… --- ## [Logtash-output-firehouse](https://discuss.elastic.co/t/logtash-output-firehouse/141320)
**Author:** [@laiyhman](https://discuss.elastic.co/u/laiyhman)\ **Replies:** 0\ **Last updated:** [July 24, 2018, 7:51am UTC](https://discuss.elastic.co/t/logtash-output-firehouse/141320 "2018-07-24T07:51:51Z")
Hi, Can anyone point me to the documentation/wiki page of logstash-output-firehose plugin ( https://github.com/logstash-plugins/logstash-output-firehose )? Couldn't find any. --- ## [Why the input-jdbc plugin automatically converts the MySQL database tinyint data type to boolean type?](https://discuss.elastic.co/t/why-the-input-jdbc-plugin-automatically-converts-the-mysql-database-tinyint-data-type-to-boolean-type/141100)
**Author:** [@Scopy](https://discuss.elastic.co/u/Scopy)\ **Replies:** 2\ **Last updated:** [July 24, 2018, 5:46am UTC](https://discuss.elastic.co/t/why-the-input-jdbc-plugin-automatically-converts-the-mysql-database-tinyint-data-type-to-boolean-type/141100 "2018-07-24T05:46:34Z")
Hi,everyone! I‘m a green hand to logstash.’This is the first attempt to synchronize MySQL data with logstash today.However, when I used the JDBC plugin, it encountered some problems. MySQL data is set to tinyint, it tur… --- ## [Character encoding](https://discuss.elastic.co/t/character-encoding/141209)
**Author:** [@safhw91](https://discuss.elastic.co/u/safhw91)\ **Replies:** 0\ **Last updated:** [July 23, 2018, 3:29pm UTC](https://discuss.elastic.co/t/character-encoding/141209 "2018-07-23T15:29:07Z")
Logstash: "Received an event that has a different character encoding than you configured." e.g. when trying to parse the character "ü" Is there any way to get logstash to parse this character as "\\xFC"? Logstash alread… --- ## [In logstash Replace elasticsearch's @timestamp with log's time](https://discuss.elastic.co/t/in-logstash-replace-elasticsearchs-timestamp-with-logs-time/140848)
**Author:** [@Haitianisgood](https://discuss.elastic.co/u/Haitianisgood)\ **Replies:** 7\ **Last updated:** [July 24, 2018, 3:59am UTC](https://discuss.elastic.co/t/in-logstash-replace-elasticsearchs-timestamp-with-logs-time/140848 "2018-07-24T03:59:27Z")
Hi , I want to replace @timestash of elasticsearch with my project log file! flilebeat Logstash Elasticsearch all are 6.2 My log format: 2018-07-20 11:40:41.196 WARN 20671 --- \[DiscoveryClient-InstanceInfoReplicato… --- ## [Logstash-output-s3 not working with auditbeat](https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194)
**Author:** [@computermaster0101](https://discuss.elastic.co/u/computermaster0101)\ **Replies:** 3\ **Last updated:** [July 24, 2018, 12:33am UTC](https://discuss.elastic.co/t/logstash-output-s3-not-working-with-auditbeat/140194 "2018-07-24T00:33:39Z")
I have setup auditbeat on a centos6 server. it ships to logstash which ships to s3. Below is my config input { beats { port =\> 9210 } } output { s3{ region =\> "us-east-1" bucket =\> "NAME-REMOVED" prefix =\> "aud… --- ## [Kv plugin not parsing null value](https://discuss.elastic.co/t/kv-plugin-not-parsing-null-value/138751)
**Author:** [@yodog](https://discuss.elastic.co/u/yodog)\ **Replies:** 5\ **Last updated:** [July 23, 2018, 11:23pm UTC](https://discuss.elastic.co/t/kv-plugin-not-parsing-null-value/138751 "2018-07-23T23:23:14Z")
using kv to parse iptables logs, i cant get it to save the MAC field \[2928868.448110\] \[iptables\] LOGDROP IN=eth0 OUT= MAC=00:50:56:9a:13:2a:02:1f:a0:00:0d:01:08:00 SRC=5.188.207.7 DST=8.8.8.8 LEN=60 TOS=0x08 PREC=0x20 … --- ## [Log stash Installation, Configuration and Running](https://discuss.elastic.co/t/log-stash-installation-configuration-and-running/141057)
**Author:** [@deepak11](https://discuss.elastic.co/u/deepak11)\ **Replies:** 2\ **Last updated:** [July 23, 2018, 9:47pm UTC](https://discuss.elastic.co/t/log-stash-installation-configuration-and-running/141057 "2018-07-23T21:47:15Z")
Hi, I have configured my logstash with simple pipeline configuration (followed Logstash documentation) when I am trying to run the command I am getting the below error. I am giving the following command at command line … --- ## [NGINX Logs not coming through while Syslogs are](https://discuss.elastic.co/t/nginx-logs-not-coming-through-while-syslogs-are/141223)
**Author:** [@tvoll](https://discuss.elastic.co/u/tvoll)\ **Replies:** 3\ **Last updated:** [July 23, 2018, 9:21pm UTC](https://discuss.elastic.co/t/nginx-logs-not-coming-through-while-syslogs-are/141223 "2018-07-23T21:21:42Z")
Just wanting some clarification on what could be causing this issue. I have Syslogs coming through just fine, but what it comes to the Nginx Logs, nothing is coming through to Elasticsearch. I believe that it might hav… --- ## [Jdbc plugin for postgresql issue](https://discuss.elastic.co/t/jdbc-plugin-for-postgresql-issue/141065)
**Author:** [@Harman\_Gurm](https://discuss.elastic.co/u/Harman_Gurm)\ **Replies:** 4\ **Last updated:** [July 23, 2018, 8:36pm UTC](https://discuss.elastic.co/t/jdbc-plugin-for-postgresql-issue/141065 "2018-07-23T20:36:55Z")
Hi, I am using jdbc plugin to fetch data from postgresql db, it seems to be work fine for first time and i am able to pull the data, but it is not working according to saved state, everytime all of data is queried and t… --- ## [Logstash HTTP output retries forever on failure](https://discuss.elastic.co/t/logstash-http-output-retries-forever-on-failure/141239)
**Author:** [@darafield](https://discuss.elastic.co/u/darafield)\ **Replies:** 4\ **Last updated:** [July 23, 2018, 7:32pm UTC](https://discuss.elastic.co/t/logstash-http-output-retries-forever-on-failure/141239 "2018-07-23T19:32:42Z")
We are using Logstash 6.2.4. We have two outputs configured: all data goes to Elasticsearch and some data also goes to an HTTP endpoint that is not always running when we are testing. In the case when the endpoint is not… --- ## [Parsing problem with both csv and dissect](https://discuss.elastic.co/t/parsing-problem-with-both-csv-and-dissect/141171)
**Author:** [@S\_Chase](https://discuss.elastic.co/u/S_Chase)\ **Replies:** 3\ **Last updated:** [July 23, 2018, 4:47pm UTC](https://discuss.elastic.co/t/parsing-problem-with-both-csv-and-dissect/141171 "2018-07-23T16:47:34Z")
for now we have snare sending windows events to a rsyslog server which then sends in json to my elastic stack. The delimiter is #011. ---example log --- Jul 23 03:24:19 dalaptop.mydomain.local MSWinEventLog#0111#011S… --- ## [Logstash doesn't receive any logs from filebeat](https://discuss.elastic.co/t/logstash-doesnt-receive-any-logs-from-filebeat/140796)
**Author:** [@Ferdous\_Shibly](https://discuss.elastic.co/u/Ferdous_Shibly)\ **Replies:** 2\ **Last updated:** [July 23, 2018, 4:21pm UTC](https://discuss.elastic.co/t/logstash-doesnt-receive-any-logs-from-filebeat/140796 "2018-07-23T16:21:06Z")
We are getting the following error message in Filebeat 2018-07-19T18:46:18.832Z ERROR logstash/async.go:235 Failed to publish events caused by: write tcp 172.17.0.2:42194-\>10.14.86.242:9191: write: connection reset… --- ## [Logstash Replace timestamp from csv](https://discuss.elastic.co/t/logstash-replace-timestamp-from-csv/141188)
**Author:** [@logesh\_j](https://discuss.elastic.co/u/logesh_j)\ **Replies:** 5\ **Last updated:** [July 23, 2018, 4:07pm UTC](https://discuss.elastic.co/t/logstash-replace-timestamp-from-csv/141188 "2018-07-23T16:07:23Z")
I am try to replace the timestamp with the field from log file. But it is not getting updated. Getting Date parse error \_dateparsefailure and still not getting updated. Attached the config file for reference. Date in th… --- ## [Unable to view all fields](https://discuss.elastic.co/t/unable-to-view-all-fields/141204)
**Author:** [@andonovj](https://discuss.elastic.co/u/andonovj)\ **Replies:** 1\ **Last updated:** [July 23, 2018, 3:45pm UTC](https://discuss.elastic.co/t/unable-to-view-all-fields/141204 "2018-07-23T15:45:10Z")
Hello Guys, I configured: Kibana, Elasticsearch and Logstash: logstash.yml: http.host: "10.132.0.4" xpack.monitoring.elasticsearch.url: \["http://10.132.0.4:9200"\] xpack.monitoring.elasticsearch.username: "elastic" … --- ## [Remove text on field](https://discuss.elastic.co/t/remove-text-on-field/141189)
**Author:** [@sysdradmin](https://discuss.elastic.co/u/sysdradmin)\ **Replies:** 4\ **Last updated:** [July 23, 2018, 3:31pm UTC](https://discuss.elastic.co/t/remove-text-on-field/141189 "2018-07-23T15:31:48Z")
hello, I have a csv file like this 2018-07-17 06:01:18 ID:784514|FD: .... As you can see it's missing a "|" between date/time and ID so first field is saw like a text I want that first field is a date/time and other … --- ## [Unable to parse my Fortigate Logs](https://discuss.elastic.co/t/unable-to-parse-my-fortigate-logs/138882)
**Author:** [@Kaviarasan](https://discuss.elastic.co/u/Kaviarasan)\ **Replies:** 7\ **Last updated:** [July 23, 2018, 3:27pm UTC](https://discuss.elastic.co/t/unable-to-parse-my-fortigate-logs/138882 "2018-07-23T15:27:56Z")
Hi Team, I am new to ELK. I have been trying to parse my Fortigate firewall logs with my my RegEx. I have Fortigate logs saved in my machine I have created a patterns directory in which I have my whole regex stuff I h… --- ## [Reserved fieldnames?](https://discuss.elastic.co/t/reserved-fieldnames/141200)
**Author:** [@rebort](https://discuss.elastic.co/u/rebort)\ **Replies:** 3\ **Last updated:** [July 23, 2018, 3:17pm UTC](https://discuss.elastic.co/t/reserved-fieldnames/141200 "2018-07-23T15:17:45Z")
i have trouble in a mutation filter. At 1st i put data trough a kv filter in a target field "kv". That works fine. After that i want to rename the fields e.g. mutate { rename =\> { "\[kv\]\[product\]" =\> "\[product\]" }} this… --- ## [Logstash Multi Pipeline breaking and logstash all pipelines completely stopping when specific Pipelines DB is down or DB details are changed](https://discuss.elastic.co/t/logstash-multi-pipeline-breaking-and-logstash-all-pipelines-completely-stopping-when-specific-pipelines-db-is-down-or-db-details-are-changed/141093)
**Author:** [@sumanbehara](https://discuss.elastic.co/u/sumanbehara)\ **Replies:** 0\ **Last updated:** [July 23, 2018, 6:52am UTC](https://discuss.elastic.co/t/logstash-multi-pipeline-breaking-and-logstash-all-pipelines-completely-stopping-when-specific-pipelines-db-is-down-or-db-details-are-changed/141093 "2018-07-23T06:52:58Z")
Hi logstash team, I have configured logstash Mutlipipeline for jdbc inputs and Pipeline is working fine and perfectly from couple of days. each pipeline contains multiple jdbc inputs. later after some days some how… --- ## [Json array input producing \_jsonparsefailure](https://discuss.elastic.co/t/json-array-input-producing-jsonparsefailure/140006)
**Author:** [@Scott\_Newby](https://discuss.elastic.co/u/Scott_Newby)\ **Replies:** 2\ **Last updated:** [July 23, 2018, 1:42pm UTC](https://discuss.elastic.co/t/json-array-input-producing-jsonparsefailure/140006 "2018-07-23T13:42:36Z")
We're trying to import a json file that starts as an array (starts with \[ and ends in \]. A truncated input file: \[ { "uri": "uri/feature", "id": "idvalue", "keyword": "Feature", "name": "name", "d… --- ## [Grok, If condition in elasticsearch section](https://discuss.elastic.co/t/grok-if-condition-in-elasticsearch-section/141186)
**Author:** [@igemon](https://discuss.elastic.co/u/igemon)\ **Replies:** 2\ **Last updated:** [July 23, 2018, 1:35pm UTC](https://discuss.elastic.co/t/grok-if-condition-in-elasticsearch-section/141186 "2018-07-23T13:35:07Z")
Hello guys! I'm trying to use if condition in the elasticsearch section. I want to write to different indexes depending on ip. what's wrong? :worried: elasticsearch{ if \[node\_ip\] == "10.6.108.100" or "10.6.108.101" { … --- ## [How to grok log file with 2 line formats?](https://discuss.elastic.co/t/how-to-grok-log-file-with-2-line-formats/141136)
**Author:** [@med\_amine](https://discuss.elastic.co/u/med_amine)\ **Replies:** 3\ **Last updated:** [July 23, 2018, 1:17pm UTC](https://discuss.elastic.co/t/how-to-grok-log-file-with-2-line-formats/141136 "2018-07-23T13:17:29Z")
iughikugkgkjk --- ## [Oracle jdbc driver](https://discuss.elastic.co/t/oracle-jdbc-driver/140829)
**Author:** [@seungtae](https://discuss.elastic.co/u/seungtae)\ **Replies:** 1\ **Last updated:** [July 23, 2018, 1:11pm UTC](https://discuss.elastic.co/t/oracle-jdbc-driver/140829 "2018-07-23T13:11:27Z")
Hello. I have an issue about jdbc filter for oracle. Who knows about this issue? filter{ jdbc\_streaming { jdbc\_driver\_library =\> "/usr/share/logstash/vendor/jar/jdbc/ojdbc6.jar" jdbc\_driver\_class =\> "java::ora… --- ## [Unable to filter csv file](https://discuss.elastic.co/t/unable-to-filter-csv-file/141169)
**Author:** [@jibinoosae](https://discuss.elastic.co/u/jibinoosae)\ **Replies:** 1\ **Last updated:** [July 23, 2018, 12:44pm UTC](https://discuss.elastic.co/t/unable-to-filter-csv-file/141169 "2018-07-23T12:44:08Z")
HI, New to ELK please help me to understand below error when i try to run logstash.conf file \[ERROR\] 2018-07-23 11:26:37.161 \[Converge PipelineAction::Create\] csv - Unknown setting 'seperator' for csv \[ERROR\] 2018-0… --- ## [Logstash xml split filter](https://discuss.elastic.co/t/logstash-xml-split-filter/141154)
**Author:** [@Naveen\_Reddy\_Sama](https://discuss.elastic.co/u/Naveen_Reddy_Sama)\ **Replies:** 5\ **Last updated:** [July 23, 2018, 12:42pm UTC](https://discuss.elastic.co/t/logstash-xml-split-filter/141154 "2018-07-23T12:42:36Z")
Hi team, using the below logstash confg file xml filter filter { xml { source =\> "message" target =\> "xml\_content" } split { field =\> "xml\_content\[station\]" } split { field =\> "xml\_content\[station\]\[name\]" } … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=715) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=717)