# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=983 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 984 --- ## [Mutual TLS FileBeat to LogStash fails with "remote error: tls: internal error"](https://discuss.elastic.co/t/mutual-tls-filebeat-to-logstash-fails-with-remote-error-tls-internal-error/85271)
**Author:** [@maglo](https://discuss.elastic.co/u/maglo)\ **Replies:** 2\ **Last updated:** [May 16, 2017, 7:00am UTC](https://discuss.elastic.co/t/mutual-tls-filebeat-to-logstash-fails-with-remote-error-tls-internal-error/85271 "2017-05-16T07:00:59Z")
I am trying to setup mutual authentication between FileBeat and LogStash as per https://www.elastic.co/guide/en/beats/filebeat/current/configuring-ssl-logstash.html I have generated certificates and keys using two diff… --- ## [Generate @timestamp in-logstash-by-concatenating-date-from-filename-and-time-from-logs](https://discuss.elastic.co/t/generate-timestamp-in-logstash-by-concatenating-date-from-filename-and-time-from-logs/82556)
**Author:** [@shammi](https://discuss.elastic.co/u/shammi)\ **Replies:** 21\ **Last updated:** [May 16, 2017, 5:49am UTC](https://discuss.elastic.co/t/generate-timestamp-in-logstash-by-concatenating-date-from-filename-and-time-from-logs/82556 "2017-05-16T05:49:53Z")
Hi, I have log file name 'log.20170410.123146.txt' and time in log entry. I want to merge date from filename and time from logs to generate @timestamp in logstash. grok { match =\> \["message", "^(?\ **Author:** [@cwhite](https://discuss.elastic.co/u/cwhite)\ **Replies:** 1\ **Last updated:** [May 16, 2017, 5:15am UTC](https://discuss.elastic.co/t/metadata-beats-not-sending-data-to-filebeat-index/85592 "2017-05-16T05:15:07Z") New to Elastic. When I am trying to output filebeat log file thru logstash with the following config: output { elasticsearch { hosts =\> "localhost:9200" manage\_template =\> false index =\> "%{\[metadata\]\[beat\]}-%{+YY… --- ## [Logstash adding duplicate rows](https://discuss.elastic.co/t/logstash-adding-duplicate-rows/85572)
**Author:** [@karim](https://discuss.elastic.co/u/karim)\ **Replies:** 1\ **Last updated:** [May 16, 2017, 5:14am UTC](https://discuss.elastic.co/t/logstash-adding-duplicate-rows/85572 "2017-05-16T05:14:08Z")
Hello , We implemented a solution to push Microsoft SQL Data in form of simple rows from SQL Server to Logstash which will index the data to Elasticsearch. The data (SQL rows and columns) are getting successfully by us… --- ## [Logstash set timestamp from json sub member field](https://discuss.elastic.co/t/logstash-set-timestamp-from-json-sub-member-field/85843)
**Author:** [@rpuserh](https://discuss.elastic.co/u/rpuserh)\ **Replies:** 1\ **Last updated:** [May 16, 2017, 4:26am UTC](https://discuss.elastic.co/t/logstash-set-timestamp-from-json-sub-member-field/85843 "2017-05-16T04:26:21Z")
Hi, I have json like this {system: {time:2017-05-15 12:12:33,version....}...} I have config for logstash like this date { match =\> \["system.time", "yyyy-MM-dd HH:mm:ss"\] target =\> "@timestamp" } But this is not m… --- ## [Creating geoip data for internal networks](https://discuss.elastic.co/t/creating-geoip-data-for-internal-networks/729)
**Author:** [@JimCheetham](https://discuss.elastic.co/u/JimCheetham)\ **Replies:** 11\ **Last updated:** [May 15, 2017, 11:42pm UTC](https://discuss.elastic.co/t/creating-geoip-data-for-internal-networks/729 "2017-05-15T23:42:48Z")
I have many internal users, and both public and private address space in use. I would like to be able to usefully separate my networks from the country we're in, and provide sensible geoip data. For my public range, I … --- ## [Grok filter - issues with spaces and special charecters](https://discuss.elastic.co/t/grok-filter-issues-with-spaces-and-special-charecters/85805)
**Author:** [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\ **Replies:** 10\ **Last updated:** [May 15, 2017, 9:50pm UTC](https://discuss.elastic.co/t/grok-filter-issues-with-spaces-and-special-charecters/85805 "2017-05-15T21:50:19Z")
Hi , I am having issues with skipping spaces and special characters which i am testing . I am trying to parse INPUT :- \[2017-05-15 00:00:07,397\] :|: INFO :|: dubprdsfe33.dub.jabodo.com I am using followi… --- ## [Create a .raw field for a given field via Dynamic Template](https://discuss.elastic.co/t/create-a-raw-field-for-a-given-field-via-dynamic-template/85822)
**Author:** [@KingBob](https://discuss.elastic.co/u/KingBob)\ **Replies:** 0\ **Last updated:** [May 15, 2017, 9:17pm UTC](https://discuss.elastic.co/t/create-a-raw-field-for-a-given-field-via-dynamic-template/85822 "2017-05-15T21:17:56Z")
Hello, I would like to modify the default Logstash template to create a host.raw field that is not analyzed (so it can be used in a terms query from Grafana). I tried the following but it is not working. Can anyone… --- ## [Groking Issue with Multiple data formats](https://discuss.elastic.co/t/groking-issue-with-multiple-data-formats/85018)
**Author:** [@kamran.matloob](https://discuss.elastic.co/u/kamran.matloob)\ **Replies:** 2\ **Last updated:** [May 15, 2017, 8:36pm UTC](https://discuss.elastic.co/t/groking-issue-with-multiple-data-formats/85018 "2017-05-15T20:36:19Z")
Hi here ! I have multiple formats in one file to grok.Below is the one type of data 2017-03-23 08:51:43,088 INFO \[ActiveMQ Session Task-142\] INVOICE02-SAP Vendor Number: SAP Vendor Number Query Result \[{DUNS\_ID=750944… --- ## [Custom log file not getting pushed to Elasticsearch via logstash](https://discuss.elastic.co/t/custom-log-file-not-getting-pushed-to-elasticsearch-via-logstash/85469)
**Author:** [@amit.karir](https://discuss.elastic.co/u/amit.karir)\ **Replies:** 4\ **Last updated:** [May 15, 2017, 8:21pm UTC](https://discuss.elastic.co/t/custom-log-file-not-getting-pushed-to-elasticsearch-via-logstash/85469 "2017-05-15T20:21:23Z")
Hi, I am trying to push a custom log file generated by my program into elasticsearch(5.3.2) via logstash(5.3.2). I dont get any error but index is also not getting created... Am I missing something? I am pretty new t… --- ## [Insert into influx on match](https://discuss.elastic.co/t/insert-into-influx-on-match/84328)
**Author:** [@bob\_monty](https://discuss.elastic.co/u/bob_monty)\ **Replies:** 3\ **Last updated:** [May 15, 2017, 6:09pm UTC](https://discuss.elastic.co/t/insert-into-influx-on-match/84328 "2017-05-15T18:09:34Z")
Greetings i have data being collected by logstash and depending on the string match i will insert into one influx or another for example: if \[@influx\_measurement\] =~ /(ABC\_.\*) then insert into influxA etc... bit i … --- ## [Logstash "Age" Filter Plugin](https://discuss.elastic.co/t/logstash-age-filter-plugin/84602)
**Author:** [@bt3388](https://discuss.elastic.co/u/bt3388)\ **Replies:** 6\ **Last updated:** [May 15, 2017, 6:01pm UTC](https://discuss.elastic.co/t/logstash-age-filter-plugin/84602 "2017-05-15T18:01:22Z")
I am hoping to get some help getting the age plugin to work. It seems like a really new filter plugin without a log of documentation. - https://www.elastic.co/guide/en/logstash/current/plugins-filters-age.html Here is m… --- ## [Stopping pipeline after sending data](https://discuss.elastic.co/t/stopping-pipeline-after-sending-data/84563)
**Author:** [@ulysse07](https://discuss.elastic.co/u/ulysse07)\ **Replies:** 2\ **Last updated:** [May 15, 2017, 3:39pm UTC](https://discuss.elastic.co/t/stopping-pipeline-after-sending-data/84563 "2017-05-15T15:39:26Z")
Hi everyone, When I launch logstash on Windows with the config file below, the pipeline is created, the data are send to elasticsearch but the pipeline stopped few seconds after. I ran it in the verbose mod et I have o… --- ## [How to use Multiline codec and Json at the same time in a config file?](https://discuss.elastic.co/t/how-to-use-multiline-codec-and-json-at-the-same-time-in-a-config-file/85692)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 4\ **Last updated:** [May 15, 2017, 2:58pm UTC](https://discuss.elastic.co/t/how-to-use-multiline-codec-and-json-at-the-same-time-in-a-config-file/85692 "2017-05-15T14:58:55Z")
Hi, I have logs in the following format, { "Name": "xxx", "Profession": "Developer", "Designation": "Senior", "StartDate": "20170317", "Salary": "xxxx" } { "Name": "xxx", "Profession": "Programmer", "Designatio… --- ## [Input method HTTP on ssl](https://discuss.elastic.co/t/input-method-http-on-ssl/85790)
**Author:** [@sahar\_q](https://discuss.elastic.co/u/sahar_q)\ **Replies:** 0\ **Last updated:** [May 15, 2017, 2:54pm UTC](https://discuss.elastic.co/t/input-method-http-on-ssl/85790 "2017-05-15T14:54:39Z")
Hi everyone, I've read about the http input method but i dont quite get how do i use it with ssl? a clearer explanation would be much appreciated . --- ## [How to get JDBC Data from Multiple Queries](https://discuss.elastic.co/t/how-to-get-jdbc-data-from-multiple-queries/25631)
**Author:** [@Kintas](https://discuss.elastic.co/u/Kintas)\ **Replies:** 4\ **Last updated:** [May 15, 2017, 1:59pm UTC](https://discuss.elastic.co/t/how-to-get-jdbc-data-from-multiple-queries/25631 "2017-05-15T13:59:46Z")
Hello, I'm trying to collect data from a DB (Derby, MySQL, Oracle) and my Schema is like this: the main table is "exchange" and I need to get data from all other tables. I'm trying to use logstash-input-jdbc, but wi… --- ## [Why is my timefield entry and @timestamp are different?](https://discuss.elastic.co/t/why-is-my-timefield-entry-and-timestamp-are-different/85741)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [May 15, 2017, 1:49pm UTC](https://discuss.elastic.co/t/why-is-my-timefield-entry-and-timestamp-are-different/85741 "2017-05-15T13:49:16Z")
Hi, I have a timefield in log entry and I have targetted it to @timestamp in Logstash, but they appear different. For example; time entry in log =\> 2017-05-15T10:14:58.992Z @timestamp =\> 2017-05-15T10:30:56.015Z below… --- ## [Get data from different servers with PATH command in logstash?](https://discuss.elastic.co/t/get-data-from-different-servers-with-path-command-in-logstash/85641)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 8\ **Last updated:** [May 15, 2017, 1:10pm UTC](https://discuss.elastic.co/t/get-data-from-different-servers-with-path-command-in-logstash/85641 "2017-05-15T13:10:22Z")
Hi, I have my ES cluster in 3 nodes(i.e.Each node in each VM) , i had enabled the slowlogs in all the node and keep slowlogs in the PATH: /var/es/slowlogs.log . So , what i had did is i writtten a logstash script to p… --- ## [Timestamp taking current time in case of a \_grokparsefailure, want log time instead](https://discuss.elastic.co/t/timestamp-taking-current-time-in-case-of-a--grokparsefailure-want-log-time-instead/85718)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 1\ **Last updated:** [May 15, 2017, 9:41am UTC](https://discuss.elastic.co/t/timestamp-taking-current-time-in-case-of-a--grokparsefailure-want-log-time-instead/85718 "2017-05-15T09:41:29Z")
Hi, My log file looks like: mydomain.com 125.15.122.12 - - \[04/Dec/2013:07:35:39 -0600\] "GET /images/icon.png HTTP/1.1" 200 358 "https://mydomain.com/welcome" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Tride… --- ## [Grok pattern for space OR double space](https://discuss.elastic.co/t/grok-pattern-for-space-or-double-space/85725)
**Author:** [@Saplog](https://discuss.elastic.co/u/Saplog)\ **Replies:** 8\ **Last updated:** [May 15, 2017, 9:33am UTC](https://discuss.elastic.co/t/grok-pattern-for-space-or-double-space/85725 "2017-05-15T09:33:53Z")
Hi, I have logs like that , Wed May 10 02:50:43 2017 Wed May 3 12:23:21 2017 My grok pattern works when they are only 1 space , like the "Wed May 10 02:50:43 2017" but with "Wed May 3 12:23:21 2017", it doesn't work. … --- ## [Why is my Logstash using huge CPU and how to fix it?](https://discuss.elastic.co/t/why-is-my-logstash-using-huge-cpu-and-how-to-fix-it/85710)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 5\ **Last updated:** [May 15, 2017, 9:25am UTC](https://discuss.elastic.co/t/why-is-my-logstash-using-huge-cpu-and-how-to-fix-it/85710 "2017-05-15T09:25:36Z")
Hi, I am using ELK GA 5.0.0 in my Linux box. I am using 2 logstash instances to pull from Kafka, and it is having high CPU utilization. I tried /\_node/hot\_threads?human=true and below are the results; node 1 ::: {}… --- ## [Logstash best practice](https://discuss.elastic.co/t/logstash-best-practice/85425)
**Author:** [@talial](https://discuss.elastic.co/u/talial)\ **Replies:** 8\ **Last updated:** [May 15, 2017, 8:05am UTC](https://discuss.elastic.co/t/logstash-best-practice/85425 "2017-05-15T08:05:38Z")
Hi, we have a 5 nodes elasticsearch cluster running 5.2.0, 2 of the nodes are configured as client nodes (data=false, master=false), 3 nodes are elasticsearch cluster (data=true, master=true) We want to install also l… --- ## [I am using default mapping but i am getting splited log messages](https://discuss.elastic.co/t/i-am-using-default-mapping-but-i-am-getting-splited-log-messages/85521)
**Author:** [@Anis\_Miled](https://discuss.elastic.co/u/Anis_Miled)\ **Replies:** 2\ **Last updated:** [May 15, 2017, 8:00am UTC](https://discuss.elastic.co/t/i-am-using-default-mapping-but-i-am-getting-splited-log-messages/85521 "2017-05-15T08:00:20Z")
Hello, I am using default mapping, my log message is like (for example): 2017-04-25 10:12:56.366 WARN 2989 --- \[DiscoveryClient-InstanceInfoReplicator-0\] com.netflix.discovery.DiscoveryClient : Can't get a respo… --- ## [Csv output problem with numbers?](https://discuss.elastic.co/t/csv-output-problem-with-numbers/85458)
**Author:** [@henrilabarre](https://discuss.elastic.co/u/henrilabarre)\ **Replies:** 2\ **Last updated:** [May 15, 2017, 7:58am UTC](https://discuss.elastic.co/t/csv-output-problem-with-numbers/85458 "2017-05-15T07:58:18Z")
Hi with ES 5.4 and LG 5.4 I try this output { csv { fields =\> \["nom","nom\_maire","CODGEO", "street", "cp","nomcommune","departement","DEPET","region","email","siteweb","tel","fax","population","location\[lat\]","l… --- ## [Logstash input jdbc plugin](https://discuss.elastic.co/t/logstash-input-jdbc-plugin/85659)
**Author:** [@FardinBehboudi](https://discuss.elastic.co/u/FardinBehboudi)\ **Replies:** 1\ **Last updated:** [May 15, 2017, 6:31am UTC](https://discuss.elastic.co/t/logstash-input-jdbc-plugin/85659 "2017-05-15T06:31:14Z")
Hi dear friends i have a very basic question, i wanna knoe how should i use Logstash input jdbc plugin? first install logstash and setup the configuration file and then install the plugin and try to index the data in e… --- ## [Need help in combining few grok](https://discuss.elastic.co/t/need-help-in-combining-few-grok/85647)
**Author:** [@cooling](https://discuss.elastic.co/u/cooling)\ **Replies:** 1\ **Last updated:** [May 15, 2017, 5:30am UTC](https://discuss.elastic.co/t/need-help-in-combining-few-grok/85647 "2017-05-15T05:30:04Z")
Hi, I have the log entry and the grok pattern as below. Log: 2017-04-11 18:31:41,938 | INFO | 195 | Process | Bundle Name | logStr: GUID: dl99X/WeN77E2SmyjH9uS1Fy+EDvFQ5R\_939bae | ReferenceID: 20170411183141500676… --- ## [How to write configuration two input for logstash with beats](https://discuss.elastic.co/t/how-to-write-configuration-two-input-for-logstash-with-beats/84816)
**Author:** [@Fly](https://discuss.elastic.co/u/Fly)\ **Replies:** 3\ **Last updated:** [May 15, 2017, 5:19am UTC](https://discuss.elastic.co/t/how-to-write-configuration-two-input-for-logstash-with-beats/84816 "2017-05-15T05:19:50Z")
Hi all, I have a model of two machines: one ubuntu server installed ELK server and one windows server 2012. I want to collect log of windows firewall and log of ubuntu system. When configuring the logstash pipeline to c… --- ## [Logstash is not open 5044 port on ubuntu server](https://discuss.elastic.co/t/logstash-is-not-open-5044-port-on-ubuntu-server/85660)
**Author:** [@Fly](https://discuss.elastic.co/u/Fly)\ **Replies:** 7\ **Last updated:** [May 15, 2017, 1:03am UTC](https://discuss.elastic.co/t/logstash-is-not-open-5044-port-on-ubuntu-server/85660 "2017-05-15T01:03:21Z")
Hi all, I configured the pipeline to collect windows firewall log of windows server 2012 as follows: etc/logstash/conf.d/10-input.conf input { beats { # The port to listen on for filebeat connections. port =\> 5044 … --- ## [Logstash JDBC plugin](https://discuss.elastic.co/t/logstash-jdbc-plugin/84874)
**Author:** [@calvinlyc](https://discuss.elastic.co/u/calvinlyc)\ **Replies:** 2\ **Last updated:** [May 15, 2017, 12:32am UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin/84874 "2017-05-15T00:32:38Z")
Hi guys, We are currently using LogStash JDBC plugin to feed the data between MYSQL and ElasticSearch. However, MySQL database has issues by using OFFSET with large set of data which can slowdown the performance by t… --- ## [Converting date to ISO8601](https://discuss.elastic.co/t/converting-date-to-iso8601/35347)
**Author:** [@Forgand](https://discuss.elastic.co/u/Forgand)\ **Replies:** 10\ **Last updated:** [May 14, 2017, 8:58pm UTC](https://discuss.elastic.co/t/converting-date-to-iso8601/35347 "2017-05-14T20:58:55Z")
Hi guys. I have a date converting problem. I have logs in this format: 2015.11.10 03:02:23.832: Some text. And I parse logs by grok: match =\> \[ "message", "%{YEAR:year}.%{MONTHNUM:month}.%{MONTHDAY:day}%{SPACE}%{TI… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=982) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=984)