# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=840 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 841 --- ## [AM to PM timestamp conversion not working as expected](https://discuss.elastic.co/t/am-to-pm-timestamp-conversion-not-working-as-expected/115723)
**Author:** [@Ashok\_Nair](https://discuss.elastic.co/u/Ashok_Nair)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 1:33pm UTC](https://discuss.elastic.co/t/am-to-pm-timestamp-conversion-not-working-as-expected/115723 "2018-01-16T13:33:28Z")
Following is my input data stream: \ \ I am using this filter in logstash: grok{ matc… --- ## [Logstash, converting fields](https://discuss.elastic.co/t/logstash-converting-fields/115075)
**Author:** [@EditAxha](https://discuss.elastic.co/u/EditAxha)\ **Replies:** 3\ **Last updated:** [January 16, 2018, 1:11pm UTC](https://discuss.elastic.co/t/logstash-converting-fields/115075 "2018-01-16T13:11:44Z")
Hello i want to make some converts in my .log files, for example converting kilometer to meter. how can i do this? --- ## [How exactly logstash-file-input plugin works?](https://discuss.elastic.co/t/how-exactly-logstash-file-input-plugin-works/113891)
**Author:** [@shreepad](https://discuss.elastic.co/u/shreepad)\ **Replies:** 13\ **Last updated:** [January 16, 2018, 12:49pm UTC](https://discuss.elastic.co/t/how-exactly-logstash-file-input-plugin-works/113891 "2018-01-16T12:49:46Z")
Hi there, We are facing some delay(5mins to 1hr) between actual log file creation and same log(file) being available in kibana. We are using file input plugin to read logs and elasticsearch as output plugin. Every hour… --- ## [Records missing while stashing data from Logstash](https://discuss.elastic.co/t/records-missing-while-stashing-data-from-logstash/115704)
**Author:** [@vinaygarg30](https://discuss.elastic.co/u/vinaygarg30)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 12:11pm UTC](https://discuss.elastic.co/t/records-missing-while-stashing-data-from-logstash/115704 "2018-01-16T12:11:53Z")
I am using logstash 6.1.0 to make a data pipeline to stash data from DEV server(where mysql data is present using jdbc input plugin) to TEST server( where elastic search in installed). The config file for logstash is as… --- ## [Logstash plugin for cloudwatch\_logs](https://discuss.elastic.co/t/logstash-plugin-for-cloudwatch-logs/115701)
**Author:** [@Saurabh\_Taneja](https://discuss.elastic.co/u/Saurabh_Taneja)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 11:52am UTC](https://discuss.elastic.co/t/logstash-plugin-for-cloudwatch-logs/115701 "2018-01-16T11:52:58Z")
Hi, I couldn't find a official plugin for cloudwatch\_logs. I was using https://github.com/lukewaite/logstash-input-cloudwatch-logs but I am facing a lot of issues lately with this. Do you guys know a way to somehow sen… --- ## [ELK without logstash?](https://discuss.elastic.co/t/elk-without-logstash/115184)
**Author:** [@wweeder](https://discuss.elastic.co/u/wweeder)\ **Replies:** 2\ **Last updated:** [January 16, 2018, 11:02am UTC](https://discuss.elastic.co/t/elk-without-logstash/115184 "2018-01-16T11:02:39Z")
Is it possible to use ELK without logstash? I have filebeats, elasticsearch and kibana running without any problems, but inserting logstash into this mix seems like a real pain. I am new to ELK stack, and probably missi… --- ## [Logstash 5.6 Date filter target setting not working](https://discuss.elastic.co/t/logstash-5-6-date-filter-target-setting-not-working/115180)
**Author:** [@ajstark123](https://discuss.elastic.co/u/ajstark123)\ **Replies:** 1\ **Last updated:** [January 16, 2018, 10:57am UTC](https://discuss.elastic.co/t/logstash-5-6-date-filter-target-setting-not-working/115180 "2018-01-16T10:57:45Z")
We have a date field filebeat\_timestamp that has the dates in the following format: 2018-01-11T21:50:50.219Z. We want to convert the date to this format: January 11 2018, 16:50:50.219 . We want to make sure that we r… --- ## [Error When loading Logstash](https://discuss.elastic.co/t/error-when-loading-logstash/115639)
**Author:** [@Methsri\_Wickramarath](https://discuss.elastic.co/u/Methsri_Wickramarath)\ **Replies:** 5\ **Last updated:** [January 16, 2018, 9:38am UTC](https://discuss.elastic.co/t/error-when-loading-logstash/115639 "2018-01-16T09:38:00Z")
Hi All, Am getting the following error with Logstash Blockquote \[2018-01-16T11:32:17,216\]\[ERROR\]\[logstash.agent \] Cannot load an invalid configuration {:reason=\>"Expected one of #, input, filter, output at … --- ## [Why can't read file by input plugin](https://discuss.elastic.co/t/why-cant-read-file-by-input-plugin/115671)
**Author:** [@3a5a5881fd90e86035b7](https://discuss.elastic.co/u/3a5a5881fd90e86035b7)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 9:06am UTC](https://discuss.elastic.co/t/why-cant-read-file-by-input-plugin/115671 "2018-01-16T09:06:27Z")
Why can't I output a file (2.txt) from the input file(test.json)? When I execute the instruction below , I did't see any oupput in cmd. However, I can see the output by stdin and that record to 2.txt successfully. /bi… --- ## [How To receive a SOAP Response message from web service](https://discuss.elastic.co/t/how-to-receive-a-soap-response-message-from-web-service/114868)
**Author:** [@s1m0ne](https://discuss.elastic.co/u/s1m0ne)\ **Replies:** 5\ **Last updated:** [January 16, 2018, 9:05am UTC](https://discuss.elastic.co/t/how-to-receive-a-soap-response-message-from-web-service/114868 "2018-01-16T09:05:42Z")
Hi Folks, NOTE: Logstash version 5.6.2 Currently, I receive events in (json format) from apache kafka and I use logstash with HTTP Output plugin for sending SOAP Request message to Web Service, via WSDL. output { … --- ## [Restheart issue with \_embedded http\_poller](https://discuss.elastic.co/t/restheart-issue-with-embedded-http-poller/115663)
**Author:** [@mahajan344](https://discuss.elastic.co/u/mahajan344)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 8:26am UTC](https://discuss.elastic.co/t/restheart-issue-with-embedded-http-poller/115663 "2018-01-16T08:26:07Z")
Hi, I want to insert data from MongoDB to ElasticSearch using Logstash. Since there is no input plugin for MongoDB, I am using http\_poller for input plugin which will communicate with MongoDB http interface restheart. N… --- ## [Retreiving an array from nested field](https://discuss.elastic.co/t/retreiving-an-array-from-nested-field/115555)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 1\ **Last updated:** [January 16, 2018, 8:11am UTC](https://discuss.elastic.co/t/retreiving-an-array-from-nested-field/115555 "2018-01-16T08:11:03Z")
I have data coming in in JSON format with a nested field which holds an array I would like to retrieve. Currently I am extracting the data from nested field via mutate plugin, such as: mutate { add\_field =\> { "nested\_… --- ## [Convert time according to timezone](https://discuss.elastic.co/t/convert-time-according-to-timezone/115648)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 1\ **Last updated:** [January 16, 2018, 7:54am UTC](https://discuss.elastic.co/t/convert-time-according-to-timezone/115648 "2018-01-16T07:54:45Z")
as for +0100 timezone we use : match =\> \["timestamp", "yyyy-MMM-dd HH:mm:ss"\] target =\> "@timestamp" timezone =\> "Europe/Berlin" what to use for +0200 timezone a help would be really appreciated Regards Shrikant --- ## [Logstash time filter not working](https://discuss.elastic.co/t/logstash-time-filter-not-working/115068)
**Author:** [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Replies:** 2\ **Last updated:** [January 16, 2018, 6:41am UTC](https://discuss.elastic.co/t/logstash-time-filter-not-working/115068 "2018-01-16T06:41:55Z")
My CSV xyz.com,1/1/2018 12:17:37 PM,62 xyz.com,7/15/2017 1:11:34 AM,62 xyz.com,7/15/2017 1:06:34 AM,62 xyz.com,7/15/2017 1:01:34 AM,78 Filter filter { csv { separator =\> "," columns =\> \[ "URL", "Date", "Respons… --- ## [Logstash Jdbc input not updating last run](https://discuss.elastic.co/t/logstash-jdbc-input-not-updating-last-run/115108)
**Author:** [@karthik\_s1](https://discuss.elastic.co/u/karthik_s1)\ **Replies:** 6\ **Last updated:** [January 16, 2018, 6:22am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-not-updating-last-run/115108 "2018-01-16T06:22:51Z")
Hi, We are pulling data from mysql and pushing to elasticsearch through logstash. Following is the configuration for jdbc input: jdbc\_driver\_library =\> '/home/software/mysql-connector-java-5.1.42/mysql-connector-java-… --- ## [Json\_parse\_exception](https://discuss.elastic.co/t/json-parse-exception/115640)
**Author:** [@karthik\_s1](https://discuss.elastic.co/u/karthik_s1)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 6:19am UTC](https://discuss.elastic.co/t/json-parse-exception/115640 "2018-01-16T06:19:44Z")
I am getting the following json\_parse\_exception while doing a bulk request from logstash to elasticsearch. \_\[ERROR\]\[logstash.outputs.elasticsearch\] Encountered a retryable error. Will Retry with exponential backoff {:c… --- ## [Patterns : Logstash - Regular Expression HELP!](https://discuss.elastic.co/t/patterns-logstash-regular-expression-help/115269)
**Author:** [@Methsri\_Wickramarath](https://discuss.elastic.co/u/Methsri_Wickramarath)\ **Replies:** 2\ **Last updated:** [January 16, 2018, 6:14am UTC](https://discuss.elastic.co/t/patterns-logstash-regular-expression-help/115269 "2018-01-16T06:14:15Z")
I'm using the Logstash "pattern" to select out a specific pattern inside the "message" field. Following is the message Blockquote January 12th 2018, 15:47:52.228 Jan 12 15:47:51 198.18.109.70 interface,info ether7 l… --- ## [Method for excluding event that cannot be used as Time Filter](https://discuss.elastic.co/t/method-for-excluding-event-that-cannot-be-used-as-time-filter/115635)
**Author:** [@j\_ram](https://discuss.elastic.co/u/j_ram)\ **Replies:** 1\ **Last updated:** [January 16, 2018, 6:12am UTC](https://discuss.elastic.co/t/method-for-excluding-event-that-cannot-be-used-as-time-filter/115635 "2018-01-16T06:12:27Z")
I am currently running a script across my network that parses windows event logs. It stores the "SystemTime" in a csv for me. The time should look something like the following: "2017-12-21T04:59:51.667607100Z". I am cur… --- ## [Logstasg KV](https://discuss.elastic.co/t/logstasg-kv/115537)
**Author:** [@neo\_levi](https://discuss.elastic.co/u/neo_levi)\ **Replies:** 6\ **Last updated:** [January 16, 2018, 4:24am UTC](https://discuss.elastic.co/t/logstasg-kv/115537 "2018-01-16T04:24:09Z")
Hi All, i am new to logstash kv , i am tring to parse message that look like {"HOSTNAME":"XXX","level":"INFO","logger\_name":"elkUaLog","serviceName":"services-log","profile":"localdev","eurekaInstanceId":"services-log:2… --- ## [Custom pattern](https://discuss.elastic.co/t/custom-pattern/115617)
**Author:** [@zpraxis](https://discuss.elastic.co/u/zpraxis)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 1:04am UTC](https://discuss.elastic.co/t/custom-pattern/115617 "2018-01-16T01:04:27Z")
Hi! I'm defining a custom pattern like the following, and at the same time trying to add a data type conversion. So far, I've tried these without success: code=(?\%{POSINT}|null|NULL|%{SPACE}) code=(?\ **Author:** [@fij29dkvn28](https://discuss.elastic.co/u/fij29dkvn28)\ **Replies:** 0\ **Last updated:** [January 16, 2018, 12:56am UTC](https://discuss.elastic.co/t/format-of-timestamp/115616 "2018-01-16T00:56:34Z") Hi, I was wondering whether I could define @timestamp in the log so I don't have to parse it with the date filter in logstash. Is this possible? If so, what's the time/date format for it? TIA! --- ## [Help in using logstash](https://discuss.elastic.co/t/help-in-using-logstash/115190)
**Author:** [@kanthimathi](https://discuss.elastic.co/u/kanthimathi)\ **Replies:** 5\ **Last updated:** [January 15, 2018, 10:30pm UTC](https://discuss.elastic.co/t/help-in-using-logstash/115190 "2018-01-15T22:30:14Z")
Hi all, Currently, i am using filebeat and metricbeat to get logs from kubernetes into the elastic cloud. I wish to test this using logstash now. Can somebody guide me how to pull the data? I am aware of setting up the … --- ## [Found a bug in date parsing!](https://discuss.elastic.co/t/found-a-bug-in-date-parsing/115311)
**Author:** [@pgervais](https://discuss.elastic.co/u/pgervais)\ **Replies:** 6\ **Last updated:** [January 15, 2018, 5:34pm UTC](https://discuss.elastic.co/t/found-a-bug-in-date-parsing/115311 "2018-01-15T17:34:46Z")
Update: As stated below, I have a file with 3,951,165 csv llines that process properly with the config file shown below. In this, I get 153 lines that fail with dateparsefailure. After looking at the ruby debug output, … --- ## [Importing MySQL in ES](https://discuss.elastic.co/t/importing-mysql-in-es/115126)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 10\ **Last updated:** [January 15, 2018, 3:32pm UTC](https://discuss.elastic.co/t/importing-mysql-in-es/115126 "2018-01-15T15:32:54Z")
Hi, I'm trying to enrich data I have in ES with data I have in a MySQL database. I have a unique id for my ES documents that I can use to reference the data in the SQL db. My ES data has just a uniqueid and time, and the… --- ## [Difference between timestamp](https://discuss.elastic.co/t/difference-between-timestamp/115265)
**Author:** [@Robert\_Naccache](https://discuss.elastic.co/u/Robert_Naccache)\ **Replies:** 2\ **Last updated:** [January 15, 2018, 11:58am UTC](https://discuss.elastic.co/t/difference-between-timestamp/115265 "2018-01-15T11:58:34Z")
Hi, I've been struggling with this for a couple of days. I'm importing from elasticsearch to elasticsearch. In Elasticsearch i have data in the from of: jobid: 1000 buisnessevent: getjob alerteventtime: 2018-01-11T… --- ## [Logstash 6.1: XML parsing doesn't work](https://discuss.elastic.co/t/logstash-6-1-xml-parsing-doesnt-work/114218)
**Author:** [@Saket\_Kumar](https://discuss.elastic.co/u/Saket_Kumar)\ **Replies:** 2\ **Last updated:** [January 15, 2018, 11:44am UTC](https://discuss.elastic.co/t/logstash-6-1-xml-parsing-doesnt-work/114218 "2018-01-15T11:44:58Z")
I am extracting array tags from xml . the defined xpath doesn't capture. Config file: input { stdin { type =\> "02Jan2018122417\_180102\_3R\_3dd37a7cf4a843a6dc90e56822b31fe6\_Swissotel-Desktop\_WebpageSummary.xml" codec =\>… --- ## [Logstash doesn't seem to recive file via TCP from nodejs](https://discuss.elastic.co/t/logstash-doesnt-seem-to-recive-file-via-tcp-from-nodejs/115532)
**Author:** [@yishain11](https://discuss.elastic.co/u/yishain11)\ **Replies:** 0\ **Last updated:** [January 15, 2018, 11:32am UTC](https://discuss.elastic.co/t/logstash-doesnt-seem-to-recive-file-via-tcp-from-nodejs/115532 "2018-01-15T11:32:35Z")
Hi everyone! I try to sent some files from node js app to logstash input tcp. What the node file do? it fetches a minified json, and try to send it. Even though the connections seems to be establishited and no errors … --- ## [Drop lines when matching a word (multiline)](https://discuss.elastic.co/t/drop-lines-when-matching-a-word-multiline/115524)
**Author:** [@cilzzz](https://discuss.elastic.co/u/cilzzz)\ **Replies:** 0\ **Last updated:** [January 15, 2018, 11:09am UTC](https://discuss.elastic.co/t/drop-lines-when-matching-a-word-multiline/115524 "2018-01-15T11:09:04Z")
Hey i have a log file i used the multiline filter to drop all the lines before the matched word filter { multiline { type =\> "somefiletype" pattern =\> "\\W\*(Copy complete.)\\W\*" what =\> "previous" } } i … --- ## [Codec line delimiter by regex](https://discuss.elastic.co/t/codec-line-delimiter-by-regex/114809)
**Author:** [@Nofar](https://discuss.elastic.co/u/Nofar)\ **Replies:** 3\ **Last updated:** [January 15, 2018, 9:55am UTC](https://discuss.elastic.co/t/codec-line-delimiter-by-regex/114809 "2018-01-15T09:55:02Z")
Is it possible? I need to create delimiter like this: \<9\> or \<13\> codec =\> line { delimiter =\> "\<\\d+\>" } It relate to this limiter as is, and expect to receive "\<\\d+\>" string... --- ## [\[Logstash-6.1.1\] Disable monitoring-license-manager](https://discuss.elastic.co/t/logstash-6-1-1-disable-monitoring-license-manager/115505)
**Author:** [@dorinand](https://discuss.elastic.co/u/dorinand)\ **Replies:** 0\ **Last updated:** [January 15, 2018, 9:21am UTC](https://discuss.elastic.co/t/logstash-6-1-1-disable-monitoring-license-manager/115505 "2018-01-15T09:21:40Z")
Hi, after installing x-pack in logstash, I am receiving errors: \[ERROR\] 2018-01-15 10:07:13.045 \[monitoring-license-manager\] licensemanager - Unable to retrieve license information from license server {:message=\>"No Ava… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=839) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=841)