# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=701 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 702 --- ## [Can i collect and parse TCP response messages from a server and port ? if so kindly help me with the config file and steps](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269)
**Author:** [@gopesh\_reddy](https://discuss.elastic.co/u/gopesh_reddy)\ **Replies:** 7\ **Last updated:** [August 16, 2018, 2:29am UTC](https://discuss.elastic.co/t/can-i-collect-and-parse-tcp-response-messages-from-a-server-and-port-if-so-kindly-help-me-with-the-config-file-and-steps/144269 "2018-08-16T02:29:03Z")
input { tcp { port =\> 5000 type =\> syslog } udp { port =\> 5000 type =\> syslog } } filter { if \[type\] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_host… --- ## [How to aggregate events containing array having duplicate hash in logstash](https://discuss.elastic.co/t/how-to-aggregate-events-containing-array-having-duplicate-hash-in-logstash/144590)
**Author:** [@pushanbhattacharya](https://discuss.elastic.co/u/pushanbhattacharya)\ **Replies:** 4\ **Last updated:** [August 16, 2018, 2:15am UTC](https://discuss.elastic.co/t/how-to-aggregate-events-containing-array-having-duplicate-hash-in-logstash/144590 "2018-08-16T02:15:17Z")
The structure of the data is- { brand\_id, { models \[ { model\_id, model\_number, model\_name } \] } } Below is the logstash output. C:\\Program Files\\Elasti… --- ## [How to check whether logsash is parsing data from a filebeat](https://discuss.elastic.co/t/how-to-check-whether-logsash-is-parsing-data-from-a-filebeat/144424)
**Author:** [@iamlearner123](https://discuss.elastic.co/u/iamlearner123)\ **Replies:** 3\ **Last updated:** [August 16, 2018, 1:18am UTC](https://discuss.elastic.co/t/how-to-check-whether-logsash-is-parsing-data-from-a-filebeat/144424 "2018-08-16T01:18:16Z")
Hi, Is there a way to know whether the log stash is parsing data from a particular filebeat or not. I am sending the data from file beat to logstash and there were configuration errors in logstash but i couldn't see th… --- ## [Logstash + OpenShift - issues with jdbc plugin when running as a non root user](https://discuss.elastic.co/t/logstash-openshift-issues-with-jdbc-plugin-when-running-as-a-non-root-user/144587)
**Author:** [@Constantine\_Abayeu](https://discuss.elastic.co/u/Constantine_Abayeu)\ **Replies:** 2\ **Last updated:** [August 15, 2018, 9:43pm UTC](https://discuss.elastic.co/t/logstash-openshift-issues-with-jdbc-plugin-when-running-as-a-non-root-user/144587 "2018-08-15T21:43:14Z")
Hi, I am having some issues with figuring out what is happening here. I am working on pushing some MS SQL data into elastic. here is my jdbc configuration in logstash. input { jdbc { jdbc\_connection\_string… --- ## [How to loop or iterate in Logstash output plugin](https://discuss.elastic.co/t/how-to-loop-or-iterate-in-logstash-output-plugin/142991)
**Author:** [@pushanbhattacharya](https://discuss.elastic.co/u/pushanbhattacharya)\ **Replies:** 2\ **Last updated:** [August 15, 2018, 9:38pm UTC](https://discuss.elastic.co/t/how-to-loop-or-iterate-in-logstash-output-plugin/142991 "2018-08-15T21:38:00Z")
So, if I have prepared a list or array of string data in LogStash filter using ElasticSearch plugin. I can save that array or list directly in another elasticsearch index from logstash output plugin. But that list actua… --- ## [Logstash - TimeStamp Normalization](https://discuss.elastic.co/t/logstash-timestamp-normalization/144531)
**Author:** [@Corey\_H](https://discuss.elastic.co/u/Corey_H)\ **Replies:** 4\ **Last updated:** [August 15, 2018, 9:02pm UTC](https://discuss.elastic.co/t/logstash-timestamp-normalization/144531 "2018-08-15T21:02:31Z")
Hi Everyone, Versions to get it out of the way, and I am not sure how to write this question up.. Wazuh Manager: 3.4.0 Filebeat and Logstash: 6.3.2 Elasticsearch 6.3.2 I have loaded the wazuh templates as per the in… --- ## [Format JSON line](https://discuss.elastic.co/t/format-json-line/144538)
**Author:** [@Ryan\_Groten](https://discuss.elastic.co/u/Ryan_Groten)\ **Replies:** 6\ **Last updated:** [August 15, 2018, 7:40pm UTC](https://discuss.elastic.co/t/format-json-line/144538 "2018-08-15T19:40:26Z")
Hey all, We have a kafka input that sends a single json line that look like this at regular intervals: {"load": {"min15": 0.05, "min5": 0.05, "cpucore": 2, "load\_warning": 1.0, "min1": 0.01, "load\_critical": 5.0, "load… --- ## [How to use current date to determine table name for jdbc\_input query?](https://discuss.elastic.co/t/how-to-use-current-date-to-determine-table-name-for-jdbc-input-query/144400)
**Author:** [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\ **Replies:** 3\ **Last updated:** [August 15, 2018, 7:28pm UTC](https://discuss.elastic.co/t/how-to-use-current-date-to-determine-table-name-for-jdbc-input-query/144400 "2018-08-15T19:28:00Z")
I have a database with a table per month for data: table\_01, table\_02, ..., table\_12 Is it possible to have logstash construct the table name at runtime, and use it in the query? --- ## [Curator configuration appears to delete too early](https://discuss.elastic.co/t/curator-configuration-appears-to-delete-too-early/144554)
**Author:** [@sandipatmca](https://discuss.elastic.co/u/sandipatmca)\ **Replies:** 1\ **Last updated:** [August 15, 2018, 6:13pm UTC](https://discuss.elastic.co/t/curator-configuration-appears-to-delete-too-early/144554 "2018-08-15T18:13:38Z")
Issue: Current configuration of “ unit\_count” in action.yml file is 180 days but when this file is invoked the logs are not getting deleted as per configured values. As per current configuration(unit\_count=180 days) , al… --- ## [Logstash json filter not working properly](https://discuss.elastic.co/t/logstash-json-filter-not-working-properly/144227)
**Author:** [@kmroz](https://discuss.elastic.co/u/kmroz)\ **Replies:** 19\ **Last updated:** [August 15, 2018, 5:24pm UTC](https://discuss.elastic.co/t/logstash-json-filter-not-working-properly/144227 "2018-08-15T17:24:49Z")
I am using all the Latest version of ELK , we are trying to parse json using the json filter. some of the data gets filtered properly but other fields have an error saying "objects in arrays are not well supported" Which… --- ## [Time field conversion](https://discuss.elastic.co/t/time-field-conversion/144542)
**Author:** [@Maddy\_M](https://discuss.elastic.co/u/Maddy_M)\ **Replies:** 4\ **Last updated:** [August 15, 2018, 5:06pm UTC](https://discuss.elastic.co/t/time-field-conversion/144542 "2018-08-15T17:06:10Z")
Hi, I have following line with timestamp and I am using grok filter, Could you please advise how to convert date into timestamp format? Line: time=2018-08-15-09-33-57 grok filter expression { %{DATA:time}=%{YEAR}-%… --- ## [How to force logstash to reread kafka topic?](https://discuss.elastic.co/t/how-to-force-logstash-to-reread-kafka-topic/144552)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 1\ **Last updated:** [August 15, 2018, 5:07pm UTC](https://discuss.elastic.co/t/how-to-force-logstash-to-reread-kafka-topic/144552 "2018-08-15T17:07:20Z")
Hi, Can someone please say how to force logstash to reread kafka topic? Just like sincedb\_path for file input plugin in Logstash, is there a seeting for Kafka Plugin? Please let me know.. Thanks --- ## [No host field when using Kafka input plugin](https://discuss.elastic.co/t/no-host-field-when-using-kafka-input-plugin/144287)
**Author:** [@fatemeh\_testing](https://discuss.elastic.co/u/fatemeh_testing)\ **Replies:** 5\ **Last updated:** [August 15, 2018, 2:35pm UTC](https://discuss.elastic.co/t/no-host-field-when-using-kafka-input-plugin/144287 "2018-08-15T14:35:04Z")
Hi, I'm using the following input input { udp { port =\> type =\> "rsyslog" } kafka { bootstrap\_servers =\> "IP-ADDRESS:PORT" topics =\> "sth" group\_id =\> "sth1" } } For UDP inputs, I receive a field named "hos… --- ## [How to get both key and value from JOSN uisng ruby](https://discuss.elastic.co/t/how-to-get-both-key-and-value-from-josn-uisng-ruby/144470)
**Author:** [@Charan\_Adabala](https://discuss.elastic.co/u/Charan_Adabala)\ **Replies:** 6\ **Last updated:** [August 15, 2018, 2:09pm UTC](https://discuss.elastic.co/t/how-to-get-both-key-and-value-from-josn-uisng-ruby/144470 "2018-08-15T14:09:03Z")
Hi, I want both key and value from json using ruby in logstash, By using the belwo snippet, able to get only value not key. i need key also. Can you please help me: Event: {"result":{"entities":{"HOST-B765":"apple"}}} … --- ## [Issues with multiple winlogbeat parsing](https://discuss.elastic.co/t/issues-with-multiple-winlogbeat-parsing/144522)
**Author:** [@Marcos\_Felix](https://discuss.elastic.co/u/Marcos_Felix)\ **Replies:** 2\ **Last updated:** [August 15, 2018, 1:33pm UTC](https://discuss.elastic.co/t/issues-with-multiple-winlogbeat-parsing/144522 "2018-08-15T13:33:21Z")
Hello, I have attempted to add another winlogbeat agent to another box and this is the output to the logstash logs: \[2018-08-15T14:01:23,485\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearc… --- ## [Parsing timestamp with extra digits](https://discuss.elastic.co/t/parsing-timestamp-with-extra-digits/144513)
**Author:** [@mladen](https://discuss.elastic.co/u/mladen)\ **Replies:** 2\ **Last updated:** [August 15, 2018, 1:27pm UTC](https://discuss.elastic.co/t/parsing-timestamp-with-extra-digits/144513 "2018-08-15T13:27:34Z")
Hello, I have a problem with parsing my Timestamp log. Format is as follows: 2018-08-07 13:31:27.248097 As you can see I have three extra digits for thousandths of a second. I managed to solve it this way: mutate… --- ## [Failed to initialize http input channel](https://discuss.elastic.co/t/failed-to-initialize-http-input-channel/144416)
**Author:** [@meetdave2611997](https://discuss.elastic.co/u/meetdave2611997)\ **Replies:** 2\ **Last updated:** [August 15, 2018, 12:35pm UTC](https://discuss.elastic.co/t/failed-to-initialize-http-input-channel/144416 "2018-08-15T12:35:04Z")
Hi, We keep seeing this error in log files of logstash nodes when we start logstash with http input. \[2018-08-14T22:03:50,292\]\[WARN \]\[io.netty.channel.ChannelInitializer\] Failed to initialize a channel. Closing: \[id: 0… --- ## [Bag or no?](https://discuss.elastic.co/t/bag-or-no/144362)
**Author:** [@Olha1](https://discuss.elastic.co/u/Olha1)\ **Replies:** 2\ **Last updated:** [August 15, 2018, 10:28am UTC](https://discuss.elastic.co/t/bag-or-no/144362 "2018-08-15T10:28:17Z")
Exception when executing JDBC query {:exception=\>#\} … --- ## [How to drop known errors in logstash](https://discuss.elastic.co/t/how-to-drop-known-errors-in-logstash/144476)
**Author:** [@psuvarna](https://discuss.elastic.co/u/psuvarna)\ **Replies:** 1\ **Last updated:** [August 15, 2018, 9:26am UTC](https://discuss.elastic.co/t/how-to-drop-known-errors-in-logstash/144476 "2018-08-15T09:26:59Z")
Hello, I am new to logstash and ELK stack. We are getting lot of errors in Kibana most of which are known to us. In order to concentrate on new errors while monitoring, we want a way to drop the known errors from logsta… --- ## [Need to parse josn object](https://discuss.elastic.co/t/need-to-parse-josn-object/144455)
**Author:** [@Charan\_Adabala](https://discuss.elastic.co/u/Charan_Adabala)\ **Replies:** 5\ **Last updated:** [August 15, 2018, 9:15am UTC](https://discuss.elastic.co/t/need-to-parse-josn-object/144455 "2018-08-15T09:15:56Z")
Hi, I need to parse this json object using logstash can you please any one help me: Sample Event: {"result":{"points":{"HOST-B765":\[\[1531504800000,84.54410552978516\],\[1531526400000,99.29006958007812\]\]},"unit":"Percent… --- ## [Split multiple CGN Logs from Cisco Router to separate Documents](https://discuss.elastic.co/t/split-multiple-cgn-logs-from-cisco-router-to-separate-documents/143809)
**Author:** [@neerajsapkota](https://discuss.elastic.co/u/neerajsapkota)\ **Replies:** 2\ **Last updated:** [August 12, 2018, 5:54am UTC](https://discuss.elastic.co/t/split-multiple-cgn-logs-from-cisco-router-to-separate-documents/143809 "2018-08-12T05:54:07Z")
Here is the syslog message from Cisco Router.. I get this output in Kibana Discover tab in message field. \<134\> 1 2018 Aug 10 05:48:17 - - NAT44 - \[SessionbasedWD 17 10.10.10.2 poolA - 192.168.1.2 40945 41011 - 172.16… --- ## [Filter - grok, date and kv example](https://discuss.elastic.co/t/filter-grok-date-and-kv-example/144437)
**Author:** [@siddhardhan](https://discuss.elastic.co/u/siddhardhan)\ **Replies:** 2\ **Last updated:** [August 15, 2018, 4:44am UTC](https://discuss.elastic.co/t/filter-grok-date-and-kv-example/144437 "2018-08-15T04:44:25Z")
I am newbie to logstash and trying to ship logs using logstash agent to splunk server. But having trouble shipping the logs. Need help / guidance. Below is my log format: 2018.07.12.12:34:31:232 GID=xxx FID=asdf STRT=1… --- ## [JSON Array split not working and how convert stirng to a JSON?](https://discuss.elastic.co/t/json-array-split-not-working-and-how-convert-stirng-to-a-json/144353)
**Author:** [@richard20427176](https://discuss.elastic.co/u/richard20427176)\ **Replies:** 2\ **Last updated:** [August 15, 2018, 2:48am UTC](https://discuss.elastic.co/t/json-array-split-not-working-and-how-convert-stirng-to-a-json/144353 "2018-08-15T02:48:33Z")
Hi All, I 'm new in using latest version of logstash, I use it to parse json format (or similar to JSON) data to a text file. But there's a JSON Array problem makes me a bit of confusion. I use the postman in Chrome … --- ## [Not creating a logstash index](https://discuss.elastic.co/t/not-creating-a-logstash-index/144433)
**Author:** [@ajhstn](https://discuss.elastic.co/u/ajhstn)\ **Replies:** 3\ **Last updated:** [August 15, 2018, 2:06am UTC](https://discuss.elastic.co/t/not-creating-a-logstash-index/144433 "2018-08-15T02:06:20Z")
Hello, i have configured a logstash input to receive syslog on port 5000. Below is some output when starting logstash. I cannot create an index in Kibana for logstash-\* because it can't find any indicies that match tha… --- ## [Logstash-keystore create generates errors and directory "${sys:ls.log.format}"](https://discuss.elastic.co/t/logstash-keystore-create-generates-errors-and-directory-sys-ls-log-format/144366)
**Author:** [@jkfields](https://discuss.elastic.co/u/jkfields)\ **Replies:** 2\ **Last updated:** [August 14, 2018, 11:58pm UTC](https://discuss.elastic.co/t/logstash-keystore-create-generates-errors-and-directory-sys-ls-log-format/144366 "2018-08-14T23:58:41Z")
We have requirements from Security to secure our Elasticstack. On Logstash, I ran into errors creating the keystore. When I execute the following command(s) it results in the errors shown. /usr/share/logstash/bin/logs… --- ## [Logstash reading the input files but not parsing it](https://discuss.elastic.co/t/logstash-reading-the-input-files-but-not-parsing-it/144404)
**Author:** [@Samvid\_Kulkarni](https://discuss.elastic.co/u/Samvid_Kulkarni)\ **Replies:** 6\ **Last updated:** [August 14, 2018, 10:19pm UTC](https://discuss.elastic.co/t/logstash-reading-the-input-files-but-not-parsing-it/144404 "2018-08-14T22:19:58Z")
logstash is reading input files but it is not displaying anything on the output. No errors are generated as well. when i do --verbose it gives me following output Sending Logstash's logs to /usr/local/Cellar/logsta… --- ## [Grok/logstash efficiency](https://discuss.elastic.co/t/grok-logstash-efficiency/144402)
**Author:** [@if-meaton](https://discuss.elastic.co/u/if-meaton)\ **Replies:** 4\ **Last updated:** [August 14, 2018, 9:20pm UTC](https://discuss.elastic.co/t/grok-logstash-efficiency/144402 "2018-08-14T21:20:00Z")
Hi Guys, I'm wondering if the below logstash conf could be improved in anyway, on one of my logstash servers (a lower powered one) we get about 300 e/s and this server is pinned at 100% cpu and 60% memory when things ar… --- ## [Logstash 6.1.1 missing logs for cloudwatch plugin: aws](https://discuss.elastic.co/t/logstash-6-1-1-missing-logs-for-cloudwatch-plugin-aws/144403)
**Author:** [@Harshad\_Velapure](https://discuss.elastic.co/u/Harshad_Velapure)\ **Replies:** 0\ **Last updated:** [August 14, 2018, 8:37pm UTC](https://discuss.elastic.co/t/logstash-6-1-1-missing-logs-for-cloudwatch-plugin-aws/144403 "2018-08-14T20:37:52Z")
Hello Team, Getting below issue while using cloud\_watch plugin with logstash Upon checking randomly on cloudwatch and kibana index some of the logs are missing I am using logstash-input-cloudwatch plugin to pull logs … --- ## [LS -\> LS (Lumberjack) - documents not being indexed?](https://discuss.elastic.co/t/ls-ls-lumberjack-documents-not-being-indexed/144224)
**Author:** [@if-meaton](https://discuss.elastic.co/u/if-meaton)\ **Replies:** 12\ **Last updated:** [August 14, 2018, 8:31pm UTC](https://discuss.elastic.co/t/ls-ls-lumberjack-documents-not-being-indexed/144224 "2018-08-14T20:31:49Z")
Hi Guys, LS 6.3.x ES 6.3.x I have two logstashs servers across 2 datacentres connected over the internet using the lumberjack protocol. I used https://www.elastic.co/guide/en/logstash/current/ls-to-ls.html as a refere… --- ## [I can't find the GROK pattern of Time format?](https://discuss.elastic.co/t/i-cant-find-the-grok-pattern-of-time-format/144104)
**Author:** [@med\_amine](https://discuss.elastic.co/u/med_amine)\ **Replies:** 2\ **Last updated:** [August 14, 2018, 7:10pm UTC](https://discuss.elastic.co/t/i-cant-find-the-grok-pattern-of-time-format/144104 "2018-08-14T19:10:48Z")
hello evreyone , i can't find the GROK pattern of this Time format ; Dec 17 00:00:01 please help me to find it !! [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=700) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=702)