# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=355 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 356 --- ## [Hi Guys , I got Error whe I connect remote filebeat with logstash](https://discuss.elastic.co/t/hi-guys-i-got-error-whe-i-connect-remote-filebeat-with-logstash/233526)
**Author:** [@Maxwell\_Chu](https://discuss.elastic.co/u/Maxwell_Chu)\ **Replies:** 5\ **Last updated:** [May 20, 2020, 1:27pm UTC](https://discuss.elastic.co/t/hi-guys-i-got-error-whe-i-connect-remote-filebeat-with-logstash/233526 "2020-05-20T13:27:16Z")
2020-05-20T21:41:56.637+1000 ERROR \[logstash\] logstash/async.go:279 Failed to publish events caused by: write tcp 192.168.110.144:51994-\>192.168.110.143:5044: write: connection reset by peer 2020-05-20T21:41… --- ## [Logstash codec graphite - not adhereing to settings](https://discuss.elastic.co/t/logstash-codec-graphite-not-adhereing-to-settings/233501)
**Author:** [@Securitybits](https://discuss.elastic.co/u/Securitybits)\ **Replies:** 0\ **Last updated:** [May 20, 2020, 9:51am UTC](https://discuss.elastic.co/t/logstash-codec-graphite-not-adhereing-to-settings/233501 "2020-05-20T09:51:45Z")
Hello, i am trying to ingest graphite metrics from my FreeNAS Server into elasticsearch-7.7 through a logstash-7.7 pipeline. my configurations are: input { tcp { port =\> 2003 type =\> graphite … --- ## [Grok filter for different kinds of log in a single file](https://discuss.elastic.co/t/grok-filter-for-different-kinds-of-log-in-a-single-file/232386)
**Author:** [@falconpurple](https://discuss.elastic.co/u/falconpurple)\ **Replies:** 3\ **Last updated:** [May 20, 2020, 9:21am UTC](https://discuss.elastic.co/t/grok-filter-for-different-kinds-of-log-in-a-single-file/232386 "2020-05-20T09:21:02Z")
Hi, I have written grok filter for 2 different types of log lines in a single file. The output is not coming but. The filter that I have written: filter { grok { match =\> { "message" =\> "%{TI… --- ## [How to run Logstash on Debian?](https://discuss.elastic.co/t/how-to-run-logstash-on-debian/233424)
**Author:** [@wanradt](https://discuss.elastic.co/u/wanradt)\ **Replies:** 1\ **Last updated:** [May 20, 2020, 8:56am UTC](https://discuss.elastic.co/t/how-to-run-logstash-on-debian/233424 "2020-05-20T08:56:16Z")
Hi! After reading this blog post I wanted to try out Mysql-Elasticsearch integration and downloaded all listed components and installed them, but did not found out, how should I run Logstash? For my surprise logstash-7… --- ## [Why logstash does not start?](https://discuss.elastic.co/t/why-logstash-does-not-start/233453)
**Author:** [@H\_EO](https://discuss.elastic.co/u/H_EO)\ **Replies:** 1\ **Last updated:** [May 20, 2020, 4:50am UTC](https://discuss.elastic.co/t/why-logstash-does-not-start/233453 "2020-05-20T04:50:12Z")
\*error message : --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console \[WARN \] 2… --- ## [Snmptrap](https://discuss.elastic.co/t/snmptrap/233444)
**Author:** [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Replies:** 0\ **Last updated:** [May 20, 2020, 3:12am UTC](https://discuss.elastic.co/t/snmptrap/233444 "2020-05-20T03:12:12Z")
Hi Elastic team. Withe below config, i am successuflly able to start logstash and snmptrap on 1062. WHen i tried to covert Cisco mib file to yaml. i am getting below error. And SNMP TRAP output in Kibana discover showed… --- ## [CSV Output Plugin - Duplicate Entries](https://discuss.elastic.co/t/csv-output-plugin-duplicate-entries/233368)
**Author:** [@muraliv](https://discuss.elastic.co/u/muraliv)\ **Replies:** 4\ **Last updated:** [May 19, 2020, 11:57pm UTC](https://discuss.elastic.co/t/csv-output-plugin-duplicate-entries/233368 "2020-05-19T23:57:14Z")
Hi All, I am using logstash to write to a csv file and elasticsearch. Elasticsearch output is fine but csv output plugin generates entries each time it runs. I have set the following based on the csv filter: skip\_he… --- ## [How to separate devices/logfiles](https://discuss.elastic.co/t/how-to-separate-devices-logfiles/233325)
**Author:** [@ELK212](https://discuss.elastic.co/u/ELK212)\ **Replies:** 2\ **Last updated:** [May 19, 2020, 11:24pm UTC](https://discuss.elastic.co/t/how-to-separate-devices-logfiles/233325 "2020-05-19T23:24:33Z")
Hello, I just want to know how a logical separation should/could look like for different types of devices. Let's say I have a bunch of switches and a bunch of servers which can send syslog messages. How should I separ… --- ## [Unable to set parse and set timestamp with logstash (7.6.2) and xml filter plugin](https://discuss.elastic.co/t/unable-to-set-parse-and-set-timestamp-with-logstash-7-6-2-and-xml-filter-plugin/233406)
**Author:** [@Kashif\_Khan](https://discuss.elastic.co/u/Kashif_Khan)\ **Replies:** 1\ **Last updated:** [May 19, 2020, 10:52pm UTC](https://discuss.elastic.co/t/unable-to-set-parse-and-set-timestamp-with-logstash-7-6-2-and-xml-filter-plugin/233406 "2020-05-19T22:52:16Z")
Environment details: ELK stack 7.6.2 Windows 10 I am unable to replace/set an elastic search timestamp from my logs while indexing via Logstash. It simply adds as a new field and does not replace the original field. It … --- ## [S3 output plugin generates invalid json](https://discuss.elastic.co/t/s3-output-plugin-generates-invalid-json/233303)
**Author:** [@Ram\_Jeghan](https://discuss.elastic.co/u/Ram_Jeghan)\ **Replies:** 1\ **Last updated:** [May 19, 2020, 7:49pm UTC](https://discuss.elastic.co/t/s3-output-plugin-generates-invalid-json/233303 "2020-05-19T19:49:55Z")
Hi, We are using s3 output plugin to forward logs into an s3 bucket with json\_lines codec. The files generated are raw text files with each line containing a single quoted json file with special chars such as \\n in it. … --- ## [Gsub replace backslash with forward slash](https://discuss.elastic.co/t/gsub-replace-backslash-with-forward-slash/233322)
**Author:** [@sjivan](https://discuss.elastic.co/u/sjivan)\ **Replies:** 2\ **Last updated:** [May 19, 2020, 6:11pm UTC](https://discuss.elastic.co/t/gsub-replace-backslash-with-forward-slash/233322 "2020-05-19T18:11:55Z")
Hi, I'm trying to replace a forward slash to back slash using gsub but can't get it to work. I've tried gsub =\> \["message", "/", "\\\\"\] and gsub =\> \["message", "/", "\\"\] The above fails with a parse error. and gsub … --- ## [Unable to unzip Logtash 7.7.0 package for windows](https://discuss.elastic.co/t/unable-to-unzip-logtash-7-7-0-package-for-windows/232697)
**Author:** [@TonyLuc](https://discuss.elastic.co/u/TonyLuc)\ **Replies:** 2\ **Last updated:** [May 19, 2020, 4:23pm UTC](https://discuss.elastic.co/t/unable-to-unzip-logtash-7-7-0-package-for-windows/232697 "2020-05-19T16:23:33Z")
There seems to be some errors when the latest Logstash package is created. This is for windows installation using the Zip file. URL of the download file: https://artifacts.elastic.co/downloads/logstash/logstash-7.7.0.z… --- ## [How to logstash patterns/regex/co](https://discuss.elastic.co/t/how-to-logstash-patterns-regex-co/232819)
**Author:** [@ELK212](https://discuss.elastic.co/u/ELK212)\ **Replies:** 6\ **Last updated:** [May 19, 2020, 3:28pm UTC](https://discuss.elastic.co/t/how-to-logstash-patterns-regex-co/232819 "2020-05-19T15:28:29Z")
Hello, I'm just playing around with logstash and hope to get help with a few topics. I'm trying to log some HP Procurve Switch syslog messages into elasticsearch. I used the example filter from the website. Which looks… --- ## [Cant parse date from log](https://discuss.elastic.co/t/cant-parse-date-from-log/233315)
**Author:** [@Deny7](https://discuss.elastic.co/u/Deny7)\ **Replies:** 4\ **Last updated:** [May 19, 2020, 3:26pm UTC](https://discuss.elastic.co/t/cant-parse-date-from-log/233315 "2020-05-19T15:26:36Z")
Hi, im trying to parse timestamp from from log to field so I can choose it when creating new index. Date format from log is: 2020-05-19 11:03:18.954 In logstash I defined: date{ match =\> \[ "timestamp", "YYYY-MM-d… --- ## [Using a variable like an index of an array in logstash pipeline](https://discuss.elastic.co/t/using-a-variable-like-an-index-of-an-array-in-logstash-pipeline/233304)
**Author:** [@Md\_Iburahimsha\_S](https://discuss.elastic.co/u/Md_Iburahimsha_S)\ **Replies:** 4\ **Last updated:** [May 19, 2020, 3:10pm UTC](https://discuss.elastic.co/t/using-a-variable-like-an-index-of-an-array-in-logstash-pipeline/233304 "2020-05-19T15:10:30Z")
filter{ ruby { code =\> 'event.set("patternmatch1", event.get("message").scan(/TEXT.\*?(\\w\[\\w,.\]\*)\\r?$/))' } # Retrieve the length of the array containing all the matches ruby { code =\> 'event.set("number… --- ## [Split a json array with same fields names](https://discuss.elastic.co/t/split-a-json-array-with-same-fields-names/233008)
**Author:** [@Claudio\_Ract\_Costa](https://discuss.elastic.co/u/Claudio_Ract_Costa)\ **Replies:** 16\ **Last updated:** [May 19, 2020, 3:01pm UTC](https://discuss.elastic.co/t/split-a-json-array-with-same-fields-names/233008 "2020-05-19T15:01:44Z")
Hi, I'm trying to split a JSON array into multiple events. Here's a sample input: "chargingContextSpecific" : \[ { "parameterID" : 16778241, "para… --- ## [Flatting nested array in Json](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066)
**Author:** [@Ullas7](https://discuss.elastic.co/u/Ullas7)\ **Replies:** 11\ **Last updated:** [May 19, 2020, 1:09pm UTC](https://discuss.elastic.co/t/flatting-nested-array-in-json/233066 "2020-05-19T13:09:19Z")
I have a Json something like this , I trying to flatten it before storing in Elasticsearch. all nested elements should be on level 1. I read about ruby script. I am new to ELK and ruby. someone can help me out? … --- ## [Amazon ES integration with K8s logstash pod](https://discuss.elastic.co/t/amazon-es-integration-with-k8s-logstash-pod/233300)
**Author:** [@Alex\_Naidyonkov](https://discuss.elastic.co/u/Alex_Naidyonkov)\ **Replies:** 0\ **Last updated:** [May 19, 2020, 10:52am UTC](https://discuss.elastic.co/t/amazon-es-integration-with-k8s-logstash-pod/233300 "2020-05-19T10:52:22Z")
Hello, need some help, I have logstash pod in k8s with connection to my Amazon ES: Define logstash outputs output { amazon\_es { hosts =\> "vpc-es-dvc-..........eu-west-1.es.amazonaws.com" … --- ## [Multiline codec and fluent codec](https://discuss.elastic.co/t/multiline-codec-and-fluent-codec/233270)
**Author:** [@VarunKS](https://discuss.elastic.co/u/VarunKS)\ **Replies:** 0\ **Last updated:** [May 19, 2020, 8:07am UTC](https://discuss.elastic.co/t/multiline-codec-and-fluent-codec/233270 "2020-05-19T08:07:51Z")
I am using fluentD to send the logs to logstash, but since am using fluent codec to receive the logs, I am having issues in using multiline codec to handle stack trace. Any ideas would be helpful. Thanks. --- ## [Logstash5.5 to kafka newer version - data discripency](https://discuss.elastic.co/t/logstash5-5-to-kafka-newer-version-data-discripency/233257)
**Author:** [@manu1](https://discuss.elastic.co/u/manu1)\ **Replies:** 0\ **Last updated:** [May 19, 2020, 6:58am UTC](https://discuss.elastic.co/t/logstash5-5-to-kafka-newer-version-data-discripency/233257 "2020-05-19T06:58:53Z")
Hello, We are using Logstash5.5 send event logs to Kafka newer version(2.4). We are seeing some data discrepancy. Issue seems to be resolving if we restart logstash service, however it is reoccurring nearly in 48 hours … --- ## [How to create kibana Dashboard from client machine through Logstash](https://discuss.elastic.co/t/how-to-create-kibana-dashboard-from-client-machine-through-logstash/233043)
**Author:** [@vijazy](https://discuss.elastic.co/u/vijazy)\ **Replies:** 3\ **Last updated:** [May 19, 2020, 5:59am UTC](https://discuss.elastic.co/t/how-to-create-kibana-dashboard-from-client-machine-through-logstash/233043 "2020-05-19T05:59:15Z")
Hi elastic team , I am new in ELK, I am trying to implement ELK on production environment. I have configured Elasticsearch,kibana and logstash.. host1 - elasticsearch,kibana ,logstash with nginx proxy host2- Clientmac… --- ## [The logstash'pipeline is running, but elasticsearch didnt create an index for data](https://discuss.elastic.co/t/the-logstashpipeline-is-running-but-elasticsearch-didnt-create-an-index-for-data/232768)
**Author:** [@Joseph-Gan-wk](https://discuss.elastic.co/u/Joseph-Gan-wk)\ **Replies:** 9\ **Last updated:** [May 19, 2020, 2:52am UTC](https://discuss.elastic.co/t/the-logstashpipeline-is-running-but-elasticsearch-didnt-create-an-index-for-data/232768 "2020-05-19T02:52:27Z")
Hi all, i am facing an issue is the elasticsearch did not create an index for data on Kibana. I checked the Logstash-plain.log that shows two pipelines are running. \[2020-05-15T15:42:41,124\]\[INFO \]\[logstash.agent … --- ## [Logstash Monitoring is Disabled when i Edit Pipeline settings](https://discuss.elastic.co/t/logstash-monitoring-is-disabled-when-i-edit-pipeline-settings/233101)
**Author:** [@Rohit\_Kumbhar](https://discuss.elastic.co/u/Rohit_Kumbhar)\ **Replies:** 2\ **Last updated:** [May 18, 2020, 10:21pm UTC](https://discuss.elastic.co/t/logstash-monitoring-is-disabled-when-i-edit-pipeline-settings/233101 "2020-05-18T22:21:12Z")
Hey, I edited Logstash,yml file to increase worker for main Pipeline in result i could not see LOGSTASH MODULE in Stack Monitoring in Kibana Here is my logstash.yml file # ------------ Pipeline Settings -------------… --- ## [Regex in mutate filter](https://discuss.elastic.co/t/regex-in-mutate-filter/233091)
**Author:** [@priyanshuk](https://discuss.elastic.co/u/priyanshuk)\ **Replies:** 3\ **Last updated:** [May 18, 2020, 10:13pm UTC](https://discuss.elastic.co/t/regex-in-mutate-filter/233091 "2020-05-18T22:13:43Z")
Hi, I need some help with regex in mutate Part of log in message field : "LogError":"30180" - Please note number can vary . in normal regex we can do grouping "LogError":"(\[0-9\]\[0-9\]\*)" and then get the matched regex… --- ## [(probably) dumb grok question](https://discuss.elastic.co/t/probably-dumb-grok-question/233192)
**Author:** [@Shawn\_Keslar](https://discuss.elastic.co/u/Shawn_Keslar)\ **Replies:** 1\ **Last updated:** [May 18, 2020, 9:46pm UTC](https://discuss.elastic.co/t/probably-dumb-grok-question/233192 "2020-05-18T21:46:54Z")
an example of my data: {"protocol": 6, "recordType": 71, "eventType": 1003, "securityGroupId": 0, "userId": 9999997, "policyRevision": "00000000-0000-0000-0000-00005e5f4ab1", "iocNumber": 0, "userAgent": {"block… --- ## [Data Mismatch between csv and logstash](https://discuss.elastic.co/t/data-mismatch-between-csv-and-logstash/233001)
**Author:** [@tenet\_testuser1](https://discuss.elastic.co/u/tenet_testuser1)\ **Replies:** 3\ **Last updated:** [May 18, 2020, 9:03pm UTC](https://discuss.elastic.co/t/data-mismatch-between-csv-and-logstash/233001 "2020-05-18T21:03:05Z")
Hello, Need help please. I am pulling data from postgres as csv on a table. The output is one record per data (date based) and the csv file shows, the record count and field values correctly when views in a table forma… --- ## [Files skipped in logstash but contained in Sincedb file](https://discuss.elastic.co/t/files-skipped-in-logstash-but-contained-in-sincedb-file/233199)
**Author:** [@aerodynamic](https://discuss.elastic.co/u/aerodynamic)\ **Replies:** 0\ **Last updated:** [May 18, 2020, 8:50pm UTC](https://discuss.elastic.co/t/files-skipped-in-logstash-but-contained-in-sincedb-file/233199 "2020-05-18T20:50:34Z")
Hello, My current setup reads gz files from coolpath and forwards them to elasticsearch. However there are files being left behind in coolpath. input { file { id =\> "my-id" path =\> "/coolpath/\*.gz" mode … --- ## [Conditional file output with overwrite leads to empty file](https://discuss.elastic.co/t/conditional-file-output-with-overwrite-leads-to-empty-file/233198)
**Author:** [@nicklan](https://discuss.elastic.co/u/nicklan)\ **Replies:** 0\ **Last updated:** [May 18, 2020, 8:46pm UTC](https://discuss.elastic.co/t/conditional-file-output-with-overwrite-leads-to-empty-file/233198 "2020-05-18T20:46:05Z")
I have a file output that is triggered conditionally set up as: output { if \[@metadata\]\[dofile\] { file { path =\> "/tmp/ts.out" codec =\> line { format =\> "%{\[afield\]}"} write\_behavior =\> "overwrit… --- ## [Logstash split function](https://discuss.elastic.co/t/logstash-split-function/233164)
**Author:** [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\ **Replies:** 5\ **Last updated:** [May 18, 2020, 7:38pm UTC](https://discuss.elastic.co/t/logstash-split-function/233164 "2020-05-18T19:38:16Z")
Hi, I have this tag fields with value Beige, Black, Blue, boohoo, Brown, cat:dresses, DZZ56895, Green, Grey, New Arrivals, Orange, Pink, Purple, Red, subcat:cocktail:cat:dresses, White, women, Yellow, z\_global\_id:3903… --- ## [Use Index templalte in logstash](https://discuss.elastic.co/t/use-index-templalte-in-logstash/232865)
**Author:** [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\ **Replies:** 2\ **Last updated:** [May 18, 2020, 4:28pm UTC](https://discuss.elastic.co/t/use-index-templalte-in-logstash/232865 "2020-05-18T16:28:07Z")
Hi, I have create a new index template. I want to reference my new template in logstash configuration. Following is my configuration file input { jdbc { jdbc\_driver\_library =\> "/usr/share/java/mysql.jar" jdbc… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=354) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=356)