# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=300 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 301 --- ## [Parsing issues with Logstash CSV plugin](https://discuss.elastic.co/t/parsing-issues-with-logstash-csv-plugin/249856)
**Author:** [@petehalatsis](https://discuss.elastic.co/u/petehalatsis)\ **Replies:** 3\ **Last updated:** [September 25, 2020, 8:13pm UTC](https://discuss.elastic.co/t/parsing-issues-with-logstash-csv-plugin/249856 "2020-09-25T20:13:19Z")
I have been beating my head on this for a while. My syslog is hitting my if statement and using the columns I have below but they are off by one. I tried adding/removing the first column to account for the shift but it d… --- ## [Plugin fails to install](https://discuss.elastic.co/t/plugin-fails-to-install/250012)
**Author:** [@tyrone](https://discuss.elastic.co/u/tyrone)\ **Replies:** 1\ **Last updated:** [September 25, 2020, 6:37pm UTC](https://discuss.elastic.co/t/plugin-fails-to-install/250012 "2020-09-25T18:37:34Z")
I am unable to install the logstash-syslog-output plugin locally I am running rhel7.8, logstash 7.8.1 I have downloaded logstash-output-syslog-master.zip from git and am running \[root@us3lpvinfelk004 bin\]# ./logstas… --- ## [Logstash 7.9.1 have issues in ARM](https://discuss.elastic.co/t/logstash-7-9-1-have-issues-in-arm/249596)
**Author:** [@goutham968](https://discuss.elastic.co/u/goutham968)\ **Replies:** 10\ **Last updated:** [September 25, 2020, 5:13pm UTC](https://discuss.elastic.co/t/logstash-7-9-1-have-issues-in-arm/249596 "2020-09-25T17:13:14Z")
\[2020-09-23T00:58:12,480\]\[ERROR\]\[logstash.javapipeline \]\[main\]\[167820c884ed4020909a7d2491232c40eac5b47a3f91495a85db21bdf02afeca\] A plugin had an unrecoverable error. Will restart this plugin. Pipeline\_id:main Plugin: \ **Author:** [@subash](https://discuss.elastic.co/u/subash)\ **Replies:** 3\ **Last updated:** [September 25, 2020, 2:53pm UTC](https://discuss.elastic.co/t/how-to-compare-the-current-event-with-the-previous-2-events-in-logstash/249864 "2020-09-25T14:53:22Z") I am receiving logs every 5 minutes from many devices. In the log, there is field called crc\_error which holds numeric value For a particular Device, if the value of the crc\_error field increases consecutively for 3 lo… --- ## [Parsing nested JSON fields to separate fields](https://discuss.elastic.co/t/parsing-nested-json-fields-to-separate-fields/249973)
**Author:** [@ottramst](https://discuss.elastic.co/u/ottramst)\ **Replies:** 2\ **Last updated:** [September 25, 2020, 2:51pm UTC](https://discuss.elastic.co/t/parsing-nested-json-fields-to-separate-fields/249973 "2020-09-25T14:51:51Z")
Hello everyone! I need to create a Logstash configuration that allows me to parse nested JSON fields to single fields. Input event looks like this: { "cloud": { "instance": { "id": "XXXX" }, "machine": { … --- ## [Filter logstash - Add simple field](https://discuss.elastic.co/t/filter-logstash-add-simple-field/249490)
**Author:** [@Thomas74](https://discuss.elastic.co/u/Thomas74)\ **Replies:** 3\ **Last updated:** [September 25, 2020, 2:39pm UTC](https://discuss.elastic.co/t/filter-logstash-add-simple-field/249490 "2020-09-25T14:39:04Z")
Hi, I'm trying to assign a value to a field. This one is the same than another variable. Here is my filter : filter{ if !(\[fields\]\[hostgroup\]) { mutate { update =\> { "\[fields\]\[hostgroup\]" … --- ## [Encryption used by Logstash to talk to Elastic Search](https://discuss.elastic.co/t/encryption-used-by-logstash-to-talk-to-elastic-search/249755)
**Author:** [@Pooja](https://discuss.elastic.co/u/Pooja)\ **Replies:** 1\ **Last updated:** [September 25, 2020, 2:20pm UTC](https://discuss.elastic.co/t/encryption-used-by-logstash-to-talk-to-elastic-search/249755 "2020-09-25T14:20:56Z")
How do you ensure data processing between logstash on premise is secured while in transit to Elasticsearch --- ## [Dynamic variable in Http\_poller url](https://discuss.elastic.co/t/dynamic-variable-in-http-poller-url/249943)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 2\ **Last updated:** [September 25, 2020, 2:03pm UTC](https://discuss.elastic.co/t/dynamic-variable-in-http-poller-url/249943 "2020-09-25T14:03:13Z")
Hi all, i try to make the endDate in the url , dynamic ( take the date of the system) ? but i got error in logstash . \[2020-09-25T10:24:22,616\]\[INFO \]\[logstash.outputs.elasticsearch\]\[main\] Attempting to install templa… --- ## [Logstash can't start](https://discuss.elastic.co/t/logstash-cant-start/249673)
**Author:** [@Ben\_said](https://discuss.elastic.co/u/Ben_said)\ **Replies:** 3\ **Last updated:** [September 25, 2020, 1:56pm UTC](https://discuss.elastic.co/t/logstash-cant-start/249673 "2020-09-25T13:56:00Z")
Hello, Can someone help me please i want to configure format date in pipeline logstash Pipeline Config : input { beats { port =\> 5044 host =\> "0.0.0.0" } } filter { if \[fileset\]\[module\] == "system" { … --- ## [Send log from SUSE 10 to logstash](https://discuss.elastic.co/t/send-log-from-suse-10-to-logstash/249927)
**Author:** [@diecode](https://discuss.elastic.co/u/diecode)\ **Replies:** 2\ **Last updated:** [September 25, 2020, 11:20am UTC](https://discuss.elastic.co/t/send-log-from-suse-10-to-logstash/249927 "2020-09-25T11:20:35Z")
Hey, I have few SUSE 10 servers (old novell servers, I cannot upgrade). I would like send a file content to remote logstash. I tried install filebeat, but the SUSE 10 is not supported. There is syslog-ng on the SUSE10, … --- ## [Beat not creating index the right way after change output from elasticsearch to logstash](https://discuss.elastic.co/t/beat-not-creating-index-the-right-way-after-change-output-from-elasticsearch-to-logstash/249951)
**Author:** [@hank](https://discuss.elastic.co/u/hank)\ **Replies:** 0\ **Last updated:** [September 25, 2020, 9:57am UTC](https://discuss.elastic.co/t/beat-not-creating-index-the-right-way-after-change-output-from-elasticsearch-to-logstash/249951 "2020-09-25T09:57:08Z")
hi installed new elastic stack with version 7.9.1 and filebeat localy everything working fine with default config installed logstash and changed the output in filebeat.yml to logstash index creating not working as bef… --- ## [What is the grok pattern for the IIS Logs](https://discuss.elastic.co/t/what-is-the-grok-pattern-for-the-iis-logs/249492)
**Author:** [@Dubru](https://discuss.elastic.co/u/Dubru)\ **Replies:** 2\ **Last updated:** [September 25, 2020, 7:32am UTC](https://discuss.elastic.co/t/what-is-the-grok-pattern-for-the-iis-logs/249492 "2020-09-25T07:32:06Z")
Hi, What is the grok pattern for the following log, as I am new to grok patterns I would need help in making the them. date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip… --- ## [What is the classpath for logstash jar and third party jar like manticore in logstash 7.4?](https://discuss.elastic.co/t/what-is-the-classpath-for-logstash-jar-and-third-party-jar-like-manticore-in-logstash-7-4/249783)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 1\ **Last updated:** [September 25, 2020, 6:36am UTC](https://discuss.elastic.co/t/what-is-the-classpath-for-logstash-jar-and-third-party-jar-like-manticore-in-logstash-7-4/249783 "2020-09-25T06:36:49Z")
any idea for this --- ## [Json array split using logstash](https://discuss.elastic.co/t/json-array-split-using-logstash/249876)
**Author:** [@subash](https://discuss.elastic.co/u/subash)\ **Replies:** 3\ **Last updated:** [September 25, 2020, 12:19am UTC](https://discuss.elastic.co/t/json-array-split-using-logstash/249876 "2020-09-25T00:19:29Z")
I have a jason array like the below The split filter is working till sdwan.status \`split { field =\> "\[parsed\_collection\]\[collection\]\[sdwan:status\]" } when I try to split till path-status like this split { field =\> "… --- ## [Targeting Field Content](https://discuss.elastic.co/t/targeting-field-content/249543)
**Author:** [@teej](https://discuss.elastic.co/u/teej)\ **Replies:** 16\ **Last updated:** [September 24, 2020, 8:10pm UTC](https://discuss.elastic.co/t/targeting-field-content/249543 "2020-09-24T20:10:42Z")
Hi, I am trying to get grok to target certain fields and extract the content that follows. For example: \3186\ I want to find c\_port and extract the number that follows, in this case 3186. I have tried … --- ## [Improper behaviour of Logstash kv filter](https://discuss.elastic.co/t/improper-behaviour-of-logstash-kv-filter/249791)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 4\ **Last updated:** [September 24, 2020, 4:06pm UTC](https://discuss.elastic.co/t/improper-behaviour-of-logstash-kv-filter/249791 "2020-09-24T16:06:33Z")
filter { if \[type\] == "rsyslog" { if \[message\] { kv { source =\> "message" field\_split =\> "," prefix =\> "syslog\_" transform\_key =\> "lowercase" } } } } { "\_index": "client\_syslog-2020.09.24", "\_type": "\_doc", "\_id":… --- ## [Multiline json over TCP is not parsed](https://discuss.elastic.co/t/multiline-json-over-tcp-is-not-parsed/249438)
**Author:** [@cristi.mitroi](https://discuss.elastic.co/u/cristi.mitroi)\ **Replies:** 7\ **Last updated:** [September 24, 2020, 2:56pm UTC](https://discuss.elastic.co/t/multiline-json-over-tcp-is-not-parsed/249438 "2020-09-24T14:56:40Z")
I have the following multiline json: " {"businessId":"335","technicId":"41180883","teller":"progr\_1 ","pid":"0024444930","timestamp":"16:03:31.0664","sursa":"test.mbv","nr\_linie":"391","mesaj":"MESSAGE PUT TIME IS 16… --- ## [Merge two index to create third index using logstash](https://discuss.elastic.co/t/merge-two-index-to-create-third-index-using-logstash/249767)
**Author:** [@Abhi6](https://discuss.elastic.co/u/Abhi6)\ **Replies:** 1\ **Last updated:** [September 24, 2020, 2:56pm UTC](https://discuss.elastic.co/t/merge-two-index-to-create-third-index-using-logstash/249767 "2020-09-24T14:56:40Z")
What I have done so far input { elasticsearch { hosts =\> "localost" index =\> "employees\_data,transaction\_data" query =\> '{ "query": { "match": { "code": 1} } }' scroll =\> "5m" docinfo =\> true } } outp… --- ## [Worker loop initialization error in logstash](https://discuss.elastic.co/t/worker-loop-initialization-error-in-logstash/248541)
**Author:** [@cp.patel](https://discuss.elastic.co/u/cp.patel)\ **Replies:** 7\ **Last updated:** [September 24, 2020, 2:08pm UTC](https://discuss.elastic.co/t/worker-loop-initialization-error-in-logstash/248541 "2020-09-24T14:08:27Z")
Hi, We are getting worker loop initialization error while starting logstash. Please help to rectify the issue. conf file is attached and also find the error logs. Error logs in logstash: \[2020-09-14T18:14:59,496\]\[ER… --- ## [Elastic search Index deleted](https://discuss.elastic.co/t/elastic-search-index-deleted/249817)
**Author:** [@kishorsekar](https://discuss.elastic.co/u/kishorsekar)\ **Replies:** 1\ **Last updated:** [September 24, 2020, 2:07pm UTC](https://discuss.elastic.co/t/elastic-search-index-deleted/249817 "2020-09-24T14:07:16Z")
Elastic search Index was mistakenly deleted. We used to take backup of each Index data in the server, how to recover it back. --- ## [Using dissect dynamicly on filepath](https://discuss.elastic.co/t/using-dissect-dynamicly-on-filepath/249663)
**Author:** [@ThePreMan](https://discuss.elastic.co/u/ThePreMan)\ **Replies:** 4\ **Last updated:** [September 24, 2020, 1:45pm UTC](https://discuss.elastic.co/t/using-dissect-dynamicly-on-filepath/249663 "2020-09-24T13:45:55Z")
Hi. Im trying to setup a workflow where people can upload there log files onto a webserver and while doing so add tags to the log file for further differenciation. I was thinking of putting tags into the file name like f… --- ## [Add documentation Pipeline-to-Pipeline Communication](https://discuss.elastic.co/t/add-documentation-pipeline-to-pipeline-communication/249814)
**Author:** [@OlegMakarikhin](https://discuss.elastic.co/u/OlegMakarikhin)\ **Replies:** 0\ **Last updated:** [September 24, 2020, 12:41pm UTC](https://discuss.elastic.co/t/add-documentation-pipeline-to-pipeline-communication/249814 "2020-09-24T12:41:20Z")
Here is feature: https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html But the logstash input-output plugins documentation does not describe these inputs and outputs. I made a mistake in the confi… --- ## [Convert HEx to ascii](https://discuss.elastic.co/t/convert-hex-to-ascii/249739)
**Author:** [@ThePreMan](https://discuss.elastic.co/u/ThePreMan)\ **Replies:** 1\ **Last updated:** [September 24, 2020, 12:34pm UTC](https://discuss.elastic.co/t/convert-hex-to-ascii/249739 "2020-09-24T12:34:41Z")
Hi again :slight\_smile: In my current Project we are using dlt-tools to convert dlt logs into ascii to be read by logstash. Our dlt viewer is using a specific plugin to convert some lines from hex to ascii. This is our … --- ## [Disable the automatic index template loading by logstash](https://discuss.elastic.co/t/disable-the-automatic-index-template-loading-by-logstash/249800)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 2\ **Last updated:** [September 24, 2020, 12:27pm UTC](https://discuss.elastic.co/t/disable-the-automatic-index-template-loading-by-logstash/249800 "2020-09-24T12:27:42Z")
We are using logstash and when it starts it is loading the index template into elasticsearch. We want to disable the auto loading of the index template. --- ## [Mongodb input to logstash output as elastic](https://discuss.elastic.co/t/mongodb-input-to-logstash-output-as-elastic/249806)
**Author:** [@Milan\_Suthar](https://discuss.elastic.co/u/Milan_Suthar)\ **Replies:** 0\ **Last updated:** [September 24, 2020, 12:05pm UTC](https://discuss.elastic.co/t/mongodb-input-to-logstash-output-as-elastic/249806 "2020-09-24T12:05:00Z")
i want MongoDB collections to insert into the elastic index I have tried but it has some issues as mentioned. is there any other standard way around? i have also tried JDBC input but also were facing issues , i fo… --- ## [Logstash adds too many fields / avoid fields from other tables](https://discuss.elastic.co/t/logstash-adds-too-many-fields-avoid-fields-from-other-tables/249802)
**Author:** [@llama0815](https://discuss.elastic.co/u/llama0815)\ **Replies:** 1\ **Last updated:** [September 24, 2020, 11:46am UTC](https://discuss.elastic.co/t/logstash-adds-too-many-fields-avoid-fields-from-other-tables/249802 "2020-09-24T11:46:08Z")
Hey guys, I'm using logstash and jdbc to import few tables. But all tables are running to the field limit (\>1000). I tried to import several other tables with several hundred columns (still each \<1000) but but they add… --- ## [Seeing 5 duplicate entries in kibana for each log entry](https://discuss.elastic.co/t/seeing-5-duplicate-entries-in-kibana-for-each-log-entry/249619)
**Author:** [@Ronnie16](https://discuss.elastic.co/u/Ronnie16)\ **Replies:** 2\ **Last updated:** [September 24, 2020, 11:21am UTC](https://discuss.elastic.co/t/seeing-5-duplicate-entries-in-kibana-for-each-log-entry/249619 "2020-09-24T11:21:23Z")
I have ELK cluster setup in which filebeat pulls the logs and sends to elasticsearch. The issue is that for each entry, the kibana shows exact 5 entries. Can someone point out the issue as to why this is happening ? H… --- ## [Logstash service problem](https://discuss.elastic.co/t/logstash-service-problem/249746)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 0\ **Last updated:** [September 24, 2020, 6:50am UTC](https://discuss.elastic.co/t/logstash-service-problem/249746 "2020-09-24T06:50:57Z")
Hey everyone! I'm struggling with logstash running as a service. I have a couple of config files in /etc/logstash/conf.d/ and when I run my config by running this bin/logstash -f /etc/logstash/conf.d/my.conf --path.sett… --- ## [Compare Dates in Logstash Pipeline](https://discuss.elastic.co/t/compare-dates-in-logstash-pipeline/247415)
**Author:** [@d-ring](https://discuss.elastic.co/u/d-ring)\ **Replies:** 4\ **Last updated:** [September 23, 2020, 9:51pm UTC](https://discuss.elastic.co/t/compare-dates-in-logstash-pipeline/247415 "2020-09-23T21:51:17Z")
I have created a data field during the pipeline and I am trying to find a way to compare that field to see if it is 30 days old or newer. I am not sure how to do about it however. This is the code I have for creating th… --- ## [SNMP/MIBs issue error: no such instance currently exists at this OID](https://discuss.elastic.co/t/snmp-mibs-issue-error-no-such-instance-currently-exists-at-this-oid/249700)
**Author:** [@ShermSherm51](https://discuss.elastic.co/u/ShermSherm51)\ **Replies:** 0\ **Last updated:** [September 23, 2020, 7:37pm UTC](https://discuss.elastic.co/t/snmp-mibs-issue-error-no-such-instance-currently-exists-at-this-oid/249700 "2020-09-23T19:37:41Z")
I'm getting logstash errors: no such instance currently exists at this OID Sep 23 15:30:04 \ logstash\[27639\]: "iso.org.dod.internet.private.enterprises.purestorage.pureObjects.pureProductVersion… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=299) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=301)