# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=288 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 289 --- ## [Pipeline aborted due to error - exception=\>java.lang.ClassCastException - Error while fetching data using Salesforce plugin](https://discuss.elastic.co/t/pipeline-aborted-due-to-error-exception-java-lang-classcastexception-error-while-fetching-data-using-salesforce-plugin/253439)
**Author:** [@Pavitra\_Poojary](https://discuss.elastic.co/u/Pavitra_Poojary)\ **Replies:** 6\ **Last updated:** [October 28, 2020, 6:35am UTC](https://discuss.elastic.co/t/pipeline-aborted-due-to-error-exception-java-lang-classcastexception-error-while-fetching-data-using-salesforce-plugin/253439 "2020-10-28T06:35:47Z")
Hi, I am trying to pull the Salesforce data using the salesforce plugin available in Logstash. Below is my configuration. ELK Version : 7.4 input { salesforce { use\_test\_sandbox =\> true client\_id =\> 'xxx' … --- ## [Dynamically move nested fields to the top level](https://discuss.elastic.co/t/dynamically-move-nested-fields-to-the-top-level/253504)
**Author:** [@jiacob](https://discuss.elastic.co/u/jiacob)\ **Replies:** 0\ **Last updated:** [October 28, 2020, 12:02am UTC](https://discuss.elastic.co/t/dynamically-move-nested-fields-to-the-top-level/253504 "2020-10-28T00:02:44Z")
Hello, I am trying to parse an XML message with lots of nested fields. I am using the XML filter to parse the data and send it to Elasticsearch. The issue that I am having is that the XML path to the value is too large. … --- ## [Logstash \[7.8\] problem with an elastic update doc](https://discuss.elastic.co/t/logstash-7-8-problem-with-an-elastic-update-doc/253370)
**Author:** [@unknown\_user](https://discuss.elastic.co/u/unknown_user)\ **Replies:** 2\ **Last updated:** [October 27, 2020, 10:27pm UTC](https://discuss.elastic.co/t/logstash-7-8-problem-with-an-elastic-update-doc/253370 "2020-10-27T22:27:34Z")
Hi Guys, Running into a bit of an issue that I am trying to overcome. I have a index that only updates documents, specifically I am monitoring devices that are either in an UP or DOWN status and the doc\_id is the name … --- ## [Masking password in logstash config file](https://discuss.elastic.co/t/masking-password-in-logstash-config-file/253368)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 3\ **Last updated:** [October 27, 2020, 9:03pm UTC](https://discuss.elastic.co/t/masking-password-in-logstash-config-file/253368 "2020-10-27T21:03:53Z")
Hello all, As logstash config file is not centrally controlled, I want to mask the elasticsearch password in logstash.conf file so that it is not visible to all. Also user with what "security role" should suffice for m… --- ## [Logstash to Logstash (Lumberjack) host details](https://discuss.elastic.co/t/logstash-to-logstash-lumberjack-host-details/253390)
**Author:** [@cliff.wakefield](https://discuss.elastic.co/u/cliff.wakefield)\ **Replies:** 3\ **Last updated:** [October 27, 2020, 8:36pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-lumberjack-host-details/253390 "2020-10-27T20:36:30Z")
I am setting up a two-tier Logstash setup for our customers. On-premises servers running syslog and beats will connect to 1 or more on-premises Logstash servers, which will be running syslog and beats pipelines configur… --- ## [S3 Input Plugin Does Not Delete The Temporary Files](https://discuss.elastic.co/t/s3-input-plugin-does-not-delete-the-temporary-files/253483)
**Author:** [@Rahul\_Kumar4](https://discuss.elastic.co/u/Rahul_Kumar4)\ **Replies:** 3\ **Last updated:** [October 27, 2020, 8:12pm UTC](https://discuss.elastic.co/t/s3-input-plugin-does-not-delete-the-temporary-files/253483 "2020-10-27T20:12:45Z")
The S3 input plugin does not delete the temporary files that it creates from the downloaded objects from S3 bucket even after processing and indexing it to Elasticsearch. Is there a work around/setting to automate this? … --- ## [Supress errors from logstash input plugin](https://discuss.elastic.co/t/supress-errors-from-logstash-input-plugin/253414)
**Author:** [@nitzan.karni](https://discuss.elastic.co/u/nitzan.karni)\ **Replies:** 1\ **Last updated:** [October 27, 2020, 2:15pm UTC](https://discuss.elastic.co/t/supress-errors-from-logstash-input-plugin/253414 "2020-10-27T14:15:30Z")
Hi, I have set up a logstash input for syslog: syslog { port =\> 6514 codec =\> cef tags =\> \[ "syslog" \] } I have different kind of syslogs running through that input. I have messages that arrives in cef fo… --- ## [Gsub backslash escape and continue when error](https://discuss.elastic.co/t/gsub-backslash-escape-and-continue-when-error/253282)
**Author:** [@bnmtl](https://discuss.elastic.co/u/bnmtl)\ **Replies:** 1\ **Last updated:** [October 27, 2020, 12:01pm UTC](https://discuss.elastic.co/t/gsub-backslash-escape-and-continue-when-error/253282 "2020-10-27T12:01:41Z")
Hi, I have json input, and using codec =\> json { charset =\> "UTF-8" } I see that logstash giving Json::ParserError: Unrecognized character escape '' when I am getting message below; "incoming": "\\u001Btb��01w" I used … --- ## [A new index for each container?](https://discuss.elastic.co/t/a-new-index-for-each-container/253394)
**Author:** [@nyquillus](https://discuss.elastic.co/u/nyquillus)\ **Replies:** 1\ **Last updated:** [October 27, 2020, 9:28am UTC](https://discuss.elastic.co/t/a-new-index-for-each-container/253394 "2020-10-27T09:28:58Z")
Hi, I am currently running a test environment to try gelf logging driver to monitor container logs with a single node stack and so far it worked well. But I need to make some changes and I'm pretty new to this so I need… --- ## [Sending Rsyslog data to logstash](https://discuss.elastic.co/t/sending-rsyslog-data-to-logstash/249066)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 6\ **Last updated:** [October 27, 2020, 7:25am UTC](https://discuss.elastic.co/t/sending-rsyslog-data-to-logstash/249066 "2020-10-27T07:25:26Z")
I've configured two linux server for sending client rsyslog data to server.The server is receiving messages from client and i've installed elk service on server side now i want to send the rsyslog data (which the server … --- ## [Logstash clustering issue](https://discuss.elastic.co/t/logstash-clustering-issue/251777)
**Author:** [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)\ **Replies:** 8\ **Last updated:** [October 27, 2020, 6:17am UTC](https://discuss.elastic.co/t/logstash-clustering-issue/251777 "2020-10-27T06:17:46Z")
Im having Elasticsearch clustering and my output.conf in logstash, looks like below hosts =\> \["node1:9200","node2:9200","node3:9200"\] So through java, Im updating output.conf and starting logstash. Now in case my elast… --- ## [How to format inregular json-string like this?](https://discuss.elastic.co/t/how-to-format-inregular-json-string-like-this/253375)
**Author:** [@knight](https://discuss.elastic.co/u/knight)\ **Replies:** 0\ **Last updated:** [October 27, 2020, 1:12am UTC](https://discuss.elastic.co/t/how-to-format-inregular-json-string-like-this/253375 "2020-10-27T01:12:39Z")
i have a json-string like this : { "aa":"123", "bb":"hello", "cc":"===this is inregular json==={"dd":"world","ee":"logstash"}" } now,the config file in logstash is : input { kafka { bootstrap\_servers… --- ## [How to remove document I logstash pipilnie that has no usefull fields](https://discuss.elastic.co/t/how-to-remove-document-i-logstash-pipilnie-that-has-no-usefull-fields/253243)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 8\ **Last updated:** [October 26, 2020, 11:53pm UTC](https://discuss.elastic.co/t/how-to-remove-document-i-logstash-pipilnie-that-has-no-usefull-fields/253243 "2020-10-26T23:53:44Z")
Hello, I use the logstash SNMP plugin to query my device. After that, I do some split and field renaming as below. input { snmp { interval =\> 5 hosts =\> \[ {host =\> "udp:ip/port" communit… --- ## [Logstash problem](https://discuss.elastic.co/t/logstash-problem/253349)
**Author:** [@dfoot](https://discuss.elastic.co/u/dfoot)\ **Replies:** 1\ **Last updated:** [October 26, 2020, 7:53pm UTC](https://discuss.elastic.co/t/logstash-problem/253349 "2020-10-26T19:53:13Z")
Logstash version 7.1.1 after restart logstash I am getting below log message but not sure what is issue. Please advise. tail /var/log/logstash/logstash-plain.log \[2020-09-28T09:23:05,009\]\[WARN \]\[logstash.outputs.elasti… --- ## [Logstash-input-salesforce](https://discuss.elastic.co/t/logstash-input-salesforce/252906)
**Author:** [@eemtzc](https://discuss.elastic.co/u/eemtzc)\ **Replies:** 5\ **Last updated:** [October 26, 2020, 7:09pm UTC](https://discuss.elastic.co/t/logstash-input-salesforce/252906 "2020-10-26T19:09:37Z")
Hello, Right now in the actuallyty we are running a Elastic Cluster v 7.5.2 with a Logstash 6.4.2 we want to upgrade our cluster to v 7.9.2 but the only reason we are not able to do this is because in this Logstash serv… --- ## [Documentation error](https://discuss.elastic.co/t/documentation-error/253330)
**Author:** [@gborg](https://discuss.elastic.co/u/gborg)\ **Replies:** 1\ **Last updated:** [October 26, 2020, 4:39pm UTC](https://discuss.elastic.co/t/documentation-error/253330 "2020-10-26T16:39:01Z")
In the logstash mutete filter documentation there seems to be an error, or at least it seems like an error to me mutate { split =\> \["hostname", "."\] add\_field =\> { "shortHostname" =\> "%{hostname\[0\]… --- ## [O/p not getting in console](https://discuss.elastic.co/t/o-p-not-getting-in-console/253344)
**Author:** [@Sakthivanan](https://discuss.elastic.co/u/Sakthivanan)\ **Replies:** 0\ **Last updated:** [October 26, 2020, 4:31pm UTC](https://discuss.elastic.co/t/o-p-not-getting-in-console/253344 "2020-10-26T16:31:41Z")
Hi All, I m trying to load CSV file for that i used below config file.its running without error in cmd prompt but I didnt get output. This my CSV File with header cds,rtype,schoolname,districtname,countyname,charter\_f… --- ## [No logs after enable TLS](https://discuss.elastic.co/t/no-logs-after-enable-tls/253337)
**Author:** [@LuWe](https://discuss.elastic.co/u/LuWe)\ **Replies:** 0\ **Last updated:** [October 26, 2020, 3:57pm UTC](https://discuss.elastic.co/t/no-logs-after-enable-tls/253337 "2020-10-26T15:57:12Z")
Hi, i have a ELK-cluster to log a firewall. That worked well until i enabled TLS/SSL. Now Kibana do not show new logs. Here ist one of the config files: input { udp { port =\> 514 type =\> firewall } } filter { if \[type\]… --- ## [Logstash combining dissect and grok to add date to my timestamps without date](https://discuss.elastic.co/t/logstash-combining-dissect-and-grok-to-add-date-to-my-timestamps-without-date/253314)
**Author:** [@bobmaza](https://discuss.elastic.co/u/bobmaza)\ **Replies:** 0\ **Last updated:** [October 26, 2020, 1:27pm UTC](https://discuss.elastic.co/t/logstash-combining-dissect-and-grok-to-add-date-to-my-timestamps-without-date/253314 "2020-10-26T13:27:50Z")
Hi, i totally new to the elastic stack, and currently working on a dash board for logs through kibana, but i'm still at the data ingestion stage unfortunately ! took me some days to learn how to make things work, frist… --- ## [Custom access\_log and ECS](https://discuss.elastic.co/t/custom-access-log-and-ecs/253148)
**Author:** [@rmrfchik](https://discuss.elastic.co/u/rmrfchik)\ **Replies:** 2\ **Last updated:** [October 26, 2020, 1:33pm UTC](https://discuss.elastic.co/t/custom-access-log-and-ecs/253148 "2020-10-26T13:33:12Z")
I have custom access\_log and want to use power of ECS. So, filebeat simply chew log file (type: log) and forward lines to logstash. Logstash groks line and fills "user\_agent.original" field. So far so good. I made sim… --- ## [Efficient kv parser - replacement?](https://discuss.elastic.co/t/efficient-kv-parser-replacement/253320)
**Author:** [@wedkarz014](https://discuss.elastic.co/u/wedkarz014)\ **Replies:** 0\ **Last updated:** [October 26, 2020, 2:17pm UTC](https://discuss.elastic.co/t/efficient-kv-parser-replacement/253320 "2020-10-26T14:17:54Z")
Hi, In filter part I have a parser like here: kv { field\_split =\> "," trim\_value =\> "\\"" value\_split =\> "=" include\_keys =\> \["Hostname","SlotId","EOCTimestamp","RequestStartTimestamp","ResponseStartTi… --- ## [LOSING OUTPUT DELETE TAKES HOURS](https://discuss.elastic.co/t/losing-output-delete-takes-hours/253305)
**Author:** [@Daniel\_Lopez](https://discuss.elastic.co/u/Daniel_Lopez)\ **Replies:** 0\ **Last updated:** [October 26, 2020, 12:37pm UTC](https://discuss.elastic.co/t/losing-output-delete-takes-hours/253305 "2020-10-26T12:37:06Z")
HI to all I will try to explain this issue: I have these lines in log 08:41:39.691 new resources port:6215 trunk:1 08:41:40.128 Release resources port:6215 The First line is index at 08:46, so late i think, but wit… --- ## [Not getting new columns with aggregation](https://discuss.elastic.co/t/not-getting-new-columns-with-aggregation/253273)
**Author:** [@micobarac](https://discuss.elastic.co/u/micobarac)\ **Replies:** 0\ **Last updated:** [October 26, 2020, 8:23am UTC](https://discuss.elastic.co/t/not-getting-new-columns-with-aggregation/253273 "2020-10-26T08:23:24Z")
This is my variation of the Logstash aggregation filter: filter { if !\[messageid\] { drop {} } else if \[program\] == "amavis" and \[message\] =~ /(?i)message\\-id/ { aggregate { task\_id =\> "%{messageid}" … --- ## [org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34](https://discuss.elastic.co/t/org-logstash-beats-invalidframeprotocolexception-invalid-version-of-beats-protocol-34/253267)
**Author:** [@tuawow](https://discuss.elastic.co/u/tuawow)\ **Replies:** 1\ **Last updated:** [October 26, 2020, 7:23am UTC](https://discuss.elastic.co/t/org-logstash-beats-invalidframeprotocolexception-invalid-version-of-beats-protocol-34/253267 "2020-10-26T07:23:43Z")
\]\[io.netty.channel.DefaultChannelPipeline\]\[main\]\[f73510d0a1046ae249c9c684ab7752f124dc9801416a7a8bc674b1874ca37218\] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the la… --- ## [Logstash Ruby not handling float value correctly](https://discuss.elastic.co/t/logstash-ruby-not-handling-float-value-correctly/253150)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 6\ **Last updated:** [October 25, 2020, 3:31pm UTC](https://discuss.elastic.co/t/logstash-ruby-not-handling-float-value-correctly/253150 "2020-10-25T15:31:07Z")
I have Kafak \<\> Logstash (then Ruby) \<\> Elastic Kafak Input: {"customerid":"smit","last\_name":"shah","age":10,"height":10,"weight":100,"automated\_email":false, "header": { "endpoint":"/pay"}, "transaction": { "amount":… --- ## [A working example of how to import a Wikipedia dump with Logstash using es\_bulk](https://discuss.elastic.co/t/a-working-example-of-how-to-import-a-wikipedia-dump-with-logstash-using-es-bulk/253228)
**Author:** [@Ola\_Gustafsson1](https://discuss.elastic.co/u/Ola_Gustafsson1)\ **Replies:** 0\ **Last updated:** [October 25, 2020, 9:46am UTC](https://discuss.elastic.co/t/a-working-example-of-how-to-import-a-wikipedia-dump-with-logstash-using-es-bulk/253228 "2020-10-25T09:46:19Z")
I'm writing a logstash configuration file for importing a Wikipedia dump, found on https://dumps.wikimedia.org/other/cirrussearch/current/ The dumps are in the es\_bulk format, ie one line for the action and id of the do… --- ## [How set IP address type on selected fields](https://discuss.elastic.co/t/how-set-ip-address-type-on-selected-fields/253220)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 1\ **Last updated:** [October 24, 2020, 8:03pm UTC](https://discuss.elastic.co/t/how-set-ip-address-type-on-selected-fields/253220 "2020-10-24T20:03:14Z")
My data incoming in json and IP is always on few fields, how can I set on this fields IP address type to query this fields via CIDR notation etc? --- ## [Logstash elsticsearch output](https://discuss.elastic.co/t/logstash-elsticsearch-output/253219)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 0\ **Last updated:** [October 24, 2020, 7:38pm UTC](https://discuss.elastic.co/t/logstash-elsticsearch-output/253219 "2020-10-24T19:38:17Z")
I am running the elasticsearch cluster of 6 node 3 master 2 data 1 coordinate.I am also running logstash in other node for testing purpose which request should go to which node i made pipeline file output { elastics… --- ## [Decode JSON hex field before push to ES](https://discuss.elastic.co/t/decode-json-hex-field-before-push-to-es/253126)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 9\ **Last updated:** [October 24, 2020, 5:37pm UTC](https://discuss.elastic.co/t/decode-json-hex-field-before-push-to-es/253126 "2020-10-24T17:37:17Z")
Hi, I have input data such as: { "timestamp": "1603363180424", "layers": { \[.. some data ..\] }, "aaa": { \[.. some data ..\] }, "bbb": { \[.. so… --- ## [Http input plugin is throwing errors](https://discuss.elastic.co/t/http-input-plugin-is-throwing-errors/252970)
**Author:** [@prajwalgmpp](https://discuss.elastic.co/u/prajwalgmpp)\ **Replies:** 3\ **Last updated:** [October 24, 2020, 9:53am UTC](https://discuss.elastic.co/t/http-input-plugin-is-throwing-errors/252970 "2020-10-24T09:53:05Z")
Still in the initial steps of learning Logstash. I have written the following pipeline in a configuration file: input { stdin { codec =\> json } http { } } output { stdout{ codec =\> rubydebug } } When I run logs… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=287) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=289)