# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=335 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 336 --- ## [Add a backslash at the end of a string](https://discuss.elastic.co/t/add-a-backslash-at-the-end-of-a-string/239336)
**Author:** [@Florent\_Fauvin](https://discuss.elastic.co/u/Florent_Fauvin)\ **Replies:** 2\ **Last updated:** [June 30, 2020, 4:54pm UTC](https://discuss.elastic.co/t/add-a-backslash-at-the-end-of-a-string/239336 "2020-06-30T16:54:02Z")
Hello, I want to add a backslash at the end of a field value, but logstash does not allow it: \`mutate { add\_field =\> { "\[prefix\]" =\> "%{\[domain\]}\\\\"}}\` Do you know how I could add a backslash at the end of the strin… --- ## [Not able to parse log file which contains xml format](https://discuss.elastic.co/t/not-able-to-parse-log-file-which-contains-xml-format/239348)
**Author:** [@Bhanu\_Praveen](https://discuss.elastic.co/u/Bhanu_Praveen)\ **Replies:** 0\ **Last updated:** [June 30, 2020, 4:49pm UTC](https://discuss.elastic.co/t/not-able-to-parse-log-file-which-contains-xml-format/239348 "2020-06-30T16:49:11Z")
Hello, I am having below webshpere GC log file which is generated in \*.log format. Different GC cycles and for every GC cycle start tag: "cycle-start" End Tage: "cycle-End" Now i need to parse each cycle as single mess… --- ## [\[ERROR\]\[logstash.filters.ruby \] Ruby exception occur red: too few arguments : Need Help](https://discuss.elastic.co/t/error-logstash-filters-ruby-ruby-exception-occur-red-too-few-arguments-need-help/238852)
**Author:** [@upasana\_sarangi](https://discuss.elastic.co/u/upasana_sarangi)\ **Replies:** 11\ **Last updated:** [June 30, 2020, 4:44pm UTC](https://discuss.elastic.co/t/error-logstash-filters-ruby-ruby-exception-occur-red-too-few-arguments-need-help/238852 "2020-06-30T16:44:12Z")
Hi , I am getting Ruby Exception too few arguments everytime I run logstash on our server . Below is the config and error msg I am trying to Parse :slight\_smile: Message : 20200515T005009.898-0500 Warning Fail Connect… --- ## [Zlib::GzipFile::Error: not in gzip format with input file codec =\> "gzip\_lines"](https://discuss.elastic.co/t/zlib-not-in-gzip-format-with-input-file-codec-gzip-lines/239334)
**Author:** [@jdelouche](https://discuss.elastic.co/u/jdelouche)\ **Replies:** 0\ **Last updated:** [June 30, 2020, 3:14pm UTC](https://discuss.elastic.co/t/zlib-not-in-gzip-format-with-input-file-codec-gzip-lines/239334 "2020-06-30T15:14:46Z")
Hi, I am not able to use the logstash-codec-gzip\_lines plugin because it tells me that .gz files are not in gzip format, but if I check the file I find no specific issue: gunzip -tv sl\_trace.trace.gz sl\_trace.trace.gz:… --- ## [Need Help on Logstash filter If-Else Condition](https://discuss.elastic.co/t/need-help-on-logstash-filter-if-else-condition/239333)
**Author:** [@Kim\_Mendoza](https://discuss.elastic.co/u/Kim_Mendoza)\ **Replies:** 0\ **Last updated:** [June 30, 2020, 3:12pm UTC](https://discuss.elastic.co/t/need-help-on-logstash-filter-if-else-condition/239333 "2020-06-30T15:12:32Z")
Please check my Logstash filter config. I cannot find my error logs cannot transfer to elasticsearch. filter { if \[fields\]\[logtype\] == "access.log" { grok { match =\> \[ "message", "%{WORD:host} %{WORD:host}\[ "%{NOTSP… --- ## [ELK : logstash with 3 different beats inputs on different ports with 3 different indexes](https://discuss.elastic.co/t/elk-logstash-with-3-different-beats-inputs-on-different-ports-with-3-different-indexes/239318)
**Author:** [@olivierdrueuil](https://discuss.elastic.co/u/olivierdrueuil)\ **Replies:** 0\ **Last updated:** [June 30, 2020, 1:50pm UTC](https://discuss.elastic.co/t/elk-logstash-with-3-different-beats-inputs-on-different-ports-with-3-different-indexes/239318 "2020-06-30T13:50:11Z")
hello I have setup an ELK with 3 beats configurations for 3 different services : httpd,mysql,ibm MQ \<\[root@elktest20 conf.d\]# pwd /etc/logstash/conf.d \[root@elktest20 conf.d\]# ls 02-beats-mysql.conf 03.beats-mq.c… --- ## [Logstash sending duplicate \_bulk requests without any apparent reason](https://discuss.elastic.co/t/logstash-sending-duplicate-bulk-requests-without-any-apparent-reason/239311)
**Author:** [@bartier](https://discuss.elastic.co/u/bartier)\ **Replies:** 0\ **Last updated:** [June 30, 2020, 1:11pm UTC](https://discuss.elastic.co/t/logstash-sending-duplicate-bulk-requests-without-any-apparent-reason/239311 "2020-06-30T13:11:37Z")
Hello, Today we have a setup like this: kafka cluster --\> (4 instances) logstash 5.6.8 --\> elasticsearch cluster Logstash instances consumes multiple topics from kafka and send \_bulk requests to elasticsearch cluster,… --- ## [Newline characters in syslog output](https://discuss.elastic.co/t/newline-characters-in-syslog-output/239279)
**Author:** [@brandonstark](https://discuss.elastic.co/u/brandonstark)\ **Replies:** 0\ **Last updated:** [June 30, 2020, 11:18am UTC](https://discuss.elastic.co/t/newline-characters-in-syslog-output/239279 "2020-06-30T11:18:41Z")
My incoming syslog has newline characters. In Kibana I see the messages as multiline text. The outgoing syslog messages change the newlines to literal "\\n". For example, the incoming message: incoming message app… --- ## [Kibana control Logstash](https://discuss.elastic.co/t/kibana-control-logstash/239275)
**Author:** [@Jasmin](https://discuss.elastic.co/u/Jasmin)\ **Replies:** 0\ **Last updated:** [June 30, 2020, 11:00am UTC](https://discuss.elastic.co/t/kibana-control-logstash/239275 "2020-06-30T11:00:16Z")
Hi , i want when kibana user specify exact date this date is writing to logstash : select \* from Tables where date = "variable" if this date does'nt exist on elasticsearch , so the logstash only select information wan… --- ## [Select Only new entries from Mssql](https://discuss.elastic.co/t/select-only-new-entries-from-mssql/239268)
**Author:** [@Jasmin](https://discuss.elastic.co/u/Jasmin)\ **Replies:** 1\ **Last updated:** [June 30, 2020, 10:13am UTC](https://discuss.elastic.co/t/select-only-new-entries-from-mssql/239268 "2020-06-30T10:13:01Z")
Hi, Logstash is listening to MSSQL using JBCD connector , i want to make logstash select only new entires because i have millions of Tables , and select them all every time gonna make MSSQL so Slow . so How can i mak… --- ## [Monitoring cluster logstash error](https://discuss.elastic.co/t/monitoring-cluster-logstash-error/238795)
**Author:** [@bevano](https://discuss.elastic.co/u/bevano)\ **Replies:** 4\ **Last updated:** [June 30, 2020, 12:53am UTC](https://discuss.elastic.co/t/monitoring-cluster-logstash-error/238795 "2020-06-30T00:53:18Z")
Hi all, I currently have a ELK stack in product and have metricbeat setup on every node logstash instances pushing metric and monitoring data to a separate monitoring cluster. Everything was working in 7.7.1, but since… --- ## [How schedule periodic task but execute when Logstash instance starts?](https://discuss.elastic.co/t/how-schedule-periodic-task-but-execute-when-logstash-instance-starts/238899)
**Author:** [@Fabio\_Naspolini](https://discuss.elastic.co/u/Fabio_Naspolini)\ **Replies:** 3\ **Last updated:** [June 29, 2020, 10:35pm UTC](https://discuss.elastic.co/t/how-schedule-periodic-task-but-execute-when-logstash-instance-starts/238899 "2020-06-29T22:35:29Z")
How schedule periodic task but execute on Logstash instance starts? I need to schedule tasks to execute every hour, but I need force re-execute this when Logstash service restarted. How to do this? Thank you! --- ## [Sflow data into Kafka?](https://discuss.elastic.co/t/sflow-data-into-kafka/236613)
**Author:** [@ceekay](https://discuss.elastic.co/u/ceekay)\ **Replies:** 1\ **Last updated:** [June 29, 2020, 9:45pm UTC](https://discuss.elastic.co/t/sflow-data-into-kafka/236613 "2020-06-29T21:45:25Z")
HI there, I'm shipping sflow data into my Elastic Stack, using Elastiflow's filters and dashboards. Input is UDP with sflow codec. My architecture has Logstash ingress nodes which are only doing minimal processing, and… --- ## [Use Bind-Mounted Setting for Local Config File?](https://discuss.elastic.co/t/use-bind-mounted-setting-for-local-config-file/238500)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 5\ **Last updated:** [June 29, 2020, 8:27pm UTC](https://discuss.elastic.co/t/use-bind-mounted-setting-for-local-config-file/238500 "2020-06-29T20:27:31Z")
Hello Logstash Wizards, I’m struggling to understand how I can spin up a Logstash Docker container using bind-mounted settings (see here) to run the container with a config file on the local host. To explain: Let’s say… --- ## [Parsing filter for nested json](https://discuss.elastic.co/t/parsing-filter-for-nested-json/239180)
**Author:** [@korrmal](https://discuss.elastic.co/u/korrmal)\ **Replies:** 1\ **Last updated:** [June 29, 2020, 8:26pm UTC](https://discuss.elastic.co/t/parsing-filter-for-nested-json/239180 "2020-06-29T20:26:52Z")
Hi. I have input string {'message': 'send data to', 'payload': {'params': {'cardNo': '77'}}, 'remote-url': 'https://api.com/status', 'cookie': {}, 'headers': {'User-Agent': 'req', 'Accept-Encoding': 'gzip, deflate', 'A… --- ## [Add Nested Field Value to a new field using Logstash Filter](https://discuss.elastic.co/t/add-nested-field-value-to-a-new-field-using-logstash-filter/239151)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 6\ **Last updated:** [June 29, 2020, 8:26pm UTC](https://discuss.elastic.co/t/add-nested-field-value-to-a-new-field-using-logstash-filter/239151 "2020-06-29T20:26:14Z")
Hello All, I hope someone here could help me as I have been stuck for days on this issue I am facing. I have got the kibana output as shown below: I am trying to add the nested field value of "key" into a new field c… --- ## [Can not add a new field using if filter and add\_field](https://discuss.elastic.co/t/can-not-add-a-new-field-using-if-filter-and-add-field/239127)
**Author:** [@brapnda23](https://discuss.elastic.co/u/brapnda23)\ **Replies:** 5\ **Last updated:** [June 29, 2020, 4:24pm UTC](https://discuss.elastic.co/t/can-not-add-a-new-field-using-if-filter-and-add-field/239127 "2020-06-29T16:24:04Z")
Newbie here. Trying to add another field but not working. I have added below set of codes to a .conf file under /etc/conf.d. Logs for logstash not showing any error. Am I doing something wrong here? Using the new 7.8 v… --- ## [Help with Logstash s3 output plugin refactoring (Feature Request #215)](https://discuss.elastic.co/t/help-with-logstash-s3-output-plugin-refactoring-feature-request-215/239141)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 0\ **Last updated:** [June 29, 2020, 3:22pm UTC](https://discuss.elastic.co/t/help-with-logstash-s3-output-plugin-refactoring-feature-request-215/239141 "2020-06-29T15:22:43Z")
I am working on refactoring logstash-output-s3 plugin so that it handles custom filenames (Feature Request #215) and I am running into an issue that has become a blocker for me and that is the refactored plugin honors th… --- ## [Logstash heap out of memory](https://discuss.elastic.co/t/logstash-heap-out-of-memory/239124)
**Author:** [@chandu5565](https://discuss.elastic.co/u/chandu5565)\ **Replies:** 0\ **Last updated:** [June 29, 2020, 2:17pm UTC](https://discuss.elastic.co/t/logstash-heap-out-of-memory/239124 "2020-06-29T14:17:58Z")
Hi , I am using logstash-7.7.0 to ingest my data from mongodb to elasticsearch using JDBC input plugin. I have around 17 configuration files which polling data from mongodb and the input section of all configuration fil… --- ## [Wrong timestamp, when trying to convert local time to UTC](https://discuss.elastic.co/t/wrong-timestamp-when-trying-to-convert-local-time-to-utc/239111)
**Author:** [@asmerb](https://discuss.elastic.co/u/asmerb)\ **Replies:** 2\ **Last updated:** [June 29, 2020, 2:21pm UTC](https://discuss.elastic.co/t/wrong-timestamp-when-trying-to-convert-local-time-to-utc/239111 "2020-06-29T14:21:09Z")
Hi everyone, I am using LogStash and ElasticSearch 6.3.1 Currently having an issue where my @timestamp is being shifted by two hours. I am picking up records in logstash that comes from filebeat, the timestamp here is… --- ## [Logstash integration Kafka: Support, or document support for, setting consumer isolation.level to read\_committed](https://discuss.elastic.co/t/logstash-integration-kafka-support-or-document-support-for-setting-consumer-isolation-level-to-read-committed/239116)
**Author:** [@Tim\_te\_Beek](https://discuss.elastic.co/u/Tim_te_Beek)\ **Replies:** 1\ **Last updated:** [June 29, 2020, 1:39pm UTC](https://discuss.elastic.co/t/logstash-integration-kafka-support-or-document-support-for-setting-consumer-isolation-level-to-read-committed/239116 "2020-06-29T13:39:41Z")
Hi; we're using https://github.com/logstash-plugins/logstash-integration-kafka to transcribe Kafka Avro message fields from Kafka to an Elastic search index for our reports. Recently we noticed that the consumer by defa… --- ## [Logstash filter plugins not operating on field](https://discuss.elastic.co/t/logstash-filter-plugins-not-operating-on-field/238870)
**Author:** [@Asinus1223](https://discuss.elastic.co/u/Asinus1223)\ **Replies:** 2\ **Last updated:** [June 29, 2020, 12:54pm UTC](https://discuss.elastic.co/t/logstash-filter-plugins-not-operating-on-field/238870 "2020-06-29T12:54:42Z")
I have been working with the notoriously difficult modsecurity\_audit.log in order to gather some useful statistics from it in order to tweak our Web Application firewall. So far, I've been able to separate the different… --- ## [How to scale out logstash when using output plugin websockets](https://discuss.elastic.co/t/how-to-scale-out-logstash-when-using-output-plugin-websockets/239106)
**Author:** [@Satish\_Kumar5](https://discuss.elastic.co/u/Satish_Kumar5)\ **Replies:** 0\ **Last updated:** [June 29, 2020, 11:45am UTC](https://discuss.elastic.co/t/how-to-scale-out-logstash-when-using-output-plugin-websockets/239106 "2020-06-29T11:45:40Z")
Hi, when using logstash output plugin for websockets, my understanding is the output plugin works as a websocket server, and sending data to websocket clients . my question is how to ensure scaling of the websockets if… --- ## [Logstash websocket input plugin throwing NoMethodError exception](https://discuss.elastic.co/t/logstash-websocket-input-plugin-throwing-nomethoderror-exception/237081)
**Author:** [@AlessandroKP](https://discuss.elastic.co/u/AlessandroKP)\ **Replies:** 3\ **Last updated:** [June 29, 2020, 10:02am UTC](https://discuss.elastic.co/t/logstash-websocket-input-plugin-throwing-nomethoderror-exception/237081 "2020-06-29T10:02:48Z")
Hi all, I am having some troubles using the Logstash websocket input plugin. I need Logstash to read a stream of events from a websocket endpoint and thus I am trying the following pipeline configuration input { webs… --- ## [Parsing S3 bucket into Logstash](https://discuss.elastic.co/t/parsing-s3-bucket-into-logstash/237996)
**Author:** [@SunilYadav](https://discuss.elastic.co/u/SunilYadav)\ **Replies:** 3\ **Last updated:** [June 29, 2020, 9:30am UTC](https://discuss.elastic.co/t/parsing-s3-bucket-into-logstash/237996 "2020-06-29T09:30:47Z")
Hi Team, when I am trying to use the S3 input plugin. S3 location:s3://bucket123/DAG/TASK/2020-04-26T16:01:00+00:00/1.log ''' The resulting error is : S3 input: Unable to download remote file {:remote\_key=\>"bucket12… --- ## [Break the string into terms](https://discuss.elastic.co/t/break-the-string-into-terms/239070)
**Author:** [@Mohammad.ali](https://discuss.elastic.co/u/Mohammad.ali)\ **Replies:** 0\ **Last updated:** [June 29, 2020, 8:08am UTC](https://discuss.elastic.co/t/break-the-string-into-terms/239070 "2020-06-29T08:08:36Z")
Hi every one Lets imagine i have a field with this string "Hi my name is mohammad ali" what iam looking for is to break the string as below into the terms: Hi my my name name is is mohammad mohammad ali then i ne… --- ## [Grok parse does not work, but looks fine on the debugger](https://discuss.elastic.co/t/grok-parse-does-not-work-but-looks-fine-on-the-debugger/238974)
**Author:** [@tenaz3](https://discuss.elastic.co/u/tenaz3)\ **Replies:** 9\ **Last updated:** [June 28, 2020, 8:26pm UTC](https://discuss.elastic.co/t/grok-parse-does-not-work-but-looks-fine-on-the-debugger/238974 "2020-06-28T20:26:28Z")
ELK: 7.71 On the debugger grok looks good, but when I send filebeat-\>logstash-\>elasticsearch, on kibana none of the fields I mapped is there. Any on has an idea? I have the following log structure: 2020-06-18T23:10:43… --- ## [Error with logstash-input-rss](https://discuss.elastic.co/t/error-with-logstash-input-rss/238997)
**Author:** [@Terizian](https://discuss.elastic.co/u/Terizian)\ **Replies:** 6\ **Last updated:** [June 28, 2020, 6:02pm UTC](https://discuss.elastic.co/t/error-with-logstash-input-rss/238997 "2020-06-28T18:02:37Z")
I was trying the RSS plugin with Logstash, and I'm facing an error as detailed below: Version: Logstash 7.7.1 Operating System: CentOS 7.8.2003 Config File: input { rss { url =\> "https://www.alit… --- ## [Can not get input from elasticsearch](https://discuss.elastic.co/t/can-not-get-input-from-elasticsearch/238988)
**Author:** [@AayushPatel](https://discuss.elastic.co/u/AayushPatel)\ **Replies:** 1\ **Last updated:** [June 28, 2020, 3:53pm UTC](https://discuss.elastic.co/t/can-not-get-input-from-elasticsearch/238988 "2020-06-28T15:53:14Z")
Hi, I wanna to get input from my elasticsearch and doing enrichment on the data in it but the problem is connecting to es via SSL, i get this error every time and can not handle it anyway the code input { elastic… --- ## [Force logstash to reindex a CSV](https://discuss.elastic.co/t/force-logstash-to-reindex-a-csv/239003)
**Author:** [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\ **Replies:** 3\ **Last updated:** [June 28, 2020, 3:46pm UTC](https://discuss.elastic.co/t/force-logstash-to-reindex-a-csv/239003 "2020-06-28T15:46:52Z")
Hello, Sometimes I need to force the re-indexing of files csv from the beginning. my config logstash is input { file { path =\> "/data/volumes/monitoring/logstash/dernier\_passage/stocks.csv" sincedb\_path =\> "/dev/null"… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=334) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=336)