# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=168 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 169 --- ## [How to escape new line character?](https://discuss.elastic.co/t/how-to-escape-new-line-character/294254)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 2\ **Last updated:** [January 13, 2022, 4:17pm UTC](https://discuss.elastic.co/t/how-to-escape-new-line-character/294254 "2022-01-13T16:17:22Z")
"MethodParameter" =\> "{\\r\\n \\"getCardholderBalance\\": {\\r\\n \\"acquirer\\": {\\r\\n \\"id\\": \\"testdemo\\",\\r\\n \\"userId\\": \\"xxxxx\\",\\r\\n \\"password\\": \\"xxxxxxxxxxxxxxx\\"\\r\\n },\\r\\n \\"card\\": {\\r\\n… --- ## [How to remove date and time in message](https://discuss.elastic.co/t/how-to-remove-date-and-time-in-message/294280)
**Author:** [@Roccof97](https://discuss.elastic.co/u/Roccof97)\ **Replies:** 3\ **Last updated:** [January 13, 2022, 2:59pm UTC](https://discuss.elastic.co/t/how-to-remove-date-and-time-in-message/294280 "2022-01-13T14:59:56Z")
Hi, how can i remove the date and time from the message field? and above all is it possible? example screen shot: --- ## [Delete value from filed in recursive way](https://discuss.elastic.co/t/delete-value-from-filed-in-recursive-way/294088)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 4\ **Last updated:** [January 13, 2022, 2:44pm UTC](https://discuss.elastic.co/t/delete-value-from-filed-in-recursive-way/294088 "2022-01-13T14:44:29Z")
Hi I need to remove from a few fields in the recursive way some value so I've tried in two steps but without success, can You check what's wrong input data: "ue\_stat" =\> "undefined-undefined", "s\_gw" =\> "122.123.205… --- ## [Elasticsearch - Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down!](https://discuss.elastic.co/t/elasticsearch-attempted-to-send-a-bulk-request-to-elasticsearch-but-elasticsearch-appears-to-be-unreachable-or-down/294287)
**Author:** [@makeshkumar](https://discuss.elastic.co/u/makeshkumar)\ **Replies:** 0\ **Last updated:** [January 13, 2022, 2:00pm UTC](https://discuss.elastic.co/t/elasticsearch-attempted-to-send-a-bulk-request-to-elasticsearch-but-elasticsearch-appears-to-be-unreachable-or-down/294287 "2022-01-13T14:00:12Z")
Hi Team, We are using the ELK stack and we are getting the below error in our logstash pods logs . Logs are sending slowly or some time it is not at all processing the logs . Note: We checked the elasatic search servi… --- ## [Is it possible to connect Logstash 7.16 with Elasticsearch "8.1.0-SNAPSHOT"](https://discuss.elastic.co/t/is-it-possible-to-connect-logstash-7-16-with-elasticsearch-8-1-0-snapshot/294272)
**Author:** [@Abhishek\_Gautam](https://discuss.elastic.co/u/Abhishek_Gautam)\ **Replies:** 1\ **Last updated:** [January 13, 2022, 1:55pm UTC](https://discuss.elastic.co/t/is-it-possible-to-connect-logstash-7-16-with-elasticsearch-8-1-0-snapshot/294272 "2022-01-13T13:55:30Z")
is it possible to connect Logstash 7.16 with Elasticsearch "8.1.0-SNAPSHOT" --- ## [Logstash with docker logs and tags](https://discuss.elastic.co/t/logstash-with-docker-logs-and-tags/294278)
**Author:** [@Spyros\_Agriopoulos](https://discuss.elastic.co/u/Spyros_Agriopoulos)\ **Replies:** 0\ **Last updated:** [January 13, 2022, 1:33pm UTC](https://discuss.elastic.co/t/logstash-with-docker-logs-and-tags/294278 "2022-01-13T13:33:57Z")
Hello, I am trying to send a docker log from one machine with filebeat and add a tag to it, and when it reaches logstash it should mutate it and the give it an ilm\_rollover\_alias. The log appears in kibana, but with the … --- ## [How to enable cache in logstash](https://discuss.elastic.co/t/how-to-enable-cache-in-logstash/294126)
**Author:** [@Roccof97](https://discuss.elastic.co/u/Roccof97)\ **Replies:** 2\ **Last updated:** [January 13, 2022, 1:29pm UTC](https://discuss.elastic.co/t/how-to-enable-cache-in-logstash/294126 "2022-01-13T13:29:33Z")
Hello, I currently have an elk cluster composed as follows: Beats= is the agent installed in linux machine and send logs to Logstash Logstash= filter logs and send logs to elk -Beats --\> Logstash --\> Elasticsearch m… --- ## [Value too large to output (49839 bytes) logstash error](https://discuss.elastic.co/t/value-too-large-to-output-49839-bytes-logstash-error/294170)
**Author:** [@ZZ.IT](https://discuss.elastic.co/u/ZZ.IT)\ **Replies:** 2\ **Last updated:** [January 13, 2022, 11:02am UTC](https://discuss.elastic.co/t/value-too-large-to-output-49839-bytes-logstash-error/294170 "2022-01-13T11:02:33Z")
We are using ELK stack on windows server 2016. ELK stack has version 7.16.2. We are sending data from filebeat to logstash using pipeline and the pipeline har grok plugin. On sending the data we are getting the error i… --- ## [Logstash running in openshift is not sending output to NFS](https://discuss.elastic.co/t/logstash-running-in-openshift-is-not-sending-output-to-nfs/294242)
**Author:** [@Ravi\_Sehgal](https://discuss.elastic.co/u/Ravi_Sehgal)\ **Replies:** 0\ **Last updated:** [January 13, 2022, 8:11am UTC](https://discuss.elastic.co/t/logstash-running-in-openshift-is-not-sending-output-to-nfs/294242 "2022-01-13T08:11:41Z")
I have a requirement to send docker container logs to NFS drive. Filebeat installed as daemonset sends logs to single logstash pod which shall write them to NFS. Logstatsh configuration look something like below outpu… --- ## ["Azure Data Explorer" output plugin error](https://discuss.elastic.co/t/azure-data-explorer-output-plugin-error/294197)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 4\ **Last updated:** [January 13, 2022, 2:45am UTC](https://discuss.elastic.co/t/azure-data-explorer-output-plugin-error/294197 "2022-01-13T02:45:11Z")
Hello All, Using the kusto output plugin for Azure Data explorer I did a successful test with the example as per the documentation below References: Ingest data from Logstash to Azure Data Explorer | Microsoft Docs G… --- ## [How to run script query on logstash](https://discuss.elastic.co/t/how-to-run-script-query-on-logstash/294227)
**Author:** [@mzinboy](https://discuss.elastic.co/u/mzinboy)\ **Replies:** 0\ **Last updated:** [January 13, 2022, 2:24am UTC](https://discuss.elastic.co/t/how-to-run-script-query-on-logstash/294227 "2022-01-13T02:24:13Z")
It doesn't work(logstash 7.15.1). I can't escape single quotes in query field. Elasticsearch input value : input { elasticsearch { hosts =\> \["..."\] index =\> "..." query =\> '{ "query": { "bool": { "… --- ## [Log4j security vulnerability for logstash 6.2.2](https://discuss.elastic.co/t/log4j-security-vulnerability-for-logstash-6-2-2/294205)
**Author:** [@ellje](https://discuss.elastic.co/u/ellje)\ **Replies:** 0\ **Last updated:** [January 12, 2022, 8:46pm UTC](https://discuss.elastic.co/t/log4j-security-vulnerability-for-logstash-6-2-2/294205 "2022-01-12T20:46:46Z")
Hi, I'm using an older version of Logstash (6.2.2) and I cannot migrate it just yet. In the meantime, I read that to remediate the log4j issue, we should be removing the jndi class as noted in Apache Log4j2 Remote Code … --- ## [Logstash with log4j 2.17.1 patch release](https://discuss.elastic.co/t/logstash-with-log4j-2-17-1-patch-release/294195)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 2\ **Last updated:** [January 12, 2022, 9:09pm UTC](https://discuss.elastic.co/t/logstash-with-log4j-2-17-1-patch-release/294195 "2022-01-12T21:09:49Z")
Hi Logstash team, The current latest logstash version 7.16.2 is bundled with log4j 2.17.0. Are there any plans to release a new patch version with log4j 2.17.1? Many thanks, --- ## [Create Parent-Child relationship with Logstash](https://discuss.elastic.co/t/create-parent-child-relationship-with-logstash/294198)
**Author:** [@RusseL](https://discuss.elastic.co/u/RusseL)\ **Replies:** 3\ **Last updated:** [January 12, 2022, 8:22pm UTC](https://discuss.elastic.co/t/create-parent-child-relationship-with-logstash/294198 "2022-01-12T20:22:19Z")
Hi Everyone; I need to create a Parent-Chield relationship with Logstash. I created index like this. Here is the mapping I try to create relationship using filter mutate as below picture But I am getting this… --- ## [Logstash pipeline filter](https://discuss.elastic.co/t/logstash-pipeline-filter/294018)
**Author:** [@max\_cyril](https://discuss.elastic.co/u/max_cyril)\ **Replies:** 7\ **Last updated:** [January 12, 2022, 6:36pm UTC](https://discuss.elastic.co/t/logstash-pipeline-filter/294018 "2022-01-12T18:36:47Z")
Hi, I am new to ELK and struggling to write my first logstash pipeline. Can anyone help me to write the filter section? Thanks in advance. i want to filter and only output the maximum of completion per users fo… --- ## [Best practice for logging in an microservice-based architecture](https://discuss.elastic.co/t/best-practice-for-logging-in-an-microservice-based-architecture/294023)
**Author:** [@peoh](https://discuss.elastic.co/u/peoh)\ **Replies:** 1\ **Last updated:** [January 12, 2022, 5:16pm UTC](https://discuss.elastic.co/t/best-practice-for-logging-in-an-microservice-based-architecture/294023 "2022-01-12T17:16:46Z")
Hi! My current environment has multiple applications on multiple servers that logs to a local file and then uses Filebeat to send logs to Logstash for processing. This works fine and I like that the logs will show up ev… --- ## [Programmatically update Logstash configuration file](https://discuss.elastic.co/t/programmatically-update-logstash-configuration-file/294096)
**Author:** [@Matt\_Jones](https://discuss.elastic.co/u/Matt_Jones)\ **Replies:** 3\ **Last updated:** [January 12, 2022, 3:43pm UTC](https://discuss.elastic.co/t/programmatically-update-logstash-configuration-file/294096 "2022-01-12T15:43:58Z")
Hi, I would like to create multiple ingestion pipelines in Elastic Cloud, each of which containing a pipeline configuration (inputs, filters, outputs). Different pipelines are being created to support different custome… --- ## [\[logstash.outputsCould not index event to Elasticsearch."error {"type"=\>"mapper\_parsing\_exception", "reason"=\>"object mapping for \[host\] tried to parse field \[host\] as object, but found a concrete value"](https://discuss.elastic.co/t/logstash-outputscould-not-index-event-to-elasticsearch-error-type-mapper-parsing-exception-reason-object-mapping-for-host-tried-to-parse-field-host-as-object-but-found-a-concrete-value/294000)
**Author:** [@jannatnishat](https://discuss.elastic.co/u/jannatnishat)\ **Replies:** 3\ **Last updated:** [January 12, 2022, 2:18pm UTC](https://discuss.elastic.co/t/logstash-outputscould-not-index-event-to-elasticsearch-error-type-mapper-parsing-exception-reason-object-mapping-for-host-tried-to-parse-field-host-as-object-but-found-a-concrete-value/294000 "2022-01-12T14:18:38Z")
Getting this error \[2022-01-11T12:48:53,887\]\[WARN \]\[logstash.outputs.elasticsearch\]\[main\]\[d60352f946d70780df98c56812b4beacd1633b0b5fb5d9c306d2c578620115b7\] Could not index event to Elasticsearch. {:status=\>400, :action=\>… --- ## [Shipping logs of system Linux with Filebeat and Logstash](https://discuss.elastic.co/t/shipping-logs-of-system-linux-with-filebeat-and-logstash/294137)
**Author:** [@Lynow](https://discuss.elastic.co/u/Lynow)\ **Replies:** 0\ **Last updated:** [January 12, 2022, 9:35am UTC](https://discuss.elastic.co/t/shipping-logs-of-system-linux-with-filebeat-and-logstash/294137 "2022-01-12T09:35:05Z")
Hello, I am setting up a log monitoring architecture, for this I am using Opensearch and Kibana to collect all the data. The data is sent by Filebeat, which retrieves the logs from Wazuh (with different agents). I then… --- ## [Logstash is terminating](https://discuss.elastic.co/t/logstash-is-terminating/294001)
**Author:** [@HELIXInternational](https://discuss.elastic.co/u/HELIXInternational)\ **Replies:** 2\ **Last updated:** [January 12, 2022, 6:37am UTC](https://discuss.elastic.co/t/logstash-is-terminating/294001 "2022-01-12T06:37:09Z")
We upgraded the logstash from7.7.0 to 7.16.2, after upgrade the pipeline service is terminating. Can you please help on this. Config file: powershell .\\bin\\logstash -f config\\harvester-pipeline.conf --… --- ## [File Input, just one file, with one line that is updated every 5 minutes](https://discuss.elastic.co/t/file-input-just-one-file-with-one-line-that-is-updated-every-5-minutes/294074)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 11:56pm UTC](https://discuss.elastic.co/t/file-input-just-one-file-with-one-line-that-is-updated-every-5-minutes/294074 "2022-01-11T23:56:05Z")
Hi, I need to read from logstash one file: number\_of\_connections.txt this file is updated every 5 minutes with a new number, so no new lines are added. can I do this with file input? if not is there another input that … --- ## [Importing file to an existing index](https://discuss.elastic.co/t/importing-file-to-an-existing-index/293932)
**Author:** [@kibanauser4](https://discuss.elastic.co/u/kibanauser4)\ **Replies:** 4\ **Last updated:** [January 11, 2022, 3:11pm UTC](https://discuss.elastic.co/t/importing-file-to-an-existing-index/293932 "2022-01-11T15:11:09Z")
I wanted to update my index by importing a file via Logstash. I used the same logstash file (the input file was updated with new records) cause I thought it's gonna overwrite the existing index. I tested this on a differ… --- ## [Json nested field](https://discuss.elastic.co/t/json-nested-field/293925)
**Author:** [@kvmuralidhar](https://discuss.elastic.co/u/kvmuralidhar)\ **Replies:** 2\ **Last updated:** [January 11, 2022, 3:02pm UTC](https://discuss.elastic.co/t/json-nested-field/293925 "2022-01-11T15:02:30Z")
Hi All, I have a field with value as a json object as follows messageParts="\[{\\"disposition\\":\\"inline\\",\\"sha256\\":\\"f3c958fe6406140b13360a42b3237477dc5bc525f3858f64cca45bf2d02fc771\\",\\"md5\\":\\"c87f14af3c845fc6d8b84b1… --- ## [Fortigate Firewall Logs to Elasticsearch](https://discuss.elastic.co/t/fortigate-firewall-logs-to-elasticsearch/294041)
**Author:** [@yogicd](https://discuss.elastic.co/u/yogicd)\ **Replies:** 2\ **Last updated:** [January 11, 2022, 2:42pm UTC](https://discuss.elastic.co/t/fortigate-firewall-logs-to-elasticsearch/294041 "2022-01-11T14:42:53Z")
Hi Team, I am trying to get the Fortigate firewall logs to elasticsearch via logstash but not able to get the data to elasticsearch, But i can see the data coming via tcpdump udp port 514. and my logstash config as… --- ## [Update to latest version compatibility](https://discuss.elastic.co/t/update-to-latest-version-compatibility/294031)
**Author:** [@FleaLes](https://discuss.elastic.co/u/FleaLes)\ **Replies:** 4\ **Last updated:** [January 11, 2022, 2:36pm UTC](https://discuss.elastic.co/t/update-to-latest-version-compatibility/294031 "2022-01-11T14:36:38Z")
Hi. I've inherited an ELK setup as follows: Elasticsearch / Logstash / Kibana / Filebeat: version 6.3.0 Curator: version 5.5.4 Docker Composer: version 3 HAProxy: version 1.8.12 I'm planning to upgrade the whole stac… --- ## [Logstash Input adds data stamp and host](https://discuss.elastic.co/t/logstash-input-adds-data-stamp-and-host/294021)
**Author:** [@pruttle](https://discuss.elastic.co/u/pruttle)\ **Replies:** 7\ **Last updated:** [January 11, 2022, 2:23pm UTC](https://discuss.elastic.co/t/logstash-input-adds-data-stamp-and-host/294021 "2022-01-11T14:23:17Z")
I am using Logstash to receive data from a syslog server and push it into different Kafka topics. Logstash appears to add a data/time stamp and hostname at that start of the raw event. Is there a way stop this happening?… --- ## [Parsing log date into timestamp](https://discuss.elastic.co/t/parsing-log-date-into-timestamp/293966)
**Author:** [@phung025](https://discuss.elastic.co/u/phung025)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 1:31pm UTC](https://discuss.elastic.co/t/parsing-log-date-into-timestamp/293966 "2022-01-11T13:31:07Z")
In the JSON-format logs sent from filebeat to logstash, I have a field named "time". In the logstash.conf, I mutate it to create a field in the kibana log called rawDate logstash.conf: mutate { add\_field =\> {"rawDa… --- ## [How to parse suricata rules with logstash](https://discuss.elastic.co/t/how-to-parse-suricata-rules-with-logstash/292548)
**Author:** [@karpaz](https://discuss.elastic.co/u/karpaz)\ **Replies:** 7\ **Last updated:** [January 11, 2022, 11:58am UTC](https://discuss.elastic.co/t/how-to-parse-suricata-rules-with-logstash/292548 "2022-01-11T11:58:54Z")
Hi, guys, I've been solving a "parsing" problem for a while now. First I tried it with GROK, then with KV filter. I have not been successful with either method. So I am turning to you for help to see if anyone experien… --- ## [Logstasch pipeline filter](https://discuss.elastic.co/t/logstasch-pipeline-filter/294009)
**Author:** [@max\_cyril](https://discuss.elastic.co/u/max_cyril)\ **Replies:** 1\ **Last updated:** [January 11, 2022, 10:13am UTC](https://discuss.elastic.co/t/logstasch-pipeline-filter/294009 "2022-01-11T10:13:14Z")
Hi, I am new to ELK and struggling to write my first logstash pipeline. Can anyone help me to write the filter section? Thanks in advance. i want to filter and only output the maximum of completion per users fo… --- ## [How to loop through array in Logstash?](https://discuss.elastic.co/t/how-to-loop-through-array-in-logstash/291747)
**Author:** [@rijo\_joseph](https://discuss.elastic.co/u/rijo_joseph)\ **Replies:** 4\ **Last updated:** [January 11, 2022, 9:15am UTC](https://discuss.elastic.co/t/how-to-loop-through-array-in-logstash/291747 "2022-01-11T09:15:10Z")
{ "modifiedTime": "2021-12-13T06:47:11.138Z", "task\_owner\_name": null, "contents": \[ { "eligible": false, "rawResponse": "454536251", "id": 868286622 }, { "eligible": false, … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=167) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=169)