# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=108 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 109 --- ## [How can I convert @timestamp from logstash to encoded format as YYYY-MM-DDThh:mm:ss.sss+/-hh:mm](https://discuss.elastic.co/t/how-can-i-convert-timestamp-from-logstash-to-encoded-format-as-yyyy-mm-ddthhss-sss-hh-mm/317608)
**Author:** [@teja\_tata](https://discuss.elastic.co/u/teja_tata)\ **Replies:** 11\ **Last updated:** [October 28, 2022, 1:50am UTC](https://discuss.elastic.co/t/how-can-i-convert-timestamp-from-logstash-to-encoded-format-as-yyyy-mm-ddthhss-sss-hh-mm/317608 "2022-10-28T01:50:18Z")
I have tried using date filter match option but didn't work. It would be great if someone can help me here. --- ## [Hanging shutdown with multiple pipelines. Shutdown of pipelines in wrong order](https://discuss.elastic.co/t/hanging-shutdown-with-multiple-pipelines-shutdown-of-pipelines-in-wrong-order/316354)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 2\ **Last updated:** [October 12, 2022, 7:17am UTC](https://discuss.elastic.co/t/hanging-shutdown-with-multiple-pipelines-shutdown-of-pipelines-in-wrong-order/316354 "2022-10-12T07:17:29Z")
Hi, I have a logstash config with multiple pipelines. One of the pipeline is a central output which ships to elasticsearch. The other pipelines have their own inputs and filters and having that central output pipeline a… --- ## [Logstash : How to extract a nested field from Json log and only index the content of the nested field](https://discuss.elastic.co/t/logstash-how-to-extract-a-nested-field-from-json-log-and-only-index-the-content-of-the-nested-field/317617)
**Author:** [@Ranjith\_kk](https://discuss.elastic.co/u/Ranjith_kk)\ **Replies:** 5\ **Last updated:** [October 27, 2022, 6:38pm UTC](https://discuss.elastic.co/t/logstash-how-to-extract-a-nested-field-from-json-log-and-only-index-the-content-of-the-nested-field/317617 "2022-10-27T18:38:57Z")
We have some logs in JSON format with a nested field called "data". We are looking for an option to extract only the content of this nested field and send it for indexing with ES. Actual log format: {"field1":"value1",… --- ## [Grok parse failure](https://discuss.elastic.co/t/grok-parse-failure/317379)
**Author:** [@Paf](https://discuss.elastic.co/u/Paf)\ **Replies:** 2\ **Last updated:** [October 27, 2022, 7:23am UTC](https://discuss.elastic.co/t/grok-parse-failure/317379 "2022-10-27T07:23:40Z")
Hello, I want to parse the field "ModifiedProperties" with this value : {"Name":"StrongAuthenticationMethod","NewValue":"\[\\r\\n {\\r\\n \\"MethodType\\": 5,\\r\\n \\"Default\\": true\\r\\n },\\r\\n {\\r\\n \\"MethodType\\":… --- ## [Logstash failing to convert from csv to json and to forward to opensearch](https://discuss.elastic.co/t/logstash-failing-to-convert-from-csv-to-json-and-to-forward-to-opensearch/317264)
**Author:** [@D\_Go](https://discuss.elastic.co/u/D_Go)\ **Replies:** 12\ **Last updated:** [October 27, 2022, 5:35am UTC](https://discuss.elastic.co/t/logstash-failing-to-convert-from-csv-to-json-and-to-forward-to-opensearch/317264 "2022-10-27T05:35:54Z")
Hi, I've been at this for a while and cant seem to load a csv file to AWS opensearch from logstash. Any help will be appreciated. """ \[ERROR\] 2022-10-23 04:09:20.063 \[Converge PipelineAction::Create\] agent - Failed to… --- ## [Configure Logstash to connect to Elasticsearch as output](https://discuss.elastic.co/t/configure-logstash-to-connect-to-elasticsearch-as-output/317578)
**Author:** [@Chma](https://discuss.elastic.co/u/Chma)\ **Replies:** 3\ **Last updated:** [October 27, 2022, 3:01am UTC](https://discuss.elastic.co/t/configure-logstash-to-connect-to-elasticsearch-as-output/317578 "2022-10-27T03:01:10Z")
I am running elasticsearch 8.4 and have configured logstash to index into elasticsearch. But I am getting the error: \]\[ERROR\]\[logstash.javapipeline \]\[main\] Pipeline error {:pipeline\_id=\>"main", :exception=\>#\ **Author:** [@Austin\_ES\_Questions](https://discuss.elastic.co/u/Austin_ES_Questions)\ **Replies:** 6\ **Last updated:** [October 26, 2022, 9:42pm UTC](https://discuss.elastic.co/t/string-inteprolation-in-logstash-data-stream-fields/317556 "2022-10-26T21:42:05Z") How can I write to different data streams for different kinesis input? We are trying to add fields in the inputs and use string interpolation in the outputs to no avail. We had been using index+ilm\_enabled in logstash, … --- ## [Action/metadata line \[1\] contains an unknown parameter \[\_script\]](https://discuss.elastic.co/t/action-metadata-line-1-contains-an-unknown-parameter-script/317548)
**Author:** [@nimish](https://discuss.elastic.co/u/nimish)\ **Replies:** 0\ **Last updated:** [October 26, 2022, 3:32pm UTC](https://discuss.elastic.co/t/action-metadata-line-1-contains-an-unknown-parameter-script/317548 "2022-10-26T15:32:41Z")
facing following issue while updating document using script Encountered a retryable error (will retry with exponential backoff) { "error": { "root\_cause": \[ { "type": "illegal\_argument\_exception", "reason"… --- ## [Referenced config files FileBeat (Windows)](https://discuss.elastic.co/t/referenced-config-files-filebeat-windows/317310)
**Author:** [@Remco1985](https://discuss.elastic.co/u/Remco1985)\ **Replies:** 6\ **Last updated:** [October 26, 2022, 1:15pm UTC](https://discuss.elastic.co/t/referenced-config-files-filebeat-windows/317310 "2022-10-26T13:15:20Z")
Hi, Recently we have successfully installed Filebeat on our (test) Windows server. Right now I’m investigating to find out the best deployment strategy for Filebeat agents on all of our Windows Server systems. We would… --- ## [Pmacct as\_path graphs](https://discuss.elastic.co/t/pmacct-as-path-graphs/317502)
**Author:** [@arphillips](https://discuss.elastic.co/u/arphillips)\ **Replies:** 0\ **Last updated:** [October 26, 2022, 10:40am UTC](https://discuss.elastic.co/t/pmacct-as-path-graphs/317502 "2022-10-26T10:40:50Z")
I am trying to make something akin to a sankey graph using showing the ASN paths. I'd like to be able to visualize the full path and each hop, pmacct is my netflow collector it sends the key "as\_path" in the format "6500… --- ## [I am getting message output data in logstash but i want to parse it to json](https://discuss.elastic.co/t/i-am-getting-message-output-data-in-logstash-but-i-want-to-parse-it-to-json/317469)
**Author:** [@ankurgupta](https://discuss.elastic.co/u/ankurgupta)\ **Replies:** 5\ **Last updated:** [October 26, 2022, 9:44am UTC](https://discuss.elastic.co/t/i-am-getting-message-output-data-in-logstash-but-i-want-to-parse-it-to-json/317469 "2022-10-26T09:44:54Z")
input{ exec { command =\> "netstat" interval =\> 30 } beats{ port =\>5044 } } filter{ grok{ match =\> {"message" =\> \[ #reqout grok: "%{DATESTAMP:time} (::slight\_smile: \[%{DATA:req}(:)\]\[(RQID:)%{DATA:rqId}\]\[(S… --- ## [Tcp output exception](https://discuss.elastic.co/t/tcp-output-exception/317227)
**Author:** [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Replies:** 5\ **Last updated:** [October 26, 2022, 9:44am UTC](https://discuss.elastic.co/t/tcp-output-exception/317227 "2022-10-26T09:44:14Z")
Hie Everyone, I have configured pipeline to read logs over tcp plugin. But im feeling tcp output exception every now and than the tcp connection is getting closed. Error logs: tcp output exception {:host=\>"10.109.0.0"… --- ## [Binary logs parsing Logstash sent via filebeat?](https://discuss.elastic.co/t/binary-logs-parsing-logstash-sent-via-filebeat/317281)
**Author:** [@ShubhamKumarJena](https://discuss.elastic.co/u/ShubhamKumarJena)\ **Replies:** 1\ **Last updated:** [October 26, 2022, 7:47am UTC](https://discuss.elastic.co/t/binary-logs-parsing-logstash-sent-via-filebeat/317281 "2022-10-26T07:47:23Z")
Hello ! I am trying to parse few logs which are in binary format hence can somebody guide how do we proceed with that? I am attaching the message field for reference: "message" =\> "\\u0000\\u0000\\u0000\\u000384p�\<�\\u0000… --- ## [Logstash trying to reprocess the same files](https://discuss.elastic.co/t/logstash-trying-to-reprocess-the-same-files/317362)
**Author:** [@Victor\_Caetano](https://discuss.elastic.co/u/Victor_Caetano)\ **Replies:** 1\ **Last updated:** [October 25, 2022, 4:20pm UTC](https://discuss.elastic.co/t/logstash-trying-to-reprocess-the-same-files/317362 "2022-10-25T16:20:19Z")
I'm using File Input Plugin, and a multiline codec, and grok filters to process .txt files that exists in a folder in my Linux VM, those files are static, i don't make changes in it's content, i only add more files to th… --- ## [Unable to output into Elasticsearch](https://discuss.elastic.co/t/unable-to-output-into-elasticsearch/317374)
**Author:** [@Hawas\_Musthafa](https://discuss.elastic.co/u/Hawas_Musthafa)\ **Replies:** 1\ **Last updated:** [October 25, 2022, 4:15pm UTC](https://discuss.elastic.co/t/unable-to-output-into-elasticsearch/317374 "2022-10-25T16:15:47Z")
Unable to output to the elasticseach. output { elasticsearch { hosts =\> "elasticsearch:9200" user =\> "logstash\_internal" password =\> "${LOGSTASH\_INTERNAL\_PASSWORD}" index =\> "testmanagement-%{+yyyy.MM.dd}" } and … --- ## [Split Document into multiple document based on condition](https://discuss.elastic.co/t/split-document-into-multiple-document-based-on-condition/316737)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 19\ **Last updated:** [October 25, 2022, 6:54am UTC](https://discuss.elastic.co/t/split-document-into-multiple-document-based-on-condition/316737 "2022-10-25T06:54:47Z")
Hi there, i plan to split field into multiple document based on OID. So, i have configured snmp pipeline for my network device. I managed to monitor the cpu and memory usage but i'm facing problem when i want to monitor… --- ## [Need help on syslog logstash input plugin](https://discuss.elastic.co/t/need-help-on-syslog-logstash-input-plugin/317181)
**Author:** [@niveditakathal](https://discuss.elastic.co/u/niveditakathal)\ **Replies:** 17\ **Last updated:** [October 24, 2022, 2:58pm UTC](https://discuss.elastic.co/t/need-help-on-syslog-logstash-input-plugin/317181 "2022-10-24T14:58:10Z")
Hi All, I have created a logstash pipeline to read the network syslog (RFC5424) data as mentioned below, However I don't see any output while running the pipeline. Can someone please assist me what I am missing. Note … --- ## [Grab everything upto a specific word](https://discuss.elastic.co/t/grab-everything-upto-a-specific-word/317280)
**Author:** [@Rajeev\_Shrestha](https://discuss.elastic.co/u/Rajeev_Shrestha)\ **Replies:** 5\ **Last updated:** [October 24, 2022, 12:19pm UTC](https://discuss.elastic.co/t/grab-everything-upto-a-specific-word/317280 "2022-10-24T12:19:02Z")
Hi Guys, I am new to GROK and appreciate your help. The goal is to put everything leading up to the word 'user-agent' to a fieldname "details" and everything after "user-agent" to a fieldname "user-agent". The excerpt… --- ## [Pattern: grok/mutate dont work?](https://discuss.elastic.co/t/pattern-grok-mutate-dont-work/317212)
**Author:** [@kurdit](https://discuss.elastic.co/u/kurdit)\ **Replies:** 9\ **Last updated:** [October 23, 2022, 10:59pm UTC](https://discuss.elastic.co/t/pattern-grok-mutate-dont-work/317212 "2022-10-23T22:59:40Z")
hi all! i ran into a problem. i have a pipeline: input { tcp { host =\> "10.10.10.10" port =\> "5959" codec =\> "json" type =\> "my\_type" mode =\> "server" } } filter { if \[… --- ## [Replace empty field with a meaning word](https://discuss.elastic.co/t/replace-empty-field-with-a-meaning-word/317266)
**Author:** [@Roger\_Huang](https://discuss.elastic.co/u/Roger_Huang)\ **Replies:** 3\ **Last updated:** [October 23, 2022, 4:23pm UTC](https://discuss.elastic.co/t/replace-empty-field-with-a-meaning-word/317266 "2022-10-23T16:23:53Z")
Hi everyone, I am trying to replace empty fields instead of %{} So i used this code: if ! \[xxevent.fw\_app\_id\] { mutate { update =\> { "xxevent.fw\_app\_id" =\> "EMPTY" } } } … --- ## [Index by hostname?](https://discuss.elastic.co/t/index-by-hostname/317187)
**Author:** [@Nikolas1306](https://discuss.elastic.co/u/Nikolas1306)\ **Replies:** 2\ **Last updated:** [October 22, 2022, 7:14am UTC](https://discuss.elastic.co/t/index-by-hostname/317187 "2022-10-22T07:14:25Z")
elasticsearch { hosts =\> \["localhost:9200"\] index =\> "logstash-%{\[host\]\[hostname\]}%{+YYYY.MM.dd}" } but have --- ## [Logstash convert single quoted json log to double quoted](https://discuss.elastic.co/t/logstash-convert-single-quoted-json-log-to-double-quoted/317164)
**Author:** [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Replies:** 4\ **Last updated:** [October 22, 2022, 3:40am UTC](https://discuss.elastic.co/t/logstash-convert-single-quoted-json-log-to-double-quoted/317164 "2022-10-22T03:40:05Z")
Hi I have log files with multiple json lines. Each json has single quotes and some of the values do not even has any quote. Due to this single quote and absence of quote issue, I cannot index them to elasticsearch. Thro… --- ## [HTTP Poller \[API\] input in Logstash - Does it support "Request\_data" on top of "Headers"?](https://discuss.elastic.co/t/http-poller-api-input-in-logstash-does-it-support-request-data-on-top-of-headers/316327)
**Author:** [@Roger\_Huang](https://discuss.elastic.co/u/Roger_Huang)\ **Replies:** 34\ **Last updated:** [October 22, 2022, 2:33am UTC](https://discuss.elastic.co/t/http-poller-api-input-in-logstash-does-it-support-request-data-on-top-of-headers/316327 "2022-10-22T02:33:41Z")
Hi everyone, I am new to ELK and I am trying to pull JSON data from Palo Alto platform into Logstash for processing. Some info redacted as xxxx. API DOCS: Get Alerts Does Logstash support such http headers? input … --- ## [Test and config logstash in docker](https://discuss.elastic.co/t/test-and-config-logstash-in-docker/317237)
**Author:** [@A\_Mightiev](https://discuss.elastic.co/u/A_Mightiev)\ **Replies:** 0\ **Last updated:** [October 21, 2022, 10:12pm UTC](https://discuss.elastic.co/t/test-and-config-logstash-in-docker/317237 "2022-10-21T22:12:12Z")
Hi Is there a way I can use the --config.test\_and\_exit\` functionality from logstash but running as a docker container? Thanks AM --- ## [\[8.4.2\] \[JDBC input plugin\] Scheduled prepared statement execution sporadically stops](https://discuss.elastic.co/t/8-4-2-jdbc-input-plugin-scheduled-prepared-statement-execution-sporadically-stops/317217)
**Author:** [@alromos](https://discuss.elastic.co/u/alromos)\ **Replies:** 0\ **Last updated:** [October 21, 2022, 4:11pm UTC](https://discuss.elastic.co/t/8-4-2-jdbc-input-plugin-scheduled-prepared-statement-execution-sporadically-stops/317217 "2022-10-21T16:11:03Z")
Hi there We have 25 logstash pipelines utilizing JDBC input plugin with schedule. input { jdbc { ... schedule =\> "30 \* \* \* \* \*" ... } } Each pipeline has its own value instead of 30, so that prepared s… --- ## [Logstash is blocked by elasticsearch-setup-passwords](https://discuss.elastic.co/t/logstash-is-blocked-by-elasticsearch-setup-passwords/317142)
**Author:** [@Nikolas1306](https://discuss.elastic.co/u/Nikolas1306)\ **Replies:** 19\ **Last updated:** [October 21, 2022, 1:34pm UTC](https://discuss.elastic.co/t/logstash-is-blocked-by-elasticsearch-setup-passwords/317142 "2022-10-21T13:34:26Z")
this a very bad procedure i have used this command for create password for kibana elasticsearch-setup-passwords but logstash is non started --- ## [Remove opening and closing parenthesis using gsub](https://discuss.elastic.co/t/remove-opening-and-closing-parenthesis-using-gsub/317100)
**Author:** [@ombit](https://discuss.elastic.co/u/ombit)\ **Replies:** 6\ **Last updated:** [October 21, 2022, 9:51am UTC](https://discuss.elastic.co/t/remove-opening-and-closing-parenthesis-using-gsub/317100 "2022-10-21T09:51:47Z")
Hi all, I am using Logstash to change the formatting of messages before forwarding them on to a QRadar reader. The syslogs are coming from various other servers, being collated onto one central 'master' server within … --- ## [Encoding issue depending on how logstash is started](https://discuss.elastic.co/t/encoding-issue-depending-on-how-logstash-is-started/317136)
**Author:** [@simon137](https://discuss.elastic.co/u/simon137)\ **Replies:** 10\ **Last updated:** [October 21, 2022, 9:32am UTC](https://discuss.elastic.co/t/encoding-issue-depending-on-how-logstash-is-started/317136 "2022-10-21T09:32:14Z")
Hello, I have two windows maschines, one is a Windows Server and one is a Standard Windows Desktop PC, both have the same elasticsearch (8.4.3) and the same logstash (8.4.3) installed with the same configuration and the… --- ## [Logstash cannot establish pipeline to communicate with elastic search](https://discuss.elastic.co/t/logstash-cannot-establish-pipeline-to-communicate-with-elastic-search/317180)
**Author:** [@Siva\_Priya](https://discuss.elastic.co/u/Siva_Priya)\ **Replies:** 0\ **Last updated:** [October 21, 2022, 9:14am UTC](https://discuss.elastic.co/t/logstash-cannot-establish-pipeline-to-communicate-with-elastic-search/317180 "2022-10-21T09:14:31Z")
I am trying to collect the logs from kubernetes pods and pass it to elastic db using logstash and filebeats.I am able to deploy the Elasticsearch and kibana in k8 cluster but i am not able to deploy my logstash and fileb… --- ## [Logstash Pipeline](https://discuss.elastic.co/t/logstash-pipeline/317155)
**Author:** [@QuestBevan](https://discuss.elastic.co/u/QuestBevan)\ **Replies:** 1\ **Last updated:** [October 21, 2022, 8:42am UTC](https://discuss.elastic.co/t/logstash-pipeline/317155 "2022-10-21T08:42:28Z")
Hi All, Urgent problem here. Have raised a support ticket but hoping to find a solution quicker! We have our logstash pipelines being managed by Kibana. A pipeline was accidently overwritten with an incorrect configura… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=107) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=109)