# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=201 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 202 --- ## ["Restored connection to ES Instance" warning at startup](https://discuss.elastic.co/t/restored-connection-to-es-instance-warning-at-startup/281752)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 4\ **Last updated:** [August 18, 2021, 9:11pm UTC](https://discuss.elastic.co/t/restored-connection-to-es-instance-warning-at-startup/281752 "2021-08-18T21:11:29Z")
At logstash startup i get these weird warnings. Here are the warning: \[2021-08-17T21:33:03,732\]\[WARN \]\[logstash.outputs.elasticsearch\]\[straight-es\] Restored connection to ES instance {:url=\>"https://logstash\_internal:x… --- ## [How can I parse 2 log files to 2 index in Kibana](https://discuss.elastic.co/t/how-can-i-parse-2-log-files-to-2-index-in-kibana/281711)
**Author:** [@MT2021](https://discuss.elastic.co/u/MT2021)\ **Replies:** 5\ **Last updated:** [August 18, 2021, 2:31pm UTC](https://discuss.elastic.co/t/how-can-i-parse-2-log-files-to-2-index-in-kibana/281711 "2021-08-18T14:31:10Z")
Hi ELK community, I am a absolutely newbie in the Elasticsearch. I am getting a stuck in creating 2 pipelines towards 2 indexes of 2 log file (2 services in corresponding). My topo likes that: Filebeat ==\> logstash =… --- ## [Logstash file input not processing (windows)](https://discuss.elastic.co/t/logstash-file-input-not-processing-windows/281720)
**Author:** [@alejandroalv87](https://discuss.elastic.co/u/alejandroalv87)\ **Replies:** 5\ **Last updated:** [August 18, 2021, 12:23pm UTC](https://discuss.elastic.co/t/logstash-file-input-not-processing-windows/281720 "2021-08-18T12:23:43Z")
I am trying to process an apache log file to logstash and file is not getting process. My config file here: input { file { path =\> "C:\\Users\\alejandro.alvarez\\Downloads\\elasticsearch\_kibana\\data\\logs\\apache\_access.lo… --- ## [Logstash data unload](https://discuss.elastic.co/t/logstash-data-unload/281506)
**Author:** [@onlinekap](https://discuss.elastic.co/u/onlinekap)\ **Replies:** 8\ **Last updated:** [August 18, 2021, 7:30am UTC](https://discuss.elastic.co/t/logstash-data-unload/281506 "2021-08-18T07:30:17Z")
Hello, I am newbie here.. I have a bunch of ELK indexes that returns below output when querying via POSTMAN GET request. \ { "took": 5, "timed\_out": false, "\_shards": { "total": 1, "successful": 1, "skipped": 0… --- ## [Logstash always outputting only 9 documents](https://discuss.elastic.co/t/logstash-always-outputting-only-9-documents/281751)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 2\ **Last updated:** [August 17, 2021, 10:19pm UTC](https://discuss.elastic.co/t/logstash-always-outputting-only-9-documents/281751 "2021-08-17T22:19:24Z")
I have a json file with 4K json objects in it. Each object is on its own line. I'm using the file plugin with the multiline codec to have Logstash parse all the objects. I'm also flattening the json objects and then send… --- ## [Logstash forwarding MSFT Server logs into Azure](https://discuss.elastic.co/t/logstash-forwarding-msft-server-logs-into-azure/281708)
**Author:** [@JBHuber](https://discuss.elastic.co/u/JBHuber)\ **Replies:** 1\ **Last updated:** [August 17, 2021, 4:45pm UTC](https://discuss.elastic.co/t/logstash-forwarding-msft-server-logs-into-azure/281708 "2021-08-17T16:45:54Z")
Hoping someone here has tried to do this same thing. We are using Logstash as a forwarder of linux & windows logs into MSFT Azure Sentinel. We do segregate (or TAG) the logs as "windows" and "linux". Linux is fine and … --- ## [XML filter - processing of an array of values](https://discuss.elastic.co/t/xml-filter-processing-of-an-array-of-values/281601)
**Author:** [@Robo](https://discuss.elastic.co/u/Robo)\ **Replies:** 4\ **Last updated:** [August 17, 2021, 2:55pm UTC](https://discuss.elastic.co/t/xml-filter-processing-of-an-array-of-values/281601 "2021-08-17T14:55:23Z")
We're processing some xml events using the xml filter. The filter setup it's quite simple: filter { xml { source =\> "message" target =\> "xml" } } This filter generates two fields with an array of v… --- ## [Filter out the same event and send to different outputs](https://discuss.elastic.co/t/filter-out-the-same-event-and-send-to-different-outputs/281611)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 4\ **Last updated:** [August 17, 2021, 2:40pm UTC](https://discuss.elastic.co/t/filter-out-the-same-event-and-send-to-different-outputs/281611 "2021-08-17T14:40:24Z")
What I want is, the event i get from the Beats by listening to the port 5044, to send a filtered version of the event to one output and send a differently filtered version to the other output. That is the general questi… --- ## [Compare float value generated from Glok Pattern](https://discuss.elastic.co/t/compare-float-value-generated-from-glok-pattern/281616)
**Author:** [@Athul\_Devkar](https://discuss.elastic.co/u/Athul_Devkar)\ **Replies:** 4\ **Last updated:** [August 17, 2021, 2:38pm UTC](https://discuss.elastic.co/t/compare-float-value-generated-from-glok-pattern/281616 "2021-08-17T14:38:26Z")
Dear All, I am trying to compare 2 float fields, which are created from Grok patterns match. But this is crashing the pipeline. Not able to figure out what is missing here! Kindly help. if \[message\] =~ "Customer Type … --- ## [Pipelines.yml failed to read with Pipeline to Pipeline model](https://discuss.elastic.co/t/pipelines-yml-failed-to-read-with-pipeline-to-pipeline-model/281659)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 2\ **Last updated:** [August 17, 2021, 2:32pm UTC](https://discuss.elastic.co/t/pipelines-yml-failed-to-read-with-pipeline-to-pipeline-model/281659 "2021-08-17T14:32:44Z")
I had previously multiple pipelines and i added Pipeline to Pipeline model in my pipelines.yml But since then i get this error: ERROR: Failed to read pipelines yaml file. Location: /usr/share/logstash/config/pipelines.… --- ## [Unable to remove "timestamp" field (not "@timestamp"!)](https://discuss.elastic.co/t/unable-to-remove-timestamp-field-not-timestamp/281654)
**Author:** [@jze1](https://discuss.elastic.co/u/jze1)\ **Replies:** 1\ **Last updated:** [August 17, 2021, 2:25pm UTC](https://discuss.elastic.co/t/unable-to-remove-timestamp-field-not-timestamp/281654 "2021-08-17T14:25:18Z")
I am seeing the error below in logstash log. So I'm trying to parse the string further but after that mutate { remove\_field =\> \[ "timestamp" \] } does NOTHING. WTF? "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"… --- ## [How can I configure logstash in this way?](https://discuss.elastic.co/t/how-can-i-configure-logstash-in-this-way/281561)
**Author:** [@xemyleex](https://discuss.elastic.co/u/xemyleex)\ **Replies:** 5\ **Last updated:** [August 17, 2021, 1:53pm UTC](https://discuss.elastic.co/t/how-can-i-configure-logstash-in-this-way/281561 "2021-08-17T13:53:31Z")
I need a way to configure a logstash instance to work as load balancer for other 5 logstash instances. How can I do that? and what are the advantages and the disavantages to use multiple instances instead of only one? … --- ## [Need to get any help in parsing a complicated logs](https://discuss.elastic.co/t/need-to-get-any-help-in-parsing-a-complicated-logs/281671)
**Author:** [@MT2021](https://discuss.elastic.co/u/MT2021)\ **Replies:** 2\ **Last updated:** [August 17, 2021, 1:49pm UTC](https://discuss.elastic.co/t/need-to-get-any-help-in-parsing-a-complicated-logs/281671 "2021-08-17T13:49:27Z")
Hi team, I am very new in Grok. I am struggling at this point. I cannot make a parser for below log file. The issue comes in \[17/Aug/2021:16:46:33 +0700\]. I cannot pass it. 192.168.40.100 - - \[17/Aug/2021:16:46:33 +0… --- ## [Creating index name as per log file name](https://discuss.elastic.co/t/creating-index-name-as-per-log-file-name/281677)
**Author:** [@mangesh\_shinde](https://discuss.elastic.co/u/mangesh_shinde)\ **Replies:** 0\ **Last updated:** [August 17, 2021, 11:52am UTC](https://discuss.elastic.co/t/creating-index-name-as-per-log-file-name/281677 "2021-08-17T11:52:04Z")
Hi, I have use beat to get log files and location specified .log files are there. I am sending this log through logstash to elastic search. in conf file what to do to have index name as file name --- ## [CITRIX NETSCALER](https://discuss.elastic.co/t/citrix-netscaler/281663)
**Author:** [@Hoffmann\_Patrick](https://discuss.elastic.co/u/Hoffmann_Patrick)\ **Replies:** 0\ **Last updated:** [August 17, 2021, 10:14am UTC](https://discuss.elastic.co/t/citrix-netscaler/281663 "2021-08-17T10:14:08Z")
Bonjour, J'ai un soucis pour parser les logs (file depuis un syslog server) d'un netscaler Citrix (Citrix VDA et loadbalancer https/smtp via logstash et grok, personne n'aurait une petite idée ou piste. J'ai … --- ## [Attempted to resurrect connection to dead ES instance, but got an error](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance-but-got-an-error/281657)
**Author:** [@naaviin](https://discuss.elastic.co/u/naaviin)\ **Replies:** 0\ **Last updated:** [August 17, 2021, 9:45am UTC](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance-but-got-an-error/281657 "2021-08-17T09:45:30Z")
Hi friends, i configured accordingly. However i am getting error logstash.yml http.host: '0.0.0.0' xpack.monitoring.elasticsearch.hosts: \["https://es1:9200","https://es2:9200","https://es3:9200"\] xpack.monitoring.elast… --- ## [How to extract a substring](https://discuss.elastic.co/t/how-to-extract-a-substring/281578)
**Author:** [@dvrla](https://discuss.elastic.co/u/dvrla)\ **Replies:** 2\ **Last updated:** [August 17, 2021, 7:10am UTC](https://discuss.elastic.co/t/how-to-extract-a-substring/281578 "2021-08-17T07:10:56Z")
Hi, I would like to extract the value of a cookie from an HTTP request but up until now I can´t make it work. I´m trying to get from a request field like this GET /trading/css/main.css?version=4.4.0 HTTP/1.1, Host: ex… --- ## [NO such file or directory](https://discuss.elastic.co/t/no-such-file-or-directory/281632)
**Author:** [@naaviin](https://discuss.elastic.co/u/naaviin)\ **Replies:** 0\ **Last updated:** [August 17, 2021, 6:12am UTC](https://discuss.elastic.co/t/no-such-file-or-directory/281632 "2021-08-17T06:12:03Z")
Can i know why i am hitting this. i am using docker. what i did wrong? /usr/share/logstash/elastic-certificates.p12 (No such file or directory)" --- ## [Condition Section after Grok section not working](https://discuss.elastic.co/t/condition-section-after-grok-section-not-working/281600)
**Author:** [@Athul\_Devkar](https://discuss.elastic.co/u/Athul_Devkar)\ **Replies:** 4\ **Last updated:** [August 16, 2021, 9:28pm UTC](https://discuss.elastic.co/t/condition-section-after-grok-section-not-working/281600 "2021-08-16T21:28:48Z")
Hello Everyone! I am trying to modify my pipeline to allow me to create few additional fields based on certain conditions. However, that section alone is not working and I don't see any errors in my log. I had used simi… --- ## [Update a field if it does not contain any value](https://discuss.elastic.co/t/update-a-field-if-it-does-not-contain-any-value/281565)
**Author:** [@Derya\_Sayar](https://discuss.elastic.co/u/Derya_Sayar)\ **Replies:** 5\ **Last updated:** [August 16, 2021, 7:34pm UTC](https://discuss.elastic.co/t/update-a-field-if-it-does-not-contain-any-value/281565 "2021-08-16T19:34:02Z")
I have log files generated from some applications. Some of applications produces messages in following format. So there is "logContent" in message field. {"instant":{"epochSecond":1628692763,"nanoOfSecond":792000000},"t… --- ## [Logstash ip protocol enriching](https://discuss.elastic.co/t/logstash-ip-protocol-enriching/281496)
**Author:** [@ThreatInter](https://discuss.elastic.co/u/ThreatInter)\ **Replies:** 2\ **Last updated:** [August 16, 2021, 3:07pm UTC](https://discuss.elastic.co/t/logstash-ip-protocol-enriching/281496 "2021-08-16T15:07:42Z")
Hello community. I was busy with the task of collecting netflow using logstash. So I noticed that the netflow data after the "netflow" codec contains the "protocol" field, which is actually the protocol number. But there… --- ## [Simple Central log monitoring with ELK](https://discuss.elastic.co/t/simple-central-log-monitoring-with-elk/280488)
**Author:** [@Prabhath\_samarasingh](https://discuss.elastic.co/u/Prabhath_samarasingh)\ **Replies:** 20\ **Last updated:** [August 16, 2021, 2:32pm UTC](https://discuss.elastic.co/t/simple-central-log-monitoring-with-elk/280488 "2021-08-16T14:32:49Z")
Need to ship my servers logs (Linux system logs " /var/logs/\*" and windows server logs) to logstash to ELK and display in KIbana dash board. Please guide to perform this task. --- ## [Logstash extract additional fields from message field of Json](https://discuss.elastic.co/t/logstash-extract-additional-fields-from-message-field-of-json/281373)
**Author:** [@Derya\_Sayar](https://discuss.elastic.co/u/Derya_Sayar)\ **Replies:** 2\ **Last updated:** [August 16, 2021, 2:18pm UTC](https://discuss.elastic.co/t/logstash-extract-additional-fields-from-message-field-of-json/281373 "2021-08-16T14:18:15Z")
I have logs files with json format.Here you can find single line of log. {"instant":{"epochSecond":1628692763,"nanoOfSecond":792000000},"thread":"AWT-EventQueue-0","level":"INFO","loggerName":"com.client.logon… --- ## [Parse Logstash message field into multiple field](https://discuss.elastic.co/t/parse-logstash-message-field-into-multiple-field/281425)
**Author:** [@buulq90](https://discuss.elastic.co/u/buulq90)\ **Replies:** 8\ **Last updated:** [August 16, 2021, 2:13pm UTC](https://discuss.elastic.co/t/parse-logstash-message-field-into-multiple-field/281425 "2021-08-16T14:13:24Z")
Hello all, I'm firstly using log stash and got some issue when parsing the JSON log on the message field into multiple field for visualizing in the Kibana dashboard. Below is my message log appear in Elasticsearch me… --- ## [IPFIX processing Setup plan & queries](https://discuss.elastic.co/t/ipfix-processing-setup-plan-queries/281558)
**Author:** [@Saravana\_Maadavan](https://discuss.elastic.co/u/Saravana_Maadavan)\ **Replies:** 0\ **Last updated:** [August 16, 2021, 2:02pm UTC](https://discuss.elastic.co/t/ipfix-processing-setup-plan-queries/281558 "2021-08-16T14:02:11Z")
Hi Everyone, I have few questions reg. processing of IPFIX logs originating from "Thunder CFW | A10 network" exporter. Would not be able to change different network exporter product or any installation in source side si… --- ## [Exclude Unicode symbol in Grok pattern](https://discuss.elastic.co/t/exclude-unicode-symbol-in-grok-pattern/280540)
**Author:** [@Avesalon\_Novinsky](https://discuss.elastic.co/u/Avesalon_Novinsky)\ **Replies:** 1\ **Last updated:** [August 16, 2021, 11:59am UTC](https://discuss.elastic.co/t/exclude-unicode-symbol-in-grok-pattern/280540 "2021-08-16T11:59:55Z")
Hi, Is there any option to exclude Unicode symbol from line directly in grok pattern. I'm trying to read json data line by line thought AWS Glue "getSourceWithFormat" method which use grok pattern for string parsing. l… --- ## [Logstash :Config file to parse nested json](https://discuss.elastic.co/t/logstash-config-file-to-parse-nested-json/281528)
**Author:** [@ippo](https://discuss.elastic.co/u/ippo)\ **Replies:** 1\ **Last updated:** [August 16, 2021, 11:26am UTC](https://discuss.elastic.co/t/logstash-config-file-to-parse-nested-json/281528 "2021-08-16T11:26:13Z")
hello i 'm new to elk and i seeking help for my mission. i want to export the data from the log of json file below so that i have visualizations related to the number of successful builds . im trying to remove the fields… --- ## [Trigger an alert when Logstash pipeline gets less events than defined in threshold](https://discuss.elastic.co/t/trigger-an-alert-when-logstash-pipeline-gets-less-events-than-defined-in-threshold/281519)
**Author:** [@nemanja](https://discuss.elastic.co/u/nemanja)\ **Replies:** 0\ **Last updated:** [August 16, 2021, 8:49am UTC](https://discuss.elastic.co/t/trigger-an-alert-when-logstash-pipeline-gets-less-events-than-defined-in-threshold/281519 "2021-08-16T08:49:43Z")
Hello, Can we setup Logstash pipeline monitoring, that would for example trigger an alert when the number of events per second drops bellow certain threshold? in Kibana Pipeline Viewer there is nice visual overview on c… --- ## [Nested JSON flattened in Logstash Filter](https://discuss.elastic.co/t/nested-json-flattened-in-logstash-filter/281502)
**Author:** [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\ **Replies:** 1\ **Last updated:** [August 16, 2021, 8:11am UTC](https://discuss.elastic.co/t/nested-json-flattened-in-logstash-filter/281502 "2021-08-16T08:11:47Z")
Hi, I am able to flatten at fields present in nested JSON excluding 3 fields that are present in double nested JSON. I need to flatten these fields so that I can create viz based on that such as Fields-"quantity", "rate… --- ## [How configuration logstash 7.6.2 email in output](https://discuss.elastic.co/t/how-configuration-logstash-7-6-2-email-in-output/281148)
**Author:** [@arisxf](https://discuss.elastic.co/u/arisxf)\ **Replies:** 5\ **Last updated:** [August 16, 2021, 6:47am UTC](https://discuss.elastic.co/t/how-configuration-logstash-7-6-2-email-in-output/281148 "2021-08-16T06:47:28Z")
how to use logstash 7.6.2 email in output ? is this email plugin available by default? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=200) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=202)