# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=792 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 793 --- ## [Monitoring the microsoft azure services](https://discuss.elastic.co/t/monitoring-the-microsoft-azure-services/125796)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 6:01pm UTC](https://discuss.elastic.co/t/monitoring-the-microsoft-azure-services/125796 "2018-03-27T18:01:18Z")
Similar to AWS cloud watch input filter, do we have the input plugin for Azure as well? --- ## [Installing Logstash on Windows using Ansible](https://discuss.elastic.co/t/installing-logstash-on-windows-using-ansible/125792)
**Author:** [@elasticprof](https://discuss.elastic.co/u/elasticprof)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 5:19pm UTC](https://discuss.elastic.co/t/installing-logstash-on-windows-using-ansible/125792 "2018-03-27T17:19:34Z")
Hi, Can anyone please provide Ansible Task to install logstash on Windows server, i have seen examples to set it up linux flavors like Debian, CentOS but not for windows server. --- ## [Some documents missing in Elasticsearch](https://discuss.elastic.co/t/some-documents-missing-in-elasticsearch/125787)
**Author:** [@ben.zeghers](https://discuss.elastic.co/u/ben.zeghers)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 4:26pm UTC](https://discuss.elastic.co/t/some-documents-missing-in-elasticsearch/125787 "2018-03-27T16:26:52Z")
I'm using Logstash (hosted in AWS Fargate) to import json files from my S3 bucket to Elasticsearch (hosted by Elastic Cloud). It works perfectly file for most of the files but some (~1%) never show up when I query for th… --- ## [Logstash ruby json parser](https://discuss.elastic.co/t/logstash-ruby-json-parser/125770)
**Author:** [@darshan](https://discuss.elastic.co/u/darshan)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 2:02pm UTC](https://discuss.elastic.co/t/logstash-ruby-json-parser/125770 "2018-03-27T14:02:38Z")
Hi i am trying to parse a json string in logstash ruby filter. My logstash ruby code is. ruby { code =\> "jsondata = event.get('\[json\]') puts jsondata jsondata.to\_hash jsondata.each { |key, value| puts key puts va… --- ## [Json filter is not working](https://discuss.elastic.co/t/json-filter-is-not-working/124434)
**Author:** [@David\_Gidony](https://discuss.elastic.co/u/David_Gidony)\ **Replies:** 6\ **Last updated:** [March 27, 2018, 1:35pm UTC](https://discuss.elastic.co/t/json-filter-is-not-working/124434 "2018-03-27T13:35:52Z")
hi, i'm trying to parse json file as simple as possible. field:value without touching the json order. when i run log stash, it starts , but nothing happened... this is the log stash console while running, just sits th… --- ## [How to generate only events in output](https://discuss.elastic.co/t/how-to-generate-only-events-in-output/125732)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [March 27, 2018, 1:23pm UTC](https://discuss.elastic.co/t/how-to-generate-only-events-in-output/125732 "2018-03-27T13:23:34Z")
I am using HTTP as input and elastic search as output. My requirement is to generate only events in output and we don't need to get data in output. How can I achieve this? Below is my conf file input { http\_poller { … --- ## [Help with Logstash conf](https://discuss.elastic.co/t/help-with-logstash-conf/125746)
**Author:** [@Faenor2105](https://discuss.elastic.co/u/Faenor2105)\ **Replies:** 2\ **Last updated:** [March 27, 2018, 1:10pm UTC](https://discuss.elastic.co/t/help-with-logstash-conf/125746 "2018-03-27T13:10:59Z")
Hi there. I'm new in ELK theme. Can you help me to make right logstash config file? Right now i have many errors like this: \[2018-03-27T11:27:37,900\]\[ERROR\]\[logstash.filters.grok \] Unknown setting 'pattern' for grok… --- ## [Can convert "10%" string to float?](https://discuss.elastic.co/t/can-convert-10-string-to-float/125574)
**Author:** [@EditAxha](https://discuss.elastic.co/u/EditAxha)\ **Replies:** 11\ **Last updated:** [March 27, 2018, 12:41pm UTC](https://discuss.elastic.co/t/can-convert-10-string-to-float/125574 "2018-03-27T12:41:40Z")
Hello, It is possible to convert with filter logstash a string value like "10%" to float? thnx. --- ## [Installing logstash plugin fails](https://discuss.elastic.co/t/installing-logstash-plugin-fails/125745)
**Author:** [@famat001](https://discuss.elastic.co/u/famat001)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 12:05pm UTC](https://discuss.elastic.co/t/installing-logstash-plugin-fails/125745 "2018-03-27T12:05:00Z")
Hi everyone, I lately running into a Problem and I hope that I can get some help. Some Background about the system we are using. We are running a server behind a firewall. The server is running Redhat 7.4 with the ela… --- ## [\[main\]\>worker0 disappears with update to logstash 6](https://discuss.elastic.co/t/main-worker0-disappears-with-update-to-logstash-6/125737)
**Author:** [@Antonio\_Oliva](https://discuss.elastic.co/u/Antonio_Oliva)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 11:15am UTC](https://discuss.elastic.co/t/main-worker0-disappears-with-update-to-logstash-6/125737 "2018-03-27T11:15:15Z")
In logstash 5.6.6 with command "ps -T -u logstash" or "top -H -u logstash" we can see de worker0 thread: PID SPID TTY TIME CMD 30185 30235 ? 1-22:02:44 \[main\]\>worker0 ... But after update to logstash… --- ## [How to read emails from Exchange or Outlook to ES or logstash](https://discuss.elastic.co/t/how-to-read-emails-from-exchange-or-outlook-to-es-or-logstash/123945)
**Author:** [@Srikanth\_FT](https://discuss.elastic.co/u/Srikanth_FT)\ **Replies:** 2\ **Last updated:** [March 27, 2018, 10:50am UTC](https://discuss.elastic.co/t/how-to-read-emails-from-exchange-or-outlook-to-es-or-logstash/123945 "2018-03-27T10:50:54Z")
Hi Team, Can someone help me on how can i read Outlook/exchange emails and send them to ES (directly or via logstasth) Thank you --- ## [Random SIGTERM stopping Logstash](https://discuss.elastic.co/t/random-sigterm-stopping-logstash/125731)
**Author:** [@999chris](https://discuss.elastic.co/u/999chris)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 10:44am UTC](https://discuss.elastic.co/t/random-sigterm-stopping-logstash/125731 "2018-03-27T10:44:21Z")
Hi, Twice (at least) in 12hrs our departments logstash is recieving a seemingly random SIGTERM: \[2018-03-27T10:54:21,680\]\[INFO \]\[logstash.outputs.file \] Closing file /apps/isilon/mamsupport/filebeat-logs/avci\_valida… --- ## [Query group by postifx](https://discuss.elastic.co/t/query-group-by-postifx/125653)
**Author:** [@zeliko79](https://discuss.elastic.co/u/zeliko79)\ **Replies:** 1\ **Last updated:** [March 27, 2018, 10:44am UTC](https://discuss.elastic.co/t/query-group-by-postifx/125653 "2018-03-27T10:44:15Z")
Hello everyone, I was able to configure ELK cluster and grok for postfix, now I have a problem in finding information. For every message sent I have 2 records similar to these postfix\_queueid, postfix\_from postfix\_qu… --- ## [Make batch size configurable in elastic search output plugin](https://discuss.elastic.co/t/make-batch-size-configurable-in-elastic-search-output-plugin/125724)
**Author:** [@Patrik\_Iselind](https://discuss.elastic.co/u/Patrik_Iselind)\ **Replies:** 8\ **Last updated:** [March 27, 2018, 10:00am UTC](https://discuss.elastic.co/t/make-batch-size-configurable-in-elastic-search-output-plugin/125724 "2018-03-27T10:00:42Z")
Is there a way to configure the batch size in the elastic search output plugin? I have an Elastic Search server at AWS on nodes with a request max size set to 10MB. If the output plugin sends more than 10MB in a request… --- ## [Logstash.yml with keystore not working](https://discuss.elastic.co/t/logstash-yml-with-keystore-not-working/125726)
**Author:** [@Izek](https://discuss.elastic.co/u/Izek)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 9:56am UTC](https://discuss.elastic.co/t/logstash-yml-with-keystore-not-working/125726 "2018-03-27T09:56:25Z")
When I try to setup logstash x-pack with keystore , but I cannot make it I have already generate logstash.keystore file and add few records. but still not working error as below: logstash: ERROR: Failed to load sett… --- ## [Why does translate not work here?](https://discuss.elastic.co/t/why-does-translate-not-work-here/123081)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 3\ **Last updated:** [March 27, 2018, 9:47am UTC](https://discuss.elastic.co/t/why-does-translate-not-work-here/123081 "2018-03-27T09:47:36Z")
We would like to prioritize log messages of type WARN if they have a certain category. So we receive {"type": "WARN", "category": "DefaultEmailService", "priority": 2} and would like to output: {"type": "WARN", "catego… --- ## [Each: active: file expired for #{path} // :timeout for #{path}, closing file](https://discuss.elastic.co/t/each-active-file-expired-for-path-timeout-for-path-closing-file/125716)
**Author:** [@cedef](https://discuss.elastic.co/u/cedef)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 9:01am UTC](https://discuss.elastic.co/t/each-active-file-expired-for-path-timeout-for-path-closing-file/125716 "2018-03-27T09:01:00Z")
Behaviour from user point of view: I'm currently facing the following issue: logstash sometimes "miss" log files. When restarting it it parses the file correctly and data are filtered then passed to output plugin. Whe… --- ## [Logstash 6.2.1 - windows memory issues](https://discuss.elastic.co/t/logstash-6-2-1-windows-memory-issues/125593)
**Author:** [@deepaks](https://discuss.elastic.co/u/deepaks)\ **Replies:** 1\ **Last updated:** [March 27, 2018, 8:44am UTC](https://discuss.elastic.co/t/logstash-6-2-1-windows-memory-issues/125593 "2018-03-27T08:44:36Z")
Hello I am having memory allocation and saturation issues with logstash. My logstash configuration has memory allocation of 20GB set in the jvm options(-Xms20g -Xmx20g) but after starting the service the total memory al… --- ## [KV plugin leaves starting bracket in key](https://discuss.elastic.co/t/kv-plugin-leaves-starting-bracket-in-key/125698)
**Author:** [@reilee](https://discuss.elastic.co/u/reilee)\ **Replies:** 3\ **Last updated:** [March 27, 2018, 8:14am UTC](https://discuss.elastic.co/t/kv-plugin-leaves-starting-bracket-in-key/125698 "2018-03-27T08:14:33Z")
Just found a difference after upgrade logstash to v6.2.3. sample data: {{key1=value1}{key2=value2}} config: kv { field\_split =\> "\\{\\}" } expected output: key1 =\> value1 key2 =\> value2 But after upgrade, it outputs … --- ## [Shutdown stalled when running LS from bash script](https://discuss.elastic.co/t/shutdown-stalled-when-running-ls-from-bash-script/125125)
**Author:** [@kevkos236](https://discuss.elastic.co/u/kevkos236)\ **Replies:** 1\ **Last updated:** [March 27, 2018, 8:14am UTC](https://discuss.elastic.co/t/shutdown-stalled-when-running-ls-from-bash-script/125125 "2018-03-27T08:14:00Z")
Greetings, I wrote a bash script that finds CSV files in specified folders and pipes them into logstash. However when running this script I run into the following error, saying that the shutdown process is stalled, cau… --- ## [Aggregate filter (Combine Logs)](https://discuss.elastic.co/t/aggregate-filter-combine-logs/124919)
**Author:** [@Yimjunhyeok](https://discuss.elastic.co/u/Yimjunhyeok)\ **Replies:** 1\ **Last updated:** [March 27, 2018, 8:13am UTC](https://discuss.elastic.co/t/aggregate-filter-combine-logs/124919 "2018-03-27T08:13:51Z")
Hello.. I have a trouble making 1 log with 2 events. 1521537121385|172.16.130.205|First 1521537121385|172.16.130.205|Second 1521537121386|172.16.130.205|Third (LastSessionTime)(SrcIP)(ExtractedText) Then i want to … --- ## [Http\_poller only outputs hash?](https://discuss.elastic.co/t/http-poller-only-outputs-hash/125426)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 5\ **Last updated:** [March 27, 2018, 6:39am UTC](https://discuss.elastic.co/t/http-poller-only-outputs-hash/125426 "2018-03-27T06:39:16Z")
Below is my pipeline. Trying to pull a list from a site and then parse it out with xml filter. input { http\_poller { urls =\> { isctop100 =\> "https://isc.sans.edu/api/topips/records/100?xml" } schedu… --- ## [How to Mask or Hash a certain details on my SMTP Logs Message](https://discuss.elastic.co/t/how-to-mask-or-hash-a-certain-details-on-my-smtp-logs-message/125687)
**Author:** [@khbaltazar](https://discuss.elastic.co/u/khbaltazar)\ **Replies:** 3\ **Last updated:** [March 27, 2018, 4:46am UTC](https://discuss.elastic.co/t/how-to-mask-or-hash-a-certain-details-on-my-smtp-logs-message/125687 "2018-03-27T04:46:40Z")
\_Hi Guys, Im sending a smtp logs to our MSSP using logstash but we are required to mask some details on our logs like sender and recipient email address and subject before forwarding it to our MSSP due to some priva… --- ## [logstash fail to start in java 9 because tools.jar](https://discuss.elastic.co/t/logstash-fail-to-start-in-java-9-because-tools-jar/125638)
**Author:** [@shyang.oblab](https://discuss.elastic.co/u/shyang.oblab)\ **Replies:** 2\ **Last updated:** [March 27, 2018, 4:16am UTC](https://discuss.elastic.co/t/logstash-fail-to-start-in-java-9-because-tools-jar/125638 "2018-03-27T04:16:26Z")
environment: windows 10 home, 64bit jdk 9.04 installed logstash-6.2.3.zip -\> unzip logstash try to find "lib\\tools.jar but it removed in java 9 and 10. S:\\elastic\\logstash-6.2.3\>java -version java version "9.0.4" J… --- ## [Parse XML file question](https://discuss.elastic.co/t/parse-xml-file-question/125527)
**Author:** [@terrymu](https://discuss.elastic.co/u/terrymu)\ **Replies:** 8\ **Last updated:** [March 27, 2018, 3:23am UTC](https://discuss.elastic.co/t/parse-xml-file-question/125527 "2018-03-27T03:23:48Z")
Hi experts, I have parsed a XML file by using Xpath, but some field can't be detect, would you please help me check it? Thank you. The input file: \ \ \"Expected one of #, {, ,, \]](https://discuss.elastic.co/t/grok-configurationerror-message-expected-one-of/125624)
**Author:** [@MrFrost](https://discuss.elastic.co/u/MrFrost)\ **Replies:** 0\ **Last updated:** [March 26, 2018, 2:06pm UTC](https://discuss.elastic.co/t/grok-configurationerror-message-expected-one-of/125624 "2018-03-26T14:06:04Z")
Hello, I tried to figure out why I get below error but I do not see what could be wrong in my logstash config file. Hopefully someone can help. I did check message patter in "Grok Debugger" and it works there. Any ideas… --- ## [Logstash ignoring a particular pipeline defined in pipelines.yml :No configuration found in the configured sources](https://discuss.elastic.co/t/logstash-ignoring-a-particular-pipeline-defined-in-pipelines-yml-no-configuration-found-in-the-configured-sources/125272)
**Author:** [@Ongun](https://discuss.elastic.co/u/Ongun)\ **Replies:** 4\ **Last updated:** [March 27, 2018, 2:16am UTC](https://discuss.elastic.co/t/logstash-ignoring-a-particular-pipeline-defined-in-pipelines-yml-no-configuration-found-in-the-configured-sources/125272 "2018-03-27T02:16:09Z")
I am a new user of ELK Stack and my main purpose is visualizing data from SQL tables. The computer I use ELK on is running 64-bit Windows 10 Enterprise Edition. The installation consists of Elastic Search v 6.2.2, Logsta… --- ## ["\_geoip\_lookup\_failure" own src/dst ip mapping don't work](https://discuss.elastic.co/t/geoip-lookup-failure-own-src-dst-ip-mapping-dont-work/125496)
**Author:** [@jiona](https://discuss.elastic.co/u/jiona)\ **Replies:** 5\ **Last updated:** [March 26, 2018, 9:41pm UTC](https://discuss.elastic.co/t/geoip-lookup-failure-own-src-dst-ip-mapping-dont-work/125496 "2018-03-26T21:41:03Z")
Hi all. Can someone help me to findout why I get a "\_geoip\_lookup\_failure". I have separate elasticsearch and logstash servers. On elasticsearch I have install the plugin "ingest-geoip" $ sudo bin/elasticsearch-plugi… --- ## [Best way to debug DeadLetterQueue crashing](https://discuss.elastic.co/t/best-way-to-debug-deadletterqueue-crashing/125635)
**Author:** [@smbullet](https://discuss.elastic.co/u/smbullet)\ **Replies:** 0\ **Last updated:** [March 26, 2018, 3:38pm UTC](https://discuss.elastic.co/t/best-way-to-debug-deadletterqueue-crashing/125635 "2018-03-26T15:38:00Z")
What's the best way to debug something like this? \[2018-03-26T11:30:57,698\]\[ERROR\]\[logstash.pipeline \] A plugin had an unrecoverable error. Will restart this plugin. Plugin: \ **Author:** [@aysala](https://discuss.elastic.co/u/aysala)\ **Replies:** 3\ **Last updated:** [March 26, 2018, 6:49pm UTC](https://discuss.elastic.co/t/configuring-logstash-to-filter-docker-logs/124633 "2018-03-26T18:49:57Z") Hi, I have an ELK stack for logging a DCOS cluster. I am trying to parse the container logs from one of the nodes and I would like to ship only the required container logs form the node. Here, I am trying to figure out … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=791) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=793)