# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=278 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 279 --- ## [Logstash Http poller plugin](https://discuss.elastic.co/t/logstash-http-poller-plugin/256154)
**Author:** [@jerin](https://discuss.elastic.co/u/jerin)\ **Replies:** 7\ **Last updated:** [November 25, 2020, 6:42am UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin/256154 "2020-11-25T06:42:47Z")
Hello , i am using logstash poller plugin to query JIRA API . it works okay .. i am trying to get the ticket status updated in last 5 mins .. the problem is when there is no update , it polls the API and retrives the fi… --- ## [Logstash Beats Handler Error](https://discuss.elastic.co/t/logstash-beats-handler-error/256576)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 0\ **Last updated:** [November 24, 2020, 9:20pm UTC](https://discuss.elastic.co/t/logstash-beats-handler-error/256576 "2020-11-24T21:20:09Z")
Not really sure what this means, the pipeline appears to come back up and processing...but it unnerves me seeing this get vomitted out into the logs. Can anybody shed light on what this is? --- ## [Multiline codec in filter](https://discuss.elastic.co/t/multiline-codec-in-filter/256520)
**Author:** [@shani](https://discuss.elastic.co/u/shani)\ **Replies:** 9\ **Last updated:** [November 24, 2020, 7:47pm UTC](https://discuss.elastic.co/t/multiline-codec-in-filter/256520 "2020-11-24T19:47:41Z")
Can I use a multiline codec in the filter? I've more than 10k+ files of 3 different patterns logs pattern on S3 bucket that will be fetched using input plugin. One log pattern requires a multiline codec before start pa… --- ## [Finding deprecated functions](https://discuss.elastic.co/t/finding-deprecated-functions/256539)
**Author:** [@Tom99](https://discuss.elastic.co/u/Tom99)\ **Replies:** 1\ **Last updated:** [November 24, 2020, 4:50pm UTC](https://discuss.elastic.co/t/finding-deprecated-functions/256539 "2020-11-24T16:50:02Z")
We use ELK 7.10. Start version was 7.7. A version prior to 7.7 was not installed/used. Logfiles will be parsed by filebeat and sent over logstash to elasticsearch. It seems that a specific call in logstash causes a mes… --- ## [Date filter return \_dateparsefailure](https://discuss.elastic.co/t/date-filter-return-dateparsefailure/256341)
**Author:** [@tong1125](https://discuss.elastic.co/u/tong1125)\ **Replies:** 4\ **Last updated:** [November 24, 2020, 3:28pm UTC](https://discuss.elastic.co/t/date-filter-return-dateparsefailure/256341 "2020-11-24T15:28:40Z")
I'm new to ELK and tried to setup my first pipeline. The input and output works fine. But when I tried to play with date filter, the "@timestamp" field is not replaced with the time in the log file. Seems there is no si… --- ## [Failed to parse field \[input\] of type \[text\]](https://discuss.elastic.co/t/failed-to-parse-field-input-of-type-text/256486)
**Author:** [@Andex](https://discuss.elastic.co/u/Andex)\ **Replies:** 1\ **Last updated:** [November 24, 2020, 2:57pm UTC](https://discuss.elastic.co/t/failed-to-parse-field-input-of-type-text/256486 "2020-11-24T14:57:28Z")
Hi, i have this problem on logstash. On index template i set "input" like "text" and when log arrives from filebeat, my logstash.log said : "failed to parse field \[input\] of type \[text\] in document with id ..." --- ## [Encrypt/protect logstash lookup files?](https://discuss.elastic.co/t/encrypt-protect-logstash-lookup-files/256515)
**Author:** [@aerodynamic](https://discuss.elastic.co/u/aerodynamic)\ **Replies:** 2\ **Last updated:** [November 24, 2020, 2:42pm UTC](https://discuss.elastic.co/t/encrypt-protect-logstash-lookup-files/256515 "2020-11-24T14:42:49Z")
Hello, For my companies product we are in the need to encrypt/protect the lookup files used in our pipeline. --- ## [Getting error in logstash output email plugin?](https://discuss.elastic.co/t/getting-error-in-logstash-output-email-plugin/256471)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 1\ **Last updated:** [November 24, 2020, 2:26pm UTC](https://discuss.elastic.co/t/getting-error-in-logstash-output-email-plugin/256471 "2020-11-24T14:26:40Z")
Error : \[2020-11-24T14:24:59,453\]\[ERROR\]\[logstash.outputs.email \]\[main\] Something happen while delivering an email {:exception=\>#\ **Author:** [@sainath](https://discuss.elastic.co/u/sainath)\ **Replies:** 3\ **Last updated:** [November 24, 2020, 11:22am UTC](https://discuss.elastic.co/t/cloudwatch-logs-input-plugin-not-streaming-all-log-groups/252300 "2020-11-24T11:22:18Z") Hi , I am using cloudwatch log input plugin to stream logs to ES using logstash. below is my configuration. input { cloudwatch\_logs { log\_group =\> \[ "/aws" \] log\_group\_prefix =\> true access\_k… --- ## [How to change logstash jdbc last run default timezone to local](https://discuss.elastic.co/t/how-to-change-logstash-jdbc-last-run-default-timezone-to-local/256490)
**Author:** [@sreedhar1](https://discuss.elastic.co/u/sreedhar1)\ **Replies:** 0\ **Last updated:** [November 24, 2020, 10:57am UTC](https://discuss.elastic.co/t/how-to-change-logstash-jdbc-last-run-default-timezone-to-local/256490 "2020-11-24T10:57:59Z")
I am using logstash to sync data between sql server and elasticsearch. how to change timezone from UTC to LOCAL. when i run logstash .conf jdbc lastrun 'logstash-7.10.0/.logstash\_jdbc\_last\_run' getting update with UTC… --- ## [Use a variable from one grok to another for all lines from a log file](https://discuss.elastic.co/t/use-a-variable-from-one-grok-to-another-for-all-lines-from-a-log-file/256470)
**Author:** [@Blazkowicz](https://discuss.elastic.co/u/Blazkowicz)\ **Replies:** 0\ **Last updated:** [November 24, 2020, 8:58am UTC](https://discuss.elastic.co/t/use-a-variable-from-one-grok-to-another-for-all-lines-from-a-log-file/256470 "2020-11-24T08:58:56Z")
Hi all, I have searched the forum for this type of question but what I have found I did not manage to make use of it. I am a newbie in all of ELK stack. Any help would be highly appreciated. I have the following log ty… --- ## [Gsub replace based on pattern](https://discuss.elastic.co/t/gsub-replace-based-on-pattern/256352)
**Author:** [@Muhammad\_Faisal](https://discuss.elastic.co/u/Muhammad_Faisal)\ **Replies:** 7\ **Last updated:** [November 24, 2020, 7:53am UTC](https://discuss.elastic.co/t/gsub-replace-based-on-pattern/256352 "2020-11-24T07:53:59Z")
i have log in below format , i need to replace ":" from 17:18:12 to 17-18-12 ...as this colon is also present at other places in line , how to match this mutate { gsub =\> \[ "message", "\\d\\d:\\d\\d:\\d\\d", "\\d\\d-\\d\\d-\\d\\d" … --- ## [Compare Logstash Old Event Against New One and Make Math Operation](https://discuss.elastic.co/t/compare-logstash-old-event-against-new-one-and-make-math-operation/256457)
**Author:** [@rknd](https://discuss.elastic.co/u/rknd)\ **Replies:** 0\ **Last updated:** [November 24, 2020, 7:48am UTC](https://discuss.elastic.co/t/compare-logstash-old-event-against-new-one-and-make-math-operation/256457 "2020-11-24T07:48:26Z")
Hello everyone, I want to compare the time object in each row with the next date object and get the difference between them and print them on the screen. I tried Aggregate plugin, Elapsed plugin, but could not reach the… --- ## [Logstash immediately shuts down upon launch in container due to YAML](https://discuss.elastic.co/t/logstash-immediately-shuts-down-upon-launch-in-container-due-to-yaml/256452)
**Author:** [@weiyentan](https://discuss.elastic.co/u/weiyentan)\ **Replies:** 1\ **Last updated:** [November 24, 2020, 7:10am UTC](https://discuss.elastic.co/t/logstash-immediately-shuts-down-upon-launch-in-container-due-to-yaml/256452 "2020-11-24T07:10:43Z")
Hi, I have a logstash.yml file that I am using the http api listener. I have it listening for some post request inputs that has some Json bod's. Now i am in the process of processing the json. I have some complex json … --- ## [Help with conversion](https://discuss.elastic.co/t/help-with-conversion/256403)
**Author:** [@Marcin\_Kr](https://discuss.elastic.co/u/Marcin_Kr)\ **Replies:** 2\ **Last updated:** [November 24, 2020, 7:09am UTC](https://discuss.elastic.co/t/help-with-conversion/256403 "2020-11-24T07:09:21Z")
Hi guys, I've created logstash file with csv filter like below. It's working but i need to export timestamp field as "date" into elastic. I decided to use convert function but it's not working. Can You help ? ''' inp… --- ## [Logatsh unable to read custom logstash.conf file](https://discuss.elastic.co/t/logatsh-unable-to-read-custom-logstash-conf-file/256450)
**Author:** [@devdatta\_mulgund](https://discuss.elastic.co/u/devdatta_mulgund)\ **Replies:** 0\ **Last updated:** [November 24, 2020, 6:24am UTC](https://discuss.elastic.co/t/logatsh-unable-to-read-custom-logstash-conf-file/256450 "2020-11-24T06:24:52Z")
Logatsh is unable to read custom logstash.conf file and it's pointing to the default file path. i.e /usr/share/logstash/pipeline/logstash.conf. docker-compose.yml version: '3.2' services: elasticsearch: build:… --- ## [Failed to create monitoring event {:message=\>"For path: http\_address. Map keys: \[:jvm, :os, :stats\]", :error=\>"LogStash::Instrument::MetricStore::MetricNotFoun](https://discuss.elastic.co/t/failed-to-create-monitoring-event-message-for-path-http-address-map-keys-jvm-os-stats-error-logstash-metricnotfoun/255939)
**Author:** [@devdatta\_mulgund](https://discuss.elastic.co/u/devdatta_mulgund)\ **Replies:** 7\ **Last updated:** [November 24, 2020, 6:05am UTC](https://discuss.elastic.co/t/failed-to-create-monitoring-event-message-for-path-http-address-map-keys-jvm-os-stats-error-logstash-metricnotfoun/255939 "2020-11-24T06:05:08Z")
I am facing the below error while starting the Logstash. Please note I am using docker-compose Below is the Logstash configuration. logstash.yml Default Logstash configuration from Logstash base image. http.host: "0… --- ## [java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/256380)
**Author:** [@Palash\_Dubey](https://discuss.elastic.co/u/Palash_Dubey)\ **Replies:** 2\ **Last updated:** [November 24, 2020, 5:57am UTC](https://discuss.elastic.co/t/java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/256380 "2020-11-24T05:57:16Z")
\[ERROR\]\[logstash.agent\] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"Java::JavaLang::IllegalStateException", :message=\>"Unable to configure plugins: (ConfigurationErr… --- ## [Split double nested json array - http poller logstash](https://discuss.elastic.co/t/split-double-nested-json-array-http-poller-logstash/256440)
**Author:** [@sachin10](https://discuss.elastic.co/u/sachin10)\ **Replies:** 0\ **Last updated:** [November 24, 2020, 5:19am UTC](https://discuss.elastic.co/t/split-double-nested-json-array-http-poller-logstash/256440 "2020-11-24T05:19:22Z")
I would like to to retrieve every element in below JSON to be a separate field so as to visualize in kibana by applying metrics in dashboard. I'm using http poller plugin to do this task and i used below filter to split… --- ## [Logstash s3 input plugin: not indexing files under individual folder of bucket using prefix option](https://discuss.elastic.co/t/logstash-s3-input-plugin-not-indexing-files-under-individual-folder-of-bucket-using-prefix-option/256313)
**Author:** [@Ravi\_Kumar\_Reddy](https://discuss.elastic.co/u/Ravi_Kumar_Reddy)\ **Replies:** 0\ **Last updated:** [November 23, 2020, 7:34am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-not-indexing-files-under-individual-folder-of-bucket-using-prefix-option/256313 "2020-11-23T07:34:11Z")
s3 bucket : elk folders: syslog, errorlog etc.. filepattern: syslog- input { s3 { access\_key\_id =\> "" secret\_access\_key =\> "" bucket =\> "elk" region =\> "eu-central-1" prefix =\> "syslog/syslog-" additional\_settin… --- ## [How to put two groks for IIS Logs?](https://discuss.elastic.co/t/how-to-put-two-groks-for-iis-logs/256111)
**Author:** [@Cristiane\_Marcarini](https://discuss.elastic.co/u/Cristiane_Marcarini)\ **Replies:** 4\ **Last updated:** [November 23, 2020, 4:38pm UTC](https://discuss.elastic.co/t/how-to-put-two-groks-for-iis-logs/256111 "2020-11-23T16:38:05Z")
\ true match =\> \["message", "%{TIMESTAMP\_ISO8601:log\_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:request}(?:%{URIPARAM:requestparam})? - %{NUMBER:port} - %{IPORHOST:clienthost} %{NOTSP… --- ## [Aggregation filter error](https://discuss.elastic.co/t/aggregation-filter-error/256372)
**Author:** [@Andex](https://discuss.elastic.co/u/Andex)\ **Replies:** 2\ **Last updated:** [November 23, 2020, 3:51pm UTC](https://discuss.elastic.co/t/aggregation-filter-error/256372 "2020-11-23T15:51:01Z")
Hi, i have this problem on aggregation filter : String can't be coerced into Integer , this is the log message : \[2020-11-23T15:55:13,479\]\[ERROR\]\[logstash.filters.aggregate\]\[main\]\[13ac83016d9f4ef50e52f1df4fd7eba1bf205… --- ## [Help with Logstash filter and private geoip data](https://discuss.elastic.co/t/help-with-logstash-filter-and-private-geoip-data/256007)
**Author:** [@ccofer](https://discuss.elastic.co/u/ccofer)\ **Replies:** 8\ **Last updated:** [November 23, 2020, 3:46pm UTC](https://discuss.elastic.co/t/help-with-logstash-filter-and-private-geoip-data/256007 "2020-11-23T15:46:17Z")
I realize I should have a better understanding of elastic, indexes, mappings, logstash and beats before coming here asking for help, but I really need to put this behind me. I have spent days searching and reading about … --- ## [Logstash failed to create template on target server](https://discuss.elastic.co/t/logstash-failed-to-create-template-on-target-server/256232)
**Author:** [@amralieg](https://discuss.elastic.co/u/amralieg)\ **Replies:** 3\ **Last updated:** [November 23, 2020, 2:17pm UTC](https://discuss.elastic.co/t/logstash-failed-to-create-template-on-target-server/256232 "2020-11-23T14:17:46Z")
Hi, I have the following logstash configuration, and when I run it it gives me the error. the input server is hosted on elastic cloud v7.9 and the output server is hosted on AWS elasticsearch v7.8, any clue how to solve… --- ## [Unable to parse field in Kibana](https://discuss.elastic.co/t/unable-to-parse-field-in-kibana/255851)
**Author:** [@tamilarasanbravo](https://discuss.elastic.co/u/tamilarasanbravo)\ **Replies:** 1\ **Last updated:** [November 23, 2020, 3:33pm UTC](https://discuss.elastic.co/t/unable-to-parse-field-in-kibana/255851 "2020-11-23T15:33:59Z")
Hi Team, I am gathering JSON formats to logstash and then parsing it to elasticsearch. In that process, there is a key which has array brackets to it. The key gets saved in Elasticsearch as unknown field9Already tried… --- ## [Logstash datetime filter if else](https://discuss.elastic.co/t/logstash-datetime-filter-if-else/254430)
**Author:** [@tamilarasanbravo](https://discuss.elastic.co/u/tamilarasanbravo)\ **Replies:** 7\ **Last updated:** [November 23, 2020, 3:22pm UTC](https://discuss.elastic.co/t/logstash-datetime-filter-if-else/254430 "2020-11-23T15:22:18Z")
Hi Team, This is regarding the Logstash Input filter. Sometimes the application from which I receive the data(Via logstash http) sends the value in one dateformat(ISO) and at times in another(UNIX). Is there anyway I … --- ## [CSV Ingest Column 1 = Time, Row 1 = Device](https://discuss.elastic.co/t/csv-ingest-column-1-time-row-1-device/256180)
**Author:** [@stevezemlicka](https://discuss.elastic.co/u/stevezemlicka)\ **Replies:** 4\ **Last updated:** [November 23, 2020, 3:06pm UTC](https://discuss.elastic.co/t/csv-ingest-column-1-time-row-1-device/256180 "2020-11-23T15:06:51Z")
I have a CSV to import that has been formatted for use in Excel. As a result, and necessary for it to open in Excel, it has not only been chopped into several files (which I can handle) but it has also been formatted su… --- ## [Logstash \[ERROR\]\[logstash.agent \] Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Could not execute action: PipelineAction::Create\, action\_result: false", :backtrace=\>nil}](https://discuss.elastic.co/t/logstash-error-logstash-agent-failed-to-execute-action-id-main-action-type-logstash-failedaction-message-could-not-execute-action-pipelineaction-create-main-action-result-false-backtrace-nil/256348)
**Author:** [@salma\_widiarti](https://discuss.elastic.co/u/salma_widiarti)\ **Replies:** 1\ **Last updated:** [November 23, 2020, 2:39pm UTC](https://discuss.elastic.co/t/logstash-error-logstash-agent-failed-to-execute-action-id-main-action-type-logstash-failedaction-message-could-not-execute-action-pipelineaction-create-main-action-result-false-backtrace-nil/256348 "2020-11-23T14:39:31Z")
Hi all, i am trying file configuration logstash like this input { http\_poller { urls =\> { users =\> { method =\> get user =\> "user" password =\> "pass… --- ## [Logstash not listening on port](https://discuss.elastic.co/t/logstash-not-listening-on-port/255356)
**Author:** [@droidus](https://discuss.elastic.co/u/droidus)\ **Replies:** 10\ **Last updated:** [November 23, 2020, 2:31pm UTC](https://discuss.elastic.co/t/logstash-not-listening-on-port/255356 "2020-11-23T14:31:01Z")
I noticed that nothing is listening on port 5044 on my ELK server. Logstash is running. Is there some configuration I missed somewhere to have it running/listening on that port? Here are the last few lines from my log … --- ## [How to specify the salesforce Host for Salesforce input plugin](https://discuss.elastic.co/t/how-to-specify-the-salesforce-host-for-salesforce-input-plugin/255119)
**Author:** [@brendanlynch](https://discuss.elastic.co/u/brendanlynch)\ **Replies:** 2\ **Last updated:** [November 23, 2020, 1:14pm UTC](https://discuss.elastic.co/t/how-to-specify-the-salesforce-host-for-salesforce-input-plugin/255119 "2020-11-23T13:14:55Z")
How do you control the Salesforce HOST/URL that you want to connect to? All I see in the input fields is a boolean switch that says it will use 'test.salesforce.com'. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=277) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=279)