# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=316 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 317 --- ## [Generic logstash config as onprem proxy for cloud](https://discuss.elastic.co/t/generic-logstash-config-as-onprem-proxy-for-cloud/244280)
**Author:** [@NightSpark](https://discuss.elastic.co/u/NightSpark)\ **Replies:** 1\ **Last updated:** [August 14, 2020, 12:15am UTC](https://discuss.elastic.co/t/generic-logstash-config-as-onprem-proxy-for-cloud/244280 "2020-08-14T00:15:37Z")
Hi, I am running 7.8.1 and hoping that someone can provide a logstash config that will allow me to have a central server that will take requests for inputs for on prem servers and send to a elastic search running on ela… --- ## [Dynamically change the index for Elasticsearch in Logstash output](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788)
**Author:** [@udesh](https://discuss.elastic.co/u/udesh)\ **Replies:** 5\ **Last updated:** [August 13, 2020, 11:47pm UTC](https://discuss.elastic.co/t/dynamically-change-the-index-for-elasticsearch-in-logstash-output/244788 "2020-08-13T23:47:31Z")
'Hi All, I'm trying to change the logstash.config file according to a log file. The log file consists three different types of log records which generated through my java application. logstash.config file is as follow… --- ## [Can't see logstash monitoring window in kibana](https://discuss.elastic.co/t/cant-see-logstash-monitoring-window-in-kibana/244858)
**Author:** [@Burga](https://discuss.elastic.co/u/Burga)\ **Replies:** 4\ **Last updated:** [August 13, 2020, 8:16pm UTC](https://discuss.elastic.co/t/cant-see-logstash-monitoring-window-in-kibana/244858 "2020-08-13T20:16:37Z")
Hi , I'm trying to configure logstash monitoring but can't see it in kibana , I see the index is created and have no related errors :slight\_smile: GET /\_cluster/settings { "persistent": { "xpack": { "monito… --- ## [Wrong logstash input filter invoked: csv filter is being invoked rather than http](https://discuss.elastic.co/t/wrong-logstash-input-filter-invoked-csv-filter-is-being-invoked-rather-than-http/244798)
**Author:** [@david6](https://discuss.elastic.co/u/david6)\ **Replies:** 2\ **Last updated:** [August 13, 2020, 3:11pm UTC](https://discuss.elastic.co/t/wrong-logstash-input-filter-invoked-csv-filter-is-being-invoked-rather-than-http/244798 "2020-08-13T15:11:13Z")
When I PUT a json body to the configured http input url, the CSV filter is being invoked instead of http. What am I missing? PUT: curl --header "Content-Type: application/json" \\ --request PUT \\ --data '{"field1": "ma… --- ## [How to change timestamp adding by logstash while sending logs to output plugin](https://discuss.elastic.co/t/how-to-change-timestamp-adding-by-logstash-while-sending-logs-to-output-plugin/244799)
**Author:** [@aksshm](https://discuss.elastic.co/u/aksshm)\ **Replies:** 5\ **Last updated:** [August 13, 2020, 3:04pm UTC](https://discuss.elastic.co/t/how-to-change-timestamp-adding-by-logstash-while-sending-logs-to-output-plugin/244799 "2020-08-13T15:04:16Z")
Hi, I'm running logstash with kafka as input plugin and syslog as output plugin. For kafka , all the messages are with local timezone/required timezone but while sending to syslog logstash is adding a timestamp in fron… --- ## [How to apply parse failure tag only when message fails BOTH filters instead of just one?](https://discuss.elastic.co/t/how-to-apply-parse-failure-tag-only-when-message-fails-both-filters-instead-of-just-one/244833)
**Author:** [@drjosephliu](https://discuss.elastic.co/u/drjosephliu)\ **Replies:** 1\ **Last updated:** [August 13, 2020, 2:28pm UTC](https://discuss.elastic.co/t/how-to-apply-parse-failure-tag-only-when-message-fails-both-filters-instead-of-just-one/244833 "2020-08-13T14:28:45Z")
My logtash config has two filters: 1) parses a log message and 2) parses a JSON message: filter { grok { match =\> { "message" =\> "\\\[%{TIMESTAMP\_ISO8601:log\_time}\\\]\\\[%{LOGLEVEL:log\_level}(?\\\s\*)\\\]\\\[%{DATA… --- ## [SSL handshake error with logstash 7.8.0 kafka integration](https://discuss.elastic.co/t/ssl-handshake-error-with-logstash-7-8-0-kafka-integration/244894)
**Author:** [@t1m\_e](https://discuss.elastic.co/u/t1m_e)\ **Replies:** 1\ **Last updated:** [August 13, 2020, 2:21pm UTC](https://discuss.elastic.co/t/ssl-handshake-error-with-logstash-7-8-0-kafka-integration/244894 "2020-08-13T14:21:28Z")
I'm having a hard time to make the kafka integration to work with logstash 7.8.0 and 7.8.1. My setup is relatively simple. I'm using self-signed certificates stored in a jks keystore. The logstash input is configured to… --- ## [Monitoring 7.8 with metricbeat drill down errors](https://discuss.elastic.co/t/monitoring-7-8-with-metricbeat-drill-down-errors/244878)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 3\ **Last updated:** [August 13, 2020, 1:46pm UTC](https://discuss.elastic.co/t/monitoring-7-8-with-metricbeat-drill-down-errors/244878 "2020-08-13T13:46:57Z")
Nodes and pipelines show up in monitoring but getting thie error trying to drill down on pipeline: {"statusCode":404,"error":"Not Found","message":"Not Found"} Pipeline is: \[beats\](https://35.245.148.101:5601/logstash/… --- ## [Logstash-input-s3 cannot back up with an S3 key that contains characters that need to be URL-encoded](https://discuss.elastic.co/t/logstash-input-s3-cannot-back-up-with-an-s3-key-that-contains-characters-that-need-to-be-url-encoded/244882)
**Author:** [@fukasawah](https://discuss.elastic.co/u/fukasawah)\ **Replies:** 0\ **Last updated:** [August 13, 2020, 12:52pm UTC](https://discuss.elastic.co/t/logstash-input-s3-cannot-back-up-with-an-s3-key-that-contains-characters-that-need-to-be-url-encoded/244882 "2020-08-13T12:52:43Z")
Hello. Regarding the title, Is there any way to fix it? I suspect it's a bug in the plugin. Version: logstash-input-s3-3.5.0 Operating System: Windows Server 2016 Config File: input { s3 { bucket =\> "fukasawah… --- ## [Logstash to Azure Log Analytics issue](https://discuss.elastic.co/t/logstash-to-azure-log-analytics-issue/244853)
**Author:** [@accts324](https://discuss.elastic.co/u/accts324)\ **Replies:** 0\ **Last updated:** [August 13, 2020, 10:22am UTC](https://discuss.elastic.co/t/logstash-to-azure-log-analytics-issue/244853 "2020-08-13T10:22:15Z")
We are using the module on our testing environment and it has been added to our logstash.conf. The problem we are facing now is with the elasticsearch module. The logs are now going properly to Log Analytics but does not… --- ## [APM Agent .NET Public API : how to configure apm server credential (username, password)](https://discuss.elastic.co/t/apm-agent-net-public-api-how-to-configure-apm-server-credential-username-password/244843)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 0\ **Last updated:** [August 13, 2020, 9:38am UTC](https://discuss.elastic.co/t/apm-agent-net-public-api-how-to-configure-apm-server-credential-username-password/244843 "2020-08-13T09:38:00Z")
I have running apm .net agent in my local and i want to send apm data to my apm cloud service. But the cloud service requires username and password. How do i configure the api to handle authentication? in my code, i use… --- ## [Date Filter Question](https://discuss.elastic.co/t/date-filter-question/244816)
**Author:** [@kvtang](https://discuss.elastic.co/u/kvtang)\ **Replies:** 2\ **Last updated:** [August 13, 2020, 8:08am UTC](https://discuss.elastic.co/t/date-filter-question/244816 "2020-08-13T08:08:26Z")
Hi all, Currently I have a use case where I have to do some calculation on the time, therefore I need to use date filter plugin in order to parse dates from field. But I dont want to use that date or timestamp as the l… --- ## [How to change Logtash internal/system logs to local timezone?](https://discuss.elastic.co/t/how-to-change-logtash-internal-system-logs-to-local-timezone/244687)
**Author:** [@aksshm](https://discuss.elastic.co/u/aksshm)\ **Replies:** 2\ **Last updated:** [August 13, 2020, 5:40am UTC](https://discuss.elastic.co/t/how-to-change-logtash-internal-system-logs-to-local-timezone/244687 "2020-08-13T05:40:55Z")
we are running logstash as a pod in k8s, we're trying to change the timezone of logstash internal/system logs. by default logs are coming in utc timezone ex: INFO \] 2020-08-12 07:26:19.757 \[Api Webserver\] agent - Succ… --- ## [Handling multiple client's filebeat on logstash](https://discuss.elastic.co/t/handling-multiple-clients-filebeat-on-logstash/244768)
**Author:** [@Jagadeesan\_A](https://discuss.elastic.co/u/Jagadeesan_A)\ **Replies:** 2\ **Last updated:** [August 12, 2020, 8:32pm UTC](https://discuss.elastic.co/t/handling-multiple-clients-filebeat-on-logstash/244768 "2020-08-12T20:32:17Z")
I want to run filebeats on "n" number of clients, on logstash how can i distinguish each client's filebeat. Is there any way to include client name in indexing. Am able to read geo name in each filebeat message. Is the… --- ## [Unable to set "ssl true", "ssl\_certificate\_verification false" in logstash http output plugin](https://discuss.elastic.co/t/unable-to-set-ssl-true-ssl-certificate-verification-false-in-logstash-http-output-plugin/244764)
**Author:** [@bimanarajesh](https://discuss.elastic.co/u/bimanarajesh)\ **Replies:** 1\ **Last updated:** [August 12, 2020, 5:54pm UTC](https://discuss.elastic.co/t/unable-to-set-ssl-true-ssl-certificate-verification-false-in-logstash-http-output-plugin/244764 "2020-08-12T17:54:03Z")
With http output plugin, I am not able to set ssl =\> true ssl\_certificate\_verification =\> false We are getting configuration error when we add the above configurations in http output Pls let us know how to enable ssl… --- ## [Example logstash.conf with multiline json to elasticsearch](https://discuss.elastic.co/t/example-logstash-conf-with-multiline-json-to-elasticsearch/244741)
**Author:** [@Annette1](https://discuss.elastic.co/u/Annette1)\ **Replies:** 1\ **Last updated:** [August 12, 2020, 4:03pm UTC](https://discuss.elastic.co/t/example-logstash-conf-with-multiline-json-to-elasticsearch/244741 "2020-08-12T16:03:24Z")
kibana error - failed to format messages from /var/lib/containers/....json.log filebeat config "paths": \[/var/lib/docker/cnotainers//.log"\] "fields": {"environement": "dev", "system" : "test" "level "application"}, … --- ## [Error in logstash-input-s3-sns-sqs plugin](https://discuss.elastic.co/t/error-in-logstash-input-s3-sns-sqs-plugin/244699)
**Author:** [@Jayakumar\_R\_R](https://discuss.elastic.co/u/Jayakumar_R_R)\ **Replies:** 4\ **Last updated:** [August 12, 2020, 3:53pm UTC](https://discuss.elastic.co/t/error-in-logstash-input-s3-sns-sqs-plugin/244699 "2020-08-12T15:53:36Z")
Hi all, I am new to logstash. I am using s3snssqs plugin to pull data from s3. input { s3snssqs { region =\> "us-east-1" s3\_default\_options =\> { "endpoint\_discovery" =\> … --- ## [Value mapping GROK\\PIPELINE](https://discuss.elastic.co/t/value-mapping-grok-pipeline/244739)
**Author:** [@dmalchikov](https://discuss.elastic.co/u/dmalchikov)\ **Replies:** 1\ **Last updated:** [August 12, 2020, 3:49pm UTC](https://discuss.elastic.co/t/value-mapping-grok-pipeline/244739 "2020-08-12T15:49:29Z")
Hi there, Could not find how to modify coded field from logs. For example in logs I have value 0 or 1 but I would like to see(store) in Kibana "OK" or "Alarm" I just use Zabbix and there "Value mapping" feature exists… --- ## [Windows Event Viewer Logs Parsing without Winlogbeat](https://discuss.elastic.co/t/windows-event-viewer-logs-parsing-without-winlogbeat/244710)
**Author:** [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\ **Replies:** 1\ **Last updated:** [August 12, 2020, 3:37pm UTC](https://discuss.elastic.co/t/windows-event-viewer-logs-parsing-without-winlogbeat/244710 "2020-08-12T15:37:53Z")
Constraint: logs are coming in collected by Splunk, forwarded, along with many other log sources, to logstash. So no winlogbeat, no logstash-input-eventlog plugin. We have to pick out the Windows events from the flood of… --- ## [How to combine Multiline events into Single Event](https://discuss.elastic.co/t/how-to-combine-multiline-events-into-single-event/244714)
**Author:** [@Pavani\_Reddy](https://discuss.elastic.co/u/Pavani_Reddy)\ **Replies:** 3\ **Last updated:** [August 12, 2020, 3:29pm UTC](https://discuss.elastic.co/t/how-to-combine-multiline-events-into-single-event/244714 "2020-08-12T15:29:44Z")
I am trying to index the sip logs (telephony) , but while indexing ,it took each line as separate event .Give me some suggestions to combine those events into one single event. Here I am giving log format \<135\>85748… --- ## [Add x-server header to kibana parsing](https://discuss.elastic.co/t/add-x-server-header-to-kibana-parsing/244346)
**Author:** [@sandikata](https://discuss.elastic.co/u/sandikata)\ **Replies:** 5\ **Last updated:** [August 12, 2020, 3:28pm UTC](https://discuss.elastic.co/t/add-x-server-header-to-kibana-parsing/244346 "2020-08-12T15:28:30Z")
Hello, i am trying to add http\_x\_server field/pattern to my installation, but without success i cannot even find any info about custom patterns or so. There's my current nginx pattern file. NGUSERNAME \[a-zA-Z\\.\\@\\-\\+\_… --- ## [Logstash sending same message 3 times to elasticsearch](https://discuss.elastic.co/t/logstash-sending-same-message-3-times-to-elasticsearch/244749)
**Author:** [@srbhklkrn](https://discuss.elastic.co/u/srbhklkrn)\ **Replies:** 0\ **Last updated:** [August 12, 2020, 3:26pm UTC](https://discuss.elastic.co/t/logstash-sending-same-message-3-times-to-elasticsearch/244749 "2020-08-12T15:26:31Z")
We have Pivotal Cloud Foundry as a platform and we are sending logs to ELK. So our setup is like we have an F5 load balancer that sends logs to 3 Logstash servers and these 3 Logstash servers send logs to 3 Elastic serve… --- ## [Writing appender.json\_console logs for logstash to file](https://discuss.elastic.co/t/writing-appender-json-console-logs-for-logstash-to-file/244746)
**Author:** [@vishnuanil](https://discuss.elastic.co/u/vishnuanil)\ **Replies:** 0\ **Last updated:** [August 12, 2020, 3:16pm UTC](https://discuss.elastic.co/t/writing-appender-json-console-logs-for-logstash-to-file/244746 "2020-08-12T15:16:52Z")
Currently all appender.json\_console logs are written as stdout and logs are written into a docker container json log for logstash which is using a lot of disk space. I am aware of the options to utilize docker options -l… --- ## [Logger.logstash.outputs.elasticsearch not printing enough logs](https://discuss.elastic.co/t/logger-logstash-outputs-elasticsearch-not-printing-enough-logs/243420)
**Author:** [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)\ **Replies:** 12\ **Last updated:** [August 12, 2020, 2:54pm UTC](https://discuss.elastic.co/t/logger-logstash-outputs-elasticsearch-not-printing-enough-logs/243420 "2020-08-12T14:54:50Z")
Hi I saw "logger.logstash.outputs.elasticsearch" : "DEBUG" is printing debug level logs only during logstash start/stop. But logstash not printing any logs when it parses an event and stores in elasticsearch My out… --- ## [Logstash aggregation filter](https://discuss.elastic.co/t/logstash-aggregation-filter/244730)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 6\ **Last updated:** [August 12, 2020, 2:54pm UTC](https://discuss.elastic.co/t/logstash-aggregation-filter/244730 "2020-08-12T14:54:12Z")
Hi all, I am trying to aggregate information of multi events to one event using aggregation filter of logstash. for example events are as following: name1=c1,name2=s1 name1=c1,name2=s1 name1=c1,name2=s1 name1=c1,name2=… --- ## ['sincedb\_path' for http\_poller not working](https://discuss.elastic.co/t/sincedb-path-for-http-poller-not-working/244734)
**Author:** [@AditiKhalatkar](https://discuss.elastic.co/u/AditiKhalatkar)\ **Replies:** 2\ **Last updated:** [August 12, 2020, 2:28pm UTC](https://discuss.elastic.co/t/sincedb-path-for-http-poller-not-working/244734 "2020-08-12T14:28:05Z")
I am trying to ingest data from an API using http\_poller plugin in logstash. Tried using some methods, however was it is throwing error like this. Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC wa… --- ## [How to calculate timestamp difference in logstash config file?](https://discuss.elastic.co/t/how-to-calculate-timestamp-difference-in-logstash-config-file/244732)
**Author:** [@Rahul\_Chougule](https://discuss.elastic.co/u/Rahul_Chougule)\ **Replies:** 1\ **Last updated:** [August 12, 2020, 2:25pm UTC](https://discuss.elastic.co/t/how-to-calculate-timestamp-difference-in-logstash-config-file/244732 "2020-08-12T14:25:10Z")
I have two timestamp fields: submit\_date("MM/dd/YYYY HH:mm") and completed date("MM/dd/YYYY HH:mm"). I want to calculate time difference between these two fields and wanted to add it as a new field while ingesting data … --- ## [Logstash plugin missing for google pub sub](https://discuss.elastic.co/t/logstash-plugin-missing-for-google-pub-sub/244724)
**Author:** [@Shalinicts](https://discuss.elastic.co/u/Shalinicts)\ **Replies:** 0\ **Last updated:** [August 12, 2020, 1:33pm UTC](https://discuss.elastic.co/t/logstash-plugin-missing-for-google-pub-sub/244724 "2020-08-12T13:33:23Z")
Hi Team, I ahve deployed Logstash in Kubernetes of version 6.3.2 and truying to get the input from google pub sub. Reecived error: Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, … --- ## [How to add values of a specific fields from 2 different version in Index?](https://discuss.elastic.co/t/how-to-add-values-of-a-specific-fields-from-2-different-version-in-index/244696)
**Author:** [@NirmitJain](https://discuss.elastic.co/u/NirmitJain)\ **Replies:** 0\ **Last updated:** [August 12, 2020, 10:21am UTC](https://discuss.elastic.co/t/how-to-add-values-of-a-specific-fields-from-2-different-version-in-index/244696 "2020-08-12T10:21:04Z")
Hi , I am looking for a way by which we can add value of a field already indexed with value of same field from new updated document to be indexed. for e.g. version 1 document: { "\_index" : "test\_index", "\_type" : "… --- ## [Multiple filebeat inputs and multiple logstash pipelines](https://discuss.elastic.co/t/multiple-filebeat-inputs-and-multiple-logstash-pipelines/244626)
**Author:** [@nino](https://discuss.elastic.co/u/nino)\ **Replies:** 4\ **Last updated:** [August 12, 2020, 7:30am UTC](https://discuss.elastic.co/t/multiple-filebeat-inputs-and-multiple-logstash-pipelines/244626 "2020-08-12T07:30:23Z")
Dear all, this is my scenario: one directory with two types of files that i want to proccess with one pipeline each. File types are identified by his name. I am very new to pipeline logstash, i usually go with a single… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=315) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=317)