# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=416 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 417 --- ## [Index Management CISCO ASA device](https://discuss.elastic.co/t/index-management-cisco-asa-device/216051)
**Author:** [@Yashwant\_Shettigar](https://discuss.elastic.co/u/Yashwant_Shettigar)\ **Replies:** 0\ **Last updated:** [January 22, 2020, 12:06pm UTC](https://discuss.elastic.co/t/index-management-cisco-asa-device/216051 "2020-01-22T12:06:03Z")
Hello, I have a centralized ELK server, where data is getting shipped via winlogbeat and filebeat installed on client machines. Now I am trying to route cisco-asa devices logs to this server. Upto some extent I am bit s… --- ## [Help with grok pattern for haproxy](https://discuss.elastic.co/t/help-with-grok-pattern-for-haproxy/215441)
**Author:** [@mmikulic](https://discuss.elastic.co/u/mmikulic)\ **Replies:** 3\ **Last updated:** [January 22, 2020, 12:02pm UTC](https://discuss.elastic.co/t/help-with-grok-pattern-for-haproxy/215441 "2020-01-22T12:02:55Z")
Hi,please assit with creating custom grok pattern for logstash. I've created a custom log format in haproxy conf for purpose of logging mutual ssl authentication details for client certificates. log-format %ci:%cp\\ \[%t… --- ## [Multiple match in Grok filter logstash is not working](https://discuss.elastic.co/t/multiple-match-in-grok-filter-logstash-is-not-working/215874)
**Author:** [@Saravana37](https://discuss.elastic.co/u/Saravana37)\ **Replies:** 14\ **Last updated:** [January 22, 2020, 11:24am UTC](https://discuss.elastic.co/t/multiple-match-in-grok-filter-logstash-is-not-working/215874 "2020-01-22T11:24:09Z")
Hello , My Grok filter looks like below grok { match =\> \["message","%{WORD:ACT\_SV\_NAME}%{SPACE}%{WORD:ACT\_CC\_Alias}%{SPACE}%{WORD:ACT\_CG\_ALIAS}%{SPACE}%{WORD:ACT\_Running\_Status}%{SPACE}%{GREEDYDATA:ACT\_OM\_Session}","… --- ## [Error with Logstash CSV Filter plugin](https://discuss.elastic.co/t/error-with-logstash-csv-filter-plugin/215879)
**Author:** [@mhyager](https://discuss.elastic.co/u/mhyager)\ **Replies:** 2\ **Last updated:** [January 22, 2020, 10:45am UTC](https://discuss.elastic.co/t/error-with-logstash-csv-filter-plugin/215879 "2020-01-22T10:45:09Z")
I am having strange errors with a specific word used in Column headers. All automatic column header recognition works fine, except for when column headers are called FName. Is this a limitation of the CSV Filter plugin? … --- ## [Getting nested object with json filter](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921)
**Author:** [@toms130](https://discuss.elastic.co/u/toms130)\ **Replies:** 9\ **Last updated:** [January 22, 2020, 10:49am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921 "2020-01-22T10:49:16Z")
Hi all, I've got problem when trying to get some nested fields in a json object with json filter. My field look like this { "total\_count" : 3, "orders" : \[ { "marketplace\_order\_id" : "malikilledme3", "order\_line… --- ## [TIBCO EMS plugin connection](https://discuss.elastic.co/t/tibco-ems-plugin-connection/216024)
**Author:** [@sumitsinghtomar09](https://discuss.elastic.co/u/sumitsinghtomar09)\ **Replies:** 1\ **Last updated:** [January 22, 2020, 10:13am UTC](https://discuss.elastic.co/t/tibco-ems-plugin-connection/216024 "2020-01-22T10:13:34Z")
Hi All, I am trying to connect my local tibco ems server from logstanch, i am getting below error. Sending Logstash logs to C:/Users/vv19sn/Downloads/logstash-7.5.1/logstash-7.5.1/logs which is now configured via log4j… --- ## [How to get the nested array and nested json from input message in logstash](https://discuss.elastic.co/t/how-to-get-the-nested-array-and-nested-json-from-input-message-in-logstash/215693)
**Author:** [@Amit\_Agarwal1](https://discuss.elastic.co/u/Amit_Agarwal1)\ **Replies:** 11\ **Last updated:** [January 22, 2020, 9:43am UTC](https://discuss.elastic.co/t/how-to-get-the-nested-array-and-nested-json-from-input-message-in-logstash/215693 "2020-01-22T09:43:00Z")
message : {"prdctId":"10066","id":"10066","owner":{"id":"E040547","name":"Shashi Raghunandan"},"regAvail":\["ap","lac","mea"\] ,"isActive":true} Applied Filter: kv { source =\> "message" trim\_key =\> """ trim\_value =\> … --- ## [Logstash wait for Elasticsearch](https://discuss.elastic.co/t/logstash-wait-for-elasticsearch/216012)
**Author:** [@marcohald](https://discuss.elastic.co/u/marcohald)\ **Replies:** 0\ **Last updated:** [January 22, 2020, 8:58am UTC](https://discuss.elastic.co/t/logstash-wait-for-elasticsearch/216012 "2020-01-22T08:58:28Z")
Hi, is there a config variable which defines the retry count to elasticsearch on startup? In the moment it does a retry 10 times within 16 seconds. I already tried this as mentioned here https://stackoverflow.com/ques… --- ## [Logstash RESTFUL API Input](https://discuss.elastic.co/t/logstash-restful-api-input/215963)
**Author:** [@BDokas](https://discuss.elastic.co/u/BDokas)\ **Replies:** 0\ **Last updated:** [January 21, 2020, 11:26pm UTC](https://discuss.elastic.co/t/logstash-restful-api-input/215963 "2020-01-21T23:26:16Z")
Has anyone had success exposing an endpoint on an AWS EC2 instance running the ELK stack using the Logstash http input plugin? I cannot for the life of me figure out how to hit the specified input. --- ## [Logstash jdbc\_static filter plugin performing very poorly (nearly not at all)](https://discuss.elastic.co/t/logstash-jdbc-static-filter-plugin-performing-very-poorly-nearly-not-at-all/214292)
**Author:** [@tomj](https://discuss.elastic.co/u/tomj)\ **Replies:** 26\ **Last updated:** [January 21, 2020, 10:35pm UTC](https://discuss.elastic.co/t/logstash-jdbc-static-filter-plugin-performing-very-poorly-nearly-not-at-all/214292 "2020-01-21T22:35:20Z")
Running Logstash 7.1.1 on Amazon Linux 2; jdbc\_static plugin version 1.0.6 (stock installation for this version of Logstash) Sending data to Elasticsearch 7.1.1 (hosted on AWS Elasticsearch service) Logstash is running… --- ## [Logstash.inputs.mongodb, fail \](https://discuss.elastic.co/t/logstash-inputs-mongodb-fail-bson-1-is-an-invalid-objectid/215570)
**Author:** [@angelma83](https://discuss.elastic.co/u/angelma83)\ **Replies:** 1\ **Last updated:** [January 21, 2020, 10:33pm UTC](https://discuss.elastic.co/t/logstash-inputs-mongodb-fail-bson-1-is-an-invalid-objectid/215570 "2020-01-21T22:33:56Z")
HI everybody, i am using logstash-input-mongodb, and i have the following mistake, \[logstash.inputs.mongodb \] MongoDB Input threw an exception, restarting {:exception=\>#\ **Author:** [@albbapm](https://discuss.elastic.co/u/albbapm)\ **Replies:** 2\ **Last updated:** [January 21, 2020, 8:37pm UTC](https://discuss.elastic.co/t/logstash-stopped-accepting-events-from-winlogbeat-since-the-new-year/215941 "2020-01-21T20:37:45Z") Posting this here as I think this is a logstash problem although I'm not really sure. ELK version 6.7.2 Winlogbeat version is a mix of 6.7.2 and 6.3.2 At midnight on January 1st, all winlogbeat events stopped going in… --- ## [Date filter not working - Logstash](https://discuss.elastic.co/t/date-filter-not-working-logstash/215880)
**Author:** [@Shahid\_Mustafa](https://discuss.elastic.co/u/Shahid_Mustafa)\ **Replies:** 6\ **Last updated:** [January 21, 2020, 8:33pm UTC](https://discuss.elastic.co/t/date-filter-not-working-logstash/215880 "2020-01-21T20:33:55Z")
I am trying to use following date filter to convert string to date but it doesn't seem to be working. Sample input data(string) - Jan 20 09:20:35 GMT 2020 I am first using a mutate gsub to remove GMT which renders foll… --- ## [Logstash-output-mongodb issues with DocumentDB](https://discuss.elastic.co/t/logstash-output-mongodb-issues-with-documentdb/215929)
**Author:** [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Replies:** 1\ **Last updated:** [January 21, 2020, 6:16pm UTC](https://discuss.elastic.co/t/logstash-output-mongodb-issues-with-documentdb/215929 "2020-01-21T18:16:06Z")
Hi, Has anyone tried accessing DocumentDB (essentially Mongo under hood) via logstash-ouput-mongodb. In logstash 7.5.1 I am getting the following error: logstash.outputs.mongodb \]\[main\] MONGODB | Unsupported URI optio… --- ## [Logstash yum updates won't reinstall my modules](https://discuss.elastic.co/t/logstash-yum-updates-wont-reinstall-my-modules/215935)
**Author:** [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Replies:** 0\ **Last updated:** [January 21, 2020, 5:53pm UTC](https://discuss.elastic.co/t/logstash-yum-updates-wont-reinstall-my-modules/215935 "2020-01-21T17:53:16Z")
I installed Logstash via yum, and now every time yum pushes the new Logstash package, it won't reinstall any of the modules I installed myself. For example, I have to reinstall logstash-input-google\_pubsub module after … --- ## [Parse Logs in Logstash](https://discuss.elastic.co/t/parse-logs-in-logstash/215805)
**Author:** [@SerSSH](https://discuss.elastic.co/u/SerSSH)\ **Replies:** 4\ **Last updated:** [January 21, 2020, 3:44pm UTC](https://discuss.elastic.co/t/parse-logs-in-logstash/215805 "2020-01-21T15:44:35Z")
Hi guys, I need help with the parsing of some logs. I have tried to do them with the grok plugin, however I need to optimize this for a large number of logs that have different formats and different fields or variables. … --- ## [How to iterate through several elements and create a new field with the content of each element](https://discuss.elastic.co/t/how-to-iterate-through-several-elements-and-create-a-new-field-with-the-content-of-each-element/215656)
**Author:** [@alejovaz](https://discuss.elastic.co/u/alejovaz)\ **Replies:** 2\ **Last updated:** [January 21, 2020, 12:09pm UTC](https://discuss.elastic.co/t/how-to-iterate-through-several-elements-and-create-a-new-field-with-the-content-of-each-element/215656 "2020-01-21T12:09:25Z")
My Logstash receive of a http input this field PANELS.memory { "availableBytes": 6212550656, "totalBytes": 8493137920, "type": "physical" }, { "availableBytes": 14702407680, "totalBytes": 17256087552, "type… --- ## [How effective translate field in logstash](https://discuss.elastic.co/t/how-effective-translate-field-in-logstash/214411)
**Author:** [@vinu89](https://discuss.elastic.co/u/vinu89)\ **Replies:** 3\ **Last updated:** [January 21, 2020, 11:44am UTC](https://discuss.elastic.co/t/how-effective-translate-field-in-logstash/214411 "2020-01-21T11:44:45Z")
Hi, I have a set of threat intel feeds i want to check whether those blacklisted ip occurs in my logs... i tried using translate field it works fine...i want to know whether we can compare billions of blacklisted ip usi… --- ## [Logstash password encryption streangth](https://discuss.elastic.co/t/logstash-password-encryption-streangth/215868)
**Author:** [@Manjula\_Piyumal](https://discuss.elastic.co/u/Manjula_Piyumal)\ **Replies:** 0\ **Last updated:** [January 21, 2020, 11:11am UTC](https://discuss.elastic.co/t/logstash-password-encryption-streangth/215868 "2020-01-21T11:11:55Z")
Hi, I'm going to use logstash-keystore to encrypt my jdbc passwords . May I know the default encryption strength(keysize) and algorithm if I use below steps, https://www.elastic.co/guide/en/logstash/current/keystore.ht… --- ## [Random loss of data import between filebeat and logstash](https://discuss.elastic.co/t/random-loss-of-data-import-between-filebeat-and-logstash/215288)
**Author:** [@tkauffmann](https://discuss.elastic.co/u/tkauffmann)\ **Replies:** 1\ **Last updated:** [January 21, 2020, 11:25am UTC](https://discuss.elastic.co/t/random-loss-of-data-import-between-filebeat-and-logstash/215288 "2020-01-21T11:25:12Z")
Hi, Every night, short after midnight we export data from a Mysql database into several csv files. Filebeat runs on the same server and is configured to send these data to the elastic server also running logstash. Log… --- ## [Logstash index naming pattern - use of an alternate tag if one is missing](https://discuss.elastic.co/t/logstash-index-naming-pattern-use-of-an-alternate-tag-if-one-is-missing/215767)
**Author:** [@jreid](https://discuss.elastic.co/u/jreid)\ **Replies:** 1\ **Last updated:** [January 21, 2020, 11:15am UTC](https://discuss.elastic.co/t/logstash-index-naming-pattern-use-of-an-alternate-tag-if-one-is-missing/215767 "2020-01-21T11:15:43Z")
Data is sourced from filebeat and packetbeat with add\_kubernetes\_metadata and passed to logstash, using the logstash helm chart. In logstash, I want the logs to be output to an elasticsearch index, named based on a kube… --- ## [lGot an error - MongoDB Input threw an exception, restarting {:exception=\>#\' is an invalid ObjectId.\>}](https://discuss.elastic.co/t/lgot-an-error-mongodb-input-threw-an-exception-restarting-exception-bson-bson-0x25ae0d1c-is-an-invalid-objectid/214830)
**Author:** [@shubhamblackstratus](https://discuss.elastic.co/u/shubhamblackstratus)\ **Replies:** 1\ **Last updated:** [January 21, 2020, 9:53am UTC](https://discuss.elastic.co/t/lgot-an-error-mongodb-input-threw-an-exception-restarting-exception-bson-bson-0x25ae0d1c-is-an-invalid-objectid/214830 "2020-01-21T09:53:58Z")
Hello Guys, While working with " \[logstash-input-mongodb\]" plugin, I got the below error in logstash logs,. \[WARN \]\[logstash.inputs.mongodb \] MongoDB Input threw an exception, restarting {:exception=\>#\ **Author:** [@saif3r](https://discuss.elastic.co/u/saif3r)\ **Replies:** 2\ **Last updated:** [January 21, 2020, 9:51am UTC](https://discuss.elastic.co/t/xml-filter-exception-in-pipelineworker-the-pipeline-stopped-processing-new-events-please-check-your-filter-configuration-and-restart-logstash-how-to-resume-logstash-or-skip-failed-event/215845 "2020-01-21T09:51:39Z") Hi, I have a Logstash pipeline that parses events from multiple filebeat sources. Those filebeats are sending multiline XML files. After a month or so, it received an event that stops logstash due to xml parsing error (… --- ## [Logstash is unable to connect to ES for Monitoring due to bad certificate](https://discuss.elastic.co/t/logstash-is-unable-to-connect-to-es-for-monitoring-due-to-bad-certificate/215841)
**Author:** [@Kosodrom](https://discuss.elastic.co/u/Kosodrom)\ **Replies:** 1\ **Last updated:** [January 21, 2020, 9:38am UTC](https://discuss.elastic.co/t/logstash-is-unable-to-connect-to-es-for-monitoring-due-to-bad-certificate/215841 "2020-01-21T09:38:52Z")
Hi guys, I cannot see my logstash node in Kibana because it cannot connect to elasticsearch due to bad certificate: \[2020-01-21T08:26:01,246\]\[ERROR\]\[logstash.licensechecker.licensereader\] Unable to retrieve license inf… --- ## [Will EOL Logstash's "logstash-plugin update" keep getting updates even when they are not supported?](https://discuss.elastic.co/t/will-eol-logstashs-logstash-plugin-update-keep-getting-updates-even-when-they-are-not-supported/215842)
**Author:** [@denisnovac](https://discuss.elastic.co/u/denisnovac)\ **Replies:** 0\ **Last updated:** [January 21, 2020, 8:49am UTC](https://discuss.elastic.co/t/will-eol-logstashs-logstash-plugin-update-keep-getting-updates-even-when-they-are-not-supported/215842 "2020-01-21T08:49:01Z")
There is no plugin versions support matrix on the documentation, but it says that "The plugins listed below are maintained and supported by Elastic. Plugin versions that are installable on an Elastic supported Logstash r… --- ## [Conditional statement not working in filter](https://discuss.elastic.co/t/conditional-statement-not-working-in-filter/215817)
**Author:** [@Charles\_Yuliansen](https://discuss.elastic.co/u/Charles_Yuliansen)\ **Replies:** 3\ **Last updated:** [January 21, 2020, 8:33am UTC](https://discuss.elastic.co/t/conditional-statement-not-working-in-filter/215817 "2020-01-21T08:33:05Z")
Im trying to learn conditional filtering, the grok outside expression successfully parsed, but not with grok inside conditional expression. target \[2020-01-09 08:32:46\] VERBOSE\[18962\]\[C-0000ceae\] pbx.c: Executing \[s@ma… --- ## [Discarding unnecessary data](https://discuss.elastic.co/t/discarding-unnecessary-data/215677)
**Author:** [@RichardH](https://discuss.elastic.co/u/RichardH)\ **Replies:** 2\ **Last updated:** [January 21, 2020, 7:34am UTC](https://discuss.elastic.co/t/discarding-unnecessary-data/215677 "2020-01-21T07:34:23Z")
Hello, I am just starting with logstash and writing filters, and trying to better understand how it works. We want to push all syslog messages to logstash but I want to ensure that events are only sent to elastic if the… --- ## [Using logstash on Elastic (GCP) hosted instance](https://discuss.elastic.co/t/using-logstash-on-elastic-gcp-hosted-instance/215698)
**Author:** [@prettysimpl](https://discuss.elastic.co/u/prettysimpl)\ **Replies:** 1\ **Last updated:** [January 21, 2020, 7:14am UTC](https://discuss.elastic.co/t/using-logstash-on-elastic-gcp-hosted-instance/215698 "2020-01-21T07:14:09Z")
Hello All, We have a new elastic hosted instance (on gcp) running and would like to use logstash on there. I could not find any documentation on this and the only recent topic on the forum on something similar has not … --- ## [How to logstash filter xml](https://discuss.elastic.co/t/how-to-logstash-filter-xml/215680)
**Author:** [@peacher](https://discuss.elastic.co/u/peacher)\ **Replies:** 8\ **Last updated:** [January 21, 2020, 2:44am UTC](https://discuss.elastic.co/t/how-to-logstash-filter-xml/215680 "2020-01-21T02:44:56Z")
Hi, i have logstash config input { file { path =\> \["/mnt/data/\*/\*.xml"\] start\_position =\> beginning #sincedb\_path =\> "/opt/logstash/sincedb-xml49" sincedb\_path =\> "/dev/null" … --- ## [How to move all the json fields at root level to a sub json node?](https://discuss.elastic.co/t/how-to-move-all-the-json-fields-at-root-level-to-a-sub-json-node/215522)
**Author:** [@STAR\_DEVX](https://discuss.elastic.co/u/STAR_DEVX)\ **Replies:** 2\ **Last updated:** [January 20, 2020, 9:55pm UTC](https://discuss.elastic.co/t/how-to-move-all-the-json-fields-at-root-level-to-a-sub-json-node/215522 "2020-01-20T21:55:32Z")
I want to move all the json node at the root level to a sub level jason node. What's the most efficient way to use filters to achieve that? root level { "a": 1, "b": "str", "c": {"key": "value"} } I want the resul… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=415) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=417)