# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=381 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 382 --- ## [Multiline codec produces ParseException: No close tag for /Events](https://discuss.elastic.co/t/multiline-codec-produces-parseexception-no-close-tag-for-events/226010)
**Author:** [@Paul\_Offord](https://discuss.elastic.co/u/Paul_Offord)\ **Replies:** 3\ **Last updated:** [April 1, 2020, 4:20pm UTC](https://discuss.elastic.co/t/multiline-codec-produces-parseexception-no-close-tag-for-events/226010 "2020-04-01T16:20:22Z")
Hi, I am ingesting an XML file that looks like this: \ \ - about 630,000 of these events ... stuff \ \ The input section of … --- ## [Send logs from native mobile apps to Kibana](https://discuss.elastic.co/t/send-logs-from-native-mobile-apps-to-kibana/226086)
**Author:** [@magdamiu](https://discuss.elastic.co/u/magdamiu)\ **Replies:** 0\ **Last updated:** [April 1, 2020, 4:08pm UTC](https://discuss.elastic.co/t/send-logs-from-native-mobile-apps-to-kibana/226086 "2020-04-01T16:08:03Z")
Hello, How could I send the logs from a native mobile app (android, ios) to Kibana? What product provided by elastic should I use? Thank you, Magda --- ## [Remove message and replace timestamp fields](https://discuss.elastic.co/t/remove-message-and-replace-timestamp-fields/226025)
**Author:** [@manikandanb87](https://discuss.elastic.co/u/manikandanb87)\ **Replies:** 2\ **Last updated:** [April 1, 2020, 3:49pm UTC](https://discuss.elastic.co/t/remove-message-and-replace-timestamp-fields/226025 "2020-04-01T15:49:56Z")
Hi Experts, I am using log stash to read my logs and I notice that the field "message" is there by default and has the complete data in it. So, I would like to remove it from my feed. Also, I want to replace "@timesta… --- ## [Parse json with multiple lists to different events](https://discuss.elastic.co/t/parse-json-with-multiple-lists-to-different-events/225987)
**Author:** [@aleixsb](https://discuss.elastic.co/u/aleixsb)\ **Replies:** 1\ **Last updated:** [April 1, 2020, 3:18pm UTC](https://discuss.elastic.co/t/parse-json-with-multiple-lists-to-different-events/225987 "2020-04-01T15:18:24Z")
Hi all, I'm trying to parse a json I get from an API which contains several lists of objects, which I would like to parse each object of each list to a different event. { "Time":"XXXXX", "listA":\[ \], "listB":\[… --- ## [Multiple logs showing Kibana](https://discuss.elastic.co/t/multiple-logs-showing-kibana/225769)
**Author:** [@anmohamed](https://discuss.elastic.co/u/anmohamed)\ **Replies:** 1\ **Last updated:** [April 1, 2020, 3:14pm UTC](https://discuss.elastic.co/t/multiple-logs-showing-kibana/225769 "2020-04-01T15:14:38Z")
Current Config is filter { grok { patterns\_dir =\> \["./patterns"\] match =\> { "message" =\> "\[%{TIMESTAMP\_ISO8601:timestamp} #%{POSINT:pid}\] \*%{RUBY\_LOGLEVEL:loglevel} -- +%{DATA:progname}: %{GREEDYDATA:message}" } } … --- ## [Technical enquiry - setting "hosts" in the "output" pipeline section](https://discuss.elastic.co/t/technical-enquiry-setting-hosts-in-the-output-pipeline-section/226028)
**Author:** [@Rysiu](https://discuss.elastic.co/u/Rysiu)\ **Replies:** 1\ **Last updated:** [April 1, 2020, 3:09pm UTC](https://discuss.elastic.co/t/technical-enquiry-setting-hosts-in-the-output-pipeline-section/226028 "2020-04-01T15:09:29Z")
Take for example the documentation for plugins-outputs-lumberjack: list of addresses lumberjack can send to This plugin can send data to hosts from an array called "hosts". This is simple. There is information in… --- ## [Dateparse failure jdbc](https://discuss.elastic.co/t/dateparse-failure-jdbc/226055)
**Author:** [@karlm](https://discuss.elastic.co/u/karlm)\ **Replies:** 1\ **Last updated:** [April 1, 2020, 3:02pm UTC](https://discuss.elastic.co/t/dateparse-failure-jdbc/226055 "2020-04-01T15:02:21Z")
i have an datetime column from sql and trying to map it to the @timetsamp field but receiving date parse error sql column date 2020-03-24 11:44:03.000 filter filter{ date { match =\> \[ "date", "yyyy-MM-dd HH:m… --- ## [I received the error ORA-01008: not all variables bound when i execute the logstash config file.can u pls tell about this error.Thanks in advance](https://discuss.elastic.co/t/i-received-the-error-ora-01008-not-all-variables-bound-when-i-execute-the-logstash-config-file-can-u-pls-tell-about-this-error-thanks-in-advance/226062)
**Author:** [@chelseia](https://discuss.elastic.co/u/chelseia)\ **Replies:** 0\ **Last updated:** [April 1, 2020, 2:18pm UTC](https://discuss.elastic.co/t/i-received-the-error-ora-01008-not-all-variables-bound-when-i-execute-the-logstash-config-file-can-u-pls-tell-about-this-error-thanks-in-advance/226062 "2020-04-01T14:18:20Z")
input{ jdbc{ clean\_run =\> true jdbc\_driver\_library =\> "C:/Program Files/SQL Developer 4/jdbc/lib/ojdbc8.jar" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver" jdbc\_connection\_string =\> "" jdbc\_user =\> "u… --- ## [Need help in capturing a pattern](https://discuss.elastic.co/t/need-help-in-capturing-a-pattern/225853)
**Author:** [@manikandanb87](https://discuss.elastic.co/u/manikandanb87)\ **Replies:** 4\ **Last updated:** [April 1, 2020, 12:41pm UTC](https://discuss.elastic.co/t/need-help-in-capturing-a-pattern/225853 "2020-04-01T12:41:41Z")
Hi Experts, Iam using filebeat to feed my logs to Logstash. Now, I need your help in capturing the occurences of "Sign on Code failed for user" in my logs. Kindly help me with the grok pattern code for it. My sample l… --- ## [Xpath throws ClassCastException for one particular XML node](https://discuss.elastic.co/t/xpath-throws-classcastexception-for-one-particular-xml-node/226038)
**Author:** [@Paul\_Offord](https://discuss.elastic.co/u/Paul_Offord)\ **Replies:** 1\ **Last updated:** [April 1, 2020, 12:02pm UTC](https://discuss.elastic.co/t/xpath-throws-classcastexception-for-one-particular-xml-node/226038 "2020-04-01T12:02:42Z")
Hi, I am using the XML filter to ingest a file that looks like this: \ ... other events \ \ \ **Author:** [@deepak\_deore](https://discuss.elastic.co/u/deepak_deore)\ **Replies:** 6\ **Last updated:** [April 1, 2020, 11:42am UTC](https://discuss.elastic.co/t/horizontal-scaling-doesnt-improve-logshash-performance-with-kinesis-input-plugin/225674 "2020-04-01T11:42:31Z") as per prometheus metrics, only 1 logstash pod is processing the kinesis input, we need to vertically scale the pods in this case. can multiple logstash not work in parallel with kinesis input? --- ## [Space in key : remove\_char\_key is not working](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 9\ **Last updated:** [April 1, 2020, 8:08am UTC](https://discuss.elastic.co/t/space-in-key-remove-char-key-is-not-working/225873 "2020-04-01T08:08:27Z")
Hello ! I have a nested part in a log I can't succeed to parse : Here is my config : filter { #replace second "msg" field by "nested" field mutate { gsub =\> \["message", "Message=", " nested="\] } kv { … --- ## [Some logs are missing in elk](https://discuss.elastic.co/t/some-logs-are-missing-in-elk/225977)
**Author:** [@Afrin](https://discuss.elastic.co/u/Afrin)\ **Replies:** 0\ **Last updated:** [April 1, 2020, 5:51am UTC](https://discuss.elastic.co/t/some-logs-are-missing-in-elk/225977 "2020-04-01T05:51:44Z")
Hi, The problem we are facing is some logs are missing in elk while few are available. We cannot find the root cause for this issue because the logs which are missing and those which are available look similar. We canno… --- ## [Need to create a timestamp by concatenating the fields](https://discuss.elastic.co/t/need-to-create-a-timestamp-by-concatenating-the-fields/225922)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 6\ **Last updated:** [April 1, 2020, 5:11am UTC](https://discuss.elastic.co/t/need-to-create-a-timestamp-by-concatenating-the-fields/225922 "2020-04-01T05:11:19Z")
log file : INFO \[Thread-6\] (newcleintt.java:1400) - Trying to connect to broker: local-broker (ssl://localhost:1234) filter used : grok { match =\> {"message" =\> "%{LOGLEVEL:loglevel} \\{%{Y… --- ## [Parsing logs from audit.log from /var/log/audit/audit.log](https://discuss.elastic.co/t/parsing-logs-from-audit-log-from-var-log-audit-audit-log/225577)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 5\ **Last updated:** [March 30, 2020, 11:05am UTC](https://discuss.elastic.co/t/parsing-logs-from-audit-log-from-var-log-audit-audit-log/225577 "2020-03-30T11:05:52Z")
Logfile contents type=USER\_END msg=audit(1585509001.049:14007): pid=15642 uid=0 auid=0 ses=1100 msg='op=PAM:session\_close grantors=pam\_loginuid,pam\_keyinit,pam\_limits,pam\_systemd acct="root" exe="/usr/sbin/crond" hostnam… --- ## [How to get the json api data into message field in http poller using logsatsh config file](https://discuss.elastic.co/t/how-to-get-the-json-api-data-into-message-field-in-http-poller-using-logsatsh-config-file/225686)
**Author:** [@Mithuna](https://discuss.elastic.co/u/Mithuna)\ **Replies:** 6\ **Last updated:** [April 1, 2020, 4:16am UTC](https://discuss.elastic.co/t/how-to-get-the-json-api-data-into-message-field-in-http-poller-using-logsatsh-config-file/225686 "2020-04-01T04:16:52Z")
Hi, Im using http poller to get API data output to be ingested into elastic search. I'm not getting any error but the message field is empty. My config file is input { stdin{} http\_poller { urls =\> { q1 =\> { … --- ## [Logstash change default @timestamp value by field from mongo](https://discuss.elastic.co/t/logstash-change-default-timestamp-value-by-field-from-mongo/225911)
**Author:** [@7cinquante](https://discuss.elastic.co/u/7cinquante)\ **Replies:** 6\ **Last updated:** [March 31, 2020, 11:07pm UTC](https://discuss.elastic.co/t/logstash-change-default-timestamp-value-by-field-from-mongo/225911 "2020-03-31T23:07:51Z")
Hello, I would like get @timestamp to be my mongo field "created\_at", i do input { jdbc { jdbc\_driver\_library =\> "/usr/share/logstash/logstash-core/lib/jars/mongojdbc1.8.jar" jdbc\_driver\_class =\> "com.dbschem… --- ## [Logstash: Replace @timestamp with time like this 2020-03-27T16:00:40.982652300+00:00](https://discuss.elastic.co/t/logstash-replace-timestamp-with-time-like-this-2020-03-27t1640-982652300-00-00/225938)
**Author:** [@Paul\_Offord](https://discuss.elastic.co/u/Paul_Offord)\ **Replies:** 5\ **Last updated:** [March 31, 2020, 11:06pm UTC](https://discuss.elastic.co/t/logstash-replace-timestamp-with-time-like-this-2020-03-27t1640-982652300-00-00/225938 "2020-03-31T23:06:44Z")
Hi, I need to replace the @timestamp value with a field value (called SystemTime) that looks like this: 2020-03-27T16:00:40.982652300+00:00 I have tried numerous variations of mutate gsub and the date filter but it's o… --- ## [Issue in loading CSV file to Elastic search because of \\n using LS 7](https://discuss.elastic.co/t/issue-in-loading-csv-file-to-elastic-search-because-of-n-using-ls-7/225945)
**Author:** [@rsuper\_6616](https://discuss.elastic.co/u/rsuper_6616)\ **Replies:** 0\ **Last updated:** [March 31, 2020, 7:51pm UTC](https://discuss.elastic.co/t/issue-in-loading-csv-file-to-elastic-search-because-of-n-using-ls-7/225945 "2020-03-31T19:51:40Z")
I am loading csv file using LS 7 to Elastic cloud. the document contain we message column that contains \\n character within the message content. below is the configuration file input { file { path =\> "D:/AzureTables/ST… --- ## [Multiple logstash intances](https://discuss.elastic.co/t/multiple-logstash-intances/224673)
**Author:** [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\ **Replies:** 4\ **Last updated:** [March 31, 2020, 7:31pm UTC](https://discuss.elastic.co/t/multiple-logstash-intances/224673 "2020-03-31T19:31:04Z")
Hi, I am trying to run multiple instances of logstash using multiple pipeline. - pipeline.id: product\_info path.config: "/home/ubuntu/dev/V4QaShop/logstash/products.conf" #pipeline.workers: 3 path.dat… --- ## [Ingested CSV Data into logstasth but not showing in kibana](https://discuss.elastic.co/t/ingested-csv-data-into-logstasth-but-not-showing-in-kibana/225772)
**Author:** [@lohit18](https://discuss.elastic.co/u/lohit18)\ **Replies:** 2\ **Last updated:** [March 31, 2020, 6:49pm UTC](https://discuss.elastic.co/t/ingested-csv-data-into-logstasth-but-not-showing-in-kibana/225772 "2020-03-31T18:49:21Z")
Dear team, I ingested CSV data path to logstash .conf file but it is not showing in kibana Dashboard .below is my .conf code . input { file { path=\>"C:\\Users\\Downloads\\Flow\_data.csv" start\_position=\>"beginning" si… --- ## [Grok filters works separately when combined it does not work. Please help](https://discuss.elastic.co/t/grok-filters-works-separately-when-combined-it-does-not-work-please-help/225579)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 2\ **Last updated:** [March 31, 2020, 6:37pm UTC](https://discuss.elastic.co/t/grok-filters-works-separately-when-combined-it-does-not-work-please-help/225579 "2020-03-31T18:37:42Z")
Please find my conf file. The filters works fine when used separate file but when combined they dont show the parsed fields in kibana. input { file { path =\> "/var/log/messages" start\_position =\> "beginning" … --- ## [How is Logstash Throughput Calculated in Stack Monitoring?](https://discuss.elastic.co/t/how-is-logstash-throughput-calculated-in-stack-monitoring/225046)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 1\ **Last updated:** [March 31, 2020, 4:02pm UTC](https://discuss.elastic.co/t/how-is-logstash-throughput-calculated-in-stack-monitoring/225046 "2020-03-31T16:02:50Z")
I'm trying to recreate the stack monitoring page for Logstash 7.6 in a dashboard , specifically so that I can see throughput for a physical node that has multiple Logstash instances deployed on it, as shown in another po… --- ## [Double quotes inside field: grok failure / unable to remove](https://discuss.elastic.co/t/double-quotes-inside-field-grok-failure-unable-to-remove/225903)
**Author:** [@holobolo0815](https://discuss.elastic.co/u/holobolo0815)\ **Replies:** 1\ **Last updated:** [March 31, 2020, 4:02pm UTC](https://discuss.elastic.co/t/double-quotes-inside-field-grok-failure-unable-to-remove/225903 "2020-03-31T16:02:01Z")
I have a message like this: result=accept, user=username, extip=1.2.3.4, nasip=5.6.7.8, realm=user(realm)"Tue Mar 31 16:59:24 2020"4KalZzYgN, internalevent= I am filtering like follows kv { source =\> "msg" field\_… --- ## [Kv Filter - allow\_duplicate\_values different behavior](https://discuss.elastic.co/t/kv-filter-allow-duplicate-values-different-behavior/225736)
**Author:** [@GitsBdr](https://discuss.elastic.co/u/GitsBdr)\ **Replies:** 6\ **Last updated:** [March 31, 2020, 3:49pm UTC](https://discuss.elastic.co/t/kv-filter-allow-duplicate-values-different-behavior/225736 "2020-03-31T15:49:47Z")
Hi guys, Thanks for taking the time to read these lines, I've got a question regarding kv filter and its allow\_duplicate\_values attribute Assuming I have a log that looks like : key1=value1|key2=value2|key3=value3|ke… --- ## [Delete elastic document by \_id found via elasticsearch filter plugin in logstash](https://discuss.elastic.co/t/delete-elastic-document-by-id-found-via-elasticsearch-filter-plugin-in-logstash/225878)
**Author:** [@Charles2](https://discuss.elastic.co/u/Charles2)\ **Replies:** 2\ **Last updated:** [March 31, 2020, 3:29pm UTC](https://discuss.elastic.co/t/delete-elastic-document-by-id-found-via-elasticsearch-filter-plugin-in-logstash/225878 "2020-03-31T15:29:24Z")
Hi, so currently i am ingestying data from kafka, filtering it throght logstash 7.2 and storying it in elasticsearch 7.2. The data im receiving are jamming events so the trap has an id and for each trap i should have 2 … --- ## [Logstash not reading config files](https://discuss.elastic.co/t/logstash-not-reading-config-files/225868)
**Author:** [@Urvi3122](https://discuss.elastic.co/u/Urvi3122)\ **Replies:** 1\ **Last updated:** [March 31, 2020, 3:16pm UTC](https://discuss.elastic.co/t/logstash-not-reading-config-files/225868 "2020-03-31T15:16:16Z")
Does Elastic search not reading the config file kept inside below location? /etc/logstash/conf.d/all\_configs/asses\_sales/a0.conf --- ## [Logstash configuration in multiple files](https://discuss.elastic.co/t/logstash-configuration-in-multiple-files/225399)
**Author:** [@jimmyvalmer19](https://discuss.elastic.co/u/jimmyvalmer19)\ **Replies:** 2\ **Last updated:** [March 31, 2020, 12:27pm UTC](https://discuss.elastic.co/t/logstash-configuration-in-multiple-files/225399 "2020-03-31T12:27:13Z")
In logstash 7.3.0 i just to have conf like 01\_input.conf 02\_parser-a 03\_parser-b ... nn\_parser-z 99\_output.conf Now i use Logstash 7.6.1 It seems it has problem, when filter {} section is spread among more files … --- ## [Logstash fields and tags. Can they be seen when being outputted to an s3 bucket? and can they be referenced on pick up from s3 bucket?](https://discuss.elastic.co/t/logstash-fields-and-tags-can-they-be-seen-when-being-outputted-to-an-s3-bucket-and-can-they-be-referenced-on-pick-up-from-s3-bucket/225848)
**Author:** [@Apollyon](https://discuss.elastic.co/u/Apollyon)\ **Replies:** 0\ **Last updated:** [March 31, 2020, 12:04pm UTC](https://discuss.elastic.co/t/logstash-fields-and-tags-can-they-be-seen-when-being-outputted-to-an-s3-bucket-and-can-they-be-referenced-on-pick-up-from-s3-bucket/225848 "2020-03-31T12:04:22Z")
Hello, Using ELK 7.6. The current set up I have at the moment is that: 1 - Data source is coming in and being thrown into an s3 bucket (Bucket A) from a remote logstash instance. 2 - An ec2 instance (instance 1) is t… --- ## [Gelf logger setup to send logs to multiple nodes of my ELK cluster](https://discuss.elastic.co/t/gelf-logger-setup-to-send-logs-to-multiple-nodes-of-my-elk-cluster/225839)
**Author:** [@Alexandros888](https://discuss.elastic.co/u/Alexandros888)\ **Replies:** 0\ **Last updated:** [March 31, 2020, 11:13am UTC](https://discuss.elastic.co/t/gelf-logger-setup-to-send-logs-to-multiple-nodes-of-my-elk-cluster/225839 "2020-03-31T11:13:53Z")
Hello I am using gelf logger to send logs from my service that is hosted in openshift to the elk. I can send successfully the logs to 1 node of my 3 nodes cluster according to the setup that the image shows below. Nev… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=380) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=382)