# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=667 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 668 --- ## [Where patterns are stored](https://discuss.elastic.co/t/where-patterns-are-stored/152662)
**Author:** [@sebasd44](https://discuss.elastic.co/u/sebasd44)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 1:15pm UTC](https://discuss.elastic.co/t/where-patterns-are-stored/152662 "2018-10-16T13:15:51Z")
Hello 'm really new to elasticsearch and i've been looking for the patterns but i cannot find them because i want to test some filters for cisco routers and second y kibana i'm only seeing counts and the pattern that i… --- ## [Index name with http input and es\_bulk codec](https://discuss.elastic.co/t/index-name-with-http-input-and-es-bulk-codec/152650)
**Author:** [@redX](https://discuss.elastic.co/u/redX)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 12:10pm UTC](https://discuss.elastic.co/t/index-name-with-http-input-and-es-bulk-codec/152650 "2018-10-16T12:10:40Z")
TL;DR: I get bulk inserts, but the index name is not in the output of logstash. How do I get the index name from the input request? Long version: I have multiple single index request in bulk form. e.g.: PUT /\_bulk HT… --- ## [Logstash input-jdbc work logic and last run sql\_last\_value](https://discuss.elastic.co/t/logstash-input-jdbc-work-logic-and-last-run-sql-last-value/152649)
**Author:** [@vara0855](https://discuss.elastic.co/u/vara0855)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 12:07pm UTC](https://discuss.elastic.co/t/logstash-input-jdbc-work-logic-and-last-run-sql-last-value/152649 "2018-10-16T12:07:23Z")
Hello I have ELK 6.3.2v. Logstash-input-jdbc : v4.3.9 In my current configuration for the JDBC pipeline there are about 15-20 inputs with JDBC queries. Such inputs scheduled for run every minute -\> "\* \* \* \* \*" or ever… --- ## [Logstash cannot correctly send data of mongo collection to elasticsaearch](https://discuss.elastic.co/t/logstash-cannot-correctly-send-data-of-mongo-collection-to-elasticsaearch/152642)
**Author:** [@NIloufarVafaei](https://discuss.elastic.co/u/NIloufarVafaei)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 11:31am UTC](https://discuss.elastic.co/t/logstash-cannot-correctly-send-data-of-mongo-collection-to-elasticsaearch/152642 "2018-10-16T11:31:42Z")
hi here is my logstash config input { mongodb { uri =\> 'mongodb://boomrang:\*\*\*\*\*\*\*/boomrang' placeholder\_db\_dir =\> '/tmp' placeholder\_db\_name =\> 'boomrang\_tarrifs.db' collection =\> 'tariffs' batch\_size =\> 5000 } … --- ## [Tags being added by whom? kafka/elasticsearch plugins](https://discuss.elastic.co/t/tags-being-added-by-whom-kafka-elasticsearch-plugins/152638)
**Author:** [@Miguel\_Oliveira](https://discuss.elastic.co/u/Miguel_Oliveira)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 11:18am UTC](https://discuss.elastic.co/t/tags-being-added-by-whom-kafka-elasticsearch-plugins/152638 "2018-10-16T11:18:53Z")
Hello, We currently have a kafka input sending to an elasticsearch output. We're not defining any tags in between but sometimes, not all the times, a document is added to ES containing the following a field tags with a… --- ## [MultiLine Message handling](https://discuss.elastic.co/t/multiline-message-handling/152229)
**Author:** [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\ **Replies:** 2\ **Last updated:** [October 16, 2018, 10:53am UTC](https://discuss.elastic.co/t/multiline-message-handling/152229 "2018-10-16T10:53:59Z")
Hi, Its already quoted in the logstash as follows If you are using a Logstash input plugin that supports multiple hosts, such as the \[beats input plugin\](http://www.elastic.co/guide/en/logstash/6.4/plugins-inputs-beat… --- ## [How to parse sub records of cloudtrail logs](https://discuss.elastic.co/t/how-to-parse-sub-records-of-cloudtrail-logs/148258)
**Author:** [@Botowi](https://discuss.elastic.co/u/Botowi)\ **Replies:** 5\ **Last updated:** [October 16, 2018, 10:37am UTC](https://discuss.elastic.co/t/how-to-parse-sub-records-of-cloudtrail-logs/148258 "2018-10-16T10:37:19Z")
Hello there I'm new to ELK and need your help for configuring ELK. I have below logstash config for parsing cloudtrail logs and this breaks/splits the 'Records' field json but i still see some sub records like - Records… --- ## [Why does my ruby filter seems to act randomly?](https://discuss.elastic.co/t/why-does-my-ruby-filter-seems-to-act-randomly/151656)
**Author:** [@OphyTe](https://discuss.elastic.co/u/OphyTe)\ **Replies:** 1\ **Last updated:** [October 9, 2018, 4:45pm UTC](https://discuss.elastic.co/t/why-does-my-ruby-filter-seems-to-act-randomly/151656 "2018-10-09T16:45:37Z")
I have multiple timestamp fields that I want to transform in long (cf. my previous post). Here is my logstash filter config : input { jdbc { ... } } filter { ruby { path =\> "/etc/logstash/date\_fields\_convertor.… --- ## [What if it is not an array after split?](https://discuss.elastic.co/t/what-if-it-is-not-an-array-after-split/152607)
**Author:** [@jmkim](https://discuss.elastic.co/u/jmkim)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 8:36am UTC](https://discuss.elastic.co/t/what-if-it-is-not-an-array-after-split/152607 "2018-10-16T08:36:14Z")
I want to treat an array as an if statement. mutate { split =\> \[ "cookie\_ids", ";" \] } if \[cookie\_ids\]\[0\] { mutate { add\_field =\> \[ "cookie\_id", "%{cookie\_ids\[0\]}" \] } } else { mutate { add\_field =\> \[ "cookie\_… --- ## [Extract log between two patterns](https://discuss.elastic.co/t/extract-log-between-two-patterns/152426)
**Author:** [@Karanth](https://discuss.elastic.co/u/Karanth)\ **Replies:** 3\ **Last updated:** [October 16, 2018, 8:43am UTC](https://discuss.elastic.co/t/extract-log-between-two-patterns/152426 "2018-10-16T08:43:57Z")
Hi We have logs that are very huge. I want to be able to extract logs \[events\] between two patterns. For example: 2018/08/29 14:33:54.646 - some log 2018/08/29 14:33:58.809 - pattern1 - rest of the log message under… --- ## [Fixed The Json But Logstash still produces Jsonparseerror](https://discuss.elastic.co/t/fixed-the-json-but-logstash-still-produces-jsonparseerror/152601)
**Author:** [@hfakar](https://discuss.elastic.co/u/hfakar)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 8:22am UTC](https://discuss.elastic.co/t/fixed-the-json-but-logstash-still-produces-jsonparseerror/152601 "2018-10-16T08:22:16Z")
Hello There I have logs coming in to syslogng. But some of the logs do not have the correct Json structure. This is the error log: \[2018-10-16T07:40:32,778\]\[WARN \]\[logstash.codecs.jsonlines\] JSON parse error, original… --- ## [How to convert my date field to my local timezone](https://discuss.elastic.co/t/how-to-convert-my-date-field-to-my-local-timezone/152263)
**Author:** [@OphyTe](https://discuss.elastic.co/u/OphyTe)\ **Replies:** 2\ **Last updated:** [October 16, 2018, 8:28am UTC](https://discuss.elastic.co/t/how-to-convert-my-date-field-to-my-local-timezone/152263 "2018-10-16T08:28:21Z")
To be perfectly honest, I'm not really sure to understand my problem properly :grimacing: So here it is. I get events from an Oracle database (with jdbc input). When I query this database (with SQL Developer), I get the… --- ## [Logstash date offset issue and indicies](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588)
**Author:** [@Pirate\_Trkmn](https://discuss.elastic.co/u/Pirate_Trkmn)\ **Replies:** 8\ **Last updated:** [October 16, 2018, 6:50am UTC](https://discuss.elastic.co/t/logstash-date-offset-issue-and-indicies/152588 "2018-10-16T06:50:56Z")
Hi guys i have two problem cant solve. 1.i have log\_time field and want to load it @timestamp with date filter. but there is time difference after i load it to @timestamp example: log\_time is 09:34, but @timestamp is 05… --- ## [Install logstash-filter-json\_encode](https://discuss.elastic.co/t/install-logstash-filter-json-encode/152586)
**Author:** [@deepsing](https://discuss.elastic.co/u/deepsing)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 5:10am UTC](https://discuss.elastic.co/t/install-logstash-filter-json-encode/152586 "2018-10-16T05:10:10Z")
I want to install logstash-filter-json\_encode plugin in offline mode on logstash 6.3.0. Please let me know the steps to follow. What I am doing: Installed plugin using logstash-plugin install logstash-filter-json\_enc… --- ## [Logstash input query](https://discuss.elastic.co/t/logstash-input-query/152519)
**Author:** [@jgitau](https://discuss.elastic.co/u/jgitau)\ **Replies:** 2\ **Last updated:** [October 16, 2018, 4:15am UTC](https://discuss.elastic.co/t/logstash-input-query/152519 "2018-10-16T04:15:00Z")
I have an application that sends events to a url (post), for it to start sending data it requires a password and expects a 'validator' token returned to it from the url's endpoint. how do I handle the validator bit? app… --- ## [Logstash Successfully started but not index in created in Elastic Search](https://discuss.elastic.co/t/logstash-successfully-started-but-not-index-in-created-in-elastic-search/151347)
**Author:** [@Venkatraman\_Muthukri](https://discuss.elastic.co/u/Venkatraman_Muthukri)\ **Replies:** 8\ **Last updated:** [October 16, 2018, 12:25am UTC](https://discuss.elastic.co/t/logstash-successfully-started-but-not-index-in-created-in-elastic-search/151347 "2018-10-16T00:25:05Z")
Hi-All, I am a newbie to logstash, When I tried to dump a csv file data by creating a index through logstash The console output says "\[2018-10-07T16:00:42,081\]\[INFO \]\[logstash.agent \] Successfully started Log… --- ## [Merging events without Unique ID](https://discuss.elastic.co/t/merging-events-without-unique-id/152565)
**Author:** [@ratman](https://discuss.elastic.co/u/ratman)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 10:27pm UTC](https://discuss.elastic.co/t/merging-events-without-unique-id/152565 "2018-10-15T22:27:27Z")
Hello! I am working with some logs and before I build the reports in Kibana, I am perfoming some transformations in Logstash. I am receiveng several types of events, but there are two types of events I need to merge in … --- ## [Logstash asking filter file path](https://discuss.elastic.co/t/logstash-asking-filter-file-path/150538)
**Author:** [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Replies:** 6\ **Last updated:** [October 15, 2018, 9:35pm UTC](https://discuss.elastic.co/t/logstash-asking-filter-file-path/150538 "2018-10-15T21:35:42Z")
Hi , I am using logstash 6.3.2 . Installed from rpm . My issuse is my filter file's path is mentioned in logstash.yml . But again the logstash asking for filter file path . My conf: path.config: /etc/logstash/conf.d… --- ## [Help parsing node stack trace\`](https://discuss.elastic.co/t/help-parsing-node-stack-trace/152558)
**Author:** [@rdooley](https://discuss.elastic.co/u/rdooley)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 9:21pm UTC](https://discuss.elastic.co/t/help-parsing-node-stack-trace/152558 "2018-10-15T21:21:09Z")
The log message below are two node stack traces. I need the multiline message to be seen in Kibana as a single message. Below the logs is the logstash configuration that is kind of working but is cutting off the top mos… --- ## [Parsing Varying Logs with Logstash](https://discuss.elastic.co/t/parsing-varying-logs-with-logstash/152549)
**Author:** [@Ryne\_Keel](https://discuss.elastic.co/u/Ryne_Keel)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 7:18pm UTC](https://discuss.elastic.co/t/parsing-varying-logs-with-logstash/152549 "2018-10-15T19:18:45Z")
Looking for some guidance on where to look for an example of how to do this properly. We have maillogs and postfix logs going into the same log file and I need to learn how to properly differentiate the logs. So far the… --- ## [Logstash showing error when connected to TLS enabled elasticsearch](https://discuss.elastic.co/t/logstash-showing-error-when-connected-to-tls-enabled-elasticsearch/152533)
**Author:** [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Replies:** 1\ **Last updated:** [October 15, 2018, 6:03pm UTC](https://discuss.elastic.co/t/logstash-showing-error-when-connected-to-tls-enabled-elasticsearch/152533 "2018-10-15T18:03:41Z")
Hi All , I am installed ES cluster and Logstash ( Both are 6.3.2) Without TLS in all machines logstash is sending logs to ES . All logs are successfully indexed in ES. ( In this setup xpack is enabled in all ES machi… --- ## [Exception in pipelineworker, the pipeline stopped processing new events](https://discuss.elastic.co/t/exception-in-pipelineworker-the-pipeline-stopped-processing-new-events/149082)
**Author:** [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Replies:** 1\ **Last updated:** [October 15, 2018, 4:45pm UTC](https://discuss.elastic.co/t/exception-in-pipelineworker-the-pipeline-stopped-processing-new-events/149082 "2018-10-15T16:45:38Z")
Hello, I have 50 pipes, and I get this error: \[2018-09-19T10:30:12,824\]\[ERROR\]\[logstash.pipeline \] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration… --- ## [Renaming/deleting inner array hash field with ruby](https://discuss.elastic.co/t/renaming-deleting-inner-array-hash-field-with-ruby/152531)
**Author:** [@jclose](https://discuss.elastic.co/u/jclose)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 4:03pm UTC](https://discuss.elastic.co/t/renaming-deleting-inner-array-hash-field-with-ruby/152531 "2018-10-15T16:03:58Z")
I'm receiving some data, and one of the fields is dynamically named with a number. So it can come out as field\_1 or field\_28 within a hash. This is inside of an array, which is inside of a hash. So, for example, the r… --- ## [Inherits logstash jdbc config](https://discuss.elastic.co/t/inherits-logstash-jdbc-config/152526)
**Author:** [@ggingras](https://discuss.elastic.co/u/ggingras)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 3:40pm UTC](https://discuss.elastic.co/t/inherits-logstash-jdbc-config/152526 "2018-10-15T15:40:09Z")
Hi, I have 6 differents logstash config file to synchronize 6 Oracle tables with their given elastic index. I was wondering if it was possible to inherit common JDBC config so I do not have to repeat myself in all thos… --- ## [Logstach config files contain non ascii characters](https://discuss.elastic.co/t/logstach-config-files-contain-non-ascii-characters/152514)
**Author:** [@kavianand](https://discuss.elastic.co/u/kavianand)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 2:41pm UTC](https://discuss.elastic.co/t/logstach-config-files-contain-non-ascii-characters/152514 "2018-10-15T14:41:27Z")
\[2018-10-15T19:35:24,547\]\[ERROR\]\[logstash.config.sourceloader\] Could not fetch a ll the sources {:exception=\>LogStash::ConfigLoadingError, :message=\>"The followi ng config files contains non-ascii characters but are no… --- ## [Logstash Data base plugin to pull uodated data](https://discuss.elastic.co/t/logstash-data-base-plugin-to-pull-uodated-data/152490)
**Author:** [@chandra0651](https://discuss.elastic.co/u/chandra0651)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 12:14pm UTC](https://discuss.elastic.co/t/logstash-data-base-plugin-to-pull-uodated-data/152490 "2018-10-15T12:14:45Z")
Hello, I have a requirement to pull sql table data in regular interval for last one month,i will be using the primary key of table as elastiseach id to avoid duplicates and update the records but is there a way i can on… --- ## [Issue in renaming fieldname which contains dots](https://discuss.elastic.co/t/issue-in-renaming-fieldname-which-contains-dots/151477)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 6\ **Last updated:** [October 15, 2018, 10:51am UTC](https://discuss.elastic.co/t/issue-in-renaming-fieldname-which-contains-dots/151477 "2018-10-15T10:51:46Z")
Hi All, I have been breaking my head in renaming a field value in logstash and got no solution. Here is my field value result.caller\_id.location.latitude and i'm trying to rename it to lalitude , the renaming filter … --- ## [Oracle DB alert log config error on Logstash 6.4](https://discuss.elastic.co/t/oracle-db-alert-log-config-error-on-logstash-6-4/151995)
**Author:** [@sandeep2](https://discuss.elastic.co/u/sandeep2)\ **Replies:** 7\ **Last updated:** [October 15, 2018, 9:44am UTC](https://discuss.elastic.co/t/oracle-db-alert-log-config-error-on-logstash-6-4/151995 "2018-10-15T09:44:51Z")
Sending Logstash logs to D:/Hexa\_Software/ELK/logstash-6.4.2/logs which is now configured via log4j2.properties \[2018-10-11T11:51:02,241\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file becau… --- ## [Unable to start logstash service Windows server 2012 64 bit](https://discuss.elastic.co/t/unable-to-start-logstash-service-windows-server-2012-64-bit/151967)
**Author:** [@hardikbhatia](https://discuss.elastic.co/u/hardikbhatia)\ **Replies:** 4\ **Last updated:** [October 15, 2018, 9:27am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-service-windows-server-2012-64-bit/151967 "2018-10-15T09:27:45Z")
Hi, I have downloaded and unzipped logstash and tried to install it as service using nssm. Logstash gets successfully installed but when i try to start the service it immediately goes to paused state: I have installe… --- ## [Limit json nested level parse](https://discuss.elastic.co/t/limit-json-nested-level-parse/152454)
**Author:** [@IngZero2](https://discuss.elastic.co/u/IngZero2)\ **Replies:** 0\ **Last updated:** [October 15, 2018, 8:35am UTC](https://discuss.elastic.co/t/limit-json-nested-level-parse/152454 "2018-10-15T08:35:23Z")
Hi all, i've a log like this : log: "{ "time": 1539591679770, "msg": "Connection is OK.", "checkCompleteInfo": { "details": "12345", "value" : 2 } }" After applying json filter to the log field, all the fields … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=666) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=668)