# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=627 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 628 --- ## [Symbolic Link File on Linux not working as Logstash File Input](https://discuss.elastic.co/t/symbolic-link-file-on-linux-not-working-as-logstash-file-input/162092)
**Author:** [@Wenqing\_Xu](https://discuss.elastic.co/u/Wenqing_Xu)\ **Replies:** 0\ **Last updated:** [December 26, 2018, 3:47am UTC](https://discuss.elastic.co/t/symbolic-link-file-on-linux-not-working-as-logstash-file-input/162092 "2018-12-26T03:47:36Z")
Hi, I am currently running a logstash in docker which use file as an input, and output to a elastic search; configuration as below: input { file { path =\> "/data/log/sandbox/primary/logic/logic-data/logic-data\_curren… --- ## [Jmx plugin and aggregate](https://discuss.elastic.co/t/jmx-plugin-and-aggregate/162083)
**Author:** [@ncasaux](https://discuss.elastic.co/u/ncasaux)\ **Replies:** 0\ **Last updated:** [December 25, 2018, 8:05pm UTC](https://discuss.elastic.co/t/jmx-plugin-and-aggregate/162083 "2018-12-25T20:05:45Z")
Hello, I try to aggregate some events coming from the jmx plugin, but I never managed to aggregate anything. As if that was not possible... Here the code I have so far: filter { grok { match =\> { "metric\_path" =\>… --- ## [Nested Json Parsing](https://discuss.elastic.co/t/nested-json-parsing/161997)
**Author:** [@Veena\_Kulkarni](https://discuss.elastic.co/u/Veena_Kulkarni)\ **Replies:** 2\ **Last updated:** [December 25, 2018, 7:22am UTC](https://discuss.elastic.co/t/nested-json-parsing/161997 "2018-12-25T07:22:23Z")
Need help in parsing nested json . Below is sample json : {"debug\_level":"ERROR","debug\_timestamp":"2018-12-21 05:15:57,559","debug\_thread":"ScalaTest","debug\_file":"Audit1.scala", "debug\_line":"27","debug\_message":{"Jo… --- ## [Grok regular expression doesn't contain a word not working](https://discuss.elastic.co/t/grok-regular-expression-doesnt-contain-a-word-not-working/162019)
**Author:** [@XanxusVer.V.R](https://discuss.elastic.co/u/XanxusVer.V.R)\ **Replies:** 2\ **Last updated:** [December 25, 2018, 6:35am UTC](https://discuss.elastic.co/t/grok-regular-expression-doesnt-contain-a-word-not-working/162019 "2018-12-25T06:35:52Z")
I have the following log： 8.8.8.8 - localhost \[24/Dec/2018:02:53:15 +0800\] "GET /test/ui/favicons/favicon.ico HTTP/1.1" 304 0 "https://www.google.com.tw/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10\_12\_6) AppleWebKit/537… --- ## [Using Logstash to transfer data from mysql to elasticsearch about TIME](https://discuss.elastic.co/t/using-logstash-to-transfer-data-from-mysql-to-elasticsearch-about-time/161995)
**Author:** [@lip](https://discuss.elastic.co/u/lip)\ **Replies:** 1\ **Last updated:** [December 24, 2018, 11:59am UTC](https://discuss.elastic.co/t/using-logstash-to-transfer-data-from-mysql-to-elasticsearch-about-time/161995 "2018-12-24T11:59:35Z")
Using elasticsearch 6.3.1 logstash 6.3.1 I m using logstash to transfer data from mysql to elasticsearch But i realized that the time field is confused.There is a field named 'gmt\_create' , it s '2018-07-13 16:50:04' i… --- ## [Logstash : Filter Error message from Logs](https://discuss.elastic.co/t/logstash-filter-error-message-from-logs/161857)
**Author:** [@Moulika\_Magatapalli1](https://discuss.elastic.co/u/Moulika_Magatapalli1)\ **Replies:** 1\ **Last updated:** [December 24, 2018, 4:29am UTC](https://discuss.elastic.co/t/logstash-filter-error-message-from-logs/161857 "2018-12-24T04:29:42Z")
Hi , I am using Logstash to parse and filter the data. The input data looks something like this: ------------------------------------------------------ Environment File - 12/07/2018 09:31:19 ----------------------------… --- ## [Logstash high memroy consumption and possibly a leak](https://discuss.elastic.co/t/logstash-high-memroy-consumption-and-possibly-a-leak/161598)
**Author:** [@ray20](https://discuss.elastic.co/u/ray20)\ **Replies:** 1\ **Last updated:** [December 24, 2018, 3:24am UTC](https://discuss.elastic.co/t/logstash-high-memroy-consumption-and-possibly-a-leak/161598 "2018-12-24T03:24:46Z")
We run logstash 6.3.1 with a single pipeline of beats input plugin, json and grok filters, and S3 output plugin. It runs in Docker and is configured with 2GB for Xms and Xmx. The node itself has 4GB total memory. When it… --- ## [No events in pipeline view](https://discuss.elastic.co/t/no-events-in-pipeline-view/161958)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 0\ **Last updated:** [December 23, 2018, 8:30pm UTC](https://discuss.elastic.co/t/no-events-in-pipeline-view/161958 "2018-12-23T20:30:47Z")
Hi, I am using LS 6.5.4 and have set up x-pack monitoring with the free license I can see the the events / second in the pipeline overview, but in the detail view they are 0. How do I solve this? Thanks in advance, an… --- ## [Unable to get total sql duration from aggregate filter example](https://discuss.elastic.co/t/unable-to-get-total-sql-duration-from-aggregate-filter-example/161890)
**Author:** [@Sanjeev7](https://discuss.elastic.co/u/Sanjeev7)\ **Replies:** 1\ **Last updated:** [December 23, 2018, 5:27pm UTC](https://discuss.elastic.co/t/unable-to-get-total-sql-duration-from-aggregate-filter-example/161890 "2018-12-23T17:27:34Z")
just run below aggregate filter url example unable to get total sql duration please find below output logs as i have same requirement . Please let me know if i missed anything https://www.elastic.co/guide/en/logstash/cu… --- ## [Logstash crashing on DNS connection timeout](https://discuss.elastic.co/t/logstash-crashing-on-dns-connection-timeout/161954)
**Author:** [@kernelpanic](https://discuss.elastic.co/u/kernelpanic)\ **Replies:** 0\ **Last updated:** [December 23, 2018, 3:50pm UTC](https://discuss.elastic.co/t/logstash-crashing-on-dns-connection-timeout/161954 "2018-12-23T15:50:24Z")
Hello, I'm using Logstash 6.3.2 with the latest version of the DNS plugin. I set up Elastiflow to ingest Netflow logs and enabled the DNS lookup option to convert the IP addresses into host names. This appeared to work f… --- ## [Twitter input plugin: only use keywords from specific follows](https://discuss.elastic.co/t/twitter-input-plugin-only-use-keywords-from-specific-follows/161942)
**Author:** [@DaniEs](https://discuss.elastic.co/u/DaniEs)\ **Replies:** 0\ **Last updated:** [December 23, 2018, 7:45am UTC](https://discuss.elastic.co/t/twitter-input-plugin-only-use-keywords-from-specific-follows/161942 "2018-12-23T07:45:34Z")
Hi! I'm new to the forum and almost new using ELK and I have some doubts about logstash twitter plugin As far as I could test every option is exclusive, they use OR, I mean if I use the option follows with some twit… --- ## [Problem with heartbeat mapping](https://discuss.elastic.co/t/problem-with-heartbeat-mapping/161947)
**Author:** [@rhombus00](https://discuss.elastic.co/u/rhombus00)\ **Replies:** 0\ **Last updated:** [December 23, 2018, 10:48am UTC](https://discuss.elastic.co/t/problem-with-heartbeat-mapping/161947 "2018-12-23T10:48:26Z")
My goal is to monitor the health of a node so my F5 LB can determine which nodes in the cluster are healthy before sending logs to them. I'm trying to sett up the heartbeat according to F5 High Speed Logging with Elastic… --- ## [Merging multiple logstash output document into one by a common field](https://discuss.elastic.co/t/merging-multiple-logstash-output-document-into-one-by-a-common-field/161944)
**Author:** [@abhishek021](https://discuss.elastic.co/u/abhishek021)\ **Replies:** 0\ **Last updated:** [December 23, 2018, 8:14am UTC](https://discuss.elastic.co/t/merging-multiple-logstash-output-document-into-one-by-a-common-field/161944 "2018-12-23T08:14:21Z")
Hi, I'm a beginner to logstash. I have created multiple documents based on some grok pattern for a particular log event. Now, before passing documents to elastic search I want to merge those documents having common fie… --- ## [Index name with local system date not using message @timestamp](https://discuss.elastic.co/t/index-name-with-local-system-date-not-using-message-timestamp/160981)
**Author:** [@noobtoelks](https://discuss.elastic.co/u/noobtoelks)\ **Replies:** 1\ **Last updated:** [December 22, 2018, 9:01pm UTC](https://discuss.elastic.co/t/index-name-with-local-system-date-not-using-message-timestamp/160981 "2018-12-22T21:01:27Z")
Hi All, I am new to ELK stack. I have logstash 6.5 setup using link and configuration below. The question I have is how do I format the index name with either receive\_at or current OS date? The reasons are because the… --- ## [Do we have any example for enabling log for a particular component as per docs?](https://discuss.elastic.co/t/do-we-have-any-example-for-enabling-log-for-a-particular-component-as-per-docs/161929)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 0\ **Last updated:** [December 22, 2018, 7:17pm UTC](https://discuss.elastic.co/t/do-we-have-any-example-for-enabling-log-for-a-particular-component-as-per-docs/161929 "2018-12-22T19:17:41Z")
Hi All I am looking for logs only for output, as per documentation it says we can enable logs for module,plugins, component. How can i achieve this if you are debugging issues with Elasticsearch Output, you can incr… --- ## [Super-noob needs help with a mapping error](https://discuss.elastic.co/t/super-noob-needs-help-with-a-mapping-error/161900)
**Author:** [@Manzobar\_Gregarin](https://discuss.elastic.co/u/Manzobar_Gregarin)\ **Replies:** 2\ **Last updated:** [December 22, 2018, 3:03pm UTC](https://discuss.elastic.co/t/super-noob-needs-help-with-a-mapping-error/161900 "2018-12-22T15:03:29Z")
Hey everyone, I have an issue indexing a log message. It's a mapping error. "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field \[messageObject\] of type \[text\]", "caused\_by"=\>{"… --- ## [How to view SNMP Traps with Logstash /Filebeat configuration](https://discuss.elastic.co/t/how-to-view-snmp-traps-with-logstash-filebeat-configuration/161919)
**Author:** [@JeyaNachiaban](https://discuss.elastic.co/u/JeyaNachiaban)\ **Replies:** 0\ **Last updated:** [December 22, 2018, 2:03pm UTC](https://discuss.elastic.co/t/how-to-view-snmp-traps-with-logstash-filebeat-configuration/161919 "2018-12-22T14:03:19Z")
I am new to ELK. I want to get the snmp traps through filebeat/logstash and show it in Kibana. Can you provide the documentation/procedure to do that? Configuration I tried in logstash. input { snmptrap { type =\> "s… --- ## [Kibana 4.6 won't let me add a new index pattern to import very basic json file](https://discuss.elastic.co/t/kibana-4-6-wont-let-me-add-a-new-index-pattern-to-import-very-basic-json-file/161897)
**Author:** [@kimr](https://discuss.elastic.co/u/kimr)\ **Replies:** 1\ **Last updated:** [December 22, 2018, 1:11am UTC](https://discuss.elastic.co/t/kibana-4-6-wont-let-me-add-a-new-index-pattern-to-import-very-basic-json-file/161897 "2018-12-22T01:11:03Z")
I have a very basic config file to import a json file into logstash and view in Kibana. Below indent preformatted text by 4 spaces input { file {path =\> "/labs/dtf/logs/level1.json" } } filter { json { source =\> "m… --- ## [How to convert SQL row to array of objects in logstash](https://discuss.elastic.co/t/how-to-convert-sql-row-to-array-of-objects-in-logstash/161888)
**Author:** [@abhiz](https://discuss.elastic.co/u/abhiz)\ **Replies:** 0\ **Last updated:** [December 21, 2018, 6:53pm UTC](https://discuss.elastic.co/t/how-to-convert-sql-row-to-array-of-objects-in-logstash/161888 "2018-12-21T18:53:18Z")
Hi..Need a help to convert sql row to array of objects in logstash. Sample output from sql - columnA - 1,3,5,7 columnB - s,c,e,p Output needed : { "details": \[ { "id": 1, "stat": s }, { "id": 3, "stat": c … --- ## [JMX plugin, grok, add\_field and type](https://discuss.elastic.co/t/jmx-plugin-grok-add-field-and-type/161672)
**Author:** [@ncasaux](https://discuss.elastic.co/u/ncasaux)\ **Replies:** 1\ **Last updated:** [December 21, 2018, 6:26pm UTC](https://discuss.elastic.co/t/jmx-plugin-grok-add-field-and-type/161672 "2018-12-21T18:26:42Z")
Hello, I managed to set-up a successfull pipeline with Logstash and the jmx plugin. filter { grok { match =\> { "metric\_path" =\> "(?\\[^\\.\]\*)\\.(?\.\*)\\.(?\\[^\\.\]\*)" } } } With this, I … --- ## [How does sincedb\_clean\_after in file{} work?](https://discuss.elastic.co/t/how-does-sincedb-clean-after-in-file-work/161471)
**Author:** [@espenk](https://discuss.elastic.co/u/espenk)\ **Replies:** 2\ **Last updated:** [December 21, 2018, 3:41pm UTC](https://discuss.elastic.co/t/how-does-sincedb-clean-after-in-file-work/161471 "2018-12-21T15:41:22Z")
Hi. I have been reading https://www.elastic.co/guide/en/logstash/master/plugins-inputs-file.html for a while, and stumbled upon the following statement: "Sincedb records can now be expired meaning that read positions of … --- ## [Export Index with Logstash?](https://discuss.elastic.co/t/export-index-with-logstash/161872)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 0\ **Last updated:** [December 21, 2018, 3:36pm UTC](https://discuss.elastic.co/t/export-index-with-logstash/161872 "2018-12-21T15:36:14Z")
Hi, I am trying to export an index to a csv file with the following config: input { elasticsearch { hosts =\> \[ "localhost:9200" \] query =\> '{ "query": { "query\_string": { "query": "\*" } } }' index =\> "ind… --- ## [Logstash Timezone Bug](https://discuss.elastic.co/t/logstash-timezone-bug/161849)
**Author:** [@Nelson\_Ferreira\_Juni](https://discuss.elastic.co/u/Nelson_Ferreira_Juni)\ **Replies:** 4\ **Last updated:** [December 21, 2018, 1:22pm UTC](https://discuss.elastic.co/t/logstash-timezone-bug/161849 "2018-12-21T13:22:16Z")
I'm running logstash 6.5 inside a container (sebp/elk:650), this container is in the UTC timezone. The host machine is in the America/New\_York timezone, this is my Logstash configuration for the timezone: date { match… --- ## [I need a very important help - need to read a DB2 database](https://discuss.elastic.co/t/i-need-a-very-important-help-need-to-read-a-db2-database/161691)
**Author:** [@angelogabeira](https://discuss.elastic.co/u/angelogabeira)\ **Replies:** 8\ **Last updated:** [December 21, 2018, 11:22am UTC](https://discuss.elastic.co/t/i-need-a-very-important-help-need-to-read-a-db2-database/161691 "2018-12-21T11:22:48Z")
Need to read a DB2 database I'm using this code: input { jdbc { jdbc\_connection\_string =\> "jdbc:db2://xxxxx.yyyyy.com.br:50100/DB2XXX" jdbc\_user =\> "DB2YYY" jdbc\_password =\> "010101" jdbc\_driver\_library =\> "/home/… --- ## [Wrap Json message in filter](https://discuss.elastic.co/t/wrap-json-message-in-filter/161549)
**Author:** [@mickt](https://discuss.elastic.co/u/mickt)\ **Replies:** 1\ **Last updated:** [December 21, 2018, 11:02am UTC](https://discuss.elastic.co/t/wrap-json-message-in-filter/161549 "2018-12-21T11:02:15Z")
I have log entries like {"timeMillis":1545209409013,"thread":"ajp-bio-9003-exec-10","level":"INFO","loggerName":"com.company.Class","message":"Timeslot ID: null","endOfBatch":false,"loggerFqcn":"com.company.MyLogger","t… --- ## [Support for source\_ip\_fieldname in all network input plugins](https://discuss.elastic.co/t/support-for-source-ip-fieldname-in-all-network-input-plugins/161798)
**Author:** [@dss](https://discuss.elastic.co/u/dss)\ **Replies:** 0\ **Last updated:** [December 21, 2018, 7:34am UTC](https://discuss.elastic.co/t/support-for-source-ip-fieldname-in-all-network-input-plugins/161798 "2018-12-21T07:34:39Z")
The udp input plugin supports an option source\_ip\_fieldname with a default value of host and described as "the name of the field where the source IP address will be stored." I'd like to begin using ECS fields everywhere… --- ## [How to get switch, storage fan, temp status info?](https://discuss.elastic.co/t/how-to-get-switch-storage-fan-temp-status-info/161779)
**Author:** [@dh77hd](https://discuss.elastic.co/u/dh77hd)\ **Replies:** 5\ **Last updated:** [December 21, 2018, 5:40am UTC](https://discuss.elastic.co/t/how-to-get-switch-storage-fan-temp-status-info/161779 "2018-12-21T05:40:57Z")
Hi. I want to get switch, storage fan, temp, power status. How to get that information? That infor is like below. Fan: ------------------------------------------------------ Fan Model Hw … --- ## [Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, ,, \]](https://discuss.elastic.co/t/failed-to-execute-action-action-logstash-create-pipeline-id-main-exception-logstash-configurationerror-message-expected-one-of/161676)
**Author:** [@wusong](https://discuss.elastic.co/u/wusong)\ **Replies:** 3\ **Last updated:** [December 21, 2018, 1:54am UTC](https://discuss.elastic.co/t/failed-to-execute-action-action-logstash-create-pipeline-id-main-exception-logstash-configurationerror-message-expected-one-of/161676 "2018-12-21T01:54:32Z")
I run cmd .\\logstash -e 'input{stdin{}} output{ elasticsearch{ hosts=\>\["127.0.0.1:9200"\] } }' ; then have following errors. \[2018-12-20T18:47:55,667\]\[ERROR\]\[logstash.agent \] Failed to execute action {:actio… --- ## [Logstash 6.5.3 inode reuse problem](https://discuss.elastic.co/t/logstash-6-5-3-inode-reuse-problem/161767)
**Author:** [@jinyoon](https://discuss.elastic.co/u/jinyoon)\ **Replies:** 0\ **Last updated:** [December 21, 2018, 1:02am UTC](https://discuss.elastic.co/t/logstash-6-5-3-inode-reuse-problem/161767 "2018-12-21T01:02:26Z")
Hi. I use logback (file appender, time-based policy, hourly) and logstash send to kafka from logback result. But when file is rolling, logstash miss some first lines. I guess that inode in sincedb file is reuse. Plea… --- ## [Do multiple jdbc input run simultaneously or sequentially?](https://discuss.elastic.co/t/do-multiple-jdbc-input-run-simultaneously-or-sequentially/160197)
**Author:** [@keysersozee](https://discuss.elastic.co/u/keysersozee)\ **Replies:** 3\ **Last updated:** [December 20, 2018, 11:52pm UTC](https://discuss.elastic.co/t/do-multiple-jdbc-input-run-simultaneously-or-sequentially/160197 "2018-12-20T23:52:29Z")
Hello, I need to query multiple database instances and I wanted to know when defining multiple jdbc input plugins in just 1 pipeline is the exact same thing (technically) as defining multiple pipelines with 1 jdbc input… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=626) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=628)