# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=647 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 648 --- ## [Convert event to JSON](https://discuss.elastic.co/t/convert-event-to-json/157126)
**Author:** [@davems](https://discuss.elastic.co/u/davems)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 7:29pm UTC](https://discuss.elastic.co/t/convert-event-to-json/157126 "2018-11-16T19:29:42Z")
I have an event from my input like this: { "Name" =\> "A", "Description" =\> "B" } I need to convert it to JSON for my output, but the event should be a sub node in the JSON - I want it to end up like so: { "event":… --- ## [Docker image doesn't expose ports in custom image](https://discuss.elastic.co/t/docker-image-doesnt-expose-ports-in-custom-image/156479)
**Author:** [@Rafael\_Simonelli](https://discuss.elastic.co/u/Rafael_Simonelli)\ **Replies:** 6\ **Last updated:** [November 16, 2018, 7:02pm UTC](https://discuss.elastic.co/t/docker-image-doesnt-expose-ports-in-custom-image/156479 "2018-11-16T19:02:18Z")
I'm working with a custom image and I want to expose some of container's ports. But when I run my image in Rancher and execute "nmap " from another service in the same host, it reaches the service but no ports are avaibl… --- ## [Convert string format to date using filter in logstash](https://discuss.elastic.co/t/convert-string-format-to-date-using-filter-in-logstash/157108)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 1\ **Last updated:** [November 16, 2018, 5:26pm UTC](https://discuss.elastic.co/t/convert-string-format-to-date-using-filter-in-logstash/157108 "2018-11-16T17:26:15Z")
filter { mutate { add\_field =\> { "docid" =\> "%{projectname}%{systemtypeid}%{username}-%{+dd-MM-YYYY}" "daymonthyear" =\> "%{+dd-MM-YYYY}" } } I get two variable docid (which I am using as my document\_id =\> second … --- ## [Logstash Not processing file in a folder simaltaneously](https://discuss.elastic.co/t/logstash-not-processing-file-in-a-folder-simaltaneously/155898)
**Author:** [@hary1987](https://discuss.elastic.co/u/hary1987)\ **Replies:** 1\ **Last updated:** [November 16, 2018, 4:23pm UTC](https://discuss.elastic.co/t/logstash-not-processing-file-in-a-folder-simaltaneously/155898 "2018-11-16T16:23:44Z")
Hello All, Logstash is not processing all files in a folder simultaneously/parallelly. It is processing one after the other. I am using one configuration file. Example: input { file { path =\> "/inputfiles/record\*.t… --- ## [Parsing a string delimited log with Dissect](https://discuss.elastic.co/t/parsing-a-string-delimited-log-with-dissect/156044)
**Author:** [@tlenya](https://discuss.elastic.co/u/tlenya)\ **Replies:** 1\ **Last updated:** [November 16, 2018, 4:17pm UTC](https://discuss.elastic.co/t/parsing-a-string-delimited-log-with-dissect/156044 "2018-11-16T16:17:10Z")
Good day all, I am trying to parse the below log using Dissect. 1541164838.872000\[20181102 16:20:38.872000\] Module: PXYZ-ModuleHandler/8.1/A/1 Event: some event initialized ID: 4800001 Type: 93669594796392449 Count: 7 … --- ## [Merge two grok](https://discuss.elastic.co/t/merge-two-grok/157083)
**Author:** [@fr3sh](https://discuss.elastic.co/u/fr3sh)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 2:39pm UTC](https://discuss.elastic.co/t/merge-two-grok/157083 "2018-11-16T14:39:27Z")
Hi I would like to grok two different lines and combine them. Assuming this is my lines: \[2016-03-01 08:55:32,073 \] message = TEST \[2016-03-01 09:55:32,073 \] something \[2016-03-01 010:55:32,073 \] telephone\_number =… --- ## [Map Data from Logstash to Elastisearch](https://discuss.elastic.co/t/map-data-from-logstash-to-elastisearch/157075)
**Author:** [@patrick007](https://discuss.elastic.co/u/patrick007)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 2:01pm UTC](https://discuss.elastic.co/t/map-data-from-logstash-to-elastisearch/157075 "2018-11-16T14:01:30Z")
Hi, I am using Logstash to fetch data from Postgres Database. I am able to fetch the data, I want to modify the data so that I can save fetched data in a proper index format to elasticsearch. I want to create json obje… --- ## [Convert a metadata String field into array](https://discuss.elastic.co/t/convert-a-metadata-string-field-into-array/156663)
**Author:** [@Panda8](https://discuss.elastic.co/u/Panda8)\ **Replies:** 1\ **Last updated:** [November 16, 2018, 12:54pm UTC](https://discuss.elastic.co/t/convert-a-metadata-string-field-into-array/156663 "2018-11-16T12:54:21Z")
Hi I have an HTTP JSON Input as follows : { "mapping": { "index": "indexname", "activity": { "action": "script\_update", "fieldList": \["field1","field2"\] }, "document\_type": "\_doc", "document\_id": "1", "routing… --- ## [Please install java in your PATH or set JAVA\_HOME while running logstash](https://discuss.elastic.co/t/please-install-java-in-your-path-or-set-java-home-while-running-logstash/155114)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 5\ **Last updated:** [November 16, 2018, 12:06pm UTC](https://discuss.elastic.co/t/please-install-java-in-your-path-or-set-java-home-while-running-logstash/155114 "2018-11-16T12:06:42Z")
I am facing the error while running logstash " Could not find any executable java binary. Please install java in your PATH or set JAVA\_HOME. " have searched a lot please someone guide me Best Regards Shrikant --- ## [Ipアドレスから国名のみ取得したい](https://discuss.elastic.co/t/ip/156579)
**Author:** [@TK1](https://discuss.elastic.co/u/TK1)\ **Replies:** 1\ **Last updated:** [November 16, 2018, 11:57am UTC](https://discuss.elastic.co/t/ip/156579 "2018-11-16T11:57:11Z")
csvファイルの緯度経度の情報からkibanaで世界地図上に可視化したいのですが上手くいきません． logstashのconfファイルをどのように改善すればよいかアドバイスもらえませんか． http://onthesoup.hatenablog.com/entry/2018/02/11/061500 上記のサイトを参考にしているのですが，ipアドレスから国名のみ取得したいのですが可能でしょうか ELKのversionは以下の通… --- ## [Logstash with stemmer token filter](https://discuss.elastic.co/t/logstash-with-stemmer-token-filter/157031)
**Author:** [@Inaam\_Chhida](https://discuss.elastic.co/u/Inaam_Chhida)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 10:56am UTC](https://discuss.elastic.co/t/logstash-with-stemmer-token-filter/157031 "2018-11-16T10:56:45Z")
I am new with logstash, i want to use stemmer token filter. is it right to do so input { jdbc { # Chaine de connexion à la BDD jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/webpackcli" # User et password de… --- ## [\[ERROR\]\[logstash.outputs.jms \] Unknown setting 'properties' for jms](https://discuss.elastic.co/t/error-logstash-outputs-jms-unknown-setting-properties-for-jms/156942)
**Author:** [@Lindmin](https://discuss.elastic.co/u/Lindmin)\ **Replies:** 1\ **Last updated:** [November 16, 2018, 9:24am UTC](https://discuss.elastic.co/t/error-logstash-outputs-jms-unknown-setting-properties-for-jms/156942 "2018-11-16T09:24:27Z")
Hello, We are currently trying to migrate our log centralisation handle by old logstash 2.3.2 to logstash 6.5.0. But, we encounter an error : \[ERROR\]\[logstash.outputs.jms \] Unknown setting 'properties' for jms th… --- ## [Logstash service failing on start up on Ubuntu 16.04](https://discuss.elastic.co/t/logstash-service-failing-on-start-up-on-ubuntu-16-04/157008)
**Author:** [@ishanip](https://discuss.elastic.co/u/ishanip)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 8:22am UTC](https://discuss.elastic.co/t/logstash-service-failing-on-start-up-on-ubuntu-16-04/157008 "2018-11-16T08:22:31Z")
I am able to run logstash through the command line but when I run it as a service using sudo systemctl start logstash.service, logstash keeps on crashing and I get the following error- Nov 16 07:59:24 nodename-0 logstas… --- ## [Grok Patterns taken from another file](https://discuss.elastic.co/t/grok-patterns-taken-from-another-file/156984)
**Author:** [@Priyaranjan\_Mudliar](https://discuss.elastic.co/u/Priyaranjan_Mudliar)\ **Replies:** 2\ **Last updated:** [November 16, 2018, 6:55am UTC](https://discuss.elastic.co/t/grok-patterns-taken-from-another-file/156984 "2018-11-16T06:55:41Z")
Can i change my logstash configuration file dynamically ? Like i need to have different grok patterns for different file paths. Can i use something like "translate plugin" to change values of grok in the configuration f… --- ## [Logstash sync mysql data to es: can't aggregation two tables' fields with id](https://discuss.elastic.co/t/logstash-sync-mysql-data-to-es-cant-aggregation-two-tables-fields-with-id/156985)
**Author:** [@SHocker-Yu](https://discuss.elastic.co/u/SHocker-Yu)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 6:48am UTC](https://discuss.elastic.co/t/logstash-sync-mysql-data-to-es-cant-aggregation-two-tables-fields-with-id/156985 "2018-11-16T06:48:16Z")
ELK version: 5.6.3. I'm using logstash-jdbc-input plugin to sync two tables data from MySQL to elasticsearch with same index and same type, the conf file is like this: The table d\_product\_set and d\_product\_tag have… --- ## [Using exec and file inputs together](https://discuss.elastic.co/t/using-exec-and-file-inputs-together/156959)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 2\ **Last updated:** [November 16, 2018, 6:38am UTC](https://discuss.elastic.co/t/using-exec-and-file-inputs-together/156959 "2018-11-16T06:38:38Z")
Curious if it's possible to use both these inputs in a single pipeline to have a script executed (that saves a file) and then the file input picks up the file? Something like below. input { exec { command =\> "d:/… --- ## [Logstash Advisory， Leave your English answer, thank you](https://discuss.elastic.co/t/logstash-advisory-leave-your-english-answer-thank-you/156974)
**Author:** [@d0d8268cbc3438684986](https://discuss.elastic.co/u/d0d8268cbc3438684986)\ **Replies:** 1\ **Last updated:** [November 16, 2018, 4:25am UTC](https://discuss.elastic.co/t/logstash-advisory-leave-your-english-answer-thank-you/156974 "2018-11-16T04:25:26Z")
if \[logger\] == "SQL" { aggregate { task\_id =\> "%{taskid}" code =\> "map\['sql\_duration'\] += event.get('duration')" map\_action =\> "update" } } 1.这里的task\_id字段、code字段都是什么意思，怎么获取学习资料 2.map、event.get( ) 这些都是什么意思，怎么获取学习资… --- ## [Is there any way to perform arithmetic calculations on values fetched from csv using logsatsh?](https://discuss.elastic.co/t/is-there-any-way-to-perform-arithmetic-calculations-on-values-fetched-from-csv-using-logsatsh/156979)
**Author:** [@puru](https://discuss.elastic.co/u/puru)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 5:30am UTC](https://discuss.elastic.co/t/is-there-any-way-to-perform-arithmetic-calculations-on-values-fetched-from-csv-using-logsatsh/156979 "2018-11-16T05:30:28Z")
hello, i need to calculate difference between two fields(rows) of the same column and store the result in the next column on the values fetched from csv. Is it possible through logstash, if so how? Thanks in advance.. --- ## [Recursively fetch content](https://discuss.elastic.co/t/recursively-fetch-content/156976)
**Author:** [@Karanth](https://discuss.elastic.co/u/Karanth)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 4:40am UTC](https://discuss.elastic.co/t/recursively-fetch-content/156976 "2018-11-16T04:40:06Z")
I have a log that has below pattern name = name1 Some message about name1 name = name1 Some more message about name1 name = name2 This is about name2 name = name2 This is also about name2 Can someone please tell me … --- ## [Grok out hex value in URL 6.4.0](https://discuss.elastic.co/t/grok-out-hex-value-in-url-6-4-0/156957)
**Author:** [@grazer](https://discuss.elastic.co/u/grazer)\ **Replies:** 0\ **Last updated:** [November 16, 2018, 12:51am UTC](https://discuss.elastic.co/t/grok-out-hex-value-in-url-6-4-0/156957 "2018-11-16T00:51:32Z")
I'm very new to this, however very keen to learn... I have grokked out the IIS log file that contains exchange active sync data that i want to draw out.. I've tried using urlparam and urldecode with no real success. th… --- ## [Specifing the size of data to load to memory](https://discuss.elastic.co/t/specifing-the-size-of-data-to-load-to-memory/156833)
**Author:** [@safaa](https://discuss.elastic.co/u/safaa)\ **Replies:** 0\ **Last updated:** [November 15, 2018, 10:35am UTC](https://discuss.elastic.co/t/specifing-the-size-of-data-to-load-to-memory/156833 "2018-11-15T10:35:03Z")
I have an xml file , and i want to parse it to logstash. The issue is that when i have a large xml file (1Go) , the memory crashes because , the logstash try to load the whole xml file in memory . Is there a way in logst… --- ## [Parsing nested arrays in json objects](https://discuss.elastic.co/t/parsing-nested-arrays-in-json-objects/156944)
**Author:** [@alexmartin](https://discuss.elastic.co/u/alexmartin)\ **Replies:** 0\ **Last updated:** [November 15, 2018, 9:38pm UTC](https://discuss.elastic.co/t/parsing-nested-arrays-in-json-objects/156944 "2018-11-15T21:38:12Z")
I'm trying to parse lines of json from Filebeat log files to Logstash to Elasticstack. The problem I'm having is that some of these lines contains nested arrays in json objects. Here's a sample (prettified): { "Even… --- ## [Separate json array into new fields](https://discuss.elastic.co/t/separate-json-array-into-new-fields/156947)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 0\ **Last updated:** [November 15, 2018, 9:56pm UTC](https://discuss.elastic.co/t/separate-json-array-into-new-fields/156947 "2018-11-15T21:56:23Z")
Hello, I have a json array into a field like this: https://pastebin.com/ZtYUQQxX I would like to split this objects each into it's own field. I have tried with split filter but that would not do the trick. Any suggesti… --- ## [SAP Netweaver logs on Kibana](https://discuss.elastic.co/t/sap-netweaver-logs-on-kibana/156943)
**Author:** [@picoroma](https://discuss.elastic.co/u/picoroma)\ **Replies:** 0\ **Last updated:** [November 15, 2018, 9:32pm UTC](https://discuss.elastic.co/t/sap-netweaver-logs-on-kibana/156943 "2018-11-15T21:32:33Z")
Hi! Does anybody tried to push SAP log file into logstash (via Beats) and then to ES and KIBANA ? If yes with kind of logstash configuration have used? REgards P. --- ## [Unable to index csv data](https://discuss.elastic.co/t/unable-to-index-csv-data/156442)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 7\ **Last updated:** [November 15, 2018, 8:24pm UTC](https://discuss.elastic.co/t/unable-to-index-csv-data/156442 "2018-11-15T20:24:41Z")
Hi All, I have been trying to upload csv data using logstash, but was unable to index the data and getting error like characterset miss match, I was able to identify these warnings are occurring only for the field valu… --- ## [Logstash on Windows - add reading Java path from registry?](https://discuss.elastic.co/t/logstash-on-windows-add-reading-java-path-from-registry/156934)
**Author:** [@gjbdebug](https://discuss.elastic.co/u/gjbdebug)\ **Replies:** 0\ **Last updated:** [November 15, 2018, 7:46pm UTC](https://discuss.elastic.co/t/logstash-on-windows-add-reading-java-path-from-registry/156934 "2018-11-15T19:46:13Z")
In setup.bat, has adding code to check the registry for setting the Java path been considered when the existing logic cannot can't find the deployment? I'm specifically referring to scenarios where installed JRE/JDK auto… --- ## [Unable to parse Meraki MX](https://discuss.elastic.co/t/unable-to-parse-meraki-mx/156712)
**Author:** [@Frank\_Shearer](https://discuss.elastic.co/u/Frank_Shearer)\ **Replies:** 1\ **Last updated:** [November 15, 2018, 6:37pm UTC](https://discuss.elastic.co/t/unable-to-parse-meraki-mx/156712 "2018-11-15T18:37:17Z")
Hi All, I'm having an issue maybe someone can help me out with, I am trying to parse Meraki flows messages which look like this: my input conf is this: and my filter conf is this: I continually get this in s… --- ## [Troubles with setting @timestamp](https://discuss.elastic.co/t/troubles-with-setting-timestamp/156906)
**Author:** [@Roma1](https://discuss.elastic.co/u/Roma1)\ **Replies:** 0\ **Last updated:** [November 15, 2018, 4:36pm UTC](https://discuss.elastic.co/t/troubles-with-setting-timestamp/156906 "2018-11-15T16:36:31Z")
In my logstash config i have this filter: filter { json { source =\> "message" target =\> "json\_data" } date { match =\> \["log\_timestamp", "yyyy-MM-dd HH:mm:ss.SSS"\] target = "@timestamp" } } log timestamp: {"log… --- ## [Take events from one pipeline and forward to another pipeline : Pipeline to Pipeline Communication](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 6\ **Last updated:** [November 15, 2018, 4:36pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903 "2018-11-15T16:36:19Z")
Hi All I have a scenario, wherein i need to read events from one data source and transfer the output of this pipeline to other pipeline as a sprintf like %{abc}, please let me know how to achieve this Here is my exampl… --- ## [ECE 2.0 / Deployment version 6.5 / Cloud ID netflow setup](https://discuss.elastic.co/t/ece-2-0-deployment-version-6-5-cloud-id-netflow-setup/155237)
**Author:** [@xersist](https://discuss.elastic.co/u/xersist)\ **Replies:** 1\ **Last updated:** [November 15, 2018, 4:26pm UTC](https://discuss.elastic.co/t/ece-2-0-deployment-version-6-5-cloud-id-netflow-setup/155237 "2018-11-15T16:26:52Z")
Hello, I hope the issue is something simple I'm just overlooking. I will hopefully give all details needed right away. :slight\_smile: Environment: FLAWLESS install of the instructions listed for the Large Deployment. C… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=646) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=648)