# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=380 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 381 --- ## [Cleaning up after installing a Logstash plugin ( 200Mb + of dependencies downloaded)](https://discuss.elastic.co/t/cleaning-up-after-installing-a-logstash-plugin-200mb-of-dependencies-downloaded/226266)
**Author:** [@deslauriersp](https://discuss.elastic.co/u/deslauriersp)\ **Replies:** 0\ **Last updated:** [April 2, 2020, 6:11pm UTC](https://discuss.elastic.co/t/cleaning-up-after-installing-a-logstash-plugin-200mb-of-dependencies-downloaded/226266 "2020-04-02T18:11:21Z")
Its seems that hundreds of dependencies are downloaded in order to install a logstash plugin. I installed the example Java filter plugin, which is very small gem (2mb), and it downloaded more than 200 Mb of dependencies… --- ## [Log collection concetp](https://discuss.elastic.co/t/log-collection-concetp/226243)
**Author:** [@Ernest\_Lachowicz](https://discuss.elastic.co/u/Ernest_Lachowicz)\ **Replies:** 0\ **Last updated:** [April 2, 2020, 3:39pm UTC](https://discuss.elastic.co/t/log-collection-concetp/226243 "2020-04-02T15:39:17Z")
Hi, I have two servers: SERVER1 with logs: /tmp/testname/server1.log SERVER2 with logs /log/server2.log I start a functional test by Jenkins. Both servers create logs during the test. On servers are run several d… --- ## [Multiline message problem](https://discuss.elastic.co/t/multiline-message-problem/226183)
**Author:** [@Rutger2000](https://discuss.elastic.co/u/Rutger2000)\ **Replies:** 2\ **Last updated:** [April 2, 2020, 2:38pm UTC](https://discuss.elastic.co/t/multiline-message-problem/226183 "2020-04-02T14:38:28Z")
Hello, I'm parsing multiline messages to elastic using Logstash. 99,9% of the messages are parsed correctly, but a small part is not. As far as I can see I'm not getting any errors. The messages that are not parsed corr… --- ## [Basic logstash not showing events](https://discuss.elastic.co/t/basic-logstash-not-showing-events/226230)
**Author:** [@lsambolino](https://discuss.elastic.co/u/lsambolino)\ **Replies:** 0\ **Last updated:** [April 2, 2020, 2:27pm UTC](https://discuss.elastic.co/t/basic-logstash-not-showing-events/226230 "2020-04-02T14:27:23Z")
Hi, I just installed Elasticsearch + Logstash 7.6.2 on a debian server. I followed the step-by-step instructions regarding elasticsearch basic configuration and I am now configuring logstash. When I arrive at the Parsi… --- ## [IP datatype field cannot be created](https://discuss.elastic.co/t/ip-datatype-field-cannot-be-created/223368)
**Author:** [@sstnt783](https://discuss.elastic.co/u/sstnt783)\ **Replies:** 2\ **Last updated:** [April 2, 2020, 1:16pm UTC](https://discuss.elastic.co/t/ip-datatype-field-cannot-be-created/223368 "2020-04-02T13:16:41Z")
Hi, i am trying to use map field as IP datatype but ending with getting string. First i tried it (as i did for NUMBER data type) do it just by GROK parsing pattern that when in log, the value would be mapped by %{IP:sor… --- ## [Stack monitoring when using Pipeline to Pipeline communication](https://discuss.elastic.co/t/stack-monitoring-when-using-pipeline-to-pipeline-communication/225861)
**Author:** [@niklaskurvinen](https://discuss.elastic.co/u/niklaskurvinen)\ **Replies:** 2\ **Last updated:** [April 2, 2020, 1:13pm UTC](https://discuss.elastic.co/t/stack-monitoring-when-using-pipeline-to-pipeline-communication/225861 "2020-04-02T13:13:21Z")
Hi all! Stack version: 7.6.0 Using the new remote monitoring (monitoring the elastic stack using metricbeat) I bumped into something that I can't find a solution for. In the Stack Monitoring view in Kibana I now, afte… --- ## [How to install logstash-output-influxdb offline in logstash](https://discuss.elastic.co/t/how-to-install-logstash-output-influxdb-offline-in-logstash/226193)
**Author:** [@Prashant\_Jain](https://discuss.elastic.co/u/Prashant_Jain)\ **Replies:** 1\ **Last updated:** [April 2, 2020, 11:51am UTC](https://discuss.elastic.co/t/how-to-install-logstash-output-influxdb-offline-in-logstash/226193 "2020-04-02T11:51:51Z")
Hi, I have a setup of logstash on a server which doesn't have internet connection. I need the logstash-output-influxdb plugin. How can i install it there? --- ## [Multiple http filters](https://discuss.elastic.co/t/multiple-http-filters/226181)
**Author:** [@akhilsharma.in](https://discuss.elastic.co/u/akhilsharma.in)\ **Replies:** 1\ **Last updated:** [April 2, 2020, 11:39am UTC](https://discuss.elastic.co/t/multiple-http-filters/226181 "2020-04-02T11:39:37Z")
We've multiple conf files which have http filters in them. And they are running simultaneously at a time. Running multiple http filters at a time seems to give us problems. Please help us to understand if its the behavio… --- ## [Incremental data](https://discuss.elastic.co/t/incremental-data/223688)
**Author:** [@chelseia](https://discuss.elastic.co/u/chelseia)\ **Replies:** 3\ **Last updated:** [April 2, 2020, 9:04am UTC](https://discuss.elastic.co/t/incremental-data/223688 "2020-04-02T09:04:20Z")
I have some records already existed in table.I have modified some records after pushed into the table.but the modified data was not reflected in elastic search.how to achieve this? --- ## [Logstash enable ssl to connect ElasticSearch but cannot run logstash and get error](https://discuss.elastic.co/t/logstash-enable-ssl-to-connect-elasticsearch-but-cannot-run-logstash-and-get-error/226037)
**Author:** [@chauncy1](https://discuss.elastic.co/u/chauncy1)\ **Replies:** 12\ **Last updated:** [April 2, 2020, 8:53am UTC](https://discuss.elastic.co/t/logstash-enable-ssl-to-connect-elasticsearch-but-cannot-run-logstash-and-get-error/226037 "2020-04-02T08:53:49Z")
Hi， I have configured the x\_pack of ElasticSearch and SSL of ElasticSearch and Kibana. But when I try to config logstash to connect ElasticSearch with SSL, I got some error. Version: Logstash7.3.2, ElasticSearch7.3.2,… --- ## [Logstash - Removing duplicity from array](https://discuss.elastic.co/t/logstash-removing-duplicity-from-array/226164)
**Author:** [@vasek](https://discuss.elastic.co/u/vasek)\ **Replies:** 0\ **Last updated:** [April 2, 2020, 7:12am UTC](https://discuss.elastic.co/t/logstash-removing-duplicity-from-array/226164 "2020-04-02T07:12:24Z")
Hello, I would like to share the experience with removing duplicity from array in Logstash. I used this approach before: I was checking whether value was already being present in array. if ( \[host\]\[name\] and (\[host… --- ## [How to format the xml logs into json format in logstash](https://discuss.elastic.co/t/how-to-format-the-xml-logs-into-json-format-in-logstash/224820)
**Author:** [@poddraj](https://discuss.elastic.co/u/poddraj)\ **Replies:** 11\ **Last updated:** [April 2, 2020, 7:46am UTC](https://discuss.elastic.co/t/how-to-format-the-xml-logs-into-json-format-in-logstash/224820 "2020-04-02T07:46:36Z")
My Log format is like below.. It is c++ app logs where we have request & response xmls within the logs. I am interested to convert those xmls into json format and save it in output file. 12/20 00:00:23.00 dte.cpp(28367)… --- ## [How to parse array\[\] in json](https://discuss.elastic.co/t/how-to-parse-array-in-json/225998)
**Author:** [@spc1jh](https://discuss.elastic.co/u/spc1jh)\ **Replies:** 2\ **Last updated:** [April 2, 2020, 7:39am UTC](https://discuss.elastic.co/t/how-to-parse-array-in-json/225998 "2020-04-02T07:39:08Z")
Hi all, I want to monitir aws waf2 logs with elastic search. I tried to insert data in elasticsearch with logstash. I faced some problems. shortly below is the json format data { "nonTerminatingMatchingRules" : \[… --- ## [Parse array of array in Logstash](https://discuss.elastic.co/t/parse-array-of-array-in-logstash/224765)
**Author:** [@sandy1139](https://discuss.elastic.co/u/sandy1139)\ **Replies:** 1\ **Last updated:** [March 24, 2020, 12:48pm UTC](https://discuss.elastic.co/t/parse-array-of-array-in-logstash/224765 "2020-03-24T12:48:51Z")
I have some logs like below which is an array within which there are two more arrays, I need to split them into individual events. \[{"nodes":\[{"id":"node1","label":\["Label1", "Label2"\],"properties":"property1"},{"id":"n… --- ## [Separate fields and values in logstash](https://discuss.elastic.co/t/separate-fields-and-values-in-logstash/226155)
**Author:** [@newbie001](https://discuss.elastic.co/u/newbie001)\ **Replies:** 0\ **Last updated:** [April 2, 2020, 5:59am UTC](https://discuss.elastic.co/t/separate-fields-and-values-in-logstash/226155 "2020-04-02T05:59:19Z")
Hi - I need help on how I can separate the fields and values via logstash. I'm not sure if grok filter is applicable on this as it only separates the values of the fields. Please see below my sample input and expected ou… --- ## [Logstash ODX input File - unconventional input file](https://discuss.elastic.co/t/logstash-odx-input-file-unconventional-input-file/225324)
**Author:** [@newbie001](https://discuss.elastic.co/u/newbie001)\ **Replies:** 2\ **Last updated:** [April 2, 2020, 5:46am UTC](https://discuss.elastic.co/t/logstash-odx-input-file-unconventional-input-file/225324 "2020-04-02T05:46:10Z")
Hi - I was hoping someone could help me regarding input in logstash. I am very new to logstash and I have a requirement where my logstash input file is an unconventional file. How do I input an ODX input file? I think d… --- ## [Logstash working on Linux but not on Windows](https://discuss.elastic.co/t/logstash-working-on-linux-but-not-on-windows/222530)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 20\ **Last updated:** [April 1, 2020, 6:06am UTC](https://discuss.elastic.co/t/logstash-working-on-linux-but-not-on-windows/222530 "2020-04-01T06:06:35Z")
I have Logstash config file with grok filters that work when I am passing logs from linux. But when logs are passed from windows, the filters arent being applied to it. below is my linux filebeat filebeat.inputs: - … --- ## [Use logstash Sync kafka data to elasticsearch](https://discuss.elastic.co/t/use-logstash-sync-kafka-data-to-elasticsearch/226022)
**Author:** [@George\_Lee](https://discuss.elastic.co/u/George_Lee)\ **Replies:** 3\ **Last updated:** [April 2, 2020, 3:50am UTC](https://discuss.elastic.co/t/use-logstash-sync-kafka-data-to-elasticsearch/226022 "2020-04-02T03:50:18Z")
use logstash (version 7.3) Sync kafka data to elasticsearch.this is logstash config file input { kafka { codec =\> "json" auto\_offset\_reset =\> "earliest" bootstrap\_servers =\> "kafka-ece.crcloud.com:9092" … --- ## [Using forked plugin in an Logstash image](https://discuss.elastic.co/t/using-forked-plugin-in-an-logstash-image/226150)
**Author:** [@Suryatej\_Mukkamalla](https://discuss.elastic.co/u/Suryatej_Mukkamalla)\ **Replies:** 0\ **Last updated:** [April 2, 2020, 3:25am UTC](https://discuss.elastic.co/t/using-forked-plugin-in-an-logstash-image/226150 "2020-04-02T03:25:21Z")
I would like to fork my own version of the official Logstash kafka-input plugin. Is there a way for me to instruct the official Logstash 7.2.0 Docker image to install and use this forked version? --- ## [If condition after parsing logs in logstash](https://discuss.elastic.co/t/if-condition-after-parsing-logs-in-logstash/226130)
**Author:** [@Sepideh\_Bayati](https://discuss.elastic.co/u/Sepideh_Bayati)\ **Replies:** 2\ **Last updated:** [April 2, 2020, 12:24am UTC](https://discuss.elastic.co/t/if-condition-after-parsing-logs-in-logstash/226130 "2020-04-02T00:24:26Z")
hello. i have a logstash configuration file and i can parse input logs correctly. my question is: can i have conditions in my parsed data ? is this even possible in logastash config file ? for example i want only odd se… --- ## [Unable to restart LS with: Logstash failed to create queue](https://discuss.elastic.co/t/unable-to-restart-ls-with-logstash-failed-to-create-queue/226134)
**Author:** [@mtudisco](https://discuss.elastic.co/u/mtudisco)\ **Replies:** 0\ **Last updated:** [April 1, 2020, 11:26pm UTC](https://discuss.elastic.co/t/unable-to-restart-ls-with-logstash-failed-to-create-queue/226134 "2020-04-01T23:26:36Z")
Hi, We have Logstash 7.5.1 configure with pipelines, our pipelines configuration is: - pipeline.id: main path.config: "/etc/logstash/config/main/\*.conf" queue.type: persisted queue.checkpoint.writes: 1 pipeline… --- ## [Logstash grok \_grokparsefailure tag](https://discuss.elastic.co/t/logstash-grok-grokparsefailure-tag/226123)
**Author:** [@Sepideh\_Bayati](https://discuss.elastic.co/u/Sepideh_Bayati)\ **Replies:** 2\ **Last updated:** [April 1, 2020, 9:20pm UTC](https://discuss.elastic.co/t/logstash-grok-grokparsefailure-tag/226123 "2020-04-01T21:20:40Z")
hi. im using syslog-ng and loggen to generate my logs and this is the one for example: \<38\>2020-04-01T23:30:02 localhost prg00000\[1234\]: seq: 0000000096, thread: 0000, runid: 1585767601, stamp: 2020-04-01T23:30:02 PADDP… --- ## [Input JDBC with mongo](https://discuss.elastic.co/t/input-jdbc-with-mongo/226125)
**Author:** [@7cinquante](https://discuss.elastic.co/u/7cinquante)\ **Replies:** 0\ **Last updated:** [April 1, 2020, 9:13pm UTC](https://discuss.elastic.co/t/input-jdbc-with-mongo/226125 "2020-04-01T21:13:18Z")
Hello, I have a config to get data from mongo , logstash get doc , but only one doc is in ES logstash conf input { jdbc { jdbc\_driver\_library =\> "/usr/share/logstash/logstash-core/lib/jars/mongojdbc1.8.jar" … --- ## [Logstash fails with error on logstash-core-plugin-api.gemspec](https://discuss.elastic.co/t/logstash-fails-with-error-on-logstash-core-plugin-api-gemspec/226054)
**Author:** [@fionamacd1](https://discuss.elastic.co/u/fionamacd1)\ **Replies:** 1\ **Last updated:** [April 1, 2020, 8:58pm UTC](https://discuss.elastic.co/t/logstash-fails-with-error-on-logstash-core-plugin-api-gemspec/226054 "2020-04-01T20:58:37Z")
Hi, I updated our logstash to use OpenJDK11 and it was working fine for a few days. Then logs files grew too big and disk space got eaten. I removed the extra log files, turned down the log level and still getting this … --- ## [Test for field name containing special characters](https://discuss.elastic.co/t/test-for-field-name-containing-special-characters/226117)
**Author:** [@sampas](https://discuss.elastic.co/u/sampas)\ **Replies:** 4\ **Last updated:** [April 1, 2020, 8:10pm UTC](https://discuss.elastic.co/t/test-for-field-name-containing-special-characters/226117 "2020-04-01T20:10:35Z")
I want to test a field name to see if it contains a "%" character and drop it if it does. I do a KV on web logs, and we regularly see threat actors trying to escape application logic with long strings in the URI, e.g. b… --- ## [Convert Field Data to UTF8 or ISO-8859-1](https://discuss.elastic.co/t/convert-field-data-to-utf8-or-iso-8859-1/226112)
**Author:** [@rviega](https://discuss.elastic.co/u/rviega)\ **Replies:** 0\ **Last updated:** [April 1, 2020, 6:24pm UTC](https://discuss.elastic.co/t/convert-field-data-to-utf8-or-iso-8859-1/226112 "2020-04-01T18:24:00Z")
Hi Everyone, I would like to convert a field called "subject" to UTF-8 but I wasn't able to do it. I have configured logstash using charset ISO-8859-1and UTF-8 but Is not working. Here is an example of the data from th… --- ## [Replacing special characters with gsub](https://discuss.elastic.co/t/replacing-special-characters-with-gsub/226098)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 2\ **Last updated:** [April 1, 2020, 6:17pm UTC](https://discuss.elastic.co/t/replacing-special-characters-with-gsub/226098 "2020-04-01T18:17:32Z")
I'm using gsub and regex to structure a field into json. Unfortunately, I'm having problems with double-quotes being escaped in the output. For example: Field: event Value: 'success': True "event", "'", "\\"", Expec… --- ## [Logstash install plugin : how to get debug info](https://discuss.elastic.co/t/logstash-install-plugin-how-to-get-debug-info/226101)
**Author:** [@deslauriersp](https://discuss.elastic.co/u/deslauriersp)\ **Replies:** 0\ **Last updated:** [April 1, 2020, 5:39pm UTC](https://discuss.elastic.co/t/logstash-install-plugin-how-to-get-debug-info/226101 "2020-04-01T17:39:46Z")
I am using the base docker image from Logstash 7.6.0. It is a cenOS image. I am trying to install a custom java filter plugin. I used the example "java filter" in the Logstash to build from there. I built my gem on th… --- ## [How to construct geo\_point field from separate fields of latitude and longitude for Kibana 7.6?](https://discuss.elastic.co/t/how-to-construct-geo-point-field-from-separate-fields-of-latitude-and-longitude-for-kibana-7-6/225847)
**Author:** [@yassine\_belhajali](https://discuss.elastic.co/u/yassine_belhajali)\ **Replies:** 2\ **Last updated:** [April 1, 2020, 5:23pm UTC](https://discuss.elastic.co/t/how-to-construct-geo-point-field-from-separate-fields-of-latitude-and-longitude-for-kibana-7-6/225847 "2020-04-01T17:23:44Z")
From input csv data I have 2 fields of location - lat and long. How can I construct a geo\_point location field to create map visualization in logstash config (version 7.6)? Thank you. ' ' ' ' filter { csv{ separator… --- ## [Logstash events out of order](https://discuss.elastic.co/t/logstash-events-out-of-order/226082)
**Author:** [@sfischer](https://discuss.elastic.co/u/sfischer)\ **Replies:** 2\ **Last updated:** [April 1, 2020, 5:20pm UTC](https://discuss.elastic.co/t/logstash-events-out-of-order/226082 "2020-04-01T17:20:08Z")
Hello, I am monitoring a Java application with the elastic agent and create some custom transactions using the Java apm client. I configured the apm server to write all data into a file, so it can be collected later. … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=379) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=381)