# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=205 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 206 --- ## [Logstash and mongodb using JDBC is not working](https://discuss.elastic.co/t/logstash-and-mongodb-using-jdbc-is-not-working/280487)
**Author:** [@TranQuangHa307](https://discuss.elastic.co/u/TranQuangHa307)\ **Replies:** 0\ **Last updated:** [August 5, 2021, 3:16am UTC](https://discuss.elastic.co/t/logstash-and-mongodb-using-jdbc-is-not-working/280487 "2021-08-05T03:16:27Z")
This is my configuration file gives output as \[2021-08-05T10:13:30,445\]\[ERROR\]\[logstash.inputs.jdbc \]\[main\]\[c5849bc82bc4f5a1e20e8bbd042d4783536556e6cc82bbb47534d1a4e0939319\] Unable to connect to database. Tried 1… --- ## [Installing a custom local plugin](https://discuss.elastic.co/t/installing-a-custom-local-plugin/280574)
**Author:** [@gusDuarte](https://discuss.elastic.co/u/gusDuarte)\ **Replies:** 2\ **Last updated:** [August 5, 2021, 8:43pm UTC](https://discuss.elastic.co/t/installing-a-custom-local-plugin/280574 "2021-08-05T20:43:27Z")
Hi all, I want to make a bit modification to http input plugin, so to start with this task, i following steps: Install logstash 7.14 downloading logstash-7.14.0-darwin-x86\_64.tar.gz, and test if installation is ok bi… --- ## [Find Out empty field](https://discuss.elastic.co/t/find-out-empty-field/280576)
**Author:** [@YASH\_SHARMA7766](https://discuss.elastic.co/u/YASH_SHARMA7766)\ **Replies:** 2\ **Last updated:** [August 5, 2021, 7:09pm UTC](https://discuss.elastic.co/t/find-out-empty-field/280576 "2021-08-05T19:09:09Z")
I have a data set of 4-5 fields.I want to check if any data field is empty , Iogstash add a field name incomplete data and add value of field name in that.Sample data and code is attached below. As shown in image,ser… --- ## [Can you put Logstash behind Nginx proxy?](https://discuss.elastic.co/t/can-you-put-logstash-behind-nginx-proxy/279665)
**Author:** [@droplet](https://discuss.elastic.co/u/droplet)\ **Replies:** 12\ **Last updated:** [August 5, 2021, 3:51pm UTC](https://discuss.elastic.co/t/can-you-put-logstash-behind-nginx-proxy/279665 "2021-08-05T15:51:25Z")
Hello, Can you put Logstash behind Nginx proxy the same way you put Kibana behind an nginx proxy to use a custom domain with SSL? All this while also using HTTP Basic authentication with Nginx? ElasticSearch, Kibana, a… --- ## [LOGSTASH - MULTILINE XML IN LOG](https://discuss.elastic.co/t/logstash-multiline-xml-in-log/280528)
**Author:** [@Daniel\_Lopez](https://discuss.elastic.co/u/Daniel_Lopez)\ **Replies:** 0\ **Last updated:** [August 5, 2021, 11:37am UTC](https://discuss.elastic.co/t/logstash-multiline-xml-in-log/280528 "2021-08-05T11:37:32Z")
Hi to all I'm trying to parse xml info that it is in a log file, I tried with different configs, without luck, the actual one is this, but I tried with TIMESTAMP too, and join lines without sense codec =\> multiline{ c… --- ## [How to connect Neo4j from logstash using JDBC](https://discuss.elastic.co/t/how-to-connect-neo4j-from-logstash-using-jdbc/280524)
**Author:** [@vivek\_Jagadeesan](https://discuss.elastic.co/u/vivek_Jagadeesan)\ **Replies:** 0\ **Last updated:** [August 5, 2021, 11:11am UTC](https://discuss.elastic.co/t/how-to-connect-neo4j-from-logstash-using-jdbc/280524 "2021-08-05T11:11:53Z")
Hi All, Could anyone help me on how to connect the neo4j from logstash using JDBC? My sample config: input { jdbc { jdbc\_driver\_library =\> "/usr/share/java/neo4j-jdbc-driver-4.0.2.jar" jdbc\_driver\_class =\> "org.neo4… --- ## [Slow/Not Indexing in Elasticsearch via Logstash](https://discuss.elastic.co/t/slow-not-indexing-in-elasticsearch-via-logstash/280523)
**Author:** [@nchalise](https://discuss.elastic.co/u/nchalise)\ **Replies:** 0\ **Last updated:** [August 5, 2021, 11:10am UTC](https://discuss.elastic.co/t/slow-not-indexing-in-elasticsearch-via-logstash/280523 "2021-08-05T11:10:16Z")
We have Elasticsearch 7.11.1 and Logstash 7.11.2. We have a scenario of indexing data from Logstash to ES. We use a java application that does some data processing and posts data to the Logstash HTTP endpoint. Normally e… --- ## [Can Logstash Base64 decode then parse json from a cloudfront log?](https://discuss.elastic.co/t/can-logstash-base64-decode-then-parse-json-from-a-cloudfront-log/280367)
**Author:** [@chrisan](https://discuss.elastic.co/u/chrisan)\ **Replies:** 2\ **Last updated:** [August 5, 2021, 9:06am UTC](https://discuss.elastic.co/t/can-logstash-base64-decode-then-parse-json-from-a-cloudfront-log/280367 "2021-08-05T09:06:05Z")
Hello, we are using AWS's image resizer Serverless Image Handler | Implementations | AWS Solutions To get an image, you need to base64 encode a json object such as {"bucket":"my-s3-bucket","key":"path/to/your/image.jpg… --- ## [To modify multiple field names through logstash](https://discuss.elastic.co/t/to-modify-multiple-field-names-through-logstash/279876)
**Author:** [@raju.d](https://discuss.elastic.co/u/raju.d)\ **Replies:** 2\ **Last updated:** [August 5, 2021, 7:38am UTC](https://discuss.elastic.co/t/to-modify-multiple-field-names-through-logstash/279876 "2021-08-05T07:38:36Z")
Hello, I came across an old discussion -\> Rename or Change Available Fields . Unfortunately in that discussion, the final configuration that worked has not been shared! I am working on same and wanted to get rid of jso… --- ## [Problem with indexing](https://discuss.elastic.co/t/problem-with-indexing/280496)
**Author:** [@ailengcon](https://discuss.elastic.co/u/ailengcon)\ **Replies:** 0\ **Last updated:** [August 5, 2021, 5:55am UTC](https://discuss.elastic.co/t/problem-with-indexing/280496 "2021-08-05T05:55:55Z")
Okey, third time's the charm. I want to use elasticsearch to search a mongoDb, and I've manage to make the connection (using monstache) but the dynamic indexing does not index the documents right. I have one db with lot… --- ## [How to optimise/cleanup config with multiple mutations?](https://discuss.elastic.co/t/how-to-optimise-cleanup-config-with-multiple-mutations/280464)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 1\ **Last updated:** [August 4, 2021, 8:23pm UTC](https://discuss.elastic.co/t/how-to-optimise-cleanup-config-with-multiple-mutations/280464 "2021-08-04T20:23:50Z")
There are a lot of new field creations, renames, and removals in my filter plugin. And if, for example, I try to club the renames together, the mutations don't get executed correctly and i end up with only half the mutat… --- ## [Base64 decoding & decompression of json](https://discuss.elastic.co/t/base64-decoding-decompression-of-json/280442)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 2\ **Last updated:** [August 4, 2021, 3:54pm UTC](https://discuss.elastic.co/t/base64-decoding-decompression-of-json/280442 "2021-08-04T15:54:02Z")
I have a JSON input containing a \[message\]\[message\_json\] field which is compressed & base64 encoded at the source. This is the python code used to decode, decompress, and deserialize the field: message\['message\_json'\] … --- ## [Logstash input txt file received from filebeat is incorrect](https://discuss.elastic.co/t/logstash-input-txt-file-received-from-filebeat-is-incorrect/280393)
**Author:** [@Mohyden](https://discuss.elastic.co/u/Mohyden)\ **Replies:** 1\ **Last updated:** [August 4, 2021, 3:07pm UTC](https://discuss.elastic.co/t/logstash-input-txt-file-received-from-filebeat-is-incorrect/280393 "2021-08-04T15:07:22Z")
Hello Community, I am new to the community, and I need help. I found that logstash is not receiving correctly from filebeat. Some of the lines are not in the correct order that they should be and other lines are missed… --- ## [Logstash exec plugin error directory not found](https://discuss.elastic.co/t/logstash-exec-plugin-error-directory-not-found/280444)
**Author:** [@Hamza\_El\_Aouane](https://discuss.elastic.co/u/Hamza_El_Aouane)\ **Replies:** 0\ **Last updated:** [August 4, 2021, 2:19pm UTC](https://discuss.elastic.co/t/logstash-exec-plugin-error-directory-not-found/280444 "2021-08-04T14:19:49Z")
I am working with logstash for syslog processing. Right now, I have logstash and elastic search for data processing and grafana for visualisation. Those data that I am receiving, I am saving them to local disk as csv an… --- ## [Logstash logging problem](https://discuss.elastic.co/t/logstash-logging-problem/280329)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [August 4, 2021, 1:50pm UTC](https://discuss.elastic.co/t/logstash-logging-problem/280329 "2021-08-04T13:50:53Z")
using 7.13.4 having problem with logstash logging. I need two thing I want is to keep my log at my location . second I don't want all stdout message that comes out from conf file when it runs via pipeline. starting … --- ## [Logstash - Nested aggregation](https://discuss.elastic.co/t/logstash-nested-aggregation/280308)
**Author:** [@MattQDev](https://discuss.elastic.co/u/MattQDev)\ **Replies:** 0\ **Last updated:** [August 3, 2021, 12:23pm UTC](https://discuss.elastic.co/t/logstash-nested-aggregation/280308 "2021-08-03T12:23:28Z")
Hey, I am trying to aggregate some data from a SQL DB with Logstash. The data that I am trying to import has the following structure: Category Type Product Where each category can have multiple types, and… --- ## [\[LogStash::Runner\] runner - Logstash shut down](https://discuss.elastic.co/t/logstash-runner-runner-logstash-shut-down/280406)
**Author:** [@travlest](https://discuss.elastic.co/u/travlest)\ **Replies:** 0\ **Last updated:** [August 4, 2021, 9:29am UTC](https://discuss.elastic.co/t/logstash-runner-runner-logstash-shut-down/280406 "2021-08-04T09:29:16Z")
I am trying to configure http\_poller for logstash. But I've encountered this error. location : /etc/logstash/conf.d/logstash\_http\_poller input { http\_poller { urls =\> {\* urlname =\> "https://randomu… --- ## [Find removed IPs from list in new data](https://discuss.elastic.co/t/find-removed-ips-from-list-in-new-data/280067)
**Author:** [@YASH\_SHARMA7766](https://discuss.elastic.co/u/YASH_SHARMA7766)\ **Replies:** 3\ **Last updated:** [August 4, 2021, 8:54am UTC](https://discuss.elastic.co/t/find-removed-ips-from-list-in-new-data/280067 "2021-08-04T08:54:07Z")
I have a file containing list of IPs and i want to check in the new data if there is any missing IP. --- ## [Timestamp copy problem with Timezone](https://discuss.elastic.co/t/timestamp-copy-problem-with-timezone/279992)
**Author:** [@Kubix0](https://discuss.elastic.co/u/Kubix0)\ **Replies:** 2\ **Last updated:** [July 29, 2021, 3:31pm UTC](https://discuss.elastic.co/t/timestamp-copy-problem-with-timezone/279992 "2021-07-29T15:31:21Z")
Hey there, my first topic here. Wanted to ask some help because I can´t find a answer/solution for my problem. So, I have a Logstash filter that copy the @timestamp filed to make Timestamp more friendly for elastalert. … --- ## [Cloudtrail to Logstash automatic roll over on prefix date](https://discuss.elastic.co/t/cloudtrail-to-logstash-automatic-roll-over-on-prefix-date/280376)
**Author:** [@TsuWeiQuan](https://discuss.elastic.co/u/TsuWeiQuan)\ **Replies:** 0\ **Last updated:** [August 4, 2021, 2:19am UTC](https://discuss.elastic.co/t/cloudtrail-to-logstash-automatic-roll-over-on-prefix-date/280376 "2021-08-04T02:19:09Z")
Hi, I am trying to ingest cloudtrail logs to logstash to detect a certain event. I have installed logstash-codec-cloudtrail plugin and was able to ingest cloudtrail logs if i point to a specific folder. Currently i am … --- ## [Refresh in pipeline: output and/or filter](https://discuss.elastic.co/t/refresh-in-pipeline-output-and-or-filter/278044)
**Author:** [@itokai](https://discuss.elastic.co/u/itokai)\ **Replies:** 6\ **Last updated:** [August 4, 2021, 6:54am UTC](https://discuss.elastic.co/t/refresh-in-pipeline-output-and-or-filter/278044 "2021-08-04T06:54:33Z")
Hi, in logstash pipeline consecutive documents are consumed from RabbitMQ. Sometimes need to perform lookup on already imported documents via filter-\>elasticsearch-\>query\_template. However, lookup doesn't query the lat… --- ## [Index files from a folder in GCS bucket using logstash GCS plugin](https://discuss.elastic.co/t/index-files-from-a-folder-in-gcs-bucket-using-logstash-gcs-plugin/280384)
**Author:** [@madhanbaskar](https://discuss.elastic.co/u/madhanbaskar)\ **Replies:** 0\ **Last updated:** [August 4, 2021, 5:12am UTC](https://discuss.elastic.co/t/index-files-from-a-folder-in-gcs-bucket-using-logstash-gcs-plugin/280384 "2021-08-04T05:12:42Z")
Hi, I have a scenario where in the gcs bucket has multiple folders like dev and qa. Now by using logstash GCS plugin I have to read only the files from dev folder and not from qa. How is this possible? The below config… --- ## [Records not indexed with Logstash](https://discuss.elastic.co/t/records-not-indexed-with-logstash/280124)
**Author:** [@deadloss](https://discuss.elastic.co/u/deadloss)\ **Replies:** 8\ **Last updated:** [August 4, 2021, 4:41am UTC](https://discuss.elastic.co/t/records-not-indexed-with-logstash/280124 "2021-08-04T04:41:36Z")
I am facing this issue, the number of events out by elasticsearch output plugin(using logstash monitoring api) is equal to records in csv this means all records in csv are sent to ES but unfortunately the records are alw… --- ## [Using ILM: error's on more then one write index](https://discuss.elastic.co/t/using-ilm-errors-on-more-then-one-write-index/272804)
**Author:** [@robbert](https://discuss.elastic.co/u/robbert)\ **Replies:** 3\ **Last updated:** [August 3, 2021, 11:15pm UTC](https://discuss.elastic.co/t/using-ilm-errors-on-more-then-one-write-index/272804 "2021-08-03T23:15:21Z")
Greetings, Currently i am in the process of migrating our ELK instance from manual index deletion to using ILM policy, however filebeat is sending logs to logstash but the following error is shown in the logs of logstas… --- ## [Http input plugin, url with param from field](https://discuss.elastic.co/t/http-input-plugin-url-with-param-from-field/280365)
**Author:** [@cibernicola](https://discuss.elastic.co/u/cibernicola)\ **Replies:** 6\ **Last updated:** [August 3, 2021, 10:32pm UTC](https://discuss.elastic.co/t/http-input-plugin-url-with-param-from-field/280365 "2021-08-03T22:32:25Z")
I have a standard logstash config, where I store several fields form a log. With one field I need to "ask something" to an external API with an specific url pattern, which is: staticString+variableField+staticString S… --- ## [Multiple Sql Server DB mapping with Elasticsearch Index for data transfer using Logstash](https://discuss.elastic.co/t/multiple-sql-server-db-mapping-with-elasticsearch-index-for-data-transfer-using-logstash/280364)
**Author:** [@vikashmarwal](https://discuss.elastic.co/u/vikashmarwal)\ **Replies:** 0\ **Last updated:** [August 3, 2021, 7:52pm UTC](https://discuss.elastic.co/t/multiple-sql-server-db-mapping-with-elasticsearch-index-for-data-transfer-using-logstash/280364 "2021-08-03T19:52:30Z")
We are having multiple databases in SQL Server, those are created dynamically for specific workspace. In these databases we have a DocumentDetail table. If there is one DB then I am able to map it in config to transfer t… --- ## [Logstash write to json no comma separator](https://discuss.elastic.co/t/logstash-write-to-json-no-comma-separator/280302)
**Author:** [@Hamza\_El\_Aouane](https://discuss.elastic.co/u/Hamza_El_Aouane)\ **Replies:** 3\ **Last updated:** [August 3, 2021, 7:18pm UTC](https://discuss.elastic.co/t/logstash-write-to-json-no-comma-separator/280302 "2021-08-03T19:18:57Z")
Hello everyone. I am saving my logstash events to a json file as follow: file { path =\> "path\\event-%{+yyyy.MM.dd}.json" codec =\> json everything is fine, until today that I realised that the file is not form… --- ## [\_grokparsefailure is occurring even after the grok pattern is success](https://discuss.elastic.co/t/grokparsefailure-is-occurring-even-after-the-grok-pattern-is-success/280352)
**Author:** [@vaseemQA](https://discuss.elastic.co/u/vaseemQA)\ **Replies:** 0\ **Last updated:** [August 3, 2021, 4:51pm UTC](https://discuss.elastic.co/t/grokparsefailure-is-occurring-even-after-the-grok-pattern-is-success/280352 "2021-08-03T16:51:02Z")
Hi Team, I'm using logstash 6.8.3, and I'm trying to parse ES slow logs and my sample field is a ES source\_query which looks like {"from":0,"size":0,"post\_filter":{"bool":{"must":\[{"term":{"someId":{"value":1234,"boost… --- ## [Possible collision with user-defined field 'parameter' and built-in fields](https://discuss.elastic.co/t/possible-collision-with-user-defined-field-parameter-and-built-in-fields/280330)
**Author:** [@macs](https://discuss.elastic.co/u/macs)\ **Replies:** 0\ **Last updated:** [August 3, 2021, 2:25pm UTC](https://discuss.elastic.co/t/possible-collision-with-user-defined-field-parameter-and-built-in-fields/280330 "2021-08-03T14:25:42Z")
Hey, I am working on an app where we tried to append a Logstash Marker with the custom key "parameter", containing a string with some information about where in the application the log event occured. Unfortunately, sinc… --- ## [Elasticsearch input plugin size](https://discuss.elastic.co/t/elasticsearch-input-plugin-size/280163)
**Author:** [@JFO\_90](https://discuss.elastic.co/u/JFO_90)\ **Replies:** 7\ **Last updated:** [August 3, 2021, 11:15am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-size/280163 "2021-08-03T11:15:25Z")
Hi, I have a logstash configuration from which I'm trying to extract elasticsearch information with the elasticsearch input plugin. The problem that i have is that despite indicating a size, it always returns all the do… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=204) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=206)