# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=350 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 351 --- ## [Masking the password and handling two client IP's in apache\_access log](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927)
**Author:** [@marco2005](https://discuss.elastic.co/u/marco2005)\ **Replies:** 4\ **Last updated:** [May 29, 2020, 3:44pm UTC](https://discuss.elastic.co/t/masking-the-password-and-handling-two-client-ips-in-apache-access-log/234927 "2020-05-29T15:44:42Z")
Sometimes, I have standard apache logs starting with a single client IP and sometimes with two client IP's like this which my logstash config does not handle. Shall I use a 3rd pattern in Grok ? 1.2.3.4,5.6.7.8 Also,… --- ## [Could not find or load main class files in logstash](https://discuss.elastic.co/t/could-not-find-or-load-main-class-files-in-logstash/234919)
**Author:** [@Sri\_Subiksha](https://discuss.elastic.co/u/Sri_Subiksha)\ **Replies:** 1\ **Last updated:** [May 29, 2020, 3:35pm UTC](https://discuss.elastic.co/t/could-not-find-or-load-main-class-files-in-logstash/234919 "2020-05-29T15:35:59Z")
Continuing the discussion from Could not find or load main class Files: can you help me with this error? --- ## [Line sequence with logstash](https://discuss.elastic.co/t/line-sequence-with-logstash/234922)
**Author:** [@Katia](https://discuss.elastic.co/u/Katia)\ **Replies:** 7\ **Last updated:** [May 29, 2020, 2:37pm UTC](https://discuss.elastic.co/t/line-sequence-with-logstash/234922 "2020-05-29T14:37:19Z")
Good morning, Can I get some help ! I have a log file, where the first data on each line is in timestamp format. to do my analysis I need to know the order of these lines, so I used a new "seq" field with the ruby fil… --- ## [Rename the message field from the original log](https://discuss.elastic.co/t/rename-the-message-field-from-the-original-log/234674)
**Author:** [@marco2005](https://discuss.elastic.co/u/marco2005)\ **Replies:** 2\ **Last updated:** [May 29, 2020, 1:29pm UTC](https://discuss.elastic.co/t/rename-the-message-field-from-the-original-log/234674 "2020-05-29T13:29:34Z")
I have a log line. 1590644837.665697 2020-05-28 07:47:17,665 to:004915122898981|id:Hello there|status\_code:SentSMSOK|status\_message:abc.exmaple.com - 200|message:(2020-05-28 07:46:41) TEST: stuck-daily@example.com… --- ## [ES query to check the existence of a document\_id?](https://discuss.elastic.co/t/es-query-to-check-the-existence-of-a-document-id/234854)
**Author:** [@Nani\_20](https://discuss.elastic.co/u/Nani_20)\ **Replies:** 9\ **Last updated:** [May 29, 2020, 1:29pm UTC](https://discuss.elastic.co/t/es-query-to-check-the-existence-of-a-document-id/234854 "2020-05-29T13:29:08Z")
I'm using a file input in logstash to input files but I encounter duplicates time to time. Currently, I'm using the following the below options in ES output plugin to avoid duplicates. document\_id =\> "%{message}" ac… --- ## [Error connecting MongoDB and Elasticseach through Logstash](https://discuss.elastic.co/t/error-connecting-mongodb-and-elasticseach-through-logstash/234762)
**Author:** [@FernandoMOTA](https://discuss.elastic.co/u/FernandoMOTA)\ **Replies:** 16\ **Last updated:** [May 29, 2020, 1:14pm UTC](https://discuss.elastic.co/t/error-connecting-mongodb-and-elasticseach-through-logstash/234762 "2020-05-29T13:14:24Z")
I can't do the ingestion between the MongoBD database and Elasticseach using Logstash, below is my .conf file input { uri =\> 'mongodb://localhost:27017/exemplo' placeholder\_db\_dir =\> '/opt/logstash-m… --- ## [External Ruby Script to enrich multiple Fields](https://discuss.elastic.co/t/external-ruby-script-to-enrich-multiple-fields/234916)
**Author:** [@muraliv](https://discuss.elastic.co/u/muraliv)\ **Replies:** 0\ **Last updated:** [May 29, 2020, 12:43pm UTC](https://discuss.elastic.co/t/external-ruby-script-to-enrich-multiple-fields/234916 "2020-05-29T12:43:53Z")
Hi All, I am currently enriching 4 fields using translate filter. I would like to know if it's possible to create an external ruby script with the following code. Here is a code for one such field. translate… --- ## [How to write a Logstash Multi-line Pattern for the Windows Events](https://discuss.elastic.co/t/how-to-write-a-logstash-multi-line-pattern-for-the-windows-events/234913)
**Author:** [@sharmad](https://discuss.elastic.co/u/sharmad)\ **Replies:** 0\ **Last updated:** [May 29, 2020, 12:38pm UTC](https://discuss.elastic.co/t/how-to-write-a-logstash-multi-line-pattern-for-the-windows-events/234913 "2020-05-29T12:38:25Z")
can someone guide me to create a Multi-line Pattern for the below windows events with event ID and need to parse my log data Event\[0\]: Log Name: Application Source: Desktop Window Manager Date: 2020-05-29T16:53:12.00… --- ## [Parsing multiline log: correct output in Logstash and error in kibana](https://discuss.elastic.co/t/parsing-multiline-log-correct-output-in-logstash-and-error-in-kibana/234411)
**Author:** [@guas5](https://discuss.elastic.co/u/guas5)\ **Replies:** 1\ **Last updated:** [May 29, 2020, 11:28am UTC](https://discuss.elastic.co/t/parsing-multiline-log-correct-output-in-logstash-and-error-in-kibana/234411 "2020-05-29T11:28:07Z")
Hi everyone, would be really gratefull for any ideas how to solve the following issue. I have defined a regex pattern for a multiline log and checked its output in Logstash, everything looks correct. But once I send dat… --- ## [How to avoid duplicates before entering the filter plugin?](https://discuss.elastic.co/t/how-to-avoid-duplicates-before-entering-the-filter-plugin/234735)
**Author:** [@Nani\_20](https://discuss.elastic.co/u/Nani_20)\ **Replies:** 5\ **Last updated:** [May 29, 2020, 8:23am UTC](https://discuss.elastic.co/t/how-to-avoid-duplicates-before-entering-the-filter-plugin/234735 "2020-05-29T08:23:55Z")
Hi, I'm using a document\_id and action =\> create in the logstash elasticsearch output filter to avoid duplicates. Since, this is in output plugin, I can see the duplicates passing through all my filter plugin operation… --- ## [Multiple inputs in logstash](https://discuss.elastic.co/t/multiple-inputs-in-logstash/234849)
**Author:** [@vikramaddagulla](https://discuss.elastic.co/u/vikramaddagulla)\ **Replies:** 1\ **Last updated:** [May 29, 2020, 5:45am UTC](https://discuss.elastic.co/t/multiple-inputs-in-logstash/234849 "2020-05-29T05:45:07Z")
Hello, We are currently running filebeats and sending the data to logstash and that is working fine. We are now working on a new set up which need to connect to an Oracle Database and get data from a table. I understa… --- ## [Fetch the Count of specific keywords in the log message](https://discuss.elastic.co/t/fetch-the-count-of-specific-keywords-in-the-log-message/232552)
**Author:** [@nikhilesh](https://discuss.elastic.co/u/nikhilesh)\ **Replies:** 2\ **Last updated:** [May 29, 2020, 5:38am UTC](https://discuss.elastic.co/t/fetch-the-count-of-specific-keywords-in-the-log-message/232552 "2020-05-29T05:38:29Z")
Hi Team, Can any help on this please? I have a requirement to capture the count of specific keywords ( i.e Exception types) from the log file and should visualize in bar/pie. and the keywords are some where in the me… --- ## [Try to enable geoip on logstash](https://discuss.elastic.co/t/try-to-enable-geoip-on-logstash/234763)
**Author:** [@rocanelo](https://discuss.elastic.co/u/rocanelo)\ **Replies:** 2\ **Last updated:** [May 29, 2020, 4:33am UTC](https://discuss.elastic.co/t/try-to-enable-geoip-on-logstash/234763 "2020-05-29T04:33:22Z")
Hi everyone, I'm try to enable geoip on my logstash but have a message say the database is corrupted, I suscribe to maxmind and get the last database, I upgdate with command "geoupdate" but not working. I dont now what… --- ## [Error in comparing date type fields on logstash](https://discuss.elastic.co/t/error-in-comparing-date-type-fields-on-logstash/234853)
**Author:** [@ed2050](https://discuss.elastic.co/u/ed2050)\ **Replies:** 0\ **Last updated:** [May 29, 2020, 2:57am UTC](https://discuss.elastic.co/t/error-in-comparing-date-type-fields-on-logstash/234853 "2020-05-29T02:57:09Z")
Hi. I have the following error when comparing two fields of date types: An unexpected error occurred! {:error=\>#\' for nil:NilClass\>, :backtrace=\>\["(eval):81714:in block in initialize'"… --- ## [How to use document id to avoid duplication of logs?](https://discuss.elastic.co/t/how-to-use-document-id-to-avoid-duplication-of-logs/234644)
**Author:** [@Nani\_20](https://discuss.elastic.co/u/Nani_20)\ **Replies:** 7\ **Last updated:** [May 29, 2020, 2:56am UTC](https://discuss.elastic.co/t/how-to-use-document-id-to-avoid-duplication-of-logs/234644 "2020-05-29T02:56:41Z")
Hi, I'm using logstash file input to index various log files available in the input location. These logfiles are generated by log4j in various systems and are fetched using a cron job to the input location. Is there a… --- ## [Logstash 401 Error](https://discuss.elastic.co/t/logstash-401-error/234029)
**Author:** [@vikramaddagulla](https://discuss.elastic.co/u/vikramaddagulla)\ **Replies:** 10\ **Last updated:** [May 29, 2020, 2:15am UTC](https://discuss.elastic.co/t/logstash-401-error/234029 "2020-05-29T02:15:40Z")
Hello, I have followed the steps in the below doc to enable security ; Everything seems to have worked. HOwever, when I start logstash, i get the below error : \[2020-05-24T02:16:15,655\]\[WARN \]\[logstash.outputs.elas… --- ## ["illegal\_argument\_exception", "reason"=\>"if \_id is specified it must not be empty"](https://discuss.elastic.co/t/illegal-argument-exception-reason-if-id-is-specified-it-must-not-be-empty/234826)
**Author:** [@tenet\_testuser1](https://discuss.elastic.co/u/tenet_testuser1)\ **Replies:** 1\ **Last updated:** [May 28, 2020, 9:49pm UTC](https://discuss.elastic.co/t/illegal-argument-exception-reason-if-id-is-specified-it-must-not-be-empty/234826 "2020-05-28T21:49:59Z")
Hello, I have setup a logstash pipeline. It works fine bringing the updates and refreshing the index. However, it throws an error while bringing updates. I am trying to understand the reasons behind. output { elastics… --- ## [Error with dates in logstash 6.7 and Oracle database](https://discuss.elastic.co/t/error-with-dates-in-logstash-6-7-and-oracle-database/234813)
**Author:** [@dooger21](https://discuss.elastic.co/u/dooger21)\ **Replies:** 3\ **Last updated:** [May 28, 2020, 9:30pm UTC](https://discuss.elastic.co/t/error-with-dates-in-logstash-6-7-and-oracle-database/234813 "2020-05-28T21:30:18Z")
I have an error message when running a .conf script that contains a connection to an oracle database using jdbc and slq\_last\_value (3). The date format setting differs between database (1) and what is shown by logstash (… --- ## [Could not index event to Elasticsearch. "reason"=\>"if \_id is specified it must not be empty"}](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-reason-if-id-is-specified-it-must-not-be-empty/234181)
**Author:** [@tenet\_testuser1](https://discuss.elastic.co/u/tenet_testuser1)\ **Replies:** 10\ **Last updated:** [May 28, 2020, 8:16pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-reason-if-id-is-specified-it-must-not-be-empty/234181 "2020-05-28T20:16:34Z")
Hello, I have setup a logstash pipeline. It works fine bringing the updates and refreshing the index. However, it throws an error while bringing updates. I am trying to understand the reasons behind. output { elastics… --- ## [Add new field using part of existing value](https://discuss.elastic.co/t/add-new-field-using-part-of-existing-value/234434)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 9\ **Last updated:** [May 28, 2020, 6:04pm UTC](https://discuss.elastic.co/t/add-new-field-using-part-of-existing-value/234434 "2020-05-28T18:04:38Z")
Hello - I would like to create a new field named "process\_name". I would like to use part of the an existing field's value to add to the newly created field. Ex: Sample JSON Log : "cb\_server":"cbserver","computer\_name"… --- ## [Error when copying a field to multiple others in a single mutate block](https://discuss.elastic.co/t/error-when-copying-a-field-to-multiple-others-in-a-single-mutate-block/234709)
**Author:** [@fgierlinger](https://discuss.elastic.co/u/fgierlinger)\ **Replies:** 1\ **Last updated:** [May 28, 2020, 5:24pm UTC](https://discuss.elastic.co/t/error-when-copying-a-field-to-multiple-others-in-a-single-mutate-block/234709 "2020-05-28T17:24:43Z")
I noticed today, that copying the same field to multiple destinations inside the same mutate block, leads to a \_mutate\_error. Splitting the copy operation up to two mutate blocks works without any problem. Is this an int… --- ## [Pattern to drop events if output is becoming clogged](https://discuss.elastic.co/t/pattern-to-drop-events-if-output-is-becoming-clogged/234764)
**Author:** [@bibblyb](https://discuss.elastic.co/u/bibblyb)\ **Replies:** 2\ **Last updated:** [May 28, 2020, 5:23pm UTC](https://discuss.elastic.co/t/pattern-to-drop-events-if-output-is-becoming-clogged/234764 "2020-05-28T17:23:04Z")
I'm looking to find a good pattern that will allow me to start discarding events that flow through one of my output pipelines which is outputting via HTTP to a 3rd party. The HTTP connections are not always fast enough … --- ## [An unexpected error occurred! {:error=\>#\ **Author:** [@Raj\_Techie](https://discuss.elastic.co/u/Raj_Techie)\ **Replies:** 0\ **Last updated:** [May 28, 2020, 1:24pm UTC](https://discuss.elastic.co/t/an-unexpected-error-occurred-error-nomethoderror-undefined-method/234752 "2020-05-28T13:24:32Z") Hi Currently using logstash version 6.0.1. connecting to ES 5.1.X. Got below error like --- An unexpected error occurred! {:error=\>#\, :backtrace=\>\["/appserver/pq-log… --- ## [Error caused by check of JAVA\_TOOL\_OPTIONS when running Logstash on Windows](https://discuss.elastic.co/t/error-caused-by-check-of-java-tool-options-when-running-logstash-on-windows/234729)
**Author:** [@paulr3](https://discuss.elastic.co/u/paulr3)\ **Replies:** 0\ **Last updated:** [May 28, 2020, 11:18am UTC](https://discuss.elastic.co/t/error-caused-by-check-of-java-tool-options-when-running-logstash-on-windows/234729 "2020-05-28T11:18:39Z")
Hi, I've encountered an issue when I try to run the Windows version of Logstash. When I run logstash.bat I receive the following error message: "Microsystems was unexpected at this time" The issue appears to be in se… --- ## [Can't see logs from Fortigate device](https://discuss.elastic.co/t/cant-see-logs-from-fortigate-device/234656)
**Author:** [@hxvxt2](https://discuss.elastic.co/u/hxvxt2)\ **Replies:** 0\ **Last updated:** [May 28, 2020, 5:32am UTC](https://discuss.elastic.co/t/cant-see-logs-from-fortigate-device/234656 "2020-05-28T05:32:53Z")
Hi There, I'm trying to set up my FortiGate device to send logs to my ELK cluster. I am running ELK inside a docker container. Here is what my Fortigate Config looks like:- config log syslogd2 setting set s… --- ## [Logstash if statement for program](https://discuss.elastic.co/t/logstash-if-statement-for-program/234020)
**Author:** [@Rasoul\_Ahmadi](https://discuss.elastic.co/u/Rasoul_Ahmadi)\ **Replies:** 6\ **Last updated:** [May 28, 2020, 5:56am UTC](https://discuss.elastic.co/t/logstash-if-statement-for-program/234020 "2020-05-28T05:56:19Z")
hi i know its elementary but i am stuck at an if statement with if statement I get grok failure and without it every thing is fine I'm using logstash 7.7 indent preformatted text by 4 spaces \`\`\` input {stdin{}} filt… --- ## [Formulas in Logsatsh](https://discuss.elastic.co/t/formulas-in-logsatsh/234650)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 2\ **Last updated:** [May 28, 2020, 5:48am UTC](https://discuss.elastic.co/t/formulas-in-logsatsh/234650 "2020-05-28T05:48:15Z")
Hi, I have a column in my source data which says if a ticket has met its SLAs. met\_sla: true or met\_sla: false I wanna calculate the below in Logstash and create a new field called target percent Target= (count(met… --- ## [S3 input plugin missing kms decrypt parameter](https://discuss.elastic.co/t/s3-input-plugin-missing-kms-decrypt-parameter/234643)
**Author:** [@s2mvk](https://discuss.elastic.co/u/s2mvk)\ **Replies:** 0\ **Last updated:** [May 28, 2020, 3:32am UTC](https://discuss.elastic.co/t/s3-input-plugin-missing-kms-decrypt-parameter/234643 "2020-05-28T03:32:46Z")
Hi We are using ELK approach, part of it Logstash is not Able to decrypt data in s3(data at rest in kms(customer key) encrypted) when pushing to Elastic Search, Do we know/have any other way to decrypt with kms key for… --- ## [Erreur d'importation avec logstash sous linux](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427)
**Author:** [@Katia](https://discuss.elastic.co/u/Katia)\ **Replies:** 19\ **Last updated:** [May 28, 2020, 12:37am UTC](https://discuss.elastic.co/t/erreur-dimportation-avec-logstash-sous-linux/233427 "2020-05-28T00:37:04Z")
Bonjour, J'importe mes logs avec logstash vers elasticserach, ça marche bien en local ( sous windows). Mais pas sous linux, j'ai pleins d'erreurs et je ne sais pas pourquoi! j'ai remarqué que le dans fichier.txt , s'… --- ## [Logstash with elastisearch cloud](https://discuss.elastic.co/t/logstash-with-elastisearch-cloud/231021)
**Author:** [@Georgios\_Nikoloudis](https://discuss.elastic.co/u/Georgios_Nikoloudis)\ **Replies:** 7\ **Last updated:** [May 28, 2020, 12:25am UTC](https://discuss.elastic.co/t/logstash-with-elastisearch-cloud/231021 "2020-05-28T00:25:27Z")
hi is it possible to give me some direction how to configure the logstash in the ES cloud? I have logstash configurations in the current logstash (server side deployment) and I would like to move them to the ES cloud. … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=349) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=351)