# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=635 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 636 --- ## [Convert mapped IPv6 to IPv4](https://discuss.elastic.co/t/convert-mapped-ipv6-to-ipv4/158651)
**Author:** [@nandrik](https://discuss.elastic.co/u/nandrik)\ **Replies:** 2\ **Last updated:** [December 8, 2018, 11:47am UTC](https://discuss.elastic.co/t/convert-mapped-ipv6-to-ipv4/158651 "2018-12-08T11:47:54Z")
Hi, I have log lines from an IPv4 network. However, in the log lines some of the IP addresses are mapped toIPv6, i.e. are in IPv6 format. I wonder if there's a good way for those addresses to be converted to IPv4 forma… --- ## [Weird multiplication error in Logstash](https://discuss.elastic.co/t/weird-multiplication-error-in-logstash/159763)
**Author:** [@Harshet\_Jain](https://discuss.elastic.co/u/Harshet_Jain)\ **Replies:** 1\ **Last updated:** [December 8, 2018, 9:40am UTC](https://discuss.elastic.co/t/weird-multiplication-error-in-logstash/159763 "2018-12-08T09:40:21Z")
I want to convert currency (to indian rupees) from USD or EUR. I am using the following code in Ruby if \[Currency\] == "EUR" or \[Currency\] == "USD" { ruby { id =\> "addingCurrencyValue" code =\>… --- ## [Outputting logs to specific index depending on log\_name](https://discuss.elastic.co/t/outputting-logs-to-specific-index-depending-on-log-name/159991)
**Author:** [@lag](https://discuss.elastic.co/u/lag)\ **Replies:** 0\ **Last updated:** [December 8, 2018, 1:52am UTC](https://discuss.elastic.co/t/outputting-logs-to-specific-index-depending-on-log-name/159991 "2018-12-08T01:52:18Z")
We are currently using Winlogbeat to directly ship Windows Server logs to Elasticsearch, but to accommodate multiple dev environments with same server names, we will be moving to Logstash as it can do additional tagging … --- ## [Logstash to elasticsearch logs forwarding](https://discuss.elastic.co/t/logstash-to-elasticsearch-logs-forwarding/159980)
**Author:** [@syedashraf](https://discuss.elastic.co/u/syedashraf)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 10:08pm UTC](https://discuss.elastic.co/t/logstash-to-elasticsearch-logs-forwarding/159980 "2018-12-07T22:08:50Z")
Hello, I am new to Elasticsearch. I've a simple 2 VM topology. VM1 running logstash 6.5.2 and VM2 running elasticsearch 6.5.1. Installation packages successfully installed and services running. How do I setup config… --- ## [Some newbie questions file path and message replacement](https://discuss.elastic.co/t/some-newbie-questions-file-path-and-message-replacement/159847)
**Author:** [@alexsamad](https://discuss.elastic.co/u/alexsamad)\ **Replies:** 5\ **Last updated:** [December 7, 2018, 9:19pm UTC](https://discuss.elastic.co/t/some-newbie-questions-file-path-and-message-replacement/159847 "2018-12-07T21:19:08Z")
Hi extracting part of the path from source I found this And it seems to say what I want to do, but I am not sure how to implement I have filebeat setup to send to a logstash input I'm using grok, i've got grok { … --- ## [Logstash - Salesforce Questions](https://discuss.elastic.co/t/logstash-salesforce-questions/159973)
**Author:** [@jeremy.smith](https://discuss.elastic.co/u/jeremy.smith)\ **Replies:** 1\ **Last updated:** [December 7, 2018, 9:25pm UTC](https://discuss.elastic.co/t/logstash-salesforce-questions/159973 "2018-12-07T21:25:41Z")
Has anyone been successful in ingesting multiple SObjects into Logstash using the Salesforce Input Plugin? It seems that it can only ingest 1 SObject, 1 time, and then you need to restart in order to grab new data. We … --- ## [Logstash input s3 Exception: Aws::S3::Errors::AccessDenied after enabling server side encryption](https://discuss.elastic.co/t/logstash-input-s3-exception-aws-accessdenied-after-enabling-server-side-encryption/159974)
**Author:** [@Raywon\_Teja\_Kari](https://discuss.elastic.co/u/Raywon_Teja_Kari)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 7:54pm UTC](https://discuss.elastic.co/t/logstash-input-s3-exception-aws-accessdenied-after-enabling-server-side-encryption/159974 "2018-12-07T19:54:31Z")
Hi all, Recently we started fetching logs from s3 using logstash input s3 and it was working fine until we enabled server side encryption to the bucket. IAM permissions look OK. I have tried with s3:\* and resource:\* as… --- ## [Logstash format for date format Dec 04 21:24:03.238 2018](https://discuss.elastic.co/t/logstash-format-for-date-format-dec-04-2103-238-2018/159508)
**Author:** [@manikandanid](https://discuss.elastic.co/u/manikandanid)\ **Replies:** 3\ **Last updated:** [December 7, 2018, 4:45pm UTC](https://discuss.elastic.co/t/logstash-format-for-date-format-dec-04-2103-238-2018/159508 "2018-12-07T16:45:09Z")
Currently my log file contains the date format in Dec 04 21:24:03.238 2018 and the loglines are being tagged as dateparsefailure. Following is the date filter i have added in my logstash conf date{ match =\> \["LogDateTi… --- ## [Grok Debugger requires ":integer" but logstash document says to use ":int" for format. Is ":integer" valid in logstash?](https://discuss.elastic.co/t/grok-debugger-requires-integer-but-logstash-document-says-to-use-int-for-format-is-integer-valid-in-logstash/159577)
**Author:** [@rducharme](https://discuss.elastic.co/u/rducharme)\ **Replies:** 2\ **Last updated:** [December 7, 2018, 4:36pm UTC](https://discuss.elastic.co/t/grok-debugger-requires-integer-but-logstash-document-says-to-use-int-for-format-is-integer-valid-in-logstash/159577 "2018-12-07T16:36:29Z")
I have a logstash configuration file with some mutates to map a matched pattern field to an integer. // mutate { // convert =\> { // "pages" =\> "integer" // … --- ## [SunCertPathBuilderException when Logstash 6.4.2 outputs to HTTPS](https://discuss.elastic.co/t/suncertpathbuilderexception-when-logstash-6-4-2-outputs-to-https/159567)
**Author:** [@vmaldosan](https://discuss.elastic.co/u/vmaldosan)\ **Replies:** 0\ **Last updated:** [December 5, 2018, 3:44pm UTC](https://discuss.elastic.co/t/suncertpathbuilderexception-when-logstash-6-4-2-outputs-to-https/159567 "2018-12-05T15:44:26Z")
Hi there, I'm trying to forward messages from RabbitMQ to an HTTPS server, using a self-signed certificate. The relevant piece of my configuration is as follows: output { http { url =\> "https://localhost:84… --- ## [I need your help using grok](https://discuss.elastic.co/t/i-need-your-help-using-grok/159952)
**Author:** [@fcalegari](https://discuss.elastic.co/u/fcalegari)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 3:12pm UTC](https://discuss.elastic.co/t/i-need-your-help-using-grok/159952 "2018-12-07T15:12:30Z")
I have this log \[2018-12-07 00:00:06,048\]\[xx.xxx.xxx.xxxxxxxxxxxxxxxxx.xxxx.xxxxl \]\[INFO \]\[http-apr-8080-exec-583 \] - TDR (id= '4048814086', channel= '', msisdn= 'xxxxxxxxxx', service= 'nomb… --- ## [Logstash conditional filter doesn't work on Linux (on Windows it's OK)](https://discuss.elastic.co/t/logstash-conditional-filter-doesnt-work-on-linux-on-windows-its-ok/159949)
**Author:** [@Victor\_M](https://discuss.elastic.co/u/Victor_M)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 2:59pm UTC](https://discuss.elastic.co/t/logstash-conditional-filter-doesnt-work-on-linux-on-windows-its-ok/159949 "2018-12-07T14:59:20Z")
I have two input sources in logstash: one is log from java application, and it is json formatted source message (json): { "@timestamp": "2018-12-07T10:15:19.244Z", "offset": 312710, "line\_number": "-1", … --- ## [Extract raw message as field](https://discuss.elastic.co/t/extract-raw-message-as-field/159946)
**Author:** [@benoit\_elk](https://discuss.elastic.co/u/benoit_elk)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 2:44pm UTC](https://discuss.elastic.co/t/extract-raw-message-as-field/159946 "2018-12-07T14:44:14Z")
Hi. I'm looking for a simple way to extract the "raw\_message" and json\_encode it into a field. I've search a lot and couldn't find a solution but I feel that there is a simple way ! Thank you --- ## [Detect empty array](https://discuss.elastic.co/t/detect-empty-array/158801)
**Author:** [@Shawcs](https://discuss.elastic.co/u/Shawcs)\ **Replies:** 2\ **Last updated:** [December 7, 2018, 1:59pm UTC](https://discuss.elastic.co/t/detect-empty-array/158801 "2018-12-07T13:59:53Z")
Hi, how could I detect when an array in a json is empty with logstash ? example: EMPTY "serviceContexts": \[ \] FULL: "serviceContexts": \[{ "service": "JIRA", "monitor": true, "client": null, "… --- ## [Logstash-6.5.2 filter worker crash with Exception in pipelineworker](https://discuss.elastic.co/t/logstash-6-5-2-filter-worker-crash-with-exception-in-pipelineworker/159933)
**Author:** [@rafaelma](https://discuss.elastic.co/u/rafaelma)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 1:09pm UTC](https://discuss.elastic.co/t/logstash-6-5-2-filter-worker-crash-with-exception-in-pipelineworker/159933 "2018-12-07T13:09:20Z")
Hello We have a problem with our Logstash installation and we think it is a bug. Logstash version: 6.5.2 OS: Centos7 Description One of the pipelines in our Logstash installation stops working with an "Exception in … --- ## [Send Output of google\_pubsub input plugin to remote server](https://discuss.elastic.co/t/send-output-of-google-pubsub-input-plugin-to-remote-server/158982)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 1\ **Last updated:** [December 7, 2018, 12:52pm UTC](https://discuss.elastic.co/t/send-output-of-google-pubsub-input-plugin-to-remote-server/158982 "2018-12-07T12:52:29Z")
Hi Everyone, I installed Logstash and I was able to successfully setup the google\_pubsub input plugin to pull the Pub/Sub logs from GCP and I can see the logs on STD out on the screen. My question is how does one re-di… --- ## [Logstash 6.5 - FATAL: auth\_param digest program /usr/lib/squidi/digest\_file\_auth: (2) No such file or directory](https://discuss.elastic.co/t/logstash-6-5-fatal-auth-param-digest-program-usr-lib-squidi-digest-file-auth-2-no-such-file-or-directory/158862)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 1\ **Last updated:** [December 7, 2018, 12:39pm UTC](https://discuss.elastic.co/t/logstash-6-5-fatal-auth-param-digest-program-usr-lib-squidi-digest-file-auth-2-no-such-file-or-directory/158862 "2018-12-07T12:39:44Z")
Hi, Just tried to upgrade 6.1 to 6.5 and getting this error FATAL: auth\_param digest program /usr/lib/squidi/digest\_file\_auth: (2) No such file or directory Looks like something in the installer is pointing to a comple… --- ## [Coaxing geoip data from logstash](https://discuss.elastic.co/t/coaxing-geoip-data-from-logstash/159176)
**Author:** [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\ **Replies:** 1\ **Last updated:** [December 7, 2018, 12:33pm UTC](https://discuss.elastic.co/t/coaxing-geoip-data-from-logstash/159176 "2018-12-07T12:33:42Z")
I have logstash setup to convert ip -\> geoip. Im getting latitude/longitude coordinates from apache2/filebeat now. What I don't seem to have is whatever is required to create a visualization from this data. I've read th… --- ## [Logstash 6.4 send duplicates when logrotate](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894)
**Author:** [@Chumu](https://discuss.elastic.co/u/Chumu)\ **Replies:** 4\ **Last updated:** [December 7, 2018, 11:30am UTC](https://discuss.elastic.co/t/logstash-6-4-send-duplicates-when-logrotate/159894 "2018-12-07T11:30:48Z")
Hi, I'm using Logstash 6.4 (OS: CentOS 6.8) to send logs from files to a message queue (a bit alike Kafka). The files are in below structure, every 15 minutes, "access.log" in "second\_dir" will be moved to sub-director… --- ## [Trouble with Logstash sincedb file](https://discuss.elastic.co/t/trouble-with-logstash-sincedb-file/158882)
**Author:** [@Amedeo\_Avogadro](https://discuss.elastic.co/u/Amedeo_Avogadro)\ **Replies:** 1\ **Last updated:** [December 7, 2018, 11:24am UTC](https://discuss.elastic.co/t/trouble-with-logstash-sincedb-file/158882 "2018-12-07T11:24:06Z")
Now I have current .conf file for Logstash (6.3): input { file { path =\> "/home/ubuntu/mail/attachment/csv/Scan\_Report/current.csv" start\_position =\> "beginning" stat\_interval =\> 300 #sincedb\_path =\> "/dev/null… --- ## [Date filter not working with json log](https://discuss.elastic.co/t/date-filter-not-working-with-json-log/159899)
**Author:** [@se681268](https://discuss.elastic.co/u/se681268)\ **Replies:** 2\ **Last updated:** [December 7, 2018, 11:12am UTC](https://discuss.elastic.co/t/date-filter-not-working-with-json-log/159899 "2018-12-07T11:12:16Z")
Hello, I'm trying to format date by using filter, and use it as "Time Filter field name" to create index pattern in Kibana. I've already have a "date" field in a json-format log, like this: {"project":"sth","date":"20… --- ## [How to implement Logstash unstructured data to structured](https://discuss.elastic.co/t/how-to-implement-logstash-unstructured-data-to-structured/159888)
**Author:** [@Dev\_01](https://discuss.elastic.co/u/Dev_01)\ **Replies:** 1\ **Last updated:** [December 7, 2018, 11:05am UTC](https://discuss.elastic.co/t/how-to-implement-logstash-unstructured-data-to-structured/159888 "2018-12-07T11:05:27Z")
Example data : 2018-10-25 11:59:20:037 2018-10-25 11:59:20:059 20181025105423815012 20000 Success Expected Results StartTime|2018-10-25 11:59:20:037|StopTime|2018-10-25 11:59:20:059|Status|20000|Description|Success --- ## [Logstash-filter-date issue](https://discuss.elastic.co/t/logstash-filter-date-issue/159886)
**Author:** [@aabababba](https://discuss.elastic.co/u/aabababba)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 8:52am UTC](https://discuss.elastic.co/t/logstash-filter-date-issue/159886 "2018-12-07T08:52:40Z")
filter { date { match =\> \["logtime1","yyyymmdd HH:mm:ss,SSS"\] match =\> \["logtime2","MMM dd, yyyy hh:mm:ss a"\] target =\> "logtime" locale =\> "cn" timezone =\> "+08:00" } } use thi… --- ## [One Tcp port with multiple sources](https://discuss.elastic.co/t/one-tcp-port-with-multiple-sources/159853)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 6:23am UTC](https://discuss.elastic.co/t/one-tcp-port-with-multiple-sources/159853 "2018-12-07T06:23:07Z")
hi community I use tcp input plugin to get streaming data in logstash. I identify different sources by port. eg: tcp { port =\> 5513 add\_field =\> { "source\_type" =\> "SourceA" } } tcp { port =\> 5512 add\_fiel… --- ## [Index creation failed](https://discuss.elastic.co/t/index-creation-failed/159710)
**Author:** [@Sanchit\_Gupta](https://discuss.elastic.co/u/Sanchit_Gupta)\ **Replies:** 1\ **Last updated:** [December 7, 2018, 5:00am UTC](https://discuss.elastic.co/t/index-creation-failed/159710 "2018-12-07T05:00:29Z")
Hi Experts, I am new to Elasticsearch and facing some error while creating the index. my flow is like filebeat --\> logstash --\> elasticsearch --\> kibana. I am trying to read the SOA (weblogic) logs, below is my logsta… --- ## [About logstash jdbc input](https://discuss.elastic.co/t/about-logstash-jdbc-input/159697)
**Author:** [@supermario](https://discuss.elastic.co/u/supermario)\ **Replies:** 4\ **Last updated:** [December 7, 2018, 1:04am UTC](https://discuss.elastic.co/t/about-logstash-jdbc-input/159697 "2018-12-07T01:04:24Z")
I have a mysql database . A big table have 2000 billion rows records. I create a config file and use jdbc input, content: input { jdbc { jdbc\_driver\_library =\> "/usr/share/logstash/jars/mysql-connector-java-5.1.47.… --- ## [How to tell what fields can be examined/coded to in a Logstash filter?](https://discuss.elastic.co/t/how-to-tell-what-fields-can-be-examined-coded-to-in-a-logstash-filter/159772)
**Author:** [@RussellBateman](https://discuss.elastic.co/u/RussellBateman)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 11:49pm UTC](https://discuss.elastic.co/t/how-to-tell-what-fields-can-be-examined-coded-to-in-a-logstash-filter/159772 "2018-12-06T23:49:33Z")
If I want to write a filter that does something different depending on the provenance of a log entry, I am encouraged by examples to write filter { if \[ source \] =~ "/var/log/acme/debug.log" { dissect { … --- ## [Comment lancer logstash avec docker-compose](https://discuss.elastic.co/t/comment-lancer-logstash-avec-docker-compose/159814)
**Author:** [@Momo](https://discuss.elastic.co/u/Momo)\ **Replies:** 0\ **Last updated:** [December 6, 2018, 10:13pm UTC](https://discuss.elastic.co/t/comment-lancer-logstash-avec-docker-compose/159814 "2018-12-06T22:13:48Z")
Quand je lance logstash en mode docker-compose, j'ai une erreur : Creating logstash ... done Attaching to logstash logstash | OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from… --- ## [Storing logs from different locations into a Central Machine](https://discuss.elastic.co/t/storing-logs-from-different-locations-into-a-central-machine/159300)
**Author:** [@Rishav\_Anand](https://discuss.elastic.co/u/Rishav_Anand)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 10:21pm UTC](https://discuss.elastic.co/t/storing-logs-from-different-locations-into-a-central-machine/159300 "2018-12-06T22:21:25Z")
Hi, I am trying to store logs from different locations into a central machine. I have different log files on different physical machines something like dev ---\> nginx.log, web.log staging ---\> nginx.log, web.log … --- ## [Logstash Upgrade from 6.3 to 6.5 Confusion](https://discuss.elastic.co/t/logstash-upgrade-from-6-3-to-6-5-confusion/159793)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 1\ **Last updated:** [December 6, 2018, 8:17pm UTC](https://discuss.elastic.co/t/logstash-upgrade-from-6-3-to-6-5-confusion/159793 "2018-12-06T20:17:44Z")
We upgraded Elastic from 6.3 to 6.5, kibana to 6.5 and do not see direct instructions for logstash 6.5 upgrade. What am I missing? https://www.elastic.co/guide/en/logstash/6.5/upgrading-logstash.html [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=634) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=636)