# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=940 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 941 --- ## [Stripping characters from a field and convert to lower case](https://discuss.elastic.co/t/stripping-characters-from-a-field-and-convert-to-lower-case/92850)

**Author:** [@bds\_Vince](https://discuss.elastic.co/u/bds_Vince)\ **Replies:** 2\ **Last updated:** [July 24, 2017, 10:35pm UTC](https://discuss.elastic.co/t/stripping-characters-from-a-field-and-convert-to-lower-case/92850 "2017-07-24T22:35:43Z")

I am importing IIS logs from the SMTP relay service. I have successfully separated out all of the fields in logstash. One of the data fields needs to have excess characters trimmed and then converted to lower case for … --- ## [Grok Filter Logstash Multiple Lines](https://discuss.elastic.co/t/grok-filter-logstash-multiple-lines/94237)

**Author:** [@lavolpem](https://discuss.elastic.co/u/lavolpem)\ **Replies:** 2\ **Last updated:** [July 24, 2017, 7:11pm UTC](https://discuss.elastic.co/t/grok-filter-logstash-multiple-lines/94237 "2017-07-24T19:11:02Z")

I have some problem about the log's format that I had to parse. My logs have a format like this: ================================================================== Report : WARNING Date : Thu Jun 18 16:5… --- ## [Decode json in \_source into fields](https://discuss.elastic.co/t/decode-json-in--source-into-fields/92137)

**Author:** [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Replies:** 1\ **Last updated:** [July 24, 2017, 5:47pm UTC](https://discuss.elastic.co/t/decode-json-in--source-into-fields/92137 "2017-07-24T17:47:46Z")

Hello, I'm using Filebeat and Logstash to analyze logs from Wazuh, the logs are json files where each line is a json string, I can read the log correctly and push it into Elasticsearch, through Logstash, and it correctl… --- ## [Multiline Codec: Autoflush causing duplicates of last log message](https://discuss.elastic.co/t/multiline-codec-autoflush-causing-duplicates-of-last-log-message/94369)

**Author:** [@CDR](https://discuss.elastic.co/u/CDR)\ **Replies:** 0\ **Last updated:** [July 24, 2017, 4:51pm UTC](https://discuss.elastic.co/t/multiline-codec-autoflush-causing-duplicates-of-last-log-message/94369 "2017-07-24T16:51:11Z")

So I was having an issue where Logstash's multiline codec was not reading in the last log message from a file. When I added the auto\_flush\_interval as suggested in the link. I found that now it adds the log message twice… --- ## [Kibana didn't load the data when using ruby filter in logstash](https://discuss.elastic.co/t/kibana-didnt-load-the-data-when-using-ruby-filter-in-logstash/94308)

**Author:** [@moabbas](https://discuss.elastic.co/u/moabbas)\ **Replies:** 4\ **Last updated:** [July 24, 2017, 4:17pm UTC](https://discuss.elastic.co/t/kibana-didnt-load-the-data-when-using-ruby-filter-in-logstash/94308 "2017-07-24T16:17:53Z")

I wrote ruby code in logstash conf file like this mutate { add\_field =\> {"\[location\]" =\> "\[0,0\]"} } ruby{ code =\> 'event.set("\[location\]", \[(event.get("%{\[lt\]\[py\]}").to\_f \* 5.6) / 10), event.get("\[lt\]\[px\]"\])) ' } w… --- ## [When processing stdin, any equivalents to awk's BEGIN and END?](https://discuss.elastic.co/t/when-processing-stdin-any-equivalents-to-awks-begin-and-end/93623)

**Author:** [@UnitedMarsupials](https://discuss.elastic.co/u/UnitedMarsupials)\ **Replies:** 4\ **Last updated:** [July 24, 2017, 3:03pm UTC](https://discuss.elastic.co/t/when-processing-stdin-any-equivalents-to-awks-begin-and-end/93623 "2017-07-24T15:03:50Z")

awk has the special actions triggered by the BEGINning and ENDing the processing of stdin (or each individual input file). (There are also variables automatically maintained by the tool (like NR).) Is there any equiv… --- ## [Logstash error with jdbc connection](https://discuss.elastic.co/t/logstash-error-with-jdbc-connection/92866)

**Author:** [@rudra](https://discuss.elastic.co/u/rudra)\ **Replies:** 1\ **Last updated:** [July 24, 2017, 2:45pm UTC](https://discuss.elastic.co/t/logstash-error-with-jdbc-connection/92866 "2017-07-24T14:45:11Z")

Unable to connect to oracle database from a logstash conf file: input { jdbc { jdbc\_validate\_connection =\> true jdbc\_connection\_string =\> "jdbc:oracle:thin:@:1522/" jdbc\_user =\> "\<\>" jdbc\_password =\> "\<\>" jdbc\_dri… --- ## [Ruby filter used for reading modified time of file](https://discuss.elastic.co/t/ruby-filter-used-for-reading-modified-time-of-file/94127)

**Author:** [@CDR](https://discuss.elastic.co/u/CDR)\ **Replies:** 1\ **Last updated:** [July 24, 2017, 10:38am UTC](https://discuss.elastic.co/t/ruby-filter-used-for-reading-modified-time-of-file/94127 "2017-07-24T10:38:10Z")

Hello community, I am currently trying to use the ruby filter to get the modified time of the file I am reading. I want to store the modified time in mtime. This is my first time using ruby so I am sort of lost. Here is… --- ## [How to divide XML data into two fields](https://discuss.elastic.co/t/how-to-divide-xml-data-into-two-fields/94301)

**Author:** [@abhisek](https://discuss.elastic.co/u/abhisek)\ **Replies:** 0\ **Last updated:** [July 24, 2017, 10:07am UTC](https://discuss.elastic.co/t/how-to-divide-xml-data-into-two-fields/94301 "2017-07-24T10:07:40Z")

I have a XML data which gets loaded in logstash and it looks like this in kibana. \ \ **Author:** [@R-Ali](https://discuss.elastic.co/u/R-Ali)\ **Replies:** 0\ **Last updated:** [July 24, 2017, 9:30am UTC](https://discuss.elastic.co/t/logstash-input-jms-client-acknowledge/94292 "2017-07-24T09:30:19Z") Hello, I'm using the plugin logstash input jms. I modified the jruby-jms-1.2.0-java library ( I modified acknowledgment mode ) JMS::Session::AUTO\_ACKNOWLEDGE TO JMS::Session::CLIENT\_ACKNOWLEDGE options: any of t… --- ## [Logstash to PostgreSQL](https://discuss.elastic.co/t/logstash-to-postgresql/93545)

**Author:** [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\ **Replies:** 2\ **Last updated:** [July 24, 2017, 9:18am UTC](https://discuss.elastic.co/t/logstash-to-postgresql/93545 "2017-07-24T09:18:50Z")

It there a way to send logs from logstash to postgresql? Thanks! :smiley: --- ## [How to improve the accurate of GeoIP](https://discuss.elastic.co/t/how-to-improve-the-accurate-of-geoip/93501)

**Author:** [@mercymx](https://discuss.elastic.co/u/mercymx)\ **Replies:** 2\ **Last updated:** [July 24, 2017, 7:56am UTC](https://discuss.elastic.co/t/how-to-improve-the-accurate-of-geoip/93501 "2017-07-24T07:56:01Z")

Hi there, I'm using GeoIP filter to add geographical information with the Maxmind GeoLite2 databases. However, I found this database has lots of mistake, the location is wrong for the IP from China in most of time. S… --- ## [Logstash docker java port \<1024 issue](https://discuss.elastic.co/t/logstash-docker-java-port-1024-issue/94128)

**Author:** [@amarchal](https://discuss.elastic.co/u/amarchal)\ **Replies:** 2\ **Last updated:** [July 24, 2017, 7:44am UTC](https://discuss.elastic.co/t/logstash-docker-java-port-1024-issue/94128 "2017-07-24T07:44:34Z")

Hello, Contexte: CentOS7 docker-CE Logstash official image: docker.elastic.co/logstash/logstash:5.5.0 SE Linux disabled When Starting Logstash container with input TCP port \< 1024, I have the following error: \[2017… --- ## [Logstash / Filebeat and "|" sparated logs](https://discuss.elastic.co/t/logstash-filebeat-and-sparated-logs/94123)

**Author:** [@basti](https://discuss.elastic.co/u/basti)\ **Replies:** 5\ **Last updated:** [July 24, 2017, 5:45am UTC](https://discuss.elastic.co/t/logstash-filebeat-and-sparated-logs/94123 "2017-07-24T05:45:35Z")

Hello, I just done my first steps with filebeat and logstash. I have some servers with log files with lines like: value1|value2|value3|..... Now i want them to be send via (1)FileBeat to (2)Logstash and then to (3)El… --- ## [No file output after I defined a file output?](https://discuss.elastic.co/t/no-file-output-after-i-defined-a-file-output/94254)

**Author:** [@KeithTt](https://discuss.elastic.co/u/KeithTt)\ **Replies:** 1\ **Last updated:** [July 24, 2017, 4:32am UTC](https://discuss.elastic.co/t/no-file-output-after-i-defined-a-file-output/94254 "2017-07-24T04:32:17Z")

ELK version: 5.4.1 I am referring to this page: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-file.html And here is my logstash pipeline config file: input { beats { port =\> 5044 codec… --- ## [Logstash can not start up after I update the license from trial to basic?](https://discuss.elastic.co/t/logstash-can-not-start-up-after-i-update-the-license-from-trial-to-basic/94266)

**Author:** [@KeithTt](https://discuss.elastic.co/u/KeithTt)\ **Replies:** 1\ **Last updated:** [July 24, 2017, 3:55am UTC](https://discuss.elastic.co/t/logstash-can-not-start-up-after-i-update-the-license-from-trial-to-basic/94266 "2017-07-24T03:55:08Z")

I followed this page to update the license: https://www.elastic.co/guide/en/x-pack/current/installing-license.html I delete the username and password in the pipeline config file of logstash, and don't modify anything e… --- ## [Create a field based on IP Range](https://discuss.elastic.co/t/create-a-field-based-on-ip-range/94139)

**Author:** [@nsur1](https://discuss.elastic.co/u/nsur1)\ **Replies:** 5\ **Last updated:** [July 23, 2017, 5:25pm UTC](https://discuss.elastic.co/t/create-a-field-based-on-ip-range/94139 "2017-07-23T17:25:56Z")

Hello everyone! I currently have logstash setup to take in flow data. I also had a file with the in the yaml format with ip-range (key), autonomous system number (value) pairs. I wanted to know how to use the lookup, fi… --- ## [Mongo::Auth::Unauthorized: User user is not authorized to access db](https://discuss.elastic.co/t/mongo-auth-unauthorized-user-user-is-not-authorized-to-access-db/94229)

**Author:** [@palmtown](https://discuss.elastic.co/u/palmtown)\ **Replies:** 0\ **Last updated:** [July 23, 2017, 5:38am UTC](https://discuss.elastic.co/t/mongo-auth-unauthorized-user-user-is-not-authorized-to-access-db/94229 "2017-07-23T05:38:54Z")

I am using mongodb as an output, however, I continue getting error Mongo::Auth::Unauthorized: User user is not authorized to access db. However, I verified user and pass is correct by connecting directly. :backtrace… --- ## [Sharepoint ULS multiline entries](https://discuss.elastic.co/t/sharepoint-uls-multiline-entries/92977)

**Author:** [@Blu3](https://discuss.elastic.co/u/Blu3)\ **Replies:** 4\ **Last updated:** [July 22, 2017, 7:14pm UTC](https://discuss.elastic.co/t/sharepoint-uls-multiline-entries/92977 "2017-07-22T19:14:58Z")

Hi I'm trying to configure Logstash to merge multiline messages into one, but I think this is a little different case than described on https://www.elastic.co/guide/en/logstash/current/multiline.html#\_timestamps Shar… --- ## [Input-jdbc Mysql error : undefined method 'close\_jdbc\_connection'](https://discuss.elastic.co/t/input-jdbc-mysql-error-undefined-method-close-jdbc-connection/93831)

**Author:** [@titovince35](https://discuss.elastic.co/u/titovince35)\ **Replies:** 2\ **Last updated:** [July 22, 2017, 11:37am UTC](https://discuss.elastic.co/t/input-jdbc-mysql-error-undefined-method-close-jdbc-connection/93831 "2017-07-22T11:37:57Z")

Hello all, I'm new to Elastic stack and I'm having a problem with input-jdbc on Mysql. 20:13:26.528 \[\[main\]\ **Author:** [@sushil1016](https://discuss.elastic.co/u/sushil1016)\ **Replies:** 1\ **Last updated:** [July 22, 2017, 10:13am UTC](https://discuss.elastic.co/t/logstash-raising-json-parse-exception-when-input-events-are-coming-from-elastisearch/94195 "2017-07-22T10:13:33Z") here is my conf input { # Read all documents from Elasticsearch matching the given query elasticsearch { hosts =\> "localhost:9200" index =\> "77777" query =\> '{ "query": { "match": { "func\_name": "MDM\_Tuner… --- ## [Logstash Periodicpoller Exception](https://discuss.elastic.co/t/logstash-periodicpoller-exception/94194)

**Author:** [@swatititame](https://discuss.elastic.co/u/swatititame)\ **Replies:** 0\ **Last updated:** [July 22, 2017, 8:17am UTC](https://discuss.elastic.co/t/logstash-periodicpoller-exception/94194 "2017-07-22T08:17:21Z")

I am getting periodicpoller exception in logstash command propmt. Please suggest me some solution for it. \[2017-07-21T22:15:03,650\]\[ERROR\]\[logstash.instrument.periodicpoller.jvm\] Periodi cPoller: exception {:poller=\>#\ **Author:** [@sushil1016](https://discuss.elastic.co/u/sushil1016)\ **Replies:** 1\ **Last updated:** [July 22, 2017, 3:32am UTC](https://discuss.elastic.co/t/logstash-not-able-to-read-data-from-elasticsearch/94185 "2017-07-22T03:32:37Z") Hello There, I've a pre-populated elasticsearch and I am trying to use logstash to further analyse the data from elasticsearch. But whenever i run logstash, it never process any events, neither do i see anything in ela… --- ## [Using Softflowd to send IPFIX data to logstash](https://discuss.elastic.co/t/using-softflowd-to-send-ipfix-data-to-logstash/93083)

**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 2\ **Last updated:** [July 21, 2017, 5:43pm UTC](https://discuss.elastic.co/t/using-softflowd-to-send-ipfix-data-to-logstash/93083 "2017-07-21T17:43:37Z")

Looking at the docs it says I can use Softflowd to send IPFIX traffic to logstash: https://www.elastic.co/guide/en/logstash/current/plugins-codecs-netflow.html#\_description\_154 I can find any examples though of how to … --- ## [Does translate filter work with logstash 5.1.2?](https://discuss.elastic.co/t/does-translate-filter-work-with-logstash-5-1-2/93394)

**Author:** [@daaakeee](https://discuss.elastic.co/u/daaakeee)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 4:24pm UTC](https://discuss.elastic.co/t/does-translate-filter-work-with-logstash-5-1-2/93394 "2017-07-21T16:24:40Z")

Hello! I am trying to split the message field into array and create many columns according to the list I got.. Can I do this? logstash filter config is like below: grok { match =\> { "message" =\> "%{GREEDYDATA:data}" … --- ## [Getting logs from azure blog storage and pushing to ES through Logstash?](https://discuss.elastic.co/t/getting-logs-from-azure-blog-storage-and-pushing-to-es-through-logstash/93393)

**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 4:08pm UTC](https://discuss.elastic.co/t/getting-logs-from-azure-blog-storage-and-pushing-to-es-through-logstash/93393 "2017-07-21T16:08:48Z")

Hi Team, I am sending my txt file which is stored in azureblobstorage to ES directly with the help of logstash with this plugin. My config is like this: input { azureblob { storage\_account\_name =\> "… --- ## [Help with time based data](https://discuss.elastic.co/t/help-with-time-based-data/93105)

**Author:** [@ahongyun](https://discuss.elastic.co/u/ahongyun)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 4:07pm UTC](https://discuss.elastic.co/t/help-with-time-based-data/93105 "2017-07-21T16:07:53Z")

**Author:** [@Pranav1](https://discuss.elastic.co/u/Pranav1)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 4:01pm UTC](https://discuss.elastic.co/t/kv-filters-behavior-when-square-brackets-in-log-data/93358 "2017-07-21T16:01:22Z")

In logstash I am using the following filter- filter { kv { field\_split =\> "\\t" } } So here I am using tab as field split and the value split I have kept default (i.e. '=') Now I am sending a log like this- ani… --- ## [Throttle Moving Window](https://discuss.elastic.co/t/throttle-moving-window/93824)

**Author:** [@brian-spot](https://discuss.elastic.co/u/brian-spot)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 4:00pm UTC](https://discuss.elastic.co/t/throttle-moving-window/93824 "2017-07-21T16:00:24Z")

Is it possible to have a moving window to monitor events through throttle filter or something similar? Ex. If I run this throttle filter: if \[event\] == "SomeEvent" { throttle { after\_count =\> 2 period =\> 1… --- ## [Logstash altering defined Elasticsearch URL](https://discuss.elastic.co/t/logstash-altering-defined-elasticsearch-url/93066)

**Author:** [@Chris\_Gatto](https://discuss.elastic.co/u/Chris_Gatto)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 3:46pm UTC](https://discuss.elastic.co/t/logstash-altering-defined-elasticsearch-url/93066 "2017-07-21T15:46:17Z")

Hello, I have my Logstash configured with the following output: output { hosts =\> \["http://myhost/elasticsearch"\] } This is a valid URL, as I can cURL commands to Elasticsearch with it, such as curl "http://myhos… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=939) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=941)