# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=750 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 751 --- ## [Logstash not updating last run metadata file](https://discuss.elastic.co/t/logstash-not-updating-last-run-metadata-file/133876)
**Author:** [@MichaelDz](https://discuss.elastic.co/u/MichaelDz)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 1:14pm UTC](https://discuss.elastic.co/t/logstash-not-updating-last-run-metadata-file/133876 "2018-05-30T13:14:13Z")
In my Logstash I want to download from a database the most recent data using :sql\_last\_value in a query and tracking\_column option in conf file. I've set last\_run\_metadata\_path because I have 2 pipelines for the same ta… --- ## [Adding field after mutate rename statement](https://discuss.elastic.co/t/adding-field-after-mutate-rename-statement/133860)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 2\ **Last updated:** [May 30, 2018, 1:03pm UTC](https://discuss.elastic.co/t/adding-field-after-mutate-rename-statement/133860 "2018-05-30T13:03:55Z")
Hi is it possible to add "post processing" to an mutate operation within the same filter? i have a filter that works but i want to add a field if specific conditions are full filled, to mark the logged event as a possib… --- ## [Parsing jenkins xml using logstash](https://discuss.elastic.co/t/parsing-jenkins-xml-using-logstash/133863)
**Author:** [@qylw](https://discuss.elastic.co/u/qylw)\ **Replies:** 4\ **Last updated:** [May 30, 2018, 12:42pm UTC](https://discuss.elastic.co/t/parsing-jenkins-xml-using-logstash/133863 "2018-05-30T12:42:48Z")
Hey there, I'm trying to parse jenkins build.xml files via filebeat/logstash. The documents gets transferred from filebeat fine, I see the documents in kibana that contains the full content, but easy xpath: filter { x… --- ## [Logstash java link error](https://discuss.elastic.co/t/logstash-java-link-error/133690)
**Author:** [@Toony](https://discuss.elastic.co/u/Toony)\ **Replies:** 5\ **Last updated:** [May 30, 2018, 11:32am UTC](https://discuss.elastic.co/t/logstash-java-link-error/133690 "2018-05-30T11:32:19Z")
I encounter a problem starting logstash. Uncompressing 6.X.tar.gz (i.e not only the last 6.2.4 version of logstash) I have the following error message: \[ERROR\] 2018-05-29 15:45:29.898 \[main\] Logstash - java.lang.Illega… --- ## [How to filter values of first pipe delimiter](https://discuss.elastic.co/t/how-to-filter-values-of-first-pipe-delimiter/133811)
**Author:** [@santhosh.mahi](https://discuss.elastic.co/u/santhosh.mahi)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 7:41am UTC](https://discuss.elastic.co/t/how-to-filter-values-of-first-pipe-delimiter/133811 "2018-05-30T07:41:21Z")
Hi I'm new to elasticsearch, here i'm facing issue while extract field. this is the message displaying in kibana. 1234567890|2018-04-05 17:50:12|NON VOUCHER|30.9.12.3|eTopUP\_xyz\_0000\_1234567890| I want only field 1,f… --- ## [Cannot auto reload config , logstash-6.2.4](https://discuss.elastic.co/t/cannot-auto-reload-config-logstash-6-2-4/133374)
**Author:** [@w14569](https://discuss.elastic.co/u/w14569)\ **Replies:** 5\ **Last updated:** [May 30, 2018, 11:39am UTC](https://discuss.elastic.co/t/cannot-auto-reload-config-logstash-6-2-4/133374 "2018-05-30T11:39:46Z")
Hello, I start logstash with command line ./bin/logstash -f ./config/std.conf -r --config.reload.interval 2s when change the config std.conf , logstash cannot reload config automatic , it report error ： \[2018-05-… --- ## [Or condition doesn't work](https://discuss.elastic.co/t/or-condition-doesnt-work/133831)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 1\ **Last updated:** [May 30, 2018, 9:10am UTC](https://discuss.elastic.co/t/or-condition-doesnt-work/133831 "2018-05-30T09:10:30Z")
Hello everybody, I use logstash to parse these messages : 28/05/2018 16:01:06.716 (SERVICE) 29/05/2018/D T UE\_TECXXX\_J\_TN\_SSH005\_LOG\_PGAH8\_SIGN(21863)/UE\_TECXXX\_X\_LL\_OPE003\_TECHNIQUE PLA STATUS : PL Toutes les contrain… --- ## [KV plugin filter not working](https://discuss.elastic.co/t/kv-plugin-filter-not-working/133704)
**Author:** [@br14nb0](https://discuss.elastic.co/u/br14nb0)\ **Replies:** 4\ **Last updated:** [May 30, 2018, 8:43am UTC](https://discuss.elastic.co/t/kv-plugin-filter-not-working/133704 "2018-05-30T08:43:57Z")
This is my config filter grok { match =\> { "message" =\> "\<%{INT:number}\>%{SYSLOGTIMESTAMP:timestamp}%{SPACE}%{NOTSPACE:app}%{SPACE}%{NOTSPACE:logsource}\[%{WORD:status}\]%{SPACE}%{INT:eventid}%{SPACE}%{GREEDYDATA:mess}.%… --- ## [Extract a string from a field: Grok Pattern](https://discuss.elastic.co/t/extract-a-string-from-a-field-grok-pattern/133698)
**Author:** [@rahulnama](https://discuss.elastic.co/u/rahulnama)\ **Replies:** 1\ **Last updated:** [May 30, 2018, 8:28am UTC](https://discuss.elastic.co/t/extract-a-string-from-a-field-grok-pattern/133698 "2018-05-30T08:28:01Z")
Hello, INSERT INTO Salesforce(ID,AccountName) VALUES ('109','name) Now I need to extract Salesforce from the above log? How should I do that. Should I use the grok here. I'm aware of (?Some Pattern) What should be … --- ## [Need grok filter for my custom log files](https://discuss.elastic.co/t/need-grok-filter-for-my-custom-log-files/133821)
**Author:** [@sana1](https://discuss.elastic.co/u/sana1)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 8:26am UTC](https://discuss.elastic.co/t/need-grok-filter-for-my-custom-log-files/133821 "2018-05-30T08:26:58Z")
I have files in a directory where all log patterns are different, I have successfully configured beats with elastic search but nor being able to filter fields or data my data in one file of a directory looks like "GET … --- ## [Is there a way to easily evaluate theoretically the necessary heap size for Logstash?](https://discuss.elastic.co/t/is-there-a-way-to-easily-evaluate-theoretically-the-necessary-heap-size-for-logstash/133817)
**Author:** [@dr01](https://discuss.elastic.co/u/dr01)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 8:10am UTC](https://discuss.elastic.co/t/is-there-a-way-to-easily-evaluate-theoretically-the-necessary-heap-size-for-logstash/133817 "2018-05-30T08:10:13Z")
Hi all, Is there a way to easily evaluate theoretically the necessary heap size for Logstash, based - for instance - on payload size and number of ingested logs per second? Thanks. --- ## [Parsing iis log](https://discuss.elastic.co/t/parsing-iis-log/133700)
**Author:** [@neo](https://discuss.elastic.co/u/neo)\ **Replies:** 3\ **Last updated:** [May 30, 2018, 7:47am UTC](https://discuss.elastic.co/t/parsing-iis-log/133700 "2018-05-30T07:47:28Z")
I am new to grok function, can someone you help me to work this out my log look like : #Software: Microsoft Internet Information Services 7.5 #Version: 1.0 #Date: 2018-04-25 00:00:00 #Fields: date time s-ip cs-metho… --- ## [Logstash and metrcibeat fields are different](https://discuss.elastic.co/t/logstash-and-metrcibeat-fields-are-different/133802)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 6:56am UTC](https://discuss.elastic.co/t/logstash-and-metrcibeat-fields-are-different/133802 "2018-05-30T06:56:17Z")
Hi, I have 2 fields in my data, one created from LS grok filter and one crated by metribeat. From LS grok filter: beat.hostname index is : myindex-\* From metricbeat : beat.hostname index is : metricbeat-\* Now my ch… --- ## [Logstash-input-azureblob slow](https://discuss.elastic.co/t/logstash-input-azureblob-slow/133801)
**Author:** [@Tarun](https://discuss.elastic.co/u/Tarun)\ **Replies:** 0\ **Last updated:** [May 30, 2018, 6:28am UTC](https://discuss.elastic.co/t/logstash-input-azureblob-slow/133801 "2018-05-30T06:28:23Z")
Dear colleague, We are exporting the Azure application insight data to Azure storage. The data is available and we are using "logstash-input-azureblob" to push this data to ELK stack. The logstash config is as below in… --- ## [Logtash issue while inserting data](https://discuss.elastic.co/t/logtash-issue-while-inserting-data/133742)
**Author:** [@saineshwar\_bageri](https://discuss.elastic.co/u/saineshwar_bageri)\ **Replies:** 1\ **Last updated:** [May 30, 2018, 4:16am UTC](https://discuss.elastic.co/t/logtash-issue-while-inserting-data/133742 "2018-05-30T04:16:41Z")
Elasticsearch new version "elasticsearch-6.2.4" and logstash version "logstash-6.2.4". I am pushing data from SQL server to Elasticsearch using logstash. Getting Error. \[2018-05-29T23:55:19,493\]\[WARN \]\[logstash.output… --- ## [Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool](https://discuss.elastic.co/t/attempted-to-send-a-bulk-request-to-elasticsearch-but-no-there-are-no-living-connections-in-the-connection-pool/132712)
**Author:** [@kimileonis](https://discuss.elastic.co/u/kimileonis)\ **Replies:** 2\ **Last updated:** [May 30, 2018, 2:13am UTC](https://discuss.elastic.co/t/attempted-to-send-a-bulk-request-to-elasticsearch-but-no-there-are-no-living-connections-in-the-connection-pool/132712 "2018-05-30T02:13:26Z")
\[2018-05-21T07:16:47,046\]\[ERROR\]\[logstash.outputs.elasticsearch\] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error\_message=\>"Elasticsearch Unreachable: \[http:… --- ## [Extract subdomain from referrer in logstash](https://discuss.elastic.co/t/extract-subdomain-from-referrer-in-logstash/133729)
**Author:** [@Med](https://discuss.elastic.co/u/Med)\ **Replies:** 6\ **Last updated:** [May 30, 2018, 1:24am UTC](https://discuss.elastic.co/t/extract-subdomain-from-referrer-in-logstash/133729 "2018-05-30T01:24:26Z")
I use logstash to send apache log to elasticsearch. In the file config of logstash I have a filter grok { match =\> { "message" =\> "%{COMBINEDAPACHELOG}"} } In the list of fields of the obtained data I have the field … --- ## [Error: Cannot assign requested address (Docker)](https://discuss.elastic.co/t/error-cannot-assign-requested-address-docker/133586)
**Author:** [@Brandonmcgr](https://discuss.elastic.co/u/Brandonmcgr)\ **Replies:** 2\ **Last updated:** [May 29, 2018, 8:50pm UTC](https://discuss.elastic.co/t/error-cannot-assign-requested-address-docker/133586 "2018-05-29T20:50:37Z")
Hello, I'm running the Logtash image with the following: docker run -it --rm -v "$PWD":/config-dir logstash -f /config-dir/logstash.conf My logstash file is: input { beats{ host =\> "192.168.0.66" … --- ## [ERROR while running configuration file in logstash](https://discuss.elastic.co/t/error-while-running-configuration-file-in-logstash/133743)
**Author:** [@hema\_chandra](https://discuss.elastic.co/u/hema_chandra)\ **Replies:** 0\ **Last updated:** [May 29, 2018, 7:01pm UTC](https://discuss.elastic.co/t/error-while-running-configuration-file-in-logstash/133743 "2018-05-29T19:01:28Z")
Sending Logstash's logs to C:/Program Files/logstash/logs which is now configured via log4j2.properties 2018-05-30 00:21:07,733 main ERROR Unable to create file C:/Program Files/logstash/logs/logstash-plain.log java.io.… --- ## [Logstash not parsing json properly](https://discuss.elastic.co/t/logstash-not-parsing-json-properly/133720)
**Author:** [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\ **Replies:** 3\ **Last updated:** [May 29, 2018, 5:33pm UTC](https://discuss.elastic.co/t/logstash-not-parsing-json-properly/133720 "2018-05-29T17:33:15Z")
2018-05-29T06:25:02.589432+00:00 ovthick\_121 bpradhcp: {"msg": "200 GET /api/v1/sessions?fields=id%2Cchildren HTTP/1.1 reqid=UBRGU4FRB7FDXBB6U4EYGE763UE", "namespace": "rasdk.rest.logger", "http\_query": "fields=id%2Cchil… --- ## [Change default logstash @timestamp with a timestamp from my logs](https://discuss.elastic.co/t/change-default-logstash-timestamp-with-a-timestamp-from-my-logs/133717)
**Author:** [@Nithin\_Nk](https://discuss.elastic.co/u/Nithin_Nk)\ **Replies:** 3\ **Last updated:** [May 29, 2018, 4:52pm UTC](https://discuss.elastic.co/t/change-default-logstash-timestamp-with-a-timestamp-from-my-logs/133717 "2018-05-29T16:52:59Z")
Hi Have a json log file which looks like: {"tenant\_id":"e100118","component\_job\_id":56153,"component\_status":"ERROR","system\_id":"GBT204","business\_type":"Test","application\_id":"e100118tmn","error\_since\_dtm":"22-05-201… --- ## [Need help parsing json array](https://discuss.elastic.co/t/need-help-parsing-json-array/133576)
**Author:** [@vcn](https://discuss.elastic.co/u/vcn)\ **Replies:** 3\ **Last updated:** [May 29, 2018, 4:49pm UTC](https://discuss.elastic.co/t/need-help-parsing-json-array/133576 "2018-05-29T16:49:04Z")
Hello everyone, I need help with parsing some Logstash logs. Incoming JSON : { "data1": { "propriety\_Version": "1.0", "source": "onetwothree" }, "parent": { "parentname": "valuezxz", "childArray": \[{ "… --- ## [Email grok filter logstash](https://discuss.elastic.co/t/email-grok-filter-logstash/133564)
**Author:** [@rouchad\_rouchad](https://discuss.elastic.co/u/rouchad_rouchad)\ **Replies:** 4\ **Last updated:** [May 29, 2018, 4:10pm UTC](https://discuss.elastic.co/t/email-grok-filter-logstash/133564 "2018-05-29T16:10:28Z")
hello everyone, I want your help please I connected my mailbox to elasticsearch and made filters on messages that come from some senders using : grok debugger what i want is how to modify my code logstash .the filters … --- ## [Logstash crashes when ES isn't available](https://discuss.elastic.co/t/logstash-crashes-when-es-isnt-available/132390)
**Author:** [@CasMeiron](https://discuss.elastic.co/u/CasMeiron)\ **Replies:** 8\ **Last updated:** [May 29, 2018, 4:09pm UTC](https://discuss.elastic.co/t/logstash-crashes-when-es-isnt-available/132390 "2018-05-29T16:09:21Z")
Hello, I turned off my elasticsearch to test what happens when it fails (or isn't running). I want to handle this kind of behaviour, sending an alarm to zabbix. The weird thing happened. When I started my logstash, I g… --- ## [Logstash with multiple pipeline doesn't process messages](https://discuss.elastic.co/t/logstash-with-multiple-pipeline-doesnt-process-messages/128589)
**Author:** [@CasMeiron](https://discuss.elastic.co/u/CasMeiron)\ **Replies:** 4\ **Last updated:** [May 29, 2018, 4:09pm UTC](https://discuss.elastic.co/t/logstash-with-multiple-pipeline-doesnt-process-messages/128589 "2018-05-29T16:09:04Z")
Hi, I'm tryin' to use logstash with multiple pipelines. Running version 6.2.3. When I run the logstash using "-f test.conf" it works as expected, but when multiple pipelines are configured and logstash runs with "bin/l… --- ## [Limit to number of pipelines](https://discuss.elastic.co/t/limit-to-number-of-pipelines/133221)
**Author:** [@dalamar666](https://discuss.elastic.co/u/dalamar666)\ **Replies:** 3\ **Last updated:** [May 29, 2018, 4:08pm UTC](https://discuss.elastic.co/t/limit-to-number-of-pipelines/133221 "2018-05-29T16:08:20Z")
Hi, I have 5 pipelines on my machine. I have 8 CPU's and enough RAM and disk space for them. Only 3 of the pipelines are reporting data and creating indices. I have verified all of my configurations to make sure I di… --- ## [JSESSIONID should be seperated](https://discuss.elastic.co/t/jsessionid-should-be-seperated/133701)
**Author:** [@Arpit\_Gulati](https://discuss.elastic.co/u/Arpit_Gulati)\ **Replies:** 0\ **Last updated:** [May 29, 2018, 2:37pm UTC](https://discuss.elastic.co/t/jsessionid-should-be-seperated/133701 "2018-05-29T14:37:22Z")
I have a log 192.168.23.56 www.jdomni.com 192.168.23.58 \[29/May/2018:04:32:53 +0530\] "GET /marketplace/login.service;JDSESSIONID=BF322E7FE8EEC70D66505838E12A486F HTTP/1.1" 302 0 \[29/May/2018:04:32:53 +0530\] 0.012 0.012 "… --- ## [Codec only support first pipeline](https://discuss.elastic.co/t/codec-only-support-first-pipeline/133435)
**Author:** [@wanglifeng](https://discuss.elastic.co/u/wanglifeng)\ **Replies:** 1\ **Last updated:** [May 29, 2018, 1:25pm UTC](https://discuss.elastic.co/t/codec-only-support-first-pipeline/133435 "2018-05-29T13:25:45Z")
I tried to use multi-pipeline to do log collection, parse and output kafka, But the log's encoding are different (i.e: UTF-8 GBK etc). i tried use pipeline.yml to setup several log inputs and listening 514, 515 udp por… --- ## [How to unzip s3 log files in Logstash](https://discuss.elastic.co/t/how-to-unzip-s3-log-files-in-logstash/133679)
**Author:** [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Replies:** 4\ **Last updated:** [May 29, 2018, 1:16pm UTC](https://discuss.elastic.co/t/how-to-unzip-s3-log-files-in-logstash/133679 "2018-05-29T13:16:52Z")
Hi, I have to load s3 log files into an ES. So I read log files from S3 through LogStash. But the problem is, the log files are in zip file format. How to unzip these log files in LogStash? Thanks, --- ## [Logstash with X-PACK Cannot reload pipeline, because the existing pipeline is not reloadable](https://discuss.elastic.co/t/logstash-with-x-pack-cannot-reload-pipeline-because-the-existing-pipeline-is-not-reloadable/132926)
**Author:** [@wuyunfeng](https://discuss.elastic.co/u/wuyunfeng)\ **Replies:** 4\ **Last updated:** [May 29, 2018, 12:56pm UTC](https://discuss.elastic.co/t/logstash-with-x-pack-cannot-reload-pipeline-because-the-existing-pipeline-is-not-reloadable/132926 "2018-05-29T12:56:44Z")
Enviroment: Logstash version: 6.2.4 Elasticsearch version: 6.2.4 kibana: 6.2.4 logstash.yml: \> config.reload.automatic: true \> config.reload.interval: 3s \> pipeline.id: elk\_logstash\_01 \> pipeline. reloadable: true \> … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=749) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=751)