# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=810 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 811 --- ## [Match timestamps using different locales](https://discuss.elastic.co/t/match-timestamps-using-different-locales/122219)
**Author:** [@friesoft](https://discuss.elastic.co/u/friesoft)\ **Replies:** 3\ **Last updated:** [March 2, 2018, 3:20pm UTC](https://discuss.elastic.co/t/match-timestamps-using-different-locales/122219 "2018-03-02T15:20:13Z")
Hi, we have some logfiles which contain three different timestamp formats. Example: \[02/Mar/2018:09:35:21\] \[Info\] \[Module\] \[3085\] \[Function\] my message \[Fri Mar 02 09:35:21.474812 2018\] \[Module\] \[pid 1819:tid 140437… --- ## [Logstash Using Anonymous User](https://discuss.elastic.co/t/logstash-using-anonymous-user/122281)
**Author:** [@Lazarbeam](https://discuss.elastic.co/u/Lazarbeam)\ **Replies:** 0\ **Last updated:** [March 2, 2018, 2:56pm UTC](https://discuss.elastic.co/t/logstash-using-anonymous-user/122281 "2018-03-02T14:56:41Z")
Team -- We're running into a wall with Logstash. We receive a 401 unauthorized error at the health check event in the logs when starting Logstash with X-pack enabled and reaching out to Elasticsearch node over HTTPS: .… --- ## [Logging Levels](https://discuss.elastic.co/t/logging-levels/122280)
**Author:** [@peter1992](https://discuss.elastic.co/u/peter1992)\ **Replies:** 0\ **Last updated:** [March 2, 2018, 2:56pm UTC](https://discuss.elastic.co/t/logging-levels/122280 "2018-03-02T14:56:12Z")
Can anybody let me know what type of data do we get on the different type of logging in Logstash. We experience a problem where logstash stops collecting data but no logs are recorded in debug level --- ## [Is it possible to use the elasticsearch template to create a filter template in logstash](https://discuss.elastic.co/t/is-it-possible-to-use-the-elasticsearch-template-to-create-a-filter-template-in-logstash/122019)
**Author:** [@nlh](https://discuss.elastic.co/u/nlh)\ **Replies:** 3\ **Last updated:** [March 2, 2018, 2:37pm UTC](https://discuss.elastic.co/t/is-it-possible-to-use-the-elasticsearch-template-to-create-a-filter-template-in-logstash/122019 "2018-03-02T14:37:56Z")
I'm using auditbeat to send syslog messages to logstash and then onto elasticsearch. I have deployed the template from auditbeat to elasticsearch and the messages being sent through are being correctly parsed and show ni… --- ## [Guidance on filtering in the logstash.conf file](https://discuss.elastic.co/t/guidance-on-filtering-in-the-logstash-conf-file/121250)
**Author:** [@nlh](https://discuss.elastic.co/u/nlh)\ **Replies:** 6\ **Last updated:** [March 2, 2018, 2:36pm UTC](https://discuss.elastic.co/t/guidance-on-filtering-in-the-logstash-conf-file/121250 "2018-03-02T14:36:19Z")
Hi, I've spent a couple of days looking at filtering and probably just cannot get the basic nuances for what I want to do to make it work. Basic set up. I have an auditbeat running on a linux server sending over to logs… --- ## [Is there a mistake?](https://discuss.elastic.co/t/is-there-a-mistake/122102)
**Author:** [@Bakkali\_Amine](https://discuss.elastic.co/u/Bakkali_Amine)\ **Replies:** 19\ **Last updated:** [March 2, 2018, 11:41am UTC](https://discuss.elastic.co/t/is-there-a-mistake/122102 "2018-03-02T11:41:41Z")
is there something wrong with this config : input { file { type =\> "wazuh-alerts" path =\> "/var/ossec/logs/alerts/alerts.json" codec =\> "json" } beats { port =\> 5044 type =\> "apache" } } filter { if \[… --- ## [Time difference between two fields in a csv using ruby plugin](https://discuss.elastic.co/t/time-difference-between-two-fields-in-a-csv-using-ruby-plugin/122115)
**Author:** [@donovans](https://discuss.elastic.co/u/donovans)\ **Replies:** 2\ **Last updated:** [March 2, 2018, 10:51am UTC](https://discuss.elastic.co/t/time-difference-between-two-fields-in-a-csv-using-ruby-plugin/122115 "2018-03-02T10:51:40Z")
Hi all New to the forum ,ive been trying to get time difference between two fields in a csv using ruby plugin but i am new to logstash and using ruby can somebody please assist with the syntax of the ruby filter (runni… --- ## [WEB log config file](https://discuss.elastic.co/t/web-log-config-file/122080)
**Author:** [@Rahul07](https://discuss.elastic.co/u/Rahul07)\ **Replies:** 6\ **Last updated:** [March 2, 2018, 9:50am UTC](https://discuss.elastic.co/t/web-log-config-file/122080 "2018-03-02T09:50:21Z")
HI i am new to ELK i have created indices and deleted through Kibana by query and manually from C:\\ELK\\elasticsearch-5.6.3\\elasticsearch-5.6.3\\data\\nodes\\0\\indices. when i try to create again indices with config file m… --- ## [Can I add leading zeros to a field?](https://discuss.elastic.co/t/can-i-add-leading-zeros-to-a-field/122081)
**Author:** [@david\_reynolds](https://discuss.elastic.co/u/david_reynolds)\ **Replies:** 4\ **Last updated:** [March 2, 2018, 9:06am UTC](https://discuss.elastic.co/t/can-i-add-leading-zeros-to-a-field/122081 "2018-03-02T09:06:01Z")
Question about Logstash (Elastic Stack). Can someone give me a pointer on how to add a new field (add\_field) that will format a number with leading zeroes? e.g. given somefield == 123 then someotherfield == "00123". --- ## [Logstash config file on docker](https://discuss.elastic.co/t/logstash-config-file-on-docker/122194)
**Author:** [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\ **Replies:** 1\ **Last updated:** [March 2, 2018, 7:13am UTC](https://discuss.elastic.co/t/logstash-config-file-on-docker/122194 "2018-03-02T07:13:44Z")
I am new to both elk and docker Wanted to know how could I run the config file of logstash and link it to elasticsearch for further analysis. I have all the images pulled on the server and elasticsearch and kibana are … --- ## [Manipulating XML before hitting XML Filter](https://discuss.elastic.co/t/manipulating-xml-before-hitting-xml-filter/119732)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 32\ **Last updated:** [March 2, 2018, 6:33am UTC](https://discuss.elastic.co/t/manipulating-xml-before-hitting-xml-filter/119732 "2018-03-02T06:33:53Z")
Alright, I have a couple changes I want to make prior to an XML file hitting the XML parser. Strip versioning/encoding statement from the file Ensure tags \ and \ are placed on their own lines if the… --- ## [WinlogBeat -\> LogStash XML filter](https://discuss.elastic.co/t/winlogbeat-logstash-xml-filter/119422)
**Author:** [@florianmulatz](https://discuss.elastic.co/u/florianmulatz)\ **Replies:** 5\ **Last updated:** [March 2, 2018, 6:15am UTC](https://discuss.elastic.co/t/winlogbeat-logstash-xml-filter/119422 "2018-03-02T06:15:28Z")
Hey Guys, Just asking myself how to configure the XML-Filter in logstash but not for a file input but a winlogbeat input. Background: We're logging the ADFS Audit from our Win-ADFS Servers via Winlogbeat to elastic. T… --- ## [Problem with cipher not available](https://discuss.elastic.co/t/problem-with-cipher-not-available/117987)
**Author:** [@iDavico](https://discuss.elastic.co/u/iDavico)\ **Replies:** 9\ **Last updated:** [March 2, 2018, 1:18am UTC](https://discuss.elastic.co/t/problem-with-cipher-not-available/117987 "2018-03-02T01:18:22Z")
Hi, I have a problem with logstash 5.3.3. When i start Logstash, the folowing error is generated: Pipeline aborted due to error {:exception=\>#\ **Author:** [@pjh](https://discuss.elastic.co/u/pjh)\ **Replies:** 1\ **Last updated:** [March 1, 2018, 10:21pm UTC](https://discuss.elastic.co/t/parsing-multiple-file-and-adding-service-name/122135 "2018-03-01T22:21:28Z") hey! I am currently using logstash-6.0.0. I have a microservices architecture where I store all my logs in /var/log/mservice/ and ops infrastrucutre (kibana, elasticsearch, logstash, grafana, etc) in /var/log/ops/ I ha… --- ## [What is the best way to ship logs from one Logstash to another Logstash?](https://discuss.elastic.co/t/what-is-the-best-way-to-ship-logs-from-one-logstash-to-another-logstash/120359)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 9\ **Last updated:** [March 1, 2018, 10:10pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-ship-logs-from-one-logstash-to-another-logstash/120359 "2018-03-01T22:10:33Z")
Hi, What would be the best way to ship logs from one Logstash to another instance of Logstash? I am currently using http input and http output plugins. However, I am not sure if all the logs are being shipped without an… --- ## [Filebeat fields are not working](https://discuss.elastic.co/t/filebeat-fields-are-not-working/122155)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 3\ **Last updated:** [March 1, 2018, 9:54pm UTC](https://discuss.elastic.co/t/filebeat-fields-are-not-working/122155 "2018-03-01T21:54:21Z")
Here is my filebeat config: filebeat.prospectors: - type: log paths: - C:/inetpub/app01/logs/IIS/\*/\*.log fields: app\_group: iis app\_id: app01 - type: log paths: - C:/inetpub/app02/logs/IIS/\*/\*.log… --- ## [Logstash twitter input - ubuntu server](https://discuss.elastic.co/t/logstash-twitter-input-ubuntu-server/122142)
**Author:** [@angeliczka](https://discuss.elastic.co/u/angeliczka)\ **Replies:** 2\ **Last updated:** [March 1, 2018, 8:22pm UTC](https://discuss.elastic.co/t/logstash-twitter-input-ubuntu-server/122142 "2018-03-01T20:22:04Z")
Hello I am a newbie and I am trying to deploy an elastic stack on a server for the first time, running an ubuntu VM. Elasticsearch and Kibana are working fine, however, I am not getting any indices from logstash, doesn'… --- ## [Import data from MySQL to elasticsearch with logstash after preprocessing](https://discuss.elastic.co/t/import-data-from-mysql-to-elasticsearch-with-logstash-after-preprocessing/122020)
**Author:** [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)\ **Replies:** 1\ **Last updated:** [March 1, 2018, 8:07pm UTC](https://discuss.elastic.co/t/import-data-from-mysql-to-elasticsearch-with-logstash-after-preprocessing/122020 "2018-03-01T20:07:51Z")
I am trying to import data from MySQL to elasticsearch using logstash, everything works fine and I have all the data imported well. However, one of the fields in MySQL called "metadata" is following a specific pattern li… --- ## [LogStash multiple files](https://discuss.elastic.co/t/logstash-multiple-files/122144)
**Author:** [@ShashiK](https://discuss.elastic.co/u/ShashiK)\ **Replies:** 2\ **Last updated:** [March 1, 2018, 8:03pm UTC](https://discuss.elastic.co/t/logstash-multiple-files/122144 "2018-03-01T20:03:10Z")
I am restarting this question as previously i was moved to a different task so now back on this. Logstash works for me when i have a direct file reference - When I use the single file reference in the logstash config- … --- ## [Grok everything until a character and include that character in a field](https://discuss.elastic.co/t/grok-everything-until-a-character-and-include-that-character-in-a-field/122122)
**Author:** [@pdna](https://discuss.elastic.co/u/pdna)\ **Replies:** 1\ **Last updated:** [March 1, 2018, 8:02pm UTC](https://discuss.elastic.co/t/grok-everything-until-a-character-and-include-that-character-in-a-field/122122 "2018-03-01T20:02:37Z")
{"level":"error","message":"Token Generation Failed {\\"error\\":{}}","timestamp":"2017-11-15T17:58:00.402Z"} In the message field, I want to extract everything until { character as message\_type and store everything start… --- ## [Parse sonarqube logs](https://discuss.elastic.co/t/parse-sonarqube-logs/122130)
**Author:** [@kunaljangade](https://discuss.elastic.co/u/kunaljangade)\ **Replies:** 5\ **Last updated:** [March 1, 2018, 7:52pm UTC](https://discuss.elastic.co/t/parse-sonarqube-logs/122130 "2018-03-01T19:52:32Z")
Hi, I have been using logstash for a month now and diving deep into its filters. I am facing issues with parsing sonarqube logs. I have configured filebeat to send 5 log files that are generated by sonarqube and added f… --- ## [Add an array in an array](https://discuss.elastic.co/t/add-an-array-in-an-array/118901)
**Author:** [@uderline](https://discuss.elastic.co/u/uderline)\ **Replies:** 2\ **Last updated:** [March 1, 2018, 4:29pm UTC](https://discuss.elastic.co/t/add-an-array-in-an-array/118901 "2018-03-01T16:29:37Z")
Hello ! I'm working with the 5.6 elastic stack. I would like to be able to include an array in an array when doing a add\_field using Logstash. mutate { add\_field =\> { "weight" =\> \[ \[0.4, 0.6\], \[0.3, 0.8\], \[0.6, 0.9… --- ## [Logstash launch on Windows 7](https://discuss.elastic.co/t/logstash-launch-on-windows-7/122106)
**Author:** [@cleaner](https://discuss.elastic.co/u/cleaner)\ **Replies:** 0\ **Last updated:** [March 1, 2018, 3:12pm UTC](https://discuss.elastic.co/t/logstash-launch-on-windows-7/122106 "2018-03-01T15:12:26Z")
Hi, Maybe it will help someone. I installed Logstash-6.2.2 on Windows 7 for testing. Starting Logstash just did nothing: no error, no success. If you face the issue just add exit /b 0 as last line of logstash-6.2.2… --- ## [What is Greedydata](https://discuss.elastic.co/t/what-is-greedydata/122078)
**Author:** [@Srikanth\_FT](https://discuss.elastic.co/u/Srikanth_FT)\ **Replies:** 1\ **Last updated:** [March 1, 2018, 2:33pm UTC](https://discuss.elastic.co/t/what-is-greedydata/122078 "2018-03-01T14:33:29Z")
Hi, Can someone explain me what is greedydata. Best Mudireddy --- ## [How to remove element value in array field](https://discuss.elastic.co/t/how-to-remove-element-value-in-array-field/122006)
**Author:** [@Eswar\_Kumar\_Musiboin](https://discuss.elastic.co/u/Eswar_Kumar_Musiboin)\ **Replies:** 3\ **Last updated:** [March 1, 2018, 2:30pm UTC](https://discuss.elastic.co/t/how-to-remove-element-value-in-array-field/122006 "2018-03-01T14:30:19Z")
hi, i want to remove an element from tags array in logstash . logstash conf: filter { kv{ source =\> "message" field\_split =\> "," } date { match =\> \[ "joiningdate", "MMMM dd yyyy H… --- ## [Logstash Grok break\_on\_filter not working](https://discuss.elastic.co/t/logstash-grok-break-on-filter-not-working/122077)
**Author:** [@vijs](https://discuss.elastic.co/u/vijs)\ **Replies:** 0\ **Last updated:** [March 1, 2018, 1:32pm UTC](https://discuss.elastic.co/t/logstash-grok-break-on-filter-not-working/122077 "2018-03-01T13:32:01Z")
Hi My Logstructure is like this: Thu 11/09/17 15:05:44 11-09-2017 03:40:53 - :: | Error: 3303 | Data1 | 123456 6489 Description1 11-09-2017 03:40:54 - :: | Error: 3303 | Data2 | 123456 6489 Description2 … --- ## [Filter does not seem to supply logs for kibana](https://discuss.elastic.co/t/filter-does-not-seem-to-supply-logs-for-kibana/121932)
**Author:** [@irb7501](https://discuss.elastic.co/u/irb7501)\ **Replies:** 4\ **Last updated:** [March 1, 2018, 1:40pm UTC](https://discuss.elastic.co/t/filter-does-not-seem-to-supply-logs-for-kibana/121932 "2018-03-01T13:40:04Z")
I have an IIS filter that seems to test fine using the grok debugger however I am not seeing any messages indexed in Kibana via filebeat. Here is my beats.conf input{ beats{ port =\> "5043" } } filter { if \[type\] … --- ## [Logstash workflow with calculation](https://discuss.elastic.co/t/logstash-workflow-with-calculation/122063)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 0\ **Last updated:** [March 1, 2018, 1:14pm UTC](https://discuss.elastic.co/t/logstash-workflow-with-calculation/122063 "2018-03-01T13:14:33Z")
I am looking to implement the following workflow: Take data from ActiveMQ with passes me a number of fields. One field that I get passed is called APID. APID is sent with each JSON message and can be equal to a number. … --- ## [Can't create a new index in elasticsearch](https://discuss.elastic.co/t/cant-create-a-new-index-in-elasticsearch/122040)
**Author:** [@geppo](https://discuss.elastic.co/u/geppo)\ **Replies:** 1\ **Last updated:** [March 1, 2018, 12:07pm UTC](https://discuss.elastic.co/t/cant-create-a-new-index-in-elasticsearch/122040 "2018-03-01T12:07:36Z")
hi, i'm new in logstash and I would try to index my apache logs in elasticsearch 5.4.3. I have followed this guide on official es blog: https://www.elastic.co/blog/logstash\_lesson\_elasticsearch\_mapping When I try to cre… --- ## [Index pattern issues](https://discuss.elastic.co/t/index-pattern-issues/122039)
**Author:** [@Bakkali\_Amine](https://discuss.elastic.co/u/Bakkali_Amine)\ **Replies:** 3\ **Last updated:** [March 1, 2018, 11:31am UTC](https://discuss.elastic.co/t/index-pattern-issues/122039 "2018-03-01T11:31:19Z")
when adding codec = jason\_lines can't add an index in elasticsearc [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=809) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=811)