# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=392 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 393 --- ## [Array of nested json objects](https://discuss.elastic.co/t/array-of-nested-json-objects/222363)
**Author:** [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Replies:** 6\ **Last updated:** [March 10, 2020, 6:25pm UTC](https://discuss.elastic.co/t/array-of-nested-json-objects/222363 "2020-03-10T18:25:25Z")
Hope this is not a repeat. I have a document like following in json format. { "request\_body": "", "bytes\_sent": 16, "response\_body": "", "response\_http\_headers": \[ { "Content-Type": "application/… --- ## [Query against REST API of a third party device](https://discuss.elastic.co/t/query-against-rest-api-of-a-third-party-device/222979)
**Author:** [@rudiz](https://discuss.elastic.co/u/rudiz)\ **Replies:** 1\ **Last updated:** [March 10, 2020, 5:14pm UTC](https://discuss.elastic.co/t/query-against-rest-api-of-a-third-party-device/222979 "2020-03-10T17:14:18Z")
I'm trying to query a storage device via RESTful API. I have to send a POST against the endpoint /auth with User and Password in the header to get a token. This token remains valid for 2hours or 30minutes if no activity.… --- ## [Remove first and last character of all fields](https://discuss.elastic.co/t/remove-first-and-last-character-of-all-fields/222897)
**Author:** [@vertigo888](https://discuss.elastic.co/u/vertigo888)\ **Replies:** 4\ **Last updated:** [March 10, 2020, 5:07pm UTC](https://discuss.elastic.co/t/remove-first-and-last-character-of-all-fields/222897 "2020-03-10T17:07:41Z")
Hi, I would like to replace the first and last character of all values in all fields ingested into my index via logstash? Please could you let me know how this is possible? I understand this may be a ruby task but plea… --- ## [Testing Logstash configurations as systemd service](https://discuss.elastic.co/t/testing-logstash-configurations-as-systemd-service/222982)
**Author:** [@rudiz](https://discuss.elastic.co/u/rudiz)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 4:56pm UTC](https://discuss.elastic.co/t/testing-logstash-configurations-as-systemd-service/222982 "2020-03-10T16:56:31Z")
I like the comfort of all components running as systemd service but for Logstash testing it is difficult as the stdout output is not directed to my terminal and sometimes hard to find in messages. For testing only: what… --- ## [Parse MSSQL logs from event log](https://discuss.elastic.co/t/parse-mssql-logs-from-event-log/222980)
**Author:** [@sref](https://discuss.elastic.co/u/sref)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 4:42pm UTC](https://discuss.elastic.co/t/parse-mssql-logs-from-event-log/222980 "2020-03-10T16:42:33Z")
Hi, Im trying to parse MSSQL logs from Windows event log, into Elastic stack. Currently MSSQL logs to event viewer, and i use Winlogbeat 7,6 and Elastic stack 7,6 also. (installed under 1 month ago) indent preformatte… --- ## [F5 Telemetry Streaming using Logstash](https://discuss.elastic.co/t/f5-telemetry-streaming-using-logstash/222970)
**Author:** [@ginu](https://discuss.elastic.co/u/ginu)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 4:03pm UTC](https://discuss.elastic.co/t/f5-telemetry-streaming-using-logstash/222970 "2020-03-10T16:03:50Z")
Hi has anyone successfully loaded the bigIP Telemetry Streaming to ES using Logstash. I have tried the following pipeline config: input { tcp { port =\> 5053 codec =\> "plain" } } filter { if \[message… --- ## [Copy different field with nanoseconds to elasticsearch/kibana @timestamp field](https://discuss.elastic.co/t/copy-different-field-with-nanoseconds-to-elasticsearch-kibana-timestamp-field/222806)
**Author:** [@savvy](https://discuss.elastic.co/u/savvy)\ **Replies:** 4\ **Last updated:** [March 10, 2020, 3:55pm UTC](https://discuss.elastic.co/t/copy-different-field-with-nanoseconds-to-elasticsearch-kibana-timestamp-field/222806 "2020-03-10T15:55:47Z")
Just posting that here, seems like logstash related. Can someone guide me here? --- ## [Fixed date-time for all record](https://discuss.elastic.co/t/fixed-date-time-for-all-record/222808)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [March 10, 2020, 3:30pm UTC](https://discuss.elastic.co/t/fixed-date-time-for-all-record/222808 "2020-03-10T15:30:30Z")
I want to start ingesting record at lets say 10:05 and keep same timestamp to all the record. i.e first record come in at 2020-03-08T10:05 and last record at 2020-03-08T10:10 but I want same time for all record. it can… --- ## [Unable to create logstash index in Kibana](https://discuss.elastic.co/t/unable-to-create-logstash-index-in-kibana/222790)
**Author:** [@gmorales](https://discuss.elastic.co/u/gmorales)\ **Replies:** 6\ **Last updated:** [March 10, 2020, 3:17pm UTC](https://discuss.elastic.co/t/unable-to-create-logstash-index-in-kibana/222790 "2020-03-10T15:17:02Z")
I am a newbie, I inherited an older elasticsearch, kibana, logstash, filebeat server and it is currently working fine, however I wanted to run the latest and went ahead and installed learticsearch, kibana, logstash, file… --- ## [Error with .csv parsing from Nessus scanner to Logstash / CSV::MalformedCSVError: Illegal quoting in line 1](https://discuss.elastic.co/t/error-with-csv-parsing-from-nessus-scanner-to-logstash-csv-illegal-quoting-in-line-1/222957)
**Author:** [@denisatanasov](https://discuss.elastic.co/u/denisatanasov)\ **Replies:** 2\ **Last updated:** [March 10, 2020, 3:08pm UTC](https://discuss.elastic.co/t/error-with-csv-parsing-from-nessus-scanner-to-logstash-csv-illegal-quoting-in-line-1/222957 "2020-03-10T15:08:49Z")
Hello All, I am trying to index into Logstash, a .csv (comma delimited) file generated from my Nessus scanner. However, I am receiving error related most probably to parsing problems, because in the Excel file, there ar… --- ## [Logging for Logstash with pipelines.yml](https://discuss.elastic.co/t/logging-for-logstash-with-pipelines-yml/222937)
**Author:** [@furhans](https://discuss.elastic.co/u/furhans)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 12:58pm UTC](https://discuss.elastic.co/t/logging-for-logstash-with-pipelines-yml/222937 "2020-03-10T12:58:37Z")
Hello, I have been trying to run Logstash (v6.5.4) with multiple pipelines by following guide here: https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html My logstash pipeline.yml looks like this: - … --- ## [Logstash elasticsearch-filter disable ssl verification](https://discuss.elastic.co/t/logstash-elasticsearch-filter-disable-ssl-verification/222926)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 12:12pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-disable-ssl-verification/222926 "2020-03-10T12:12:11Z")
How to disable hostname verification in elasticsearch filter like below https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-ssl\_certificate\_verification --- ## [\[SOLVED\] Failed to execute PipelineAction::Create](https://discuss.elastic.co/t/solved-failed-to-execute-pipelineaction-create/222782)
**Author:** [@alesabat](https://discuss.elastic.co/u/alesabat)\ **Replies:** 3\ **Last updated:** [March 10, 2020, 11:40am UTC](https://discuss.elastic.co/t/solved-failed-to-execute-pipelineaction-create/222782 "2020-03-10T11:40:54Z")
Hi everyone! I'm running ELK on docker. Everything works fine, but, since I added some grok patterns to my pipeline, logstash never start again. This is the error message: \[2020-03-09T18:10:36,634\]\[ERROR\]\[logstash.ag… --- ## [Journalctl logstash has issue "logstash.service: main process exited, code=exited, status=1/FAILURE"](https://discuss.elastic.co/t/journalctl-logstash-has-issue-logstash-service-main-process-exited-code-exited-status-1-failure/222891)
**Author:** [@gilang\_bilbisri](https://discuss.elastic.co/u/gilang_bilbisri)\ **Replies:** 2\ **Last updated:** [March 10, 2020, 11:30am UTC](https://discuss.elastic.co/t/journalctl-logstash-has-issue-logstash-service-main-process-exited-code-exited-status-1-failure/222891 "2020-03-10T11:30:10Z")
Halo i want to ask something about my logstash configuration. i've already installed logstash and giving some configure in /etc/logstash/logstash-sample.conf and for the configuration is input { beats { port =\> 5044… --- ## [Remove timezone/timestamp from my data field?](https://discuss.elastic.co/t/remove-timezone-timestamp-from-my-data-field/222768)
**Author:** [@samargh](https://discuss.elastic.co/u/samargh)\ **Replies:** 6\ **Last updated:** [March 10, 2020, 10:17am UTC](https://discuss.elastic.co/t/remove-timezone-timestamp-from-my-data-field/222768 "2020-03-10T10:17:00Z")
how can i delete the timezone/timestamp from my data field ? i need it to get only : Disconnection Date Sep 16, 2020 not this !! Disconnection Date Sep 16, 2020 @ 23:00:00.000 Activation Date Aug 13, 2019 @ 23:00:00.… --- ## [Is this still valid syntax for grok filters?](https://discuss.elastic.co/t/is-this-still-valid-syntax-for-grok-filters/222908)
**Author:** [@calanon](https://discuss.elastic.co/u/calanon)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 10:15am UTC](https://discuss.elastic.co/t/is-this-still-valid-syntax-for-grok-filters/222908 "2020-03-10T10:15:27Z")
I want to make sure I identify the that the logs are coming from nginx before processing this grok filter, is this syntax still valid in version 7x? filter { if \[program\] == "nginx\_access" { grok { match… --- ## [Logstash jdbc\_streaming to keep the casing of fields?](https://discuss.elastic.co/t/logstash-jdbc-streaming-to-keep-the-casing-of-fields/222901)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 9:53am UTC](https://discuss.elastic.co/t/logstash-jdbc-streaming-to-keep-the-casing-of-fields/222901 "2020-03-10T09:53:43Z")
I use jdbc\_streaming filter to lookup from sql but the columns returned were in small caps. I understand there was a setting in jdbc input lowercase\_column\_names =\> false but theres nothing like this in jdbc\_streaming fi… --- ## [If else ruby check for nil](https://discuss.elastic.co/t/if-else-ruby-check-for-nil/222681)
**Author:** [@samiujan](https://discuss.elastic.co/u/samiujan)\ **Replies:** 2\ **Last updated:** [March 10, 2020, 9:51am UTC](https://discuss.elastic.co/t/if-else-ruby-check-for-nil/222681 "2020-03-10T09:51:51Z")
Hi I am using a ruby codeblock to add values from redis for specific cases - I am getting some null values and want to log those - but this check for null values and isn't working, I get a complication error - looking a… --- ## [Config filter not found when using Logstash docker image](https://discuss.elastic.co/t/config-filter-not-found-when-using-logstash-docker-image/222753)
**Author:** [@danielfmsouza](https://discuss.elastic.co/u/danielfmsouza)\ **Replies:** 2\ **Last updated:** [March 10, 2020, 9:45am UTC](https://discuss.elastic.co/t/config-filter-not-found-when-using-logstash-docker-image/222753 "2020-03-10T09:45:35Z")
When I run Logstash directly on my PC (Windows) it runs fine. Everything works. Then when I try to use logstash official image (docker.elastic.co/logstash/logstash:7.6.0) it gives me this error: \[2020-03-09T15:14:46,2… --- ## [Logstash jdbc plugin schedule](https://discuss.elastic.co/t/logstash-jdbc-plugin-schedule/222881)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 8:02am UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-schedule/222881 "2020-03-10T08:02:41Z")
Hey everyone, I was wondering if there is any possibility to query a database via logstash jdbc plugin in intervals that are lower than one second? Thanks for reply! --- ## [I have tried to ingested the new record ..but everytime the existing record is overriding with the new one.how to clear this?](https://discuss.elastic.co/t/i-have-tried-to-ingested-the-new-record-but-everytime-the-existing-record-is-overriding-with-the-new-one-how-to-clear-this/222863)
**Author:** [@chelseia](https://discuss.elastic.co/u/chelseia)\ **Replies:** 0\ **Last updated:** [March 10, 2020, 6:16am UTC](https://discuss.elastic.co/t/i-have-tried-to-ingested-the-new-record-but-everytime-the-existing-record-is-overriding-with-the-new-one-how-to-clear-this/222863 "2020-03-10T06:16:43Z")
input{ jdbc{ clean\_run =\> true jdbc\_driver\_library =\> "C:/Program Files/SQL Developer 4/jdbc/lib/ojdbc8.jar" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver" jdbc\_connection\_string =\> "..................… --- ## [How to transfer logs from different VM's (Digitalocen) to a single Logstash using Filebeat](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847)
**Author:** [@Pathshala\_Gopi](https://discuss.elastic.co/u/Pathshala_Gopi)\ **Replies:** 4\ **Last updated:** [March 10, 2020, 5:40am UTC](https://discuss.elastic.co/t/how-to-transfer-logs-from-different-vms-digitalocen-to-a-single-logstash-using-filebeat/222847 "2020-03-10T05:40:28Z")
Hi all, From Last two days i got stuck in this task.Task is there are multiple Ubuntu machines are running on cloud (DigitalOcen), I have to take the logs of those machines and ship those logs to Logstash where complete… --- ## [Logstash unable to run conf file in background?](https://discuss.elastic.co/t/logstash-unable-to-run-conf-file-in-background/222633)
**Author:** [@kopilim96](https://discuss.elastic.co/u/kopilim96)\ **Replies:** 7\ **Last updated:** [March 10, 2020, 2:11am UTC](https://discuss.elastic.co/t/logstash-unable-to-run-conf-file-in-background/222633 "2020-03-10T02:11:39Z")
Hi there, how can I run the logstash .conf file in background ? I am using Centos 6.9 and I installed logstash with rpm package. file: /etc/logstash/logstash.yml pipeline.id: logstash\_for\_nagios path.config: "/etc/log… --- ## [Logstash config for Haproxy and Nginx not working together](https://discuss.elastic.co/t/logstash-config-for-haproxy-and-nginx-not-working-together/222798)
**Author:** [@Sona\_Das](https://discuss.elastic.co/u/Sona_Das)\ **Replies:** 2\ **Last updated:** [March 9, 2020, 9:42pm UTC](https://discuss.elastic.co/t/logstash-config-for-haproxy-and-nginx-not-working-together/222798 "2020-03-09T21:42:21Z")
I am having a ELK setup for processing haproxy and nginx logs, for this i have used separate config files for logstash, the main data which i want from logs are the "content url" and the "response time", in haproxy the r… --- ## [Unable to create logstash indice](https://discuss.elastic.co/t/unable-to-create-logstash-indice/222788)
**Author:** [@gmorales](https://discuss.elastic.co/u/gmorales)\ **Replies:** 1\ **Last updated:** [March 9, 2020, 8:42pm UTC](https://discuss.elastic.co/t/unable-to-create-logstash-indice/222788 "2020-03-09T20:42:09Z")
I am a newbie and juts installed the latest elasticsearch, Kibana, filebeat, logstash, I inherited an old version of elartic, kibana and the whole 9 yards, the older one works just fine but I wanted to run the latest and… --- ## [Multiple Pipelines with the Logstash Helm Chart](https://discuss.elastic.co/t/multiple-pipelines-with-the-logstash-helm-chart/222359)
**Author:** [@dbwest](https://discuss.elastic.co/u/dbwest)\ **Replies:** 1\ **Last updated:** [March 6, 2020, 5:08pm UTC](https://discuss.elastic.co/t/multiple-pipelines-with-the-logstash-helm-chart/222359 "2020-03-06T17:08:10Z")
Hi, we have Logstash deployed using the Helm Chart at https://github.com/elastic/helm-charts/tree/master/logstash . We want to configure multiple pipelines. The values.yaml has these comments in it: ## NOTE: To achieve… --- ## [I cannot see two indices in Kibana](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755)
**Author:** [@calanon](https://discuss.elastic.co/u/calanon)\ **Replies:** 13\ **Last updated:** [March 9, 2020, 4:55pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755 "2020-03-09T16:55:27Z")
I have the following output in my logstash output file. When I look at the loaded indices in Kibana I only see the var\_log\_common-\* 30-elasticsearch-output.conf output { if \[source\] == "/var/log/nginx/access.log" or… --- ## [Format date field as a string?](https://discuss.elastic.co/t/format-date-field-as-a-string/222733)
**Author:** [@macau](https://discuss.elastic.co/u/macau)\ **Replies:** 3\ **Last updated:** [March 9, 2020, 4:36pm UTC](https://discuss.elastic.co/t/format-date-field-as-a-string/222733 "2020-03-09T16:36:05Z")
I would like to create a new string field from a date field (@timestamp) in the format "yyyy-MM-dd" (e.g. 2020-12-31) - is that possible with Logstash? --- ## [Input csv from website](https://discuss.elastic.co/t/input-csv-from-website/222765)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 0\ **Last updated:** [March 9, 2020, 4:34pm UTC](https://discuss.elastic.co/t/input-csv-from-website/222765 "2020-03-09T16:34:27Z")
I would like to parse a csv file from a website with logstash and the results put into elasticsearch. Something like this: input { http\_poller { urls =\> { test =\> { method =\> get url =\> "ht… --- ## [Grok parsing failure but working in the debugger](https://discuss.elastic.co/t/grok-parsing-failure-but-working-in-the-debugger/222609)
**Author:** [@lucia\_vargas\_a](https://discuss.elastic.co/u/lucia_vargas_a)\ **Replies:** 3\ **Last updated:** [March 9, 2020, 3:22pm UTC](https://discuss.elastic.co/t/grok-parsing-failure-but-working-in-the-debugger/222609 "2020-03-09T15:22:51Z")
Hi friends, Cn you help me? I am trying to parse a log and ingest to elasticsearch. One line looks like this: 2020-03-05 20:42:32 ERROR ConnectionError 404 1000 http://localhost/resources My config file looks like… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=391) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=393)