ElasticSearch + Kibana + Logstash Config in Windows

I am working on configuring Elasticsearch with Kibana and Logstash. My goal is to be able to import AWS S3 Bucket Access logs, cloudtrail and other aws logs into Elastic either by manual import or via logstash.

When I manually load an aws file I see some fields (5) and the others are just labeled field. Do I need to create some sort of mapping or index first?

LogStash gives these errors:

Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :exception=>Manticore::ClientProtocolException, :cause=>#<Java::JavaxNetSsl::SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>}

ALSO:

Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@x.x.x.x:9200/", :exception=>LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://x.x.x.x:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}

Update: My Logstash now loops:

I am running logstash version 8.9.1
[2023-09-08T22:05:38,932][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=1335924351} forced-compaction result (captures: 3 span: PT10.0136766S)

[2023-09-08T22:05:38,932][DEBUG][org.logstash.instrument.metrics.ExtendedFlowMetric] RetentionWindow{policy=current id=346568156} forced-compaction result (captures: 3 span: PT10.014082599S)

[2023-09-08T22:05:42,416][DEBUG][org.logstash.execution.PeriodicFlush][main] Pushing flush onto pipeline.

[2023-09-08T22:05:43,468][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu

[2023-09-08T22:05:43,558][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Young Generation"}

[2023-09-08T22:05:43,558][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"G1 Old Generation"}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.