Kibana - Not showing any data on Discover

Elastic Stack Kibana
1

Kibana / logstash are running and Elastic search nodes are accessible from Kibana server. But when I check on Discover, there is nothing to view other than "Add integrations" as shown below.

image
image1292×814 76.8 KB

Below is what we got in the kibana.yml

server.port: 5601

 
server.host: "20.332.11.90"
 
server.publicBaseUrl: "https://20.332.11.90:5601"
 
server.name: "kibana01"
 
server.ssl.enabled: true
server.ssl.certificate: "C:\\elastic\\kibana-8.6.2\\config\\kibana01.crt"
server.ssl.key: "C:\\elastic\\kibana-8.6.2\\config\\kibana01.key"
 
elasticsearch.hosts: ["https://20.332.11.91:9200", "https://20.332.11.92:9200", "https://20.332.11.93:9200"]
 
elasticsearch.username: "kibana_system"
elasticsearch.password: "########"

 
elasticsearch.ssl.certificateAuthorities: "C:\\elastic\\kibana-8.6.2\\config\\elasticsearch-ca.pem"

 
logging.root.level: info
logging.appenders.default:
  type: rolling-file
  fileName: "D:/kibana-8.6.2/logs/kibana.log"
  layout:
    type: pattern        
  policy:
    type: size-limit
    size: 100mb

And below is the Kibana log

2023-08-24T16:34:12.823+02:00][INFO ][node] Kibana process configured with roles: [background_tasks, ui]
[2023-08-24T16:34:37.046+02:00][INFO ][plugins-service] Plugin "cloudChat" is disabled.
[2023-08-24T16:34:37.046+02:00][INFO ][plugins-service] Plugin "cloudExperiments" is disabled.
[2023-08-24T16:34:37.046+02:00][INFO ][plugins-service] Plugin "cloudFullStory" is disabled.
[2023-08-24T16:34:37.046+02:00][INFO ][plugins-service] Plugin "cloudGainsight" is disabled.
[2023-08-24T16:34:37.053+02:00][INFO ][plugins-service] Plugin "profiling" is disabled.
[2023-08-24T16:34:37.171+02:00][INFO ][http.server.Preboot] http server running at https://10.112.11.82:5601
[2023-08-24T16:34:37.261+02:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2023-08-24T16:34:37.314+02:00][WARN ][config.deprecation] The default mechanism for Reporting privileges will work differently in future versions, which will affect the behavior of this cluster. Set "xpack.reporting.roles.enabled" to "false" to adopt the future behavior before upgrading.
[2023-08-24T16:34:37.652+02:00][INFO ][plugins-system.standard] Setting up [127] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,usageCollection,taskManager,cloud,guidedOnboarding,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,screenshotMode,banners,newsfeed,fieldFormats,expressions,screenshotting,dataViews,embeddable,uiActionsEnhanced,charts,esUiShared,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,advancedSettings,spaces,security,snapshotRestore,lists,encryptedSavedObjects,telemetry,licenseManagement,files,eventLog,actions,notifications,console,bfetch,data,watcher,reporting,fileUpload,ingestPipelines,alerting,unifiedSearch,unifiedFieldList,savedSearch,savedObjects,graph,savedObjectsTagging,savedObjectsManagement,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,eventAnnotation,dataViewFieldEditor,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,discover,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,rollup,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,dashboardEnhanced,expressionXY,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,dataVisualizer,cases,timelines,sessionView,kubernetesSecurity,threatIntelligence,osquery,observability,aiops,ml,synthetics,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,visTypeGauge,dataViewManagement]
[2023-08-24T16:34:37.673+02:00][INFO ][plugins.taskManager] TaskManager is identified by the Kibana UUID: bc35aa1a-6c29-4f7d-8700-7b3ee5a8c818
[2023-08-24T16:34:37.767+02:00][WARN ][plugins.security.config] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-08-24T16:34:37.795+02:00][WARN ][plugins.security.config] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-08-24T16:34:37.810+02:00][WARN ][plugins.encryptedSavedObjects] Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-08-24T16:34:37.822+02:00][WARN ][plugins.actions] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-08-24T16:34:37.829+02:00][INFO ][plugins.notifications] Email Service Error: Email connector not specified.
[2023-08-24T16:34:37.937+02:00][WARN ][plugins.reporting.config] Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-08-24T16:34:37.943+02:00][WARN ][plugins.alerting] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
[2023-08-24T16:34:38.001+02:00][INFO ][plugins.ruleRegistry] Installing common resources shared between all indices
[2023-08-24T16:34:38.052+02:00][INFO ][plugins.cloudSecurityPosture] Registered task successfully [Task: cloud_security_posture-stats_task]
[2023-08-24T16:34:38.605+02:00][INFO ][plugins.screenshotting.config] Chromium sandbox provides an additional layer of protection, and is supported for Win32 OS. Automatically enabling Chromium sandbox.
[2023-08-24T16:34:38.672+02:00][INFO ][plugins.screenshotting.chromium] Browser executable: C:\elastic\kibana-8.6.2\x-pack\plugins\screenshotting\chromium\chrome-win\chrome.exe
[2023-08-24T16:34:38.743+02:00][INFO ][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
[2023-08-24T16:34:38.743+02:00][INFO ][savedobjects-service] Starting saved objects migrations
[2023-08-24T16:34:38.789+02:00][INFO ][savedobjects-service] [.kibana] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 21ms.
[2023-08-24T16:34:38.793+02:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 4ms.
[2023-08-24T16:34:38.796+02:00][INFO ][savedobjects-service] [.kibana_task_manager] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 24ms.
[2023-08-24T16:34:38.799+02:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 3ms.
[2023-08-24T16:34:38.802+02:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 9ms.
[2023-08-24T16:34:38.805+02:00][INFO ][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> CHECK_TARGET_MAPPINGS. took: 3ms.
[2023-08-24T16:34:38.806+02:00][INFO ][savedobjects-service] [.kibana] CHECK_TARGET_MAPPINGS -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 1ms.
[2023-08-24T16:34:38.806+02:00][INFO ][savedobjects-service] [.kibana] CHECK_VERSION_INDEX_READY_ACTIONS -> DONE. took: 0ms.
[2023-08-24T16:34:38.807+02:00][INFO ][savedobjects-service] [.kibana] Migration completed after 39ms
[2023-08-24T16:34:38.808+02:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 9ms.
[2023-08-24T16:34:38.811+02:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> CHECK_TARGET_MAPPINGS. took: 3ms.
[2023-08-24T16:34:38.811+02:00][INFO ][savedobjects-service] [.kibana_task_manager] CHECK_TARGET_MAPPINGS -> CHECK_VERSION_INDEX_READY_ACTIONS. took: 0ms.
[2023-08-24T16:34:38.811+02:00][INFO ][savedobjects-service] [.kibana_task_manager] CHECK_VERSION_INDEX_READY_ACTIONS -> DONE. took: 0ms.
[2023-08-24T16:34:38.812+02:00][INFO ][savedobjects-service] [.kibana_task_manager] Migration completed after 40ms
[2023-08-24T16:34:38.817+02:00][INFO ][plugins-system.preboot] Stopping all plugins.
[2023-08-24T16:34:38.819+02:00][INFO ][plugins-system.standard] Starting [127] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,usageCollection,taskManager,cloud,guidedOnboarding,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,screenshotMode,banners,newsfeed,fieldFormats,expressions,screenshotting,dataViews,embeddable,uiActionsEnhanced,charts,esUiShared,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,advancedSettings,spaces,security,snapshotRestore,lists,encryptedSavedObjects,telemetry,licenseManagement,files,eventLog,actions,notifications,console,bfetch,data,watcher,reporting,fileUpload,ingestPipelines,alerting,unifiedSearch,unifiedFieldList,savedSearch,savedObjects,graph,savedObjectsTagging,savedObjectsManagement,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,eventAnnotation,dataViewFieldEditor,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,discover,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,rollup,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,dashboardEnhanced,expressionXY,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,dataVisualizer,cases,timelines,sessionView,kubernetesSecurity,threatIntelligence,osquery,observability,aiops,ml,synthetics,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,visTypeGauge,dataViewManagement]
[2023-08-24T16:34:39.440+02:00][INFO ][plugins.fleet] Task Fleet-Usage-Sender-1.1.0 scheduled with interval 1h
[2023-08-24T16:34:39.483+02:00][INFO ][plugins.monitoring.monitoring] config sourced from: production cluster
[2023-08-24T16:34:41.386+02:00][INFO ][http.server.Kibana] http server running at https://10.112.11.82:5601
[2023-08-24T16:34:41.405+02:00][INFO ][plugins.fleet] Task Fleet-Usage-Logger-Task scheduled with interval 15m
[2023-08-24T16:34:41.453+02:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: scheduled with interval 1h
[2023-08-24T16:34:41.545+02:00][INFO ][plugins.monitoring.monitoring.kibana-monitoring] Starting monitoring stats collection
[2023-08-24T16:34:41.546+02:00][INFO ][plugins.fleet] Beginning fleet setup
[2023-08-24T16:34:41.561+02:00][INFO ][status] Kibana is now degraded
[2023-08-24T16:34:41.562+02:00][INFO ][plugins.ruleRegistry] Installed common resources shared between all indices
[2023-08-24T16:34:41.562+02:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.uptime.alerts
[2023-08-24T16:34:41.562+02:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-security.alerts
[2023-08-24T16:34:41.562+02:00][INFO ][plugins.ruleRegistry] Installing resources for index .preview.alerts-security.alerts
[2023-08-24T16:34:41.563+02:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.logs.alerts
[2023-08-24T16:34:41.563+02:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.metrics.alerts
[2023-08-24T16:34:41.563+02:00][INFO ][plugins.ruleRegistry] Installing resources for index .alerts-observability.apm.alerts
[2023-08-24T16:34:41.623+02:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.metrics.alerts
[2023-08-24T16:34:41.624+02:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.uptime.alerts
[2023-08-24T16:34:41.625+02:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-security.alerts
[2023-08-24T16:34:41.625+02:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.logs.alerts
[2023-08-24T16:34:41.668+02:00][INFO ][plugins.ruleRegistry] Installed resources for index .alerts-observability.apm.alerts
[2023-08-24T16:34:41.673+02:00][INFO ][plugins.ruleRegistry] Installed resources for index .preview.alerts-security.alerts
[2023-08-24T16:34:44.824+02:00][INFO ][status] Kibana is now available (was degraded)
[2023-08-24T16:34:44.827+02:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization
[2023-08-24T16:34:44.829+02:00][ERROR][plugins.fleet] Failed to fetch latest version of synthetics from registry: Error connecting to package registry: request to https://epr.elastic.co/search?package=synthetics&prerelease=true&kibana.version=8.6.2 failed, reason: connect ECONNREFUSED 34.120.127.130:443
[2023-08-24T16:34:44.903+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T16:34:44.910+02:00][INFO ][plugins.fleet] Fleet setup completed
[2023-08-24T16:34:44.916+02:00][INFO ][plugins.securitySolution] Dependent plugin setup complete - Starting ManifestTask
[2023-08-24T16:34:47.941+02:00][ERROR][plugins.fleet] Failed to fetch latest version of synthetics from registry: Error connecting to package registry: request to https://epr.elastic.co/search?package=synthetics&prerelease=true&kibana.version=8.6.2 failed, reason: connect ECONNREFUSED 34.120.127.130:443
[2023-08-24T16:34:47.944+02:00][INFO ][plugins.synthetics] Installed synthetics index templates
[2023-08-24T16:49:45.080+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T17:02:54.159+02:00][INFO ][plugins.fleet] Running Fleet Usage telemetry send task
[2023-08-24T17:04:45.344+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T17:19:45.675+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T17:34:46.016+02:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization
[2023-08-24T17:34:46.049+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T17:49:46.257+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T18:02:49.467+02:00][INFO ][plugins.securitySolution.endpoint:metadata-check-transforms-task:0.0.1] no endpoint installation found
[2023-08-24T18:02:55.481+02:00][INFO ][plugins.fleet] Running Fleet Usage telemetry send task
[2023-08-24T18:04:46.632+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T18:19:46.971+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T18:34:47.830+02:00][INFO ][plugins.ml] Task ML:saved-objects-sync-task: No ML saved objects in need of synchronization
[2023-08-24T18:34:47.867+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T18:49:50.548+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T19:02:56.718+02:00][INFO ][plugins.fleet] Running Fleet Usage telemetry send task
[2023-08-24T19:04:50.897+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
[2023-08-24T19:17:56.346+02:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic)
[2023-08-24T19:19:51.300+02:00][INFO ][plugins.fleet] Fleet Usage: {"agents_enabled":true,"agents":{"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"total_all_statuses":0,"updating":0},"fleet_server":{"total_all_statuses":0,"total_enrolled":0,"healthy":0,"unhealthy":0,"offline":0,"updating":0,"num_host_urls":0}}
2

@Kvoyce2023

Go to Kibana > Dev Tools

And run
GET _cat/indices?v

And show the results.

Also did you look at the logstash logs to see if it is actually connected and sending data?

Can you share your logstash conf file?

What are you expecting to be sent to elasticsearch?

3

image
image1553×421 35.3 KB

And logstash.yml got only 3 lines of code.

node.name: logstash01
path.data: S:/logstash/data
pipeline.id: main

Expectation is to sent filebeat logs from customer server and view in Kibana.

4

Hi @Kvoyce2023
First please do not post images of text in the future.

Your command shows
There is no index data index meaning I data is getting to elasticsearch so most likely your logstash is not working.

Apologies I did not mean your logstash.yml file the the should be one or more files that end in .conf which are the logstash pipeline configuration files which define inputs , filters and outputs that is what we are interested in .

Also the startup logs from logstash we would want to see those.

5

Apologies for posting images of text @stephenb

Below is my filebeatlogstash conf file.

input {
  beats {
    port => 5046
    host => "20.332.11.90"
    tags => ["filebeat"]
  }
}

output{
  if "filebeat" in [tags] {
   elasticsearch  {
      hosts =>["https://20.332.11.91:9200", "https://20.332.11.92:9200", "https://20.332.11.93:9200"]
      manage_template => false
      index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
      action => "create" 
      user => elastic
      password => "######"
      ssl_certificate_verification => false
    }
  }
}

Below is my filebeat.yml on customer server.

filebeat.inputs:

- type: filestream
  id: app-01
  enabled: true
  paths:
    - C:\ProgramData\app01\Logs\app01.ExportLog.txt
  fields:
    log_type: exportappp
    log_subtype: DataExportLog    
  parsers:
  - multiline:
      type: pattern
      pattern: \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3}
      negate: true
     

filebeat.config.modules:
  path: "C:\\elastic\\filebeat-8.6.2-windows-x86_64\\modules.d\\*.yml"
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:

output.logstash:
  # The Logstash hosts
  hosts: ["20.332.11.90:5046", "20.332.11.91:5046"]
  loadbalance: true
  ttl: 1

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~


type or paste code here

I am not getting any filebeat* indices on either data view or discover.

6

So there's a couple places to debug.

First, are you and Sherry reading the file and sending it via filebeat.

You can start filebeat like this..
Which will provide a lot of debug output.

filebeat -e -d "*"

Also you need to look at the logstash logs and add the stdout for debug in the output section

stdout { codec => rubydebug }

7

@stephenb executed as suggested. Please see below the logs.

filebeat json log file

{"log.level":"info","@timestamp":"2023-08-25T22:33:37.334+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4109},"total":{"ticks":5562,"value":5562},"user":{"ticks":1453}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3454465},"version":"8.6.2"},"memstats":{"gc_next":51396664,"memory_alloc":25380296,"memory_total":144654928,"rss":83292160},"runtime":{"goroutines":115}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":4096}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":3}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-08-25T22:34:07.333+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4125,"time":{"ms":16}},"total":{"ticks":5578,"time":{"ms":16},"value":5578},"user":{"ticks":1453}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3484465},"version":"8.6.2"},"memstats":{"gc_next":51396664,"memory_alloc":25742528,"memory_total":145017160,"rss":83292160},"runtime":{"goroutines":113}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":-3}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:34:21.399+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.82:5046)): dial tcp 10.112.11.82:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:34:21.399+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.82:5046)) with 55 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:34:37.328+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4140,"time":{"ms":15}},"total":{"ticks":5593,"time":{"ms":15},"value":5593},"user":{"ticks":1453}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3514459},"version":"8.6.2"},"memstats":{"gc_next":51378504,"memory_alloc":24620256,"memory_total":145373040,"rss":83292160},"runtime":{"goroutines":114}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":3}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:34:38.796+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.83:5046)): dial tcp 10.112.11.83:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:34:38.796+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.83:5046)) with 55 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:35:07.340+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4140},"total":{"ticks":5593,"value":5593},"user":{"ticks":1453}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3544471},"version":"8.6.2"},"memstats":{"gc_next":51378504,"memory_alloc":25009320,"memory_total":145762104,"rss":83296256},"runtime":{"goroutines":113}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":2}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-08-25T22:35:37.338+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4250,"time":{"ms":110}},"total":{"ticks":5734,"time":{"ms":141},"value":5734},"user":{"ticks":1484,"time":{"ms":31}}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3574470},"version":"8.6.2"},"memstats":{"gc_next":51378504,"memory_alloc":25376824,"memory_total":146129608,"rss":83283968},"runtime":{"goroutines":113}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107}}},"registrar":{"states":{"current":0}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:35:37.758+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.82:5046)): dial tcp 10.112.11.82:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:35:37.758+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.82:5046)) with 56 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-08-25T22:35:39.433+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.83:5046)): dial tcp 10.112.11.83:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:35:39.433+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.83:5046)) with 56 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:36:07.328+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4375,"time":{"ms":125}},"total":{"ticks":5921,"time":{"ms":187},"value":5921},"user":{"ticks":1546,"time":{"ms":62}}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3604458},"version":"8.6.2"},"memstats":{"gc_next":51378504,"memory_alloc":25728384,"memory_total":146481168,"rss":83296256},"runtime":{"goroutines":113}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":4096}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":-3}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-08-25T22:36:37.340+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4375},"total":{"ticks":5968,"time":{"ms":47},"value":5968},"user":{"ticks":1593,"time":{"ms":47}}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3634471},"version":"8.6.2"},"memstats":{"gc_next":51374856,"memory_alloc":24651240,"memory_total":146886256,"rss":83296256},"runtime":{"goroutines":113}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":2}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:36:55.046+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.82:5046)): dial tcp 10.112.11.82:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:36:55.046+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.82:5046)) with 57 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-08-25T22:36:56.330+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.83:5046)): dial tcp 10.112.11.83:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:36:56.330+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.83:5046)) with 57 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:37:07.340+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4406,"time":{"ms":31}},"total":{"ticks":5999,"time":{"ms":31},"value":5999},"user":{"ticks":1593}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3664470},"version":"8.6.2"},"memstats":{"gc_next":51374856,"memory_alloc":25001568,"memory_total":147236584,"rss":83300352},"runtime":{"goroutines":115}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":4096}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":1}}},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-08-25T22:37:37.330+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4515,"time":{"ms":109}},"total":{"ticks":6108,"time":{"ms":109},"value":6108},"user":{"ticks":1593}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3694460},"version":"8.6.2"},"memstats":{"gc_next":51374856,"memory_alloc":25361776,"memory_total":147596792,"rss":83308544},"runtime":{"goroutines":113}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":-3}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:37:53.787+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.82:5046)): dial tcp 10.112.11.82:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:37:53.787+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.82:5046)) with 58 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:38:07.339+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4531,"time":{"ms":16}},"total":{"ticks":6124,"time":{"ms":16},"value":6124},"user":{"ticks":1593}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3724468},"version":"8.6.2"},"memstats":{"gc_next":51374856,"memory_alloc":25752544,"memory_total":147987560,"rss":83308544},"runtime":{"goroutines":114}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":3}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:38:16.525+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.83:5046)): dial tcp 10.112.11.83:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:38:16.525+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.83:5046)) with 58 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:38:37.339+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4656,"time":{"ms":125}},"total":{"ticks":6265,"time":{"ms":141},"value":6265},"user":{"ticks":1609,"time":{"ms":16}}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3754468},"version":"8.6.2"},"memstats":{"gc_next":51374856,"memory_alloc":26157960,"memory_total":148392976,"rss":83316736},"runtime":{"goroutines":114}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":-1}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:38:53.523+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.82:5046)): dial tcp 10.112.11.82:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:38:53.523+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.82:5046)) with 59 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:39:07.327+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4687,"time":{"ms":31}},"total":{"ticks":6530,"time":{"ms":265},"value":6530},"user":{"ticks":1843,"time":{"ms":234}}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3784457},"version":"8.6.2"},"memstats":{"gc_next":51695528,"memory_alloc":25003016,"memory_total":148723912,"rss":83316736},"runtime":{"goroutines":114}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":1}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:39:22.678+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.83:5046)): dial tcp 10.112.11.83:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:39:22.678+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.83:5046)) with 59 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:39:37.337+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4687},"total":{"ticks":6530,"value":6530},"user":{"ticks":1843}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3814467},"version":"8.6.2"},"memstats":{"gc_next":51695528,"memory_alloc":25377848,"memory_total":149098744,"rss":83316736},"runtime":{"goroutines":114}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":2}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:39:55.679+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.82:5046)): dial tcp 10.112.11.82:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:39:55.679+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.82:5046)) with 60 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:40:07.326+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4750,"time":{"ms":63}},"total":{"ticks":6609,"time":{"ms":79},"value":6609},"user":{"ticks":1859,"time":{"ms":16}}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3844457},"version":"8.6.2"},"memstats":{"gc_next":51695528,"memory_alloc":25732184,"memory_total":149453080,"rss":83304448},"runtime":{"goroutines":114}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":-4}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:40:29.784+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.83:5046)): dial tcp 10.112.11.83:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:40:29.784+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.83:5046)) with 60 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:40:37.334+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4750},"total":{"ticks":6609,"value":6609},"user":{"ticks":1859}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3874464},"version":"8.6.2"},"memstats":{"gc_next":51695528,"memory_alloc":26113376,"memory_total":149834272,"rss":83320832},"runtime":{"goroutines":114}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}}},"ecs.version":"1.6.0"}}
{"log.level":"error","@timestamp":"2023-08-25T22:40:48.648+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":150},"message":"Failed to connect to backoff(async(tcp://10.112.11.82:5046)): dial tcp 10.112.11.82:5046: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:40:48.648+0200","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(async(tcp://10.112.11.82:5046)) with 61 reconnect attempt(s)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-08-25T22:41:07.331+0200","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":4750},"total":{"ticks":6625,"time":{"ms":16},"value":6625},"user":{"ticks":1875,"time":{"ms":16}}},"info":{"ephemeral_id":"1a324f7d-97fe-4e6f-b4e2-ae3889d3c280","uptime":{"ms":3904461},"version":"8.6.2"},"memstats":{"gc_next":51486472,"memory_alloc":24897880,"memory_total":150199736,"rss":83386368},"runtime":{"goroutines":114}},"filebeat":{"events":{"active":4107},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":11,"events":{"active":4107,"retry":2048}}},"registrar":{"states":{"current":0}},"system":{"handles":{"open":4}}},"ecs.version":"1.6.0"}}

logstash-plain.log

[2023-08-25T21:30:19,192][INFO ][logstash.runner          ] Log4j configuration path used is: C:\elastic\logstash-8.6.2\config\log4j2.properties
[2023-08-25T21:30:19,192][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.6.2", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-mswin32]"}
[2023-08-25T21:30:19,192][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2023-08-25T21:30:19,239][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2023-08-25T21:30:20,502][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2023-08-25T21:30:20,864][INFO ][org.reflections.Reflections] Reflections took 107 ms to scan 1 urls, producing 127 keys and 444 values
[2023-08-25T21:30:21,394][INFO ][logstash.javapipeline    ] Pipeline `Main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2023-08-25T21:30:21,410][INFO ][logstash.outputs.elasticsearch][Main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://x.x.x.x:9200", "https://x.x.x.1:9200", "https://1x.x.x.4:9200"]}
[2023-08-25T21:30:21,418][WARN ][logstash.outputs.elasticsearch][Main] You have enabled encryption but DISABLED certificate verification, to make sure your data is secure remove `ssl_certificate_verification => false`
[2023-08-25T21:30:21,648][INFO ][logstash.outputs.elasticsearch][Main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@x.x.x.x:9200/, https://elastic:xxxxxx@x.x.x.1:9200/, https://elastic:xxxxxx@1x.x.x.4:9200/]}}
[2023-08-25T21:30:21,850][WARN ][logstash.outputs.elasticsearch][Main] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@x.x.x.x:9200/"}
[2023-08-25T21:30:21,856][INFO ][logstash.outputs.elasticsearch][Main] Elasticsearch version determined (8.6.2) {:es_version=>8}
[2023-08-25T21:30:21,857][WARN ][logstash.outputs.elasticsearch][Main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2023-08-25T21:30:21,915][WARN ][logstash.outputs.elasticsearch][Main] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@x.x.x.1:9200/"}
[2023-08-25T21:30:21,953][WARN ][logstash.outputs.elasticsearch][Main] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@1x.x.x.4:9200/"}
[2023-08-25T21:30:21,969][INFO ][logstash.outputs.elasticsearch][Main] Not eligible for data streams because config contains one or more settings that are not compatible with data streams: {"index"=>"%{[@metadata][beat]}-%{+YYYY.MM.dd}"}
[2023-08-25T21:30:21,969][INFO ][logstash.outputs.elasticsearch][Main] Data streams auto configuration (`data_stream => auto` or unset) resolved to `false`
[2023-08-25T21:30:21,971][WARN ][logstash.outputs.elasticsearch][Main] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2023-08-25T21:30:22,011][INFO ][logstash.javapipeline    ][Main] Starting pipeline {:pipeline_id=>"Main", "pipeline.workers"=>16, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2000, "pipeline.sources"=>["C:/elastic/logstash-8.6.2/config/beats_conf/filebeatlogstash.conf"], :thread=>"#<Thread:0x4c1e86a0@C:/elastic/logstash-8.6.2/logstash-core/lib/logstash/java_pipeline.rb:131 run>"}
[2023-08-25T21:30:23,469][INFO ][logstash.javapipeline    ][Main] Pipeline Java execution initialization time {"seconds"=>1.46}
[2023-08-25T21:30:23,480][INFO ][logstash.inputs.beats    ][Main] Starting input listener {:address=>"10.112.11.82:5046"}
[2023-08-25T21:30:23,515][INFO ][logstash.javapipeline    ][Main] Pipeline started {"pipeline.id"=>"Main"}
[2023-08-25T21:30:23,532][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:Main], :non_running_pipelines=>[]}
[2023-08-25T21:30:23,793][INFO ][org.logstash.beats.Server][Main][418148730d1cc33dafad15cc228993d2381b63de996bca77c253faa6a4ef25e0] Starting server on port: 5046
8

Right there 2nd line of the log. Filebeat can not connect to logstash.

Perhaps Firewall issue etc ....

9

Thanks Stephen. it was firewall issue.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.