# Visibility Monitoring - Have Kibana Ingest a File that Contains a List of Log Sources and Have Kibana Compare that List to What Sources Are Actually Being Collected

**URL:** <https://discuss.elastic.co/t/visibility-monitoring-have-kibana-ingest-a-file-that-contains-a-list-of-log-sources-and-have-kibana-compare-that-list-to-what-sources-are-actually-being-collected/269734>\
**Category:** Kibana\
**Created:** [April 9, 2021, 4:47pm UTC](https://discuss.elastic.co/t/visibility-monitoring-have-kibana-ingest-a-file-that-contains-a-list-of-log-sources-and-have-kibana-compare-that-list-to-what-sources-are-actually-being-collected/269734 "2021-04-09T16:47:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Log\_Gobbler](https://avatars.discourse-cdn.com/v4/letter/l/a183cd/32.png) [@Log\_Gobbler](https://discuss.elastic.co/u/Log_Gobbler)\
**Post date:** [April 9, 2021, 4:47pm UTC](https://discuss.elastic.co/t/visibility-monitoring-have-kibana-ingest-a-file-that-contains-a-list-of-log-sources-and-have-kibana-compare-that-list-to-what-sources-are-actually-being-collected/269734/1 "2021-04-09T16:47:59Z")

</div>

Hi Everybody. We're new to ELK but not new to SIEMs. We're trying to improve our visibility by creating something in the ELK stack that might be considered "Visibility Monitoring". Our "Visibility Monitoring" comes with two objectives:  
1) Identify any new log source (a new server, a new device, a new cloud  
service, etc) that has been added to the company's environment which  
ELK is currently not collecting-from. (So it can be added for SIEM log  
collection to improve Visibility) AKA - _New Log Source Detection_  
2) Identify any existing log source (a server, a device, a cloud service, etc)  
that was initially successfully set-up for log collection in ELK, but now ELK  
is failing to obtain logs from that same specific log source. (So the issue  
with collecting logs from that specific source can be identified and  
resolved to improve Visibility.) AKA - _Failed Log Source Detection_

In both cases, for "New Log Source Detection" and "Failed Log Source Detection", we would use scripts that would periodically gather information, from various resources in our environment, and create different lists for each log source type that would be written to separate files. These different files would be considered separate baselines for each log source type (list of servers, list of devices, list of cloud services, etc).

Then, we want to compare what's contained in a given file (a list of servers, a list of devices, or a list of cloud services, etc) to what is actually being ingested by ELK and produce a Visualization that would display what ELK is currently missing in log collection.

Given all of this, how do we have Kibana ingest a file (a list of servers, a list of devices, or a list of cloud services, etc) that can be used in Kibana to compare-to what is actually being ingested in ELK and produce a Visualization that displays what is missing in ELK log collection?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 7, 2021, 4:48pm UTC](https://discuss.elastic.co/t/visibility-monitoring-have-kibana-ingest-a-file-that-contains-a-list-of-log-sources-and-have-kibana-compare-that-list-to-what-sources-are-actually-being-collected/269734/2 "2021-05-07T16:48:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.