Hey @SanderP,
You're right, you can access the App Search indexes with an index pattern as mentioned here: Elastic index name with enterprise app search engine names - #2 by JasonStoltz.
Definitely be sure not to modify any of the data in those indexes. Otherwise, the performance impact is pretty minimal assuming you're not querying against that data very frequently. It depends on your cluster size, but less frequently than once a minute should be negligible on any cluster size.
What field would you intend to use as the time field? Most documents should have a created at field, and many should have updated at. You could also use a custom schema field contained in your document structure. Note that if you use system fields like the data used to track created at, there is no future guarantee that those fields will be preserved as they currently exist.
Ross