I have fleet agents with a custom logs.
I want to route some of the logs to a different index.
I created the index
task-logs and in the integration, I defined an
reroute ingest pipeline. However,the documents don't reach the index, and instead I get an error.
This is the error I see in the main
I suspect this might be related to the permissions error mentioned in the docs.
How do I (a) prove it and (b) fix it?
I instead used the
dataset option to receive the tasks.
It created an index (?) called logs-taks for me.
Is this the correct way to go?
Yes, using the
dataset option is a common way to route different types of logs to different indices in Elastic Stack.
Did you try to simply filling in static fields for the namespace and dataset?
I don't think just
logs-task is proper syntax I think you need all three parts
or something like that
from this article:
The Elastic data stream naming scheme is made for time series data and consists of splitting datasets into different data streams using the following naming convention.
- type: Generic type describing the data
- dataset: Describes the data ingested and its structure
- namespace: User-configurable arbitrary grouping
These three parts are combined by a “-” and result in data streams like
logs-nginx.access-production. In all three parts, the “-” character is not allowed. This means all data streams are named in the following way:
For both dataset and namespace there is a default value, which is
namespace=default. In the case of Elastic Agent, if a user just starts to ingest a log file, the data ends up in
An introduction to the Elastic data stream naming scheme | Elastic Blog
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.