2, For redis module, the metricset.name "info" always can not feed into elasticsearch with below error in metricbeat log file, think this need to be fixed.
2016-09-30T15:16:26+08:00 DBG PublishEvents: 25 events have been published to elasticsearch in 688.516542ms.
2016-09-30T15:16:26+08:00 WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"Field name [bgrewrite.last_status] cannot contain '.'"}
i checked the original info command from redis, the orginal parameter is called aof_last_bgrewrite_status:ok
why metricbeat translate into some other name which elasticsearch can not handle???
3, is there any ways I can monitor the redis queue for each key??? so far in keyspace it only shows the no. of keys which is not so useful.
Can you run the following ES queries to confirm that this is happening on all data points.
# Get total number of documents.
GET /metricbeat-*/_search?pretty
{
"size": 0,
"query": {
"query_string": {
"query": "_exists_:system.process.memory.rss.pct"
}
}
}
# Get total number where pct > 0.
GET /metricbeat-*/_search?pretty
{
"size": 0,
"query": {
"query_string": {
"query": "system.process.memory.rss.pct:>0"
}
}
}
# Check the mapping for the field. It should be a float or scaled_float depending on ES version.
GET /metricbeat-*/_mapping/field/system.process.memory.rss.pct
The redis metricset does appear to use field names with dots in them which are not compatable with ES 2.0-2.3. Elasticsearch 2.4.0 adds a system property called mapper.allow_dots_in_name which will disable the check for dots in field names. And ES 5.0 allows dots in field names. Source: https://www.elastic.co/guide/en/elasticsearch/reference/current/dots-in-names.html#dots-in-names
Please open an new issue in elastic/beats for this problem. We can correct the metricset to not use dots in the field name.
You can't do this with the redis info metricset. Maybe there could be a separate metricset to monitor size of various keys. I'm open to additional discussion of the feature. How would you want it to be configured? What data would useful? What parts of the redis api would be used to collect the data?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.