Hello,
The process of working with enrichments has been quite challenging for me, and I've been struggling with it for hours. It's first time so I may have done something wrong. However, I believe I'm close to a solution, though I’m still having difficulty finding something that works.
I have two dynamically generated fields from the MITRE ATT&CK framework:
attack_technique: "T1027.010"
attack_tactic: "TA0005"
I would like to enrich these fields to display their full names for better readability. For example:
TA0005 → "Defense Evasion"
T1027.010 → "Command Obfuscation"
I have already ingested the MITRE framework in JSON format. However, when I check Analytics → Discover in the logs, I cannot find fields like attack_tactic_description or attack_technique_description.
Below is my current configuration. Could you please help me identify what might be missing or what I could improve?
Thanks in advance!
#######################################
Bulk Indexing: Mitre Framework into json.
jq -c '. | {"index": {"_index": "mitre_mapping"}}, .' tactics.json > tactics_bulk.json
jq -c '. | {"index": {"_index": "mitre_mapping"}}, .' techniques.json > techniques_bulk.json
curl -X POST "https://x:9200/_bulk" -H "Content-Type: application/json" --data-binary "@tactics_bulk.json" -u elastic:xxx -k
curl -X POST "https://x:9200/_bulk" -H "Content-Type: application/json" --data-binary "@techniques_bulk.json" -u elastic:xxx -k
#######################################
#######################################
Filebeat Configuration:
output.elasticsearch.indices:
- index: "carbon_black_observations-%{+yyyy.MM.dd}"
pipeline: "mitre_attack_pipeline"
#######################################
Index Mapping:
PUT mitre_mapping
{
"mappings": {
"properties": {
"attack_tactic": {
"type": "keyword"
},
"attack_tactic_description": {
"type": "text"
},
"attack_technique": {
"type": "keyword"
},
"attack_technique_description": {
"type": "text"
}
}
}
}
#######################################
#######################################
Ingest Pipelines: mitre_attack_pipeline
[
{
"enrich": {
"policy_name": "mitre_tactic_policy",
"field": "attack_tactic",
"target_field": "attack_tactic_description",
"ignore_missing": true
}
},
{
"enrich": {
"policy_name": "mitre_technique_policy",
"field": "attack_technique",
"target_field": "attack_technique_description",
"ignore_missing": true
}
}
]
#######################################
#######################################
Enrich Policies:
mitre_tactic_policy
{
"policies": [
{
"config": {
"match": {
"name": "mitre_tactic_policy",
"indices": [
"mitre_mapping"
],
"match_field": "attack_tactic",
"enrich_fields": [
"attack_tactic_description"
]
}
}
}
]
}
mitre_technique_policy
{
"policies": [
{
"config": {
"match": {
"name": "mitre_technique_policy",
"indices": [
"mitre_mapping"
],
"match_field": "attack_technique",
"enrich_fields": [
"attack_technique_description"
]
}
}
}
]
}
#######################################
#######################################
Execute Enrich Policies
POST _enrich/policy/mitre_tactic_policy/_execute
POST _enrich/policy/mitre_technique_policy/_execute
#######################################
#######################################
Testing pipleing:
Which looks like it work
POST _ingest/pipeline/mitre_attack_pipeline/_simulate
{
"docs": [
{
"_source": {
"attack_tactic": "TA0005",
"attack_technique": "T1027.010"
}
}
]
}
{
"docs": [
{
"doc": {
"_index": "_index",
"_version": "-3",
"_id": "_id",
"_source": {
"attack_tactic": "TA0005",
"attack_tactic_description": {
"attack_tactic_description": "Defense Evasion",
"attack_tactic": "TA0005"
},
"attack_technique": "T1027.010",
"attack_technique_description": {
"attack_technique_description": "Command Obfuscation",
"attack_technique": "T1027.010"
}
},
"_ingest": {
"timestamp": "2025-02-04T13:28:23.091222826Z"
}
}
}
]
}
#######################################
Also I can't find the new fields.
