Applying analyzer to certain field in mapping


(Charlie) #1

Hello.
I have defined analyzer for stripping off the html tags from data ingested by logstash by rss feed and outputted to ElasticSearch.

My index settings are:
GET rss-news

{
"rss-news": {
"aliases": {},
"mappings": {
"default": {
"properties": {
"@timestamp": {
"type": "date"
},
"@version": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Feed": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"description": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"link": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"message": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"published": {
"type": "date"
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"settings": {
"index": {
"number_of_shards": "5",
"provided_name": "rss-news",
"creation_date": "1541154984069",
"analysis": {
"analyzer": {
"my_analyzer": {
"char_filter": [
"my_char_filter"
],
"tokenizer": "keyword"
}
},
"char_filter": {
"my_char_filter": {
"type": "html_strip"
}
}
},
"number_of_replicas": "1",
"uuid": "TR_0PhsBTUussADaZMFJwQ",
"version": {
"created": "6040099"
}
}
}
}
}

When I attempt to apply analyzer to the field:

PUT rss-news/_settings
{
"mappings": {
"default": {
"properties": {
"description": {
"type": "keyword",
"analyzer": "char_filter",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}

I receive an error:

{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[elk1-node][172.18.70.11:9300][indices:admin/settings/update]"
}
],
"type": "illegal_argument_exception",
"reason": "unknown setting [index.mappings.default.properties.description.analyzer] please check that any required plugins are installed, or check the breaking changes documentation for removed settings",
"suppressed": [
{
"type": "illegal_argument_exception",
"reason": "unknown setting [index.mappings.default.properties.description.fields.keyword.ignore_above] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"
},
{
"type": "illegal_argument_exception",
"reason": "unknown setting [index.mappings.default.properties.description.fields.keyword.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"
},
{
"type": "illegal_argument_exception",
"reason": "unknown setting [index.mappings.default.properties.description.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"
}
]
},
"status": 400
}

I am running newest kibana and elasticsearch (6.4).
How can I apply the analyzer to the description field?

The analyzer works well, example:
INPUT:

POST rss-news/_analyze

{
"analyzer": "my_analyzer",
"text": "<img width="300" height="146" src="https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/11/BLEEDINGBIT.jpg?fit=300%2C146&ssl=1" class="webfeedsFeaturedVisual wp-post-image" alt="" style="display: block; margin-bottom: 5px; clear:both;max-width: 100%;" srcset="https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/11/BLEEDINGBIT.jpg?w=716&ssl=1 716w, https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/11/BLEEDINGBIT.jpg?resize=300%2C146&ssl=1 300w, https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/11/BLEEDINGBIT.jpg?resize=700%2C340&ssl=1 700w, https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/11/BLEEDINGBIT.jpg?resize=533%2C261&ssl=1 533w, https://i2.wp.com/gbhackers.com/wp-content/uploads/2018/11/BLEEDINGBIT.jpg?resize=696%2C338&ssl=1 696w" sizes="(max-width: 300px) 100vw, 300px" />

Researcher’s discovered 2 critical Bluetooth vulnerabilities in BLE (Bluetooth Low Energy) is named as ” BLEEDINGBIT ” affected millions of BLE embedded devices that allows an attacker to access enterprise network without authentication. These serious vulnerabilities existing in the BLE which is made by Texas Instruments (TI) that embedded in access points to provide Wi-Fi […]

\n

The post <a rel="nofollow" href="https://gbhackers.com/bleedingbit-bluetooth-vulnerabilities/">BLEEDINGBIT – Two Bluetooth Chip-level Vulnerabilities Affected Millions of Enterprise Wi-Fi Access Point Devices appeared first on <a rel="nofollow" href="https://gbhackers.com">GBHackers On Security.

\n"
}

OUTPUT:

{
"tokens": [
{
"token": """

Researcher’s discovered 2 critical Bluetooth vulnerabilities in BLE (Bluetooth Low Energy) is named as ” BLEEDINGBIT ” affected millions of BLE embedded devices that allows an attacker to access enterprise network without authentication. These serious vulnerabilities existing in the BLE which is made by Texas Instruments (TI) that embedded in access points to provide Wi-Fi […]

The post BLEEDINGBIT – Two Bluetooth Chip-level Vulnerabilities Affected Millions of Enterprise Wi-Fi Access Point Devices appeared first on GBHackers On Security.

""",
"start_offset": 838,
"end_offset": 1568,
"type": "word",
"position": 0
}
]
}


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.