Hi everybody!!!
I have a postgres table that have a json field. it looks like this:
customer_id ==> integer
categories ==> json ==> please note that this field is type json
and the table have some records like this:
1 [{"first_level":359,"second_level":null}]
2 [{"first_level":62,"second_level":null}]
3 null
4 [{"first_level":585,"second_level":[1559,2445]},{"first_level":987,"second_level":[2}]
5 [{"first_level":592,"second_level":[20521]},{"first_level":335,"second_level":null}]
now I want to load this data in an elasticsearch index. to do this I have created a mapping like this:
POST index_to_test/
{
"settings": {
"number_of_shards": 3
},
"mappings": {
"docu": {
"properties": {
"customer_id": {
"type": "integer"
},
"categories": {
"type": "nested",
"properties": {
"firs_level": {
"type": "integer"
},
"second_level": {
"type": "integer"
}
}
}
}
}
}
}
I will need the field categories as "nested" type, because I will need to search by separate first_level and second_level
the config logstash looks like this:
input {
jdbc {
jdbc_connection_string => "jdbc:postgresql://dev:5432/database"
jdbc_user => "myuser"
jdbc_password => "mypassword"
jdbc_validate_connection => true
jdbc_driver_library => "/usr/share/elasticsearch/lib/postgresql-9.4.1208.jar"
jdbc_driver_class => "org.postgresql.Driver"
statement => "SELECT * from table
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
type => "docu"
}
}
output {
elasticsearch{
index => "index_to_test"
document_id => "%{customer_id}"
}
}
When I execute the process to load the data, the mapping has been changed for the categories field. They has created:
"categories": {
"type": "nested",
"properties": {
"first_level": {
"type": "long"
},
"second_level": {
"type": "long"
},
"type": {
"type": "string"
},
"value": {
"type": "string"
}
}
},
and the document look like this:
"customer_id": "1",
"categories": {
"type": "json",
"value": "[{"first_level":26,"second_level":[342]},{"first_level":826,"second_level":null}]"
},
"@version": "1",
"@timestamp": "2016-04-18T08:02:06.243Z",
"type": "docu"
}
So, may anyone tell me how I have to mapping this field in order to receive it as a nested field?
In advance thanks a lot for your suppor
Jorge von Rudno