# How to map fields from Oracle table to Index fields in Logstash?

**URL:** <https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675>\
**Category:** Logstash\
**Created:** [February 17, 2020, 7:42pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675 "2020-02-17T19:42:13Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![fefontana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fefontana/32/62843_2.png) [@fefontana](https://discuss.elastic.co/u/fefontana)\
**Post date:** [February 17, 2020, 7:42pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675/1 "2020-02-17T19:42:13Z")

</div>

Hello, It is not clear for me how to tell Logstash which field from my Oracle table corresponds to specific field of my predefined Elasticsearch index.

For example, I have the following SQL configured in the .conf file using jdbc input in Logstash:

Select id\_log, nombre\_procedimiento, mensaje, detalle, fecha, to\_char(trunc(fecha),'YYYY-MM-DD') fechadia from STGCBPRD.STG\_FT\_LOG where fecha BETWEEN TRUNC(SYSDATE)-1 AND TRUNC(SYSDATE)

On the other side, I have an index (named: logbim) with the following mapping structure:  
{  
"settings": {  
"index": {  
"number\_of\_shards": 1,  
"number\_of\_replicas": 1  
}  
},  
"mappings": {  
"properties": {  
"@timestamp": {  
"type": "date"  
},  
"@version": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"detalle": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"fecha": {  
"type": "date"  
},  
"fechadia": {  
"type": "keyword"  
},  
"id\_log": {  
"type": "long"  
},  
"mensaje": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"nombre\_procedimiento": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
}  
}  
}  
}

The question is, how to specify in Logstash (for example) that the field "id\_log" from the Oracle SQL, corresponds to the field "id\_log" at the index predefined in Elasticsearch ?. Does Logstash do a match by name ?.

Below is my .conf file for Logstash. Thank you!.

input {  
jdbc {  
jdbc\_validate\_connection =\> true  
jdbc\_connection\_string =\> "jdbc:oracle:thin:@172.26.150.124:1525/BIPRD"  
jdbc\_user =\> "DWHCBPRD"  
jdbc\_password =\> "xxxxxxxx"  
jdbc\_driver\_library =\> "D:\Temp\SW\ELK\OJDBC-Full\ojdbc7.jar"  
jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver"  
statement =\> "Select id\_log, nombre\_procedimiento, mensaje, detalle, fecha, to\_char(trunc(fecha),'YYYY-MM-DD') fechadia from STGCBPRD.STG\_FT\_LOG where fecha BETWEEN TRUNC(SYSDATE)-1 AND TRUNC(SYSDATE)"  
}  
}

output {  
elasticsearch {  
hosts =\> ["[http://127.0.0.1:9200](http://127.0.0.1:9200)"]  
index =\> "logbim"  
#user =\> "elastic"  
#password =\> "changeme"  
}  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 17, 2020, 8:06pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675/2 "2020-02-17T20:06:21Z")

</div>

If you do "Select id\_log, nombre\_procedimiento, ..." then your events will have fields called id\_log, nombre\_procedimiento etc., and the documents written to elasticsearch will also have fields with those names.

---

<div class="post-metadata">

**Author:** ![fefontana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fefontana/32/62843_2.png) [@fefontana](https://discuss.elastic.co/u/fefontana)\
**Post date:** [February 17, 2020, 8:21pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675/3 "2020-02-17T20:21:22Z")

</div>

I don't have events, Logstash is working with the index (in Elasticsearch) and the .conf file (for Logstash) defined as I described previously.  
What I mean is how can I tell Logstash, take the value from the field "fecha" in Oracle and then put it in the field "fechadia" in the Elasticsearch index ?. Can you point me to any reference regarding this point ?.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 17, 2020, 8:43pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675/4 "2020-02-17T20:43:15Z")

</div>

You do have events. Each row fetch by the jdbc input is an event.

If you want to copy a field to another field, or rename it, you can use a mutate filter.

---

<div class="post-metadata">

**Author:** ![fefontana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fefontana/32/62843_2.png) [@fefontana](https://discuss.elastic.co/u/fefontana)\
**Post date:** [February 17, 2020, 8:59pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675/5 "2020-02-17T20:59:27Z")

</div>

Thank you for your answer, I'm sorry but I'm new and I didn't know each line at input section in .conf file is called an event!.  
Anyway, if you take a look at the SQL statement, all the Oracle fields are in one line.  
How is the syntax to tell Logstash, take the field fecha from source and put it into fechadia target field ?. I'm assuming that Logstash by default read the fields from source and seek to match the same names to target fields. If this is true, how can be changed.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 17, 2020, 9:52pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675/6 "2020-02-17T21:52:25Z")

</div>

As I said, you can use a mutate filter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 16, 2020, 9:52pm UTC](https://discuss.elastic.co/t/how-to-map-fields-from-oracle-table-to-index-fields-in-logstash/219675/7 "2020-03-16T21:52:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
