# Indexer Mongodb dans ES en temps réel

**URL:** <https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190>\
**Category:** Elasticsearch\
**Created:** [December 20, 2020, 8:51am UTC](https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190 "2020-12-20T08:51:27Z")\
**Posts on this page:** 5\
**Page:** 2

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [December 22, 2020, 1:39pm UTC](https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190/21 "2020-12-22T13:39:06Z")

</div>

Logstash ne peut pas sérialiser les champs de type ObjectId, en l'occurence \_id et end.\_id qu'il faut aussi exclure, Je pense via un statement de ce type

```auto
statement => "db.trips.find({},{'_id': false, 'end._id': false});"

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 22, 2020, 2:22pm UTC](https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190/22 "2020-12-22T14:22:43Z")

</div>

Je pense que le mapping que tu as indiqué à [Indexer Mongodb dans ES en temps réel](https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190/14) devrait être bon.

---

<div class="post-metadata">

**Author:** ![Jalil](https://avatars.discourse-cdn.com/v4/letter/j/fbc32d/32.png) [@Jalil](https://discuss.elastic.co/u/Jalil)\
**Post date:** [December 22, 2020, 7:12pm UTC](https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190/23 "2020-12-22T19:12:32Z")

</div>

Bonsoir,

Je vais tester ce mapping, mais là j'ai du mal à faire tourner la config logstash.  
J'ai mis le jdbc driver dans le dossier config pour ne pas avoir de chemin, je n'ai plus l'exception mais j'ai une erreur maintenant :

```auto
  jdbc {
    jdbc_driver_library => "mongojdbc3.0.jar"
    jdbc_driver_class => "com.dbschema.MongoJdbcDriver"
    jdbc_connection_string => "jdbc:mongodb://localhost:27017/atseeone?authSource=admin"
	jdbc_user => ""
    jdbc_password => ""
    schedule => "*/30 * * * * *"
    statement => "db.trips.find({},{'_id': false});"
  }

```

Erreur :

```auto
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Jdbc schedule=>"*/30 * * * * *", jdbc_password=><password>, statement=>"db.trips.find({},{'_id': false});", jdbc_driver_library=>"mongojdbc3.0.jar", jdbc_connection_string=>"jdbc:mongodb://localhost:27017/atseeone?authSource=admin", id=>"99473a9d2c60035f754dc0fdfc90d4e5ecd62b1a56c4c17de536ed564db5ef5a", jdbc_driver_class=>"com.dbschema.MongoJdbcDriver", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_2a455c80-b95f-48fa-a164-dde8f9bb7545", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc", last_run_metadata_path=>"C:\\Users\\karou/.logstash_jdbc_last_run", use_column_value=>false, tracking_column_type=>"numeric", clean_run=>false, record_last_run=>true, lowercase_column_names=>true, use_prepared_statements=>false>
  Error: unable to load mongojdbc3.0.jar from :jdbc_driver_library, file not readable (please check user and group permissions for the path)
  Exception: LogStash::PluginLoadingError

```

---

<div class="post-metadata">

**Author:** ![Corben\_Dallas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/corben_dallas/32/77287_2.png) [@Corben\_Dallas](https://discuss.elastic.co/u/Corben_Dallas)\
**Post date:** [December 25, 2020, 9:03am UTC](https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190/24 "2020-12-25T09:03:53Z")

</div>

we use rest service with json data (spring boot) for save into kafka and to push via connectors into mongo and elasticsearch or something else ,that work well and are scalable, the db reference is kakfa, in case of rebuild or migration on version elastic or mongo, that's near real time.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 22, 2021, 9:04am UTC](https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190/25 "2021-01-22T09:04:12Z")

</div>

Closed on January 22, 2021, 9:04am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/indexer-mongodb-dans-es-en-temps-reel/259190.md?page=1)