# Multiple tables as input. Multiple pipelines? **URL:** \ **Category:** Logstash\ **Created:** [March 6, 2018, 2:47am UTC](https://discuss.elastic.co/t/multiple-tables-as-input-multiple-pipelines/122630 "2018-03-06T02:47:11Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![justin\_Kim](https://avatars.discourse-cdn.com/v4/letter/j/e9a140/32.png) [@justin\_Kim](https://discuss.elastic.co/u/justin_Kim)\ **Post date:** [March 6, 2018, 2:47am UTC](https://discuss.elastic.co/t/multiple-tables-as-input-multiple-pipelines/122630/1 "2018-03-06T02:47:11Z")
Hello, I'm a complete n00b at Logstash, so please excuse my ignorance. I have multple tables that I'd like to read using the JDBC input. They have different schemas, different purposes, etc. Consuming these data as an end-user, they probably will end up with different visualisations, albeit on the same dashboard on Kibana. Therefore, I'm thinking that I will put them in different indices in elasticsearch - my first question is, am I on the right track thinking this way? If I do do that, although I'm sure I can configure it using one config file, I think each config file will be simpler and more maintainable if I split them into multiple pipelines... Are there any best practices guide relating to this? What do you guys do in such cases? Thanks, Justin ---
**Author:** ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Post date:** [March 6, 2018, 2:58am UTC](https://discuss.elastic.co/t/multiple-tables-as-input-multiple-pipelines/122630/2 "2018-03-06T02:58:52Z")
Sounds like you are on the right track. I have no experience with the JDBC input but reading the documentation makes it seem pretty straight-forward. I'd suggest that for each table you ingest, it be given a unique index name and then in Kibana, create a unique Index Pattern. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 6, 2018, 6:58am UTC](https://discuss.elastic.co/t/multiple-tables-as-input-multiple-pipelines/122630/3 "2018-03-06T06:58:41Z")
> Therefore, I'm thinking that I will put them in different indices in elasticsearch - my first question is, am I on the right track thinking this way? Yes. You may want to have different mappings for fields with a given name and then you _have_ to store them in different indexes. > If I do do that, although I'm sure I can configure it using one config file, I think each config file will be simpler and more maintainable if I split them into multiple pipelines... Yes, perhaps. Multiple pipelines is probably the better choice here (unless you need a very large number of them I suppose) but either will work fine. ---
**Author:** ![justin\_Kim](https://avatars.discourse-cdn.com/v4/letter/j/e9a140/32.png) [@justin\_Kim](https://discuss.elastic.co/u/justin_Kim)\ **Post date:** [March 6, 2018, 11:20pm UTC](https://discuss.elastic.co/t/multiple-tables-as-input-multiple-pipelines/122630/4 "2018-03-06T23:20:37Z")
Hi Magnus, thanks for replying. I have another question though - I'm reading multiple different files (IIS log and SQL server log) using filebeat. They have generally the same problem as the JDBC input - they'll end up in different indices, contains different formats (and therefore filters) Should I create multiple pipelines with multiple beats input on different ports? Or would it be better to have one beats input? Thanks, Justin ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 7, 2018, 7:00am UTC](https://discuss.elastic.co/t/multiple-tables-as-input-multiple-pipelines/122630/5 "2018-03-07T07:00:26Z")
It's either way really. Personally I wouldn't bother splitting them because I'd want them to share some filters, and eventually I'd want to pick up additional log types and then the port allocation becomes unnecessary overhead. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 4, 2018, 7:00am UTC](https://discuss.elastic.co/t/multiple-tables-as-input-multiple-pipelines/122630/6 "2018-04-04T07:00:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.