# Can't use two ports and 2 types logs using beats **URL:** \ **Category:** Beats\ **Created:** [November 26, 2017, 12:50pm UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136 "2017-11-26T12:50:14Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![arye](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@arye](https://discuss.elastic.co/u/arye)\ **Post date:** [November 26, 2017, 12:50pm UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/1 "2017-11-26T12:50:14Z")
Hi All, I have configured two conf files using the followings: input { beats { port =\> 5044 type =\> "logbeat\_general" } } input { beats { port =\> 5045 type =\> "openlogtst" } } And configured the output conf as follow: output { if [type] == "openlogtst" { elasticsearch { hosts =\> "localhost:9200" manage\_template =\> false index =\> "open\_log\_tst-%{+YYYY.MM.dd}" document\_type =\> "sivanbeat-openstash" } } else { elasticsearch { hosts =\> "localhost:9200" manage\_template =\> false index =\> "logstash-%{+YYYY.MM.dd}" document\_type =\> "filebeat-appstash" } } } The idea was to be able to log different logs using beats on different servers with the type separate them and I can do analyzing on each as I want to. I do not know why But I can only log the filebeat-appstash and even using Port 5045. can someone assist me with that so i will be able tro understand what i did wrong and how I can fix it? thank You ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 26, 2017, 3:00pm UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/2 "2017-11-26T15:00:35Z")
Do you have monitoring installed so you can check if both input plugins are receiving data? ---
**Author:** ![arye](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@arye](https://discuss.elastic.co/u/arye)\ **Post date:** [November 27, 2017, 4:55am UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/3 "2017-11-27T04:55:43Z")
I have beats install on both and I can see that i am getting traffic on both ports using sudo lsof -i :5044 and 5045 - I can see that ESTABLISHMENT on both when I send the logs. The problem is that I am getting all the logs under the same type and i can NOT filter them out and use a template for each individual logging. There is nothing wrong with the communication - this is pure logic configuration issue that i would be happy to resolve as soon as possible so I would be testing more machines and see how that ELK is stashing logs by indexes and patterns. I would be happy if someone can assist me on that Thank you ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 27, 2017, 6:22am UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/4 "2017-11-27T06:22:07Z")
Do the documents end up in the correct index? Are you using the default Beats mapping templates? ---
**Author:** ![arye](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@arye](https://discuss.elastic.co/u/arye)\ **Post date:** [November 27, 2017, 7:15am UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/5 "2017-11-27T07:15:22Z")
As I have mentioned above: file one located at conf.d called 02-beats.conf - input { beats { port =\> 5044 type =\> "logbeat\_general" } } and second file 03-beats.conf- input { beats { port =\> 5045 type =\> "openlogtst" } } That I want to get from both ports and have each one with its own type. as mentioned above. and i have the out like this: output { if [type] == "openlogtst" { elasticsearch { hosts =\> "localhost:9200" manage\_template =\> false index =\> "open\_log\_tst-%{+YYYY.MM.dd}" document\_type =\> "sivanbeat-openstash" } } else { elasticsearch { hosts =\> "localhost:9200" manage\_template =\> false index =\> "logstash-%{+YYYY.MM.dd}" document\_type =\> "filebeat-appstash" } } } I wanted to have the output to elasticsearch with the type and index - i.e. if [type] == "openlogtst" then index =\> "open\_log\_tst-%{+YYYY.MM.dd}" document\_type =\> "sivanbeat-openstash" otherwise the default is taking place. But at this point the logs I get marked as \_type: filebeat-appstash and \_index: logstash-%{+YYYY.MM.dd} - It looks like the type in the input files is being ignored and If it did NOT had default at all I would not be getting any logs at all. Any idea why I can't control the logs input using beats the way i do it? I want to be able to modulate inputs for each system and target each log-stash with its own indexing pattern. I would be happy to get help on how to do that the correct way and make it work. Thank You ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 27, 2017, 7:34am UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/6 "2017-11-27T07:34:13Z")
Which version are you using? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [November 28, 2017, 2:30pm UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/7 "2017-11-28T14:30:49Z")
Please properly format logs and config files using the ``-button. The `type` field can not be set via the input. See the [beats plugin documentation](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html#_description). You can either tag events in in the input or have the type (or other custom field) being set in filebeat itself. Beats use the `@metadata` field to pass information like index and other information to logstash. The `@metadata` field is dropped by all outputs. You can easily simplify your output configuration by storing and re-using the index name in `@metadata`. Elasticsearch is removing support for `_type`. That's why beats and logstash set the document type to `docs` by default. You can still have a `type` field without setting the document\_type. ---
**Author:** ![arye](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@arye](https://discuss.elastic.co/u/arye)\ **Post date:** [December 3, 2017, 7:42am UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/8 "2017-12-03T07:42:32Z")
version 5.6.2 - would be upgrading to version 6.0 when I understand how? 🙂 ---
**Author:** ![arye](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@arye](https://discuss.elastic.co/u/arye)\ **Post date:** [December 3, 2017, 7:51am UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/9 "2017-12-03T07:51:04Z")
OK. Will do from now on. I have setup the document\_type on the yml of the filebeat on the server and that I got the type correctly and output the correct index. In order to undersatnd it more - Each server that send beats to ELK need to be pre-configured with the document\_type before hand? The control of the indexing type is on the side of the SERVER and not the ELK, correct? In order to change the type I should change the YML file on the SERVER senduing the logs and I ca not do that from within the ELK itself. I would be happy if I can get the following to work - I want to have three indexes seperated using type on each. One is coming using Filebeats the second is coming using http (API calls) and the last should be the default . I have tried with `if [type]=="filebeat"{.....} else if [type]=="apicall"{......} else {......}` I could not catch the http\_input\_plugin type when I configured it in the input config file. How can I do that? Thanks, Arye ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [December 4, 2017, 1:28pm UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/10 "2017-12-04T13:28:10Z")
You can use `tags` or `type` in the http input. For debugging, it's sometimes helpful to replace the output section with ```auto output { stdout { codec => rubydebug } } ``` and test with a few events of one or the other kind only. This get's you an idea about fields being actually available. Using ruby one can even add some debug output in between filters: ```auto filter { ... ruby { init => "require 'json'" code => "puts 'myfilter'; puts JSON.pretty_generate(event); puts '=' * 80" } ... } ``` This will print: ```auto myfilter {....} # <- json encoded event ================================================================================ ``` to console. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 17, 2017, 12:50pm UTC](https://discuss.elastic.co/t/cant-use-two-ports-and-2-types-logs-using-beats/109136/11 "2017-12-17T12:50:19Z")
This topic was automatically closed after 21 days. New replies are no longer allowed.