# Sending logging data to ES using tcp input plugin, no use of file
**URL:** https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159
**Category:** Logstash
**Created:** [June 29, 2019, 10:10pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159 "2019-06-29T22:10:54Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [June 29, 2019, 10:10pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/1 "2019-06-29T22:10:54Z")
Hi All
We get very good log file in LS, but i want to mute this file logging and send the logging data directly to ES
I am using following configuration, but it is not working. It does not show anything in console and does not create index
> input {
> tcp {
> host =\> localhost
> port =\> 3456
> codec =\> json
> ssl\_enable =\> false
> }
> }
> filter {
> date {
> match =\> ["timeMillis", "UNIX\_MS"]
> }
> }
I have updated the log4j.properties file like this in LS config folder
```
log4j.rootLogger=DEBUG,tcp
log4j.appender.tcp=org.apache.log4j.net.SocketAppender
log4j.appender.tcp.Port=3456
log4j.appender.tcp.RemoteHost=localhost
log4j.appender.tcp.ReconnectionDelay=10000
log4j.appender.tcp.Application=test
log4j.appender.tcp.append=true
```
while i am running other pipeline in parallel in port 9006
I saw, some guys asked similar que, but no one replied those. Did anyone tried this?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 29, 2019, 11:16pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/2 "2019-06-29T23:16:38Z")
The following works in 7.2.0
```
rootLogger.appenderRef.tcp.ref = JsonOverTcp
appender.json_tcp.name = JsonOverTcp
appender.json_tcp.layout.type = JSONLayout
appender.json_tcp.layout.compact = true
appender.json_tcp.layout.eventEol = true
appender.json_tcp.type = Socket
appender.json_tcp.protocol = TCP
appender.json_tcp.port = 3456
appender.json_tcp.host = localhost
appender.json_tcp.reconnectionDelayMillis = 10000
```
That will get you messages like
```
{
"@timestamp" => 2019-06-29T23:05:55.946Z,
"level" => "WARN",
"port" => 56570,
"host" => "localhost",
"thread" => "SIGINT handler",
"loggerName" => "logstash.runner",
"timeMillis" => 1561849555604,
"logEvent" => {
"message" => "SIGINT received. Shutting down."
}
[...]
}
```
That said, I do not like the setup very much. You miss all the messages at startup, because the tcp input is not listening, and log4j appears not to buffer if the output is unavailable. I could easily see you missing messages during shutdown, because the input gets closed. (For example, if logstash crashes, you may miss the reason why.) Finally if you are sending the log of an instance back through the same instance that created them there could be weird cases where processing a message results in a message, and it goes in to an infinite loop (or even worse an exponential increase in volume). If you are going to do this I strongly recommend that you have a dedicated logstash instance (not a pipeline, an instance) that just feeds the tcp input into elasticsearch (a date filter should be OK 🙂 ).
---
### Author:  [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [July 2, 2019, 5:19am UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/3 "2019-07-02T05:19:55Z")
Hi, thanks for the properties file, with the given property file, its working.
I see whats the limitation with this solution, that you described above
Will it be possible to extend the field value.
How can i expand this for
pipeline.id
node.name
etc.
> {
> "@timestamp" =\> 2019-06-29T23:05:55.946Z,
> "level" =\> "WARN",
> "port" =\> 56570,
> "host" =\> "localhost",
> "thread" =\> "SIGINT handler",
> "loggerName" =\> "logstash.runner",
> "timeMillis" =\> 1561849555604,
> "logEvent" =\> {
> "message" =\> "SIGINT received. Shutting down."
> }
---
### Author:  [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [July 2, 2019, 4:48pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/4 "2019-07-02T16:48:59Z")
Hi @Badger
"port" =\> 56570,
and the port we get in LS 9600, both are not same
Can you please let me know this 56570 port is for what and from where it is coming
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 2, 2019, 4:56pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/5 "2019-07-02T16:56:53Z")
I believe 56570 is the port on the client side of the TCP connection. I cannot think of a use for it, but I guess it does no harm.
---
### Author:  [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [July 2, 2019, 5:00pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/6 "2019-07-02T17:00:18Z")
The reason i have asked because
i run multiple pipeline, by using --path.data from the same instance and it does create different port for each pipeline like 9600,9601....
But with this port# 56570, it is difficult to identify the response is for which pipeline
That is the reason , i was asking for how to change the tcp response or extend the response by pipeline.id, so that can identify the log message is for which pipeline
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 2, 2019, 5:22pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/7 "2019-07-02T17:22:29Z")
If you are using a multiple tcp inputs you can use the [tags](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-tcp.html#plugins-inputs-tcp-tags) option to label the events.
---
### Author:  [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [July 2, 2019, 5:56pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/8 "2019-07-02T17:56:48Z")
i am using one pipeline whose input is tcp and output is ES
And multiple other pipelines to do logging of all the events to ES( from above pipeline). In ES i want to identify which logs events for which pipeline.
Tags may add other errors tags as well, so may not be useful. I am looking for ways to extended the response from tcp input
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 30, 2019, 5:56pm UTC](https://discuss.elastic.co/t/sending-logging-data-to-es-using-tcp-input-plugin-no-use-of-file/188159/9 "2019-07-30T17:56:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.