# File in S3 has additional timestamp and host name as prefix to each message processed

**URL:** <https://discuss.elastic.co/t/file-in-s3-has-additional-timestamp-and-host-name-as-prefix-to-each-message-processed/172571>\
**Category:** Logstash\
**Created:** [March 15, 2019, 4:26pm UTC](https://discuss.elastic.co/t/file-in-s3-has-additional-timestamp-and-host-name-as-prefix-to-each-message-processed/172571 "2019-03-15T16:26:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![kaushik.vankayala](https://avatars.discourse-cdn.com/v4/letter/k/aca169/32.png) [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\
**Post date:** [March 15, 2019, 4:26pm UTC](https://discuss.elastic.co/t/file-in-s3-has-additional-timestamp-and-host-name-as-prefix-to-each-message-processed/172571/1 "2019-03-15T16:26:29Z")

</div>

Hi There,

I am having S3 as one of the output plugins in my logstash conf. Belos is my s3 plugin configuration details;

```
s3
{
	id => "uat-s3-backup"
	bucket => "mt-uat-project"
	access_key_id => "BK7U8I90DGFN236BQ62A"
	secret_access_key => "uwvzbkNMSBFNSBDFKJDFKJSAHEvOX14rkZRBl5W"
	region => "ap-south-1"
	prefix => "%{+YYYY}/%{+MMM}/%{+dd}"
}

```

The sample content of the s3 text file is;

```
2019-03-14T00:11:23.679Z ZE34P2907 [2019-03-14 05:41:23.679] INFO Audit-IN [[project].HTTP_Experience_Listener_Configuration.worker.142]: {"APITransactionId": "2bcdfbad9abc4862ba63472178c28fdd","CorrelationId": "cd72b9a0-461b-11e9-b9b4-02ee19ab148c","Request": "GET /api/experience/member-profile/v1/fetch-profile","TimeStamp": "2019-03-14 11:11:23.679","ApplicationName": "experience-app","Environment": "Production","SourceIP": "13.229.125.192:32238","PartnerID": "Partner","PartnerUniqueID": "1100f4932232481c85910debecc71e9b"} 
2019-03-14T00:00:28.900Z ZE34P2907 [2019-03-14 05:30:28.900] INFO Audit-IN [[project].HTTP_Experience_Listener_Configuration.worker.142]: {"APITransactionId": "41621fd54a2c4c6081b999313f235e94","CorrelationId": "47296520-461a-11e9-b9b4-02ee19ab148c","Request": "GET /api/experience/member-profile/v1/fetch-profile","TimeStamp": "2019-03-14 11:00:28.900","ApplicationName": "experience-app","Environment": "Production","SourceIP": "13.229.125.192:31438","PartnerID": "Partner","PartnerUniqueID": "1100f4932232481c85910debecc71e9b"} 
2019-03-14T00:00:29.178Z ZE34P2907 [2019-03-14 05:30:29.178] INFO Audit-OUT [[project].HTTP_Experience_Listener_Configuration.worker.142]: {"APITransactionId": "41621fd54a2c4c6081b999313f235e94","StatusCode": "200","TimeStamp": "2019-03-14 11:00:29.178","PartnerID": "Partner","PartnerUniqueID": "1100f4932232481c85910debecc71e9b","Request": "/api/experience/member-profile/v1/fetch-profile","ServiceType": "Service", "ResponseTime": "327 ms","ResponseDescription": ""}
2019-03-14T00:04:34.790Z ZE34P2907 [2019-03-14 05:34:34.790] INFO Health-Check [[project].HTTP_Experience_Listener_Configuration.worker.142]: {"http.status": 200, "Server Status": "Online", "Environment": "Production"}
2019-03-14T00:04:36.942Z ZE34P2907 [2019-03-14 05:34:36.942] INFO Health-Check [[project].HTTP_Experience_Listener_Configuration.worker.142]: {"http.status": 200, "Server Status": "Online", "Environment": "Production"}
2019-03-14T00:04:37.045Z ZE34P2907 [2019-03-14 05:34:37.045] INFO Health-Check [[project].HTTP_Experience_Listener_Configuration.worker.142]: {"http.status": 200, "Server Status": "Online", "Environment": "Production"}
2019-03-14T00:04:37.101Z ZE34P2907 [2019-03-14 05:34:37.101] INFO Health-Check [[project].HTTP_Experience_Listener_Configuration.worker.142]: {"http.status": 200, "Server Status": "Online", "Environment": "Production"}
2019-03-14T00:04:37.155Z ZE34P2907 [2019-03-14 05:34:37.155] INFO Health-Check [[project].HTTP_Experience_Listener_Configuration.worker.142]: {"http.status": 200, "Server Status": "Online", "Environment": "Production"}

```

The first two columns are not part of the original message and they are timestamps and the host name.

Does anyone know how to not print them in the s3 files?

Regards

Kaushik

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 15, 2019, 4:45pm UTC](https://discuss.elastic.co/t/file-in-s3-has-additional-timestamp-and-host-name-as-prefix-to-each-message-processed/172571/2 "2019-03-15T16:45:56Z")

</div>

The default codec for s3 is [line](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-line.html). The default format for a line codec prepends message with @timestamp and host. Set the codec format explicitly in your output.

```
codec => line { format => "%{message}" }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2019, 4:46pm UTC](https://discuss.elastic.co/t/file-in-s3-has-additional-timestamp-and-host-name-as-prefix-to-each-message-processed/172571/3 "2019-04-12T16:46:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.