File beat throwing warning for custom index pattern

Elastic Stack Beats
1

Hi
I am seeing this weird issue on my file beat module 6.6.2 .

When I have default configuration in the filebeat.yml and iis module enabled. The file beat sends logs fine but when I try to add an index pattern it is throwing error in logs and not sendiing logs to elastic search. Please let me know what is wrong with my configuration.

Here is my configuration.

#==================== Elasticsearch template setting ==========================
setup.template.name: 'filebeat-win'
setup.template.pattern: 'filebeat-win*'
setup.template.settings:
  index.number_of_shards: 3
  #index.codec: best_compression
  #_source.enabled: false
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["reldev0000318.ags.esri.com:9200"]
  index: 'filebeat-win-%{[host][name]}-%{+yyyy.MM.dd}'

File Beat Logs ==

        2019-03-20T09:40:10.424-0700	WARN	elasticsearch/client.go:523	Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbf1cb91a5845a4f8, ext:8840551301, loc:(*time.Location)(0x19e7840)}, Meta:common.MapStr{"pipeline":"filebeat-6.6.2-iis-access-default"}, Fields:common.MapStr{"input":common.MapStr{"type":"log"}, "beat":common.MapStr{"name":"Changedhostnameforsecurity", "hostname":"Changedhostnameforsecurity", "version":"6.6.2"}, "log":common.MapStr{"file":common.MapStr{"path":"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex181117.log"}}, "source":"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex181117.log", "offset":541655, "prospector":common.MapStr{"type":"log"}, "fileset":common.MapStr{"name":"access", "module":"iis"}, "event":common.MapStr{"dataset":"iis.access"}, "host":common.MapStr{"name":"Changedhostnameforsecurity", "architecture":"x86_64", "os":common.MapStr{"family":"windows", "name":"Windows Server 2012 R2 Standard", "build":"9600.19302", "platform":"windows", "version":"6.3"}, "id":"b2bde5bc-d73d-488f-bc74-30b392ef71ef"}, "message":"2018-11-17 23:38:47 10.44.10.223 HEAD /portal/home/ - 443 - 10.49.102.104 - - 200 0 0 61"}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc0421e1b00), Source:"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex181117.log", Offset:541745, Timestamp:time.Time{wall:0xbf1cb9199ac3abbc, ext:5882384301, loc:(*time.Location)(0x19e7840)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{IdxHi:0x1870000, IdxLo:0x23f84, Vol:0xeee5eeed}}}, Flags:0x1} (status=400): {"type":"invalid_index_name_exception","reason":"Invalid index name [filebeat-win-changedhostname for security-2019.03.20], must be lowercase","index_uuid":"_na_","index":"filebeat-win-changedhostname for security-2019.03.20"}
        2019-03-20T09:40:10.424-0700	WARN	elasticsearch/client.go:523	Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbf1cb91a5845a4f8, ext:8840551301, loc:(*time.Location)(0x19e7840)}, Meta:common.MapStr{"pipeline":"filebeat-6.6.2-iis-access-default"}, Fields:common.MapStr{"source":"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex181117.log", "prospector":common.MapStr{"type":"log"}, "input":common.MapStr{"type":"log"}, "fileset":common.MapStr{"module":"iis", "name":"access"}, "event":common.MapStr{"dataset":"iis.access"}, "log":common.MapStr{"file":common.MapStr{"path":"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex181117.log"}}, "message":"2018-11-17 23:54:02 10.44.10.223 HEAD /portal/home/ - 443 - 10.49.102.104 - - 200 0 0 26", "host":common.MapStr{"name":"changedhostname for security", "id":"b2bde5bc-d73d-488f-bc74-30b392ef71ef", "architecture":"x86_64", "os":common.MapStr{"name":"Windows Server 2012 R2 Standard", "build":"9600.19302", "platform":"windows", "version":"6.3", "family":"windows"}}, "offset":541745, "beat":common.MapStr{"version":"6.6.2", "name":"changedhostname for security", "hostname":"Changedhostnameforsecurity"}}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc0421e1b00), Source:"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex181117.log", Offset:541835, Timestamp:time.Time{wall:0xbf1cb9199ac3abbc, ext:5882384301, loc:(*time.Location)(0x19e7840)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{IdxHi:0x1870000, IdxLo:0x23f84, Vol:0xeee5eeed}}}, Flags:0x1} (status=400): {"type":"invalid_index_name_exception","reason":"Invalid index name [filebeat-win-Changedhostnameforsecurity-2019.03.20], must be lowercase","index_uuid":"_na_","index":"filebeat-win-Changedhostnameforsecurity-2019.03.20"}
        2019-03-20T09:40:33.902-0700	INFO	[monitoring]	log/log.go:144	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":3515,"time":{"ms":3515}},"total":{"ticks":9796,"time":{"ms":9796},"value":9796},"user":{"ticks":6281,"time":{"ms":6281}}},"handles":{"open":378},"info":{"ephemeral_id":"7c83ed07-0fae-477a-9404-e94890fb73c8","uptime":{"ms":33126}},"memstats":{"gc_next":58327984,"memory_alloc":53029696,"memory_total":796026240,"rss":101793792}},"filebeat":{"events":{"added":56502,"done":56502},"harvester":{"open_files":160,"running":160,"started":160}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"events":{"batches":865,"dropped":43187,"total":43187},"read":{"bytes":381220},"type":"elasticsearch","write":{"bytes":39382519}},"pipeline":{"clients":4,"events":{"active":0,"filtered":13315,"published":43187,"retry":50,"total":56502},"queue":{"acked":43187}}},"registrar":{"states":{"current":160,"update":56502},"writes":{"success":587,"total":587}},"system":{"cpu":{"cores":4}}}}}
2

I see that in my logs the hostname is in higher letters like "Server123" and Filebeat is not liking this in the index name. How can I make kibana ignore this.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.