Relative Content

Tag Archive for elk

How to identify and extract field data in log files in ELK

We have many disparate (some node, some tomcat, some Spring) services which provide rest API to other services or web apps.
Each service writes logs which are pulled into ELK.
Log entries are unstructured, but most start with the date, and most have INFO/ERROR etc.
Elk doesn’t understand what’s in the log lines, you can only search on keyword. You cant even search using the log files time stamp, as ELK stores its @timestmap field as when the log file was ingested, not the time stamp of the log line in question. Elk has no concept of fields like a database would, its just a list of strings.