Loki is a centralised log aggregation system.
...
It stores the logs in chunks which are then compressed to significantly reduce the cost of loki.
Log Flow using PLG Logging Stack
.png?version=1&modificationDate=1622902654831&cacheVersion=1&api=v2)
The Application writes different level of logs to a log file.
Promtail is one of the agents of loki which can be configured to look for logs in a folder or a file.
Promtail Discovers the log files and monitors them and attaches labels associated with each log as specified in the configuration file of promtail.
Once promtail discovers change in the logs file, it ships the logs to Loki.
Loki stores the logs in chunks and maintains the labels in indexes. The configuration file of Loki specifies where to store these chunks of logs and indexes.
Loki provides few APIs through which we can query(LogQL) the chunks to fetch the logs.
Loki Architecture
...
Read path:
The querier receives the request for logs.
First, the querier passes the request to ingester for in-memory logs. Ingester returns the data to querier.
Querier lazily loads/queries the backend storage for the logs.
The querier de-duplicate service removes any redundant data as it fetches from ingester and backend store.
The querier returns the logs to the front end.
...
For more information, do visit Loki Documentation.
Loki APIs
Loki provides APIs through which we can send LogQL queries as request parameters, through which and we can get log messages.
GET /loki/api/v1/query
...
For more information, refer Loki's HTTP API.
Loki and Promtail Installation
On Mac:
We can use brew to install both loki and promtail.
...