1# A minimal logging API for Go
2
3[](https://pkg.go.dev/github.com/go-logr/logr)
4[](https://goreportcard.com/report/github.com/go-logr/logr)
5[](https://securityscorecards.dev/viewer/?platform=github.com&org=go-logr&repo=logr)
6
7logr offers an(other) opinion on how Go programs and libraries can do logging
8without becoming coupled to a particular logging implementation. This is not
9an implementation of logging - it is an API. In fact it is two APIs with two
10different sets of users.
11
12The `Logger` type is intended for application and library authors. It provides
13a relatively small API which can be used everywhere you want to emit logs. It
14defers the actual act of writing logs (to files, to stdout, or whatever) to the
15`LogSink` interface.
16
17The `LogSink` interface is intended for logging library implementers. It is a
18pure interface which can be implemented by logging frameworks to provide the actual logging
19functionality.
20
21This decoupling allows application and library developers to write code in
22terms of `logr.Logger` (which has very low dependency fan-out) while the
23implementation of logging is managed "up stack" (e.g. in or near `main()`.)
24Application developers can then switch out implementations as necessary.
25
26Many people assert that libraries should not be logging, and as such efforts
27like this are pointless. Those people are welcome to convince the authors of
28the tens-of-thousands of libraries that *DO* write logs that they are all
29wrong. In the meantime, logr takes a more practical approach.
30
31## Typical usage
32
33Somewhere, early in an application's life, it will make a decision about which
34logging library (implementation) it actually wants to use. Something like:
35
36```
37 func main() {
38 // ... other setup code ...
39
40 // Create the "root" logger. We have chosen the "logimpl" implementation,
41 // which takes some initial parameters and returns a logr.Logger.
42 logger := logimpl.New(param1, param2)
43
44 // ... other setup code ...
45```
46
47Most apps will call into other libraries, create structures to govern the flow,
48etc. The `logr.Logger` object can be passed to these other libraries, stored
49in structs, or even used as a package-global variable, if needed. For example:
50
51```
52 app := createTheAppObject(logger)
53 app.Run()
54```
55
56Outside of this early setup, no other packages need to know about the choice of
57implementation. They write logs in terms of the `logr.Logger` that they
58received:
59
60```
61 type appObject struct {
62 // ... other fields ...
63 logger logr.Logger
64 // ... other fields ...
65 }
66
67 func (app *appObject) Run() {
68 app.logger.Info("starting up", "timestamp", time.Now())
69
70 // ... app code ...
71```
72
73## Background
74
75If the Go standard library had defined an interface for logging, this project
76probably would not be needed. Alas, here we are.
77
78When the Go developers started developing such an interface with
79[slog](https://github.com/golang/go/issues/56345), they adopted some of the
80logr design but also left out some parts and changed others:
81
82| Feature | logr | slog |
83|---------|------|------|
84| High-level API | `Logger` (passed by value) | `Logger` (passed by [pointer](https://github.com/golang/go/issues/59126)) |
85| Low-level API | `LogSink` | `Handler` |
86| Stack unwinding | done by `LogSink` | done by `Logger` |
87| Skipping helper functions | `WithCallDepth`, `WithCallStackHelper` | [not supported by Logger](https://github.com/golang/go/issues/59145) |
88| Generating a value for logging on demand | `Marshaler` | `LogValuer` |
89| Log levels | >= 0, higher meaning "less important" | positive and negative, with 0 for "info" and higher meaning "more important" |
90| Error log entries | always logged, don't have a verbosity level | normal log entries with level >= `LevelError` |
91| Passing logger via context | `NewContext`, `FromContext` | no API |
92| Adding a name to a logger | `WithName` | no API |
93| Modify verbosity of log entries in a call chain | `V` | no API |
94| Grouping of key/value pairs | not supported | `WithGroup`, `GroupValue` |
95| Pass context for extracting additional values | no API | API variants like `InfoCtx` |
96
97The high-level slog API is explicitly meant to be one of many different APIs
98that can be layered on top of a shared `slog.Handler`. logr is one such
99alternative API, with [interoperability](#slog-interoperability) provided by
100some conversion functions.
101
102### Inspiration
103
104Before you consider this package, please read [this blog post by the
105inimitable Dave Cheney][warning-makes-no-sense]. We really appreciate what
106he has to say, and it largely aligns with our own experiences.
107
108### Differences from Dave's ideas
109
110The main differences are:
111
1121. Dave basically proposes doing away with the notion of a logging API in favor
113of `fmt.Printf()`. We disagree, especially when you consider things like output
114locations, timestamps, file and line decorations, and structured logging. This
115package restricts the logging API to just 2 types of logs: info and error.
116
117Info logs are things you want to tell the user which are not errors. Error
118logs are, well, errors. If your code receives an `error` from a subordinate
119function call and is logging that `error` *and not returning it*, use error
120logs.
121
1222. Verbosity-levels on info logs. This gives developers a chance to indicate
123arbitrary grades of importance for info logs, without assigning names with
124semantic meaning such as "warning", "trace", and "debug." Superficially this
125may feel very similar, but the primary difference is the lack of semantics.
126Because verbosity is a numerical value, it's safe to assume that an app running
127with higher verbosity means more (and less important) logs will be generated.
128
129## Implementations (non-exhaustive)
130
131There are implementations for the following logging libraries:
132
133- **a function** (can bridge to non-structured libraries): [funcr](https://github.com/go-logr/logr/tree/master/funcr)
134- **a testing.T** (for use in Go tests, with JSON-like output): [testr](https://github.com/go-logr/logr/tree/master/testr)
135- **github.com/google/glog**: [glogr](https://github.com/go-logr/glogr)
136- **k8s.io/klog** (for Kubernetes): [klogr](https://git.k8s.io/klog/klogr)
137- **a testing.T** (with klog-like text output): [ktesting](https://git.k8s.io/klog/ktesting)
138- **go.uber.org/zap**: [zapr](https://github.com/go-logr/zapr)
139- **log** (the Go standard library logger): [stdr](https://github.com/go-logr/stdr)
140- **github.com/sirupsen/logrus**: [logrusr](https://github.com/bombsimon/logrusr)
141- **github.com/wojas/genericr**: [genericr](https://github.com/wojas/genericr) (makes it easy to implement your own backend)
142- **logfmt** (Heroku style [logging](https://www.brandur.org/logfmt)): [logfmtr](https://github.com/iand/logfmtr)
143- **github.com/rs/zerolog**: [zerologr](https://github.com/go-logr/zerologr)
144- **github.com/go-kit/log**: [gokitlogr](https://github.com/tonglil/gokitlogr) (also compatible with github.com/go-kit/kit/log since v0.12.0)
145- **bytes.Buffer** (writing to a buffer): [bufrlogr](https://github.com/tonglil/buflogr) (useful for ensuring values were logged, like during testing)
146
147## slog interoperability
148
149Interoperability goes both ways, using the `logr.Logger` API with a `slog.Handler`
150and using the `slog.Logger` API with a `logr.LogSink`. `FromSlogHandler` and
151`ToSlogHandler` convert between a `logr.Logger` and a `slog.Handler`.
152As usual, `slog.New` can be used to wrap such a `slog.Handler` in the high-level
153slog API.
154
155### Using a `logr.LogSink` as backend for slog
156
157Ideally, a logr sink implementation should support both logr and slog by
158implementing both the normal logr interface(s) and `SlogSink`. Because
159of a conflict in the parameters of the common `Enabled` method, it is [not
160possible to implement both slog.Handler and logr.Sink in the same
161type](https://github.com/golang/go/issues/59110).
162
163If both are supported, log calls can go from the high-level APIs to the backend
164without the need to convert parameters. `FromSlogHandler` and `ToSlogHandler` can
165convert back and forth without adding additional wrappers, with one exception:
166when `Logger.V` was used to adjust the verbosity for a `slog.Handler`, then
167`ToSlogHandler` has to use a wrapper which adjusts the verbosity for future
168log calls.
169
170Such an implementation should also support values that implement specific
171interfaces from both packages for logging (`logr.Marshaler`, `slog.LogValuer`,
172`slog.GroupValue`). logr does not convert those.
173
174Not supporting slog has several drawbacks:
175- Recording source code locations works correctly if the handler gets called
176 through `slog.Logger`, but may be wrong in other cases. That's because a
177 `logr.Sink` does its own stack unwinding instead of using the program counter
178 provided by the high-level API.
179- slog levels <= 0 can be mapped to logr levels by negating the level without a
180 loss of information. But all slog levels > 0 (e.g. `slog.LevelWarning` as
181 used by `slog.Logger.Warn`) must be mapped to 0 before calling the sink
182 because logr does not support "more important than info" levels.
183- The slog group concept is supported by prefixing each key in a key/value
184 pair with the group names, separated by a dot. For structured output like
185 JSON it would be better to group the key/value pairs inside an object.
186- Special slog values and interfaces don't work as expected.
187- The overhead is likely to be higher.
188
189These drawbacks are severe enough that applications using a mixture of slog and
190logr should switch to a different backend.
191
192### Using a `slog.Handler` as backend for logr
193
194Using a plain `slog.Handler` without support for logr works better than the
195other direction:
196- All logr verbosity levels can be mapped 1:1 to their corresponding slog level
197 by negating them.
198- Stack unwinding is done by the `SlogSink` and the resulting program
199 counter is passed to the `slog.Handler`.
200- Names added via `Logger.WithName` are gathered and recorded in an additional
201 attribute with `logger` as key and the names separated by slash as value.
202- `Logger.Error` is turned into a log record with `slog.LevelError` as level
203 and an additional attribute with `err` as key, if an error was provided.
204
205The main drawback is that `logr.Marshaler` will not be supported. Types should
206ideally support both `logr.Marshaler` and `slog.Valuer`. If compatibility
207with logr implementations without slog support is not important, then
208`slog.Valuer` is sufficient.
209
210### Context support for slog
211
212Storing a logger in a `context.Context` is not supported by
213slog. `NewContextWithSlogLogger` and `FromContextAsSlogLogger` can be
214used to fill this gap. They store and retrieve a `slog.Logger` pointer
215under the same context key that is also used by `NewContext` and
216`FromContext` for `logr.Logger` value.
217
218When `NewContextWithSlogLogger` is followed by `FromContext`, the latter will
219automatically convert the `slog.Logger` to a
220`logr.Logger`. `FromContextAsSlogLogger` does the same for the other direction.
221
222With this approach, binaries which use either slog or logr are as efficient as
223possible with no unnecessary allocations. This is also why the API stores a
224`slog.Logger` pointer: when storing a `slog.Handler`, creating a `slog.Logger`
225on retrieval would need to allocate one.
226
227The downside is that switching back and forth needs more allocations. Because
228logr is the API that is already in use by different packages, in particular
229Kubernetes, the recommendation is to use the `logr.Logger` API in code which
230uses contextual logging.
231
232An alternative to adding values to a logger and storing that logger in the
233context is to store the values in the context and to configure a logging
234backend to extract those values when emitting log entries. This only works when
235log calls are passed the context, which is not supported by the logr API.
236
237With the slog API, it is possible, but not
238required. https://github.com/veqryn/slog-context is a package for slog which
239provides additional support code for this approach. It also contains wrappers
240for the context functions in logr, so developers who prefer to not use the logr
241APIs directly can use those instead and the resulting code will still be
242interoperable with logr.
243
244## FAQ
245
246### Conceptual
247
248#### Why structured logging?
249
250- **Structured logs are more easily queryable**: Since you've got
251 key-value pairs, it's much easier to query your structured logs for
252 particular values by filtering on the contents of a particular key --
253 think searching request logs for error codes, Kubernetes reconcilers for
254 the name and namespace of the reconciled object, etc.
255
256- **Structured logging makes it easier to have cross-referenceable logs**:
257 Similarly to searchability, if you maintain conventions around your
258 keys, it becomes easy to gather all log lines related to a particular
259 concept.
260
261- **Structured logs allow better dimensions of filtering**: if you have
262 structure to your logs, you've got more precise control over how much
263 information is logged -- you might choose in a particular configuration
264 to log certain keys but not others, only log lines where a certain key
265 matches a certain value, etc., instead of just having v-levels and names
266 to key off of.
267
268- **Structured logs better represent structured data**: sometimes, the
269 data that you want to log is inherently structured (think tuple-link
270 objects.) Structured logs allow you to preserve that structure when
271 outputting.
272
273#### Why V-levels?
274
275**V-levels give operators an easy way to control the chattiness of log
276operations**. V-levels provide a way for a given package to distinguish
277the relative importance or verbosity of a given log message. Then, if
278a particular logger or package is logging too many messages, the user
279of the package can simply change the v-levels for that library.
280
281#### Why not named levels, like Info/Warning/Error?
282
283Read [Dave Cheney's post][warning-makes-no-sense]. Then read [Differences
284from Dave's ideas](#differences-from-daves-ideas).
285
286#### Why not allow format strings, too?
287
288**Format strings negate many of the benefits of structured logs**:
289
290- They're not easily searchable without resorting to fuzzy searching,
291 regular expressions, etc.
292
293- They don't store structured data well, since contents are flattened into
294 a string.
295
296- They're not cross-referenceable.
297
298- They don't compress easily, since the message is not constant.
299
300(Unless you turn positional parameters into key-value pairs with numerical
301keys, at which point you've gotten key-value logging with meaningless
302keys.)
303
304### Practical
305
306#### Why key-value pairs, and not a map?
307
308Key-value pairs are *much* easier to optimize, especially around
309allocations. Zap (a structured logger that inspired logr's interface) has
310[performance measurements](https://github.com/uber-go/zap#performance)
311that show this quite nicely.
312
313While the interface ends up being a little less obvious, you get
314potentially better performance, plus avoid making users type
315`map[string]string{}` every time they want to log.
316
317#### What if my V-levels differ between libraries?
318
319That's fine. Control your V-levels on a per-logger basis, and use the
320`WithName` method to pass different loggers to different libraries.
321
322Generally, you should take care to ensure that you have relatively
323consistent V-levels within a given logger, however, as this makes deciding
324on what verbosity of logs to request easier.
325
326#### But I really want to use a format string!
327
328That's not actually a question. Assuming your question is "how do
329I convert my mental model of logging with format strings to logging with
330constant messages":
331
3321. Figure out what the error actually is, as you'd write in a TL;DR style,
333 and use that as a message.
334
3352. For every place you'd write a format specifier, look to the word before
336 it, and add that as a key value pair.
337
338For instance, consider the following examples (all taken from spots in the
339Kubernetes codebase):
340
341- `klog.V(4).Infof("Client is returning errors: code %v, error %v",
342 responseCode, err)` becomes `logger.Error(err, "client returned an
343 error", "code", responseCode)`
344
345- `klog.V(4).Infof("Got a Retry-After %ds response for attempt %d to %v",
346 seconds, retries, url)` becomes `logger.V(4).Info("got a retry-after
347 response when requesting url", "attempt", retries, "after
348 seconds", seconds, "url", url)`
349
350If you *really* must use a format string, use it in a key's value, and
351call `fmt.Sprintf` yourself. For instance: `log.Printf("unable to
352reflect over type %T")` becomes `logger.Info("unable to reflect over
353type", "type", fmt.Sprintf("%T"))`. In general though, the cases where
354this is necessary should be few and far between.
355
356#### How do I choose my V-levels?
357
358This is basically the only hard constraint: increase V-levels to denote
359more verbose or more debug-y logs.
360
361Otherwise, you can start out with `0` as "you always want to see this",
362`1` as "common logging that you might *possibly* want to turn off", and
363`10` as "I would like to performance-test your log collection stack."
364
365Then gradually choose levels in between as you need them, working your way
366down from 10 (for debug and trace style logs) and up from 1 (for chattier
367info-type logs). For reference, slog pre-defines -4 for debug logs
368(corresponds to 4 in logr), which matches what is
369[recommended for Kubernetes](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md#what-method-to-use).
370
371#### How do I choose my keys?
372
373Keys are fairly flexible, and can hold more or less any string
374value. For best compatibility with implementations and consistency
375with existing code in other projects, there are a few conventions you
376should consider.
377
378- Make your keys human-readable.
379- Constant keys are generally a good idea.
380- Be consistent across your codebase.
381- Keys should naturally match parts of the message string.
382- Use lower case for simple keys and
383 [lowerCamelCase](https://en.wiktionary.org/wiki/lowerCamelCase) for
384 more complex ones. Kubernetes is one example of a project that has
385 [adopted that
386 convention](https://github.com/kubernetes/community/blob/HEAD/contributors/devel/sig-instrumentation/migration-to-structured-logging.md#name-arguments).
387
388While key names are mostly unrestricted (and spaces are acceptable),
389it's generally a good idea to stick to printable ascii characters, or at
390least match the general character set of your log lines.
391
392#### Why should keys be constant values?
393
394The point of structured logging is to make later log processing easier. Your
395keys are, effectively, the schema of each log message. If you use different
396keys across instances of the same log line, you will make your structured logs
397much harder to use. `Sprintf()` is for values, not for keys!
398
399#### Why is this not a pure interface?
400
401The Logger type is implemented as a struct in order to allow the Go compiler to
402optimize things like high-V `Info` logs that are not triggered. Not all of
403these implementations are implemented yet, but this structure was suggested as
404a way to ensure they *can* be implemented. All of the real work is behind the
405`LogSink` interface.
406
407[warning-makes-no-sense]: http://dave.cheney.net/2015/11/05/lets-talk-about-logging