# LogLayer for Go: Comprehensive LLM Reference > Transport-agnostic structured logging for Go. A fluent API on top of zerolog, zap, logrus, phuslu/log, charmbracelet/log, or any custom transport. This is the comprehensive reference. For the concise index see llms.txt. Module path: `go.loglayer.dev`. GitHub: `github.com/loglayer/loglayer-go`. ## Core Concepts LogLayer separates structured data into three categories with distinct scopes: | Category | Method | Scope | Purpose | |----------|---------------|----------------------------|----------------------------------------| | Fields | WithFields | Persistent across all logs | Request IDs, user info, session data | | Metadata | WithMetadata | Single log entry only | Per-event details: durations, counts | | Errors | WithError | Single log entry only | An error value, serialized for output | Each category can be muted, renamed, or nested under a configurable field. ## API Surface ### Construction `loglayer.New(Config) *LogLayer` panics on misconfiguration (no transport set; reports `loglayer.ErrNoTransport`). For explicit error handling use `loglayer.Build(Config) (*LogLayer, error)`. ```go log := loglayer.New(loglayer.Config{ Transport: someTransport, // or Transports: [] Prefix: "[auth]", Enabled: ptr(true), ErrorSerializer: func(err error) map[string]any { ... }, ErrorFieldName: "err", CopyMsgOnOnlyError: false, FieldsKey: "", // empty = merge at root MuteFields: false, MuteMetadata: false, DisableFatalExit: false, // false = Fatal calls os.Exit(1) Source: SourceConfig{ Enabled: false, // capture file/line/function of every emission FieldName: "source", // default; output key for captured source }, Routing: RoutingConfig{ // group-based dispatch; zero value = no routing Groups: map[string]LogGroup{}, ActiveGroups: nil, Ungrouped: UngroupedRouting{}, }, }) // Or: log, err := loglayer.Build(loglayer.Config{...}) ``` ### Log Levels Seven levels, integer-typed: - `loglayer.LogLevelTrace` (5) - `loglayer.LogLevelDebug` (10) - `loglayer.LogLevelInfo` (20) - `loglayer.LogLevelWarn` (30) - `loglayer.LogLevelError` (40) - `loglayer.LogLevelFatal` (50) - `loglayer.LogLevelPanic` (60) Level methods on `*loglayer.LogLayer`: ```go log.Trace(...messages any) log.Debug(...messages any) log.Info(...messages any) log.Warn(...messages any) log.Error(...messages any) log.Fatal(...messages any) // dispatches then calls os.Exit(1) unless DisableFatalExit log.Panic(...messages any) // dispatches then panics with the joined message string (recoverable) ``` ### Builder Chain `WithMetadata` and `WithError` return `*LogBuilder`. Chain then terminate with a level method. `WithContext` on `*LogLayer` returns `*LogLayer` (binds ctx to subsequent emissions); on `*LogBuilder` it's a per-call override that returns `*LogBuilder`. ```go log.WithContext(ctx). WithMetadata(loglayer.Metadata{"k": "v"}). WithError(err). Error("failed") ``` A `*LogBuilder` is single-use. Calling `WithMetadata` twice replaces the value (it does not merge). ### Special Methods ```go log.MetadataOnly(loglayer.Metadata{...}) // logs metadata at Info log.MetadataOnly(loglayer.Metadata{...}, loglayer.LogLevelWarn) // override level log.ErrorOnly(err) // logs error at Error log.ErrorOnly(err, loglayer.ErrorOnlyOpts{LogLevel: ..., CopyMsg: ...}) log.Raw(loglayer.RawLogEntry{ // bypass builder LogLevel: ..., Messages: ..., Metadata: ..., Err: ..., Fields: ..., Ctx: ..., }) log = log.WithContext(ctx) // bind ctx to logger (returns *LogLayer) log.Info("...") // bound ctx propagates log.WithContext(otherCtx).Info("...") // per-call override (builder) ``` ### Fields ```go log = log.WithFields(loglayer.Fields{"requestId": "abc"}) // returns new logger; assign fields := log.GetFields() // shallow copy log = log.WithoutFields() // remove all (returns new logger) log = log.WithoutFields("key1", "key2") // remove specific (returns new logger) log.MuteFields().UnmuteFields() // suppress in output (mutate, atomic) ``` `loglayer.Fields` is a type alias for `map[string]any`. Fields are keyed because they support merge, clear-by-key, and copy-on-Child semantics that only make sense over keys. `WithFields` and `WithoutFields` return a new logger instead of mutating the receiver; this matches zerolog/zap/slog/logrus convention. The compiler does not catch a discarded return value, so always assign. ### Metadata `loglayer.Metadata` is a type alias for `map[string]any`. The two forms are runtime-identical; the alias is the idiomatic form in user code. `loglayer.M` is a shorter alias for `loglayer.Metadata` (same for `loglayer.F` ↔ `loglayer.Fields`); use them for terser call sites. ```go log.WithMetadata(loglayer.Metadata{"userId": 42}).Info("login") // map log.WithMetadata(loglayer.M{"userId": 42}).Info("login") // short alias log.WithMetadata(MyStruct{...}).Info("event") // struct ``` `WithMetadata` accepts `any`. The transport decides serialization: - structured, console, pretty: maps merge at root, structs JSON-roundtrip into root fields - zerolog, zap, phuslu, logrus: maps merge at root, structs nest under MetadataFieldName (default "metadata") - charmlog: maps flatten to keyvals, structs as a single keyval - otellog: maps flatten to typed KeyValue attributes (StringValue, Int64Value, etc., with recursive MapValue/SliceValue), structs JSON-roundtrip into a nested MapValue under MetadataFieldName (default "metadata") ### Errors ```go log.WithError(err).Error("failed") // attach to a log log.ErrorOnly(err) // log only the error ``` Default serialization: `{"message": err.Error()}`. Override via `Config.ErrorSerializer`. Recommended: `github.com/rotisserie/eris` whose `ToJSON(err, withTrace)` returns `map[string]any` directly. ```go import "github.com/rotisserie/eris" loglayer.New(loglayer.Config{ Transport: structured.New(structured.Config{}), ErrorSerializer: func(err error) map[string]any { return eris.ToJSON(err, true) }, }) ``` ### Level Control ```go log.SetLevel(loglayer.LogLevelWarn) // enables warn and above log.EnableLevel(loglayer.LogLevelDebug) // toggle one level log.DisableLevel(loglayer.LogLevelDebug) log.IsLevelEnabled(loglayer.LogLevelInfo) log.EnableLogging() / log.DisableLogging() // master switch ``` ### Child Loggers ```go child := log.Child() // shallow copy of fields + level state prefixed := log.WithPrefix("[auth]") // child with prefix override ``` Child mutations do not affect parent. Transports are shared (same instances). ### Transport Management ```go log.AddTransport(t) // append, replaces if same ID log.RemoveTransport("id") // returns bool log.SetTransports(t1, t2) // replace all log.GetLoggerInstance("id") // underlying logger from a transport ``` ### Mocking for Tests Three patterns: 1. `loglayer.NewMock()`: drop-in `*LogLayer` backed by a discard transport. Sets `DisableFatalExit: true` automatically. 2. `transports/testing`: captures every entry into a mutex-safe library with typed `LogLine` fields for assertions. 3. `log.DisableLogging()`: keeps real wiring but suppresses output. ```go log := loglayer.NewMock() // silent lib := &lltest.TestLoggingLibrary{} log = loglayer.New(loglayer.Config{ Transport: lltest.New(lltest.Config{Library: lib}), }) // then assert on lib.Lines() / lib.PopLine() ``` There is no `loglayer.Logger` interface. Go convention is "consumer defines the interface" so application code accepts the concrete `*loglayer.LogLayer` and `NewMock()` returns the same concrete type. ## Fatal Behavior `log.Fatal(...)` writes the entry to all transports, then calls `os.Exit(1)`. This matches the Go convention used by `log.Fatal` in stdlib, zerolog, zap, logrus, etc. To opt out, set `Config.DisableFatalExit: true`. `loglayer.NewMock()` does this automatically. Caveat: the phuslu transport calls `os.Exit` from its underlying library before the core's check ever runs. For tests or library code that needs guaranteed non-exit, use a different transport for fatal-level paths. ## Transports ### Renderers Self-contained transports that format the entry and write to an `io.Writer`. - **console**: plain `fmt.Println`-style output. Stdout for trace/debug/info, stderr for warn/error/fatal. Useful for fixture generation; for human-readable dev output prefer pretty. - **pretty**: colorized terminal output. Five themes (Moonlight default, Sunlight, Neon, Nature, Pastel). Three view modes (inline default, message-only, expanded). Pulls in `github.com/fatih/color`. - **structured**: one JSON object per log entry. Default fields msg/level/time, all configurable. Stdlib only. - **testing**: in-memory capture into `TestLoggingLibrary` with typed `LogLine` exposing Messages/Data/HasData/Metadata/Ctx. Stdlib only. - **blank**: delegates `SendToLogger` to a user-supplied `func(loglayer.TransportParams)`. For prototyping a transport inline, one-off integrations (metrics, queues, HTTP forwarders), or tests that want raw TransportParams. Honors level filtering; nil function silently drops. ### Supported Loggers Transports that hand the entry to an existing logger. - **zerolog**: wraps `github.com/rs/zerolog`. Routes fatal entries through `WithLevel(FatalLevel)` to defer the exit decision to core. - **zap**: wraps `go.uber.org/zap`. Always wraps the supplied logger with `zap.WithFatalHook(noopFatalHook{})` because zap silently overrides `WriteThenNoop` back to `WriteThenFatal`. A custom hook is required. - **slog**: wraps the stdlib `*log/slog.Logger`. Forwards `WithContext` to `LogAttrs` so handlers downstream can extract trace context. Fatal maps to `slog.LevelError + 4`. - **phuslu**: wraps `github.com/phuslu/log`. **Always exits on Fatal** regardless of `DisableFatalExit`. phuslu calls `os.Exit` from every fatal dispatch path. - **logrus**: wraps `github.com/sirupsen/logrus`. Builds a fresh `*logrus.Logger` copying the user's settings with `ExitFunc` neutralized so the user's original logger is never mutated. - **charmlog**: wraps `github.com/charmbracelet/log` (package named `charmlog` to avoid stdlib `log` collision). Uses `Log(level, ...)` which doesn't exit on Fatal. - **otellog**: emits each entry as an OpenTelemetry `log.Record` on a `log.Logger` (`go.opentelemetry.io/otel/log`; package named `otellog` to avoid colliding with the OTel root package). `Config.Name` is required when `Logger` is nil (instrumentation scope name). `Config.LoggerProvider` defaults to `global.GetLoggerProvider`. `Config.Logger` is an escape hatch for tests/advanced wiring. Severity maps `Debug/Info/Warn/Error/Fatal` to OTel `SeverityDebug/Info/Warn/Error/Fatal` (numeric 5/9/13/17/21); `SeverityText` carries the original LogLayer level name. `WithContext` is forwarded to `Logger.Emit` so SDK processors can correlate the record with the active span. Fatal exit is the LogLayer core's decision via `Config.DisableFatalExit`. ### TransportParams Shape Every transport receives: ```go type TransportParams struct { LogLevel LogLevel Messages []any // already prefix-applied Data Data // assembled fields + error map (nil if HasData false) HasData bool Metadata any // raw value passed to WithMetadata Err error Fields Fields // logger's persistent key/value bag Ctx context.Context // per-call Go context attached via WithContext; nil if not set } ``` `Data` is a convenience map (fields + error). `Metadata` is the raw value the user passed; transports decide serialization. `Ctx` carries deadlines, cancellation, and request-scoped values; transports can extract trace IDs etc. ## Custom Transports Implement four methods: ```go type Transport interface { ID() string IsEnabled() bool SendToLogger(params loglayer.TransportParams) GetLoggerInstance() any } ``` Embed `transport.BaseTransport` to get ID, IsEnabled, and ShouldProcess for free. See the transports/creating-transports doc for a minimal example. ## Integrations - **integrations/loghttp**: HTTP middleware that derives a per-request logger from a base logger via `WithFields`, attaches `requestId` (from `X-Request-ID` header or generated), `method`, and `path`, stores the derived logger in the request context via `loglayer.NewContext`, and emits a "request completed" log line at the end of the request with status, bytes, and duration in metadata. Status-based level escalation by default (5xx → Error, 4xx → Warn, else → Info). Wraps any net/http-compatible router. Functional options: `WithRequestIDHeader`, `WithRequestIDGenerator`, `WithFieldNames`, `WithStartLog`, `WithStatusLevels`, `WithExtraFields`. `loghttp.FromRequest(r)` retrieves the per-request logger inside a handler. ```go http.ListenAndServe(":8080", loghttp.Middleware(log)(mux)) func handler(w http.ResponseWriter, r *http.Request) { log := loghttp.FromRequest(r) log.Info("doing work") // includes requestId, method, path } ``` - **integrations/sloghandler**: A `log/slog.Handler` whose `Handle` dispatches into a loglayer logger, so every `slog.Info(...)` call participates in the loglayer plugin pipeline, multi-transport fan-out, group routing, and runtime level state. Opposite direction from `transports/slog` (loglayer dispatches through a `*slog.Logger`). Mapping: slog `WithAttrs` accumulates persistent fields; inline record attrs become per-call fields; `WithGroup(name)` opens a nested map; `slog.Group("", ...)` inlines members at the parent (per slog spec); empty groups and empty-key attrs are dropped; `slog.LogValuer` is resolved before encoding; native kinds preserve their typed Go values (int64, time.Time, time.Duration, ...). Levels above `slog.LevelError` pin to `LogLevelError` so a slog emission cannot trigger loglayer's `os.Exit(1)`. Context passed via `slog.InfoContext(ctx, ...)` is forwarded to dispatch-time plugin hooks via `TransportParams.Ctx` so `oteltrace`/`datadogtrace` extract trace IDs the same way they do for native loglayer calls. Persistent fields set on the underlying logger before the handler is installed are preserved on every emission. Source forwarding: `slog.Record.PC` is converted to a `*loglayer.Source` via `loglayer.SourceFromPC` and passed on `RawLogEntry.Source`, so callers using `slog.New(...)` (which always captures PC) get caller info rendered automatically without setting `Config.Source.Enabled`. ```go log := loglayer.New(loglayer.Config{Transport: structured.New(structured.Config{})}) log.AddPlugin(redact.New(redact.Config{Keys: []string{"password"}})) slog.SetDefault(slog.New(sloghandler.New(log))) slog.Info("user signed in", "userId", 42, "password", "hunter2") // → flows through loglayer; redact replaces password with [REDACTED]. ``` ## Thread Safety Every method on `*LogLayer` is safe to call from any goroutine, including concurrently with emission. There is no setup-only category. - **Emission methods** (`Info`, `WithMetadata`, `Raw`, ...): read-only. - **Returns-new** (`WithFields`, `WithoutFields`, `Child`, `WithPrefix`, `(*LogLayer).WithGroup`): build a new logger; receiver untouched. - **Level mutators** (`SetLevel`, `EnableLevel`, `DisableLevel`, `EnableLogging`, `DisableLogging`): atomic.Uint32 bitmap. Mirrors `zap.AtomicLevel`. Safe for live runtime toggling (SIGUSR1, admin endpoints). - **Transport mutators** (`AddTransport`, `RemoveTransport`, `SetTransports`): atomic.Pointer[transportSet]. Concurrent mutators serialize via internal mutex; dispatch hot path is lock-free. - **Plugin mutators** (`AddPlugin`, `RemovePlugin`): atomic.Pointer[pluginSet], same pattern. - **Group mutators** (`AddGroup`, `RemoveGroup`, `EnableGroup`, `DisableGroup`, `SetGroupLevel`, `SetActiveGroups`, `ClearActiveGroups`): atomic.Pointer[groupSet], same pattern. - **Mute toggles** (`MuteFields`, `UnmuteFields`, `MuteMetadata`, `UnmuteMetadata`): atomic.Bool. Verified by `concurrency_test.go` under `-race`. ## Plugins `Plugin` is a one-method interface (`ID() string`). Plugins implement zero or more hook interfaces (`FieldsHook`, `MetadataHook`, `DataHook`, `MessageHook`, `LevelHook`, `SendGate`) plus an optional `ErrorReporter` for recovered-panic observation. Register via `*LogLayer.AddPlugin`. Hook membership is pre-indexed at registration time (one type assertion per hook per plugin) so the dispatch path only walks plugins that implement the hook. Safe to add/remove from any goroutine. Child loggers inherit plugins. Six hook interfaces: - `FieldsHook { OnFieldsCalled(Fields) Fields }`. Fires from WithFields. Return new fields or nil to drop. - `MetadataHook { OnMetadataCalled(any) any }`. Fires from WithMetadata / MetadataOnly. Return new metadata or nil to drop. - `DataHook { OnBeforeDataOut(BeforeDataOutParams) Data }`. Per emission, after data assembly. Return a map to merge. - `MessageHook { OnBeforeMessageOut(BeforeMessageOutParams) []any }`. Per emission. Return replacement messages or nil to keep. - `LevelHook { TransformLogLevel(TransformLogLevelParams) (LogLevel, bool) }`. Per emission. Last ok=true wins. - `SendGate { ShouldSend(ShouldSendParams) bool }`. Per (entry, transport). Any false vetoes that transport. All four dispatch-time hook param structs (`BeforeDataOutParams`, `BeforeMessageOutParams`, `TransformLogLevelParams`, `ShouldSendParams`) include a `Ctx context.Context` field populated from `WithContext`. `FieldsHook` and `MetadataHook` do not receive a context (they fire from the builder phase, before the level method). Hook panics are recovered centrally by LogLayer. A buggy plugin can't tear down the calling goroutine: each hook returns its no-op value on panic (nil for input/output hooks, level unchanged for `LevelHook`, fail-open for `SendGate`). When the plugin implements `ErrorReporter { OnError(err error) }` the recovered panic is forwarded to it; otherwise a one-line description is written to `os.Stderr`. The recovered value is wrapped in a `*RecoveredPanicError` (hook name + recovered value), with `errors.Unwrap` reaching the original when the panic value implemented `error`. Adapter constructors for inline single-hook plugins: `NewFieldsHook`, `NewMetadataHook`, `NewDataHook`, `NewMessageHook`, `NewLevelHook`, `NewSendGate`, plus `NewPlugin(id)` for a no-op plugin. Multi-hook plugins (like `plugins/redact`) declare a type implementing `Plugin` plus the relevant hook interfaces. ```go log.AddPlugin(loglayer.NewMetadataHook("redact-password", func(metadata any) any { if m, ok := metadata.(map[string]any); ok { if _, has := m["password"]; has { // clone before mutating; never edit caller-owned input out := make(map[string]any, len(m)) for k, v := range m { out[k] = v } out["password"] = "[REDACTED]" return out } } return metadata })) ``` Built-in plugins: - `plugins/redact`. Match by `Keys` (exact key names; honors `json` tags when matching struct fields) or `Patterns` (regex against string values). Walks nested maps, structs, slices, arrays, and pointers at any depth via reflection; preserves the caller's runtime type (struct in → struct out). Caller's input is never mutated. Dependency-free. - `fmtlog` (sub-package of the main module). Single MessageHook plugin: opt the logger into `fmt.Sprintf` semantics for multi-arg messages. After `log.AddPlugin(fmtlog.New())`, calls like `log.Info("user %d", id)` resolve to `"user 1234"` before downstream MessageHooks see them. Without the plugin, multi-arg messages remain space-joined (the default). Composes naturally with the builder chain. - `plugins/datadogtrace`. Reads the active span from each entry's `WithContext` context via a user-supplied `Extract` function and emits `dd.trace_id`, `dd.span_id`, plus optional `dd.service` / `dd.env` / `dd.version` for Datadog's log/trace correlation. Tracer-agnostic: works with `dd-trace-go` v1, v2, or any custom extractor; LogLayer itself takes no Datadog dependency. - `plugins/oteltrace`. Reads the active OTel span from each entry's `WithContext` context via `trace.SpanContextFromContext` (`go.opentelemetry.io/otel/trace`) and emits `trace_id` / `span_id` in OTel's lowercase-hex canonical form, plus optional `trace_flags`, `trace_state` (W3C vendor-specific routing/sampling info), and W3C baggage members (`go.opentelemetry.io/otel/baggage`) keyed ``. Baggage rides independently of the trace span, so a context with baggage but no active span still surfaces baggage attributes. Configurable keys (defaults match OTel JSON serialization; switch to `trace.id`/`span.id` for ECS, etc.). Companion to `transports/otellog`: use the plugin when shipping logs to non-OTel destinations (structured stdout, Datadog HTTP, Loki); use the transport when shipping through the OTel pipeline (the SDK handles correlation automatically). ## Shared Utilities `utils/maputil` exposes two primitives used by transports and plugins, both safe to call from any goroutine: - `ToMap(v any) map[string]any` normalizes any value (struct, pointer, map) to a flat map via JSON roundtrip. Used by `transport.MetadataAsMap` (still available as a thin wrapper). Lossy on type. - `Cloner{MatchKey, MatchValue, Censor}.Clone(v any) any` deep-clones a value with key-name and string-value predicates applied at any depth. Honors `json` tags on struct fields, skips unexported fields, freshly allocates every container, returns a value of the same runtime type as the input. The redact plugin is a thin shell over this. Third-party plugin and transport authors can import these directly from `go.loglayer.dev/utils/maputil`. ## Groups Named routing rules port directly from TS loglayer's `withGroup` feature. Define groups in `Config.Routing.Groups map[string]LogGroup`; each group lists transport IDs (`Transports []string`), an optional minimum `Level`, and a `Disabled` toggle. Tag entries with one or more groups via `(*LogBuilder).WithGroup(...string)` (per-call) or `(*LogLayer).WithGroup(...string)` (returns a child where every log is tagged). Tags accumulate (deduplicated) across chained calls; multi-group entries route to the union of their groups' transports. `Config.Routing.ActiveGroups []string` restricts routing to a subset (nil/empty = no filter). `Config.Routing.Ungrouped` is a typed enum struct controlling untagged entries: `UngroupedToAll` (default; backward compatible), `UngroupedToNone` (drop), `UngroupedToTransports` (allowlist). Disabled groups drop their entries (no fall-back); undefined groups in the tag list fall back to ungrouped (graceful for typos and not-yet-registered groups). Runtime mutators (atomic publish, mutex-serialized): `AddGroup`, `RemoveGroup`, `EnableGroup`, `DisableGroup`, `SetGroupLevel`, `SetActiveGroups`, `ClearActiveGroups`, `GetGroups`. `Raw(RawLogEntry{Groups: ...})` overrides assigned groups for forwarded entries. `loglayer.ActiveGroupsFromEnv("LOGLAYER_GROUPS")` parses a comma-separated env-var list explicitly (we don't read env vars on your behalf). ```go log := loglayer.New(loglayer.Config{ Transports: []loglayer.Transport{...}, Groups: map[string]loglayer.LogGroup{ "database": {Transports: []string{"datadog"}, Level: loglayer.LogLevelError}, "auth": {Transports: []string{"sentry"}, Level: loglayer.LogLevelWarn}, }, }) log.WithGroup("database").Error("connection lost") log.WithGroup("auth", "database").Error("auth db failure") // routes to both groups' transports dbLog := log.WithGroup("database") // child logger; all logs tagged ``` ## What's Out of Scope (v1) These exist in upstream TS loglayer but are not in the Go port: - Field managers (TS calls these "context managers": Linked / Isolated / Default) - Log level managers (LinkedLogLevelManager / etc.) - Lazy / async lazy evaluation - Mixins (the `useLogLayerMixin` augmentation pattern) ## Versioning Currently single-module, single-version. Tag at the repo root (e.g. v1.0.0). All packages move together. Subject to change as the project matures.