Inconsistency between format string and dynamic formatting

[brainbytes42]

Thanks for this library - seems very promising, as far as I can say. Just trying it out to see if it fits my needs (release v0.7.3).

I tried to print some extra-Infos which aren't contained in the default format.
Thanks to your reply here (#1164), I could extend the default format (which - as it's internal - shouldn't be done this way for production, I know) as follows:

logger.add(sys.stderr,
    format=lambda rec: loguru._defaults.LOGURU_FORMAT
                       + (" @ {extra}" if rec['extra'] else "")
                       + "\n{exception}",         ###  <-- INCONSISTENCY
)

At first, I hadn't added the last row containing line break and exception, which caused the logging to miss line breaks and exception bodies.

I dug deeper into this, why the default format - which doesn't contain exception and linebreak - works differently, just comparing those two loggers one after the other for their format and behaviour at exceptions:

logger.add(sys.stderr, format="{message}")
logger.add(sys.stderr, format=lambda r:"{message}")

Even if the format string is effectively the same simple string ("{message}"), the lambda-generated string results in a different format than the plain string.

I think, this is the relevant part in the code, where a string format gets extended by terminator and exception (L989), but the dynamic formatter does not (L1002).

loguru/loguru/_logger.py

Lines 987 to 1003 in 5ec8d00

	if isinstance(format, str):
	try:
	formatter = Colorizer.prepare_format(format + terminator + "{exception}")
	except ValueError as e:
	raise ValueError(
	"Invalid format, color markups could not be parsed correctly"
	) from e
	is_formatter_dynamic = False
	elif callable(format):
	if format == builtins.format:
	raise ValueError(
	"The built-in 'format()' function cannot be used as a 'format' parameter, "
	"this is most likely a mistake (please double-check the arguments passed "
	"to 'logger.add()')."
	)
	formatter = format
	is_formatter_dynamic = True

For me, this distinction seems counterintuitive and as far as I could find, is not documented.

format (str or callable, optional) – The template used to format logged messages before being sent to the sink.

I'd suggest to unify the behaviour between string and dynamic formatting, or at least document the differences.
(And ideally, to add the extra-output in the default format.)

[RedZapdos123]

Opened a PR to fix this.

[Delgan]

Hey @brainbytes42.

The inconsistency you've pointed out is certainly true, and I can understand why it might be surprising or even frustrating. It is the result of a trade-off. On one hand, we want the format to be as straightforward as possible for the most common cases. On the other hand, for advanced use cases, the user must be able to fully control the output format, and decide if and how the exception should be rendered.

For these reasons, it was decided that \n{exception} would be implicitly added when format is a str, since it is much more convenient. However, when format is a function, user takes responsibility for adding it (or skipping it entirely). The \n{exception} suffix should not be appended unconditionally, as there are valid use cases where it is undesirable: for example, when formatting and writing log messages to a JSONL file.

This API caveat is mentioned in the documentation, but it maybe isn't emphasized enough.

However, note that in such a case, you should take care of appending the line ending and exception field to the returned format, while "\n{exception}" is automatically appended for convenience if format is a string.

In latest version of docs (not yet on stable branch), a new entry was also added to the FAQ: Why is the captured exception missing from the formatted message?.

The downside is that if generally requires to be bitten once before reaching these pages. Ideally, I agree the API should be more consistent and intuitive, but as I explained, it was a compromise to try covering all use cases. There is another behavior that often surprises users when the dynamic format is a function: it must return a string to be formatted, rather than formatting the message itself. So, maybe it's a hint the API should be changed entirely. I haven't thought about it deeply, though.