Enhanced Interpolations to Replace @printf

[ChrisRackauckas]

One way to handle the replacing @printf may be to enhance string interpolation. To do this, one could introduce a to_str function (I am not set on the name, but am using it for discussion. This is a proposal for the architecture). For example, currently ”piece1$var piece2” parses similarly to string(“piece1", var, “ piece2”). In reality, "$var" is the same as "$(var)". My idea is to extend the interpolating $() to a full function call, to_str, and allow arguments.

For example, we could instead use ”piece1$(var,args...) piece2” which would extend to string(“piece1", to_str(var,args...), “ piece2”). to_str would be a function which returns a String (or if not, to_str is reursively called on the output). The standard to_str would be

to_str(x::Any) = string(x)

which would just give the string output of the print method. But this let's us generalize a lot. For example, we could do

to_str(x::AbstractFloat,style::Symbol;dec=2) = ...

to be a function which gives back a string which which is in the chosen style. This would allow one to make $(var,:sci,dec=3) is the result" interpolate into the string var in scientific notation with 3 decimal places.

I am not proposing the exact details for the Base to_str functions, but I think this architecture would work really well for Base since it would cover most use cases while being very extendable (anyone could add dispatches for their own types, so they would work seamlessly for interpolation). This is very different from the idea of #10610, but I think this is a conservative extension to current interpolation behavior that matches well with Julia intuition and could cover most of what people need with @printf (with anything super special handled in packages).

[andyferris]

Does "$x" call string(x)?

Could we just allow keyword arguments:

"$(x; kwargs...)" == string(x; kwargs...)

And have more complex strings nest calls to "string" (one call for each $ and a final one for concatenation) or whatever.

[ChrisRackauckas]

Someone who knows more about the current implementation would have to comment. I assume that it's something similar to that, which means this wouldn't be really hard to implement. A small description of the current setup could probably make make this an easy PR.

[musm]

Does "$x" call string(x)?

I don't think it does?

julia> x = 1
1

julia> @code_lowered("$x")
CodeInfo(:(begin
        nothing
        return (Base.dec)(x)
    end))

julia> x = 1.0
1.0

julia> @code_lowered("$x")
CodeInfo(:(begin
        nothing
        return (Core._apply)(Base.print_to_string, xs)
    end))

but someone more familiar with internals should comment

[vtjnash]

It does:

julia> x = 1
1

julia> @which "$x"
string(x::Union{Int128, Int16, Int32, Int64, Int8}) in Base at intfuncs.jl:484

julia> x = 1.0
1.0

julia> @which "$x"
string(xs...) in Base at strings/io.jl:120

julia> expand(Main, :( "$x" ))
:((Base.string)(x))

[PallHaraldsson]

@ararslan Do you remember why the thumbs down? Is this just not needed in Base or some other better PR suggesting? Or as in discourse, pointed out as alternative:

https://github.com/JuliaString/StringLiterals.jl

[Should people in general explain thumps downs?]

[StefanKarpinski]

[Should people in general explain thumps downs?]

I would generally appreciate this. An unexplained thumbs down generally comes off as kind of passive aggressive and unconstructive. Unless it's clear why you have a problem with something, it's not really helping the conversation.

[ararslan]

Do you remember why the thumbs down?

No, but rereading this I find it odd and don't see the benefit over @printf.

[quinnj]

Is this still relevant/desired? I haven't seen too many requests to overload how an object is interpolated into strings, but maybe I've missed if people still need something like this. It seems that just overloading string for your custom type let's you hook into a wide variety of printing situations.

[simonbyrne]

The main use case would be being able to write something like

"$(t, digits=2) $units"

instead of

@sprintf("%.2 %s", t, units)

A long time ago I started a Julep along these lines (JuliaLang/Juleps#49). I'd still be in favor of this, but the main technical challenge is that it requires mucking about in the parser.

[simonbyrne]

If I try this now, I get

julia> "$(1.0, digits=2)"
ERROR: syntax: invalid interpolation syntax
Stacktrace:
 [1] top-level scope
   @ none:1

@c42f as the last person to touch the relevant piece of code in the parser

julia/src/julia-parser.scm

Lines 2368 to 2385 in c4d162e

	(define (parse-interpolate s)
	(let* ((p (ts:port s))
	(c (peek-char p)))
	(cond ((identifier-start-char? c)
	(let* ((t (require-token s))
	(c (peek-char p)))
	(if (ends-interpolated-atom? c)
	(take-token s)
	(error (string "interpolated variable $" t " ends with invalid character \"" c "\"; use \"$(" t ")\" instead.")))))
	((eqv? c #\()
	(read-char p)
	(let ((ex (parse-eq* s))
	(t (require-token s)))
	(cond ((eqv? t #\) )
	(take-token s)
	ex)
	(else (error "invalid interpolation syntax")))))
	(else (error (string "invalid interpolation syntax: \"$" c "\""))))))

do you have any idea what would be involved in supporting this?

[c42f]

This isn't actually a parser error! The error arises from lowering:

julia> dump(Meta.parse("\$(1.0, digits=2)"))
Expr
  head: Symbol $
  args: Array{Any}((1,))
    1: Expr
      head: Symbol tuple
      args: Array{Any}((2,))
        1: Float64 1.0
        2: Expr
          head: Symbol =
          args: Array{Any}((2,))
            1: Symbol digits
            2: Int64 2

Currently syntax like "aa$(x)bb" lowers to Base.string("aa", x, "bb"), so what's involved here is figuring out a good lowering to hold the keyword arguments.

I guess I'd suggest something like "aa$(1.0, digits=2)bb" becoming

Base.string("aa", Base.stringformat(1.0, digits=2), "bb")

where Base.stringformat() might return a lazy wrapper which can be picked up by Base.string and printed along with the other interpolations?

[c42f]

It's probably relatively easy to add this to the first syntax desugaring pass in lowering, somewhere around here:

julia/src/julia-syntax.scm

Line 2696 in c4d162e

'string

[simonbyrne]

Isn't that just parsing $(1.0, digits=2)? When you try to parse a string it throws an error:

julia> Meta.parse("\"\$(1.0, digits=2)\"")
ERROR: Base.Meta.ParseError("invalid interpolation syntax")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:237 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:268
 [3] parse(str::String)
   @ Base.Meta ./meta.jl:267
 [4] top-level scope
   @ REPL[12]:1

[c42f]

Haha oops! You're right of course.

[c42f]

Huh, so ah... turns out I didn't copy this restriction into JuliaSyntax.jl ... so the parser fix could be just merging #46372

julia> dump(JuliaSyntax.parsestmt(Expr, "\"\$(x, digits=1)\""))
Expr
  head: Symbol string
  args: Array{Any}((1,))
    1: Expr
      head: Symbol tuple
      args: Array{Any}((2,))
        1: Symbol x
        2: Expr
          head: Symbol =
          args: Array{Any}((2,))
            1: Symbol digits
            2: Int64 1

(But also I should really add this restriction back into the parser. I'm not sure how it got left out.)

[c42f]

Ok I've fixed this in JuliaSyntax now and also had a look in the reference parser as part of doing that. To answer the question, it should be fairly easy to change this in the parser - it's roughly a matter of changing the (parse-eq* s) part to call one of the other parsing functions. Or maybe to do something special with how it consumes commas. (In JuliaSyntax we'd probably call parse_brackets() for this and reject a nonzero number of semicolons but the bracket parsing machinery over there is a little different.)

Having changed this we'd need a new expression head as a way to distinguish between "$(1.0, digits=2)" vs "$((1.0, digits=2))" in the returned expression. It seems we can't reuse Expr(:tuple) as the representation of the (1.0, digits=2) and likewise can't use Expr($) because that's ambiguous with "$($(1.0, digits=2))".

I'd suggest something like "aa$(1.0, digits=2)bb" parsing to

Expr(:string, "aa", Expr(:stringformat, 1.0, Expr(:(=), :digits, 2)), "bb")

I guess. (maybe there's a better name than :stringformat?)

[simonbyrne]

That seems like the obvious lowering. Given how string is already defined, I don't see much other choice.

I was actually going to suggest just :format: the idea being it could return a wrapper that formats an object when printed or converted to a string.

[c42f]

Expr(:format) works for me :-)

Here's a prototype implementation in JuliaSyntax JuliaLang/JuliaSyntax.jl#308