Generic instantiation breaks template with untyped parameter overload resolution depending upon import order

[c-blake]

Example

#=> asum.nim <==
template sum*(F, i, itr, expr): untyped =
  var s: F
  for i in itr: s += expr
  s

#=> bsum.nim <==
func sum*[T](x: seq[T]): T = # std/math reduced
  for i in items(x): result = result + i

#=> main.nim <==
when defined(fail): import bsum, asum
else              : import asum, bsum

proc gen1*[G](x: seq[G]): G = sum(float, j, 0..<x.len, G(x[j]))
proc def2*(x: seq[float]): float = sum(x)
proc foo*(cols: seq[float]) = echo gen1(cols)

when isMainModule: (let x = @[1.0,2,3]; foo(x); echo def2(x))

Current Output

With -d:fail

main.nim(4, 42) Error: undeclared identifier: 'j'

Expected Output

Without -d:fail

6.0

Additional Information

$ nim -v
Nim Compiler Version 1.7.1 [Linux: amd64]
Compiled at 2022-01-03
Copyright (c) 2006-2021 by Andreas Rumpf

git hash: 19bcb43a0e21dd222ffd3fad8c5fb2d8e85d59ea
active boot switches: -d:release -d:danger -d:nimUseLinenoise -d:nimHasLibFFI

but I do not think the version matters much. It fails & works the same way on Nim from 0.19.2 to present day devel (0.20.2, 1.4, 1.6, etc). I also had someone else test it on their Nim build. So, the failure seems to be robust/reproducible.

isMainModule is unneeded to fail|work; Just to check/provide some correct output. (Someone could maybe do an exponentially expanding search of past git versions to see if it ever worked and if so git bisect to see what broke things.)

Things that do not alter the fail to compile|work pattern in the above mentioned versions:

• changing for i in itr to a var i = 0; while i <= limit loop
• Swapping order of gen1,def2 definitions
• Making def2 or foo generic
• Making any seq -> openArray (sum,gen1,def2)
• import asum before/after def in bsum
• import bsum before/after def in asum

Things that do alter the fail|work pattern:

• Making gen1 non-generic always works.
• EDIT: Changing to from bsum import nil; from asum import nil with module-qualified name reference always works (i.e. in either b,a or a,b order).
• Changing def&call to sum*[F](i, itr, expr): untyped
  • Same undeclared identifier: 'j' but now always fails.
• Changing sum*(F, i, ..) to sum*(F, i: typed{nkSym}, ..)
  • Same undeclared identifier: 'j' (more context) but now always fails.

This may relate to #13572 & #16128 , but it may also be its own thing. I could not find any other related issues, but maybe someone else can. Note that both the kind of routine for sum and the parameter count vary (and node kind constraints with i: typed{nkSym}). So, if it is caching then something may be causing the "keys" for the cache to not include any signature information.

Users sometimes import enough modules that even finding a working permutation might be intractable for frequent overloads like add. So, this would seem a high priority bug to me.

[Araq]

This is simply the good old "you cannot overload routines that use untyped reliably" issue. It's indeed decades old. We can "improve" the situation but the real solution here is to abandon untyped, IMO.

[c-blake]

Ok. Thanks for the quick reply. Maybe some more related issues cross-linked might help other bug searchers, although now that I know exactly what to look for there seem like dozens of these issues:

• overload resolution broken if untyped param shares position with typed param in some overload #14827 (this has many internal cross-links, but I haven't looked at them all)
• this Exporting a toSeq overload messes up iterator visibility RFCs#512 also seems to relate.
• Unsure how I didn't find this before -very similar but marked resolved: strange interaction between untyped templates and import in overload resolution #4675
• At least RFC Improve overloaded resolution for template with untyped parameter RFCs#402 ; I didn't know how much lipstick this particular pig needed. Lol. The arity/parameter count idea for when it can work (no varargs,etc.) seems sensible to me.

I'll also edit the title to include overload resolution.

[metagn]

This is not nim-lang/RFCs#402, the issue is generics thinking j has to be declared when it can be an untyped parameter, #20086 should be the same

[c-blake]

As mentioned by the EDIT in top post, the above works with either import order (-d:fail or not) if you qualify the sum symbol, i.e.: asum.sum(float, .. and bsum.sum(x). This is true with or without import nil - the latter only forces module qualification. I'm not sure this fact is conclusive as to your point, but it seemed worth reiterating in light of it.

[metagn]

Generics do not semcheck arguments of symbols where the first discovered overload is a template or macro with completely untyped arguments. That's why the import order changes it

[c-blake]

Just since the linked issue suggests this workaround, in this case changing to

template sum*(F: typedesc[SomeNumber], i, itr, expr): untyped =

does not fix the problem / alter behavior.