Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: lock down future uses of linkname #67401

Closed
rsc opened this issue May 15, 2024 · 171 comments
Closed

cmd/link: lock down future uses of linkname #67401

rsc opened this issue May 15, 2024 · 171 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone

Comments

@rsc
Copy link
Contributor

rsc commented May 15, 2024

Overuse of //go:linkname to reach into Go standard library internals (especially runtime internals) means that when we do change the standard library internals in ways that should not matter, we can end up breaking packages that are depended on by a large swath of the Go ecosystem. For example, https://go.dev/cl/583756 broke github.com/goccy/go-json because it turns out that package copied most of the runtime's internal type API. Now we can't change anything in that list, despite that being an ostensibly internal package, without breaking goccy/go-json. And goccy is used by many packages, including Kubernetes.

This situation is unsustainable. Internals are internal for a reason. We can't keep Go programs working when they create explicit dependencies on details that we have kept internal. But we also care a lot about compatibility: we don't want to break Go programs either. The obvious conclusion is that we have to stop Go programs from being able to create these dependencies on internal details in the first place.

This issue tracks work to prevent new //go:linkname-based dependencies and contain existing ones.

Right now, if package A has a symbol and package B wants to refer to it with //go:linkname, there are three patterns:

  • (Push) Package A uses a //go:linkname to rename one of its own symbols to B.foo, and then B declares func foo() without a body. In this form, A clearly intends for B to use foo, although the compiler cannot quite tell what's going on in B and warns about foo not having a body unless you create an empty dummy.s file.

  • (Pull) Package A defines foo without any annotation, and package B uses //go:linkname to access A.foo. In this form, A may not intend for B to use foo at all. That's a serious problem: when A renames foo and/or changes its type signature, B breaks, and A may never even have heard of B.

  • (Handshake) Package A defines foo with a //go:linkname and package B defines foo also with a //go:linkname, and the two agree on the name (either A.foo or B.foo). This is the ideal form, and it avoids the dummy.s workaround that is needed in the Push case.

The ideal goal state is a world where all //go:linkname usage must be in the Handshake form: both sides must agree to use linkname for a given symbol in order for it to succeed. This will mean that arbitrary packages cannot create new dependencies on runtime internals. At the same time, we realize that the current world is not this ideal world, and we don't want to break all existing uses.

Our plan is as follows.

  1. Introduce a new -checklinkname=1 flag to cmd/link that requires the Handshake form for symbols in the standard library. That flag is already landed in at tip, but it is not the default.

  2. Survey all existing open-source Go packages to find standard library symbols that are being //go:linkname'd (behind our backs!) using the Pull pattern. Add the necessary //go:linkname annotations to the standard library to keep those working, documenting why each exists. The explicit //go:linkname lines and documentation will help avoid accidental breakage in future refactoring. We have done a preliminary survey, but we haven't yet added all the necessary //go:linkname lines.

  3. Make -checklinkname=1 the default for Go 1.23. If this breaks anything, users can use -ldflags=-checklinkname=0 to get unbroken, and we hope they will also file reports letting us know what we missed.

  4. As we get reports of additional breakage we missed, add more //go:linkname annotations to the standard library.

At the completion of that plan, we won't be in the ideal world, but we will have accomplished two important things:

  • We won't have broken anything.

  • We will have stopped new damage from accumulating: there will be no more new references to runtime internals introduced. In particular, new internals we added during the Go 1.23 cycle, like coro and weak pointers, cannot be linknamed, now or ever. And anything that wasn't linknamed yet won't grow new linknames in the future.

Note that anyone who wants to experiment can always build with -ldflags=-checklinkname=0 and linkname whatever they like. That's fine. We like experimenting too. But the fact that the code won't build without special flags should help prevent code that digs into internal details from becoming a core dependency in the Go ecosystem that we end up having to maintain forever.

Note also that for now, //go:linkname can still be used in Pull mode to get at internals of non-standard library packages. We'd like to change that eventually too, insisting on Handshakes everywhere. For now, we are starting with the standard library. If all goes well, we'll circle back and try to devise a plan for the rest of the ecosystem.

@rsc rsc added this to the Go1.23 milestone May 15, 2024
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label May 15, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/585820 mentions this issue: runtime/coverage: remove uses of //go:linkname

@Jorropo
Copy link
Member

Jorropo commented May 15, 2024

I think github.com/goccy/go-json pulled internal runtime APIs in a reckless manner and there are safe ways to pull internal packages from the std.
For example quic-go did an equally dangerous thing with crypto/tls by accessing private fields using unsafe.Pointer and maintaining forks of crypto/tls:

However key differences:

As a downstream of quic-go I clearly understood the situation, the worst was the need to wait a couple of days for quic-go to release a new release compatible with the new go release before updating my go toolchain and the inability to run on tip or RCs.


In the case of github.com/goccy/go-json the situation could have been even better than quic-go's as it claims to be compatible with encoder/json:

Fast JSON encoder/decoder compatible with encoding/json for Go

instead of creating a compile time error they could have stubbed their API by forwarding calls to encoding/json when the release of go were to be unknown. (I don't know the details, maybe due to some edge cases or behaviors only go-json implements this wouldn't have been possible)


What I actually propose:

Continue to allow the Pull kind of linkname from std packages when the file is locked with build tags:

//go:build go1.23.4 && !go1.23.5 && !go1.24

To know if a file is properly locked down, the toolchain can evaluate the build tags with the next future release and it MUST fail to be allowed. That means if a file pass the current version but not the next one, then Pull linknames from the std would be allowed.

There is a downside to this approach which is that if there is no change required between two releases you still need a different file per version with the same implementation, to satisfy this constraint. Maybe parsing the build tags would be better.

I am not sure if goX.Y or goX.Y.Z should be used. goX.Y might be more dangerous in case a fix require breaking some internal API, quic-go used that and it was fine and created this couple of days period where you can't use latest go only every 6 months (note that goX.Y.Z build tags didn't existed back then).

@rsc
Copy link
Contributor Author

rsc commented May 15, 2024

I think ... there are safe ways to pull internal packages from the std.

I completely disagree. Quic-go's use of linkname caused all manner of problems for us release after release too, because anyone using quic-go couldn't update to a new Go version until quic-go did.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/585916 mentions this issue: internal/coverage/cfile: remove //go:linkname into testing

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/585915 mentions this issue: internal/coverage/cfile: remove more //go:linkname usage

@ericlagergren

This comment has been minimized.

@randall77
Copy link
Contributor

We can't fix what we don't know about.

  1. As we get reports of additional breakage we missed, add more //go:linkname annotations to the standard library.

We're open to any reports from closed-source packages. It would be particularly useful to hear about these when the release candidate comes out so they can make the .0 release.

@ericlagergren

This comment has been minimized.

@randall77

This comment was marked as outdated.

@ericlagergren

This comment was marked as outdated.

@ruyi789
Copy link

ruyi789 commented May 16, 2024

Misuse is undesirable, disabling is even worse, you want to change you maintain that package (go-json), that doesn't solve it?

@bjorndm
Copy link

bjorndm commented May 16, 2024

I would like to say that //go:linkname is very useful for certain types of low level programming such as Ebitengine , etc. While I agree the situation with goccy/go-json is bad, we should look at how it is being used in detail and think of alternatives for the legitimate uses.

@DmitriyMV
Copy link
Contributor

@bjorndm should't -ldflags=-checklinkname=0 which disables this check be enough for low level programming? And if you can always raise a proposal if something is missing and truly needed without ld flags.

As we get reports of additional breakage we missed, add more //go:linkname annotations to the standard library.

@cherrymui
Copy link
Member

In C, static symbols are not accessible outside of the compilation unit, full stop. There is no way to pull a static symbol from a C library. A number of other languages have similar strict visibility rules. They are very successful languages and are widely used. This suggests that a lot programs can be written and things can go very well without a mechanism to break into a library's internal details.

I don't think Go is fundamentally different. Ideally we could also have strict visibility rules. I would think Go unexported symbols are meant to be similar to C static symbols. In fact, that is what gccgo does. Unfortunately for the gc toolchain it is not the case today. But we can get closer to it. And as we care a lot about compatibility, we'll keep the existing code continue to build in Go 1.23 (Step 2 in the plan). And we have a linker flag to disable the restriction (e.g. for experiments; as far as I know, the C linker doesn't seem to have such an option).

Also, I think the authors of the code should have a way to decide which symbols are visible externally and which are not.

@TotallyGamerJet
Copy link

TotallyGamerJet commented May 16, 2024

Purego which is a dependency of Oto, Beep, and Ebitengine as well as others doesn't just pull symbols it also pushes since it reimplements runtime/cgo package when CGO_ENABLED=0 entirely in Go. Is there any way the symbols defined in the runtime that hook into that package also get comments to avoid breaking us?

Another potential solution for us is to prebuild runtime/cgo into a cgo_GOOS_GOARCH_GOVERSION.syso that ships with purego. That would save us from having to keep up with any changes that package has and allows the Go team the freedom to change it as they like. Is this actually possible? I tried with setting different -buildmode but none of them would link.

Of course, if the Go team wanted to port runtime/cgo to Go that would be optimal.

@cherrymui
Copy link
Member

Is there any way the symbols defined in the runtime that hook into that package also get comments to avoid breaking us?

Push linknames are still allowed. If they are currently pushed from runtime/cgo, I believe you can still push them from Purego. Does Purego push symbols more than runtime/cgo?

Of course, if the Go team wanted to port runtime/cgo to Go that would be optimal.

This might be a possible option, but I think we need to understand the rationales better. The runtime/cgo package is intended to work with cgo, that is, interacting with C code. I'm not sure I understand the use case of runtime/cgo with CGO_ENABLED=0. This is probably better to be a separate discussion. Thanks.

@TotallyGamerJet
Copy link

TotallyGamerJet commented May 16, 2024

Push linknames are still allowed. If they are currently pushed from runtime/cgo, I believe you can still push them from Purego. Does Purego push symbols more than runtime/cgo?

No, Purego pushes the same symbols that runtime/cgo does which means the "Handshake" is already satisfied for those. Step 2 of the suggested plan only mentions surveying for Pull linknames and marking them with comments to avoid future breakage. I'm wondering if there is any plans for Pushes as changes to those would break Purego?

This might be a possible option, but I think we need to understand the rationales better. The runtime/cgo package is intended to work with cgo, that is, interacting with C code. I'm not sure I understand the use case of runtime/cgo with CGO_ENABLED=0. This is probably better to be a separate discussion. Thanks.

Indeed, runtime/cgo is required to allow Go code and C code to play well with each other. Purego provides an entirely Go version of it so that you can call C code using purego.Dlopen and purego.Dlsym without the need of a C compiler so cross-compiling is again possible. We can discuss this further elsewhere.

@cherrymui
Copy link
Member

cherrymui commented May 16, 2024

Thanks.

I'm wondering if there is any plans for Pushes as changes to those would break Purego?

I don't think there is any plan to break the use case of Purego's pushes. If we do anything to restrict push-only ones, they will be equally applied to the ones runtime/cgo pushing to runtime. So we'll need to fix those first (I think many of them are already in handshake form, but it is possible we missed some). And that should make Purego work as well.

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 16, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/586259 mentions this issue: runtime: move exit hooks into internal/runtime/exithook

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/586137 mentions this issue: all: add push linknames to allow legacy pull linknames

fuweid added a commit to fuweid/containerd that referenced this issue Aug 24, 2024
The Go runtime has started to [lock down future uses of linkname][1] since
go1.23. In the go source code, containerd project has been marked in the
comment, [hall of shame][2]. Well, the go:linkname is used to fork no-op
subprocess efficiently. However, since that comment, I would like to use
ptrace and remove go:linkname in the whole repository.

With go1.22 `go:linkname`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 2440            533320 ns/op            1145 B/op         43 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 342           3661616 ns/op           11562 B/op        421 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  2.983s
```

With go1.22 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1785            739557 ns/op            3948 B/op         68 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 328           4024300 ns/op           39601 B/op        671 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.104s
```

With go1.23 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1815            723252 ns/op            4220 B/op         69 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 319           3957157 ns/op           42351 B/op        682 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.051s
```

Diff:

The `ptrace` is slower than `go:linkname` mode. However, it's accepctable.

```
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
                                    │ go122-golinkname │             go122-ptrace              │             go123-ptrace              │
                                    │      sec/op      │    sec/op     vs base                 │    sec/op     vs base                 │
BatchRunGetUsernsFD_Concurrent1-16        533.3µ ± ∞ ¹   739.6µ ± ∞ ¹        ~ (p=1.000 n=1) ²   723.3µ ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16       3.662m ± ∞ ¹   4.024m ± ∞ ¹        ~ (p=1.000 n=1) ²   3.957m ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                   1.397m         1.725m        +23.45%                   1.692m        +21.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │              go122-ptrace               │              go123-ptrace               │
                                    │       B/op       │     B/op       vs base                  │     B/op       vs base                  │
BatchRunGetUsernsFD_Concurrent1-16       1.118Ki ± ∞ ¹   3.855Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   4.121Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16      11.29Ki ± ∞ ¹   38.67Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   41.36Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
geomean                                  3.553Ki         12.21Ki        +243.65%                   13.06Ki        +267.43%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │             go122-ptrace             │             go123-ptrace             │
                                    │    allocs/op     │  allocs/op   vs base                 │  allocs/op   vs base                 │
BatchRunGetUsernsFD_Concurrent1-16         43.00 ± ∞ ¹   68.00 ± ∞ ¹        ~ (p=1.000 n=1) ²   69.00 ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16        421.0 ± ∞ ¹   671.0 ± ∞ ¹        ~ (p=1.000 n=1) ²   682.0 ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                    134.5         213.6        +58.76%                   216.9        +61.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
```

[1]: <golang/go#67401>
[2]: <https://github.com/golang/go/blob/release-branch.go1.23/src/runtime/proc.go#L4820>

Signed-off-by: Wei Fu <fuweid89@gmail.com>
fuweid added a commit to fuweid/containerd that referenced this issue Aug 26, 2024
The Go runtime has started to [lock down future uses of linkname][1] since
go1.23. In the go source code, containerd project has been marked in the
comment, [hall of shame][2]. Well, the go:linkname is used to fork no-op
subprocess efficiently. However, since that comment, I would like to use
ptrace and remove go:linkname in the whole repository.

With go1.22 `go:linkname`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 2440            533320 ns/op            1145 B/op         43 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 342           3661616 ns/op           11562 B/op        421 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  2.983s
```

With go1.22 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1785            739557 ns/op            3948 B/op         68 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 328           4024300 ns/op           39601 B/op        671 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.104s
```

With go1.23 `ptrace`:

```bash
$ go test -bench=.  -benchmem ./ -exec sudo
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
BenchmarkBatchRunGetUsernsFD_Concurrent1-16                 1815            723252 ns/op            4220 B/op         69 allocs/op
BenchmarkBatchRunGetUsernsFD_Concurrent10-16                 319           3957157 ns/op           42351 B/op        682 allocs/op
PASS
ok      github.com/containerd/containerd/v2/core/mount  3.051s
```

Diff:

The `ptrace` is slower than `go:linkname` mode. However, it's accepctable.

```
goos: linux
goarch: amd64
pkg: github.com/containerd/containerd/v2/core/mount
cpu: AMD Ryzen 7 5800H with Radeon Graphics
                                    │ go122-golinkname │             go122-ptrace              │             go123-ptrace              │
                                    │      sec/op      │    sec/op     vs base                 │    sec/op     vs base                 │
BatchRunGetUsernsFD_Concurrent1-16        533.3µ ± ∞ ¹   739.6µ ± ∞ ¹        ~ (p=1.000 n=1) ²   723.3µ ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16       3.662m ± ∞ ¹   4.024m ± ∞ ¹        ~ (p=1.000 n=1) ²   3.957m ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                   1.397m         1.725m        +23.45%                   1.692m        +21.06%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │              go122-ptrace               │              go123-ptrace               │
                                    │       B/op       │     B/op       vs base                  │     B/op       vs base                  │
BatchRunGetUsernsFD_Concurrent1-16       1.118Ki ± ∞ ¹   3.855Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   4.121Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16      11.29Ki ± ∞ ¹   38.67Ki ± ∞ ¹         ~ (p=1.000 n=1) ²   41.36Ki ± ∞ ¹         ~ (p=1.000 n=1) ²
geomean                                  3.553Ki         12.21Ki        +243.65%                   13.06Ki        +267.43%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                                    │ go122-golinkname │             go122-ptrace             │             go123-ptrace             │
                                    │    allocs/op     │  allocs/op   vs base                 │  allocs/op   vs base                 │
BatchRunGetUsernsFD_Concurrent1-16         43.00 ± ∞ ¹   68.00 ± ∞ ¹        ~ (p=1.000 n=1) ²   69.00 ± ∞ ¹        ~ (p=1.000 n=1) ²
BatchRunGetUsernsFD_Concurrent10-16        421.0 ± ∞ ¹   671.0 ± ∞ ¹        ~ (p=1.000 n=1) ²   682.0 ± ∞ ¹        ~ (p=1.000 n=1) ²
geomean                                    134.5         213.6        +58.76%                   216.9        +61.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05
```

[1]: <golang/go#67401>
[2]: <https://github.com/golang/go/blob/release-branch.go1.23/src/runtime/proc.go#L4820>

Signed-off-by: Wei Fu <fuweid89@gmail.com>
@szmcdull

This comment was marked as off-topic.

@szmcdull

This comment was marked as off-topic.

@ghost
Copy link

ghost commented Sep 8, 2024

In fact, if linknames require handshaking, then why don't we export these symbols? This seems to defeat the purpose of linknames themselves.

In addition, when we use ldflags=-checklinkname=0 add a new tag like //go:build linkname, just like //go:build go1.23.
Or define ldflags=-checklinkname=0 in go.mod?

eiffel-fl added a commit to inspektor-gadget/inspektor-gadget that referenced this issue Sep 11, 2024
Before this commit, we used the private variable safeSet available in
encoding/json through linkname.
But, since golang 1.23, linkname is not authorized by default [1].

So, this commit introduces the safeset package which defines the SafeSet map.
This map has the exact same value as encoding/json.safeSet [2].
It is also safe to add it in our sources, as this variable was not modified
upstream since its introduction in 2016 [3].

This permits the whole project to be compiled using golang 1.23.

Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
[1]: golang/go#67401
[2]: https://cs.opensource.google/go/go/+/release-branch.go1.23:src/encoding/json/tables.go;l=15-112
[3]: https://cs.opensource.google/go/go/+/ed8f207940c8787d344664a43071b235e2ce5c68
eiffel-fl added a commit to inspektor-gadget/inspektor-gadget that referenced this issue Sep 11, 2024
Before this commit, we used the private variable safeSet available in
encoding/json through linkname.
But, since golang 1.23, linkname is not authorized by default [1].

So, this commit introduces the safeset package which defines the SafeSet map.
This map has the exact same value as encoding/json.safeSet [2].
It is also safe to add it in our sources, as this variable was not modified
upstream since its introduction in 2016 [3].

This permits the whole project to be compiled using golang 1.23.

Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
[1]: golang/go#67401
[2]: https://cs.opensource.google/go/go/+/release-branch.go1.23:src/encoding/json/tables.go;l=15-112
[3]: https://cs.opensource.google/go/go/+/ed8f207940c8787d344664a43071b235e2ce5c68
eiffel-fl added a commit to inspektor-gadget/inspektor-gadget that referenced this issue Sep 11, 2024
Before this commit, we used the private variable safeSet available in
encoding/json through linkname.
But, since golang 1.23, linkname is not authorized by default [1].

So, this commit introduces the safeset package which defines the SafeSet map.
This map has the exact same value as encoding/json.safeSet [2].
It is also safe to add it in our sources, as this variable was not modified
upstream since its introduction in 2016 [3].

This permits the whole project to be compiled using golang 1.23.

Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
[1]: golang/go#67401
[2]: https://cs.opensource.google/go/go/+/release-branch.go1.23:src/encoding/json/tables.go;l=15-112
[3]: https://cs.opensource.google/go/go/+/ed8f207940c8787d344664a43071b235e2ce5c68
eiffel-fl added a commit to inspektor-gadget/inspektor-gadget that referenced this issue Sep 11, 2024
Before this commit, we used the private variable safeSet available in
encoding/json through linkname.
But, since golang 1.23, linkname is not authorized by default [1].

So, this commit introduces the safeset package which defines the SafeSet map.
This map has the exact same value as encoding/json.safeSet [2].
It is also safe to add it in our sources, as this variable was not modified
upstream since its introduction in 2016 [3].

This permits the whole project to be compiled using golang 1.23.

Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
[1]: golang/go#67401
[2]: https://cs.opensource.google/go/go/+/release-branch.go1.23:src/encoding/json/tables.go;l=15-112
[3]: https://cs.opensource.google/go/go/+/ed8f207940c8787d344664a43071b235e2ce5c68
@ghost
Copy link

ghost commented Sep 22, 2024

Why not learn Android and limit reflection like that. Or don't include the go: linkname feature from the start.

gopherbot pushed a commit that referenced this issue Oct 21, 2024
Remove linkname directives that are no longer necessary given
parquet-go/parquet-go#142 removes the dependency on the `memhash{32,64}`
functions.

This change also removes references to segmentio/parquet-go since that
repository was archived in favor of parquet-go/parquet-go.

Updates #67401

Change-Id: Ibafb0c41b39cdb86dac5531f62787fb5cb8d3f01
GitHub-Last-Rev: e14c4e4
GitHub-Pull-Request: #67784
Reviewed-on: https://go-review.googlesource.com/c/go/+/589795
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Sefi4 pushed a commit to Sefi4/inspektor-gadget that referenced this issue Nov 4, 2024
Before this commit, we used the private variable safeSet available in
encoding/json through linkname.
But, since golang 1.23, linkname is not authorized by default [1].

So, this commit introduces the safeset package which defines the SafeSet map.
This map has the exact same value as encoding/json.safeSet [2].
It is also safe to add it in our sources, as this variable was not modified
upstream since its introduction in 2016 [3].

This permits the whole project to be compiled using golang 1.23.

Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
[1]: golang/go#67401
[2]: https://cs.opensource.google/go/go/+/release-branch.go1.23:src/encoding/json/tables.go;l=15-112
[3]: https://cs.opensource.google/go/go/+/ed8f207940c8787d344664a43071b235e2ce5c68
@rbqvq
Copy link

rbqvq commented Dec 2, 2024

@rsc

The best resolution is if the package want to use linkname to expose std function. They must define support go version in go.mod or use build tag protect the go version.

Then the linker allow to use linkname.
Of course, thats should require compiler check.

If the package not defined support go version in go.mod
We should set toolchain to lastest Go v1.22 version.

If user compile with newer or unsupport go version, we should set toolchain to lastest toolchain which version the package support.

Finally, import and compile with any package with linkname will be warned.

In fact, Go community should not to care those packages are used linkname, thats the package owner does. If compiling are broken. The user should ask the package owner instead of Go community.
ALL problems are go not warning when imports and compile which package is using linkname.

As the community says, we are hackers and don't need child-safe toys.

If these limits also keep here. That means Go was dead.
We can not build some hacker code after Go v1.23

So please make changes.

@rbqvq
Copy link

rbqvq commented Dec 2, 2024

One more things.

Thats u not provided build tags when -checklinkname=0

That means the package owner must make a decision: child-safe toys or hacker.

We can not do it both!

Don't let developers reinvent the wheel.
Don't let redundant code become more and more bulky.

@ianlancetaylor
Copy link
Member

@rbqvq The reasons that we did not follow the path you describe were covered extensively on the discussion on this issue.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/634476 mentions this issue: runtime: remove datadog-agent from prof labels hall of shame

gopherbot pushed a commit that referenced this issue Dec 9, 2024
github.com/DataDog/datadog-agent has stopped using runtime_setProfLabel
and runtime_getProfLabel, remove them from the hall of shame.

Updates #67401

Change-Id: I4a66c5e70397d43d7f064aeae5bad064e168316f
Reviewed-on: https://go-review.googlesource.com/c/go/+/634476
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
Development

No branches or pull requests