proposal: Go 2: Add struct and interface properties in the style of C#

[jsshapiro]

• Would you consider yourself a novice, intermediate, or experienced Go programmer?

  Intermediate

• What other languages do you have experience with?

  C, C++, Python, C#, Javascript, Typescript, Basic, FORTRAN, COBOL, too many assembly languages to count, Yacc, Python, BitC, lots and lots more.

• Would this change make Go easier or harder to learn, and why?

  Neither harder nor easier

• Has this idea, or one like it, been proposed before?

  Yes

• If so, how does this proposal differ?

  I suspect this is a different syntax, but the main purpose here is to give a clear motivating use case for discuttion.

• Who does this proposal help, and why?

  Anyone wrestling with certain backwards compatible source API revisions

• What is the proposed change?

  Add C#-style properties to Go.

• Please describe as precisely as possible the change to the language.

  C# has a property syntax that can be borrowed nearly verbatim. The interesting part is that it re-frames fields as getters and setters without requiring parenthesis, which makes it possible to solve certain API compatibility issues.

• What would change in the language spec?

  New kind of element for struct types

  New kind of declaration for interface types

• Please also describe the change informally, as in a class teaching Go.

  See below

• Is this change backward compatible?

  Source-level: yes

  Binary-level: non-breaking for new uses, potentially breaking when used to implement legacy API compatibility (depending on how the low-level interface is implemented).

• Show example code before and after the change.

  Refer to many uses in C#. If there's actually any interest here, I'll be happy to expand.

• What is the cost of this proposal? (Every language change has a cost).

  1. Cost of implementation: This is best implemented as a high-level AST rewrite, so it doesn't carry far-reaching implications in the compiler.
  2. Cost for existing (property oblivious) code: none
  3. Cost for code using properties: depends entirely on what the property code blocks are doing. For the most simple properties, cost should be zero assuming basic inlining. For more complex property implementations, these are function calls in complex cases. The cost of a function call can't be quantified in abstract.
  4. Debugging type information updates, if properties are to be identified as such for debugging purposes.

• How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?

  Not yet analyzed adequately, but certainly gofmt, gopls, and the compiler itself. I suspect in all cases the modifications are light.

• What is the compile time cost?

  Not measured. This is essentially a substitution of a getter/setter call for a field use/update. The rewrite itself should be quite fast and can probably be fused with an existing pass. The subsequent cost incurred in the optimizer can only be quantified with real use cases.

  I'm concerned that some changes may need to be made in the handling of multiple value return, though I can see a rewrite strategy for that as well.

• What is the run time cost?

  zero-argument (get) and single-argument (set) function call, usually inlined.

• Can you describe a possible implementation?

  My personal inclination would be to handle this in a high-level AST->AST transformation that does the necessary rewrites.
  It seems likely this could be fused with an existing pass.

• Do you have a prototype? (This is not required.)

  No, but I'd be happy to build a change set if there is interest.

• How would the language spec change?

  Specifications of struct and interface types, possibly some fine print on multi-value assignment at procedure return.

• Orthogonality: how does this change interact or overlap with existing features?

  It does not.

• Is the goal of this change a performance improvement?

  No. This addresses a real-world engineering concern.

• Does this affect error handling?

  Yes. When getter/setter is not inlined, this changes what appears in the debugger stack trace. Otherwise, I can't think of an error handling change from this.

• Is this about generics?

  No

Summary: the absence of properties impedes source-level backwards compatibility, and somewhat restricts what can be expressed in interfaces without added syntactic cruft.

Compatibility concerns: Go source level: none. Cross-language and binary level: potentially.

Various forms of getter/setter patterns have been proposed and rejected before, and I suspect this one will be no different. The intended contribution here is to clearly describe a real-world use case that illustrates a source-level API compatibility problem in Go. I do not see a way to address this compatibility limitation without something like properties. In the interest of concreteness, I'll describe the specific issue and context where I tripped on this, but the problem I describe is a general problem for source-level API backwards compatibility.

Problem Statement For various reasons, I've been poking at a successor to pigeon. It is a transitional goal that existing pigeon grammars should migrate with minimal change. In particular, existing user-supplied code blocks should not require modification in order to be processed by the new tool. Pigeon code blocks are passed the pigeon parse state object (the *current type), which directly exposes structure fields. Because they are not guarded by getters and setters (of any sort), these fields have become part of a de facto API interface. Some of them were not especially well thought out from a space or runtime efficiency perspective, and are rarely accessed in practice. The new tool will maintain parse state a bit differently, but legacy API compatibility requires that these fields continue to "work" from a source-level perspective.

This is a a "compatibility pattern" that eventually arises whenever a concrete type having fields is exposed by an API. At some level, the root problem is that a concrete type was exposed where an interface should have been exposed instead. Once that is done, source compatibility perpetuates the API design error indefinitely.

Solution Sketch In C#, there is a notion of attributes. These implement a getter/setter pattern without requiring function call syntax at the point of access. Use occurrences of the attribute name are transparently translated to getter calls. Update occurrences (assignments) are transparently translated to setter calls. Their implementation is defined in terms of a high-level syntactic rewrite to methods with a specific name rewriting convention.

This approach can be lifted wholesale and unchanged from C# (right down to the syntax) for use in Go. For those not familiar with this corner of C# syntax, it is presented here in the C# guide. The surface syntax would want to be adapted to a more Go-like syntax.

Doing so addresses or mitigates four problems:

1. It provides a recovery path for inadequately encapsulated APIs (at least in many cases).
2. It implements the intermittently requested feature of fields within interfaces.
3. It eliminates the syntactic overheads of current alternatives at use and update occurrences.
4. It simplifies some concurrency patterns by making lock acquisition transparent.

I'm sure other objections will be raised, but here are the objections I see to adding properties in Go:

1. They feel like a bell and whistle addition to an intentionally spare language. This, I suspect, is likely to be the main objection, and I think it is an objection worth considering carefully.
2. In contrast to the .Net environment, properties are not a compatible replacement for fields when viewed from other languages. From the C perspective, getters and setters look like methods on an object, and the underlying concrete field still (or at least may still be) exposed.¹ So they address a set of issues within Go code, but the solution will not extend naturally across some very popular language boundaries.
3. This means that they introduce a new "level" of compatibility objectives to be considered by Go designers: go source compatibility or cross-language compatibility.
4. Go prefers parsimonious surface syntax. I suspect one could do better than C# with some thought, but some syntactic bulk seems to be inherent in defining what amounts to a pair of procedures.
5. If properties are treated as a type, rather than handled entirely as a high-level syntactic transformation, some work in the type checker may be needed. Offhand, I see no advantage to treating them as a type.
6. The keywords get and set would either need to be reserved, or would need to be specified as syntactically significant only in the syntactic context of property definition. Either is straightforward. Given the conceptual weight often placed on these identifiers as prefixes or suffixes, the "only in property syntactic context" approach may be preferable.

These objections being noted, some recurring issues would be simplified by introducing properties:

1. They make source-level backwards compatible revisions possible in the face of incompletely encapsulated APIs.
2. They address the intermittently requested feature of fields within interfaces.

Closing Since I suspect this will be rejected quickly, I haven't yet attempted to adapt the C# property surface syntax to Go. If interest is strong enough, I'm happy to do so, and I suspect I'd be able to create a suitable change set for the Go compiler.

Footnotes

1. Modern versions of C permit anonymous struct fields that enable low-level compatible structure layout without exposing private fields. I do not know whether this is effectively utilized at current Go/C boundaries as a way to enforce Go field visibility rules across language boundaries. ↩

[seankhliao]

Please fill out https://github.com/golang/proposal/blob/master/go2-language-changes.md when proposing language changes

[jsshapiro]

Updated to use language changes template.

[ianlancetaylor]

I think we need to see some examples of what this would look like in Go. You say that we can use the C# syntax verbatim, which may well be true, but when I look at the C# code I see that it starts with public class Person and I assume that you are not proposing that. Thanks.

[jsshapiro]

Fair. And I definitely think the C# syntax would need to be adapted to be Go-like. Here's an attempted translation:

In C#, you would define a property within a class as follows:

   public double Hours
   {
       get { return _seconds / 3600; }
       set {
          if (value < 0 || value > 24)
             throw new ArgumentOutOfRangeException(
                   $"{nameof(value)} must be between 0 and 24.");

          _seconds = value * 3600;
       }
   }

As a first step to a Go-like surface syntax:

// We assume here that _seconds is an already-defined private field of the same struct.
struct TimePeriod {
  _seconds int
  Hours double {
    get { return _seconds / 3600 }
    set {
      if value < 0 || value > 24 {
        return error... // I actually don't know the Go idiom for out-of-range, sorry
      }

      _seconds = value *3600
    }
}

The problem with this, mainly, is that Go doesn't hold with this style of method definition. Offhand (and I'm very much making this up as I go), one approach might be:

struct TimePeriod {
  _seconds int
  // declare Hours to be a property - this syntax also okay in interfaces
  Hours double { get; set } // Has both a getter and a setter; either may be absent
}

// The get/set declaration means the compiler will do the needed rewrite
// at use/update occurrences, to GetHours and SetHours, which in turn
// requires definitions to resolve the references:
func (tp *TimePeriod) GetHours() int {
  return _seconds / 3600
}

// IIRC Go treats assignments as statements rather than expressions, so no return value here
func (tp *TimePeriod) SetHours(val int) {
    if value < 0 || value > 24 {
      // Not clear how to translate the raised exception, but that's a separate topic
    }

    _seconds = value *3600
  }
}

So the declaration identifies the field as a property, which triggers special handling at use and update occurrences, and also signals the requirement to supply the associated functions.

I'm tempted to suggest that the declaration part could be reduced to something more spare, but I'm of two minds on that:

• On the one hand, Go doesn't currently seem to have a notion of a const field, so there's no compelling motivation for properties that have a getter without a setter.
• On the other hand, I suspect this is going to emerge over time as a thing the community wants to reconsider (perhaps in the form of init-only fields), so I'm hesitant to remove the explicit identification of the getter and setter as expected.

As I say, I'm making this up as I go. Hopefully this is "good enough to suck" and we might iterate on it if we think such a feature is conceptually desirable.

The interesting part, really, is the change in behavior at field use and update occurrences. The thing the declaration part is doing that is actually important is signaling that the compiler has to do the rewrites at these locations.

[beoran]

You can already implement setters and getters in go that work like this minus an user defined operator for = . Go has no user defined operarors, and I think that is a good thing.

https://go.dev/play/p/RPOi7UU0u2u

// You can edit this code!
// Click here and start typing.
package main

import "fmt"

type Hours struct {
	_seconds *int
}

func (h Hours) Get() int {
	return *(h._seconds) / 3600
}

func (h *Hours) Set(val int) {
	*(h._seconds) = val * 3600
}

type TimePeriod struct {
	_seconds int
	Hours
}

func MakeTimePeriod() TimePeriod {
	t := TimePeriod{}
	t.Hours._seconds = &t._seconds
	return t
}

func main() {
	t := MakeTimePeriod()
	t.Hours.Set(2)
	fmt.Printf("Hours: %d\n", t.Hours.Get())
}

[jsshapiro]

@beoran: I'm certainly aware that the pattern you suggest is possible. It does not address the API compatibility issue that I have raised, which is the motivation for the proposal. Given a pre-existing API that has exposed fields, your proposal does not provide an evolution path that is source compatible with the existing API.

I mostly share your reservations about the operator overloading rabbit hole, because it is incredibly easy to get overloading wrong (in a whole bunch of ways). Properties aren't the same thing as operator overloading, and they do not carry the same design risk. Offhand, I cannot think of any "property enabled" language that has encountered major issues because properties are present in the language. In some cases, the ability to convert fields to properties has enabled very interesting behavior.

There are a number of Go APIs out there whose authors did not fully internalize the requirements for future proofing and didn't come up with perfect designs the first time. There will be more such APIs over time, if only because many new programmers will create new APIs that don't deal with future proofing either.

So the questions, to my mind, are:

1. Can this kind of API be evolved without a language change? I do not see how to do it within the current language specification.
2. If a language change is needed, what change should it be? Speaking as a programming language architect myself, my bias in this case is "don't invent something new when something that exists already solves the problem well."
3. Is the issue important enough that we should address it at all or at this time?

The last one is may be the most interesting. Because the API evolution issue will become a source of increasing "design pressure" over time, some solution will eventually be needed. It's the type of thing that is part of the price of success for any programming language.

Reasonable people could certainly disagree. Eventually, I think those voices are going to get overridden by accreted code and evolution requirements. That doesn't necessarily have to mean today, but I believe that the question is "when and how" rather than "if".

[beoran]

If an API has some fields exposed which turn out to be undesirable after, then I would slap a depreciation comment on it and then make a next module version where it is gone.

It's not a smooth migration path, but converting the the field to a getter/setter only for backwards compatibility with a mistake seems like a mistake as well.

[ianlancetaylor]

In Go we generally want the code on the page to indicate the execution cost. A function call may take some unknown amount of time. An references to a variable or struct field, on the other hand, will not. When the user just refers to a field, they expect that it will simply load the field, and similarly for an assignment. This proposal would break this property.

There are various idioms for handling this in general. For example, name the field f and add methods F and SetF.

The emoji voting on the proposal is not in favor.

Therefore, this is a likely decline. Leaving open for four weeks for final comments.

[apparentlymart]

When I consider this from the (very reasonable) perspective of evolving an existing API with new capabilities while staying source compatible, it does still seem to have some rough spots:

• If I have a value v of a struct type that has a field Foo, I can write &v.Foo to take the address of that field inside that object. Since a getter invents an ephemeral value on request rather than committing that value to storage, there presumably isn't any location to take the address of.

  Letting the ephemeral value escape to the heap and then returning the address of that heap allocation is possible in principle, but that seems to mean that if I evaluate &v.Foo twice I will get a different address each time.

  Does C# avoid this problem as a result of not having explicit pointers? I do remember there being address and dereference operators in C#, but I suspect I'm remembering unsafe C# rather than the typical safe language.

• Fallible setters as we see in languages like C# rely on the use of exceptions to signal that the assigned value is out of range for the property. I don't think there's precedent for assignment to panic or otherwise fail in Go and so existing users of your API presumably expect to be able to assign any value assignable to the field's type without a panic. (I'd consider assigning to a field through a pointer to be a panicking pointer deference rather than an a panicking assignment, but will concede that current Go syntax hides that distinction by doing the pointer deref implicitly.)

  Would your assumption be that although you can't change the callers you can still review them and determine that none of them currently assign a value outside the range of what your setter would accept?

---

Thinking about the above potential problems reminded me tangentially of the design of mutable index overloading in Rust. Notice that this trait is required to return an mutable borrow, which for our purposes here is essentially analogous to returning a pointer in Go. This guarantees that there must be some real location in memory that this index refers to.

If we instead allowed only hooking the "address of" for a field and required that code to return a pointer to a memory location then it could potentially return either a pointer into part of the receiver or a pointer to something on the heap that could then be read or written through, but again there would be no guarantee that two accesses would yield the same pointer. And even if that's fine, it does kinda seem to miss the point of allowing the type to hook into reads and writes of the field: once the type has exposed a pointer to a memory location, anything holding that pointer can read and write arbitrarily from that location without any opportunity to intervene.

Of course this proposal is talking about (essentially) overloading member access rather than indexing, but it seems like it turns up a comparable set of design challenges either way. The Rust community has been debating whether and how to allow non-reference-based indexing for a long time with many questions still unanswered; rust-lang/rfcs#997 seems like the best entry-point into all of those discussions.

---

Overall it seems to me like accessing a field in Go is just a fundamentally different thing to calling a function, and so I'm having trouble imagining ways to make a hidden method call behave exactly like reading from or writing to a field, such that I would be confident in asserting that my change from a regular field to a getter/setter pair would not be a breaking change to any existing caller. 🤔

[ianlancetaylor]

No change in consensus.