Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create storage conversion helper functions #1495

Merged
merged 63 commits into from
May 27, 2021
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
a381d35
Move supporting types into subpackage
theunrepentantgeek Mar 1, 2021
40d0640
Move type visitor creation onto StorageTypeFactory
theunrepentantgeek Mar 1, 2021
2232244
Remove StorageTypesVisitorContext
theunrepentantgeek Mar 1, 2021
8e32767
Rename all receivers
theunrepentantgeek Mar 1, 2021
4886c47
Use AsEnumType() instead of hard cast
theunrepentantgeek Mar 1, 2021
98bb10f
Move creation of storage types into the factory
theunrepentantgeek Mar 1, 2021
e0a8960
Use a separate factory for each group
theunrepentantgeek Mar 2, 2021
1f65c9f
Fix bug generating storage versions of resources
theunrepentantgeek Mar 2, 2021
e9eec16
Use queue for just in time creation of storage types
theunrepentantgeek Mar 22, 2021
14445fb
Improve logging of conversion issues
theunrepentantgeek Mar 31, 2021
1632b82
Don't case convert endpoint names
theunrepentantgeek Mar 31, 2021
26bb92b
Update StorageTypeFactory to use the name queue
theunrepentantgeek Mar 31, 2021
39d9bce
Code gardening
theunrepentantgeek Apr 20, 2021
6f1bc35
Add injection of conversion functions
theunrepentantgeek Apr 20, 2021
8d19b86
Improve debug diagnostics
theunrepentantgeek Apr 20, 2021
cc635d4
Change to using TypeVisitorBuilder
theunrepentantgeek Apr 20, 2021
ae17cc6
Code gardening
theunrepentantgeek Apr 28, 2021
59e9c9e
Rename member to otherDefinition
theunrepentantgeek May 5, 2021
24a855b
Use PropertyContainer to generate conversions
theunrepentantgeek May 5, 2021
48dca90
Code gardening
theunrepentantgeek May 5, 2021
60b112d
Move visitor methods to end of file
theunrepentantgeek May 5, 2021
0c02a15
Create and use visitor to inject functions
theunrepentantgeek May 5, 2021
3bb029c
Code gardening
theunrepentantgeek May 5, 2021
4e7dc96
Make TypeFlag.IsOn() handled nested types
theunrepentantgeek May 5, 2021
d68972b
Implement AsZero for ValidatedType
theunrepentantgeek May 5, 2021
8b4948b
Change implementation of type conversion skipping
theunrepentantgeek May 5, 2021
797ce20
Code gardening
theunrepentantgeek May 5, 2021
ad39514
Clear PackageReferenceSet
theunrepentantgeek May 5, 2021
4b4cdba
Address issues identified by CI build
theunrepentantgeek May 10, 2021
8c9703b
Modify generator to reuse the err local in generated code
theunrepentantgeek May 11, 2021
78412a3
Remove variable shadowing
theunrepentantgeek May 11, 2021
d13be36
Resources need to use function references too
theunrepentantgeek May 11, 2021
5d55e57
Ensure "errors" is imported
theunrepentantgeek May 11, 2021
0301500
Use nested conversion contexts where needed
theunrepentantgeek May 11, 2021
42fce85
Use pointer receiver for converion types
theunrepentantgeek May 11, 2021
e589430
Pass address of local
theunrepentantgeek May 11, 2021
717c8ce
Exclude storage version types from hub version selection
theunrepentantgeek May 12, 2021
0e0e788
Update from k8s-infra PR
theunrepentantgeek May 19, 2021
dd25c46
Create FunctionInjector abstraction
theunrepentantgeek May 19, 2021
027d044
Create HubVersionMarker abstraction
theunrepentantgeek May 19, 2021
00884b3
Create PropertyConverter abstraction
theunrepentantgeek May 19, 2021
f2b15df
Create TypeConverter abstraction
theunrepentantgeek May 19, 2021
e635b87
Add required headers
theunrepentantgeek May 19, 2021
c0c74cb
Fix references
theunrepentantgeek May 19, 2021
97c712f
go fmt
theunrepentantgeek May 23, 2021
7d7b08f
Fix bug with missing KnownLocalsSet in context
theunrepentantgeek May 23, 2021
7de2c99
Ensure pointers are captured, not values
theunrepentantgeek May 24, 2021
e0b6a65
Add Process() to Types
theunrepentantgeek May 25, 2021
ace581a
Preserve ResourceReferences
theunrepentantgeek May 25, 2021
366eb26
Code gardening
theunrepentantgeek May 25, 2021
6d2ce02
Add support for ResourceReference properties
theunrepentantgeek May 25, 2021
f1b5efe
Simplify error handling when creating conversions
theunrepentantgeek May 25, 2021
3bcd98b
Refactor StorageTypeFactory
theunrepentantgeek May 25, 2021
5a55825
Add warning
theunrepentantgeek May 25, 2021
53ed56c
Simplify handling of aggregate errors
theunrepentantgeek May 25, 2021
89009cd
Rename member to group
theunrepentantgeek May 25, 2021
52c3aaf
Include property type in error
theunrepentantgeek May 26, 2021
1cec073
Go fmt
theunrepentantgeek May 26, 2021
e8d4af1
Merge branch 'master' into feature/storage-conversions
theunrepentantgeek May 26, 2021
aa83be6
Apply suggestions from code review
theunrepentantgeek May 26, 2021
21df630
Fix terminology, using package instead of namespace
theunrepentantgeek May 26, 2021
5435b85
Merge from master
theunrepentantgeek May 26, 2021
e28774c
Add GitHub errors reference
theunrepentantgeek May 26, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions hack/generator/pkg/astbuilder/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -291,9 +291,10 @@ func ReturnNoError() dst.Stmt {
//
// errors.Wrap(err, <message>)
matthchr marked this conversation as resolved.
Show resolved Hide resolved
//
func WrappedErrorf(template string, args ...interface{}) dst.Expr {
// (actual package name will be used, which will usually be 'errors')
func WrappedErrorf(errorsPackage string, template string, args ...interface{}) dst.Expr {
return CallQualifiedFunc(
"errors",
errorsPackage,
"Wrap",
dst.NewIdent("err"),
StringLiteralf(template, args...))
Expand Down
1 change: 1 addition & 0 deletions hack/generator/pkg/astmodel/known_locals_set.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ func (locals *KnownLocalsSet) Add(local string) {
locals.names[name] = struct{}{}
}

// HasName returns true if the specified name exists in the set, false otherwise
func (locals *KnownLocalsSet) HasName(name string) bool {
_, ok := locals.names[name]
return ok
Expand Down
2 changes: 1 addition & 1 deletion hack/generator/pkg/astmodel/optional_type.go
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ func (optional *OptionalType) BaseType() Type {

// String implements fmt.Stringer
func (optional *OptionalType) String() string {
return fmt.Sprintf("(optional: %s)", optional.element.String())
return fmt.Sprintf("optional(%s)", optional.element.String())
}

// Unwrap returns the type contained within the wrapper type
Expand Down
16 changes: 10 additions & 6 deletions hack/generator/pkg/astmodel/package_reference.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,19 @@ func PackageAsLocalPackage(pkg PackageReference) (LocalPackageReference, error)

func SortPackageReferencesByPathAndVersion(packages []PackageReference) {
sort.Slice(packages, func(i, j int) bool {
comparer := versionComparer{
left: []rune(packages[i].PackagePath()),
right: []rune(packages[j].PackagePath()),
}
compare := comparer.Compare()
return compare < 0
return ComparePathAndVersion(packages[i].PackagePath(), packages[j].PackagePath())
})
}

// ComparePathAndVersion compares two paths containing versions and returns true if left should go before right
func ComparePathAndVersion(left string, right string) bool {
comparer := versionComparer{
left: []rune(left),
right: []rune(right),
}
return comparer.Compare() < 0
}

// versionComparer captures our state while doing an alphanumeric version comparision
// We need separate indexes for each side because we're doing a numeric comparison, which will
// compare "100" and "0100" as equal (leading zeros are not significant)
Expand Down
5 changes: 5 additions & 0 deletions hack/generator/pkg/astmodel/package_reference_set.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@ func (set *PackageReferenceSet) Remove(ref PackageReference) {
delete(set.references, ref)
}

// Clear removes everything from the set
func (set *PackageReferenceSet) Clear() {
set.references = make(map[PackageReference]struct{})
}

// Contains allows checking to see if an import is included
func (set *PackageReferenceSet) Contains(ref PackageReference) bool {
_, ok := set.references[ref]
Expand Down
4 changes: 4 additions & 0 deletions hack/generator/pkg/astmodel/resource_type.go
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,10 @@ func (resource *ResourceType) RequiredPackageReferences() *PackageReferenceSet {
references.Merge(resource.status.RequiredPackageReferences())
}

for _, fn := range resource.functions {
references.Merge(fn.RequiredPackageReferences())
}

// Interface imports
references.Merge(resource.InterfaceImplementer.RequiredPackageReferences())

Expand Down
30 changes: 15 additions & 15 deletions hack/generator/pkg/astmodel/std_references.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,31 +7,31 @@ package astmodel

var (
// References to standard Go Libraries
ErrorsReference PackageReference = MakeExternalPackageReference("errors")
FmtReference PackageReference = MakeExternalPackageReference("fmt")
JsonReference PackageReference = MakeExternalPackageReference("encoding/json")
ReflectReference PackageReference = MakeExternalPackageReference("reflect")
TestingReference PackageReference = MakeExternalPackageReference("testing")
FmtReference = MakeExternalPackageReference("fmt")
JsonReference = MakeExternalPackageReference("encoding/json")
ReflectReference = MakeExternalPackageReference("reflect")
TestingReference = MakeExternalPackageReference("testing")

// References to our Libraries
GenRuntimeReference PackageReference = MakeExternalPackageReference(genRuntimePathPrefix)
GenRuntimeReference = MakeExternalPackageReference(genRuntimePathPrefix)

// References to other libraries
ErrorsReference = MakeExternalPackageReference("github.com/pkg/errors")
ApiExtensionsReference = MakeExternalPackageReference("k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1")
ApiExtensionsJsonReference = MakeExternalPackageReference("k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1/JSON")

// References to libraries used for testing
CmpReference PackageReference = MakeExternalPackageReference("github.com/google/go-cmp/cmp")
CmpOptsReference PackageReference = MakeExternalPackageReference("github.com/google/go-cmp/cmp/cmpopts")
DiffReference PackageReference = MakeExternalPackageReference("github.com/kylelemons/godebug/diff")
GopterReference PackageReference = MakeExternalPackageReference("github.com/leanovate/gopter")
GopterGenReference PackageReference = MakeExternalPackageReference("github.com/leanovate/gopter/gen")
GopterPropReference PackageReference = MakeExternalPackageReference("github.com/leanovate/gopter/prop")
GomegaReference PackageReference = MakeExternalPackageReference("github.com/onsi/gomega")
PrettyReference PackageReference = MakeExternalPackageReference("github.com/kr/pretty")
CmpReference = MakeExternalPackageReference("github.com/google/go-cmp/cmp")
CmpOptsReference = MakeExternalPackageReference("github.com/google/go-cmp/cmp/cmpopts")
DiffReference = MakeExternalPackageReference("github.com/kylelemons/godebug/diff")
GopterReference = MakeExternalPackageReference("github.com/leanovate/gopter")
GopterGenReference = MakeExternalPackageReference("github.com/leanovate/gopter/gen")
GopterPropReference = MakeExternalPackageReference("github.com/leanovate/gopter/prop")
GomegaReference = MakeExternalPackageReference("github.com/onsi/gomega")
PrettyReference = MakeExternalPackageReference("github.com/kr/pretty")

// Imports with specified names
GomegaImport PackageImport = NewPackageImport(GomegaReference).WithName(".")
GomegaImport = NewPackageImport(GomegaReference).WithName(".")

// Type names
ResourceReferenceTypeName = MakeTypeName(GenRuntimeReference, "ResourceReference")
Expand Down
44 changes: 41 additions & 3 deletions hack/generator/pkg/astmodel/storage_conversion_context.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,40 @@ type StorageConversionContext struct {
types Types
// functionName is the name of the function we're currently generating
functionName string
// knownLocals is a reference to our set of local variables
// (Pointer because it's a reference type, not because it's optional)
knownLocals *KnownLocalsSet
}

// NewStorageConversionContext creates a new instance of a StorageConversionContext
func NewStorageConversionContext(types Types) *StorageConversionContext {
func NewStorageConversionContext(types Types, idFactory IdentifierFactory) *StorageConversionContext {
return &StorageConversionContext{
types: types,
types: types,
knownLocals: NewKnownLocalsSet(idFactory),
}
}

// WithFunctionName returns a new context with the specified function name included
func (c *StorageConversionContext) WithFunctionName(name string) *StorageConversionContext {
result := NewStorageConversionContext(c.types)
result := c.clone()
result.functionName = name
return result
}

// WithKnownLocals returns a new context with the specified known locals set referenced
func (c *StorageConversionContext) WithKnownLocals(knownLocals *KnownLocalsSet) *StorageConversionContext {
result := c.clone()
result.knownLocals = knownLocals
return result
}

// NestedContext returns a new context with a cloned knownLocals so that nested blocks
// can declare and reuse locals independently of each other.
func (c *StorageConversionContext) NestedContext() *StorageConversionContext {
result := c.clone()
return result
}

// ResolveType resolves a type that might be a type name into both the name and the actual
// type it references, returning true iff it was a TypeName that could be resolved
func (c *StorageConversionContext) ResolveType(t Type) (TypeName, Type, bool) {
Expand All @@ -42,3 +61,22 @@ func (c *StorageConversionContext) ResolveType(t Type) (TypeName, Type, bool) {

return name, actualType, true
}

// TryCreateLocal returns true if the specified local was successfully created, false if it already existed
func (c *StorageConversionContext) TryCreateLocal(local string) bool {
if c.knownLocals.HasName(local) {
return false
}

c.knownLocals.Add(local)
return true
}

// clone returns a new independent copy of this context
func (c *StorageConversionContext) clone() *StorageConversionContext {
return &StorageConversionContext{
types: c.types,
functionName: c.functionName,
knownLocals: c.knownLocals.Clone(),
}
}
4 changes: 1 addition & 3 deletions hack/generator/pkg/astmodel/storage_conversion_endpoint.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@
package astmodel

import (
"strings"

"github.com/gobuffalo/flect"
)

Expand All @@ -28,7 +26,7 @@ func NewStorageConversionEndpoint(
knownLocals *KnownLocalsSet) *StorageConversionEndpoint {
return &StorageConversionEndpoint{
theType: theType,
name: strings.ToLower(name),
name: name,
knownLocals: knownLocals,
}
}
Expand Down
Loading