Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large row sizes panic on insert #7524

Closed
max-hoffman opened this issue Feb 21, 2024 · 0 comments · Fixed by #7533
Closed

Large row sizes panic on insert #7524

max-hoffman opened this issue Feb 21, 2024 · 0 comments · Fixed by #7533
Labels
panic sql Issue with SQL

Comments

@max-hoffman
Copy link
Contributor

MySQL prevents creating tables when column sizes are too big, for example:

mysql>  create table t (x varchar(30) primary key, y varchar(65535), z varchar(65535));
ERROR 1074 (42000): Column length too big for column 'y' (max = 16383); use BLOB or TEXT instead

We permit this, and allow writes for small string sizes, but panic in different ways approaching the max row size. A setup script is below, with some of the panics I see depending on the specific y column length.

setup:

	longField := ""
	for i := 0; i < 65535; i++ {
		longField += "a"
	}
	var scripts = []queries.ScriptTest{
		{
			Name: "physical columns added after virtual one",
			SetUpScript: []string{
				"create table xy (x varchar(1) primary key, y varchar(65535));",
				fmt.Sprintf("insert into xy values ('%s', '%s')", "a", "a"),
			},
			Assertions: []queries.ScriptTestAssertion{
				{
					Query: "select count(*) from xy",
					Expected: []sql.Row{
						{1},
					},
				},
			},
		},
	}

length 65529:

panic: assertion failed: in-progress chunk must be non-empty to create chunk boundary

goroutine 144 [running]:
github.com/dolthub/dolt/go/store/prolly/tree.assertTrue(...)
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/prolly/tree/node_cursor.go:622
github.com/dolthub/dolt/go/store/prolly/tree.(*chunker[...]).handleChunkBoundary(0x1?, {0x1057ab2f0?, 0x140007d15e0?})
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/prolly/tree/chunker.go:331 +0x158
github.com/dolthub/dolt/go/store/prolly/tree.(*chunker[...]).append(0x10584d020, {0x1057ab2f0, 0x140007d15e0}, {0x14000010108, 0x4, 0x4}, {0x14001b24000, 0xfffc, 0xfffc}, 0x1)
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/prolly/tree/chunker.go:293 +0xa8

length 65535:

panic: runtime error: index out of range [65535] with length 0 [recovered]
	panic: runtime error: index out of range [65535] with length 0

goroutine 163 [running]:
testing.tRunner.func1.2({0x107b5b700, 0x140001b2288})
	/usr/local/go/src/testing/testing.go:1545 +0x1c4
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1548 +0x360
panic({0x107b5b700?, 0x140001b2288?})
	/usr/local/go/src/runtime/panic.go:914 +0x218
github.com/dolthub/dolt/go/store/val.writeByteString(...)
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/val/codec.go:591
github.com/dolthub/dolt/go/store/val.writeString(...)
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/val/codec.go:576
github.com/dolthub/dolt/go/store/val.(*TupleBuilder).PutString(0x14000ed5290, 0x14000060250?, {0x14001542000?, 0xffff})
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/val/tuple_builder.go:300 +0x20c
github.com/dolthub/dolt/go/store/prolly/tree.PutField({0x107cfb2f0, 0x1400099bb80}, {0x107d02290, 0x1400027bb80}, 0x14000ed5290, 0x0, {0x1078280a0?, 0x140000603e0?})
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/prolly/tree/prolly_fields.go:224 +0x1778
github.com/dolthub/dolt/go/libraries/doltcore/sqle/writer.prollyIndexWriter.Insert({0x14000702900, 0x14000ed4f30, {0x140001b18e8, 0x1, 0x1}, 0x14000ed5290, {0x140001b18f0, 0x1, 0x1}}, {0x107cfb2f0, ...}, ...)
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/libraries/doltcore/sqle/writer/prolly_index_writer.go:144 +0x130

length 65533:

panic: runtime error: slice bounds out of range [-2:] [recovered]
	panic: runtime error: slice bounds out of range [-2:]

goroutine 127 [running]:
testing.tRunner.func1.2({0x107193700, 0x140001b38f0})
	/usr/local/go/src/testing/testing.go:1545 +0x1c4
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1548 +0x360
panic({0x107193700?, 0x140001b38f0?})
	/usr/local/go/src/runtime/panic.go:914 +0x218
github.com/dolthub/dolt/go/store/val.writeFieldCount(...)
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/val/tuple.go:194
github.com/dolthub/dolt/go/store/val.allocateTuple({0x107324e18?, 0x10a57a233?}, 0xfffe, 0x1)
	/Users/maxhoffman/go/github.com/dolthub/dolt/go/store/val/tuple.go:119 +0xdc

length 65551:

Received unexpected error:
        	            	length is 65551 but max allowed is 16383
        	            	
        	            	github.com/dolthub/go-mysql-server/sql/types.CreateString
        	            		/Users/maxhoffman/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.17.1-0.20240220224127-651f4cc10f14/sql/types/strings.go:138
        	            	github.com/dolthub/go-mysql-server/sql/types.ColumnTypeToType
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
panic sql Issue with SQL
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant