Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Mismatch column sizes" when inserting default Tuple from JSONEachRow #5653

Closed
akuzm opened this issue Jun 17, 2019 · 3 comments · Fixed by #8606
Closed

"Mismatch column sizes" when inserting default Tuple from JSONEachRow #5653

akuzm opened this issue Jun 17, 2019 · 3 comments · Fixed by #8606
Assignees
Labels
bug Confirmed user-visible misbehaviour in official release comp-formats Input / output formats st-accepted The issue is in our backlog, ready to take

Comments

@akuzm
Copy link
Contributor

akuzm commented Jun 17, 2019

create table t(a Tuple(int, int) default (7, 7));
set input_format_defaults_for_omitted_values = true;
insert into t values {}

Exception on client:
Code: 9. DB::Exception: Mismatch column sizes while adding defaults
@akuzm akuzm added the bug Confirmed user-visible misbehaviour in official release label Jun 17, 2019
@akuzm akuzm self-assigned this Jun 17, 2019
@akuzm
Copy link
Contributor Author

akuzm commented Jun 19, 2019

Turns out the Tuple type is irrelevant. The core problem is that we can't determine the size of a Block that has no columns. We run into this problem at some point, when all the columns of a block are to be filled with defaults.

We have not received any complaints about this issue, and the way we work with Blocks is going to be changed in processors pr (#4914), so let's wait for it.

@ishirav
Copy link

ishirav commented Dec 14, 2019

I got the same problem when using default values with CSV format. I'm using ClickHouse 19.17.5 and input_format_defaults_for_omitted_fields is on by default. For example:

~ clickhouse-client -q "CREATE TABLE foo (id UInt32 DEFAULT 1, date Date DEFAULT today(), random UInt32 DEFAULT rand()) ENGINE = Log"
~ echo -ne "1,'2019-12-31',17\n2,'2019-12-31',18" | clickhouse-client -q "INSERT INTO foo FORMAT CSV"
~ clickhouse-client -q "SELECT * FROM foo"                                                           
1	2019-12-31	17
2	2019-12-31	18
~ echo -ne "3,,19\n2,'2019-12-31'," | clickhouse-client -q "INSERT INTO foo FORMAT CSV"
Code: 9. DB::Exception: Mismatch column sizes while adding defaults

Same problem with TSKV format:

~ echo -ne "id=3\tdate=2019-12-31\trandom=19\n" | clickhouse-client -q "INSERT INTO foo FORMAT TSKV"
~ clickhouse-client -q "SELECT * FROM foo"                                                                                                                                                                             
3	2019-12-31	19
1	2019-12-31	17
2	2019-12-31	18
~ echo -ne "id=3\trandom=19\n" | clickhouse-client -q "INSERT INTO foo FORMAT TSKV"                 
Code: 9. DB::Exception: Mismatch column sizes while adding defaults

No matter what I tried, the same error was returned. The input_format_defaults_for_omitted_fields feature doesn't seem to work at all.

@filimonov
Copy link
Contributor

filimonov commented Dec 20, 2019

create table json_insert (a UInt32 DEFAULT 1, b UInt32 DEFAULT 1) Engine=Log;
insert into json_insert FORMAT JSONEachRow {"a":2}
Exception on client:
Code: 9. DB::Exception: Mismatch column sizes while adding defaults
Stack trace:
0. 0x55b34754d147 StackTrace::StackTrace() /build/obj-x86_64-linux-gnu/../dbms/src/Common/StackTrace.cpp:228
1. 0x55b34754cf43 DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) /build/obj-x86_64-linux-gnu/../dbms/src/Common/Exception.h:26
2. 0x55b34723a05e DB::AddingDefaultsBlockInputStream::checkCalculated(DB::ColumnWithTypeAndName const&, DB::ColumnWithTypeAndName const&, unsigned long) const [clone .cold] /usr/include/c++/9/bits/basic_string.h:222
3. 0x55b34abd29f5 DB::AddingDefaultsBlockInputStream::readImpl() /usr/include/c++/9/bits/stl_bvector.h:187
4. 0x55b34abdb28f DB::IBlockInputStream::read() /usr/include/c++/9/bits/stl_vector.h:108
5. 0x55b34abd4594 DB::AsynchronousBlockInputStream::calculate() /usr/include/c++/9/bits/stl_vector.h:108
6. 0x55b34abd491a std::_Function_handler<void (), DB::AsynchronousBlockInputStream::next()::{lambda()#1}>::_M_invoke(std::_Any_data const&) /usr/include/c++/9/bits/atomic_base.h:551
7. 0x55b34759774a ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::_List_iterator<ThreadFromGlobalPool>) /usr/include/c++/9/bits/atomic_base.h:551
8. 0x55b347597d79 ThreadFromGlobalPool::ThreadFromGlobalPool<ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::function<void ()>, int, std::optional<unsigned long>)::{lambda()#3}>(ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::function<void ()>, int, std::optional<unsigned long>)::{lambda()#3}&&)::{lambda()#1}::operator()() const /build/obj-x86_64-linux-gnu/../dbms/src/Common/ThreadPool.h:155
9. 0x55b34759524a ThreadPoolImpl<std::thread>::worker(std::_List_iterator<std::thread>) /usr/include/c++/9/bits/atomic_base.h:551
10. 0x55b34d75a410 execute_native_thread_routine /usr/lib/debug/usr/bin/clickhouse
11. 0x7fd0158a16db start_thread /lib/x86_64-linux-gnu/libpthread-2.27.so
12. 0x7fd0151be88f clone /build/glibc-OTsEL5/glibc-2.27/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Confirmed user-visible misbehaviour in official release comp-formats Input / output formats st-accepted The issue is in our backlog, ready to take
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants