test: stability of bucket_id -> replicaset mapping is not guaranteed

[Totktonada]

There are tests (at least mine one from PR #222), which lean on stability of bucket_id -> replicaset mapping: this mapping should be the same from run to run. See:

crud/test/integration/select_test.lua

Lines 1593 to 1614 in 5e8c91e

	-- gh-220: bucket_id argument is ignored when it cannot be deduced
	-- from provided select/pairs conditions.
	pgroup.test_select_no_map_reduce = function(g)
	local customers = helpers.insert_objects(g, 'customers', {
	{
	-- bucket_id is 477, storage is s-2
	id = 1, name = 'Elizabeth', last_name = 'Jackson',
	age = 12, city = 'New York',
	}, {
	-- bucket_id is 401, storage is s-2
	id = 2, name = 'Mary', last_name = 'Brown',
	age = 46, city = 'Los Angeles',
	}, {
	-- bucket_id is 2804, storage is s-1
	id = 3, name = 'David', last_name = 'Smith',
	age = 33, city = 'Los Angeles',
	}, {
	-- bucket_id is 1161, storage is s-2
	id = 4, name = 'William', last_name = 'White',
	age = 81, city = 'Chicago',
	},
	})

The test uses it several lines below:

crud/test/integration/select_test.lua

Lines 1632 to 1639 in 5e8c91e

	t.assert_equals(storage_stat.diff(stat_b, stat_a), {
	['s-1'] = {
	select_requests = 1,
	},
	['s-2'] = {
	select_requests = 0,
	},
	})

It works, when several assumptions are true:

• We don't add or remove storage replicasets.
• Replicaset UUIDs are the same.
• pairs() iteration over replicasets table in vshard is stable (the same from run to run).

In this conditions vshard just traverse over replicaset in the same order and created the same ranges of bucket ids on the same replicasets.

The point about stability of pairs() is a bit vague. It seems, it works on tarantool, however there is no such guarantee. It would not work on recent LuaJIT and may become broken in tarantool if we'll pulldown some LuaJIT patches (I guess, it is relevant to this one; Update: @Buristan corrects me that pairs() non-deterministic behavior is actually introduced in LuaJIT/LuaJIT@ff34b48).

There are two ways to make the code safe:

1. Write the test without assumption about where particular bucket resides.

   • I don't like that the test (and all such tests) would become less straightforward and will be more abstract, so harder to understand.

     To illustrate: what if I'll need two keys, which gives different bucket ids, which should reside on different replicasets? Sure, it is possible to generate them, but it requires extra code and will make things cumbersome. We'll make the idea of the test less obvious for a reader.

2. Don't call vshard.router.bootstrap(), disable rebalancer and create buckets manually using vshard.storage.bucket_force_create().

   • It may be tricky, because we use 'vshard-router' cartridge role, which bootstraps vshard router on its own.

     But, maybe, implement the ability to replace vshard's router bootstrap code with your own one for the 'vshard-storage' role? (Filed Allow to redefine vshard router boostrapping code in the vshard-router role cartridge#1606.)

(Thanks @Gerold103 and @olegrok for answering questions and far-beyound-working-time discussions, so finally I ended up with conslusion that the testing code is not well written and filed this issue.)