Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Mongoid::Shardable::ClassMethods#shard_key params from array t… #4650

Closed
wants to merge 8 commits into from
Closed

Change Mongoid::Shardable::ClassMethods#shard_key params from array t… #4650

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3e72cae
Change Mongoid::Shardable::ClassMethods#shard_key params from array t…
xuxiangyang Jul 23, 2019
9545fcb
add doc for shard_key
xuxiangyang Aug 1, 2019
d88c7cd
fix mongoid-sharding.txt Title overline too short
xuxiangyang Aug 7, 2019
7dbbb8c
enhance shard_key with other args in sh.shardCollection api
xuxiangyang Aug 8, 2019
7ae9e2b
tunning
xuxiangyang Aug 8, 2019
2a62181
support old syntax
xuxiangyang Aug 21, 2019
43562af
do not try shard if collection already sharded
xuxiangyang Sep 5, 2019
d93cf7f
Merge branch 'master' of github.com:mongodb/mongoid
xuxiangyang Jan 6, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions lib/mongoid/railties/database.rake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ namespace :db do
task :remove_indexes => "mongoid:remove_indexes"
end

unless Rake::Task.task_defined?("db:shard_collections")
task :shard_collections => "mongoid:shard_collections"
end

namespace :mongoid do
task :load_models do
::Rails.application.eager_load! if defined?(::Rails)
Expand Down
14 changes: 7 additions & 7 deletions lib/mongoid/shardable.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ module Shardable

included do
cattr_accessor :shard_key_fields
self.shard_key_fields = []
self.shard_key_fields = {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an API change. Why was it made?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if there is no one using shard_key, it may be better to change api. I think it is ugly to write like this

self.shard_key_fields = []
self.shard_key_hash = {}

and there will be two method to declare a shard_key

class User
  include Mongoid::Document
  include Mongoid::Timestamps
  
  shard_key :created_at
  shard_key_config created_at: "hashed"
end

shard_key_hash and shard_key_config may have a better name, but it will be confused to have two ways to declare shard_key

end

# Get the shard key fields.
Expand All @@ -21,7 +21,7 @@ module Shardable
# @example Get the shard key fields.
# model.shard_key_fields
#
# @return [ Array<String> ] The shard key field names.
# @return [ Hash ] The shard key field names.
#
# @since 1.0.0
def shard_key_fields
Expand All @@ -38,7 +38,7 @@ def shard_key_fields
# @since 2.0.0
def shard_key_selector
selector = {}
shard_key_fields.each do |field|
shard_key_fields.keys.each do |field|
selector[field.to_s] = new_record? ? send(field) : attribute_was(field)
end
selector
Expand All @@ -55,13 +55,13 @@ module ClassMethods
# field :first_name, :type => String
# field :last_name, :type => String
#
# shard_key :first_name, :last_name
# shard_key :first_name => 1, :last_name => 1
# end
#
# @since 2.0.0
def shard_key(*names)
names.each do |name|
self.shard_key_fields << self.database_field_name(name).to_sym
def shard_key(**shard_configs)
shard_configs.each do |key, value|
self.shard_key_fields[self.database_field_name(key).to_sym] = value
end
end
end
Expand Down
5 changes: 5 additions & 0 deletions lib/mongoid/tasks/database.rake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ namespace :db do
::Mongoid::Tasks::Database.remove_indexes
end

desc "Shard the collections with shard_key defined on your mongoid models"
task :shard_collections => [:environment, :load_models] do
::Mongoid::Tasks::Database.shard_collections
end

desc "Drops the default client database"
task :drop => :environment do
::Mongoid::Clients.default.database.drop
Expand Down
20 changes: 20 additions & 0 deletions lib/mongoid/tasks/database.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,26 @@ def remove_indexes(models = ::Mongoid.models)
end.compact
end

# Shared collection for each model given the provided globs and the class is
# not embedded..
#
# @example Shard all collections
# Mongoid::Tasks::Database.shard_collections
#
# @return [ Array<Class> ] The sharded models
def shard_collections(models = ::Mongoid.models)
models.each do |model|
next unless !model.embedded? || model.cyclic?
next unless model.collection.cluster.sharded?
next if model.shard_key_fields.blank?

admin_db = model.collection.client.list_mongo_databases(name: :admin).first
admin_db.command(enableSharding: model.collection.database.name)
admin_db.command(shardCollection: model.collection.namespace, key: model.shard_key_fields)
logger.info("MONGOID: Shard for #{model.collection.namespace}")
end.compact
end

private

def logger
Expand Down
12 changes: 6 additions & 6 deletions spec/mongoid/shardable_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
expect(klass).to respond_to(:shard_key_fields)
end

it "defaults shard_key_fields to an empty array" do
it "defaults shard_key_fields to an empty hash" do
expect(klass.shard_key_fields).to be_empty
end
end
Expand All @@ -29,11 +29,11 @@
end

before do
Band.shard_key(:name)
Band.shard_key(name: 1)
end

it "specifies a shard key on the collection" do
expect(klass.shard_key_fields).to eq([:name])
expect(klass.shard_key_fields).to eq({name: 1})
end

context 'when a relation is used as the shard key' do
Expand All @@ -43,11 +43,11 @@
end

before do
Game.shard_key(:person)
Game.shard_key(person: 1)
end

it "converts the shard key to the foreign key field" do
expect(klass.shard_key_fields).to eq([:person_id])
expect(klass.shard_key_fields).to eq({person_id: 1})
end
end
end
Expand All @@ -57,7 +57,7 @@
let(:klass) { Band }
let(:value) { 'a-brand-name' }

before { klass.shard_key(:name) }
before { klass.shard_key(name: 1) }

context 'when record is new' do
let(:instance) { klass.new(name: value) }
Expand Down