Skip to content

Commit

Permalink
Merge pull request #529 from toptal/dynamic-names
Browse files Browse the repository at this point in the history
Dynamic names
  • Loading branch information
pyromaniac authored Jun 26, 2017
2 parents 298831b + ed4c2e5 commit bcd4cc9
Show file tree
Hide file tree
Showing 19 changed files with 240 additions and 105 deletions.
35 changes: 24 additions & 11 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,20 @@
# master

## Breaking changes

* Changed behavior of `Chewy::Index.index_name`, it doesn't cache the values anymore.

## Changes

* Deprecate `Chewy::Index.build_index_name`.

* Rename `Chewy::Index.default_prefix` to `Chewy::Index.prefix`. The old one is deprecated.

* Add `Chewy::Type.derivable_name` for consistency.

* Rename `Chewy::Index.derivable_index_name` to `Chewy::Index.derivable_name`.
`Chewy::Index.derivable_index_name` and `Chewy::Type.derivable_index_name` are deprecated.

* Use normal YAML loading, for the config, we don't need the safe one.

* Consistency checks and synchronization: `rake chewy:sync`.
Expand All @@ -14,7 +27,7 @@

* Brand new request DSL. Supports ElasticSearch 2 and 5, better usability, architecture and docs.

* Kaminari 1.0 support.
* Add Kaminari 1.0 support.

* `skip_index_creation_on_import` option (@sergey-kintsel, #483)

Expand Down Expand Up @@ -108,6 +121,12 @@

# Version 0.8.3

## Breaking changes:

* `Chewy.atomic` and `Chewy.urgent_update=` methods was removed from the codebase, use `Chewy.strategy` block instead.

* `delete_from_index?` hook is removed from the codebase.

## Changes

* Sequel support completely reworked to use common ORM implementations + better sequel specs covarage.
Expand All @@ -124,12 +143,6 @@

* Correct custom assets path silencer (@davekaro)

## Incompatible changes:

* `Chewy.atomic` and `Chewy.urgent_update=` methods was removed from the codebase, use `Chewy.strategy` block instead.

* `delete_from_index?` hook is removed from the codebase.

# Version 0.8.2

## Changes
Expand Down Expand Up @@ -165,7 +178,7 @@

# Version 0.8.0

## Incompatible changes:
## Breaking changes:

* `:atomic` and `:urgent` strategies are using `import!` method raising exceptions

Expand All @@ -191,7 +204,7 @@

# Version 0.7.0

## Incompatible changes:
## Breaking changes:

* `Chewy.use_after_commit_callbacks = false` returns previous RDBMS behavior
in tests
Expand Down Expand Up @@ -340,7 +353,7 @@
# Version 0.5.2
## Incompatible changes:
## Breaking changes:
* `Chewy::Type::Base` removed in favour of using `Chewy::Type` as a base class for all types
Expand Down Expand Up @@ -380,7 +393,7 @@
# Version 0.5.0
## Incompatible changes:
## Breaking changes:
* 404 exception (IndexMissingException) while query is swallowed and treated like an empty result set.
Expand Down
2 changes: 1 addition & 1 deletion lib/chewy.rb
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ def derive_type(name)
return name if name.is_a?(Class) && name < Chewy::Type

types = derive_types(name)
raise Chewy::UnderivableType, "Index `#{types.first.index}` has more than one type, please specify type via `#{types.first.index.derivable_index_name}#type_name`" unless types.one?
raise Chewy::UnderivableType, "Index `#{types.first.index}` has more than one type, please specify type via `#{types.first.index.derivable_name}#type_name`" unless types.one?
types.first
end

Expand Down
136 changes: 106 additions & 30 deletions lib/chewy/index.rb
Original file line number Diff line number Diff line change
Expand Up @@ -19,43 +19,95 @@ class Index
self._settings = Chewy::Index::Settings.new

class << self
# Setups or returns ElasticSearch index name
# @overload index_name(suggest)
# If suggested name is passed, it is set up as the new base name for
# the index. Used for the index base name redefinition.
#
# class UsersIndex < Chewy::Index
# end
# UsersIndex.index_name # => 'users'
# @example
# class UsersIndex < Chewy::Index
# index_name :legacy_users
# end
# UsersIndex.index_name # => 'legacy_users'
#
# class UsersIndex < Chewy::Index
# index_name 'dudes'
# end
# UsersIndex.index_name # => 'dudes'
# @param suggest [String, Symbol] suggested base name
# @return [String] new base name
#
# @overload index_name(prefix: nil, suffix: nil)
# If suggested name is not passed, returns the base name accompanied
# with the prefix (if any) and suffix (if passed).
#
def index_name(suggest = nil)
raise UndefinedIndex unless _index_name(suggest)
# @example
# class UsersIndex < Chewy::Index
# end
#
# Chewy.settings = {prefix: 'test'}
# UsersIndex.index_name # => 'test_users'
# UsersIndex.index_name(prefix: 'foobar') # => 'foobar_users'
# UsersIndex.index_name(suffix: '2017') # => 'test_users_2017'
# UsersIndex.index_name(prefix: '', suffix: '2017') # => 'users_2017'
#
# @param prefix [String] index name prefix, uses {.prefix} method by default
# @param suffix [String] index name suffix, used for creating several indexes for the same alias during the zero-downtime reset
# @raise [UndefinedIndex] if the base name is blank
# @return [String] result index name
def index_name(suggest = nil, prefix: nil, suffix: nil)
if suggest
@index_name = nil
_index_name(suggest)
@base_name = suggest.to_s.underscore.presence
else
@index_name ||= build_index_name(_index_name, prefix: default_prefix)
[
prefix || prefix_with_deprecation,
base_name,
suffix
].reject(&:blank?).join('_')
end
end

def _index_name(suggest = nil)
if suggest
@_index_name = suggest
elsif name
@_index_name ||= name.sub(/Index\Z/, '').demodulize.underscore
end
@_index_name
# Base name for the index. Uses the default value inferred from the
# class name unless redefined.
#
# @example
# class Namespace::UsersIndex < Chewy::Index
# end
# UsersIndex.index_name # => 'users'
#
# Class.new(Chewy::Index).base_name # => raises UndefinedIndex
#
# @raise [UndefinedIndex] when the base name is blank
# @return [String] current base name
def base_name
@base_name ||= name.sub(/Index\z/, '').demodulize.underscore if name
raise UndefinedIndex if @base_name.blank?
@base_name
end

def derivable_index_name
@_derivable_index_name ||= name.sub(/Index\Z/, '').underscore
# Similar to the {.base_name} but respects the class namespace, also,
# can't be redefined. Used to reference index with the string identifier
#
# @example
# class Namespace::UsersIndex < Chewy::Index
# end
# UsersIndex.derivable_name # => 'namespace/users'
#
# Class.new(Chewy::Index).derivable_name # => nil
#
# @return [String, nil] derivable name or nil when it is impossible to calculate
def derivable_name
@derivable_name ||= name.sub(/Index\z/, '').underscore if name
end

# Setups or returns pure Elasticsearch index name
# without any prefixes/suffixes
def default_prefix
# Used as a default value for {.index_name}. Return prefix from the configuration
# but can be redefined per-index to be more dynamic.
#
# @example
# class UsersIndex < Chewy::Index
# def self.prefix
# 'foobar'
# end
# end
# UsersIndex.index_name # => 'foobar_users'
#
# @return [String] prefix
def prefix
Chewy.configuration[:prefix]
end

Expand Down Expand Up @@ -166,11 +218,6 @@ def journal?
types.any?(&:journal?)
end

def build_index_name(*args)
options = args.extract_options!
[options[:prefix], args.first || index_name, options[:suffix]].reject(&:blank?).join('_')
end

def settings_hash
_settings.to_hash
end
Expand Down Expand Up @@ -199,6 +246,35 @@ def index_params
def specification
@specification ||= Specification.new(self)
end

def derivable_index_name
ActiveSupport::Deprecation.warn '`Chewy::Index.derivable_index_name` is deprecated and will be removed soon, use `Chewy::Index.derivable_name` instead'
derivable_name
end

# Handling old default_prefix if it is not defined.
def method_missing(name, *args, &block) # rubocop:disable Style/MethodMissing
if name == :default_prefix
ActiveSupport::Deprecation.warn '`Chewy::Index.default_prefix` is deprecated and will be removed soon, use `Chewy::Index.prefix` instead'
prefix
else
super
end
end

def prefix_with_deprecation
if respond_to?(:default_prefix)
ActiveSupport::Deprecation.warn '`Chewy::Index.default_prefix` is deprecated and will be removed soon, define `Chewy::Index.prefix` method instead'
default_prefix
else
prefix
end
end

def build_index_name(*args)
ActiveSupport::Deprecation.warn '`Chewy::Index.build_index_name` is deprecated and will be removed soon, use `Chewy::Index.index_name` instead'
index_name(args.extract_options!)
end
end
end
end
42 changes: 22 additions & 20 deletions lib/chewy/index/actions.rb
Original file line number Diff line number Diff line change
Expand Up @@ -52,17 +52,18 @@ def create(*args)
# Suffixed index names might be used for zero-downtime mapping change, for example.
# Description: (http://www.elasticsearch.org/blog/changing-mapping-with-zero-downtime/).
#
def create!(*args)
options = args.extract_options!.reverse_merge!(alias: true)
name = build_index_name(suffix: args.first)
def create!(suffix = nil, **options)
options.reverse_merge!(alias: true)
general_name = index_name
suffixed_name = index_name(suffix: suffix)

if Chewy::Runtime.version >= 1.1
body = specification_hash
body[:aliases] = {index_name => {}} if options[:alias] && name != index_name
result = client.indices.create(index: name, body: body)
body[:aliases] = {general_name => {}} if options[:alias] && suffixed_name != general_name
result = client.indices.create(index: suffixed_name, body: body)
else
result = client.indices.create(index: name, body: specification_hash)
result &&= client.indices.put_alias(index: name, name: index_name) if options[:alias] && name != index_name
result = client.indices.create(index: suffixed_name, body: specification_hash)
result &&= client.indices.put_alias(index: suffixed_name, name: general_name) if options[:alias] && name != index_name
end

Chewy.wait_for_status if result
Expand All @@ -78,7 +79,7 @@ def create!(*args)
# UsersIndex.delete '01-2014' # deletes `users_01-2014` index
#
def delete(suffix = nil)
result = client.indices.delete index: build_index_name(suffix: suffix)
result = client.indices.delete index: index_name(suffix: suffix)
Chewy.wait_for_status if result
result
# es-ruby >= 1.0.10 handles Elasticsearch::Transport::Transport::Errors::NotFound
Expand Down Expand Up @@ -164,15 +165,18 @@ def reset!(suffix = nil, journal: false)
result = if suffix.present? && (indexes = self.indexes).present?
create! suffix, alias: false

optimize_index_settings suffix
general_name = index_name
suffixed_name = index_name(suffix: suffix)

optimize_index_settings suffixed_name
result = import suffix: suffix, journal: journal, refresh: !Chewy.reset_disable_refresh_interval
original_index_settings suffix
original_index_settings suffixed_name

client.indices.update_aliases body: {actions: [
*indexes.map do |index|
{remove: {index: index, alias: index_name}}
{remove: {index: index, alias: general_name}}
end,
{add: {index: build_index_name(suffix: suffix), alias: index_name}}
{add: {index: suffixed_name, alias: general_name}}
]}
client.indices.delete index: indexes if indexes.present?
result
Expand All @@ -187,27 +191,25 @@ def reset!(suffix = nil, journal: false)

private

def optimize_index_settings(suffix)
def optimize_index_settings(index_name)
settings = {}
settings[:refresh_interval] = -1 if Chewy.reset_disable_refresh_interval
settings[:number_of_replicas] = 0 if Chewy.reset_no_replicas
update_settings suffix: suffix, settings: settings if settings.any?
update_settings index_name, settings: settings if settings.any?
end

def original_index_settings(suffix)
def original_index_settings(index_name)
settings = {}
if Chewy.reset_disable_refresh_interval
settings.merge! index_settings(:refresh_interval)
settings[:refresh_interval] = '1s' if settings.empty?
end
settings.merge! index_settings(:number_of_replicas) if Chewy.reset_no_replicas
update_settings suffix: suffix, settings: settings if settings.any?
update_settings index_name, settings: settings if settings.any?
end

def update_settings(*args)
options = args.extract_options!
name = build_index_name suffix: options[:suffix]
client.indices.put_settings index: name, body: {index: options[:settings]}
def update_settings(index_name, **options)
client.indices.put_settings index: index_name, body: {index: options[:settings]}
end

def index_settings(setting_name)
Expand Down
3 changes: 2 additions & 1 deletion lib/chewy/index/aliases.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ def indexes
end

def aliases
client.indices.get_alias(index: index_name, name: '*')[index_name].try(:[], 'aliases').try(:keys) || []
name = index_name
client.indices.get_alias(index: name, name: '*')[name].try(:[], 'aliases').try(:keys) || []
rescue Elasticsearch::Transport::Transport::Errors::NotFound
[]
end
Expand Down
4 changes: 2 additions & 2 deletions lib/chewy/index/specification.rb
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def initialize(index)
# @return [true] if everything is fine
def lock!
Chewy::Stash::Specification.import!([
id: @index.derivable_index_name,
id: @index.derivable_name,
value: current.to_json
], journal: false)
end
Expand All @@ -33,7 +33,7 @@ def lock!
#
# @return [Hash] hash produced with JSON parser
def locked
filter = {ids: {values: [@index.derivable_index_name]}}
filter = {ids: {values: [@index.derivable_name]}}
JSON.parse(Chewy::Stash::Specification.filter(filter).first.try!(:value) || '{}')
end

Expand Down
Loading

0 comments on commit bcd4cc9

Please sign in to comment.