-
Notifications
You must be signed in to change notification settings - Fork 369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ElasticSearch v. 8 upgrade #962
Conversation
docker-compose.yml
Outdated
- "ES_JAVA_OPTS=-Xms${TEST_ES_HEAP_SIZE:-500m} -Xmx${TEST_ES_HEAP_SIZE:-500m}" | ||
- discovery.type=single-node | ||
- xpack.security.enabled=false | ||
- action.destructive_requires_name=false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest leaving this default
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice works, the things that I miss
- let's support both ES 7.x and 8.x, it will be easier to use it in our projects
- provide a documentation
If you want to keep working on it, but still deliver stuff in smaller pieces, we can create a feature branch and merge your changes there.
'type' => 'mapper_parsing_exception', | ||
'reason' => 'object mapping for [name] tried to parse field [name] as object, but found a concrete value' | ||
'type' => 'document_parsing_exception', | ||
'reason' => '[1:9] object mapping for [name] tried to parse field [name] as object, but found a concrete value' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we try to map with regex? Or what exactly this number [1:9]
mean?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's token location, not likely to change IMO.
private static void throwOnConcreteValue(ObjectMapper mapper, String currentFieldName, DocumentParserContext context) {
throw new DocumentParsingException(
context.parser().getTokenLocation(),
"object mapping for ["
+ mapper.fullPath()
+ "] tried to parse field ["
+ currentFieldName
+ "] as object, but found a concrete value"
);
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, let's see how it goes
@@ -32,7 +32,7 @@ def exists? | |||
# | |||
def create(*args, **kwargs) | |||
create!(*args, **kwargs) | |||
rescue Elasticsearch::Transport::Transport::Errors::BadRequest | |||
rescue Elastic::Transport::Transport::Errors::BadRequest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that we have very small difference between 7.x and 8.x, can we make it work with both version? For instance as a possibility, define these constants like
if es_8?
ELASTIC_ERROR_NOT_BAD_REQUEST = Elastic::Transport::Transport::Errors::BadRequest end
else
ELASTIC_ERROR_NOT_BAD_REQUEST = Elasticsearch::Transport::Transport::Errors::BadRequest
end
and use it
def create(*args, **kwargs)
create!(*args, **kwargs)
rescue ELASTIC_ERROR_NOT_BAD_REQUEST
Or even make this prefix configurable
if es_8?
ELASTIC_LIB_CONST_PREFIX = Elastic
else
ELASTIC_LIB_CONST_PREFIX = Elasticsearch
end
and then ELASTIC_LIB_CONST_PREFIX::Transport::Transport::Errors::BadRequest
, not sure if it's going to work :D
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, it's not likely that we need it: #962 (comment)
@konalegi it looks like we need to use here we have here we don't: And official migration guide also recommends to start with 7.17 in compatibility mode, fix critical issues and then update libs to 8: |
with: | ||
stack-version: 7.10.1 | ||
port: 9250 | ||
- name: Start containers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use services
for that? The same way we start redis.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@barthez the containers created this way is quite hard to configure (yeah, we have custom configs :D ) , given that the same containers needed for local development as well 🤷 so I find it quite convenient use docker compose based ES.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, sorry, it's possible, I don't remember why we didn't use them from the start.
|
@amol21p this code is not supposed to support ES 7.17 as per official upgrade guide. Compatibility mode is in the ES server, not in libs, so that 7.* libs can work with 8.* server, not the other way around. So theoretically upgrade goes like this:
I guess for smooth upgrade we need to make sure Chewy 7 plays nice with ES 8 with the flag |
However we had already some changes in our forked version. What would you suggest in that case ? |
Things to consider:
HTTPS + Basic Auth are now enabled by default -- need to mention setup process in README and update CI workflows