Rebase yjit_backend_ir branch on ruby/master #358
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bumps [rb-sys](https://github.com/oxidize-rb/rb-sys) from 0.9.20 to 0.9.26. - [Release notes](https://github.com/oxidize-rb/rb-sys/releases) - [Commits](oxidize-rb/rb-sys@v0.9.20...v0.9.26) --- updated-dependencies: - dependency-name: rb-sys dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> rubygems/rubygems@fe76234cf1
The `rb_profile_frames` API did not skip the two dummy frames that each thread has at its beginning. This was unlike `backtrace_each` and `rb_ec_parcial_backtrace_object`, which do skip them. This does not seem to be a problem for non-main thread frames, because both `VM_FRAME_RUBYFRAME_P(cfp)` and `rb_vm_frame_method_entry(cfp)` are NULL for them. BUT, on the main thread `VM_FRAME_RUBYFRAME_P(cfp)` was true and thus the dummy thread was still included in the output of `rb_profile_frames`. I've now made `rb_profile_frames` skip this extra frame (like `backtrace_each` and friends), as well as add a test that asserts the size and contents of `rb_profile_frames`. Fixes [Bug #18907] (<https://bugs.ruby-lang.org/issues/18907>)
Revert "Fix sync_default_gems.rb to use absolute path" This reverts commit 8fa6646.
The result of String#b is a string with an ASCII_8BIT/BINARY encoding. That encoding is ASCII-compatible and has no byte sequences that are invalid for the encoding. If we know the receiver's code range, we can derive the resulting string's code range without needing to perform a full code range scan.
ary_discard should not be used as it should be handled by the GC. The only user of ary_discard is rb_ary_product, which doesn't neeed to use ary_discard.
rb_ary_tmp_new suggests that the array is temporary in some way, but that's not true, it just creates an array that's hidden and not on the transient heap. This commit renames it to rb_ary_hidden_new.
Adding the doc/rdoc directory to .document will allow files in that directory to be included in the documentation.
Tabs were expanded because the file did not have any tab indentation in unedited lines. Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
This parameter was coupling the concept of lockfile with the index. I don't think it's necessary. Also I believe it's causing some flaky test failures, which might leak into realworld issues. They are like this: ```` Invoking `/opt/hostedtoolcache/Ruby/3.0.4/x64/bin/ruby -I/home/runner/work/rubygems/rubygems/bundler/spec -r/home/runner/work/rubygems/rubygems/bundler/spec/support/artifice/fail.rb -r/home/runner/work/rubygems/rubygems/bundler/spec/support/hax.rb /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/bin/bundle lock` failed with output: ---------------------------------------------------------------------- --- ERROR REPORT TEMPLATE ------------------------------------------------------- ``` NoMethodError: undefined method `identifier' for #<Gem::Specification:0x00005639ad0db0a0 @extension_dir=nil, @full_gem_path=nil, @gem_dir=nil, @ignored=nil, @bin_dir=nil, @cache_dir=nil, @cache_file=nil, @doc_dir=nil, @ri_dir=nil, @spec_dir=nil, @spec_file=nil, @gems_dir=nil, @base_dir=nil, @loaded=false, @activated=false, @loaded_from=nil, @original_platform="ruby", @installed_by_version=nil, @autorequire=nil, @Date=2022-07-25 00:00:00 UTC, @description="This is a completely fake gem, for testing purposes.", @email="foo@bar.baz", @homepage="http://example.com", @name="win32-process", @post_install_message=nil, @signing_key=nil, @summary="This is just a fake gem for testing", @Version=#<Gem::Version "0.8.3">, @authors=["no one"], @bindir="bin", @cert_chain=[], @Dependencies=[<Gem::Dependency type=:runtime name="ffi" requirements=">= 1.0.0">], @executables=[], @extensions=[], @extra_rdoc_files=[], @files=[], @licenses=[], @metadata={}, @platform="ruby", @rdoc_options=[], @require_paths=["lib"], @required_ruby_version=#<Gem::Requirement:0x00005639ad0dbc80 @requirements=[[">=", #<Gem::Version "0">]]>, @required_rubygems_version=#<Gem::Requirement:0x00005639ad0dba50 @requirements=[[">=", #<Gem::Version "0">]]>, @requirements=[], @rubygems_version="3.2.33", @specification_version=4, @test_files=[], @new_platform="ruby", @full_name="win32-process-0.8.3", @has_rdoc=true, @license=["MIT"] win32-process-0.8.3> /home/runner/work/rubygems/rubygems/bundler/tmp/rubygems/lib/rubygems/specification.rb:2116:in `method_missing' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/remote_specification.rb:115:in `method_missing' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/lazy_specification.rb:34:in `eql?' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/index.rb:189:in `eql?' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/index.rb:189:in `search_by_dependency' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/index.rb:96:in `local_search' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/index.rb:64:in `unsorted_search' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/index.rb:60:in `search' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/resolver.rb:179:in `results_for' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/resolver.rb:113:in `search_for' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/resolver.rb:216:in `block in sort_dependencies' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/resolver.rb:207:in `each' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/resolver.rb:207:in `sort_by' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/resolver.rb:207:in `sort_dependencies' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/delegates/specification_provider.rb:60:in `block in sort_dependencies' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/delegates/specification_provider.rb:77:in `with_no_such_dependency_error_handling' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/delegates/specification_provider.rb:59:in `sort_dependencies' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/resolution.rb:754:in `push_state_for_requirements' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/resolution.rb:744:in `require_nested_dependencies_for' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/resolution.rb:727:in `activate_new_spec' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/resolution.rb:684:in `attempt_to_activate' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/resolution.rb:254:in `process_topmost_state' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/resolution.rb:182:in `resolve' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/molinillo/lib/molinillo/resolver.rb:43:in `resolve' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/resolver.rb:50:in `start' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/resolver.rb:24:in `resolve' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/definition.rb:480:in `reresolve' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/definition.rb:283:in `resolve' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/definition.rb:181:in `resolve_remotely!' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/cli/lock.rb:53:in `run' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/cli.rb:674:in `lock' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/cli.rb:31:in `dispatch' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/cli.rb:25:in `start' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/exe/bundle:48:in `block in <top (required)>' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/lib/bundler/friendly_errors.rb:120:in `with_friendly_errors' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/gems/bundler-2.4.0.dev/exe/bundle:36:in `<top (required)>' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/bin/bundle:23:in `load' /home/runner/work/rubygems/rubygems/bundler/tmp/1/gems/system/bin/bundle:23:in `<main>' ``` ```` I think the issue is that now we eagerly materialize some base specifications before resolving in order to give better errors if user specified an incorrect source in the Gemfile. This means that the key for the index hash will have heterogeneous specification objects (some LazySpecification, some real Specification), and `LazySpecification#eql?` is incompatible with that. By dropping the base parameter from the index, we should no longer have these heterogenous objects as hash keys. rubygems/rubygems@dc179d41c3
Not sure why, but I run into the following flaky test failure
````
(...)
Invoking `/Users/deivid/.asdf/installs/ruby/3.1.2/bin/ruby -I/Users/deivid/Code/rubygems/rubygems/bundler/spec -r/Users/deivid/Code/rubygems/rubygems/bundler/spec/support/artifice/fail.rb -r/Users/deivid/Code/rubygems/rubygems/bundler/spec/support/hax.rb /Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/bin/bundle install` failed with output:
----------------------------------------------------------------------
--- ERROR REPORT TEMPLATE -------------------------------------------------------
```
NameError: constant Pathname::FileUtils not defined
FileUtils.mkpath(@path, mode: mode)
^^^^^^^^^
/Users/deivid/.asdf/installs/ruby/3.1.2/lib/ruby/3.1.0/pathname.rb:585:in `mkpath'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/shared_helpers.rb:103:in `filesystem_access'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/rubygems_gem_installer.rb:78:in `build_extensions'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/rubygems_gem_installer.rb:28:in `install'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/source/rubygems.rb:207:in `install'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/installer/gem_installer.rb:54:in `install'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/installer/gem_installer.rb:16:in `install_from_spec'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/installer/parallel_installer.rb:186:in `do_install'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/installer/parallel_installer.rb:177:in `block in worker_pool'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/worker.rb:62:in `apply_func'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/worker.rb:57:in `block in process_queue'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/worker.rb:54:in `loop'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/worker.rb:54:in `process_queue'
/Users/deivid/Code/rubygems/rubygems/bundler/tmp/4/gems/system/gems/bundler-2.4.0.dev/lib/bundler/worker.rb:91:in `block (2 levels) in create_threads'
(...)
```
Whatever it was, this small change should fix it.
rubygems/rubygems@71d7503ce4
- Former section "Directives in Trailing Comments" is reworked. The important thing about a directive is what it does, not whether it's trailing or stand-alone. Therefore I've worked the directives in the former section into the appropriate sections, based on function. - Each directive is now explicitly marked as trailing or stand-alone. - C-code directives are mentioned only for those directives that actually appear in our ruby/ruby C files, which are :startdoc:, :stopdoc:, :enddoc:, :include:, and :call-seq:. What effect, if any, other directives have in C, I'm not sure about. ruby/rdoc@b00978bfa5
What we want to test should be the bundled and to be installed files, but not the upstream.
Treats:
::_httpdate
::_iso8601
::_jisx0301
::_parse
::_rfc2822
::_rfc3339
::_xmlschema
::httpdate
::iso8601
::jisx0301
::parse
::rfc2822
::rfc3339
::xmlschema
ruby/date@24bdab600a
Treats:
::_strptime
::strptime
Adds 'Related' entry to some methods' doc.
ruby/date@a6c2129273
Co-authored-by: Hiroshi SHIBATA <hsbt@ruby-lang.org>
It should be an alias of `--filter-strict`. `--update-strict` is essentially a dummy option with no special behavior associated and should be deprecated. rubygems/rubygems@ec1e5d83c8
They occasionaly fail with;
```
FLeaked thread: TestThreadQueue#test_queue_pop_timeout: #<Thread:0x0000000108e38e48 /Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:123 sleep>
.Finished thread: TestThreadQueue#test_deny_pushers: #<Thread:0x0000000108e38e48 /Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:123 dead>
...
Retrying...
1) Failure:
TestThreadQueue#test_sized_queue_pop_timeout [/Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:157]:
<#<Thread:0x00000001084bc7e8 /Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:156 sleep>> expected but was
<nil>.
2) Failure:
TestThreadQueue#test_queue_pop_timeout [/Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:124]:
<#<Thread:0x00000001083ff058 /Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:123 sleep>> expected but was
<nil>.
```
I'm hoping joining for longer should help avoid this.
ld64 shipped with Xcode 14 emits a warning when using `-undefined dynamic_lookup`. ``` ld: warning: -undefined dynamic_lookup may not work with chained fixups ``` Actually, `-undefined dynamic_lookup` doesn't work when: 1. Link a *shared library* with the option 2. Link it with a program that uses the chained-fixup introduced from macOS 12 and iOS 15 because `-undefined dynamic_lookup` uses lazy-bindings and they won't be bound while dyld fixes-up by traversing chained-fixup info. However, we build exts as *bundles* and they are loaded only through `dlopen`, so it's safe to use `-undefined dynamic_lookup` in theory. So the warning produced by ld64 is false-positive, and it results failure of option checking in configuration. Therefore, it would be an option to ignore the warning during our configuration. On the other hand, `-undefined dynamic_lookup` is already deprecated on all darwin platforms except for macOS, so it's good time to get rid of the option. ld64 also provides `-bundle_loader <executable>` option, which allows to resolve symbols defined in the executable symtab while linking. It behaves almost the same with `-undefined dynamic_lookup`, but it makes the following changes: 1. Require that unresolved symbols among input objects must be defined in the executable. 2. Lazy symbol binding will lookup only the symtab of the bundle loader executable. (`-undefined dynamic_lookup` lookups all symtab as flat namespace) This patch adds `-bundle_loader $(RUBY)` when non-EXTSTATIC configuration by assuming ruby executable can be linked before building exts. See "New Features" subsection under "Linking" section for chained fixup https://developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes
For the macOS -bundle_loader linker option, we need a path to the Ruby exectuable. $(RUBY) is not necessarily a path since it could be a command line invocation. That happens during build with runruby.rb and can happen post installation if the user passes the --ruby option to a extconf.rb. Use $(bindir) to locate the executable instead. Before installation, $(bindir) doesn't exist, so we need to be able to override $(BUILTRUBY) in such situations so test-spec and bundled extensions could build. Use a new mkmf global, $builtruby, to do this; set it in fake.rb and in extmk.rb. Our build system is quite complex...
Adding `ruby` to `PREP` causes the following circular dependencies because `PREP` is used as a prerequisite by some targets required to build `ruby` target itself. ``` make: Circular .rbconfig.time <- ruby dependency dropped. make: Circular builtin_binary.inc <- ruby dependency dropped. make: Circular ext/extinit.c <- ruby dependency dropped. make: Circular ruby <- ruby dependency dropped. ``` Adding a new Make variable like `EXTPREP` only for exts may be also reasonable, but it would introduce another complexity into our build system. `-bundle_loader` doesn't care that link-time and run-time loader executables are different as long as bound symbols are provided, so it's ok to resolve from miniruby to simplify our build.
When I removed the SizeQueue#push timeout from my PR, I forgot to update the `queue_sleep` parameters to be a `queue_sleep_arg`. Somehow this worked on most archs, but on Solaris/Sparc it would legitimately crash when trying to access the `timeout` and `end` members of the struct.
``` /Users/deivid/Code/rubygems/rubygems/test/rubygems/test_gem_resolver_installer_set.rb:55: warning: assigned but unused variable - a_1_local ``` rubygems/rubygems@9ea4534800
* YJIT: Add known_* helpers for Type This adds a few helpers to Type which all return Options representing what is known, from a Ruby perspective, about the type. This includes: * known_class_of: If known, the class represented by this type * known_value_type: If known, the T_ value type * known_exact_value: If known, the exact VALUE represented by this type (currently this is only available for true/false/nil) * known_truthy: If known, whether or not this value evaluates as true (not false or nil) The goal of this is to abstract away the specifics of the mappings between types wherever possible from the codegen. For example previously by introducing Type::CString as a more specific version of Type::TString, uses of Type::TString in codegen needed to be updated to check either case. Now by using known_value_type, at least in theory we can introduce new types with minimal (if any) codegen changes. I think rust's Option type allows us to represent this uncertainty fairly well, and should help avoid mistakes, and the matching using this turned out pretty cleanly. * YJIT: Use known_value_type for checktype * YJIT: Use known_value_type for T_STRING check * YJIT: Use known_class_of in guard_known_klass * YJIT: Use known truthyness in jit_rb_obj_not * YJIT: Rename known_class_of => known_class
…ing (ruby#6205) Allow str-concat arg to be any string subtype, not just rb_cString
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.