Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't use lex_state emulation on Ruby 2.5 or later #546

Merged
merged 2 commits into from
Nov 7, 2017
Merged

Don't use lex_state emulation on Ruby 2.5 or later #546

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
72ced08
Don't use lex_state emulation on Ruby 2.5 or later
aycabta Oct 19, 2017
ee9d50a
Use Ripper::Filter.method_defined?(:state) instead of RUBY_VERSION
aycabta Oct 25, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 25 additions & 7 deletions lib/rdoc/parser/ripper_state_lex.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
require 'ripper'

class RDoc::RipperStateLex
# TODO: Remove this constants after Ruby 2.4 EOL
RIPPER_HAS_LEX_STATE = Ripper::Filter.method_defined?(:state)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


EXPR_NONE = 0
EXPR_BEG = 1
EXPR_END = 2
Expand Down Expand Up @@ -283,7 +286,22 @@ def each(&block)
@callback = block
parse
end
end
end unless RIPPER_HAS_LEX_STATE

class InnerStateLex < Ripper::Filter
def initialize(code)
super(code)
end

def on_default(event, tok, data)
@callback.call({ :line_no => lineno, :char_no => column, :kind => event, :text => tok, :state => state})
end

def each(&block)
@callback = block
parse
end
end if RIPPER_HAS_LEX_STATE

def get_squashed_tk
if @buf.empty?
Expand All @@ -297,10 +315,10 @@ def get_squashed_tk
when :on_tstring_beg then
tk = get_string_tk(tk)
when :on_backtick then
if (EXPR_FNAME & tk[:state]) != 0
@inner_lex.lex_state = EXPR_ARG
if ((EXPR_FNAME | EXPR_ENDFN) & tk[:state]) != 0
@inner_lex.lex_state = EXPR_ARG unless RIPPER_HAS_LEX_STATE
tk[:kind] = :on_ident
tk[:state] = @inner_lex.lex_state
tk[:state] = EXPR_ARG
else
tk = get_string_tk(tk)
end
Expand All @@ -310,7 +328,7 @@ def get_squashed_tk
tk = get_embdoc_tk(tk)
when :on_heredoc_beg then
@heredoc_queue << retrieve_heredoc_info(tk)
@inner_lex.lex_state = EXPR_END
@inner_lex.lex_state = EXPR_END unless RIPPER_HAS_LEX_STATE
when :on_nl, :on_ignored_nl, :on_comment, :on_heredoc_end then
unless @heredoc_queue.empty?
get_heredoc_tk(*@heredoc_queue.shift)
Expand Down Expand Up @@ -541,9 +559,9 @@ def get_squashed_tk
private def get_op_tk(tk)
redefinable_operators = %w[! != !~ % & * ** + +@ - -@ / < << <= <=> == === =~ > >= >> [] []= ^ ` | ~]
if redefinable_operators.include?(tk[:text]) and EXPR_ARG == tk[:state] then
@inner_lex.lex_state = EXPR_ARG
@inner_lex.lex_state = EXPR_ARG unless RIPPER_HAS_LEX_STATE
tk[:state] = EXPR_ARG
tk[:kind] = :on_ident
tk[:state] = @inner_lex.lex_state
elsif tk[:text] =~ /^[-+]$/ then
tk_ahead = get_squashed_tk
case tk_ahead[:kind]
Expand Down