Speeding up RDoc by 85% in CRuby documents generation #640
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aycabta
force-pushed
the
optimize-hot-spot
branch
from
e877d55 to
01c65fa
Compare
aycabta
changed the title
aycabta
changed the title
aycabta
force-pushed
the
optimize-hot-spot
branch
3 times, most recently
from
7eecde8 to
cfaf229
Compare
I used Stackprof for this optimization of CRuby documents generation
with a patch below.
--- a/exe/rdoc
+++ b/exe/rdoc
@@ -16,8 +16,11 @@ end
require 'rdoc/rdoc'
begin
- r = RDoc::RDoc.new
- r.document ARGV
+ require 'stackprof'
+ StackProf.run(mode: :cpu, out: 'stackprof.dump') do
+ r = RDoc::RDoc.new
+ r.document ARGV
+ end
rescue Errno::ENOSPC
$stderr.puts 'Ran out of space creating documentation'
$stderr.puts
Before this commit, RDoc::TopLevel#text? uses processing time about
19.3%.
~/rdoc$ stackprof stackprof.dump --text --limit 5
==================================
Mode: cpu(1000)
Samples: 54814 (0.47% miss rate)
GC: 9022 (16.46%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
10579 (19.3%) 10579 (19.3%) RDoc::TopLevel#text?
9022 (16.5%) 9022 (16.5%) (garbage collection)
21611 (39.4%) 2503 (4.6%) RDoc::Generator::Darkfish#generate_class
3151 (5.7%) 1482 (2.7%) Ripper::Lexer#lex
1271 (2.3%) 1271 (2.3%) Ripper::Lexer::State#initialize
After this commit, processing time of RDoc::TopLevel#text? is reduced to
6.8%.
~/rdoc$ stackprof stackprof.dump --text --limit 5
==================================
Mode: cpu(1000)
Samples: 42712 (0.47% miss rate)
GC: 6066 (14.20%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
6066 (14.2%) 6066 (14.2%) (garbage collection)
2892 (6.8%) 2892 (6.8%) RDoc::TopLevel#text?
14059 (32.9%) 2379 (5.6%) RDoc::Generator::Darkfish#generate_class
3015 (7.1%) 1389 (3.3%) Ripper::Lexer#lex
1265 (3.0%) 1265 (3.0%) Ripper::Lexer::State#initialize
Therefore, this commit speeds up RDoc by 9.9%.
I'm using Stackprof for a performance check of CRuby documents
generation.
Before this commit, RDoc::TopLevel#text? uses processing time about
6.8%.
~/rdoc$ stackprof stackprof.dump --text --limit 5
==================================
Mode: cpu(1000)
Samples: 42712 (0.47% miss rate)
GC: 6066 (14.20%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
6066 (14.2%) 6066 (14.2%) (garbage collection)
2892 (6.8%) 2892 (6.8%) RDoc::TopLevel#text?
14059 (32.9%) 2379 (5.6%) RDoc::Generator::Darkfish#generate_class
3015 (7.1%) 1389 (3.3%) Ripper::Lexer#lex
1265 (3.0%) 1265 (3.0%) Ripper::Lexer::State#initialize
After this commit, processing time of RDoc::TopLevel#text? is reduced to
0.1%.
~/rdoc$ stackprof stackprof.dump --method 'RDoc::TopLevel#text?'
RDoc::TopLevel#text? (/home/aycabta/rdoc/lib/rdoc/top_level.rb:274)
samples: 20 self (0.1%) / 20 total (0.1%)
callers:
17 ( 85.0%) RDoc::Generator::Darkfish#generate_page
2 ( 10.0%) RDoc::Store#add_file
1 ( 5.0%) RDoc::Store#complete
I'm using Stackprof for a performance check of Rails documents
generation.
Before this commit, RDoc::Text#wrap uses processing time about 12.7%.
~/rdoc$ stackprof stackprof-rails.dump --text --limit 3:
==================================
Mode: cpu(1000)
Samples: 94637 (1.54% miss rate)
GC: 18742 (19.80%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
18742 (19.8%) 18742 (19.8%) (garbage collection)
12022 (12.7%) 12022 (12.7%) RDoc::Text#wrap
17468 (18.5%) 6836 (7.2%) RDoc::Generator::Darkfish#generate_class
After this commit, processing time of RDoc::Text#wrap is reduced to
0.0%.
$ stackprof stackprof-rails.dump --method 'RDoc::Text#wrap'
RDoc::Text#wrap (/home/aycabta/rdoc/lib/rdoc/text.rb:269)
samples: 9 self (0.0%) / 9 total (0.0%)
callers:
The wrap processing is just for readability of HTML, but it's too heavy
and unnecessary in the present day.
I'm using Stackprof for a performance check of Rails documents
generation.
Before this commit, RDoc::Text#wrap uses processing time about
0.8%.
~/rdoc$ stackprof stackprof-rails.dump
==================================
Mode: cpu(1000)
Samples: 68079 (0.92% miss rate)
GC: 15749 (23.13%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
15749 (23.1%) 15749 (23.1%) (garbage collection)
14045 (20.6%) 6002 (8.8%) RDoc::Generator::Darkfish#generate_class
6107 (9.0%) 5962 (8.8%) RDoc::Markup::AttributeManager#convert_specials
(snip)
2955 (4.3%) 541 (0.8%) RDoc::Parser::RubyTools#skip_tkspace
After this commit, processing time of RDoc::Text#wrap is reduced to
0.5%.
~/rdoc$ stackprof stackprof-rails.dump
==================================
Mode: cpu(1000)
Samples: 66262 (0.79% miss rate)
GC: 15673 (23.65%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
15673 (23.7%) 15673 (23.7%) (garbage collection)
13878 (20.9%) 5836 (8.8%) RDoc::Generator::Darkfish#generate_class
5624 (8.5%) 5489 (8.3%) RDoc::Markup::AttributeManager#convert_specials
(snip)
2178 (3.3%) 316 (0.5%) RDoc::Parser::RubyTools#skip_tkspace_without_nl
I'm using Stackprof for a performance check of Rails documents
generation.
I checked methods below:
- RDoc::Parser::RipperStateLex#get_squashed_tk
- RDoc::Parser::RipperStateLex#initialize
- RDoc::Parser::RipperStateLex::InnerStateLex#on_default
Before this commit, these use processing time about 19.6%.
aycabta@x270:~/rdoc$ stackprof stackprof-rails-before.dump --limit 1
==================================
Mode: cpu(1000)
Samples: 66894 (0.83% miss rate)
GC: 15700 (23.47%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
15700 (23.5%) 15700 (23.5%) (garbage collection)
aycabta@x270:~/rdoc$ stackprof stackprof-rails-before.dump | \
grep 'RipperStateLex'
1727 (2.6%) 1472 (2.2%) (snip)RipperStateLex#get_squashed_tk
9352 (14.0%) 1281 (1.9%) (snip)RipperStateLex#initialize
2018 (3.0%) 475 (0.7%) (snip)RipperStateLex::InnerStateLex#on_default
After this commit, processing time is reduced to 15.0%.
aycabta@x270:~/rdoc$ stackprof stackprof-rails-after.dump --limit 1
==================================
Mode: cpu(1000)
Samples: 61020 (1.01% miss rate)
GC: 13824 (22.65%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
13824 (22.7%) 13824 (22.7%) (garbage collection)
aycabta@x270:~/rdoc$ stackprof stackprof-rails-after.dump --limit 0 | \
grep '#get_squashed_tk\|RipperStateLex#initialize\|#on_default'
735 (1.2%) 533 (0.9%) (snip)RipperStateLex::InnerStateLex#on_default
760 (1.2%) 517 (0.8%) (snip)RipperStateLex#get_squashed_tk
7663 (12.6%) 1 (0.0%) (snip)RipperStateLex#initialize
aycabta
force-pushed
the
optimize-hot-spot
branch
from
cfaf229 to
2b96a8b
Compare
aycabta
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.