Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related issues: #227, #278
__
The problem:
contains
method inRange
has nooverride
modifier. So after upcastRange
toSeq[Int]
, usage ofcontains
cost usO(n)
(or evenO(n) * n
in total, since we callcontains
for each row). That means the bigger file is, the more rows file has the longer spark-excel will read the file.I found out this was fixed in Scala 2.13.3.