Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a formula evaluator to replace TFormula calls #11516

Merged
merged 1 commit into from
Sep 30, 2015

Conversation

Dr15Jones
Copy link
Contributor

Created a reco::FormulaEvaluator class which can parse a subset of
the TFormula expressions. The parsing and function evaluation are
done using stateless classes so are thread safe and extremely thread
efficient. The formula parser is shared by all reco::FormulaEvaluators
to save memory and since the parser is stateless it can efficiently
be called by multiple threads simultaneously.

Created a reco::FormulaEvaluator class which can parse a subset of
the TFormula expressions. The parsing and function evaluation are
done using stateless classes so are thread safe and extremely thread
efficient. The formula parser is shared by all reco::FormulaEvaluators
to save memory and since the parser is stateless it can efficiently
be called by multiple threads simultaneously.
@cmsbuild
Copy link
Contributor

A new Pull Request was created by @Dr15Jones (Chris Jones) for CMSSW_7_6_X.

Added a formula evaluator to replace TFormula calls

It involves the following packages:

CommonTools/Utils

@cmsbuild, @cvuosalo, @vadler, @monttj, @slava77 can you please review it and eventually sign? Thanks.
@makortel this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

@Dr15Jones
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

The tests are being triggered in jenkins.

@Dr15Jones
Copy link
Contributor Author

My intentions is to try this in SimpleJetCorrector to avoid the serious serialization problem it is having with TFormula when we use ROOT 6.02. The serialization problem is gone when we use ROOT 6.04 but we still pay a large startup penalty in time and memory with the use of cling.

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

{

public:
enum class Precidence {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it Precedence or my poor English?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just because I'm a native speaker doesn't mean my English is any good :)

@slava77
Copy link
Contributor

slava77 commented Sep 28, 2015

+1

for #11516 61a724b

  • new code added to parse TFormula syntax and call relevant functions. Unit tests completion suggest the code integrity is fine.
  • jenkins tests pass: static analysis is clean, no differences as expected

@Dr15Jones
Copy link
Contributor Author

With this change added on top of the changes to ROOT and the pull request #11469 I am getting significantly improved performance on an Tier 0 like job when using 8 threads. I'm working on getting the precise measurements now but the short job I ran for VTune showed no major stalls or slowdowns due to unwanted synchronization.

@lgray
Copy link
Contributor

lgray commented Sep 29, 2015

@Dr15Jones I think I will start anew from something like this (or a hybrid of what already exists) for the StringCutObjectParser.

@lgray
Copy link
Contributor

lgray commented Sep 29, 2015

@Dr15Jones Wouldn't it be possible to cover N input args (even zero) using variadic templates? Can poke at that if needed.

@Dr15Jones
Copy link
Contributor Author

From a threading stand point the fix I did to cache the function pointer created by cling pretty much solved the StringCutObjectParser efficiency problem.

@Dr15Jones
Copy link
Contributor Author

The reason I could get away with doing a hand parser and not use boost spirit is everything was a double and the syntax was much simpler.

@lgray
Copy link
Contributor

lgray commented Sep 29, 2015

@Dr15Jones Hmm, should I continue with that project then? boost::spirit isn't the memory hog, it's really clang, which I think we will keep around any way due to simply reading our root data.

@lgray
Copy link
Contributor

lgray commented Sep 29, 2015

@Dr15Jones Probably a better use of my time is completely getting rid of TMVA (where possible) for BDTs.

@Dr15Jones
Copy link
Contributor Author

Removing the use of TMVA is probably better since the cut parser inefficiency seems to be under control.

davidlange6 added a commit that referenced this pull request Sep 30, 2015
Added a formula evaluator to replace TFormula calls
@davidlange6 davidlange6 merged commit 4140f59 into cms-sw:CMSSW_7_6_X Sep 30, 2015
@Dr15Jones Dr15Jones deleted the formulaEvaluator_7_6 branch October 19, 2015 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants