Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: add y-intercept for logistic regression #143

Merged
merged 9 commits into from
Jul 24, 2017

Conversation

facaiy
Copy link
Contributor

@facaiy facaiy commented Jul 14, 2017

Hi, the PR is aimed at adding intercept for logistic regression #142 .

The work is not done yet (mostly finished in fact), but feedback is necessary for me to determine whether to go further or not. Hence, the PR is opened.

Mathematical Derivation

Given:

  • linear model: f(x) = w^T x + b
  • loss function: L(y, f(x))

the derivative of b is: \frac{dL}{df}

Implementation

The PR is inspired by:

How to test

  • code review by expert.
  • Perhaps some tests are needed, however, more feedback is needed for me now.

@TAAAN
Copy link
Contributor

TAAAN commented Jul 14, 2017

Thanks for your PR.
When preprocessing train data, set the first dim 1 for every instance, then the first dim of w is b.

@TAAAN TAAAN requested review from andyyehoo and removed request for andyyehoo July 17, 2017 06:55
@andyyehoo
Copy link
Contributor

@facaiy it's a good idea. Just go on with this PR and finish it. We will review and merge it with certain modification.

@facaiy
Copy link
Contributor Author

facaiy commented Jul 18, 2017

OK. I'll ping you when finished.

@facaiy
Copy link
Contributor Author

facaiy commented Jul 21, 2017

The PR is tested on a4a dataset.

There are two questions:

  1. The package failed to submit its tasks to YARN. I believe the bug is introduced from the latest branch-1.0.0, instead of the PR. I will create a new issue to report it later.

  2. I picked those commits to commit 3ff4329825fd355118384d5be438dd132f1015c3, and everything is OK. However, the result of experiment on a4a dataset implies that AUC decreased when y-intercept is used.

use y-intercept iteration auc duration reg_l2 learn rate
No 10 0.8991570991465441 47s 0 4
Yes 10 0.898142663326485 50s 0 4

@facaiy facaiy changed the title WIP: ENH: add y-intercept for logistic regression ENH: add y-intercept for logistic regression Jul 21, 2017
@andyyehoo andyyehoo merged commit 252b02f into Angel-ML:branch-1.0.0 Jul 24, 2017
@facaiy
Copy link
Contributor Author

facaiy commented Jul 24, 2017

Many thanks, @andyyehoo @TAAAN .

@facaiy facaiy deleted the ENH/lr_support_intercept branch July 24, 2017 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants