Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3.0.0rc1 #3071

Merged
merged 33 commits into from
Aug 7, 2020
Merged

v3.0.0rc1 #3071

merged 33 commits into from
Aug 7, 2020

Conversation

guolinke
Copy link
Collaborator

@StrikerRUS @jameslamb @Laurae2 @henry0312 @btrotta
please list PRs or issues that you think should be fixed in this new release.

@StrikerRUS
Copy link
Collaborator

StrikerRUS commented May 11, 2020

As 3.0 major release is a big step forward, I'd like to propose the following TODOs:

PRs:

New Features

Fixes:

@guolinke
Copy link
Collaborator Author

@StrikerRUS
I agree with you, except for the GPU bugs.
most of them are hard to reproduce and to fix. also ping @huanzhang12

@guolinke
Copy link
Collaborator Author

BTW, I am busy with the 'company work' recently.
so the new features assigned to me may delay, maybe we can move them to the next release (3.0.0rc2 or 3.0.0).
I will have more time after June.

@StrikerRUS
Copy link
Collaborator

StrikerRUS commented May 11, 2020

@guolinke

I agree with you, except for the GPU bugs.
most of them are hard to reproduce and to fix. also ping @huanzhang12

OK, maybe at least it worth to revisit them? A lot of them haven't got any updates for months.

BTW, I am busy with the 'company work' recently.
so the new features assigned to me may delay, maybe we can move them to the next release

Sure! Are you speaking mostly about #2884, right?
From new features section, I personally really would like to see #1493 in next release. I remember you said

I think it is easy to implement.
#1493 (comment)

@StrikerRUS
Copy link
Collaborator

StrikerRUS commented May 11, 2020

Also, I'd like to propose a release checklist which we can copy-paste from old releases and it will help us to not forget any required action. Not sure that it will be actual for RC updates though.

@guolinke
Copy link
Collaborator Author

refer to the version normalizing problem: pypa/setuptools#308

@henry0312
Copy link
Contributor

I think it's time to drop suuport of Python 2.7, since it has reached the end of life.
cf. https://www.python.org/doc/sunset-python-2/
If users that use Python 2.7, they can use LightGBM v2.x

@henry0312
Copy link
Contributor

And if you want, I'll create a PR to drop the support.

@StrikerRUS
Copy link
Collaborator

@henry0312

I think it's time to drop suuport of Python 2.7, since it has reached the end of life.

Please take a look at the following comment: #3056 (comment).
I propose that we mark 3.0.0 release as the latest one which supports Python 2, and stop supporting it with our next release (3.0.1 or 3.1.0 whatever will be the next after 3.0.0). I'm looking forward to hear from you what do you think about it!

@henry0312
Copy link
Contributor

@StrikerRUS
Returning to the previous discussion about Semantic Versioning (SemVer) in #2949, increment of major version can only breaks backward compatibility.
When we drop the support of Python 2.x, we will break backward compatibility because we maintain some (dirty) codes (like

if is_py3:
zip_ = zip
string_type = str
numeric_types = (int, float, bool)
integer_types = (int, )
range_ = range
) to support it.
Therefore, I think it is a just time to only support Python 3.x or later.
And I think as long as popular libraries (including numpy, scipy and pandas) support Python 2. the number of the users to use Python 2 won't decrease, that is not the future Python community hope.

@henry0312
Copy link
Contributor

So if we need support of Python 2.x (I don't need any more), I propose we will release both v2.4.x that will be the last version to support it and v3.x that will support Python 3.5 or later.

@jameslamb jameslamb closed this May 12, 2020
@jameslamb jameslamb reopened this May 12, 2020
@jameslamb
Copy link
Collaborator

Sorry, closing was an accident.

@jameslamb
Copy link
Collaborator

jameslamb commented May 12, 2020

I agree with @StrikerRUS 's suggestions in #3071 (comment).

#629 is my primary focus, and I do think we should try to finish it for 3.0.0. In my experience more than half of the issues we receive about the R package are related to installation challenges, so I want the 3.0.0 release to come with easier install for R users.

Improving installation

Improving documentation:

Simplifying the usage of the R package

I just created #3075. I think we should take the opportunity of a new major release to remove lgb.prepare() and lgb.prepare_rules() and move people only to lgb.prepare2() and lgb.prepare_rules2().

@StrikerRUS
Copy link
Collaborator

@henry0312

When we drop the support of Python 2.x, we will break backward compatibility because we maintain some (dirty) codes

That's true.

And I think as long as popular libraries (including numpy, scipy and pandas) support Python 2. the number of the users to use Python 2 won't decrease, that is not the future Python community hope.

All these libraries (our dependencies) are dropping or have already dropped Python 2 support.
pandas:

Officially Python 3.6.1 and above, 3.7, and 3.8.
https://pandas.pydata.org/pandas-docs/stable/getting_started/install.html#python-version-support

numpy: https://numpy.org/neps/nep-0014-dropping-python2.7-proposal.html
scipy : https://www.scipy.org/scipylib/faq.html#do-numpy-and-scipytill-support-python-2-7

Also, from the plot I provided earlier, it can be observed that number of our Python 2 users constantly decreasing starting from January 2020:

image

But still it is several thousand downloads per day for now (~2k). For example daily downloads for macOS are never greater 400, but it is not the reason to not support macOS and stop building wheels for it.

I propose we will release both v2.4.x that will be the last version to support it and v3.x that will support Python 3.5 or later.

What commits do you want to include in 2.4.x branch?

@guolinke
Copy link
Collaborator Author

guolinke commented Aug 6, 2020

due to the problems in R's version number, I think the release will delay for an additional day.

@guolinke
Copy link
Collaborator Author

guolinke commented Aug 6, 2020

BTW, this cause the CI error.

* checking if this is a source package ... WARNING
Subdirectory ‘src’ contains:
  VERSION.txt
These are unlikely file names for src files.

.github/workflows/main.yml Outdated Show resolved Hide resolved
.gitignore Outdated Show resolved Hide resolved
@guolinke
Copy link
Collaborator Author

guolinke commented Aug 6, 2020

@jameslamb I am trying to de-couple R with version.txt.
For this:

AC_INIT([lightgbm], [m4_esyscmd_s([cat src/VERSION.txt])], [], [lightgbm], [])

, is that possible to read the version number from Description?

@jameslamb
Copy link
Collaborator

@jameslamb I am trying to de-couple R with version.txt.
For this:

AC_INIT([lightgbm], [m4_esyscmd_s([cat src/VERSION.txt])], [], [lightgbm], [])

, is that possible to read the version number from Description?

I think reading from DESCRIPTION is too complicated. I don't want to do YAML processing inside that configure script.

The documentation on how to update configure is here: https://github.com/microsoft/LightGBM/blob/master/R-package/README.md#changing-the-cran-package

But I can update it and fix the other warnings you're seeing right now! Can I push to this branch?

@jameslamb
Copy link
Collaborator

@guolinke I just pushed 5d79e77 which should fix the other issues with the R package. That hard-coding is fine for right now to not delay the release. I will improve the automation in a follow-up PR.

@StrikerRUS
Copy link
Collaborator

@guolinke Will RCs be a part of our regular development process? I'm asking because 3.0 looks like an edge case as 3.0 will never be published on CRAN (R-package is not ready yet as per #629). So I believe we can set VERSION to 3.0, then manually postprocess names of files for PyPI (add rc suffix) and bump the version for development (as we do after each release) to 3.0.1.

@guolinke
Copy link
Collaborator Author

guolinke commented Aug 6, 2020

@StrikerRUS
Actually, I think the new solution for R version number is better.
In previous, it depends on both the number in DESCRIPTION and version.txt, and version.txt is mainly used to locate the generate package file in tests. So it is safe to decouple it.
And using 3.0.0-1 is more uniform with 3.0.0rc1. Refer to https://semver.org/#spec-item-9 .

@jameslamb in this case, Maybe we can generate the version number to DESCRIPTION, instead of manually update it?
I mean, auto-fill the version number (and date) to DESCRIPTION in recreate-configure.sh.
Not in this release, we can enhance this latter.

@guolinke
Copy link
Collaborator Author

guolinke commented Aug 6, 2020

@StrikerRUS @jameslamb
I will merge this in about 2 hours. Do we have any other problems to discuss?

@StrikerRUS
Copy link
Collaborator

@guolinke

I will merge this in about 2 hours.

OK!

Are you planning to go through all items from #3071 (comment) (except updating stable tag of course)? I think RC release to PyPI will be enough. Not sure about NuGet though.

@jameslamb
Copy link
Collaborator

I'm ok with a release any time. Trying to fix #3280 for it right now.

For R, after this release, we could do this:

  1. add a placeholder in DESCRIPTION like ~~VERSION~~
  2. in build_r.R, and build-cran-package.sh, replace that with the version in VERSION.txt (and similar approach for Date)
  3. add a similar placeholder to configure.ac, and have recreate-configure.sh and build-cran-package.sh replace it. That way, we never have to rely on having configure.ac read a file
  4. All this code that reads VERSION.txt can replace any rc with -

If we did all of that, it could be totally automated and the only thing that has to be changed is VERSION.txt.

And I think we could create a Makefile with a release recipe that handles all these things.

@guolinke guolinke merged commit 63c8287 into master Aug 7, 2020
@guolinke guolinke deleted the v3-release branch August 7, 2020 00:11
@guolinke
Copy link
Collaborator Author

guolinke commented Aug 7, 2020

@StrikerRUS very strange, it seems azure pipeline does not build for tag now.

updated:
now it is working after re-push.

@guolinke
Copy link
Collaborator Author

guolinke commented Aug 7, 2020

Thanks all contributors so much! The new version is released! 🍺🍺🍺

The python package is uploaded. you can install it via pip install lightgbm==3.0.0rc1.

For the NuGet package, I think we can release it in the 3.0.0 version.
@StrikerRUS how about the homebrew version? should we update it now or wait for 3.0.0?

@StrikerRUS
Copy link
Collaborator

StrikerRUS commented Aug 7, 2020

@guolinke

how about the homebrew version?

I think PyPI is enough for and homebrew can wait for the stable version.

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants