Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.0.0a1 #1523

Merged
merged 83 commits into from
Jul 1, 2020
Merged

3.0.0a1 #1523

merged 83 commits into from
Jul 1, 2020

Conversation

ColdHeat
Copy link
Member

@ColdHeat ColdHeat commented Jun 30, 2020

Alpha release of CTFd v3.

3.0.0a1 / 2020-07-01

General

  • CTFd is now Python 3 only
  • Render markdown with the CommonMark spec provided by cmarkgfm
  • Render markdown stripped of any malicious JavaScript or HTML.
    • This is a significant change from previous versions of CTFd where any HTML content from an admin was considered safe.
  • Inject Config, User, Team, Session, and Plugin globals into Jinja
  • User sessions no longer store any user-specific attributes.
    • Sessions only store the user's ID, CSRF nonce, and an hmac of the user's password
    • This allows for session invalidation on password changes
  • The user facing side of CTFd now has user and team searching
  • GeoIP support now available for converting IP addresses to guessed countries

Admin Panel

  • Use EasyMDE as an improved description/text editor for Markdown enabled fields.
  • Media Library button now integrated into EasyMDE enabled fields
  • VueJS now used as the underlying implementation for the Media Library
  • Fix setting theme color in Admin Panel
  • Green outline border has been removed from the Admin Panel

API

  • Significant overhauls in API documentation provided by Swagger UI and Swagger json
  • Make almost all API endpoints provide filtering and searching capabilities
  • Change GET /api/v1/config/<config_key> to return structured data according to ConfigSchema

Themes

  • Themes now have access to the Configs global which provides wrapped access to get_config.
    • For example, {{ Configs.ctf_name }} instead of get_ctf_name() or get_config('ctf_name')
  • Themes must now specify a challenge.html which control how a challenge should look.
  • The main library for charts has been changed from Plotly to Apache ECharts.
  • Forms have been moved into wtforms for easier form rendering inside of Jinja.
    • From Jinja you can access forms via the Forms global i.e. {{ Forms }}
    • This allows theme developers to more easily re-use a form without having to copy-paste HTML.
  • Themes can now provide a theme settings JSON blob which can be injected into the theme with {{ Configs.theme_settings }}
  • Core theme now includes the challenge ID in location hash identifiers to always refer the right challenge despite duplicate names

Plugins

  • Challenge plugins have changed in structure to better allow integration with themes and prevent obtrusive Javascript/XSS.
    • Challenge rendering now uses challenge.html from the provided theme.
    • Accessing the challenge view content is now provided by /api/v1/challenges/<challenge_id> in the view section. This allows for HTML to be properly sanitized and rendered by the server allowing CTFd to remove client side Jinja rendering.
    • challenge.html now specifies what's required and what's rendered by the theme. This allows the challenge plugin to avoid having to deal with aspects of the challenge besides the description and input.
    • A more complete migration guide will be provided when CTFd v3 leaves beta
  • Display current attempt count in challenge view when max attempts is enabled
  • get_standings(), get_team_stanadings(), get_user_standings() now has a fields keyword argument that allows for specificying additional fields that SQLAlchemy should return when building the response set.
    • Useful for gathering additional data when building scoreboard pages
  • Flags can now control the message that is shown to the user by raising FlagException
  • Fix override_template() functionality

Deployment

  • Enable SQLAlchemy's pool_pre_ping by default to reduce the likelihood of database connection issues
  • Mailgun email settings are now deprecated. Admins should move to SMTP email settings instead.
  • Postgres is now considered a second class citizen in CTFd. It is tested against but not a main database backend. If you use Postgres, you are entirely on your own with regards to supporting CTFd.
  • Docker image now uses Debian instead of Alpine. See Switch from Alpine to python:3.7-slim-buster #1215 for rationale.
  • docker-compose.yml now uses a non-root user to connect to MySQL/MariaDB
  • config.py should no longer be editting for configuration, instead edit config.ini or the environment variables in docker-compose.yml

ColdHeat and others added 30 commits May 28, 2020 23:40
* Experimental ideas to make SQLAlchemy queries faster by switching them to SQLAlchemy Core
* Starts work on #1402
* Add `view=admin` GET param to `/api/v1/users`, `/api/v1/teams`, and `/api/v1/challenges` to bypass filtering for admins
* Closes #1423 #1445
* Related to #1165
* Update Flask-Migrate to 2.5.3
* Regenerate the `env.py` file for migrations and delete useless README file
* Supercedes #1447
* Added github actions workflow
* Switch from mistune to cmarkgfm
* Starts work on #1317
* Update markdown test to be commonmark compliant
* Remove Python 2 specific code
* Require imports to have a proper isort-supported order
* Only test/lint on Python 3
* Bump most dependencies to latest supported version
* Closes #1027 
* Default to using the non-root user in `docker-compose`
* Switch Dockerfile from alpine to debian. Switch entrypoint from sh to bash
* Closes #1215
* Replace user facing pagination with Flask SQLAlchemy Pagination objects
* Closes #1353

I think this is a big improvement but I feel like this is harder to create a theme construct around.
Start cleaning up a lot of the core theme. 
Extract pieces into components that can be included into overarching templates. 
Work on #1234
* Closes #1362
* Reduces the session object to just an id, nonce, and security hash
ColdHeat and others added 21 commits June 24, 2020 14:04
* Require that markdown files are linted by prettier
* Adds Media Library access to EasyMDE editor
* Chooses VueJS as front end framework for #1508 
* Closes #1320
* Adds User and Team globals to Jinja theme files. 
* Closes #1234
* Paginate only the `/api/v1/users`, `/api/v1/teams`, and `/api/v1/submissions` endpoints.
* Add a `PaginatedAPIListSuccessResponse` class with a customized `apidoc` method to hack in the pagination scheme
* Works on #1318
* Works on #1318 
* Adds searching and filtering to most of the bulk API endpoints
* Adds documentation on the GET parameters used to conduct searches
)

* Adds support for looking up countries by IP address. 
* Shows country and flag for a user/team's IP addresses in the admin panel
* Adds support for `GEOIP_DATABASE_PATH` in config.py
* Closes #1115
* Change `/api/v1/config/<config_key>` to return properly structured data
* Closes #1506
* Add account searching to the user facing side of CTFd
* Closes #1354
* Lint `Dockerfile` and `docker-compose.yml`
* Closes #1458
* Allows for configuration to be specified via `config.ini` instead of direct Python manipulation
* Adds type hints to `config.py` to make it clearer what each variable expects
* Integrates SQLAlchemy `pool_pre_ping` to check SQLAlchemy connections on checkout. 
* Closes #1509, #1438, #1395, #467
…#1522)

* Adds challenge ID into location hashes so that the right challenge is always loaded regardless of duplicate names
* Closes #1120
@codecov
Copy link

codecov bot commented Jun 30, 2020

Codecov Report

Merging #1523 into master will increase coverage by 0.20%.
The diff coverage is 91.89%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1523      +/-   ##
==========================================
+ Coverage   89.05%   89.25%   +0.20%     
==========================================
  Files         105      129      +24     
  Lines        5692     6554     +862     
==========================================
+ Hits         5069     5850     +781     
- Misses        623      704      +81     
Impacted Files Coverage Δ
CTFd/api/v1/scoreboard.py 95.71% <ø> (ø)
CTFd/constants/__init__.py 92.00% <ø> (ø)
CTFd/utils/security/auth.py 97.29% <ø> (-0.27%) ⬇️
CTFd/utils/humanize/words.py 25.00% <25.00%> (ø)
CTFd/constants/teams.py 50.00% <41.66%> (-50.00%) ⬇️
CTFd/constants/users.py 50.00% <41.66%> (-50.00%) ⬇️
CTFd/utils/user/__init__.py 78.15% <46.87%> (-12.38%) ⬇️
CTFd/admin/pages.py 79.31% <50.00%> (+0.73%) ⬆️
CTFd/utils/encoding/__init__.py 78.37% <57.89%> (-3.44%) ⬇️
CTFd/api/v1/helpers/models.py 66.66% <66.66%> (ø)
... and 103 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ca6270...d642ef0. Read the comment docs.

@ColdHeat ColdHeat merged commit adc70fb into master Jul 1, 2020
UnknownSilicon pushed a commit to UnknownSilicon/CTFd that referenced this pull request Oct 4, 2023
Alpha release of CTFd v3. 

# 3.0.0a1 / 2020-07-01

**General**

- CTFd is now Python 3 only
- Render markdown with the CommonMark spec provided by `cmarkgfm`
- Render markdown stripped of any malicious JavaScript or HTML.
  - This is a significant change from previous versions of CTFd where any HTML content from an admin was considered safe.
- Inject `Config`, `User`, `Team`, `Session`, and `Plugin` globals into Jinja
- User sessions no longer store any user-specific attributes.
  - Sessions only store the user's ID, CSRF nonce, and an hmac of the user's password
  - This allows for session invalidation on password changes
- The user facing side of CTFd now has user and team searching
- GeoIP support now available for converting IP addresses to guessed countries

**Admin Panel**

- Use EasyMDE as an improved description/text editor for Markdown enabled fields.
- Media Library button now integrated into EasyMDE enabled fields
- VueJS now used as the underlying implementation for the Media Library
- Fix setting theme color in Admin Panel
- Green outline border has been removed from the Admin Panel

**API**

- Significant overhauls in API documentation provided by Swagger UI and Swagger json
- Make almost all API endpoints provide filtering and searching capabilities
- Change `GET /api/v1/config/<config_key>` to return structured data according to ConfigSchema

**Themes**

- Themes now have access to the `Configs` global which provides wrapped access to `get_config`.
  - For example, `{{ Configs.ctf_name }}` instead of `get_ctf_name()` or `get_config('ctf_name')`
- Themes must now specify a `challenge.html` which control how a challenge should look.
- The main library for charts has been changed from Plotly to Apache ECharts.
- Forms have been moved into wtforms for easier form rendering inside of Jinja.
  - From Jinja you can access forms via the Forms global i.e. `{{ Forms }}`
  - This allows theme developers to more easily re-use a form without having to copy-paste HTML.
- Themes can now provide a theme settings JSON blob which can be injected into the theme with `{{ Configs.theme_settings }}`
- Core theme now includes the challenge ID in location hash identifiers to always refer the right challenge despite duplicate names

**Plugins**

- Challenge plugins have changed in structure to better allow integration with themes and prevent obtrusive Javascript/XSS.
  - Challenge rendering now uses `challenge.html` from the provided theme.
  - Accessing the challenge view content is now provided by `/api/v1/challenges/<challenge_id>` in the `view` section. This allows for HTML to be properly sanitized and rendered by the server allowing CTFd to remove client side Jinja rendering.
  - `challenge.html` now specifies what's required and what's rendered by the theme. This allows the challenge plugin to avoid having to deal with aspects of the challenge besides the description and input.
  - A more complete migration guide will be provided when CTFd v3 leaves beta
- Display current attempt count in challenge view when max attempts is enabled
- `get_standings()`, `get_team_stanadings()`, `get_user_standings()` now has a fields keyword argument that allows for specificying additional fields that SQLAlchemy should return when building the response set.
  - Useful for gathering additional data when building scoreboard pages
- Flags can now control the message that is shown to the user by raising `FlagException`
- Fix `override_template()` functionality

**Deployment**

- Enable SQLAlchemy's `pool_pre_ping` by default to reduce the likelihood of database connection issues
- Mailgun email settings are now deprecated. Admins should move to SMTP email settings instead.
- Postgres is now considered a second class citizen in CTFd. It is tested against but not a main database backend. If you use Postgres, you are entirely on your own with regards to supporting CTFd.
- Docker image now uses Debian instead of Alpine. See CTFd#1215 for rationale.
- `docker-compose.yml` now uses a non-root user to connect to MySQL/MariaDB
- `config.py` should no longer be editting for configuration, instead edit `config.ini` or the environment variables in `docker-compose.yml`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants