3.1.0 upgrade with data migration #271
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
in org model spec
org spec failing with no abbrev for some reason.
Used to be that org had to have an abbrev. That stopped the Super Admin creating the org so was removed. Remove check from this test.
…admins-unable-to-add-new-organisations DCC Issue 679 - Fix for issue that Super Admins unable to create new
… Org. Fixes issues: https://github.com/DigitalCurationCentre/DMPonline-Service/issues/592 https://github.com/DigitalCurationCentre/DMPonline-Service/issues/645 https://github.com/DigitalCurationCentre/DMPonline-Service/issues/641 The reason we are seeing deleted (de-activated) Plans is that there is no filter for plans with Roles active in the Org model's org_admin_plans() method. Change added .where(roles: { active: true }) to filter plans in org_admin_plans() method of Org model.
…g_admins' of github.com:DMPRoadmap/roadmap into bug_dcc_592_and_645_deleted-private_plans_visible_to_org_admins
…5_deleted-private_plans_visible_to_org_admins Bug DCC Issues 592, 645 - Fix for Org Admins seeing deleted Plans for…
…hub.com:DMPRoadmap/roadmap into bug_dcc_674_csv_download_plans_fails_for_admins
…wnload_plans_fails_for_admins Issue DCC-674: Fix for broken CSV Download noticed because a nil
pagination of user's plans broken.
Fix for issue DMPRoadmap#3069 and DCC issue
https://github.com/DigitalCurationCentre/DMPonline-Service/issues/675.
Changes:
- Replaced view files /paginable/plans/org_admin_other_user with /paginable/plans/_index.html.erb
with extra checks for plan.owner.present? as missing plan.owner broke a DCC user's plans being render by org_admin.
- Replaced partial with '/paginable/plans/org_admin_other_user' with '/paginable/plans/index',
replaced action'org_admin_other_user' with 'index' in paginable_renderiser() method in views
/org_admin/users/edit.html.erb. /org_admin/users/plans.html.erb and /super_admin/users/edit.html.erb.
- In Paginable::PlansController replaced -
# GET /paginable/plans/org_admin/:page with # GET /paginable/users/:id/plans
associated with method org_admin_other_user() renamed index()
- Routes replaced
get "org_admin_other_user/:page", action: :org_admin_other_user,
on: :collection, as: :org_admin_other_user
# Paginable actions for users
resources :users, only: [] do
get "index/:page", action: :index, on: :collection, as: :index
with
resources :users, only: %i[index] do
member do
resources :plans, only: %(index)
end
end
…75_paged_results_on_plans_org_admin_other_user_breaks Issue#3069 - (DCC Issue 675) - Org Admin and Super Admin searches and
form for Plan based on whether email and/or name are required. Fix for DCC bug DMPRoadmap#2983 Changes: - Added two new properties to config/initializers/_dmproadmap.rb (both devault to false): config.x.application.require_contributor_name = false config.x.application.require_contributor_email = false -In Contributor model the private method name_or_email_presence() has been updated to take account of the above mentioned property settings.
… of github.com:DMPRoadmap/roadmap into bug_DCC2983-possible_to_add_contributor_without_a_name
…le_to_add_contributor_without_a_name Issue#dcc2983 - Fix for preventing adding a Contributor via Contributor
#trival resolve issue308 by restore title and description public methods
add static page for 3.1.0 upgrade
Fix memory usage issue in translation sync process
pengyin-shan
changed the title
lagoan
approved these changes
Feb 21, 2023
There was a problem hiding this comment.
This has been a big upgrade. Thank you very much for your hard work @pengyin-shan
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This release includes changes in the 3.0.5, 3.0.6, and 3.1.0 versions from main codebase
Due to the large changes in the 3.1.0 version, I used all remote code from the
appandspecfolders, then reverse back to reflect DMP Assitant changes by doing through all GitHub issues. The status of issues can be found in issues.xlsx.Data migration rake tasks
mysql_to_postgresandrewrite_postgresare in the task folder. I documented the process for UAT database migration in the second tab of the excel file. Neil has tested the data migration process on staging (using a different approach than UAT).rubocop.ymlis updated to be the same as the one in the main codebase. I finished most "double to single quote" transfers, but there may still be some left for the 3.1.0 upgrade. Plus, rubocop fixes are done. The status of rubocop can be found in my document Rubocop change documentation #264All new issues arising in the migration process (e.g. translation sync issue, unmanaged org issue...) have been resolved, and I added them in the release notes below.
Release notes, including the main codebase side changes and DMP Assistant side changes:
Added
In project detail page, users can now specify a primary research domain for their project (e.g. Biology, Computer Science, Humanities, etc.). This information appears in the DMP's JSON as a 'tag'
In project detail page, users can now indicate and describe any ethical concerns that concern their research data. Clicking the checkbox will display a field to describe the concerns and a URL field to link to a report. This information appears in the DMP's JSON.
Added a new 'Research Outputs' tab that allows users to specify information about their intended research outputs #2738. User can specify the type (e.g. dataset, software, image, etc.), name and a description of the output, size (if applicable), whether or not it contains sensitive info or PII, what the initial access level will be (e.g. open, restricted, closed) and the anticipated publication date:
Allowed user to select an appropriate license in Research Outputs tab (note that the options come from the SPDX license registry) #2607
Allowed user to any metadata standards that will be followed (e.g. Dublin Core) in Research Outputs tab (note that the list of standards comes from the RDA Metadata standards catalog) #2798
Allowed user to select the repositories they intend to preserve the object in (e.g. Dryad, Zenodo, GitHub, or an institutional repository) in Research Outputs tab (note that the list of repositories comes from the re3data registry) #2605
Added research outputs result to downloaded PDF file in a table format #2812
Updated APIs to add research outputs result the downloaded JSON file, appears as 'Dataset' information
Added helpdesk email address field to
Organisation's details. This email will be used in the email signature for emails sent from the system (assuming the recipient of the email is a member of the same Organisation) #3140Added data migration rake tasks and migrated DMP Assistant database from MariaDB 10 to PostgreSQL 12 to better accommodate JSON format data
Sandbox testing flag can be turned on/off so org admin can change locally hosted DMP Assistant to a sandbox testing environment
Added rubocop test and code side support #264
Updated rubocop setting to disable the debug checking after upgrading to rubocop 1.45 #316
Added more clear error message to remind user not to fresh the page when 'saving plan' process is stuck and success message is not shown as expected #241
Added static page for server upgrading time #310
Make comments/guidances area collapsible to gives the researcher more space to see their answers #3123
Changed
DMP Assistant migrated all data from MariaDB 10 to PostgreSQL 12
Updated all Gems and JS dependencies
Used excel sheet to track upgrading process #187
Updated th french translations for Portage template for systematic reviews #269
Adjusted Dockerfile.Production to move
rails assets:precomilestep todocker-compose.ymlfile #282Relived memory load in translation syncing process by adjusting configuration so that only code in
appfolder will be scanner and translated #313Adjusted customized translation.io source code to add
unscopedmethod for necessary table to fit PostgreSQL syntax (Improve translation speed by apply unscope to only necessary table ualbertalib/translation_io_rails#2)Fixed
Reversed
perm_helper.rbchanges to reduce translation.io sync memory pressure and added rubocop exception fortranslation.rb#315Unmanaged organization will not show in all org selector except the creating new organization page #260
Allowed contributors to choose organization #273
Allowed admin to see unmanaged org name when creating new org to avoid duplication #275
Force Github actions using Ubuntu 20.04 to fix wkhtmltopdf-binary issue #266
Clarified of String and Array behavior for plan exports #268
Fixed the error that
app.potheaders accidentals appear in the empty text box of writing plans area #308Fixed for pagination of plans retrieved using the REST API V0 #3105
Security vulnerability patches #3100, #3097, #3093, #3077
Fixed issues/inconsistencies with policies #3084 #3099
Fix for issue downloading PDFs #3054#3049 #3098
Fix for issue displaying users as collaborators on a plan after they have removed themselves Fix for issue downloading PDFs #3095
Fix for API v0 not supplying the owner email address Fix for issue downloading PDFs #3094
Fix for bug preventing Super Admins from creating new Orgs #3091
Fix for issue causing deleted plans to appear in the Org Admin list of plans #3087
Fix for CSV downloads #3085#3075
Fix for pagination and search on the Admin plans page #3069 #3073
Fix for strings that were not included in translation content #3081#3050
Fix for issue with contributor org logic not adhering to the restrict_orgs config flag #3078#3060
Fix for adding contributors #3071
Removed phone number field from contributor page #3067
Fixed issue with merging users #3065
Fixed issue with emails #3052
Fix for users unable to see API page #3047
Upgraded Rubocop and updated all files accordingly #3048 #3045
Fixed an issue that was preventing Org Admins from seeing the 'Share' tab #3131
Fixed an issue that was causing errors when Org Admins tried to save changes to a template #3071
Patched an issue that continued to show users as collaborators on a Plan's Share tab even after they had removed themselves from the Plan #3126
Fixed an issue that was causing TinyMCE editors to have an incorrect size when they initially load #3141
Various fixes for the APIs
Patched issue that was always using the default template when creating a new DMP in API v1 #3137
Fixed an issue that was preventing plans from being created via API v0 #3135
Addressed an issue that was preventing users from being unassigned from a department in API v0 #3132
Reverted change to our use of string format tokens that was made during an upgrade of Rubocop. The preferred format is once again Some %{variable_name} text #3138
Fixed some potential PDF downloading problems