Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(emx2): spark to javalin migration #4165

Merged
merged 37 commits into from
Oct 8, 2024
Merged

chore(emx2): spark to javalin migration #4165

merged 37 commits into from
Oct 8, 2024

Conversation

harmbrugge
Copy link
Member

@harmbrugge harmbrugge commented Aug 26, 2024 •
edited
Loading

What are the main changes you did:

  • Migrated the API code from Spark to Javalin

how to test:
Please test all features of emx2 to see if everything behaves as expected. Everything should work as before.

todo:

  • migrate api code to javalin

@harmbrugge harmbrugge marked this pull request as draft August 26, 2024 19:58
@harmbrugge
changed api classes to javalin code
d8455df
@harmbrugge
different paths for jars
3e07554
@harmbrugge
added needed graphql mappings
f968f7f
@harmbrugge
Added css and static routes
34fadfa
@harmbrugge
Updated beacon api
4e0bb47
@harmbrugge
updated tests
e25f66e
@harmbrugge
handling of static files
d3de6cf
@harmbrugge
fix: correct return types in zip api
7a5cbcf
@harmbrugge
updated beacon test with javalin context
f5f0781
@harmbrugge
Merge branch 'refs/heads/master' into chore/spark-to-javalin
149a6ec 
# Conflicts:
#	backend/molgenis-emx2-fairdatapoint/src/main/java/org/molgenis/emx2/fairdatapoint/FAIRDataPointCatalog.java
#	backend/molgenis-emx2-fairdatapoint/src/main/java/org/molgenis/emx2/fairdatapoint/FAIRDataPointDistribution.java
#	backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/CsvApi.java
#	backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/ExcelApi.java
#	backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/GroupPathMapper.java
#	backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/MolgenisWebservice.java
#	backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/RDFApi.java
#	backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/TaskApi.java
#	backend/molgenis-emx2-webapi/src/main/java/org/molgenis/emx2/web/ZipApi.java
@harmbrugge
updated tests
080896d
@harmbrugge
migration to javalin
15f00d8
@harmbrugge
spark to javalin migration
5b0d426
@harmbrugge
spark to javalin migration
27735dd
@harmbrugge
spark to javalin migration
19d2c80
@harmbrugge
spark to javalin migration
ecbac76
mswertz
mswertz approved these changes Sep 30, 2024
View reviewed changes
Copy link
Member

@mswertz mswertz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good.
I tested:

  • catalogue app
  • simple form and explorer using pet store
  • set theme to use another color
  • viewed a sql report

connoratrug
connoratrug previously requested changes Sep 30, 2024
View reviewed changes
Copy link
Contributor

@connoratrug connoratrug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go to https://preview-emx2-pr-4165.dev.molgenis.org/apps/ui/#/
select a schema
reload page: 'File not found' error
same issue for sub pages

@harmbrugge
Copy link
Member Author

go to https://preview-emx2-pr-4165.dev.molgenis.org/apps/ui/#/ select a schema reload page: 'File not found' error same issue for sub pages

Should be fixed now

@connoratrug connoratrug dismissed their stale review October 3, 2024 14:52

issue has been fixed

connoratrug
connoratrug reviewed Oct 3, 2024
View reviewed changes
backend/molgenis-emx2-webapi/src/test/java/org/molgenis/emx2/web/StaticFileMapperTest.java Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice :)

@dtroelofsprins
Copy link
Contributor

Did some functional tests: Successful:

  • Create Directory schema from template including demo data
  • Schema editor
  • Use DataCatalogueTemplate
  • Fair data hub template
  • Data upload (zipfile with several csv's) using UI
  • Other templates => tested several, but not all
  • Schedule a script
  • Failure email when script fails => NOT POSSIBLE on preview server

Fixed:

  • Directory App => footer is broken, pictures not shown (f.e. logo)

  • Manually adding data

  • Manually updating data

  • Create ERN Dashboard from template when going to the schema a white screen is shown, should go to /tables/#

  • Run a script on the server

  • Run a script locally using the EMX2-pyclient =>

    • create schema using template WORKS
    • downloading real BBMRI-ERIC biobanks

New issues (probably not tested before):

  • Directory App => Load json settings does not have the proper layout
  • CatalogueTemplate => SSR Catalogue shows white screen => Might also be due to recent changes to the catalogue that it's not working properly

From these two new issues, the first one, loading the JSON settings in the Directory App is still not working OK.
Second thing: It seems the refback fix that Morris merged last week is not included in here? As creating a schema with Directory template takes a whole lot longer than it currently does in production.

esthervanenckevort
esthervanenckevort previously requested changes Oct 4, 2024
View reviewed changes
Copy link
Member

@esthervanenckevort esthervanenckevort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Should update with master and resolve merge conflicts.
  • There is some inconsistency in using static function call vs instance method calls, but I don't see this as a blocking issue since the code does work and part of that issue was already in the existing code base.

Directory app works
Catalogue (old app) gives error. Probably not related to this PR, but to the data model changes
SSR catalogue seems to work fine
Table explorer, editing of data and downloads in different formats work

@harmbrugge harmbrugge dismissed stale reviews from esthervanenckevort and dtroelofsprins October 8, 2024 08:50

Master is merged as requested

@sonarcloud SonarCloud
Copy link

sonarcloud bot commented Oct 8, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
80.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

@harmbrugge harmbrugge merged commit 0d30372 into master Oct 8, 2024
5 of 6 checks passed
@harmbrugge harmbrugge deleted the chore/spark-to-javalin branch October 8, 2024 10:03
This was referenced Oct 21, 2024
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@connoratrug connoratrug connoratrug left review comments

@mswertz mswertz mswertz approved these changes

@dtroelofsprins dtroelofsprins dtroelofsprins left review comments

@esthervanenckevort esthervanenckevort esthervanenckevort left review comments

@BrendaHijmans BrendaHijmans Awaiting requested review from BrendaHijmans

@Fernananas Fernananas Awaiting requested review from Fernananas

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@harmbrugge @dtroelofsprins @mswertz @esthervanenckevort @connoratrug