Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

2021 Cylc Meetings #139

Closed
hjoliver opened this issue Dec 7, 2021 · 17 comments
Closed

2021 Cylc Meetings #139

hjoliver opened this issue Dec 7, 2021 · 17 comments
Milestone
Meetings 2021

Comments

@hjoliver
Copy link
Member

hjoliver commented Dec 7, 2021

Jan 2021

Cylc 7.8.7/7.9.2 released

  • deployed at MO and NIWA, no probs

Event-driven datastore PR: almost done

rose suite-run migration:

  • cylc install, reinstall, run1/2/3 dirs, rose-suite.conf plugin
  • cylc clean
  • done by end of Jan, if not before

UI

  • editable mutations
    • only thing missing is testing
  • delta-driven gscan
    • need to check workflow "unregister" comes through as a delta
    • DS: workflow status flicker hopefully fixed by event-driven PR
    • if WF started from UI, need to trigger an immediate scan (config 5 sec delay at the moment)
  • message triggers (almost done?)
    • choose ellipsis demo, with mods
  • hierarchical gscan
    • a bit longer, needs deltas
    • (needs more to the datastream? DS)

TODO for cylc-8.0b0 (by end of Feb):

UI

  • finish existing PRs

UIS

  • minimal mutation-driven services: start stopped flows
  • (authorization, if time)

Scheduler

  • finish existing PRs

  • finish task state rationalization: queued as xtrigger (plus some trivial renaming?)

    • queued as xtrigger
    • (need to consider behaviour of multiple queues ... global queues)
    • (ditch the trivial changes)

  • (delta-driven TUI? - nope; beta-1+)

Docs?

  • HO to consider minimal requirements for beta?

Other:

  • JR has time on Cylc 8 now 🎉
@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

Feb 2021

Cylc 7?

  • no problems?

Cylc 8

Admin

  • applying for project funding extension next FY
  • possible 10 week internship at NIWA (CS Masters student)
  • other?

Done, or as good as, since last meeting

  • editable mutations
    • just tests to do
  • cylc install, cylc reinstall, cylc play, cylc clean ( and rose-suite.conf plugin)
  • event-driven datastore
  • removed obsolete queued task state (and centralize queue logic)
    • revisit sorted queues etc. before 8.0
  • other?

Outstanding problems and issues

  • display task output messages (reactivity?) Display message triggers cylc-ui#530
    • added delta contains no outputs, so no initial object to subsequently be reactive
  • deltas in gscan Use deltas in GScan cylc-ui#543
  • hierarchical gscan (blocked by item above) Hierarchical GScan cylc-ui#502
  • UIS: start stopped flows
    • DS to look at
  • rose-suite-run plugin slightly broken on re-install (TP on it)
  • intelligent host select not working (needs fixing soon, but post-beta is fine)
  • 5 most recent failed tasks gscan tooltip:
    • scheduler datastore to maintain a separate FIFO list/queue to update on submit-failed or failed events - DS
  • other?

TODO beta release

  • (finish all above)
  • Docs:
    • "What's new in Cylc 8"?
    • What's still missing, to be done by 8.0
    • Exactly how can the beta can be used (e.g. remote spawning not well tested yet?)
  • other? ("execute user scripts in a subshell"??)

Next meeting

  • hash out priorities and assign issues for 8.0, and for beta-1

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

March

Main topic: Cylc 8.0b0 TODO list

  • (updated)

Also:

  • what exactly should gscan (and its tooltips) display? (in Cylc 7 it is a summary of the current scheduler task pool, which makes less sense for Cylc 8)
    • show n=0 (active + unhandled failed) in icon list, 5-most-recent in tooltips
  • should we allow cylc play --re-run? (without cylc clean first ... probably not)
    • agreed not, stick with the plan to extend cylc clean to prep for in-place re-run
  • confirm what to do if the UI misses deltas or receives bad deltas
    • agreed: it should not happen, but if it does start from scratch is required. Freeze the UI, alert the user, and advise to reload the browser
  • queued state indicators in the UI occasionally not removed when the task is de-queued?
    • need to go back and investigate, it seemed to be a back-end problem
  • user scripts in a subshell
    • DM has approved, let's get it in for beta if possible
  • gscan deltas probs:
    • UIS sometimes sends out-of-order deltas that can't be applied, leaving incorrect state ("missing family node"...)
    • delta representation of non-running workflow transitions (registered, deleted)
    • OS noticed some data incorrectly appearing in the "added" deltas?

"Question" labeled issues:

Beta release timeline:

  • end of March at latest, frequent updates thereafter
  • get delta gscan in if possible (decide by early next week)

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

April

Cylc 7

Problems?

Cylc 8

UI

Packaging

  • proposal: meta-package proposal: meta package cylc-flow#4158
    • use conda environment.yml files (examples of different recipes for different site needs: spawners, authenticators etc.)
    • may need to rename cylc-flow back to cylc, but think through the implications! (before 8.0!!)
    • (conda outputs PR done)
  • close Packaging end-game issue?
    Packaging end-game. #27

Documentation

  • things that don't work as expected, e.g. cylc hold
    • 2-column comparison of how to do X in Cylc 7
  • terminology: time to completely excise "suite"
  • see also glossary (OS) on cylc-doc

8.0b1 release?

  • content: minimal authorisation, ...?
    • (cylc install; logging of shutdown)
  • Priorities pre-8.0.0:
    1. bug fixes
    2. deployment testing
    3. cross-user and authorization - fine-grained via graphql middle-ware; in-progress MH
    4. performance testing
    5. basic graph and static vis
    6. edit-run (CLI and UI) - prob broadcast
    7. documentation
    8. task hold!

Other

  • GH actions DNS problems
    • 2 different DNS configs for GHA runners, only one is compat with cylc! (socket.gethostbyname()??) - (also problem with MacOS - the way that python socket lib binds to the BSD system socket module) ... can we stop using that, post platforms - OS to take a look.

Next meeting

  • 1st Wednesday of May?

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

May

Cylc 8 beta deployment delays at our sites

  • NIWA:

    • PAM: "In order to use PAM you need root privileges to configure the PAM module you’re targeting to login and we’d need to modify images and automation to make the changes persist." (Or ... use the existing JupyterHub instance on the system)
    • Integration with existing JupyterHub instance: "I think we can set up
      multiple spawners with https://github.com/jupyterhub/wrapspawner there’s an issue about specifying cmd, but no examples."

  • Met Office:

    • can't use conda (glibc old)
    • installing on Monsoon (same platform, internet access)
    • quite reliant on pre-compiled binary versions of some libraries
    • post to Discourse
    • (manylinux1 check is in now, with Python 3.7)
    • OS: distributed spawner - UIS is currently assuming hub is on localhost?
    • DM: want to switch on J-Hub internal SSL, needs support in UIS?

  • (BOM?)

Work in progress

  • authorization (MH): anything we need to discuss here?
    • read, modify, execute? (modify is dangerous)
  • task hold
    • (trigger etc. is ok for ghost tasks)
    • (support globs because it's more general than hold)
    • (what about trigger foo.*??)
    • (solution: globs ONLY apply to n=0)
  • runhead pool -> runhead-limited indicator flag
    • (consider representing as a queue - cycle point priority queue)
    • (but: user sets it with "runahead limit")
  • xtriggered indicator flag, as interim measure? (before xtriggers: tree view and beyond cylc-ui#331)
    • (multiple flags?)
    • (maybe we can go straight to the full solution? - xtriggers in schema, including status? - DS to follow-up)
  • intelligent host select (TP)
    • approaching POC (a week ... or more?)
  • "expanded workflow status deltas": looks like DS has nailed this, but we need to review it expanded workflow status deltas cylc-uiserver#212
    • OS: doesn't deal with the added deltas yet (DS: when workflow goes from running to stopped; or installed, or deleted; OS: for testing, to see the generated deltas, can we fake the hub?)
    • DS to look at the seemingly spurious workflow status added deltas showing up
  • UI: 9 PRs in progress, most are fairly simple, except for gscan deltas & hierarchical
  • UIS: load balanced spawner (OS)

TODO

Other

  • (Check on scheduler reload, we get a new "initial data burst" - check for the reload flag in the delta)

  • Altair update

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021 •
edited
Loading

June

Sites

  • any new Cylc 8 issues?
    • possible gscan-caused UI "stability issues", waiting for gscan deltas
    • tasks appearing beyond end of workflow
    • DM concerns about partially-satisfied prereqs
    • ignore pre-initial for start-tasks

Quick summaries of what we've each been working on, main issues and problems, etc.

MH

  • authorization
    • can we resolve remaining questions now?
      • access group names and mapping to commands and queries
        • agreed on CONTROL name
        • agreed to NOT allow per-workflow permissions
        • JR: timed-out privileges would be desirable
        • BK: J-hub has concept of inactivity (kill on inactivity)
        • MO Ops: too easy to hit the pause button
      • behaviour if config file permission are too loose
      • implementation options A, B, or C??
        • Option B: middleware! ... just config to go; what to do in the UI for unauthorised actions; tidying, maybe by end of month
        • DM: less essential than it was because of J-Server

BK

  • gscan deltas
    • up for review now: OS and HO to do
  • merging subscriptions
    • issue going up on combining fragments

DS

  • workflow status deltas: done?
  • multiple subscriptions per request issue: done?
  • ...

TP

  • intelligent host select
    • PR up, failing on some GH tests, not failing locally, very nearly there (needed for 8.0, otherwise degraded from rose host-select)
  • cylc review status
    • suite to workflow, nearly ready to go in
    • one outstanding issue, but it's bodgeable

OS

  • load balanced spawner
    done! but UIS can't use because it has to be on the hub host! .. which will be achieved by jupyter server
  • jupyter server
    run Jupyter lab and Cylc UIS under J-server, with exist auth ... couple of PRs to J-server. Launch UIS without going via the Hub (and opens browser tab)... for standalone (hub-less) use! ... J-S used to be part of Jupyter lab; solve issues like support for internal SSL, because J-S supports that; only catch is you don't get multi-user for this
  • universal ID
    • RD to look at centralising parsing of the workflow arg, as a prereq to this

DM

  • (ICM Cylc support?)
  • ...

RD

  • task hold
    -done - awaiting final review
  • targeted clean
    • safety if user symlinks present, TBD

HO

  • done: remove runahead pool, expose runahead limiting (c.f. queued, held; note not done with xtriggered)
  • done: start from specified tasks (SoD)
  • some smaller stuff e.g.:
    • done: fix manual triggering of queued tasks (SoD)
    • done: inherited parameter env fix
    • NIWA end of FY Cylc and UMP project reporting and start of FY proposals 😬

8.0.0 release

  • end of June?
    • DM: end of Sept at the latest for 8.0 pushing out to new users; end of Feb for default version, production ready
  • exactly what do we need to finish first?

AOB?

What should HO present the UM Workshop next week?

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

July

Apologies:

  • DS

Cylc 7 next release

  • minor bug fix(es) - GUI and cylc review
  • need to avoid messing with Cylc 8 run-dirs

Cylc 8.0b2

Cylc 8.0

What's still TBD?

  • BUG: cylc reinstall nukes scheduler work dir
    • MH: easy to fix (skip log, work, share)
  • optional/expected outputs
  • intelligent host select (~done?)
    • still in review with OS
    • (w/o platform from platform groups - follow-up)
  • authorisation (~done?)
    • documentation added (auto-gen)
    • ready for review!
    • OS fixing an error on his JupyterServer branch
  • integer flow labels (~done)
    • (HO almost review-able)
  • infer run number (~done)
    • ready for review
  • UI: hierarchical gscan (~done)
    • ready for review
    • possible performance improvements as follow-up
    • sorting bug pending review (separate fix PR)
  • UI: table view (~done?)
    • missing multi-jobs
    • and sorting
    • and columns not configurable
  • a bunch of ~smaller PRs
  • anything else?
  • JupyterServer changes will probably be good to go!
    • new release of J-S thanks to OS
  • Universal ID?
    • not a breaking change
    • CLI implementation easy, pushing it throughout the system
  • Graph view is 8.x
  • Static graph view essential??
    • tide us over: sed, cylc graph --ref, dot => PNG. About an hour's work!
  • RELEASE DATE?
    • "September hard deadline"? (If possible!) If not, a lot of beta users with heavy use
  • documentation!
    • review issues
    • Quick Start Guide: simple as possible

AOB?

  • UMP contract
  • "semi-absolute triggers"? (Yikes) Support semi-absolute offset triggers? cylc-flow#4316
  • post-8.0 stuff in progress
    • GraphQL subscriptions at the scheduler
  • next meeting?
    • DM are from 18 August
    • ~Tuesday 17th August, same time
  • JR: a week in August to do a POC, at BOM
  • MO has some include/exclude-at-startup suites, is SOS-specific. Is impossible to support with SoD? Need to obsolete it, or say "unsupported" and point to a migration page. Probably easily down with "optional outputs". Note an 8.0 blocker
  • put links to migration guide in certain error messages

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

August

Sites

  • any new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

BK

  • Learn about Jupyter Server and review Oliver's PR's
    • (also corrects deficiencies, e.g. internal SSL, distributed UISs, which we won't have to reinvent ourselves; has a magic switch to go between single user and hub use - for using J-hub on the front multi-user)
    • OS have contributed a change to J-server
  • Review Mel's Authorization PR's
    • ready to go in
    • OS J-server to go in first then MH to rebase
    • Let's get J-server in quickly
  • Trying to start using milestones for UI, focusing on 0.6 and avoiding adding more things, or work on 1.0 issues
  • 0.6
    • Table view
      • (priority)
    • Propagate status in the GScan hierarchy
    • hunt down any tracebacks and errors we might see in the UI
    • TODO test UI master memory footprint

OS

  • 8.0b2 release.
    • auto-release actions worked great
    • now have unix builds incl. mac os
    • TODO: can we delete the earlier meta-package releases? and turn the meta package into a stub?
  • Jupyter Server work
  • Trying and failing to keep up with reviewing

RD

  • Centralised reg parsing & infer run number
    ready to go in: one-stop for CLI, infers runN
  • Conditional prerequisite bug
    • in progress
  • Cylc 7
    • better handling of Cylc 8 workflows
      • stops Cylc messing with Cylc 8 workflows
    • store cycle point TZ in DB
      • (back port)
    • TODO NEED A Cylc 7 RELEASE FOR THIS - HO

HO

  • required and optional outputs
  • integer flow numbers (already discussed, but not finished off and reviewed)

MH

  • Finished Auth work
  • greyed-out UI mutations for not-authorized
  • Review Jupyter Server
  • Getting Auth working with Jupyter Server

TP

  • IHS From Platforms
    • ready to go
  • Cylc Rose Fixes
    • Work with list, validate, graph, config
    • Template var handling bug
  • Documentation - Various
  • Cylc Report timings
    • Cylc 7: Cause mysterious (probably pandas version related), PR up.
    • Cylc 8: Most obvious failure merged. Ticket raised to look for other failures.
    • We have at least one user at MO.
  • Warnif StopCP > FinalCP

8.0.0 release

  • In September
  • Next UIS release should be all the multi-user multi-server functionality
    • another beta release with J-server

What's still TBD?

Discuss

AOB?

Open Questions (any time left over)

https://github.com/issues?q=is%3Aopen+is%3Aissue+archived%3Afalse+label%3Aquestion+org%3Acylc

Next videoconference

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

09 September

Sites

Quick summaries of what we've each been working on, main issues and problems, etc.

BK

OS

  • Reviewing.
  • Reviewing.

RD

  • Reviewing (mainly optional outputs)
  • cylc clean - docs + improve logging
    • PR coming
  • A couple of bugfixes

HO

  • Optional outputs Optional outputs cylc-flow#4343
    • new approach to family triggers and output "optionality"
    • refactored workflow timeout code (there are 3 timeouts now)
    • ongoing management of conflicts with the integer flow label branch 🤢
  • Some reviewing and merging

MH

  • Continuing Auth work
    working with Jupyter Server - good to go; and the associated UI PR
    • (and reviewing auth in general, with OS)
  • Reviewing

TP

8.0b3

8.0.0 release

  • In September (which is, this month 😱 )
    • 🎉 hashtag yesplease
  • Next UIS release should be all the multi-user multi-server functionality
    • another beta release with J-server

What's still TBD?

cylc-flow

cylc-ui

cylc-uiserver

Discuss

Lets concentrate 100% on:

  • get current PRs merged

  • bump all issues that can reasonably be bumped

  • hammer the system to find any problems

  • full docs revision

  • "Release candidate" - once major features are frozen (once current PRs are in)

  • UI post 1.0 can do a continuous release process to get new features in

NOTE: - one more Cylc 7 release required, for cylc review compat

AND: final beta release with Jup-Server, optional outputs, and distributed spawner...

AOB?

Open Questions

Next videoconference

  • meetings every 2 weeks from now, for those can attend, every second one can be smaller

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

22 September

Sites

  • any new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

BK

OS

RD

HO

  • UM Partnership contract - can BK be replaced 😭
  • presenting Cylc 8 to NIWA HPC and Data Science management and teams
  • "optional outputs" merged to master
  • reviewing and testing ...

MH

  • Finalising auth ui-server work (mainly fixing tests and refactoring)
  • Adding tests and responding to feedback on ui disabling unauthorised buttons
  • Couple of reviews
  • Met Office related admin

TP
Done:

  • Documentation
  • Holiday
  • Make booleans line continuation chars in Graph language
    Waiting:
  • Reviews of platform from platform group

TODO

Cylc 7 - one more release required, for cylc review compat

cylc-flow

cylc-ui

cylc-uiserver

  • Next UIS release should be all the multi-user multi-server functionality

    • another beta release with J-server

  • 0.6 milestone

  • 1.0 milestone

Priorities

Lets concentrate 100% on:

  • (avoid new developments as far as possible?)
  • get current PRs merged
  • bump all issues that can reasonably be bumped
  • hammer the system to find problems
  • full docs revision

AOB?

Open Questions

Next videoconference

  • meetings every 2 weeks from now, for those can attend, every second one can be smaller

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

08 October

Apologies

David Sutherland, (DM & MH 30 mins late)

Sites

  • any new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

BK

OS

  • Catching up.
  • Reviewing.

RD

HO

  • (various reviews)
  • (4 days leave)
  • finished off integer flow labels and fixes related to triggering tasks beyond the scheduler window
  • discussions with NIWA and UM Partnership management on backfilling Bruno's position

MH

  • Reviewing
  • Finishing Auth Documentation
  • Responding to review comments.

TP
Done

  • Platforms from Platform groups now in!
  • Prevent recursive installation
  • Polling bug
  • Assorted bugfixes/documentation/reviewing
    Priorities
  • Reviewing as asked
  • Closing my own PRS
  • Add checks to Rose 1 to prevent it damaging installed Cylc workflows

TODO

cylc-flow

cylc-ui

cylc-uiserver

  • Next UIS release should be all the multi-user multi-server functionality

    • another beta release with J-server

  • 0.6 milestone

  • 1.0 milestone

Meeting Notes

  • beta-3 release: let's get authorization merged
  • mutations bug on start-up (mutations menu does not appear) - problem with XSRF token, see the Issue, BK to look
  • UIS memory leak? middleware objects get created but never destroyed.Are we using Graphene middleware properly? One middleware object per request, so less of a problem now that we have persistent subscriptions, but still needs fixing (still have auth requests etc.). OS: may be able to reuse the same object each time.
  • BK: will post reminders for others to take over pypi and conda admin before he leaves
  • Vuetify is blocking migration to Vue 3, which would allow us to drop vuex, for simpler code and better reactivity. Vuetify itself to be updated for Vue3 but has been delayed, now ETA Q1 2022.
  • OS and BK: need to complete the job of purely datastore-driven views (currently still need initial burst deltas for constructing the view)
  • gscan performance is an issue with many workflows running - frequent updates all of which cause DOM reflow. This is worse on the "propagate states" work but performance not great on master either. BK: can we have the UIS construct the tree and send the result to the UI?
  • HO: will meet with new guy AC early next week

AOB?

  • HO: demote preparing from task state to badged waiting?

Open Questions

Next videoconference

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

21 October

Apologies

  • (BK)

Sites

Any new Cylc 8 issues?

  • no

Quick summaries of what we've each been working on, main issues and problems, etc.

OS

  • reviewing!

RD

HO

  • lots of PR reviewing and merging
  • investigating cross-user hub
  • UMP contract (truncated "tranche 3" + "tranche 4")
  • lining up the BK -> AC handover

MH

  • Authorization follow-up questions to discuss
  • Reviewing
  • UI Familiarisation
  • Auth docs

TP
Done

  • CYLC_W_NAME => CYLC_W_ID [and lots of follow up]
  • 2 Small UI changes (mostly training)
  • Assorted documentation & small fixes

In progress

  • Differentiate between Cylc config not set and config not a config.
  • Avoid recursive installation
  • Stop Rose 2019.x fiddling with Cylc 8 installs.
  • Cylc 8 timezone default == UTC

DS

  • upgraded graphql-ws in the UIS: -160/+16 code lines 👍
  • understands the delta processing bug and has a fix in the works
    • will need quick reviewing once the PR is up

TODO

cylc-flow

cylc-ui

cylc-uiserver

  • Next UIS release should be all the multi-user multi-server functionality

    • another beta release with J-server

  • 0.6 milestone

  • 1.0 milestone

AOB?

New team member AC has been indoctrinated by BK. First cylc-ui issues to start on?

  • A few of easier reviews and changes?
  • Tableview + finish datastore-driven view framework

Reminder of BK's final docs and notes:

Open Questions

Next videoconference

  • ~2 weeks, day time NZ?
  • let's go back to Element VC, not Teams (we are an open source project :-)

Meeting Notes

  • HO: UIS spawning by hub requires J-Hub "admin user" status, or target UIS running already

    • upcoming changes in J-Hub and J-server may help us address this without modifying the hub code
    • try a custom hub service (that can be permitted to do just what we need)?
    • not a super high priority right now because single-user use is now much easier
    • worth posting a question to the JupyterX forum?

  • MH working on the xsrf issue, then authorization config validation

  • beta-3 release by end of next week if possible. Needed:

    • DS delta processing fix
    • MH cylc-ui PR for greying out unauthorized operations
    • MH to look at the mutations not loaded issue too
    • (all cylc-ui 0.6.0 issues if should be done)

  • but note many of us are taking leave during this time: NZ 4-day weekend, most UK people taking several days to a week off

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

02 November

Sites

  • any shocking new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

OS

  • Reviewing.
  • Preparing task state and related fixes.
  • Rose CLI: Bash -> Python

RD

HO

  • project admin: UM P "tranche 3" contract executed (and invoiced) at NIWA, "tranche 4" in the works
  • AC induction
  • UI triggering and "set-outputs" bug fix PRs
  • reviewing
  • testing:
    • "preparing" job state problem (re)discovered
    • delta log jam in the UI?

MH

  • Holiday
  • Reviewing
  • _xsrf token problem
  • continued follow up auth in ui and docs work

TP

Outstanding assigned issues

TODO

cylc-flow

cylc-ui

cylc-uiserver

  • Next UIS release should be all the multi-user multi-server functionality

    • another beta release with J-server

  • 0.6 milestone

  • 1.0 milestone

Priorities

AOB?

Open Questions

Next videoconference

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

18 November

Any important new issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

OS

  • 8.0b3 release
  • Rose Python CLI: Bash -> Python

RD

HO

  • mostly: marathon doc upgrade for Cylc 8

Questions:

  • aborting a retry sequence in Cylc 8, without cylc reset
  • (SoD with cycle offset errors in the graph?)

MH

Questions:

  • Scrap configurability of comms timeout? comms timeout cylc-flow#4112,
    • bump config issue to 8.x; issue is must be configure on remote platforms as
      well, but that's OK
    • leave the --comms-timeout CLI option, it actually works
  • Are we happy for other users' ui-servers, available in pulldown menu in UI, to be configurable? user-profile: add authorised-users field cylc-uiserver#273
    • yes (user decides who else's UIS they want look at if authorized)
    • also need access to other hubs (probably another browser tab)

AC

  • multiples views driven by the UI shared datastore alone
  • tableview PR: let's get is merged now; further improvements as follow-up

TP

Still in progress

  • Reviewing Oliver's Rose CLI → Python change.
  • No recursive install - waiting on Ronnie's finessing
  • Only Run Cylc Rose once - nearly done
  • Cylc 7 Bug

Done

  • Fixed UIS bug for Dave and min weird groups config (probably a common issue here)
  • Review of Config Spec internal docs - painful, but less painful than I feared.
  • Messy CYLC_WORKFLOW_NAME/ID follow up 😞

Questions

Can we review where we are at with the wrapper script changes?

DS

TODO

8.0b3 release

  • new conda outputs new documenting
  • cylc-uiserver dependent on graphql-ws, BK and HO are maintainers of the conda-forge package. We're two releases behind (need merge). Should ping the pypi package owners to see if they are willing to take it on
  • need to update admin release process docs with graphql-ws

cylc-flow

cylc-ui

  • 1.0 milestone
  • UI post 1.0 can do a continuous release process to get new features in

cylc-uiserver

Priorities

Open RC1 Questions

https://github.com/cylc/cylc-flow/issues?q=is%3Aissue+is%3Aopen+label%3Aquestion+milestone%3Acylc-8.0rc1

AOB?

Open Questions

Next videoconference

  • 30 Nov: 8.0r1 issue triage; and decide if we release in December.

@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021

30 November

Sites

  • any new Cylc 8 issues?

TODO

Particular issues (?):

cylc-flow

  • final beta release with Jupyter Server, optional outputs, and distributed spawner...

  • "Release candidate 1" - once major features are frozen (once current PRs are in)

  • 8.0rc1 issues

cylc-ui

  • 1.0 milestone

  • UI post 1.0 can do a continuous release process to get new features in

cylc-uiserver

  • Next UIS release should be all the multi-user multi-server functionality

    • another beta release with J-server

  • 1.0 milestone

Open Questions

Open Questions

Priorities

(What are your priorities?)
(Are there any blockers you need to discuss?)

OS

  • Maintenance, admin & boring stuff.
  • Universal ID

RD

  • In progress: client/server + GraphQL refactor to reduce unexpected traceback: cylc-flow#4529
  • Upcoming: cylc install should respect max scan depth: cylc-flow#4363

HO

  • (off sick for a few days 😬 ... well, not really "off", but sick)
  • bulk docs upgrade for Cylc 8 ... then beta3 release announcement

MH

TP

  • Cylc Tutorial/Resources
  • Rose issues marked rc1. Functional first, then documentation.

DS

AC

  • Ditch the filter button
  • Table view merged (driven by shared store)
  • Table view new PR
    • Expand jobs
    • filter toolbar pinned

AOB?

Next videoconference

@hjoliver hjoliver mentioned this issue Dec 7, 2021
Closed
@hjoliver
Copy link
Member Author

hjoliver commented Dec 7, 2021 •
edited by oliver-sanders
Loading

8 December Cylc-UI Meeting

1. Table View (and general UI) MVP for 8.0rc1

2. Begin prioritizing further UI development

OS from Element chat:

Ok, here's my attempt at triage. We can probably agree a path forward before the meeting, have a skim through, see what issues you think should be higher up the pecking order.

I think the main priority right now is to get what we have working properly and fill in the obvious functionality gaps to make it usable. My picks:

  • Addressing user feedback (will be plenty before long).
  • Multi-user functionality.
  • Finishing off GScan, Tree & Table views (mutations, polish, performance).
  • Finishing off the data model (& any changes to the view interface).
  • Improving the mutation editor.
  • Implement the view toolbar.
  • Preserve user's tabs.

Then we can start implementing the more ambitious features e.g:

  • Graph view.
  • Offline data.
  • N-Window.

I've been through the UI issues (wrote up a few things that were missing).
Here's a quick (and scary) overview of the main things:

3. Conclusions

Immediate priorities:

Longer term priorities:

  • Graph view.
  • Offline data.
  • N-Window.
  • Any data model improvements that are required.

@wxtim
Copy link
Member

wxtim commented Dec 15, 2021 •
edited by hjoliver
Loading

16/17 December (UK/NZ respectively)

Christmas Leave/Apologies?

  • RD, DS, AC

Any new Cylc 8 issues?

Work Done/Priorities / Blockers

DS - on leave

OS

  • Universal ID

RD - on leave

HO

  • Deployed and documented 8.0b3 for users on NIWA HPC
    • used and adapted wrapper from TP's umerged PR
  • Reviewing

MH

TP

AC - on leave

TODO

can we release 8.0rc1 before end-of-year?

Possible blockers:

  • universal ID
  • task prerequisites outside beyond n=0
  • (change in hub/server config file path)

Decision: release rc1 in early January. It should have universal ID, task prereqs for the UI, and fixed tutorials

extant issues

cylc-flow

cylc-ui

cylc-uiserver

AOB?

Open Questions

  • HO to check the old rose symlink config issue - does NIWA still need it? DM says no way to do it with Cylc 8 at the moment.

Next video conference

  • mid Jan

@hjoliver hjoliver added this to the Meetings 2021 milestone Jan 16, 2022
@hjoliver
Copy link
Member Author

EOY

@hjoliver hjoliver mentioned this issue Jan 16, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Meetings 2021
Development

No branches or pull requests
2 participants
@hjoliver @wxtim