Skip to content
This repository has been archived by the owner on Apr 1, 2022. It is now read-only.

Expose the status of currently-running tasks #239

Merged
merged 32 commits into from
May 24, 2021
Merged

Expose the status of currently-running tasks #239

merged 32 commits into from
May 24, 2021

Conversation

cnr
Copy link
Contributor

@cnr cnr commented May 18, 2021

Fixes https://github.com/fossas/team-analysis/issues/418 . Details available in that ticket

The user-visible changes for this include:

  • The Diagnostics context stack is visible at the bottom of the terminal for each running task (when the terminal is ANSI-compatible)
  • --debug mode (very) noisily outputs each context change for each task as discrete log messages (this works even when the terminal is not ANSI-compatible)
  • Each running task displays a unique integer ID
  • Each analyzer has context added for:
    • Project discovery
    • Project analysis
      • the type of analysis; dynamic/static; and specific analysis strategy names as appropriate
      • for complex analyzers (e.g. the golang analyzers that attempt to use a command for deep dependency analysis), it should be clear from the context what step is currently running in the analyzer
  • Exec
    • has context for running commands
  • ReadFS
    • has context for parsing various types of files
    • has context for walking the filetree

This PR also addresses some tech debt:

  • Sticky regions in the console should work better now on all platforms, but especially windows: we're using the concurrent-output package for this
  • Migrates other entrypoints to use logWithExit helper
  • Adds convenient string conversion functions in Data.String.Conversion
    • This eliminates our use of the unsafe decodeUtf8 functions and replaces them with decodeUtf8With lenientDecode
  • Eliminate ResultBundle from Diagnostics, as the warnings from the bundle were never actually used (and it just made for a worse dev experience)

Copy link
Contributor

@skilly-lily skilly-lily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I think the approach is great and I'm happy with almost all of the changes.
I am still working on the full in-depth review, but I had some initial feedback about the API for non-diagnostic stickies, and a comment about docs for diagnostic contexts

src/App/Fossa/API/BuildWait.hs Outdated Show resolved Hide resolved
src/Strategy/Gomodules.hs Show resolved Hide resolved
Copy link
Contributor

@skilly-lily skilly-lily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more nits, but I'm satisfied that the only major concern is the sticky API comment I left first.

Once some of my questions are answered, I can be more confident that I do understand each part, and I can probably test and approve quickly after that.

Awesome work! Super creative, idiomatic in a lot of ways, really good API decisions, and solid code.

src/Control/Effect/Diagnostics.hs Show resolved Hide resolved
src/App/Pathfinder/Scan.hs Outdated Show resolved Hide resolved
src/Data/String/Conversion.hs Show resolved Hide resolved
src/Data/String/Conversion.hs Show resolved Hide resolved
-- - warnings from outside of the error boundary do not impact the FailureBundles produced by the action
--
-- This returns a FailureBundle if the action failed; otherwise returns the result of the action
errorBoundary :: Has Diagnostics sig m => m a -> m (Either FailureBundle a)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[no action, maybe?]: We probably need a guideline to decide when this is appropriate. Originally, the only way to "catch" was to use runDiagnostics or recover, and this provides another option. I think we should document or at least agree when each should/shouldn't be used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I'll put together some docs. errorBoundary is primarily a hack, and is otherwise analogous to runDiagnostics.

errorBoundary is a workaround to allow us to send the Diagnostics constraint upward, so we can intercept context calls from the inner action. We use this for stickyDiag specifically

src/Strategy/Gradle.hs Show resolved Hide resolved
src/Strategy/Cocoapods.hs Outdated Show resolved Hide resolved
src/Strategy/Maven.hs Show resolved Hide resolved
src/Control/Carrier/Fresh.hs Outdated Show resolved Hide resolved
src/Control/Carrier/Diagnostics/StickyContext.hs Outdated Show resolved Hide resolved
Using several incurs a minor performance overhead
@cnr cnr requested a review from skilly-lily May 24, 2021 16:05
Copy link
Contributor

@skilly-lily skilly-lily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One future concern and a few nits, but this LGTM!
giphub

src/App/Fossa/VPS/Report.hs Outdated Show resolved Hide resolved
@@ -136,16 +137,18 @@ uploadBuildGraphChunk url httpOptions targets = do
_ <- sendIO $ runDiagnostics $ runHTTP $ req POST url (ReqBodyJson jsonChunk) ignoreResponse httpOptions
pure ()

updateProgress :: Has Logger sig m => Progress -> m ()
updateProgress :: Has StickyLogger sig m => Progress -> m ()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[future work]: Should we move this to the TaskPool modules? I see it everywhere, and I don't think there are any changes between implementations.

src/Control/Effect/Diagnostics.hs Outdated Show resolved Hide resolved
src/Effect/Logger.hs Outdated Show resolved Hide resolved
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants