Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PeekPokeAPI: include source location on failed expect() calls. (backport #4144) #4149

Merged
merged 3 commits into from
Jun 6, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jun 6, 2024

Fixes #4128. (cc @jackkoenig)

Some notes:

  • b3fca5e adds the SourceInfo collection and reporting in the exception.
    • I haven't tried stack trace collection. Other than simply not getting to it yet, I don't think I have a rigorous enough setup (or a large enough testcase, which I have a working build going for) to meaningfully assess the performance cost.
    • Minor API design question: do we ever expect testableData.expect (with expected, encode and buildMessage arguments) to be called by a test user directly? If not, I have a slight preference for making the sourceInfo argument non-implicit there, and only having it implicit in the user-facing calls on SimulationData.
  • 2d58c7e adds a baseline testcase that expect() actually does anything. This isn't specific to this work.
  • In f507170, I try a very simple refactor of the source line collection in ErrorLog, and use it to embellish the exception.
    • It's not very pretty — I just nyonk getErrorLineInFile over to ExceptionHelpers and mark it private[chisel3] so it doesn't become a new public API to maintain. We have to take sourceRoots as an argument.
    • I couldn't actually find a meaningful way to collect source roots at the point of calling expect() (no Builder context, etc., so I'm not sure how to get at the ChiselOptions for a given module, if it's even possible), so we just use none, which defaults to Seq(new File(".")). This works fine for Chisel's own test cases and mine in my testing, but maybe there's more complicated setups out there.
    • I also add some very primitive and somewhat brittle checks to the test case added in the previous commit.

I'm not emotionally wed to any of this, so do feel free to suggest other ways for it to be done, or to rework it yourself if that's less overall work! (Particularly thinking of the refactor here, as it's not the cleanest.)

It does work nicely, though:

[info] UARTSpec:
[info] UART
[info] - should receive a byte *** FAILED ***
[info]   chisel3.simulator.PeekPokeAPI$FailedExpectationException: Failed Expectation: Observed value '1' != 0. Expectation failed: observed value 1 != 0 @[src/test/scala/ee/hrzn/athena/uart/UARTSpec.scala 64:29]
[info]         c.io.rx.valid.expect(false.B)
[info]                             ^
[info]   at chisel3.simulator.PeekPokeAPI$testableData.$anonfun$expect$9(PeekPokeAPI.scala:157)
[info]   at chisel3.simulator.PeekPokeAPI$testableData.$anonfun$expect$9$adapted(PeekPokeAPI.scala:147)
[info]   at svsim.Simulation$Port$$anonfun$check$1.applyOrElse(Simulation.scala:434)
[info]   at svsim.Simulation$Port$$anonfun$check$1.applyOrElse(Simulation.scala:432)
[info]   at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
[info]   at svsim.Simulation$Controller.$anonfun$completeInFlightCommands$1(Simulation.scala:203)
[info]   at svsim.Simulation$Controller.$anonfun$completeInFlightCommands$1$adapted(Simulation.scala:200)
[info]   at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
[info]   at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
[info]   at scala.collection.AbstractIterable.foreach(Iterable.scala:935)
[info]   ...

Contributor Checklist

  • Did you add Scaladoc to every public function/method?
  • Did you add at least one test demonstrating the PR?
  • Did you delete any extraneous printlns/debugging code?
  • Did you specify the type of improvement?
  • Did you add appropriate documentation in docs/src?
    • The only relevant documentation appears to be the "Migrating from ChiselTest" page, but there's not anything to add here really.
  • Did you request a desired merge strategy?
  • Did you add text to be included in the Release Notes for this change?

Type of Improvement

  • Feature (or new API)
    • -ish. Simulator debugging info improvement.
  • Internal or build-related (includes code refactoring/cleanup)

Desired Merge Strategy

  • Rebase: You will rebase the PR onto master and it will be merged with a merge commit.

Release Notes

SimulationData.expect calls now record source location and report it in the FailedExpectationException on failure.

Reviewer Checklist (only modified by reviewer)

  • Did you add the appropriate labels? (Select the most appropriate one based on the "Type of Improvement")
  • Did you mark the proper milestone (Bug fix: 3.6.x, 5.x, or 6.x depending on impact, API modification or big change: 7.0)?
  • Did you review?
  • Did you check whether all relevant Contributor checkboxes have been checked?
  • Did you do one of the following when ready to merge:
    • Squash: You/ the contributor Enable auto-merge (squash), clean up the commit message, and label with Please Merge.
    • Merge: Ensure that contributor has cleaned up their commit history, then merge with Create a merge commit.

This is an automatic backport of pull request #4144 done by [Mergify](https://mergify.com).

* simulator: add SourceInfo to expect calls and report.
* simulator: add test for failed expects.
* simulator: attempt to extract source line.
* simulator: make testableData.expect's sourceInfo parameter explicit.
* simulator: add factory method for giving failed expect sourceInfo/extraContext.

(cherry picked from commit 45dd82a)

# Conflicts:
#	src/test/scala/chiselTests/simulator/SimulatorSpec.scala
@mergify mergify bot added Backport Automated backport, please consider for minor release bp-conflict labels Jun 6, 2024
Copy link
Contributor Author

mergify bot commented Jun 6, 2024

Cherry-pick of 45dd82a has failed:

On branch mergify/bp/6.x/pr-4144
Your branch is up to date with 'origin/6.x'.

You are currently cherry-picking commit 45dd82a70.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   core/src/main/scala/chisel3/internal/Error.scala
	modified:   src/main/scala/chisel3/simulator/PeekPokeAPI.scala

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   src/test/scala/chiselTests/simulator/SimulatorSpec.scala

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@github-actions github-actions bot added the Feature New feature, will be included in release notes label Jun 6, 2024
@chiselbot chiselbot merged commit 26ae5a8 into 6.x Jun 6, 2024
15 checks passed
@chiselbot chiselbot deleted the mergify/bp/6.x/pr-4144 branch June 6, 2024 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backport Automated backport, please consider for minor release Feature New feature, will be included in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants