Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track t3000 nightly regression #12986

Closed
SeanNijjar opened this issue Sep 23, 2024 · 2 comments · Fixed by #13164
Closed

Track t3000 nightly regression #12986

SeanNijjar opened this issue Sep 23, 2024 · 2 comments · Fixed by #13164
Assignees
Labels

Comments

@SeanNijjar
Copy link
Contributor

T3000 nightly has recently regressed. Issue tracks root causing the problem commit and resolving the issue. I suspect the issue is related to 4-chip reduce scatter tests where the 4 chosen chips do not form a ring and so error out when trying to identify receiver and sender chips.

Please reassign accordingly after problem commit has been identified.

@Aswinmcw
Copy link
Contributor

Aswinmcw commented Sep 23, 2024

Hi @cfjchu , after your PR (#12719), t3k nightly tests are failing due to device not connected issue, Can you look into it?
Failing pipeline (https://github.com/tenstorrent/tt-metal/actions/runs/10948889220)

@Aswinmcw Aswinmcw assigned cfjchu and unassigned Aswinmcw Sep 23, 2024
@SeanNijjar
Copy link
Contributor Author

Hey @cfjchu, I assume these (4chip) tests fail now because we don't by default specify the inner 4 chip ring. I'd l8ke some input from you about the best way to proceed so we can still have a4chip ring? Do we just need to invoke the APIs differently in order to get that ring?

@SeanNijjar SeanNijjar linked a pull request Sep 26, 2024 that will close this issue
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants