-
Notifications
You must be signed in to change notification settings - Fork 272
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Yash refactor #254
Yash refactor #254
Conversation
Let's not change the setup script or the Batch script. |
Alright should I just replace them with the standard scripts that come with the pipeline?
Thank you,
Yash Thacker
…________________________________
From: glasserm ***@***.***>
Sent: Monday, March 20, 2023 4:14:32 PM
To: Washington-University/HCPpipelines ***@***.***>
Cc: Thacker, Yash ***@***.***>; Author ***@***.***>
Subject: Re: [Washington-University/HCPpipelines] Yash refactor (PR #254)
* External Email - Caution *
Let's not change the setup script.
—
Reply to this email directly, view it on GitHub<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FWashington-University%2FHCPpipelines%2Fpull%2F254%23issuecomment-1476943735&data=05%7C01%7Cythacker%40wustl.edu%7C531488fe580d47f0892e08db29881a64%7C4ccca3b571cd4e6d974b4d9beb96c6d6%7C0%7C0%7C638149436804628272%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2B7FWOljmV0TmNYhPDxEpyNO%2BedDRMW%2BkiOK%2FN8kAZCs%3D&reserved=0>, or unsubscribe<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALEHJY7I7XXKYEDUGBMOF6DW5DCDRANCNFSM6AAAAAAWBTQHFA&data=05%7C01%7Cythacker%40wustl.edu%7C531488fe580d47f0892e08db29881a64%7C4ccca3b571cd4e6d974b4d9beb96c6d6%7C0%7C0%7C638149436804628272%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9QJ8hsMgbB0Q%2BF%2BGGwmrQwpNFAztJaeksV579iEa%2FsM%3D&reserved=0>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
|
I wouldn't have committed those two files to this pull request. I don't know the best way to back that out, perhaps Tim can help. |
Can you make a side by side comparison of the usage statements when you run the script without arguments/trigger the usage? |
I have removed the changes to other files. Please check that the changes to the pipeline are all of the changes you intended to commit. |
f0fd0b3
to
2863dc2
Compare
PreFreeSurfer_new.txt |
Thanks. I noticed a new parameter at the end for the new script (perhaps the last line of the old was just omitted) and also there are some places were there are too many spaces in the sentences, e.g.: "for hcp-style" has two spaces instead of one. |
I believe the old script accepted
|
I see. Perhaps @demsarjure or @sivcek have some ideas on acceptance testing in their environment we could do to thoroughly test all the options? |
{ | ||
if [ ! $HCP_DEBUG_USETRAP == 'OFF' ]; then | ||
{ | ||
if [[ "$HCP_DEBUG_USETRAP" == "ON" ]]; then |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI, these edits are from the branch I told him to use. This PR probably shouldn't go into master, but into the debug-function-enable
branch. Once all of the pipelines have gone through testing, we can rebase that branch onto master.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do think we want to do this expeditiously, as Misha needs to use the updated structural pipelines for longitudinal pipeline development.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will Misha need all 3 structural pipelines converted before starting? If so, we should still do them into the debug-function-enable
branch, test them with that change, and then I can rearrange the commits so the new versions are on master, debug-function-enable
is based on that, and the rest of the PRs can continue the same way.
Even if he needs each one as soon as they are available, I can do that kind of rearrangement 3 times without much trouble, so that we don't have to test them twice (without, and then with) the debug.shlib changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be best if he had all three pipelines converted before doing any development on them. There is also the longitudinal pipeline. Perhaps I can convince Jure to convert that one too.
The extra spaces look like they came from where the old help info had newlines. There is a style choice here, do we convert it into sentences by adding a period, or do we put in newlines, making it longer but with less line wrapping towards the start of each sentence? Looks like I kept the newlines when I converted hcp_fix_multi_run, but I don't care that much. |
I would rather we didn't have double spaces in the middle of sentences. I think letting your code do the wrapping is fine. |
I can build a develop QuNex container and acceptance test the refactored FS on our end. This acceptance test runs HCP MPP over 7 different datasets (3 HCP and 4 internal, from Siemens, GE and Phillips scanners). |
Would that test your legacy options thoroughly? Both SE and standard fieldmaps? |
One of the datasets is legacy style, it has SE pairs though, but is not multi-band. If there is something else/bespoke to test, we can discuss it. |
Well for now this is just HCP Structural Preprocessing and PreFreeSurfer. It would be good to test a dataset with standard field maps, another with SE fieldmaps, and any legacy options you guys find important. |
OK, just to be clear the idea is to test PreFreeSurfer on the What is the timeline here? If we do this by the end of next week, would this work? |
Yes, that's fine. |
@demsarjure were you able to do these tests? |
Sorry, for not getting back to you sooner! The command is not backwards compatible, what QuNex generates right now is missing a number of mandatory parameters. So it seems like many parameters that previously had defaults set in your scripts are now mandatory. See below for details. Please advise about how to proceed. The agreement we have is that we set only mandatory parameters QuNex side, while optional ones can be set through QuNex but use HCP defaults if not set.
|
@ythackerCS @coalsont can you look at this? The goal here was not to change the requirements of the command, but just to simplify the I/O interface code. |
Several parameters above should not be mandatory as they are connected with acquisition devices and protocols (e.g., |
Yes I can go ahead and make those not mandatory! Sorry about that some of them aren’t as clear to me if they are mandatory or not.
Thank you,
Yash Thacker
…________________________________
From: Jure Demsar ***@***.***>
Sent: Tuesday, April 11, 2023 5:59:04 AM
To: Washington-University/HCPpipelines ***@***.***>
Cc: Thacker, Yash ***@***.***>; Mention ***@***.***>
Subject: Re: [Washington-University/HCPpipelines] Yash refactor (PR #254)
* External Email - Caution *
Several parameters above should not be mandatory as they are connected with acquisition devices and protocols (e.g., fmapgeneralelectric, fmapphase, SEPhaseNeg). I think the issue might be just that some parameters were erroneously labeled as mandatory instead of as optional.
—
Reply to this email directly, view it on GitHub<#254 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALEHJY5XNU3TNZLIO6XFYQDXAU2PRANCNFSM6AAAAAAWBTQHFA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
|
Do we have a list of all vars that are non mandatory? That way I can make sure they are correct. Also Jure, Matt said you wanted to convert longitudinal Freesurfer, I built a command converter for it to make it a bit easier to do the conversion to use Tim’s input library. I have attached it above it is a notebook. In the first cell where there’s command = replace the content between the triple quotes (“””) with the old command run the cell and the next one and it should give you a refactored command that should be close to what you need for Tim’s library! Let me know if you have any questions.
From: Thacker, Yash ***@***.***>
Date: Tuesday, April 11, 2023 at 11:48 AM
To: Washington-University/HCPpipelines ***@***.***>, Washington-University/HCPpipelines ***@***.***>
Cc: Mention ***@***.***>
Subject: Re: [Washington-University/HCPpipelines] Yash refactor (PR #254)
Yes I can go ahead and make those not mandatory! Sorry about that some of them aren’t as clear to me if they are mandatory or not.
Thank you,
Yash Thacker
…________________________________
From: Jure Demsar ***@***.***>
Sent: Tuesday, April 11, 2023 5:59:04 AM
To: Washington-University/HCPpipelines ***@***.***>
Cc: Thacker, Yash ***@***.***>; Mention ***@***.***>
Subject: Re: [Washington-University/HCPpipelines] Yash refactor (PR #254)
* External Email - Caution *
Several parameters above should not be mandatory as they are connected with acquisition devices and protocols (e.g., fmapgeneralelectric, fmapphase, SEPhaseNeg). I think the issue might be just that some parameters were erroneously labeled as mandatory instead of as optional.
—
Reply to this email directly, view it on GitHub<#254 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALEHJY5XNU3TNZLIO6XFYQDXAU2PRANCNFSM6AAAAAAWBTQHFA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
|
Hi @ythackerCS , Yes, I am waiting for the Thanks, Jure |
In the old-style code, the mandatory/optional logic is sometimes implemented deep in the processing code, in this case the fieldmap stuff is passed as-is to the For cases like this, maybe we could borrow QuNex's understanding of what is mandatory vs optional? |
Sounds good! Yea from what I have implemented it usually just looks for square brackets. And if option arguments are passed in if not it’s considered mandatory. Also, Jure it should be attached in the email I just set yesterday. Let me know if you still can’t find it, I can message it through slack.
Thank you,
Yash Thacker
…________________________________
From: Tim Coalson ***@***.***>
Sent: Wednesday, April 12, 2023 3:22:13 PM
To: Washington-University/HCPpipelines ***@***.***>
Cc: Thacker, Yash ***@***.***>; Mention ***@***.***>
Subject: Re: [Washington-University/HCPpipelines] Yash refactor (PR #254)
* External Email - Caution *
In the old-style code, the mandatory/optional logic is sometimes implemented deep in the processing code, in this case the fieldmap stuff is passed as-is to the T2wToT1wDistortionCorrectAndReg.sh sub-script and it deals with whether the arguments are valid. The --avgrdcmethod option is probably the only fieldmap-related option that is mandatory.
For cases like this, maybe we could borrow QuNex's understanding of what is mandatory vs optional?
—
Reply to this email directly, view it on GitHub<#254 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALEHJYYKCB4BRSPWAC7AMA3XA4FHLANCNFSM6AAAAAAWBTQHFA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
|
So, PreFS is also one of the oldest scripts, and may be less consistent than others about the help info. I also notice that the way to tell it not to do distortion correction is currently "any string that isn't recognized", which is not good design (not typo-safe). I think we should use something like "NONE" here to mean no distortion correction (though this is different than how other scripts unfortunately use "NONE" to represent an empty list), and the |
400e7f1
to
84a6729
Compare
Finished refactoring PreFresurfer ready for others to test I did do one run seems to get through without variable errors