How to set MTrurk assignments requested

[seme0021]

Is there a way to set the HIT Assignments Requested in Mechanical Turk? Default is 1.

[alexholdenmiller]

I think we might not support this yet, but should. We'd be happy to accept a pull request (add it as a param to the parlai/core/params.py file and to the mturk API call in the mturk code), otherwise @yf225 could you look into this when you get the chance?

[yf225]

There would be some design changes if we allow MaxAssignments > 1. I will look into this and put up a pull request soon.

[alexholdenmiller]

Pardon me for underestimating--thanks Will!

[alexholdenmiller]

We could always require this to be done manually by the user--e.g., create only 1 assignment per HIT, but user can make some HITs the same text (shouldn't be hard to match them together). If this is a significant design change then I don't know if it's worth it.

[yf225]

I agreed. One benefit of having multiple assignments per HIT is that each HIT can have multiple unique Turkers working on it, so it could still be a valid use case and we should probably address it.

[seme0021]

Thanks @yf225 and @alexholdenmiller - will look for the PR!

[yf225]

#156

[yf225]

@seme0021 We now support multiple assignments per HIT via the -na <num_assignments> parameter. In run.py, we can use hit_index and assignment_index to differentiate between different HITs and assignments, and set the content of the task accordingly.

[seme0021]

@yf225 Great! Thank you.