by Andreas Panagiotopoulos.
You should set the option "Seperate groups" and secondly, set the capability of the teacher role "accessallgroups" to prevent. With both options will have the result you want (each teacher has access only to his/her students' assignments).
Andreas