Re: I thought I had my Groups all set up but,... ):

It's actually very simple to say:

You can assign an activity to all groups or one grouping.
A grouping contains one or more groups.
You cannot assign an activity to just ONE of the groups in a course without using a grouping.

Check out the tracker item here: https://tracker.moodle.org/browse/MDL-28082

Everyone (even people who don't teach with Moodle) realises this is clunky.

You say:

I am not able to create a Group of specific students and then create separate groups in the assignment module that specify only that Group.(ie in the assignment module it only recognizes groupings not Groups!) 

Correct

Why create a duplicate task? 

Good question.

-Derek