You may run into this error when upgrading your UserProfile database. I had to track this down using Reflector and some database sleuthing.
Turns out, some people created groups on their mysite and put several people into it. Any groups created like this (which I had two) will cause this error. You can get the group id and my site location by finding the third value in the duplicate key value (in this case 20795) and can be found by using the "MemberGroup" table. You can simply delete this faulty group and then the upgrade will proceed.
12/09/2014 19:36:57.06 powershell (0x245C) 0x3204 SharePoint Foundation Upgrade ProfileDatabaseSequence ajywk INFO ProfileDatabase Name=Profile_DB 4ed7d9fe-19a3-47a2-b5cb-8f9981bcc813
12/09/2014 19:36:57.06 powershell (0x245C) 0x3204 SharePoint Foundation Upgrade ProfileDatabaseSequence ajywk ERROR Exception: Violation of UNIQUE KEY constraint 'AK_UserMemberships_RecordId_MemberGroupId_SID'. Cannot insert duplicate key in object 'dbo.UserMemberships'. The duplicate key value is (0c37852b-34d0-418e-91c6-2ac25af4be5b, 21236, 20795, 0x010500000000000515000000b4b7cd221ed1xxxx). The statement has been terminated. 4ed7d9fe-19a3-47a2-b5cb-8f9981bcc813
Enjoy!
Chris