Maybe old but delegation works in such way that when you put that delegated amount there, it is the actual total amount you are going to delegate to that account. It does not add up. So simply to undelegate just delegate 0 sp to them and youll get your sp back in a week.