Why does unfriending players not remove them as followers?

So generally, if I unfriend someone I'd like to completely disconnect from them, and I think most people are trying to do the same. So why is it that they continue to remain as followers? I understand if you set your followers to "friends only" they won't be able to see anything, but why not let us remove them all together?

I don't know if this is something that'll be changed in the future, but it really should be. Doesn't seem that difficult. Not friends, neither follows the other.