If you try to remove a vote on a deleted post, you will get the "this post has been deleted" popup and the vote count will be reset back to what it was before you tried to rescind your vote, but the orange marker indicating your vote will not be reinstated until you reload the page.
(I assume this also happens with upvotes, I have only tested it on downvotes.)