When renaming a test case, is it not supposed to remove older one?

I've gone in and renamed a couple of testcases and noticed that it renames them in place however when you refresh the composite project, the old test cases pop-back up.  Additionally, the folders for these test cases are on the file system.

Is this by design?  Doesn't seem it should be if it is.

Please advise,
Mason