Collaborator is configured such that once any defect has been found in any file of a review, then no files in that review can be removed. Your only option is to delete the review and make a new rev...
Removing the files needs to be blocked for many users because absolute audit history must be maintained. In 11.0.11000 the "hide" option for files was added that effectively implements what you describe. It is limited to hiding files which have no comments however, including the Overall Accepted that marks a version of a file as reviewed by a user.