Deleted product images don't disappear.

I have uploaded an image for a product, then used the trashcan on the image in the product edit to delete it, then added a new image. The image goes away in the edit screen and the new one shows. But... it never corrects itself on the product page and never uses the new image. This seems like it would be a straightforward thing and it would just replace the image with the new one. Many times I have had to completely delete the product and add a new one all over just to be able to get the new image to show. What's going on?

Thank you!