I've ran into this problem for WELL more than a year. And lately, I find almost every time I edit my images, I get not only a "REMOVE SELECTED" error, but ENTIRE IMAGES ARE BEING DELETED. I'm not doing anything but the image I'm editing a caption just "auto deletes" by itself. I'm lucky that every time I have the non-refreshed website open in another browser, so I can copy & paste the content and rebuild.
But these are very time-consuming glitches that I think Squarespace should focus on.