I'm having the same issue (YouTube link to video). From what I can make out it is to do with the pushed quality of the video. For some reason it defaults to 480p. If I go to the video on YouTube and change the settings in the player to 1080p and return to my site, it's fine. There has to be a way to force externally linked videos to only play at the best settings.