After going into Settings/Audio and setting Music volume to 0%, the music is muted as expected.
When I turn off my headphones and my laptop switches to its internal speakers, the music begins playing again.
Opening the Settings/Audio menu shows it still at 0%. Moving the slider up and back to 0% mutes the music again as expected.
Music should stay muted when changing audio devices.