Depends on which band it is. If it's a band I like I don't really care what the album cover looks like (some of My Dying Bride covers have been a bit poo). However, I have gotten into Bal Sagoth purely from the cover of Starfire Burns; and Drudkh through Blood in the Wells album cover. So in a way, album covers do matter.