....I lived in Alphabet City mostly.....the one record store I remember most fondly was Moodies, up in The Bronx.....they were 24/7 if I'm not mistaken....most of my record shopping in the East Village was at merch tables at shows, but I'd also go over to the outdoor vendors off of Lex and Great Jones, on the weekends....mainly tapes and singles but you could also find good vinyl and even CDs....Village Music on Bleecker might be where you're thinking of....they stayed open late but I don't remember how late...
MELISSA - II