The more i read this sentiment, the more I feel his scene w/ Ned in season 1 was a waste. :/
http://www.youtube.com/watch?v=mkPMA7R9hek
Anything significant about the scene (imo) doesn't really originate with Selmy's character. Most of the expository info could have come from a ton of other characters (re: history, Ned's family, Robert's state of mind, etc.). The character as a whole just didn't register as memorable in season 1, imo, save that awesome exit scene in the throne room.