Maybe you can include the phone number below the contact's name on top of the conversation. It would make it easier to determine as some contacts have 2 numbers.