Have the notification stay in the notifications with the conversation going on. The way Google messenger has it is very convenient. Another app has the feature as well, it's called pulse.