maybe have an "unknown" folder, like the spam folder in emails, where unknown contacts get filtered to without being seen.
an email spam filter function
not that complex