It’s inappropriate to describe a Black man as colored in 2000-anything. Especially when you want the library to call the police because you are “concerned” with the content of his phone conversation. Save that for when you are talking to other white people.
Or, don’t use it at all.
Or, use it in a room full of Black people. You tried it,
A Black Librarian