The followers eye is never hostile. They just stare at you, and sometimes when you get close, start to follow you until you give them a thumbs up.