Allowing athletes from Russia and Belarus to compete at the Paris Olympics could lead to a large-scale boycott and render the Games "pointless," Poland's sports minister said on Thursday.