Atheists, like everyone else, have to base their sense of morality on emotions.
Rationality is great when we already know what we want to optimize for. If we want to know how to buy the most fruit given our budget, or whether a conclusion follows from two premises, it’s wonderful. The problem is that rationality cannot tell us what we should optimize for.
Morality, David Hume and others have argued, cannot be derived from rationality as you cannot derive an “ought” from an “is.” Morality comes from our base emotions and conventions arisen in society to structure relationships between individuals and groups.
Atheists can say they believe in rationality, humanism, and compassion. They look a lot better than those basing decisions on what was thrown together as a holy book from different, contradictory texts written in the Levant over the course of a thousand years. But they still cannot claim anything more about what is ultimately good and bad and why. Humanism must promote a given idea of what it is to be human and a good human. Utilitarians need to decide what goes in the "utility" function.
It’s common for atheists and liberals to say that when “the religious” dictate behavior they are imposing morality but when the rational good guys do it it’s just making the world better. But anytime you make a claim about how people should live, you are assuming certain outcomes are optimal based on your own raw emotions. Christianity and Islam are belief systems in the same way nationalism, liberalism, humanism, communism, or fascism are. Any of these ideologies professing a belief in a greater good have justified and will continue to justify a whole lot of killing.
Simple statements like “Treat others as I’d like to be treated,” “Take action x if it’s okay if everyone takes action x,” and “Love thy neighbor as thyself” lead to very different moral behaviors, as UCSB economist Ted Bergstrom has shown. People’s moral leanings cannot avoid relying on more than rationality, and often involve little of it.
Atheists don’t go around murdering and thieving because they don’t fear divine retribution or don’t have any moral compass. But on the other hand, they probably don’t refrain from stealing and killing just because they have decided it is irrational, given a consistent set of beliefs about how to treat other humans.
Atheists don’t steal because most people around them don’t, it’s looked down upon, and it can get them imprisoned. Self-interest and fear of violating social norms may be enough for humans to end up treating each other decently. Maybe it’s better people disagree about morality or don’t truly have moral beliefs, so no extreme measures can be taken by a majority.