An atheist is just a person that lacks a belief in a god or gods. In that case, I agree with Kekerusey here (and many other people as well), that babies are people who do not believe in a god or gods. That doesn't mean that all atheists are babies, not does it mean anything useful in regards to babies. But the utility is not in discussing theism about babies, the utility is a consistent definition.
But then I got to thinking about our ignorance of the thoughts of babies. Because we can't talk about things with babies (nor do we presently have any other reliable means to discover their thoughts), they might develop some kind of concept of a deity that we wouldn't know about. We know if that ever happens or, if it does happen, how often it happens. Which means that speculation is fine, but discussing the beliefs of a baby is rather pointless and useless.
With this definition, things that are not people cannot be an atheist, so cars, rocks, and shoes, not being people, are not atheists just because they too lack belief in a god or gods.