@skyfaller You're probably after a tablet or e-ink device with a locked down desktop UI.
Maybe only a single app (drawing? push button to make a noise?) in full screen mode to start.
Balanced against a collection of physical toys, books, etc, curated with the idea that your kid is building a brain right now and needs a lot of play and as little commercial interference as humanly possible.
Distro will matter less than the curation. Did OLPC quit?