The Ka Stone was originally a fragment of the Lifestone Tree, a collaboration of various alien species. Many of the fragments ended up on Earth, after the space ship transporting them was shot down thousands of years ago. Around 5,000 years ago, the Ka Stone was found by Anath-Na Mut in the temple of the Ka. He had wandered in the desert for 90 years after his defeat at the hands of Moses and being expelled from his duties as chief wizard to Ramses II. The Ka Stone empowered Anath-Na Mut and turned him into the Sphinx. As the Sphinx he became an immortal, with godlike power, but he eventually grew weary of immortality and sought a way to end his life. For a period of months after gaining the Ka stone the Sphinx was overwhelmed by the power he had gained and was nursed back to health by a nomad woman. This woman would later become a perpetual reincarnate and became obsessed with the Sphinx.
The Ka Stone was destroyed by Galactus before he sent the Sphinx back in time to relive his life. The Sphinx stumbled onto a flaw in Galactus's punishment and engineered an encounter with his past self and together they recreated the destroyed Ka Stone. While one Sphinx was doomed to continuously relive his life, as Galactus's punishment required, the other Sphinx slept in suspended animation and was awakened in the present day. This Sphinx later fought the Thing but was killed when the Puppet Master forced him to crush the Ka Stone.
Later, Meryet Karim, the current name of the reincarnated woman who was obsessed with the Sphinx, used another relic, called the Sceptre of Ka, to reunite the energies of the Ka Stone. Meryet reconstituted the Ka Stone but was unable to control the power. She recreated the timeline and in it she was happily married to the male Sphinx before his suicide and then she became the Sphinx. The New Warriors convinced her this reality she created was a lie, and the universe reverted back to its original condition.
By reconstituting the Ka Stone, Meryet unwittingly triggered the resurrection of Anath-Na Mut, who like her before him, reincarnated into a new body. He became obsessed with regaining the Ka Stone and its power. 
Years later, the original Sphinx used a fissure in space to acquire a second Ka Stone via time travel.