There's a story where the First Firmament comes back from the place where universes go when they die and tries to reclaim Everything. The current Marvel multiverse is the 7th/8th Cosmos. The First Cosmos wasn't even a multiverse, it was a singular universe. The multiversal aspect didn't start until the 2nd Cosmos. Well, the Celestials were originally created back during the First Cosmos, but since then, they've stuck around, although some have died, and others have been created. So on one hand, they should be multiversal entities, but in actuality, they are not.
Footnote: the 7th Cosmos was the original Marvel multiverse until they destroyed it. When they brought it back, it wasn't a new multiverse like all the previous times, but a continuation, although they still call it the 8th Cosmos instead of 7th.