My interpretation of the Season 6 was that the "Fear Dimension" was never a thing. When the Monoliths exploded, the energy of the Creation Monoloith started manifesting people's nightmares because the Humans had no idea of how properly handle it. The term "Fear Dimension" was coined by Fitz to describe what they were dealing with, but eventually that conjecture was proven incorrect after they got more information about the Monoliths from Izel.
I tried to point that Retcon on the MCU wiki, but they have been slow to incorporate certain information and I'm not active there enough to get into edit conflicts over it.
It's definitely an alias, don't get me wrong, but it's the most notable alias we have for the location also generically referred to as Izel's realm, but regardless a location article's name doesn't have to be more than an alias afaik, plus the alias is used in universe multiple times, so there's no question it's a canon alias.