... probably have wizards as well. They would need to have them of course, because each world is evidently teetering on the brink of falling into the Dark Dimension.

I've seen this sub's reactions to the use of magic by an Android (AIDA) in the most recent episode of Agents of SHIELD, with a few of us claiming that it somehow diminishes the mysticism behind magic in the Universe. It doesn't, and even the film clarified this.

Magic is in essence a program for controlling the flow of extra-dimensional energy to the point of even manipulating other dimensions, as the Ancient One stated. There needs to be some mechanism for manipulating energy, and a sling ring is an example of this. We should take it that Agamotto somehow divined the properties of the technological marvel that is the sling ring, by some other celestial being, be it a Celestial or an Asgardian or a benevolent Kree.

Then several centuries later comes an AI that is able to peer into the machine language (source code) of this energy manipulation, which is described in a Lovecraftian Book of Sins. The technological device that she's able to create takes only a few minutes for a superintelligent artificial being, what would have taken decades for an ordinary human, considering that the pinnacle of humanity's energy technology - Tony's arc reactor - is only essence replicating the Tesseract's seemingly infinite energy (as per Howard's research).

Which brings me to the Infinity Stones. The fact that it is usable in the Dark Dimension whilst being seemingly only occurring in our regular Universe, perhaps suggests that these other aspects of the Multiverse are merely pocket dimensions of our regular Universe. The Universe that was once in fact, in a period of "darkness" either before or after its birth. And from that darkness came, as per our knowledge, the Dark Elves and the Cosmic Beings (Infinity, Eternity, Death, Entropy). It's safe to say that the latter were much more advanced than the former, yet the former was said to have ruled the Universe prior to this one. Whether the latter just happened to exist spontaneously or were beings from the previous Universe (or are omnipresent aspects of the everlasting Multiverse) is debatable, but it is doubtless that they were able to concentrate the various aspects of the Multiverse into our regular Universe in the form of these omnipotent concentrated ingots.

It is safe to conclude that the Infinity Stones such as the Tesseract obtain the energy used for its functions, such as creating wormholes, from energy derived from the other far more energetic dimensions. And the Asgardians evidently began using this form of power source for their interstellar transport system, the Bifrost. They pretty much are leaps ahead of Tesseract-inspired technology, to the point of Tesseract-based technology. And this advancement wasn't divine-inspired (that we know of), as shown with the fact that the oldest civilization of this Universe was not helped by divine providence, by the most powerful beings, in their dealings with a pre-Universe civilization that wanted existence to return to its former state.

A civilization's rank in the Kardashev scale determines not only its energy usage, but also the feasibility of its technology as well as the likelihood of its survivability. The Asgardians gained their position of Type-3 (on the scale) with their acquisition of the Tesseract, and thus secured their dominant position in the known Universe (barring any impending Ragnarok). And by corollary from the previous passage, we can conclude that the Asgardians attained this position only after millennia or so of striving their technology forward, which Agamotto and AIDA did in a lifetime or less.

We can also assume that this striving is what led them to the point of security from any of Dormammu's ambitions of succumbing them into his fold, and the general use of technology for other interdimensional purposes (the Soul Forge which uncovered the presence of the Aether on Jane's soul, or any of Loki's illusions). Perhaps even the former process is automated, and they don't even need to have sorcerers for worrying about that. Culturally however, they seemingly refuse to go beyond the age limits of 5000 years, or even resurrecting the fallen (Frigga). And this is why their automated processes haven't evolved into their guards or even conscious individuals, to the point that they'd have some Vision or Ultron-like entities. Or maybe they did and that's what the Celestials are.

The same could be said for Xandar too. Either their world is in a part of the Universe which isn't tugged by the Dark Dimension, or they have masters of mystic arts there as well, or even some automated processes for maintaining their sanctums.

In conclusion: Asgard (using objects that were generated for manipulating these aspects), and maybe Xandar and Hala, perhaps took millennia of technological marvels to uncover regarding interdimensional energy manipulation, and this was done by two members of Earth's population in a briefer period. This doesn't take away from the fact that magic is essentially the rules of how the "souls" of living organisms affect other dimensions. These rules were created by some unknown beings, but aspects of them intended to cater to material living organisms are rendered by books such as the Darkhold (Book of Sins).