Disclaimer: We're only talking about the original Japanese lore here, not the US or European localisations that changed a bunch of things.

The manual for Sonic 1 (http://info.sonicretro.org/Sonic_the_Hedgehog_(16-bit)/Manuals/Manuals)) says that South Island - an island that floats around in the ocean, constantly changing location - is the place where the Chaos Emeralds lay dormant.

The manual for Sonic 2 (http://info.sonicretro.org/Sonic_the_Hedgehog_2_(16-bit)/Manuals#Translated_from_Japanese/Manuals#Translated_from_Japanese)) says that Westside Island - an island that is difficult to locate because it constantly floats around in the ocean, always changing location - was the home of an ancient civilization which used the Chaos Emeralds to achieve prosperity until they misused them, causing the gods to seal them away somewhere on the island.

The manual for Sonic 3 (http://info.sonicretro.org/Sonic_the_Hedgehog_3/Manuals#Translated_from_Japanese) says that the Floating Island - an island that floats in the sky - back when it was part of a continent, was the home of an ancient civilization which used the Master Emerald to achieve prosperity until they misused it, causing the destruction of their civilisation, leading to the gods restoring a small part of the continent as the Floating Island which they then raised into the sky with the power of the Master Emerald. This same manual also says that Knuckles guarded the altar of the Master Emerald in which the Chaos Emeralds also resided until, one day, they started vibrating and then, with a flash of light, they knocked Knuckles unconscious and, when he woke up, they had disappeared.

So what gives? How can the Chaos Emeralds reside on three different islands at once? And how come all three islands basically share the same backstory about ancient civilisations and gods and emeralds? And why do all three islands share the same trait of not being stationary in the ocean but floating around instead?

Well, here's my theory...

All three islands used to make up a single large continent (the one the Sonic 3 manual says the Floating Island used to be part of) upon which the Echidna civilisation prospered thanks to the Chaos and Master Emeralds. The Echidnas kept the emeralds in and around the altar of the Master Emerald (as seen in Sonic & Knuckles), but also used giant power rings and star posts to create gateways in which they could - with enough ring or blue sphere energy - summon the emeralds to themselves. Eventually, the Echidnas misused the emeralds, causing their continent to break apart into the three separate islands (South Island, Westside Island, and the Floating Island), leading to the gods sealing the emeralds back in the altar of the Master Emerald on the Floating Island.

So, when Sonic 1 takes place, the Chaos Emeralds actually resided on the Floating Island, perched on top of the pedestals that surround the altar of the Master Emerald (as we see in Sonic & Knuckles). However, South Island and Westside Island still retained their giant rings and star posts with which one could summon the emeralds to themselves and away from the Floating Island's Master Emerald altar.

As such, when Sonic claimed the emeralds during the events of Sonic 1 and 2, that caused them to vibrate and disappear, knocking Knuckles unconscious until the events of Sonic 3. It's important he be unconscious for that long since the Sonic 3 manual states that mere moments after he woke up and realised the Chaos Emeralds were gone, the Death Egg crashed into the Floating Island. This isn't impossible to imagine since the power of the Chaos Emeralds could easily have put Knuckles into some sort of long but temporary stasis where he doesn't need to eat or drink to survive, but he still remains unconscious the whole time.

This theory, from what I can tell, accounts for all discrepancies and plot holes that would otherwise exist within the lore of the classic games. It explains why the Chaos Emeralds were thought to reside on all 3 islands, since they can all be found on all 3 islands (using giant rings or star posts), though they only actually resided on the Floating Island. It also explains how Knuckles could have had the Chaos Emeralds physically with him around the altar of the Master Emerald, like the Sonic 3 manual says he did, until they disappeared before the events of Sonic 3. It also explains how it could be simultaneously true that Sonic and Eggman physically possessed the Chaos Emeralds prior to Sonic 3, and that Knuckles physically possessed them before Sonic 3 but was knocked unconscious by their disappearance before waking up just moments before the Death Egg hit the Floating Island.

I welcome any thoughts, critiques, opinions, criticisms, and alternative explanations.