Part of a grand cloning experiment that led to a horrific end for the instigators, 54 clones of "Gary" were created, but each began to exhibit psychotic tendencies to the scientists who created them. Eventually they rebelled and took over the Vault completely, and have evolved a complex and subtle series of ways to say "Gary" to convey their emotions. They are deranged and attack anyone that isn't Gary.
A holodisk found in the Vault 108 cloning lab, named "Cloning log," explains that Gary was cloned at least 54 times; all clones exhibited hostility towards non-clones, with each successive clone being more and more violent. It is uncertain why the scientists continued to create homicidal clones or, indeed, why they started.
When the Vault scientists started running out of room to store the clones, they planned to destroy some of them to make room for additional test subjects. Apparently the clones objected to this, escaped, and slaughtered the rest of the Vault's inhabitants. It seems most of the Garys either escaped or were killed, as the Vault door is found open upon discovery, and Gary 23 is located far away from the Vault.
Curiously, the Gary clones found in Vault 108 use almost entirely different ways of saying "Gary." The reason for this is not explained or even hinted at, but might be an intended joke by the developers.
In the Outcast outpost (from the Operation: Anchorageadd-on), a dead Gary (Gary 23) can be found wearing a Vault 108 jumpsuit in the locked room (Lockpick 25) across from Protector McGraw. Gary 23 has nothing on him and his jumpsuit cannot be removed. His left arm has been cut off, and there are medical tools nearby. It seems that the Outcasts tried to remove his Pip-Boy 3000 through removal of the limb in order to use the military VR pod in order to gain access to an armory that opens once the simulation is complete. Specialist Olin also comments on how it wasn't her fault "what happened to the first guy." Gary's complete Interrogation log was cut from the game, but can be obtained through console commands if one wants to know exactly what happened to him.
There is evidence to suggest that at least some of the Gary clones were curious about the nature of non-clones. One such Gary can be observed studying a wastelander's corpse laid out on an operating table in the cloning labs.
Although the Gary clones usually say nothing but their name, their death quotes are similar to those of regular non-player characters: a Gary might shout, "No!" upon death. Also, if you stay hidden (by wearing Chinese stealth armor or Stealth Boy) and get close enough to two Garys, they can be heard talking like normal wastelanders, saying things like "morning," "hey," or "be careful with that."
The Gary clones are referenced in Fallout 4 as a running gag, where four wooden blocks can be found in assorted locations spelling out "GARY."
In Fallout Shelter, during the quest Vault-Tec Experiment Review, the location will sometimes contain enemies and characters that only say "Gary." They will attack if the Dwellers respond with anything other than "Gary".
Several audio files can be found in the game's files of the character shouting "Gary" in various different ways indicated that the character(s) was going to be included in Fallout 4, but were cut from the game. [verification overdue]