He can be met during the first visit to the Atomic Wrangler casino, standing close to one of the guards near the entrance of the gambling area.
The dialog with him will imply he's working as the Garrets' debt collector. He'll become the final target of the side quest Debt Collector and relocate in The Strip. Caleb is incredibly arrogant and condescending, deriding the Courier as beneath his notice. A Speech check is required to deal with him in any way other than violently.
Debt Collector: Caleb is a target in the quest Debt Collector. The Courier is asked to kill him, since he fled to The Strip. He is standing just outside of Gomorrah. Alternatively, if your Speech skill is high enough, you can let him walk free by just taking his hat and a few bottle caps so that the Garrett twins think that he's dead.
The Courier can ask him for tips on bounty hunting, if talked to before the Debt Collector quest. He will give the advice "Be quiet, be quick, and don't get dead." If you ask him for advice again, he tells you he isn't a "career counselor".
After choosing the speech option and letting him live, he will go back to the Wrangler and shoot up the place. He will be killed by the guards, and you can loot his body.
When attacking him, no one on the strip will turn hostile.
ps3xbox360pc If you request Caleb's hat and then kill him without equipping the hat it will become permanently embedded in your inventory unable to remove as it is a quest item and cannot be given to the Garrets because they are quite happy he's dead. This presents no issues as quest items carry no weight. [verified]
ps3pcIf you kill him in the Atomic Wrangler, his body can be found in the Strip. [verified]
xbox360pc His body might remain in the Strip indefinitely when killed. [verified]