He can be met during the first visit to the Atomic Wrangler casino, standing close to one of the guards near the entrance of the gambling area.
The dialog with him will imply he's working as the Garrets' debt collector. He'll become the final target of the side quest Debt Collector and relocate in the Strip. Caleb is incredibly arrogant and condescending, deriding the Courier as beneath his notice. A Speech check is required to deal with him in any way other than violently.
Debt Collector: Caleb is a target in the quest Debt Collector. The Courier is asked to kill him, since he fled to the Strip. He is standing just outside of Gomorrah. Alternatively, if your Speech skill is high enough, you can let him walk free by just taking his hat and a few bottle caps so that the Garrett twins think that he's dead.
The Courier can ask him for tips on bounty hunting, if talked to before the Debt Collector quest. He will give the advice "Be quiet, be quick, and don't get dead." If you ask him for advice again, he tells you he isn't a "career counselor".
When attacking him, no one on the strip will turn hostile.
Sometimes he will equip a Cattleman cowboy hat instead of the normal hat that looks like the Old cowboy hat. It will replace the normal hat required in the quest Debt Collector.
pcps3xbox360 If you request Caleb's hat and then kill him without equipping the hat it will become permanently embedded in your inventory unable to remove as it is a quest item and cannot be given to the Garrets because they are quite happy he's dead. This presents no issues as quest items carry no weight.[verified]
pcps3 If you kill him in the Atomic Wrangler, his body can be found in the Strip.[verified]
pcxbox360 His body might remain in the Strip indefinitely when killed.[verified]