Many of the short stories of Jorge Luis Borges have both specific mathematical objects and abstract mathematical concepts embedded in their very structure. The most prevalent of these is the concept of Infinity, which Borges examines not only as a philosophical idea, as many have done before him, but also as a precisely defined mathematical object. The mathematical properties of Infinity are incorporated into his stories as metaphors, lending a strange and magical feel to them. In this paper we examine four of his most famous stories and the mathematics they contain.