Modern mathematics owes a big debt to India's contributions to the subject. Of particular importance is the decimal, place value number system that appeared in India during the Vedic period or soon after, circa 1300 BC to 300 AD, and made its way to Europe during the Middle Ages. That period of time in India also produced a heady mixture of poetic works: poems, songs, grand epics, biographies and books of instruction in verse covering millions of pages. Mathematical ideas are interwoven into the metaphysical, religious and aesthetic fabric of many of these works. This article brings a selection of poems from that time period that provides a taste of ancient India's mathematical preoccupations in their cultural and esthetic context. They also highlight India's mathematical accomplishments of the period, and uncover instances where seeds of future mathematical concepts made their first appearance. The concluding remarks touch lightly on current Indian-inspired uses of mathematical poetry as a pedagogical tool.