This paper considers groups of musical “contextual” transformations, the most famous of which is a group of bijections between minor and major triads described by the music theorist Hugo Riemann (1880). Mathematically, contextual transformations act on chords or melodies and commute with transposition (shifting by the same number of pitches in the same direction). This is important because most people naturally identify two melodies or chord progressions as “the same” if one is a transposition of the other. Music theorists have studied contextual transposition and inversion groups extensively; in particular, Lewin (1987), Kochavi (1998), and Fiore and Satyendra (2005) used discrete group theory, while Clough (1998) and Gollin (1998) considered symmetries of discrete geometric spaces. The action of contextual transpositions and inversions on the continuous geometric “voice-leading spaces” of Callender, Quinn, and Tymoczko (2008) reveals subtleties that do not arise in the traditional discrete approach. I propose two ways of understanding contextual transpositions and inversions, one employing a “bundle” construction and the other representing contextual transformations as a family of linear transformations. The first involves topological group theory, the second dynamics. I discuss the advantages and drawbacks of each method.