A simplified version of information processing in the brain minimally requires cortex and limbic processing modules. In this paper, we discuss how to construct training data for auditory and visual cortex models using a 17th century construction technique called a Wurfelspiel matrix. Traditionally, this was used to develop many equally valid musical compositions for use as templates on which the actual composition of the musician would be based. For example, a musician would compose 10 openings, 10 transitions and 10 closings which would be fitted into a 10 x 3 matrix. From this a total of 103 musical prototypes could rapidly be assembled by combining an entry from column 0 to one from column 1 and then ending with an entry from column 2. The artistry of the musician was essentially captured in the 30 fragments which could be combined in such a combinatorial fashion. This idea can be used to develop musical data for use in training a model of auditory cortex. In addition, the notion of Wurfelspiel matrices can be extended to the design of painting data for use in the training of the visual cortex. In this paper, we discuss how these specialized training sets provide us with the data to construct interesting cortical models which can eventually be used to create models of musical and painting composition. In addition, since the data can be generated with different emotional modalities, there is also a potential for building limbic processing modules.