This paper presents a theoretical framework to create never-ending stories. Our theoretical framework is developed using discrete-time stochastic processes with a Markov property. To demonstrate the effectiveness of our theoretical framework, we have developed a visual storytelling system that helps users to interactively create narratives. This preliminary visual storytelling system can help users to create never-ending narrative segments in cartoon form.