In this paper, we present a theoretical framework to represent and manipulate narrative structures for visual storytelling. This framework can be used in applications beyond visual storytelling, which includes formal representation of stories, emotional, social and even economical interactions among agents. Our framework significantly extends and formalizes classical narratology theories. In our framework, we represent narratological functions as interventions by employing an extension of causal inference theory, as directed graphs that provide cause and effect relationships among agents. Moreover, we categorize them as real, expressed and observed interventions. This differentiation allows us to represent beliefs, lies and misunderstandings. In our framework, any transformation in causality graph structure is called an event by providing a non-linear temporal dimension that can even allow time-travel. This approach provides a general framework to develop tools for modeling narration and can help to investigate social and economic interactions.