A system for evaluating novelty in computer generated narratives

Abstract

The outputs of computational creativity systems need to be evaluated in order to gain insight into the creative process. Automatic evaluation is a useful technique in this respect because it allows for a large number of tests to be carried out on a system in a uniform and objective way, and produce reports of its behaviour. Furthermore, it provides insights about an essential aspect of the creative process: selfcriticism. Novelty, interest and coherence are three main characteristics a creative system must have in order for it to be considered as such. We describe in this paper a system to automatically evaluate novelty in a plot generator for narratives. We discuss its core characteristics and provide some examples.