But at the core of Storyteller is its alignment algorithm. Storyteller can take an ebook and an audiobook that you provide it and align them, finding where each word of the ebook is spoken in the audiobook. It does this automatically, without any input from you. Then it uses the EPUB specification’s built-in audio synchronization system, called Media Overlays, to embed the audio and synchronization info into the EPUB. That lets you read your book immersively, with your reader app highlighting each sentence as it’s narrated by the narrator, just like in the demo above (which is using real Storyteller alignments!).
Initially, that’s all Storyteller was: one Python script. It took one audiobook file and one ebook file, and it output a new ebook file with audio synchronization metadata. At the time, there were a very small number of ereader apps (and zero ereader devices) that could consume these files, which used the EPUB Media Overlay specification for their functionality. I would run the script on my computer, copy the resulting EPUB to my phone, and then use BookFusion’s fledging Media Overlay support to read and listen to my books.