Talking About Talking

If you've read the first blogpost, you will know the game I'm currently working on is an RPG with heavy emphasis on dialogue and moral choices. To this end, one vital part of developing the featureset for the game is making a robust dialogue system. I had a good idea of what I wanted it to be able to do before I set out to make it, and one of my main goals was to make a very automated system that, when properly set up, would be able to handle just about every situation I could throw at it and automatically handle everything for me after being fed a single file for each encounter - one single file containing all relevant data - beyond just dialogue.

Here are the bullet points:

• Based on external files to accommodate for localization, easy fixes, updates and possibly modding
• Needs to work in seamless conjunction with the internal quest handling module
• One file for each conversation should provide everything - not just dialogue, but automated handling of quest data, scripted events etc.
• Wildcard attributes in a file can do stuff like changing in-game values (money, health, good/evil points etc.) at specific points in a conversation
• Should also be able to change facial expressions at certain points, determine who's talking to who - maybe a second NPC suddenly gets involved
• Handle dynamically created, context based dialogue option; based on certain criteria like quests, weather conditions or other factors
• Needs to be able to show customized control keys (images and/or text) - with localization taken into consideration
• Anyone, regardless of programming skills, should be able to create content for the dialogue system with very little to no need to read documentation

So let's look at them one by one in a bit more depth.

Based on external files

I won't dwell too much on this - anything other than external files would just be rather stupid, quite frankly. It provides some significant advantages;
It will be much easier to correct any errors (typos, grammatical mistakes etc.), it's much more organized, it allows for easy localization, and it even potentially allows for modding/addition of user created content. There's simply no other way.
A little bonus anecdote; when we made Spoiler Alert as rookies, we made the huge mistake of making all text either hardcoded in the code or, even worse, ingrained onto the graphics themselves. After release, we got multiple good offers for specialized distribution on foreign markets in both Germany and Asia, but had to decline, because the main criterium was for the game to be localized - and even with a game as text-light as Spoiler Alert, this would just be too big a task. Lesson learned!

One file should provide all needed data

For games with dialogue systems, it's fairly normal that they do more than "just" talk - typically, you want to be able for the system to trigger certain events - for example, you might want to update the player's map mid-conversation when an NPC tells you about a new location, update the player's inventory when given an item, subtract some money when getting information from a snitch, just to name a few examples. So while a dialogue file should obviously contain all the spoken dialogue and dialogue options, it should also contain information for special triggers such as these, and anything else potentially needed. In short, every dialogue encounter should only take up one single file, complete with everything.

Work seamlessly with internal quest module

The quest module is, like the dialogue system, a separate module all of its own. It handles all quests - the ones you accepted, the ones you completed, the ones you botched etc., as well as when you got them and much more. It keeps track of how far along you are in each quest, and it lets you filter your log based on all this data (for example; show only quests I have completed, and order them alphabetically, or show all quests regardless of their state, and sort them by the order in which I got them).

But this needs to work with the dialogue system in a few ways. Some of them are rather obvious; for example, you don't want it to present the dialogue option where you tell the innkeeper that you've cleared his basement of rats when in fact you still haven't done that, just as you don't want the innkeeper to offer you the quest when he's already done it.

But due to the free nature of the game, there are quests you may be able to, even unknowingly, complete before you even get them. For a simple fetch quest, you might come across the item in question before knowing it's related to a quest. When you talk to the NPC asking you to fetch it, the system should acknowledge this, so that two important things happens;
1: You won't get the normal "I accept your quest" option and then have to end the conversation to talk again so you can go "Here it is" and complete the quest
2: You will instead get a more fluent and organic "Oh, you mean THIS item? I already found it!" option - in this way, it feels way more immersive and natural.

Wildcard attributes

The system also needs "wildcard" attributes. These are attributes that don't necessarily have anything to do with the conversation; look at them as "keywords", a little similar to very simple code, that tells the system to do something specific. One such example could be a wildcard keyword like "EMOTION", which can have several possible values like "ANGRY", "HAPPY", "EXCITED", "SCARED" etc., and this would then tell the system to change the emotional expression of someone in the conversation - make them frown when angry, smile when happy etc. Another example could be the ability to change simple values in-game, such as give or subtract money, update a map location and many other things.

Who's talking to who

A small thing, but important to immersion; the system should be able to keep track of who is currently talking, and who they're talking to. A simple example of how this is used;
The player walks past an NPC who's idle and looking towards the left side of the screen. Once the player is on the right side of the NPC, he or she decides to trigger a conversation; the NPC can recognize that he is being adressed by the player, and as a result, he faces the player (turns to the right). It also means that, if the player was facing away from the NPC before engaging in conversation, the player will automatically be flipped to face the NPC. Anything else would look incredibly arrogant... Or stupid.

One more very important thing about this, is the ability for the system to recognize who is present and who isn't. A dialogue file can potentially point towards several speakers; for example, the player may initially just be talking to an NPC, but then another NPC close by may choose to participate based on something he overhears, and so the system needs to be able to automatically identify this new NPC as the speaker. But what happens if he is no longer present? What happens if he was murdered or something like that?
Then the dialogue file will jump to an alternative branch that handles that case!

Essentially, this is basically just an extension of the wildcards; but integrated to work with character IDs and give them orders based on this.

Dynamic, context based dialogue options

Many dialogue options are probably going to be fairly fixed, but we need to be able to define special context based dialogue options that are only available under certain criteria.
For example, the player shouldn't have the option to tell about an event he or she hasn't witnessed, tell a password they haven't learned etc.. Furthermore, parts of the conversation (not necessarily just the options for the player, but also whatever the NPC is saying) could possibly be slightly changed due to external factors, such as the weather. Even though the weather may be completely random and dynamic, the system should be able to tell how the weather is and reflect it; for example, instead of his usual "Hello there", the local merchant may opt to greet you with a sarcastic remark about the "great weather" when it's raining.

Show customized control keys

The UI for the dialogue system needs to be able to display control instructions (such as "press SPACE to continue"), but here are some small complicating factors we need to take into consideration;
1: Localization
2: Displaying a combination of images and text. For example, showing "press <INSERT PICTURE OF SPACEBAR> to continue" looks better than just the text "spacebar".
3: Not only does the text part of that need to be dynamic for localization puporses, but so do the images; what if the player changed the spacebar to another button? Or is using a gamepad?

Anyone should be able to use it

While making the game itself obviously takes programming skill, this particular system all on its own should be so simple and user friendly in terms of creating the content files that anyone, regardless of skill level, should be able to make content for it with very little or no need at all to read documentation. Preferably, the only part that should really need a little more elaborate explanation should be the various options when it comes to wildcard attributes (obviously those needs to be explained and documented), but that's about it. This ensures a fast and simple workflow (without compromising complexity and flexibility), and it also allows me to get additional writers and content creators on board who doesn't necessarily know the first thing about coding.
Also... Might I, once again, mention those modders?

Great... Now Make It!

So, these are all the things I need the system to able to do. Now I just need to make it.

Basically, the system consists of two parts - the interpreter (which is basically the system), and the content (the files being fed to the system).
The interpreter was by far the biggest piece of work - it reads the file it's given, and then decides what to do. It needs to be able to recognize what goes where - which part is the text currently being spoken, which is the part with the dialogue options it needs to generate (and in what order), what are the special dialogue options that are based on special circumstances, and are those special circumstances being met right now - what wildcards are we currently working with, and what do we need to with them, etc..

That part was made a long time ago, and while it's basically finished and working, it's still being reworked or have stuff added to it every now and then, getting better and more flexible all the time.

Now, even though the interpreter part was by far the biggest work, the files was actually the biggest problem. 
Up until very recently, I was creating the files by hand, which worked just fine (although it was a little tedious and fairly error prone).

However, I discovered one huge design flaw - when saving a game, the save stored where in your conversations you were (in order to be able to pick up that conversation the same place next time you load up the game), but the problem was, it saved it as an identifier string in order to point to what branch of the dialogue file you were currently in.
Problem being... What happens if you are playing a game in, say, Spanish, and the next day, your Italian friend comes by and you want to show him the game - so you switch language, load up the game, and continue the conversation you were having with an NPC. Now the interpreter will be looking for a branch with a string identifier that doesn't exist, because the savegame has stored a Spanish string identifier, but is now actually looking for the Italian version of the same one.

So... I needed to convert the string identifiers to number based ones, because numbers are fairly universal. This, however, presented a few new problems;

1: I need to make sure that no one ID is ever used twice
2: Numbers aren't as human friendly for readability as strings are for identifiers  (this is relevant in the eyes of a content creator)
3: Making files by hand was already very error prone, but this is even more; I need to keep track of what numbers have been used all the time

It was clear to me that I needed to move to using numbers as identifiers instead of strings. But it was also clear that this meant making files by hand was just no longer viable. Too tedious and way, way too error prone.

So, I needed to make a tool - a tool that would allow me and other people to create the files through a visual interface that would handle all the error prone stuff for me behind the scenes and let me and others focus on just the content creation.

So that's what I did. The tool is now done, and has its very own blog post, which you can read right here!