Introduction to the Nonvisual Gaming Toolkit
Thank you for downloading the Nonvisual Gaming Toolkit (NVGT) by Sam Tupy Productions and contributors. We hope you find the time that you invest in this engine to be worthwhile!
The NVGT documentation contains:
- The NVGT user manual; containing everything from programming resources to general tutorials to example games,
- API references; Class/function references that specify exactly how to use each feature of the engine,
- Advanced topics; Documentation on the c++ side of the NVGT engine including how to build the engine from source, modify it's security functions, write documentation for the engine and more,
- Appendix; data tables that don't fit in the API references, miscellaneous topics, attributions and license agreements.
Documentation status
It should be noted that this engine was only released to the public on May 31st 2024, and that the documentation is not yet complete as a result. Though the engine had been in development for years previously, the decision to open source it, and thus to write documentation for it, only took place a couple of months before the initial release. The choice was made to release the engine in an unpolished state specifically so that the community can help improve it, otherwise the engine would have remained private for years while only a couple of people worked in their spair time to complete the documentation. The engine is very usable and a growing community exists on Discord for any questions you might have, certainly we welcome all users! However those involved with the engine's development are also not personally advertising the engine yet due to it's unpolished state. Contributions to the documentation and other materials are therefore very appreciated, though you may want to hold off if you are the type looking for a particularly polished and pretty looking product at this time. Whether you decide to try the engine now or wait until the documentation is complete, we hope you enjoy your time engaging with NVGT and thank you for trying it!
offline documentation
The latest version of this documentation can be downloaded for offline viewing in several formats using the following links.
- nvgt.chm (compressed html help file)
- nvgt-html.zip (rendered html)
- nvgt-markdown.zip (github flavored markdown)
- nvgt.txt (plain text)
NVGT User Manual
Getting Started
Welcome to the Nonvisual Gaming Toolkit! This user manual contains all of the information you will need to get up and running using this powerful engine! whether you are new at programming, someone who used to create games using the Blastbay Game Toolkit or an advanced programmer who just wants an easy, no-hassle method of creating games, you're sure to find what you need in this manual to not only get up and running, but to also develop some confidence in it's usage.
Upgrading From BGT
Since for a while this is where most people who have heard of NVGT will likely start, we will begin by describing the relationship between NVGT and BGT, and more importantly the steps required to upgrade a BGT project to an NVGT one. If you have not heard of BGT or are completely new to game development and are wondering how NVGT can help you, you can probably skip this and move to the fresh start topic.
BGT stands for the Blastbay Gaming Toolkit, and is what initially inspired NVGT to be created after BGT was made into abandonware.
It is important to note that NVGT is not officially endorsed by BGT's developers and contributors, and NVGT is not in any way derived from BGT's source code. The only shared assets between NVGT and BGT are some of BGT's include files such as the audio_form and sound_pool used by many bgt games, which are released under a zlib license.
our goal is to make the transition as seamless as possible from BGT to NVGT, but here are some things you should know when porting an existing BGT game.
Missing features
NVGT currently has some missing features compared with BGT. These may be added in the future, or have already been replaced with more flexible options.
- Security functions such as
file_encrypt,file_decrypt,file_hashdo not exist in NVGT. These may or may not be added, but in the meantime you can encrypt/decrypt/hash files manually by using string_aes_encrypt/decrypt/hash_sha256/sha512.
Differences
Functional
Some functions which have the same name as those in BGT work slightly differently.
get_last_error: Currently doesn't trigger in all the situations that BGT does.- When refering to an array's length, use length as a method call and not as a property. For example, you would use
array.length();rather thanarray.length;. - Screen reader functions no longer need to specify the screen reader. If an active screen reader is detected it will automatically speak through there.
- Data encrypted with BGT cannot be decrypted with NVGT, and vice versa.
- The
sound::stream()method does exist in NVGT, but it's simply an alias tosound::load(). For this reason it is recommended that you change all yourstream()calls toload()instead. Theloadfunction performs an efficient combination of streaming and preloading by default. load()contains a second optional argument to pass a pack file.set_sound_storage()is no longer used for the moment. However, you can setsound_pool.pack_fileor even the globalsound_default_packproperty to a pack handle to get a similar effect.- Take care to check any method calls using the
tts_voiceobject as a few methods such asset_voicehave changed from their BGT counterparts. - When splitting a string, matching against \r\n is advised as BGT handles this differently. This will result in not having spurious line breaks at the ends of split text.
- The
settingsobject no longer writes to the registry, but instead has been replaced by the settings.nvgt include which wraps the previous settings object API, but instead writes to configuration files in various formats. - The dynamic_menu.bgt include is now called bgt_dynamic_menu.nvgt. This is because NVGT now includes it's own menu class called menu.nvgt.
- There is a type called
varin the engine now, so you may need to be careful if your project contains any variables named var. - It's worth noting that unlike BGT, NVGT by default attempts to fully package your game for you including sounds, libraries, documents and any other assets into a .zip file or similar on other platforms intended for distrobution. If you don't like this behavior, you can create a file next to nvgt.exe called config.properties and add the line build.windows_bundle = 0 which will cause NVGT to just produce a standalone executable like BGT did, though you now may need to copy some libraries from the lib folder for the compiled product to run. For more information, see the Compiling your Project for Distribution topic.
- In bgt, you could include pack files with the
#includedirective. In nvgt, we've decided that this should only include code. To embed packs, you can instead add the line#pragma embed packname.datto your project, the extension can be any of your choosing for both code and packs this way. - Unlike BGT, you can embed multiple packs into an exe. For BGT compatibility, therefore, opening a pack file called
*will open the first available pack, but it is recommended that you specify a filename such as*mypack.dat. - The
generate_profilefunction returns a string which you can write into a file if desired. - Encrypting/decrypting sounds is done with the
asset_encryptor/asset_decryptordatastreams, rather than withfile_encrypt/file_decryptfunctions. You can play encrypted sounds by setting thesound_default_decryption_keyproperty.
Nominal
The names of many functions have changed in NVGT. Some have even been moved into the objects they are linked to.
Below is a list starting with the BGT function and the NVGT equivalent:
- clipboard_copy_text: clipboard_set_text
- clipboard_read_text: clipboard_get_text
- generate_computer_id: generate_system_fingerprint
- is_game_window_active: is_window_active
- show_game_window: show_window
- screen_reader_is_running: screen_reader_detect (see function in API reference for differing usage)
- screen_reader_speak_interrupt: screen_reader_speak
- screen_reader_stop_speech: screen_reader_silence
- absolute: abs
- arc_cosine: acos
- arc_sine: asin
- arc_tangent: atan
- ceiling: ceil
- cosine: cos
- power: pow
- sine: sin
- square_root: sqrt
- tangent: tan
- serialize: dictionary.serialize
- string_compress: string_deflate
- string_contains: string.find
- string_decompress: string_inflate
- string_decrypt: string_aes_decrypt
- string_encrypt: string_aes_encrypt
- string_hash: separated into two functions (string_hash_sha256, string_hash_sha512)
- string_is_alphanumeric: string.is_alphanumeric
- string_is_digits: string.is_digits
- string_is_lower_case: string.is_lower
- string_is_upper_case: string.is_upper
- string_len: string.length
- string_replace: string.replace
- string_reverse: string.reverse_bytes
- string_split: string.split
- string_to_lower_case: string.lower
- string_to_number: parse_float/parse_double
- string_to_upper_case: string.upper
- The five substring functions string_left, string_mid, string_right, string_trim_left, string_trim_right, have been replaced with just two methods: string.substr and string.slice.
The following key constants are also different:
- KEY_PRIOR: KEY_PAGEUP
- KEY_NEXT: KEY_PAGEDOWN
- KEY_LCONTROL: KEY_LCTRL
- KEY_RCONTROL: KEY_RCTRL
- KEY_LWIN: KEY_LGUI
- KEY_RWIN: KEY_RGUI
- KEY_LMENU: KEY_LALT
- KEY_RMENU: KEY_RALT
- KEY_LBRACKET: KEY_LEFTBRACKET
- KEY_RBRACKET: KEY_RIGHTBRACKET
- KEY_NUMPADENTER: KEY_NUMPAD_ENTER
- KEY_DASH: KEY_MINUS
BGT compatibility helper
You can #include "bgt_compat.nvgt", which implements a lot of aliases to BGT-compatible functions and classes. However when doing this, please be aware of the following limitations:
- Using built-in functions may improve performance so this is more or less a stop-gap to get you up and running quickly; however if you wish for your code to just run, bgt_compat will certainly be of use.
- Please note you will not be able to use this script if you have disabled global variables.
- Since the screen reader internal functions no longer require you to specify a screen reader, the reader parameter is ignored.
- The hardware_only parameter of generate_computer_id is ignored.
Fresh start
This is the starting point for anyone who is interested in game development and who has found this engine, but who does not have much experience with programming concepts or the history of older game engines that are similar to this one.
Effective Programming and Game Development with NVGT
Please note that this tutorial is still incomplete and though you should read it as a beginner to gain new information, you should be ware that it will cut off abruptly as the author has not come forward to complete what they started at this time.
What is Game Development?
If you are an experienced programmer looking to get skilled up with NVGT as fast as possible, you can safely skip this section.
To someone who has never programmed before, the development of games might seem like an insurmountable task. This manual hopes to help with the steep learning curve of programming, and to get you ready to make the games of your dreams all on your own.
There are many things that need to be considered carefully in the development of a game. One of the most curious is programming; very quickly, programming stops aligning with real-world concepts. The precise instructions and rigid syntactic rules are often the bane of a new programmer's existence.
As you learn, you will begin to "speak" programming more intuitively. You will eventually realize that programming languages are nothing more than a readable abstraction over common paradigms: variables, functions, conditions, loops, data structures, and many more. Start thinking of programming as a new way to think and construct new ideas, rather than a way of expressing already-imagined thoughts and ideas.
And if you don't understand these paradigms yet, don't worry - reading this manual, my hope is that you will learn to make games with nvgt, but I have a secondary goal: by the end of this text, I hope that you will also be confident with programming in general, as well!
Especially in the world of audiogames, whose developers are often small teams at most, skill in game design is correlated with skill in programming; this manual will attempt to teach you both. However, game designing itself can also be made into a career!
When making games, you must first consider what your game will be: what are the rules of the game? What is the lore, if it has a story? Plan out your project in detail, so that it's easier to turn it into code later on. This is especially true of projects that have a plot, or at least backstory.
As well as coding and designing the game, sounds must also be found or created; many high-quality resources can be found for this, be they free or paid.
It is also the case that a released game is not necessarily a finished one: multiplayer games need administration to keep them running, and games might need to be maintained or updated to fix bugs that you did not encounter until post-launch.
Your First NVGT Script
A script can also be considered one "unit" of code. You have probably seen many of these before: .py, .js, .vbs, perhaps .bgt, and now .nvgt. A script can be executed, meaning that the code contained within will be run by nvgt.
Usually, one game will consist of not just one, but many scripts, each of which has some specific function or purpose. For now, though, a single script will be sufficient for our needs.
A common tradition among programmers is to write a program that outputs "hello, world!" to the screen, to make sure whichever language they're using is working properly.
Let's do that now and say hello to NVGT! Open a file in your text editor of choice (I recommend notepad++) and paste the following code inside:
void main(){
alert("hello", "Hello, world!");
}
Now, save this file as hello.nvgt, and head over to it in your file explorer.
Press enter on it, and you should see a dialog box appear, with our message inside!
Congratulations - you've just written your first nvgt script!
This script might look simple, but there's actually quite a bit going on here. Let's analyze our various lines:
void main(){ - this is the beginning of a function (more on those later) which doesn't return anything, and takes no arguments, or parameters.
The { afterwards opens a code block. It's customary in many codebases to put the { on the line which opens the block, and to put the closing brace on a new blank line after the block ends.
Inside our block, we have one line: alert("hello", "Hello, world!");
Let's break it down:
alert() is a function in nvgt that opens a dialog box containing a message. It supports many arguments, but most are optional. Here, we have used the two that are not: title, and text.
We've separated our parameters by a comma and a space, similar to what we do when listing items in English, although we don't need to use "and" like we do in English, and there is only one rule to remember: a comma after every item, except the last.
Our parameters are enclosed in parentheses () but why? This tells NVGT what we'd like to do with the alert function: we would like to call it (calling meaning running the code inside based on some values). Finally, we end the line with a semicolon ; that tells NVGT this piece of code is finished.
Together, alert("hello", "Hello, world!"); is known as one statement, the smallest unit of code that can be executed. As you can see, functions aren't so daunting after all!
However, there's one missing piece of this puzzle: what's the deal with the main() function?
Actually, it's arbitrary, although extremely common. Many programming languages (rust, c, c++, java, and NVGT, to name a few) require you to use the main() function as what's called the "entry point": in other words, NVGT will call the main function, just as we called the alert function. If it doesn't find the main function, it won't know where to start, and will simply give you an error stating as much.
It's very possible to write scripts without a main function. These are sometimes called modules, and sometimes called include scripts, helper scripts, or any number of other names. In NVGT, since a module is quite a different (and advanced) concept, we call them include scripts.
There's something interesting which nvgt can do with this script: not only can you run it, but you can also pack it into an exe file, ready to be shared with whoever you wish to have it. The advantage is that the exe will obfuscate your code and protect it from bad actors, which is especially useful for multiplayer projects!
Not just that, but anyone can run your game if you compile it, whether or not they have nvgt installed on their computers.
It's easy to do: when you've selected your script in windows explorer, don't run it. Instead, press the applications key (alternatively shift+f10 if you don't have one of those) and a "compile script (release)" option should appear.
Click it, and a new file should appear next to the script: hello.exe.
Running this file, you should get a similar effect to just running the script, but the code is now protected and can no longer be accessed.
Now that we've learned the basics of nvgt scripts, let's move on and create a simple program!
Learning Project: Calculator
Let's make a calculator program with nvgt. We should be able to type mathematical expressions into it, and then get results out. We'll then build on our program by adding more features, and hopefully learn lots along the way!
You'll encounter many new programming concepts at once, so don't feel discouraged if you are lost. All will be explained!
In this chapter, we'll learn about many of the fundamental concepts of nvgt, and of programming as well! I will try to make this interesting, but learning the basics can admittedly be boring. It is not shameful if you need a coffee or two reading through this.
And once we've learned the basics, we'll create several versions of our project, each one better than the last!
I will first explain the nvgt and programming concepts we'll need to understand it, using some short examples. They are called snippets, and won't all actually run. They are just to demonstrate.
comments
Sometimes, you might want to document what a particularly complex bit of code does, or perhaps remind yourself to make it better later.
For these reasons and more, programming languages usually have some way of keeping some notes in your code that they will not attempt to execute.
These are usually called comments, and nvgt has three types of them:
The single-line comment is written with a //, and looks like this:
// Hello!
The multi-line comment is written slightly differently, enclosed by a /* and a */. It looks like this.
/*
Hello!
This is a multiline comment
Lorem ipsum
Dolor sit amet
*/
Lastly, the end-of-line comment is sort of like a single-line comment, but goes after a line of code. For example:
int drink_price=5; // should this be able to store fractional values?
As well as documenting your code, comments (especially multi-line ones) can be used to surround code which temporarily doesn't need to be run, or is broken for some reason.
This is called "commenting out" code. Code which is commented out will be ignored by the compiler, just like regular textual comments.
Example:
void main(){
//alert("hello", "hello, world!")
alert("how are you", "My name is NVGT!");
}
If you run this, you will only see the "how are you" dialog box, and not our "hello" one. You see, I made a little error when I was writing that line - can you spot why we might have commented it out?
include scripts
The truth about programming, as with anything, is that organization is the most important thing. Always remember this: reading code is more difficult than writing it.
As a programmer, your highest-priority task is to get the code working. But arguably just as important is to keep it working.
Imagine you are a version of yourself, with similar programming experience. However, you have never read your project's code before. If you can intuitively understand what all of its components do, then you have succeeded in writing readable code!
Another thing experienced programmers like to have is modularity. This philosophy dictates that as little of your code as possible should depend on other code.
I'll explain more about this later, but an include script is how one achieves modularity in nvgt: by using the #include directive at the top of your program, you can load in another file of code, adding its code to your own in doing so.
NVGT ships with a host of include scripts (or includes for short), which you are free to use to speed up the process of game development.
For example, the speech.nvgt include has functions for making your game speak out text, without needing to worry about what screen reader or sapi voice the user has.
If you wanted to use the speech.nvgt include, you would put it at the very top of your script, like this:
#include "speech.nvgt"
Why the #? In nvgt, # signifies what's called a preprocessor directive: usually one line of code, these are used before your program is run to change something about it.
NVGT has what's called an include path. It searches multiple folders for your include scripts, and if it can't find them, it will give you an error. It works a bit like this:
- Search the directory of the script from which another script was included
- Search the include folder in nvgt, in which all the built-in includes are stored. [add more later here maybe]
Here is a full code example, which you can copy into an nvgt script and run.
#include "speech.nvgt"
void main(){
speak("Hello from the NVGT speech include!");
}
variables
If you know about variables in algebra, then you will have a similar concept in your head to those in programming. They are, at their core, names of data.
Quite a few things in nvgt are variables: functions, normal variables, classes and many more.
In this section we will learn about how to make some simple variables, and how to change them.
integers (ints) and unsigned integers (uints)
In NVGT, there are two main types of numbers, integers and floating-point numbers.
The easiest to understand is an integer, otherwise known as a discrete number. We'll learn about those first, then move on to floats, the type we will make the most use of in our calculator project.
Here are some examples of declaring the two kinds of integers:
int x = 3;
x = -3;
uint y = 3;
y=-3; // Oh no!
As you can see, we used both kinds of integers. One is called an int, as we'd expect, but the other is called a uint. What does the u mean? You might have already guessed!
We'll talk about that in a second. And if you don't want to learn about binary now, it's enough to know that unsigned ints sacrifice the ability to store negative values for double+1 the positive number range.
First, let's break down our int declaration statement
int is the type of variable (int)
x and y are the "identifier" of a variable. In other words, its name.
= is the assignment operator. The first time you assign a value to a variable, it is called initialization.
After the =, a value is assigned to the variable. Then, the ; is used to end our statement, making it complete and ending the line.
You'll notice that only the first reference of a variable needs its type; this is because this is the declaration of the variable, whereas the second line is a reassignment of the same variable and does not need to be re-declared.
You can also declare what are called global variables. I'll give a full example to demonstrate this.
int unread_emails = 20;
void main(){
alert("important", "You have " +unread_emails + " unread emails!");
}
As you can see, despite the fact that the global variable was not declared within the function (or its scope), we can still use it. This is because it's not declared in any function, and can thus be used from all functions in your program.
This author personally does not recommend much usage of globals. A more elegant way to use variables in lots of different functions at once will be demonstrated shortly. The exception is consts, which will also be discussed later.
To create the message, I used the string concatenation operator to glue one string, one variable, and another string together. This will be further explained in the section on Strings.
As we know, inside a function (but not outside) variables can be changed, or re-assigned. You might have realized that changing a variable's value in relation to itself (like giving the player some money or points) is a handy side effect, since you can simply reference a variable when re-assigning it, like this:
int level = 2;
level = level + 1;
But there is a simpler, more readable way of doing this, which saves lots of time (and cuts down on typos!).
If you want to change a variable in relation to itself like this, you use what's called compound assignment.
This combines an operator, like an arithmetic operation or string concatenation, with the assignment operator.
For example, we could rewrite our previous code using compound assignment:
int level = 2;
level +=1;
As you can see, it's much cleaner!
Here's a full example to consolidate what we've learned. You can copy and paste it into an nvgt script and run it. We'll also demonstrate includes and comments again!
#include "speech.nvgt"
int g = 3; // a global variable
void main(){
/*
This program demonstrates integers in NVGT, by declaring one (a) and performing a variety of arithmetic operations.
After each operation, the value of a will be spoken.
*/
int a = 0; // This is the variable we'll use.
speak("a is now " + a);
a+=2;
speak("After adding 2, a is now " + a);
a*=6;
speak("After multiplying by 6, a is now " + a);
a /=3;
speak("After dividing by 3, a is now " + a);
//something new!
a -= g;
speak("After subtracting g, a is now " + a);
}
bonus: binary 1100101 (or 101)
To understand signed and unsigned integers (and to understand integers) we must first understand binary. Otherwise known as base 2, it is a system based on 0s and 1s. It's most well known for appearing on terminals in poorly written movies about hackers, but it is integral to understand it as a programmer, so let's talk about it.
The unsigned integer version of binary is actually easier to explain, so we'll start with that one.
Consider a row of bits. In base 2, we can already surmise that the maximum possible value is 2^bits-1. NVGT's ints are 32-bit, although it does also support int64 and uint64 types if you want them.
The unsigned integer (uint) type in nvgt, thus, can store a maximum value of 4.294 billion. This is a huge number, and is suitable for most, if not all, requirements. The unsigned 64-bit integer type can store a value of up to 18.446 quintillion, which is more than two billion times the world population and more than a thousand times the amount of money in circulation in the entire economy, in US cents.
The first bit on the left is worth 2 raised to the n-1th power, where n is the number of bits.
If the bit is set to 0, it means no value is added to the total in base 10. If it's set to 1, you add its worth.
From left to right, each bit is worth half the bit before it. Let's give examples with 8 bits, since that's much easier to think about than 32 bits.
Consider this set of bits: 01100101
The leftmost bit in this group would be worth 128, since that's the value of 2^(8-1).
But it's set to 0, so we don't do anything
Right another bit, and we get a bit of worth 64, set to 1. So, we add 64.
Next, another 1 bit, with a value of 32, which we add, giving us 96.
Next, two 0 bits, each worth 16 and 8. We will ignore them.
Another 1 bit, worth 4. Let's add it, for a total of 100.
Then, a 0 bit, worth 2, and another 1 bit, worth 1. Adding the last 1 bit, we have our final total, 101.
This is all you need to know about unsigned binary representation.
float variables
The main difference between ints and floats is that floats can store fractional values, whereas ints are restricted to exclusively whole numbers. Although they do come with some drawbacks, this makes floats more suitable for tasks where a high level of precision is needed. They are also useful for dealing with imprecise, but extremely large, numbers.
There are two main types of floats in nvgt: float and double.
Float is a 32-bit (or single-precision) variable, and double is a 64-bit variant which can store a greater number of decimal digits and higher exponents.
In most cases, you should be okay to use a double, but this is not always required and is often not a good choice anyway.
The inner workings of floats are beyond the scope of this tutorial, but it's enough to know that computers don't think about fractional values like we do: the concept of decimal does not exist to them.
Instead, they use a binary representation, called the IEEE754 standard.
You cannot rely on floats storing a number perfectly. Sometimes, the IEEE754 standard has no exact representation for a number, and its closest equivalent must be used instead.
To demonstrate this, run this script. The result should be 1.21, but it isn't.
#include "speech.nvgt"
void main(){
double result = 1.1 * 1.1;
screen_reader_speak(result, false); // implicit cast from double to string
}
As you can see, the value is very close, but not quite right. We even used the double type, with 64 bits of precision, but it wasn't enough.
There are several ways to get around this, but we don't need to worry about them for this project, so let's learn about another very useful type of variable: strings!
string variables
The easiest and most reliable way to think about string variables is that they are text. "hello" is a string, "this is a test" is a string, and "1" is a string (this last one is confusing but will be explained shortly).
We have actually seen string variables before. When we were making our hello world program, we used two of them for the title and text parameter to the alert function.
Now knowing about variables and their identifiers, you can probably see why we used quotes (") around them, and why that is necessary.
If we hadn't, "hello, world!" would've ended up being interpreted as two function parameters, the variable identifiers hello and world, neither of which existed in the program.
NVGT would not have liked this at all; in fact, it would've thrown numerous errors our way in response.
So, quotes must enclose strings to let NVGT know that it should ignore the text inside - the computer doesn't need to know that it's text, only that it's data like text, which it can then show to the user for them to interpret.
It's almost, if not quite, like delivering letters: you don't know or care about the information in a letter (and if you did, then your manager probably needs to talk to you!) but the letter's recipient does.
In the same way, NVGT will happily place text in quotes in strings, which can then be passed to functions, put into variables, or concatenated onto other variables or strings.
In this case, it was assigning them to the title and text variables, arguments of the alert function.
String variables are created using a similar syntax to the int variable we just saw:
string name = "Rory";
You can also create a string with multiple words:
string message = "How are you today?";
Or even a string with non-ascii characters:
string message2 = "Hello, and 你好 👋";
Just like integer variables, strings also have operations which can be performed on them. By far, the most common is concatenation.
Concatenation is the process of stringing strings together. The + symbol is used for this, and compound assignment with += is also supported.
Let's see how it works:
string sentence = "The quick brown fox";
sentence += " jumps over the lazy dog";
To give you some familiarity with string concatenation, let's go over a full example. Copy this into an NVGT script, and run it:
#include "speech.nvgt"
void main(){
int a = 1;
int b = 2;
string c = "1";
int d = 2;
string result1 = a + b;
string result2 = c + d;
speak("a + b is " + result1);
speak("c + d is " + result2);
}
The output should be:
a + b is 3
c + d is 12
Is that what you expected?
What's happening here is called casting, specifically implicit or automatic casting. In programming, casting means converting some value of one type (int) to another type (string).
When we calculate result1, we perform addition on a + b (1 + 2) and get 3, which makes sense.
But when we calculate result2, NVGT automatically converts d, an int, into a string, making it "2", so it can be concatenated.
It then ignores the data inside, and just adds it together. So, instead of having 1 + 2, you get "1" + "2" - which is just two pieces of data combined together into one string, making "12".
This is why strings are so useful: they can hold any data in a text-like form, and then the meaning of that data can be interpreted by something else. In this case, it is output to our screen readers, and we can interpret it; the sound waves by themselves do not hold any semantic information.
boolean variables (bool)
Leading up to a powerful idea called conditionals, boolean variables are another fundamental datatype in programming, and are usually present in some form in programming languages.
Named in honour of the mathematician and logician George Boole, the variables can store only two possible values: true or false - or 1 or 0, on or off, yes or no.
They are extremely powerful and there are quite a few things you can do with them, but most of them don't really make sense without conditionals. Still, we can give a basic example, using not (!), a logical operator:
void main(){
bool state = true;
speak("State is: " + state);
state = !state;
speak("Flipped, state is: " + state);
}
This shows how to declare a bool: it's fairly similar to other variables. Unlike strings, the values true or false do not need to be put in quotes, despite the fact that they are not variables in the traditional sense. These variables are actually consts, which means you can never accidentally overwrite their values; trying will yield an error.
That's all we'll learn about variables for now. We'll come back to them later on, but for our calculator project, this is all that we'll need to know.
Const Keyword
For this project, the last thing we will explore regarding variables is consts.
Const variables (or constants) are variables which can never change after they are first assigned. This must be done when they are initialized.
They are particularly useful in avoiding "magic numbers": using numbers directly in our code is bad!
Let's write some code to demonstrate:
#include "speech.nvgt"
void main(){
speak("Welcome to the camping store! You need to have 30 dollars to buy a folding chair.");
}
This looks fine, but we're using this value in only one area of our code (mostly because there is only one place in which to use it, but that's beside the point).
Suppose, now, that we use the value 30 in many areas: not just telling the user how much it is to buy a chair, but also for the logic of buying and selling them itself.
This is also valid in that it works, but it is frowned upon.
Consider this: inflation is making everything more expensive these days, so what if we need to raise this price to 35 dollars next year?
The answer to that question is that it would be a coding nightmare! We would have to go through our code, painstakingly changing every reference of the value 30 to 35. But we could get it wrong: we might accidentally make one of the values 53, or change the number of centimeters in a foot from 30 to 35 in our frantic search - wouldn't it be much better if we only had to change the value once?
This is where consts will save the day!
Using them , we could rewrite our code like this:
#include "speech.nvgt"
const int price_folding_chair = 30;
void main(){
speak("Welcome to the camping store! You need to have " + price_folding_chair + " dollars to buy a folding chair.");
}
Much better! Now we can use this value wherever we want to, and all we have to do to change it is update the one declaration at the top of the file!
conditionals and the if statement
The if statement is perhaps one of the most overwhelmingly useful pieces of code. Its ubiquity is almost unparalleled and a variation of it can be found in just about every programming language in use today.
Say you want to run some code, but only if a specific thing is true - the if statement is what you use. Let's give a full example:
#include "speech.nvgt"
void main(){
int dice = random(1, 6);
if(dice == 1)
speak("Wow, you got " + dice + "! That's super lucky!");
else if(dice < 4 )
speak("You got " + dice + " - that's still pretty lucky, but aim for a 1 next time!");
else
speak("Ah, better luck next time. You got a " + dice + ".");
}
This small dice game is very simple. Roll a dice, and see how low a number you can get.
There are three options: you get a 1 (the luckiest roll), 2 or 3 (the 2nd luckiest), or 4-6 (which means you lose).
We can express these three options using the "if", "else if", and optional "else" constructions. They work like this:
if(conditional)
statement/block
else if(conditional)
statement/block
else if(conditional)
statement/block
else
statement/block
These are slightly different than normal statements, because they do not always have the ; where you would expect, at the end: if multiple lines are to be run (a block, surrounded by braces) then you use the ; on the last line of the block, before the }.
But what is a conditional? A condition is any value that returns either true or false. This can be something which is already a bool variable, or the result of a comparison operator. There are six comparison operators in nvgt, each of which takes two values and returns true or false based on their relationship.
| Operator | Purpose | Opposite |
|---|---|---|
| == | Checks if two values are exactly equal | != |
| != | Checks if two values are not exactly equal | == |
| <= | Checks if a value is less than or equal to another | > |
| >= | Checks if a value is greater than or equal to another | < |
| > | Checks if a value is greater than another | <= |
| < | Checks if a value is less than another | >= |
There are also four logical operators which work on bools, one of which we explored in the section on booleans. They are:
- && (and) returns true if the bools on the left and right are both true, but not neither or just one
- || (or) returns true if either or both of the bools on the left or right are true
- ^^ (xor) returns true only if one of the left and right bools is true, but not neither or both
- ! (not) returns the opposite of the bool on the right (it takes only one bool and nothing on the left)
Using these comparison operators and logical operators is how one creates conditionals to use in if statements, though remember that bools themselves can already be used directly.
This is all a lot of information at once, so here's a full example to demonstrate:
#include "speech.nvgt"
void main(){
int your_level = 5;
int monster_level=10;
int your_xp=50;
bool alive=true;
if(alive)
speak("You are alive!");
if(monster_level<=your_level){
speak("You manage to kill the monster!");
}
else{
speak("The monster is higher level than you, so it kills you!");
alive=false;
}
if(!alive)
speak("You are no longer alive!");
if(your_level*10==your_xp)
speak("Your level is equal to a tenth of your experience points.");
}
This example demonstrates an important distinction we touched upon earlier: if statements which use a block instead of a single statement need to have the block surrounded by braces.
Where a block might be used, a single statement can usually be used as well; the exception is functions, which must always use a block, whether or not it is a single line.
loops
While the if statement is used to create conditional checks for whether certain pieces of code will be run, loops are used to repeat sections of code more than once.
Loops have many uses, including infinite loops (which are already a well-known concept), loops that run a known number of times, and loops that we use to run through certain data structures like arrays.
For advanced programmers, NVGT does not support iterators; the traditional c-style for loop must be used to iterate through arrays, although this is substantially more convenient because one can query their length.
NVGT has three main types of loop, which we will discuss in this section: while loops, while-do loops, and for loops. We will also discuss the break and continue statements.
While Loops
The most simple form of loop is the while loop. It is written almost exactly the same as the if statement, and runs exactly the same, except that it will check the condition and run the code over and over, checking before each run, until the condition is determined to be false.
Here is an example which you can run:
#include "speech.nvgt"
void main(){
int counter = 1;
while(counter <6){
speak(counter);
wait(1000);
counter+=1;
}
}
This should speak out the numbers 1 through 5, with a second's delay between each.
Just like if statements (as well as other types of loops), a while loop can also be written with only one line of code inside. If so, it does not need to be surrounded by braces.
The while loop is the standard way to write an infinite loop: a piece of code that will run forever, perhaps until broken out of with the break keyword.
To make an infinite loop, we simply use
true
as the condition, since it will logically always return true, every time.
Do-While Loops
In while loops, the condition is checked before the code is run. This means that, just like an if statement, there is always the possibility that the code may never run at all.
On the other hand, in do-while loops, the condition is checked after the code has run. As a result, code will always run at least one time.
It's often up to the programmer which type they think is best, and there is no real standard. Whichever is more convenient and maps best to the situation can be used, and neither has a performance advantage over the other.
Let's rewrite our counter example using a do-while loop:
#include "speech.nvgt"
void main(){
int counter = 1;
do{
speak(counter);
wait(1000);
counter+=1;
} while(counter < 6);
}
As you can see, since the condition is at the end, we need to check it based on the value after the counter as updated, instead of before.
If we had used
while(counter < 5);
as we had done with our while loop, the code would have only counted to 4, since the counter gets updated after it speaks its current value.
For Loops
One of the most-used types of loop is something similar to our counter example from earlier in this chapter.
There are a lot of lines here which for loops can compress into just one line; they make code easier to write as well as read later on.
For loops also have an additional unique property, which is extremely useful, but will be discussed in a moment after some background.
However, they are admittedly difficult to grasp at first, because they're very different than both the while and do-while loops.
Consisting of four parts, a for loop might look like this:
for(declarations; condition; final)
statement/block
How it works:
- The declarations code is run, and a variable (or more) is declared.
- The loop starts to run.
- At the beginning of each loop iteration, the condition is checked. If false, the loop stops, potentially even before it has run once.
- Assuming the condition is true, The code inside the for loop is run.
- After that code has finished, the final step is run.
- Repeat steps 3-5
Note: all three parts of a for-loop are optional. For instance, you may omit the declaration by simply writing nothing before the first ;
As you can imagine, a for loop would help us rewrite our counter example in a much more readable way.
Let's go ahead and do this now:
void main(){
for(uint i = 1; i < 6; i ++){
screen_reader_speak(i, false);
wait(1000);
}
}
This example will yield the same results as the previous one, but it does it in a way which is more concise. Pretty code is very important for your sanity!
Break And Continue Statements
There are times when you might want to get out of loops completely. This is called "breaking out", and it's very easy to do:
break;
There are no additional components to this statement, as there are in some other languages such as rust.
This is particularly useful in infinite loops (typically created by simply using true as the condition in a while loop)
When this is done, the loop immediately ends, and will iterate no longer.
If you want to break out of a loop, but still want it to try and run again (IE. not run the rest of this iteration), the continue statement can help.
As is the break statement, the continue statement is simply written on a line by itself:
continue;
This immediately ends the current iteration. For while and do-while loops, nothing more happens, and the final component in a for loop is run. Then, the next iteration may begin.
The reason that for loops behave differently is by design. It is not possible to achieve this behaviour in any other way.
functions
Functions in nvgt are pieces of code that can be "called" or executed by other parts of the code. They take parameters (or arguments) as input, and output a value, called the "return value".
The return value is so named because it gives some critical piece of information back to the calling function, without which work cannot continue.
Let's use baking a cake as an example: in the process of baking a cake (simplifying greatly) you put the batter in the oven. The oven cooks the cake, and then you open it up and take a cooked cake out.
You can't do something else related to baking a cake while the oven is baking, because you don't have the cake: the oven does. In the same vein, only one function can be running at once, and we say that function is currently executing.
When a function ends, execution returns to the function from which it was called, just like taking the cake back out of the oven. It returns, along with execution, whatever is specified using a return statement (discussed shortly).
Here is a snippet for the purpose of example:
int add(int a, int b){
return a + b;
}
Frankly, this code is a needless abstraction over an already-simple task (the addition operator) and you should almost never do it like this in production. Nonetheless, it's a good example of what a function can do. It takes data in, and then outputs some other data.
The way we declare functions is a little bit strange, and this example is packed with new ideas, so let's break it down piece by piece:
int add(
The beginning of the function's declaration, letting the compiler know that it is a function with the return datatype (int), the name (add) and the left parenthesis
int
the type of the first variable
a,
the name (identifier) of the first parameter/argument variable, and a comma and space to separate it from the next variable, just as we use when calling functions
int b)
The declaration of the second parameter/argument variable, another integer, and a right parenthesis to tell the compiler there are no more parameter variables
{
The beginning of our function (remember, there must always be braces enclosing functions, even if the execution step is only one line in length.)
Then, the next line:
return a + b;
This is the only line in our function, and returns an expression evaluating to an int variable. It adds the a and b variables' values together.
}
The end of our function, which lets the compiler know that we're back in the outer scope (probably global)
Bonus: Some notes About References (Advanced)
If you are new, you can skip this brief section for now, as it's discussed in another article in great depth for easier understanding.
There are a couple of common misconceptions and mistakes made by even experienced coders when it comes to function parameters.
For performance, it may seem intuitive to declare your primitive functions as const x&in references, but this is almost always useless, except in the case of strings.
The reason for this is the fundamental property of a reference: a pointer itself is a value of 8 bytes storing a memory address. As opposed to most primitives (ints etc), there is no advantage, as the new bytes still need to be allocated - it is just a different value that is placed into them (the address instead of a copy of the value).
Arrays (Lists)
If you have programmed prior to reading this manual, you may have seen them before: an array is angelscript's equivalent to a vector or a list.
Via this powerful data structure, we can store lots of the same type of variable in one neat package.
In NVGT, arrays are dynamic. This means that you can add and remove from them at any time you'd like, as opposed to arrays in c or rust, which are trickier to expand or remove from.
Before we move on, here is a quick example:
void main(){
int[] scores = {20, 30};
scores.insert_last(50);
screen_reader_speak("The scores of the players in this game are:", false);
for(uint i = 0; i < scores.length(); i ++)
screen_reader_speak(scores[i], false);
}
The variables in an array are called "items" or "elements", and you can imagine them as being stored in a line (ordered).
As a consequence, we can access the 1st, 2nd, or any other item in our line of elements, via "indexing".
Slightly more complicated to think about, however, is that arrays in NVGT use "0-based indexing".
This is true of most programming languages, with a notable exception being lua.
0-based indexing means that, instead of the 1st item being at index 1, it is at index 0. Item 2 would be at index 1, item 3 at 2, and item 20 at 19.
To be Continued
Unfortunately, this game development tutorial is not yet complete, you should stop reading at this point if you are a beginner.
- classes (methods and properties)
- datastreams and files
Toolkit Configuration
One highly useful aspect of NVGT is it's ever growing list of ways that an end user can configure the engine. Whether trying to add an extra include directory, tweak an Angelscript configuration property, choose how warnings and errors are printed, deciding whether to print output on successful compilation or any number of other changes, NVGT provides various options and directives that should help you to get it functioning much closer to the way you desire.
From #pragma directives to command line argument syntax to configuration files, this document attempts to keep track of all such available configuration options NVGT has to offer.
Command line arguments
NVGT's compiler program has a variety of command line arguments you can pass to it which can alter it's behavior. Some of these options can also be set in configuration files, however an explicit command line option is very likely to override any contradictory options that are in a configuration file.
Generally, these command line options are self-documenting. If you open a command prompt or terminal and just run the nvgt application directly with no arguments, you will be informed that you should either provide an input file or run nvgt --help for usage instructions. You can also actually just run nvgt -h
In almost all cases, command line arguments have 2 methods of invocation, both a short form and a long form. The short form of a command usually consists of just a letter or two and is easier to type, while the long form of a command is always more descriptive and thus might be suited to an automation script where you want anyone reading such a script to be able to understand exactly what they are asking NVGT to do. What form you wish to use is completely up to you. In either case you should put a space between arguments, and mixing short and long form arguments is completely fine.
The short form of an argument always begins with a single hyphen character (-) followed by one or 2 letters. For example, -C would tell NVGT to compile your script with debug information. Some arguments take values, in which case the value shall follow directly after the argument when using short form invocations. For example you could set the platform to compile with using the -pwindows argument.
On the other hand, the long form of an argument begins with two hyphens followed by what usually amounts to a few words separated by hyphens. Usually it's just one or 2 words, but could be more in rare cases. For example, --compile-debug would tell NVGT to compile the given script with debug information. If an option takes an argument, you use an equals (=) character to define such a value. For example --platform=windows would tell NVGT to compile a script for the windows platform.
Finally, a special argument, two hyphens without any following text, indicates that any further arguments should be passed onto the nvgt script that is about to run.
Argument list
The following is a list of all available command line arguments, though note that it is best to directly run nvgt --help yourself encase this list is in any way out of date as nvgt's --help argument will always be more accurate because the text it prints is dynamically generated.
- -c, --compile: compile script in release mode
- -C, --compile-debug: compile script in debug mode
- -pplatform, --platform=platform: select target platform to compile for (auto|android|windows|linux|mac|ios)
- -q, --quiet: do not output anything upon successful compilation
- -Q, --QUIET: do not output anything (work in progress), error status must be determined by process exit code (intended for automation)
- -d, --debug: run with the Angelscript debugger
- -wlevel, --warnings=level: select how script warnings should be handled (0 ignore (default), 1 print, 2 treat as error)
- -iscript, --include=script: include an additional script similar to the #include directive
- -Idirectory, --include-directory=directory: add an additional directory to the search path for included scripts
- -V, --version: print version information and exit
- -h, --help: display available command line options
Configuration files
Both because command line arguments can become exhausting to type for every script execution, and because NVGT contains far too many configuration options to make command line arguments out of, NVGT also supports configuration files which can be used either on a global or a project level to further direct NVGT.
Configuration files can either be in ini, json or properties formats and are loaded from multiple locations. Values in these configuration files are usually separated into sections in some form. Settings controlling the user interface typically go in the application subsection, while directives that provide control over Angelscript's engine properties are in a subsection called scripting. The way these directives in various subsections are defined depends on the configuration format you choose, however you can use any combination of configuration formats at once to suit your needs. Though all supported configuration formats are more or less standard, you can find examples of and notes on each format below.
Configuration files at any given location have a priority attached to them to resolve directive conflicts. For instance, if a global configuration file and a project configuration file both contain the same option, the project file takes priority.
Loaded configuration files
The following configuration files are loaded listed in order of priority from highest to lowest:
- Script-dependent project configuration: script_basename.ini, script_basename.json or script_basename.properties in the same directory as the nvgt script about to be run, where script_basename is the name of the nvgt script without an extension (mygame.nvgt = mygame.properties)
- Script independent project configuration: .nvgtrc (ini format) in the same directory containing the nvgt script that is getting executed or in any parent
- Compiler dependent global configuration: exe_basename.ini, exe_basename.json and exe_basename.properties in NVGT's installed directory or any parent, where exe_basename is the name of nvgt's running executable without the extension (for example nvgt.json or nvgtw.ini)
- Compiler independent global configuration: config.ini, config.json and config.properties in NVGT's installed directory.
Supported configuration formats
NVGT supports 3 widely standard configuration formats, all of which have their own advantages and disadvantages. It is perfectly acceptable to create multiple configuration files with the same name but with different extension, all keys from all files will load and be combined. If the files nvgt.json and nvgt.ini both exist and both set the same key to a different value, however, the result is a bit less defined as to what key is actually used and typically depends on what file was internally loaded first by the engine.
It is entirely up to you what formats you want to use in your projects.
json
{"application": {
"quiet": true
}, "scripting": {
"compiler_warnings": 1,
"allow_multiline_strings": true
\}\}
Likely the most widely used and most modern, NVGT can load configuration options from standard JSON formatted input.
This format receives the most points for standardization. Included directly within many higher level programming languages as preinstalled packages or modules, this is the easiest format to use if you need some code to generate or modify some configuration options used in your program. For example, a python script that prepares your game for distribution might write a gamename.json file next to your gamename.nvgt script that modifies some Angelscript engine options or changes debug settings before release, and sets them to other values during development.
Furthermore if NVGT ever includes an interface to set some of these options in a manner where they save, .json files would be the format written out by the engine.
The disadvantage to JSON though is that even though it is the most modern and widely recognised of nvgt's options, it's relatively the least human friendly of the formats. For example if a configuration option takes a path on the filesystem and one wants to manually set this option in NVGT, they would need to escape either the slash or backslash characters in the path if they go with the JSON option. Furthermore NVGT contains many boolean options that are just enabled by being present, where an option key has no needed value. While JSON of course supports setting a value to an empty string, it is certainly extra characters to type in order to just quickly tweak an NVGT option.
ini
[application]
quiet
[scripting]
compiler_warnings = 1
allow_multiline_strings
NVGT can also load configuration data from good old ini formatted text files.
This format is probably the least standardized of the ones NVGT supports, likely due to it's origin from closed source windows. While on the surface it is a very simple format, there are a few deviations that exist in various parsers as the syntax is not as set in stone as other formats. For example whether subsections can be nested, whether strings should be quoted, whether escaping characters is supported or whether a value is required with a key name are all up to an individual ini parser rather than a mandated standard. In this case, the ini parser that handles these configuration files does not require that a value be provided with a key name for simple boolean options, escaping characters is not supported and strings do not need to be quoted.
The biggest advantage to this format as it pertains to NVGT is the simplicity of setting several options in the same section without constantly redeclaring the sections name. One can just declare that they are setting options in the scripting section and can then begin setting various engine properties without typing the name "scripting" over and over again for each option.
The biggest disadvantage to this format is the inability to escape characters. For example, typing \n in a configuration string would insert the text verbatim instead of a newline character as might be expected.
properties
application.quiet
scripting.compiler_warnings=1
scripting.allow_multiline_strings
Finally, NVGT is able to load configuration options from java style .properties files.
The biggest advantage to this format is it's easy simplicity. The format just consists of a linear list of key=value pairs, though the =value is not required for simple boolean options that just need to be present to exist. Unlike ini, it also supports character escaping, causing \n to turn into a line feed character.
The only real disadvantage to this format over ini is the need to keep constantly redeclaring the parent section names whenever defining keys, for example you need to type scripting.this, scripting.that where as with ini and JSON you can specify that you are working within a section called scripting before adding keys called this and that.
Available configuration options
Many options listed here do not require a value, simply defining them is enough to make them have an effect. Cases where a value is required are clearly noted.
The available options have been split into sections for easier browsing. While the method for defining options changes somewhat based on configuration format, a line in a .properties file that contains one of these options might look like application.quiet or scripting.allow_multiline_strings, for example.
application
This section contains options that typically control some aspect of the user interface, as well as a few other miscellaneous options that don't really fit anywhere else.
- as_debug: enables the angelscript debugger (same as -d argument)
- compilation_message_template = string: allows the user to change the formatting of errors and warnings (see remarks at the bottom of this article)
- GUI: attempts to print information using message boxes instead of stdout/stderr
- quiet: no output is printed upon successful compilation (same as -q argument)
- QUIET: attempts to print as little to stdout and stderr as possible (though this option is still a work in progress)
build
This section contains options that are directly related to the compiling/bundling of an NVGT game into it's final package. It contains everything from options that help NVGT find extra build tools for certain platforms to those that define the name and version of your product.
- android_home string defaults to %ANDROID_HOME%: path to the root directory of the Android sdk
- android_install = integer default 1: should Android apks be installed onto connected devices if signing was successful, 0 no, 1 ask, 2 always
- android_jaava_home string defaults to %JAVA_HOME%: path to the root of a java installation
- android_manifest string defaults to prepackaged: path to a custom AndroidManifest.xml file that should be packaged with an APK file instead of the builtin template
- android_path string defaults to %PATH%: where to look for android development tools
- android_signature_cert = string: path to a .keystore file used to sign an Android apk bundle
- android_signature_password = string: password used to access the given signing keystore (see remarks at the bottom of this article)
- linux_bundle = integer default 2: 0 no bundle, 1 folder, 2 .zip, 3 both folder and .zip
- mac_bundle = integer default 2: 0 no bundle, 1 .app, 2 .dmg/.zip, 3 both .app and .dmg/.zip
- no_success_message: specifically hides the compilation success message if defined
- output_basename = string default set from input filename: the output file or directory name of the final compiled package without an extension
- precommand = string: a custom system command that will be executed before the build begins if no platform specific command is set
precommand_<platform>= string: allows the execution of custom prebuild commands on a per-platform basisprecommand_<platform>_<debug or release>= string: allows the execution of custom prebuild commands on a per-platform basis with the condition of only exicuting on debug or release builds- postcommand = string: a custom system command that will be executed after the build completes but before the success message
postcommand_<platform>= string: allows the execution of custom postbuild commands on a per-platform basispostcommand_<platform>_<debug or release>= string: allows the execution of custom postbuild commands on a per-platform basis with the condition of only exicuting on debug or release builds- product_identifier=string default com.NVGTUser.InputBasenameSlug: the reverse domain bundle identifier for your application (highly recommended to customize for mobile platforms, see compiling for distribution tutorial)
- product_identifier_domain = string defaults to com.NVGTUser: everything accept the final chunk of a reverse domain identifier (used only if build.product_identifier is default)
- product_name=string defaults to input file basename: human friendly display name of your application
- product_version = string default 1.0: human friendly version string to display to users in bundles
- product_version_code = integer default (UnixEpoch / 60): an increasing 32 bit integer that programatically denotes the version of your application (default is usually ok)
- product_version_semantic = string default 1.0.0: a numeric version string in the form major.minor.patch used by some platforms
- shared_library_excludes = string default "plist TrueAudioNext GPUUtilities systemd_notify sqlite git2 curl": Partial names of shared libraries that should not be copied from NVGT's source lib directory into the bundled product.
- shared_library_recopy: If this is set, any shared libraries will be copied from scratch instead of only if they're newer than already copied.
- windows_bundle = integer default 2: 0 no bundle, 1 folder, 2 .zip, 3 both folder and .zip
- windows_console: when compiling for windows, build with the console subsystem instead of GUI
scripting
This section contains options that directly effect the Angelscript engine, almost always by tweaking a property with the SetEngineProperty function that Angelscript provides.
The result is undefined if any value is provided outside suggested ranges.
For more information on these properties, the Angelscript custom options documentation is a great resource. There are some engine properties shown on that page which NVGT does not support the configuration of, as a result of such properties being used internally by the engine to function properly.
- allow_multiline_strings: allow string literals to span multiple lines
- allow_unicode_identifiers: allow variable names and other identifiers to contain unicode characters
- allow_implicit_handle_types: experimentally treat all class instance declarations as though being declared with a handle (classtype@)
- alter_syntax_named_args = integer default 2: control the syntax for passing named arguments to functions (0 only colon, 1 warn if using equals, 2 colon and equals)
- always_impl_default_construct: create default constructors for all script classes even if none are defined for one
- compiler_warnings = integer default 0: control how Angelscript warnings should be treated same as -w argument (0 discard, 1 print and continue, 2 treat as error)
- do_not_optimize_bytecode: disable bytecode optimizations (for debugging)
- disallow_empty_list_elements: disallow empty items in list initializers such as {1,2,,3,4}
- disallow_global_vars: disable global variable support completely
- disallow_value_assign_for_ref_type: disable value assignment operators on reference types
- disable_integer_division: Defer to floatingpoint division internally even when only integer variables are involved
- expand_default_array_to_template: cause compilation messages which would otherwise contain something like string[] to instead contain array<string>
- heredoc_trim_mode = integer default 1: decide when to trim whitespace of heredoc strings (0 never, 1 if multiple lines, 2 always)
- ignore_duplicate_shared_interface: allow shared interfaced with the same name to be declared multiple times
- init_call_stack_size = integer default 10: the size of the call stack in function calls to initially allocate for each script context
- init_stack_size = integer default 4096: the initial stack size in bytes for each script context
- max_nested_calls = integer default 10000: specify the number of nested calls before a stack overflow exception is raised and execution is aborted
- max_stack_size = integer default 0 (unlimited): the maximum stack size in bytes for each script context
- max_call_stack_size = integer default 0: similar to max_nested_calls but can possibly include calls to system functions (angelscript docs is unclear)
- private_prop_as_protected: private properties of a parent class can be accessed from that class's children
- property_accessor_mode = integer default 3: control the support of virtual property accessors (0 disabled, 1 only for c++ registrations, 2 no property keyword required, 3 property keyword required)
- require_enum_scope: access to enum values requires prepending the enum name as in enumname::enumval instead of just enumval
- use_character_literals: cause single quoted one-character string literals to return an integer with that character's codepoint value
#pragma directives
In a few cases, it is also possible to configure some aspects of NVGT's behavior directly from within nvgt scripts themselves using the #pragma preprocessor directive.
This directive is used to safely tell the engine about anything that doesn't directly have to do with your script code but also without causing some sort of compilation error due to bad syntax. A pragma directive could do anything, from embedding a file to selecting assets to choosing to adding include directories and more.
The syntax for a pragma directive looks like #pragma name value or sometimes just #pragma name if the option does not require a value. In some cases when a value is to contain a long or complex enough string such as a path, you may need to surround the value in quotes such as #pragma name "value."
Available directives
#pragma include <directory>: search for includes in the given directory (directive can be repeated)#pragma stub <stubname>: select what stub to compile using (see remarks at the bottom of this article)#pragma embed <packname>: embed the given pack into the compiled executable file itself#pragma asset <pathname>: copy the given asset/assets into the bundled product as resources#pragma document <pathname>: copy the given asset/assets into the bundled product as documents intended for the user to access rather than the programmer#pragma plugin <plugname>: load and activate a plugin given it's dll basename#pragma compiled_basename <basename>: the output filename of the compiled executable without the extension#pragma bytecode_compression <level from 0 to 9>: controls the compression level for bytecode saved in the compiled executable (0 disabled 9 maximum)#pragma console: produce the compiled executable on windows using the console subsystem for CLI based programs
Remarks on complex options
This section contains any explanations of topics that were too bulky to fit in the documentation of each specific configuration option or article section.
application.compilation_message_template
This useful option allows you to control the format that errors and warnings are printed in. The default template looks like this, for example:
file: %s\r\nline: %u (%u)\r\n%s: %s\r\n
Most specifically, the format string is passed to the function Poco::format in the c++ codebase, and that function receives 5 dynamic arguments. string file, uint line, uint column, string message_type, string message_content.
It is not needed to understand how this c++ string formatting function works to customize the message template however, you can just reorder the arguments and add text to the following example template:
%[1]u %[2]u %[0]s; %[3]s: %[4]s
This example moves the line number and column to the beginning of the string, before printing the filename, the message type and the content all on a single line. NVGT automatically adds one new line between each engine message printed regardless of the template.
platform and stub selection
One possible area of confusion might be how the platform and stub directives fit together. In short, the stub option lets you choose various features or qualities included in your target executable while platform determines what major platform (mac, windows) you are compiling for.
Though explaining how NVGT's compilation process works is a bit beyond the scope of this article, the gist is that Angelscript bytecode is attached to an already compiled c++ program which is used to produce the final executable. There are several versions of the c++ program (we call that the stub) available with more that could appear at any time. For example a stub called upx will produce an executable that has already been compressed with the UPX executable packer, while a stub available on windows called nc will insure that the Angelscript compiler is not included in your target application. In the future we hope to include several more stubs such as one focusing much more on performance over file size, one focusing on minimal dependencies to insure no third party code attributions are needed, etc.
Internally, you can see this at play by looking in the stub directory of NVGT's installation. You can see several files with the format nvgt_<platform>_<stubname>.bin, or sometimes just nvgt_<platform>.bin which is the default stub for a given platform. Such files are used to create the final game executable produced by NVGT's compiler, and are selected exactly using the configuration pragmas and options described. If platform is set to windows and stub is set to nc, the file nvgt_windows_nc.bin is used to produce the compiled executable of your game. The only exception is if the platform is set to auto (the default), which will cause your executable to be compiled for the host platform you are compiling on.
bundle naming and versioning
NVGT includes several directives particularly in the build configuration section that allow you to control how your product is identified to end users and to the systems the app is running on. While some are purely for display, others are manditory in certain situations especially when you plan to distribute your app on networks like the app store, google play and similar. For more info on these directives, you should read the "configuring bundling facility" section of the compiling for distribution topic.
Conclusion
Hopefully this tutorial has given you a good idea of how you can get NVGT to perform closer to how you would like in various areas from the user interface to the syntax handling. If you have any ideas as to configuration options you'd like to see added, please don't hesitate to reach out on github either with a discussion or pull request!
Compiling your project for distribution
Once your game is ready to be distributed to others, you may want to turn it into a compiled binary executable, or perhaps even a complete bundle. The compilation procedure assembles all the individual nvgt files of your source code into a single file that can run natively on its targeted operating system without having Nvgt available to interpret it, and the bundling facility can then take that binary along with your sounds/assets, documentation, shared libraries/plugins as well as any other material your game requires and compress/bundle all of that into a complete package ready for you to directly share with the world.
Here are some thoughts about reasons to do this:
- Doing this can let you make your work more of a closed-source affair which limits the ease at which someone can reverse engineer your work and hack in changes if that is something you don't want to happen, which is probably the case if your game isn't a free one for sure.
- The end user will just be able to run the executable file natively, no need to have Nvgt installed. Just like a regular program they download anywhere else!
- Fewer files, especially if you have a lot of scripts included. That all can turn into just one file, except for the libraries, which can clean up the directory considerably.
- Stability. No need to worry about whether someone's going to be trying to run your code on an older or newer version of Nvgt with script-breaking changes.
Some thoughts about reasons not to do this:
- Your project is open for all to hack and slash at, with the code readily available.
- You want people to be able to review your work for what ever reason. Since the code is right there, they can read it if they know how and determine what the code does if they so choose.
- You're creating tutorials or test scripts for others to learn by or to provide examples for how to do something or get feedback, such as trying to find a bug or teaching others how to do something in Nvgt. Hey, that's kinda the same thing as item 2!
What is an executable file?
This will be sometimes called a binary. It's a file that is a complete program directly compatible with the operating system for which it is intended. It can be launched directly, and is usually bundled along side sounds, shared libraries and other data required to run your game.
What is a bundle?
A bundle, unlike a plain executable binary, is instead a full game package which contains everything your game needs to run including any sounds, shared libraries, and the aforementioned executable file. It might be a .zip that the user extracts, a .dmg on MacOS, a .apk file for android and similar for other supported platforms.
What about all the different operating systems?
Since Nvgt is intended to be able to run on Windows, Linux, Mac OS and mobile platforms like Android and iOS, there will be different executables for each targeted operating system. On windows, the resulting binary will have a .exe extension like other programs you may have seen. On other platforms though, there is no standard file extension for executable files and on Android, you never see the executable directly but instead only the final .APK file as there is no reasonable way to run NVGT applications on mobile without fully bundling them first.
e.g. if you disable the bundling facility and then compile my_game.nvgt on Windows you'll get a my_game.exe but if you compile on linux, you'll just get a file called my_game. Keep this in mind when trying to compile on a non-windows platform. If you have a file called my_game.nvgt and next to it is a directory called my_game, the compilation process will fail because my_game already exists and is a directory!
When compiling on a non-windows platform, the executable permission bitt on the resulting file is automatically set for you.
How to compile:
- on Windows, navigate to the main script file of your code (the one with the void main () function in it). For example it might be called my_game.nvgt
- Right click the .nvgt file (you can use the context key or shift+f10) and expand the "Compile Script" sub menu. Each platform you'll see there has 2 options:
- Choose "platform (debug)" if you will want extra debugging information to be included in the program. This can be handy if you're working on it still and want to gain all information about any crashes that may happen that you can get.
- Choose "platform (release)" if you don't want the extra debug information. You probably should always use release for anything you don't want others to be able to hack and slash at, since the debug information could make it easier on them.
- If you've disabled the bundling facility, ensure that the necessary library files are located in the same directory your new exe is, or within a lib folder there.
- If you would rather, you can also run the nvgtw.exe program itself and use the buttons there to compile for any given platform.
- Right click the .nvgt file (you can use the context key or shift+f10) and expand the "Compile Script" sub menu. Each platform you'll see there has 2 options:
- On Linux, cd to the path containing a .nvgt script.
- Now run /path/to/nvgt -c scriptname.nvgt where scriptname.nvgt should of course be replaced with the name of your script file. You can select a platform with the -p argument E. -pwindows.
- On Mac OS, it's best to just click on the NVGT application directly.
- After launching NVGT you can select compile a script in release/debug mode, choose a platform, and browse to your script file.
- If, however, you wish to build on MacOS using the command line, cd to the directory containing a .nvgt script and run the command /applications/nvgt.app/Contents/MacOS/nvgt -c scriptname.nvgt, or alternatively open -a nvgt --args -c `pwd`/scriptname.nvgt
You will receive a popup or STDOut print letting you know how long the compilation took. You should find a new file in the same folder you just compiled that has the same name it does but with a different extension e.g. when compiling my_game.nvgt you'll have my_game.exe. Distribute this along with the necessary libraries in what ever form is required. You can set it all up with an installer, such as Inno Setup, or if your game will be portable you can just zip it up, etc.
Configuring the bundling facility:
NVGT tries to provide a one-click bundling facility, that is, clicking the compile option on an NVGT script will directly create a package that you can distribute to your end users with little hastle. You don't need to go searching for libraries, you don't need too learn how app bundles on MacOS or Android APK files work, you can just build your game into a distribution ready package.
While this is a very powerful system that does work out of the box, it likely needs input from you if it is to produce the most optimal results. The facility needs to know where your assets are, it is best if it knows what version your game is / a good display name string for the app (used on various platforms), and various other options allow for anything from custom app manifest files to custom system commands that you can cause to be executed before or after a successful compile.
Other than including assets like sounds, most inputs to the bundling facility are provided via configuration options. Usually a file called .nvgtrc containing configuration directives is created along side a project, though it can also be called gamename.properties, gamename.json or gamename.ini where gamename is the name of your main nvgt script without the extension. You can learn more about the configuration subsystem specifically in the toolkit configuration tutorial, which includes a list of every available bundling option.
All of the bundling related options are in the build section of the configuration. So if you want to disable the bundling facility entirely on windows for your project, for example, you might add the line build.windows_bundle = 0 to your .nvgtrc file which would cause a standalone executable to be created when the game is compiled rather than a full bundle. If the bundling facility is not for you at all, you can disable it globally by creating a file alongside the nvgt.exe application itself called config.properties with the same key, more details and alternate configuration filenames are also described in the configuration tutorial.
Many of the directives here allow you to control how your product is identified to end users and to the systems the app is running on. While some are purely for display, others are manditory in certain situations especially when you plan to distribute your app on networks like the app store, google play and similar.
If you plan to release your game on MacOS, iOS, or Android, it is highly recommended that you create what's known as a reverse domain identifier for your app by setting the build.product_identifier property. It's called the reverse domain identifier because the format is like a website in reverse, such as com.samtupy.nvgt. In mild cases it might be used to group similar apps together E. the operating system or distribution service can implicitly tell that apps with an identifier starting with com.samtupy all belong to the same company, but in extreme cases this identifier might be used by the system as the main way your app is identified, such as on Android for example where the filesystem paths to the apps data folders actually include this identifier. The identifier can contain periods, uppercase / lowercase letters, and numbers, so long as a number is not the first character following a period. There should be at least 2 period delimited segments in this identifier (prefferably at minimum 3), and it might be a good idea to limit the first segment to popular top level domains like com, org, net etc. While in reality a couple of other characters like dashes and underscores might work in identifiers, they may not be as platform agnostic as the rules listed above. It is not important that the reverse website/app identifier you come up with here actually exists on the internet. If you do not provide this value, the default is com.NVGTUser.scriptname where scriptname is the name of the NVGT file passed to the compiler without the extension. It is possible, however, to only customize the com.NVGTUser part if you want NVGT to use the script filename to derive the rest of the identifier for you. So for example if you know that all games you'll be developing on your computer will be from the company com.epicdevelopers, you can set the key build.product_identifier_domain = com.epicdevelopers in nvgt's global config files. Now when you compile mygame.nvgt, the full reverse domain identifier for that app will be set to com.epicdevelopers.mygame.
Unfortunately, different operating systems and services use different versioning schemes. Windows and MacOS really want version numbers in major.minor.patch format, in fact the windows version resource actually contains 4 binary words in it's structure to store each period separated version component. While the parsed format isn't actually embedded into a binary structure on MacOS, it's still manditory as soon as you want to distribute your game in the app store, because that service reads such a property from the bundle to identify the version of the app to track updates. As such, the build.product_version_semantic property must be set for MacOS app store distributions, while on windows it is an optional display feature that will cause version information to show for your compiled executable. On the other hand, Android and the google play store use an integer version code to track when a given version of an APK is newer than any installed version. This simple requirement makes it very easy to manage this property automatically on android, it is done by dividing the unix timestamp in seconds by 60, you can also set the build.product_version_code property if you wish to control this manually. Finally, the build.product_version configuration option allows you to set, where possible, the version string that is actually displayed to end users.
Another configuration option worth mentioning is the build.shared_library_excludes property. This defaults to the string "plist TrueAudioNext GPUUtilities systemd_notify sqlite git2 curl", meaning that by default the shared libraries for plugins are not bundled. As such, you may need to replace this property with your own list of excludes if you use any of NVGT's plugins. We intend to make this step unnecessary, instead determining the list of plugin libraries based on the plugin pragmas contained in the script. So for example if you wanted to include the curl plugin at the moment, you might set the property build.shared_library_excludes = "plist TrueAudioNext GPUUtilities systemd_notify sqlite git2" in your project's configuration file.
To include assets like sounds and documents into your bundle, the asset and document pragmas are used. A pragma is a type of code statement, similar to #include, that allows you to feed NVGT various extra information about your game. For example if one wants to bundle the sounds.dat file with their product, they can add the line of code#pragma asset sounds.dat to the top level of one of any .nvgt files that make up the project, or #pragma document changelog.txt to include a text file intended for the user rather than the game to access.
The asset and document pragmas are very similar, on most platforms indeed they do the same thing. The distinction is that in some cases it can be useful for NVGT to know whether an included asset is intended for access by the game or by the user. The changelog and readme documents for your game should be included as documents rather than assets, for example, because they are included outside the .app folder within a .dmg bundle on MacOS as opposed to assets which are placed in the MacOS/Content/Resources directory, which may be inconvenient for the user to access. Similar distinction is applied on other platforms when required/possible.
Sometimes you may have a file that you wish to include as an asset, but as a different name in the bundle than that of the file stored on disk. To do this, you can express the asset pragma like #pragma asset sounds.dat;audio.dat if you want sounds.dat to actually be called audio.dat in the bundle. The asset and document pragmas will copy/include directories recursively. For example if you have a folder called sounds, the code #pragma asset sounds will successfully include the entire sounds folder in your bundle.
Libraries needed for distribution:
The following information can be helpful if you have disabled NVGT's bundling facility and you wish to package your compiled games yourself.
There are a few libraries (dependencies) that Nvgt code relies on. When running from source, Nvgt is just interpreting your .nvgt scripts and using the libraries found in its own installation folder. Generally, if all is normal that will be:
- On Windows, c:\Nvgt
- On Linux, where you extracted NVGT's tarball to.
- On Mac OS,
/Applications/nvgt.app/Contents
Within that folder you should have a subfolder that is called "lib" (or "Frameworks" on macOS). Lib contains the redistributable dependencies Nvgt uses for certain functionality. When you run from source, the Nvgt interpreter uses this very folder. However, when you compile, you must provide the files in the same directory as the compiled program so that it can find them. You may copy the entire lib folder to the location of the compiled game, or you can optionally just copy the files directly there if you don't want a lib folder for some reason. In other words, you can have a subdirectory within the program's directory called lib with the files inside, or you can also just place the dlls right in the program's directory where the binary is. On windows, you can omit some of the dll files not needed by your code, and we plan to add more and more support for this on other platforms as time goes on. A list is provided below of the basic purpose and importance of each library so you can hopefully know what to omit if you don't want all of them. These files must be included where ever the program ends up, so installers need to copy those over as well, and they should be included in any zip files for portable programs, etc. Again remember that NVGT usually handles this for you and this step is only required if you wish to create your own bundles by hand.
At time of writing, the files are:
- bass.dll is required for the sound object and the tts_voice object, though will not be later once we switch from bass to the miniaudio library. The moment one of these objects exists in your script, your program will crash if this dll is not present.
- bassmix.dll is just as important as bass.dll and allows things like nvgt's mixer class to exist, where we can combine many bass channels into one. It is always required when bass.dll is used
- bass_fx.dll is used for reverb, filters, etc and is also required whenever bass.dll is used.
- git2.dll is the libgit2 library which allows people to programmatically access git repositories (can be good for adding version control to your online game's map world for example). You don't usually need it.
- git2nvgt.dll is the plugin for nvgt itself that wraps git2. So if your script does not include the line #pragma plugin git2nvgt, then you don't need either of the git2 dlls.
- nvgt_curl.dll is another plugin that wraps the libcurl library. It used to be the only way to do http requests, but it's now being phased out in favor of more portable options. It's only required if your script includes the line #pragma plugin nvgt_curl
- nvgt_sqlite.dll is similarly for interfacing with SQLite and is only needed if #pragma plugin nvgt_sqlite is defined.
- GPUUtilities.dll is used for the GPU acceleration of ray tracing used for geometric reverb in steam audio. It should always be completely optional.
- phonon.dll is steam audio, that's the HRTF and geometric reverb and cool things like that. You should only need it if you create a sound_environment class or set sound_global_hrtf=true in your script, but at time of writing, sound output using normal sound objects wasn't working as expected without this.
- TrueAudioNext.dll is more optional GPU acceleration for steam audio.
- systemd_notify.dll is a plugin wrapping the sd_notify linux function which can be useful if you are writing a systemd service. You probably do not need this otherwise.
These are organized here roughly in order of most critical to least, but it all depends generally on your project. The first few items are required for the game to even run, but the rest are only necessary if you're using them, and the documentation for using those plugins will ideally make that apparent.
Detecting library availability
Sometimes, you might want to display a friendly error message to your users if a library is unavailable on platforms that allow you to do so rather than the application just crashing. You can do this with a function in NVGT called bool preglobals() which executes automatically similar to the main function but before any global variables that might tap into one of these libraries are ever initialized. It can return false to abort the execution of the application. Then, you can use the properties called SOUND_AVAILABLE and SCREEN_READER_AVAILABLE to safely determine if the subsystems could be loaded. The following code snippet would be OK, for example.
bool preglobals() {
if (!SCREEN_READER_AVAILABLE) alert("error", "cannot load screen reader components");
else if (!SOUND_AVAILABLE) alert("error", "cannot load soundsystem");
else return true; // success
return false; // one of the libraries failed to load.
}
Crediting open source libraries
NVGT is an open source project that uses open source dependencies. While aside from bass (which we will replace) we have been very careful to avoid any open source components that forbids or restricts commercial usage in any way, we do not always avoid dependencies that ask for a simple attribution in project documentation such as those under a BSD license. We'd rather focus on creating a good engine which we can fix quickly that can produce some epic games with awesome features, without needing to discard a library that implements some amazing feature just because it's devs simply ask users to do little more than just admit to the fact that they didn't write the library.
To facilitate the proper attribution of such libraries, a file called 3rd_party_code_attributions.html exists in the lib folder. We highly recommend that you distribute this file with your game in order to comply with all attribution requirements which you can read about in the aforementioned document.
The idea is that by simply distributing this attributions document within the lib folder of your app, any user who is very interested in this information can easily find and read it, while any normal game player will not stumble upon random code attributions in any kind of intrusive manner. The attributions document is currently well under 100kb, so it should not bloat your distribution.
The existing file attributes all open source components in NVGT, not just those that require it. If you want to change this, you can regenerate the file by looking in the doc/OSL folder in NVGT's github repository to see how.
For those who really care about not distributing this file, we hope to provide extra stubs in the future which do not include any components that require a binary attribution. However, there is no denying that this goal is very costly to reach and maintain, while yielding absolutely no true reward for all the time spent sans not needing to distribute an html file with game releases which players and even most developers will not be bothered by anyway. As such, it might be a while before this goal comes to fruition, and we could decide at any point that it is not a viable or worthwhile project to attempt if we find that doing so in any way detriments the development of NVGT, such as by making a feature we want to add very difficult due to lack of available public domain libraries for that feature.
Concurrency Tutorial
Introduction to multithreading and parallelism in NVGT
Multithreading, often referred to as concurrency, is the process by which a program can divide tasks into separate, independent flows of execution. These tasks, known as threads, can run concurrently within a single process, allowing for more efficient use of resources and faster execution of complex programs. This approach differs from parallelism, where the tasks are executed simultaneously on multiple processors or computers.
Concurrency focuses on the structure and design of the program to handle multiple tasks that can overlap in execution. This is particularly useful in scenarios where tasks can be interleaved and do not need to run at exactly the same time, such as managing user interfaces, I/O operations, or handling multiple client requests in a server.
In contrast, parallelism aims to improve computational speed by executing multiple tasks simultaneously. This can be achieved through multi-core processors, where each core can execute a separate thread, or distributed systems, where tasks are distributed across different machines in a network.
While NVGT does not support parallelism as one might use it in other programming languages, it does support threads, and this article is all about that. The above distinctions are important, however, because they are quite different, and the way one would go about implementing parallelism is vastly different (and more complicated) than how one would implement concurrency. We denote the distinction explicitly here because often these terms are used interchangeably when they do, in fact, mean very different things, although on a single computer they can appear to be very similar. We will not be discussing parallelism any further in this article, however, as concurrency is the more important aspect here.
Why is this so important?
By far, the easiest way to program is by sticking to synchronous execution, which is what happens by default unless any concurrency is specifically introduced by the programmer. Synchronous execution means that each statement in your code runs to completion before the next is executed. However as your game grows, you may find that it begins to execute slowly, which may temmpt you to quickly implement concurrency into your application.
This however poses a significant issue because NVGT supports three types of concurrency, and not knowing what method to use or when to use it can often result in slower or more buggy code than if either a different method of concurrency had been used, or if concurrency had not been used at all. People most often jump strait to threads to solve a problem of performance, when, 99 percent of the time, the performance degradation comes from an easily solvable issue in the person's code and not NVGT or a true need for threads or the lowest levels of concurrency. It can be not only bad practice but also detrimental to your program's development to use more advanced concurrency methods when a simple loop would have sufficed, but knowing when and how to spin up a thread when synchronous execution just won't cut it can also save the day in such cases. As such, this article attempts to explain:
- What exactly concurrency is;
- How to use concurrency correctly; and
- The golden rules of concurrency.
The above explainers are important because concurrency is easy to use improperly. Users of it who don't understand it are bound to make critical mistakes that can cause any number of things, such as:
- Sequential execution, i.e., absolutely no actual gain and a waste of resources
- Data races
- Deadlocks
- And worse
They'll also notice their code suffering very strange bugs or exhibiting very odd behaviors, all of which is very, very difficult to debug, let alone track down in the first place, especially on larger projects, and it's best that you be aware of these issues before ever considering threads.
Three types of concurrency?
There are three types of concurrency that NVGT supports: async, coroutines, and threads. We'll discuss all of them in this article.
Note
Any plugins you load may add even more methods of concurrency or even parallelism. Though this article will help you use those extra methods correctly, you should always read the documentation of those plugins to get a firm grasp of how their extra methods work.
Async
Async is the first type of concurrency, and one of the easiest to use. It's also one of the simplest to understand. Although async (may) use a thread (or even multiple) under the hood, it is entirely possible that it may not, and you, as the programmer, needn't care how it works as long as your able to do what needs to be done. In the majority of cases, this is probably the furthest you will ever need concurrency in your game.
Unlike the other two forms of concurrency, the engine insulates you (mostly) from the problems and mistakes of concurrency that you may make. This is particularly true for functions like url_get and url_post where the engine can complete the request while you do other things, or in any case that involves writing an asyncronous function that does not share any state with the rest of your program. Take, for example, this code:
async<string> result(url_get, "https://nvgt.dev");
This class is known as a templated class. Though it is beyond the scope of this article, the string part is the most important, besides the arguments of the constructor, which specify both the function to be executed and it's arguments. When you create (or instantiate) a class that's templated, NVGT generates the code for that specific class automatically for you, using the types you specify in the angle brackets. This is done on the fly and doesn't harm performance in any manner.
You may not realize it, but this constructor can take up to 16 parameters. (Woohoo, that's a lot!) When the constructor completes, result has some things of interest:
- The
valueproperty, which is (in this case) of typestring, and contains the return result of the function invoked. If you changestringabove to, say,int, then this will be of typeint, notstring. - A
completeandfailedproperty, which tells you if the task has finished executing, or failed in some manner, respectively. - An
exceptionproperty which provides you any information that you need if, during the execution of the function, it failed and threw one. - A
wait()method which pauses, or blocks, your code from continuing until the function completes or fails. - a
try_waitfunction, returning typebool, which takes a timeout and will block until either the timeout( also known as the deadline) expires or the task completes.
As stated previously, this is, for the majority of cases, the only thing you will need to reach for in your toolbox, and it's the highest level of concurrency available. The next two are much lower level, and give you more control, at the cost of raising the steaks by quite a bit. Even then, we strongly recommend reading the sections below about what can go wrong when 2 bits of code running at the same time try accessing the same global variable or bit of memory if you intend to write your own functions that are to be called with this async construct.
Coroutines
A coroutine is a function that suspends itself at certain points. This suspension is called yielding, and using coroutines is known as cooperative multitasking.
In computer architecture and operating systems, there are generally two types of multitasking approaches. Multitasking, for those who are unsure of the definition (and no, we don't mean multitasking in normal human language), is the concurrent execution or handling of multiple computational processes by a CPU or other processing units within a computer system, such that multiple tasks are performed during overlapping time periods. The key definition is "overlapping time periods." The two types that are generally accepted are known as cooperative multitasking and preemptive multitasking.
Cooperative multitasking is the first type and works by requiring that a task executes until it reaches a point where it can allow another task to run; this is known as a yield point. When the task reaches this point, it makes a call to a function provided by the operating system, runtime environment, or whatever is running each task, telling it that it is ready to step aside and allow something else to run. In NVGT, this is done through the yield() function. Cooperative multitasking relies on each task being well-behaved and voluntarily yielding control, ensuring that all tasks get a chance to execute. This approach is simpler and can be more efficient in environments where tasks can be trusted to yield regularly. However, if a task fails to yield, it can cause the entire system to become unresponsive.
Preemptive multitasking, on the other hand, is used in all well-known operating systems. Preemptive multitasking works by allocating a task a certain amount of time to run, known as a time slice, and allowing it to run until that time slice ends. When the time slice has elapsed, the operating system forcefully suspends (or "preempts") the execution of the task and replaces it with another task, and the cycle repeats forever. Usually, time slices are very short (perhaps only a few microseconds) and so this gives the illusion that the computer is able to do many things simultaneously. In multicore and multiprocessor systems, this is actually true: the computer can and does do many things all at the same time since the multitasking can occur on all the processors at the same time. However, all that's really happening is that the system is constantly performing this song and dance on all the processors in your computer, all at the same time!
You may have noticed that we used the word "task" instead of "process" or "thread." This is because, in a computer, there are a few different types of "tasks," and they're all the same to the computer:
- Process: A fully loaded program that runs. It has code, data, assets, etc.
- Thread: A task within a process. It shares the code and data of its parent program and is able to access all the state within. More on this later.
So, how does this have anything to do with coroutines? Well, when you create a coroutine, you are engaging in cooperative multitasking. The coroutine will begin execution as soon as you call create_coroutine and will pause the execution of all your other code until you call yield(). So, you must remember to yield at some point! The best places to do this are in loops or when your function is about to do something that could take a while, for example, some network-based IO. If you don't, none of your other code can run!
However, the same applies to the rest of your code: it, too, must yield; otherwise, your coroutine will never be able to proceed!
Unlike threading, which involves preemptive multitasking, it is (theoretically) impossible for a coroutine that executes in this manner to cause data races or other concurrency problems. This is because only one flow of execution is allowed at any given moment. You can definitely make it appear that multiple things are happening at once if you yield enough, but you'll never be able to duplicate the kind of problems that threading has, so you usually don't need to worry about those. This does not, however, mean that coroutines are the go-to option for all scenarios.
Coroutines are particularly useful for scenarios where tasks need to perform lengthy operations without blocking the entire program. For instance, they are excellent for handling asynchronous I/O operations, long-running computations that need to provide intermediate results, and any situation where tasks need to cooperate smoothly without complex synchronization mechanisms.
There are some rules to remember when using coroutines, however:
- Ensure that your coroutines yield control back to the caller at appropriate points, particularly in loops or before performing lengthy operations.
- Do not perform blocking operations (like waiting for I/O) within a coroutine without yielding. Instead, yield control and resume the coroutine once the operation completes. The
async<T>class can help with this: you can store it in a variable your code has access to, and when it completes, you canyieldto allow one of your coroutines to continue. - Always clean up resources before yielding or ensure that resources are properly managed upon resumption. This helps prevent resource leaks and ensures that the coroutine does not hold onto resources unnecessarily.
- Design your coroutines to work well with others.
- Thoroughly test your coroutines, particularly their yield and resume behavior. Debugging coroutines can be challenging due to their non-linear execution flow, so use logging and debugging tools to track their state and behavior.
To create a coroutine, call the create_coroutine function:
void create_coroutine(coroutine @func, dictionary @args);
Your coroutine should take the form:
void coroutine(dictionary@);
In both of these, dictionary@ is a dictionary of arguments that the coroutine needs to execute. When your ready to yield, simply call yield:
void yield();
As an example, let's say we wanted to calculate the factors of an integer. Although this is fast for small numbers, it can very quickly become quite computationally expensive when the numbers are particularly large. We might define the coroutine as:
void generate_factors(dictionary@ args) {
const uint64 number = uint64(args["n"]);
uint64[] factors = {1, number}; // Initialize factors with 1 and the number itself.
for (uint64 i = 2; i * i <= number; ++i) {
if (number % i == 0) {
factors.insert_last(i);
if (i * i != number) {
factors.insert_last(number / i);
}
}
yield(); // Yield control after each iteration
}
factors.sort_ascending();
for (uint64 i = 0; i < factors.length(); ++i) {
println("%0".format(factors[i]));
yield(); // Yield control after printing each factor
}
}
Then, before we start our game loop:
dictionary args;
args["n"] = 100000;
create_coroutine(@generate_factors, @args);
Now, in our game loop, we need somewhere to yield:
while (true) {
wait(1);
yield();
...
}
Threads
STOP!
Before you consider using threads, understand that they are the most complex and error-prone way to handle concurrency. Threads share state with your entire program, meaning they can easily cause data races and deadlocks if not managed correctly. These issues can make your game crash, behave unpredictably, or become very difficult to debug. Unless you have a specific, compelling reason to use threads, stick to async (or coroutines if you really need them).
Note
This section is extremely technical in places. This is because multi-threading can be difficult. Even after doing it for a few years you will begin to learn that there are subtle mistakes you can make without realizing it. After we get through the different methods of protecting your code, we'll get into how to use threads and the rules on using them properly. If you want to know how to use any of the synchronization primitives discussed here, the documentation is your friend, as well as the NVGT community.
Introduction
A thread is much different than a coroutine. A thread runs alongside your code, and could even run on other processors in the system. A thread introduces many other problems that coroutines don't: they share state with the rest of your game, meaning they have access to all the global variables the rest of your code does.
One of the most critical rules of threads is to avoid global or shared state. Global or shared state means any global variables that you have in your code, as well as any state that your thread may access that other threads (also) might access.
If two or more threads access shared state without any kind of protection, this is known as a data race. A data race occurs when two threads want to perform different operations on a variable, and it just so happens that they do it the same time, or close enough that it doesn't matter. For example, if thread 1 wants to read a sound handle and another thread wants to initialize it, it may just so happen that both operations overlap, causing the handle that thread 1 gets to be in some weird undefined state. Data races can have all kinds of dangerous consequences that could make your program crash, cause unpredictable behavior, and so on. It's even known to cause a write to a variable to mysteriously vanish!
Data races are also one of the hardest things for a programmer to debug. This is because it can't be predicted when they'll even occur. One run of your code might be fine, but the next run might cause the race, or it might only occur every 200 runs of your code. This is why protection is so important.
Protecting against data races
There are several ways of protecting your code against data races if you do share state:
- Locks or semaphores
- Atomic variables
- Events/condition variables
- Just don't share any state
- Message passing (which will be explained here after it is added to NVGT)
Locks and semaphores
A lock is a synchronization primitive used to manage access to a shared resource by multiple threads. When a thread wants to "acquire" the lock, it checks if another thread already holds it. If the lock is already acquired, the requesting thread is blocked until the lock becomes available. When the lock is released, the blocked thread can proceed. This mechanism prevents concurrent access to the resource, ensuring data consistency and integrity.
Semaphores are another important synchronization mechanism. A semaphore is a signaling mechanism that can be used to control access to a common resource by multiple threads in a concurrent system. Semaphores can be used to solve various synchronization problems, such as controlling access to a finite number of resources or coordinating the order of thread execution. Semaphores maintain a counter representing the number of available resources. Threads can increment the counter to signal the release of a resource or decrement it to wait for a resource. When the counter is zero, any thread attempting to decrement it is blocked until the counter is incremented by another thread.
There are several types of locks and synchronization mechanisms. One common type is the mutex, or mutual exclusion lock. A mutex ensures that only one thread can access a resource at a time. When a thread acquires a mutex, other threads attempting to acquire the same mutex are blocked until it is released. This is useful for protecting critical sections of code where shared resources are accessed or modified.
Spinlocks are another type of lock, where the thread repeatedly checks if the lock is available in a loop, or "spins," until it can acquire the lock. Spinlocks are efficient when the wait time is expected to be short, as they avoid the overhead of putting the thread to sleep and waking it up. However, they are not suitable for long wait times, as they consume CPU cycles while spinning.
Read-write locks, or RW locks, allow multiple threads to read a resource simultaneously but provide exclusive access for writing. This lock has two modes: read mode and write mode. Multiple readers can hold the lock concurrently, but a writer must wait until all readers have released the lock, and vice versa. This is particularly useful in scenarios where read operations are frequent and write operations are infrequent, as it allows for greater concurrency.
Recursive locks are designed to be acquired multiple times by the same thread without causing a deadlock. They keep track of the number of times they have been acquired by the thread and require the same number of releases to unlock completely. This is useful in scenarios where the same thread might need to re-enter a critical section of code.
Binary semaphores, also known as mutex semaphores, can have only two states: 0 or 1, representing locked or unlocked. They function similarly to mutexes but do not have ownership, meaning any thread can release them, not just the thread that acquired them. This is useful for simple synchronization tasks where the ownership of the lock is not important.
Lastly, counting semaphores can have a value greater than one, allowing them to manage a finite number of resources. They maintain a counter representing the number of available resources. Threads increment, or signal, the counter when a resource is released and decrement, or wait, the counter when a resource is acquired. This is useful for limiting access to a pool of resources, such as a fixed number of database connections.
Wait wait, deadlocks?
yeah, did I forget to tell you about those? A deadlock is where two or more threads are unable to proceed with their execution because each is waiting for the other to release a resource they need.
To understand deadlocks, consider an example where two threads, Thread A and Thread B, need access to two resources, Resource 1 and Resource 2. If Thread A acquires Resource 1 and Thread B acquires Resource 2, both threads might then try to acquire the resource held by the other. Thread A will wait for Resource 2 to be released by Thread B, while Thread B waits for Resource 1 to be released by Thread A. Since neither thread can release the resource it holds until it acquires the resource held by the other, they are both stuck, leading to a deadlock.
Four necessary conditions for a deadlock to occur are:
- At least one resource must be held in a non-shareable mode, meaning only one thread can use the resource at any given time.
- A thread holding at least one resource must be waiting to acquire additional resources currently held by other threads.
- Resources cannot be forcibly taken from the threads holding them; they must be released voluntarily by the holding thread.
- There must exist a set of threads such that each thread is waiting for a resource held by the next thread in the set, forming a circular chain.
Preventing or resolving deadlocks typically involves ensuring that at least one of these conditions cannot hold. Techniques include:
- Impose a strict order in which resources must be acquired and ensure that all threads adhere to this order.
- Allow deadlocks to occur, but have mechanisms to detect and recover from them, such as terminating one or more of the threads involved.
- Use algorithms that dynamically analyze resource allocation requests to ensure that they do not lead to deadlocks, such as the Banker's Algorithm.
- Implement timeouts for resource requests, forcing a thread to release its held resources if it cannot acquire the required resources within a certain period (this, by far, may be the easiest algorithm to implement).
Atomic variables
An atomic variable is a special type of variable used in concurrent programming to ensure that operations on the variable are completed without interruption. The key characteristic of atomic variables is that any operation performed on them -- such as reading, writing, or modifying -- must be completed entirely before any other operation can start. This means that the state of the variable is always consistent and predictable, even when multiple threads are accessing or modifying it simultaneously.
To understand this better, consider a scenario where multiple threads are trying to increment a shared counter variable. In a non-atomic scenario, one thread might read the value of the counter, and another thread might read the same value before the first thread has a chance to write the incremented value back, leading to both threads writing back the same value. This causes the counter to be incremented only once instead of twice, leading to incorrect results.
However, with an atomic variable, the increment operation is indivisible. This means that when one thread starts to increment the counter, no other thread can read or write to the counter until the increment operation is fully completed. The operation is guaranteed to either fully succeed or fail, but it cannot be partially completed. This prevents the problem of race conditions, where the outcome depends on the unpredictable sequence of thread execution.
Modern processors support atomic operations through special hardware instructions that ensure these operations are performed without interruption. These instructions include atomic read-modify-write operations, such as compare-and-swap (CAS) and fetch-and-add, which are used to implement atomic variables.
For example, consider the compare-and-swap (CAS) operation. It works by checking if the value of the variable is equal to an expected value. If it is, the value is replaced with a new value. If it isn't, the operation fails. This all happens in one atomic step, ensuring that no other thread can interfere between the check and the update.
Events/condition variables
Events are synchronization primitives used to signal between threads, allowing one thread to notify others about the occurrence of a specific condition. In the context of concurrency, an event can be set or reset, and threads can wait for an event to be set before proceeding. This is particularly useful when you need one or more threads to pause execution until a certain condition is met.
For instance, imagine a scenario where a worker thread is processing tasks from a queue. The worker thread can wait for an event that gets set when new tasks are added to the queue. Once the event is set, the worker thread wakes up and processes the new tasks. If there are no tasks, the worker thread goes back to waiting. This mechanism ensures efficient use of CPU resources, as the worker thread is not continuously polling the queue but instead waits for a signal.
Events can be manual-reset or auto-reset. A manual-reset event remains signaled until it is explicitly reset, allowing multiple waiting threads to be released. An auto-reset event automatically resets after releasing a single waiting thread, ensuring that only one thread is woken up per signal.
Condition variables are another synchronization mechanism used to block a thread until a particular condition is met. They are typically used in conjunction with mutexes. When a thread waits on a condition variable, it releases the associated mutex and enters a waiting state. When the condition is signaled, the mutex is reacquired, and the thread resumes execution.
The primary use case for condition variables is to manage complex thread interactions that require waiting for certain states or conditions. For example, in a producer-consumer scenario, a consumer thread might wait on a condition variable until there are items available in a buffer. The producer thread, after adding an item to the buffer, signals the condition variable to wake up the consumer thread.
Here’s a simplified example: a producer and consumer thread are started one after the other. The producer thread (which is what produces data for the consumer thread to act upon) acquires the mutex, adds an item to the buffer, signals the condition variable, and releases the mutex. By contrast, the consumer thread acquires the mutex, waits on the condition variable while the buffer is empty, processes the item from the buffer the condition variable has been signaled, and releases the mutex. The "mutex," in this case, could (and should most likely be) a recursive mutex.
Condition variables support two main operations: wait and notify. The wait operation puts the thread into a waiting state and releases the mutex. The notify_one operation wakes up one waiting thread, while notify_all wakes up all waiting threads. These operations are critical for coordinating complex interactions and ensuring that threads only proceed when the required conditions are met.
Events are simpler to use when you need to signal one or more threads to proceed. They are particularly effective when dealing with straightforward signaling scenarios, such as waking up worker threads when a new task arrives.
Condition variables are more suitable for scenarios requiring complex synchronization, especially when multiple conditions and interactions between threads need to be managed. They provide more control over the waiting and signaling process, making them ideal for use cases like producer-consumer problems.
Not sharing any state
If all of the above has left you utterly confused and wondering what you've just read, that's okay; you have lots of other options. The simplest would be to just ignore this section entirely and to just not use threads at all. If you really, really, really do need threads, though, the best method would be to figure out how you don't need to share state. This might be quite difficult in some cases, but the less state you share, the less likely it is you'll need to worry about any of this stuff. If you do need to share state, all of the above is important to understand to do things properly.
Creating and managing threads
To create a thread, call the thread constructor and give it a name of your thread. This is usually an ID that you can use to look up the thread later. This will NOT start your thread, though; to do that, call start passing in both a function to execute and (optionally) a dictionary handle of arguments that that thread should receive.
Your thread callback should take as input a dictionary@ which contains the arguments given in start().
Once started, a thread immediately begins execution. Your original code that spawned the thread will continue executing after this function returns.
If you want to wait until the thread completes, call join() without any arguments. Alternatively, join can take a timeout which, if expires, causes the function to return false.
On the thread object there are a few other things you can do as well, such as changing the threads priority via the priority property.
Warning!
Do not change the priority of a thread unless you absolutely have to. Messing around with the thread priority of a thread -- particularly with threads which do a lot of computationally expensive work -- can cause your entire system to hang and have other undesireable side-effects!
Golden rules of threads
When using threads, always remember:
- Only use threads when you need to. Do not just use them because you feel they would make your code faster. More often than not, this will just cause your code to run slower.
- When using locks, hold the lock for the shortest time possible. The area in which a lock is acquired is known as a critical section, and you should do only what you neeed to do in these and then immediately release the lock. If you don't do this, you could cause resource leaks, sequential execution, deadlocks and all kinds of other problems.
- Do not acquire a lock at the start of a function. This may seem tempting, but it is wrong, and will cause your code to be far slower, if not eliminate the benefit of using threads entirely.
- Avoid recursive mutexes unless you know what you are doing.
- When acquiring a lock, there are two ways of doing so:
lockandtry_lock. If your able to, usetry_lockwhich will return false if the lock cannot be acquired. If this does return false, go do something else and try acquiring the lock later. - If your code consists of many threads that mostly perform reads of shared state and only a few cases where shared state is modified, use a reader-writer lock and not a mutually exclusive one.
- Do not mess with thread priorities.
- The less shared state your threads have to manage, the fewer synchronization issues you'll encounter. Where possible, avoid global variables or shared objects.
- For simple counters or flags, use atomic operations instead of locks to avoid the overhead and complexity of mutexes.
- Do not use busy-waiting (spinning) to wait for a lock or condition. This wastes CPU resources and can lead to performance issues. If you need to wait for a lock to be available for acquisition, call
lockon it and allow the underlying operating system to do the waiting for you. - Creating and destroying threads can be expensive. Reuse threads with thread pools or similar techniques to manage resources efficiently.
- Where possible, use higher-level concurrency constructs like
async<T>. These abstractions often handle the complex details of synchronization for you. - When using condition variables, always protect the condition check with a lock and use a loop to recheck the condition after waking up, as spurious wakeups can occur. That is, it is possible for your thread to be awoken when the condition variable hasn't actually been signaled.
- While using timeouts can prevent deadlocks, ensure that your program can handle the scenario where a timeout occurs gracefully.
- Clearly document the synchronization strategy in your code, including which locks protect which data. This helps maintainers understand the concurrency model and avoid introducing bugs.