• packed up everything I owned except the stuff I sold or threw out
  • moved, really far
  • made a decision that I haven’t decided whether it’s good or bad yet

Some time has passed since my last post. I have a reasonably good excuse, I think – I moved me, my wife, my cat, and most of our stuff (what we could justify keeping anyway) thousands of miles across the country, and got a new job. Things have been a bit hectic (and, really, still are!)

Not Unproductive

That said, I haven’t gotten NOTHING done on Black Mountain. After driving across the country (and my stuff, separately, being shipped across) I discovered my desktop computer no longer boots. It doesn’t even really try to power on, so it’s probably the motherboard. I got a new laptop right away (since I didn’t have an alternative that was up to the task of what my new job was going to demand of it), but the files (other than what was committed to git) were locked away on the hard drive in the desktop. I got the new Godot 4 beta and started messing around with that (trying very hard not to think too hard about “what if all my original art files are gone”), and didn’t make anything particularly interesting but I got a feel for how stable the beta is (quite stable!) and what had changed (quite a bit!)

Eventually we got settled in a new apartment and I got myself an external enclosure for my hard drive. The hard drive was totally fine, all my files were there, and I decided that what I should do is just bite the bullet and migrate the game to Godot 4!

A Mistake… Maybe?

I have gone back and forth a lot on just how much of a mistake this was. I’m still in the middle of the process, though I’m making progress at least. Nearly every change from Godot 3 -> 4 seems to be good (the rest seem pretty neutral, honestly) but I have wrestled a lot with what the automatic upgrade process gave me.

To be clear: I went into this fully aware that I was doing something that wasn’t recommended, I have a backup of the original project so I can always go back to Godot 3 if I want, etc. I did the automatic migration with beta 2, and beta 3 is already out (and beta 4 should be out soon). The migration process is part of what they’re improving as they go, so in the future this should be a lot smoother

That said, it’s been rough. Thanks to Godot 3’s wide-spread usage of strings for things like deferred function calls and properties etc, it means that if they changed a name for something, the migration process has to update all instances of that name to the new one… even if that occurs inside of a string (and, apparently, inside of a comment). This meant things like having my “Fight On Touch” behaviour being renamed to “Fight Checked Behaviour”.

Another thing that ended up being surprisingly rough was the new asynchronous stuff – being able to use await was one of the changes I was most excited for, but it turns out I had made use of the old yield-based asynchronicity in ways that weren’t compatible with the new way of doing things! There were a number of places where I would hold onto a GDScriptFunctionState object from a call on a stack, for instance, and check each frame to see if it was completed. Turns out you can’t really do that anymore, though I have figured out how to rework things for the new system (at least, so far). I’ll talk more about that below.

Overall, though, I do like Godot 4. I’ve said it’s rough, but I want to stress again that it’s the migration process that has been. Godot 4 itself is still a beta, it’s quite solid, and if I were starting a new project I would absolutely start it in Godot 4.

Many Async Calls at Once

In Godot 3, coroutines are done with yield, which is actually used to wait for a signal.

1
2
3
4

def f():
    print("A")
    yield(get_tree(), "idle_frame")
    print("B")

Calling f() here will print “A”, then wait for the “idle_frame” signal from the root tree (which is fired every frame), and then will print “B” – specifically idle_frame is triggered right before _process is called one each of your objects.

What yield actually does is pack up the function’s execution state into a GDScriptFunctionState object, and returns immediately, thus ending execution of the function. Whenever the signal you’re waiting on fires, it unpacks that GDScriptFunctionState object and resumes executing. Since it jumps right back into the execution, you can stick that yield wherever you need in the function and start it right back up there…

1
2
3
4
5
6

def wait(time):
	var done = OS.get_ticks_msec() + (time * 1000.0)
	while OS.get_ticks_msec() < done:
        var t = OS.get_ticks_msec()
        yield(get_tree(), "idle_frame")
        return (OS.get_ticks_msec() - t) / 1000.0

This wait() function will keep yielding out of the while loop every frame, and going right back in to check how much time has passed. Once it passes the requested threshold (time in seconds) it will exit out of the loop and be done.

The next important part is that async functions (aka coroutines) fire a signal called completed when they finish. (Technically this is probably not quite correct; I don’t think completed is really a signal but it works like one, so I think it’s relatively safe to think of it that way.)

Using that wait function above would look like…

1
2
3
4

def f():
    print("A")
    yield(wait(5), "completed")
    print("B")

This version of f() would print A, wait 5 whole seconds (without locking up the engine, so rendering and input can still happen just fine) and then print B.

This is an aside, but another cool thing with this is that you can get a value returned from an async call.

1
2
3
4
5
6
7
8
9

func wait_for_next_frame():
	var t = OS.get_ticks_msec()
	yield(get_tree(), "idle_frame")
	return (OS.get_ticks_msec() - t) / 1000.0

func f():
    print("A")
    var dt = yield(wait_for_next_frame(), "completed")
    print("B %s" % [dt])

Here, wait_for_next_frame() is a slightly fancier version of yield(get_tree(), "idle_frame") which will not only wait until the next frame, but will tell you how many seconds have passed since (which hopefully for you is a very small fraction of a second!)

What happens, though, if you call a coroutine without yield?

1
2
3
4

func f():
    print("A")
    var what = wait_for_next_frame()
    print("B %s" % [what])

In this case, you will print “A” and “B” immediately. Since there’s no yield, you’re not pausing execution. What you’ll see printed next to “B”, though, is a GDScriptFunctionState object – in fact, it’s the one that represents the state of the coroutine when it called yield! That object has a function you can call to see if the function state is still valid; that is, can it still re-enter. You can hold onto that state object, and keep checking is_valid(), in fact, until it becomes false… which means the function has finally completed.

So that means, say we have three coroutines we all want to run, but we don’t know how long they’ll all take. Say they’re part of a cutscene system – you want to have two characters walk around independently, and you also want to have a textbox show up that waits for the player’s input. You then want to end the cutscene once all three things have finished. You don’t know if the player will hit the button first or after the characters have finished moving, and you’re not sure exactly how long the characters will take to move (let’s say they’re doing some path finding or whatever).

Well… to cut to the chase, you write this function (pulled from Black Mountain’s Godot 3 source):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

func multiyield(fs):
    yield(wait_for_next_frame(), "completed")
	var done = false
	while not done:
		done = true
		var s = ""
		for f in fs:
			if f is GDScriptFunctionState and f.is_valid():
				done = false
		yield(wait_for_next_frame(), "completed")

func f():
    var async_things = [
        move_first_character(),
        move_second_character(),
        show_textbox()
    ]
    yield(multiyield(async_things, "completed")

multiyield() takes in a list of GDScriptFunctionState objects (at least, theoretically that’s what they are, but just in case we check to make sure). We start up a loop and in each iteration of the loop, we check every one of those objects. If any of them are still valid, we keep the loop going (but wait for the next frame to give things a chance to process). If every single one is no longer valid, we exit. In other words, we’ve created an async call that gloms all of those coroutines together and doesn’t complete until they’re all done!

I will note that I am yielding for a single frame at the beginning because of a really cool quirk in the Godot 3 coroutines. Functions that don’t yield? They don’t have a “completed” signal. So if you have a normal, non-coroutine function, and you call yield(foo(), "completed") with it, it just blows up. If you look at my multiyield, you’ll notice that if fs is empty, or if none of them are valid in the very first iteration, it’ll just fall through and return without needing to wait for a frame – meaning it would never yield at all. I throw in one frame of delay right at the beginning so I know that, no matter what, multiyield will always yield at least once. (This sucks; it’s a good thing I’m making a fairly slow JRPG and not a tight action game.)

(At this point I should note that if you’re familiar with Promises in Javascript, this multiyield() is essentially Promise.all() – I haven’t needed to write any of the rest of the various “multi-Promise” functions, but I’m reasonably sure you could write them all with a similar approach.)

And Now in Godot 4…

So then what happens if you call this in Godot 4? Well, there is no yield() in Godot 4, so you’ll get that error. You might also see this error:

Parser Error: Function "foo()" is a coroutine, so it must be called with "await".

Godot 4 is a lot more smart about what is and isn’t a coroutine, and it has introduced a new statement, await foo() which functions kind of like yield(foo(), "completed") but is a lot less kludgy. You no longer yield in a way that returns a function state object – I am pretty sure Godot still uses those same GDScriptFunctionState objects internally but they don’t just hand them back to you when you call the coroutine in a particular way, which means you don’t shoot yourself in the foot by forgetting to yield on a coroutine’s completed “signal” so often! You can await on a normal non-coroutine function just fine, it simply continues on immediately! It’s great!

It also means my exceedingly clever code to make a multiyield doesn’t work. Which was a bit disappointing. But I did find a work around, and it’s even kind of not-gross. Here’s what I’ve figured out:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

func async_all(fs:Array):
	var id = 0
	var rtnvalues = []
	var completed = [] # use an array instead of an int because arrays are reference types

	rtnvalues.resize(fs.size())

	var complete_one = func(rtnvalue, i):
		rtnvalues[i] = rtnvalue
		completed.push_back(i)

	for f in fs:
		var AO = AsyncWrapper.new(f[0], f.slice(1))
		AO.async_done.connect(complete_one.bind(id))
		AO.run()
		id += 1

	while completed.size() < fs.size():
		await get_tree().process_frame
	
	return rtnvalues

class AsyncWrapper extends Object:
	signal async_done(returnValue)

	var _callable:Callable
	var _args:Array

	func _init(callable:Callable, args:Array):
		_callable = callable
		_args = args

	func run():
		var r = await _callable.callv(_args)
		async_done.emit(r)

async_all() here is my new multiyield(). AsyncWrapper is a cool new Godot 4 internal class. You pass in a list of functions, just like with multiyield, but instead of calling those functions you pass a list of arrays like

    async_all([
        [foo, "first_param", 2],
        [bar, "bars_param"]
    ])

which isn’t great but it works. (Note that callables in Godot 4 are first-class values, so no more passing string names around! I could probably replace these with .bind() calls instead, now that I think about it… hmm.) It sets up a few values, and an inline function complete_one (another new feature!), then sets about calling each one. It calls them with the AsyncWrapper class, which awaits the result of the call, and then triggers a signal – a signal we’ve connected to our complete_one function.

complete_one adds a value to the completed array, and once the completed array has grown in size to match the number of functions pass in… we’re done! As the comment notes, I’m using an array for completed instead of just incrementing a number because (as near as I can tell) the anonymous inline function captures the value of variables at their declaration time, meaning every invocation of complete_one would think it was 0. Since arrays are reference values, every call is adding to the same array.

Okay Time to Put the Keyboard Away

Okay I need to sleep and it is not making my explanations more coherent. Hopefully someone out there gleaned SOME useful information from this. Have a good week!