TIL Ray remote calls aren't lazy

TIL that I've been misunderstanding how Ray Core schedules work. I was under the mistaken impression that remote calls such as foo.remote() or actor.bar.remote() were lazy; that unless I triggered a terminal action like ray.get(), that Ray would skip scheduling the task. By extension, I assumed that dropping the returned ObjectRef would prevent execution because the reference count would hit zero.

In reality, Ray queues the task immediately 🤯 ! This changes how I think about ray.get(): it's really about materializing the result back to your process, and isn't needed if you don't need the result. ray.get() is also useful for control flow, since it's a blocking operation. If no references remain to the ObjectRef, Ray garbage collects the result from the object store, but the computational graph is otherwise unaffected.

(Contrast this with Ray Data, which does actually use a lazy execution model)

This is handy for fire-and-forget patterns, which is perfect for launching background work without blocking a caller. This whole time, I've been using ray.get() even if I didn't use the result, such as ray.get([background_task.remote(job) for job in jobs]) 🤦‍♂

Demo:

import ray


@ray.remote
class EchoActor:
    def __init__(self):
        self.calls = 0

    def echo(self, message):
        print(f"[EchoActor] {message}", flush=True)
        self.calls += 1

    def total_calls(self):
        return self.calls


@ray.remote
def echo(message):
    print(f"[echo task] {message}", flush=True)


ray.init()
echo_actor = EchoActor.remote()

print("1) Fire an actor task and wait for the result with ray.get()")
ray.get(echo_actor.echo.remote("actor call with ray.get()"))

print("\n2) Fire the same actor task without ray.get(), and drop the ObjectRef")
echo_actor.echo.remote("actor call (fire-and-forget)")

print(f"\n3) Total actor calls: {ray.get(echo_actor.total_calls.remote())}")

print("\n4) Run a normal Ray task and wait for the result")
ray.get(echo.remote("normal task with ray.get()"))

print("\n5) Fire a normal Ray task without saving the ObjectRef")
echo.remote("normal task (fire-and-forget)")