How to make callbacks run in the background

Prerequisites

This guide assumes familiarity with the following concepts:

• Callbacks

By default, LangChain.js callbacks are blocking. This means that execution will wait for the callback to either return or timeout before continuing. This is to help ensure that if you are running code in serverless environments such as AWS Lambda or Cloudflare Workers, these callbacks always finish before the execution context ends.

However, this can add unnecessary latency if you are running in traditional stateful environments. If desired, you can set your callbacks to run in the background to avoid this additional latency by setting the LANGCHAIN_CALLBACKS_BACKGROUND environment variable to "true". You can then import the global awaitAllCallbacks method to ensure all callbacks finish if necessary.

To illustrate this, we’ll create a custom callback handler that takes some time to resolve, and show the timing with and without LANGCHAIN_CALLBACKS_BACKGROUND set. Here it is without the variable set:

import { RunnableLambda } from "@langchain/core/runnables";

Deno.env.set("LANGCHAIN_CALLBACKS_BACKGROUND", "false");

const runnable = RunnableLambda.from(() => "hello!");

const customHandler = {
  handleChainEnd: async () => {
    await new Promise((resolve) => setTimeout(resolve, 2000));
    console.log("Call finished");
  },
};

const startTime = new Date().getTime();

await runnable.invoke({ number: "2" }, { callbacks: [customHandler] });

console.log(`Elapsed time: ${new Date().getTime() - startTime}ms`);

Call finished
Elapsed time: 2005ms

And here it is with backgrounding on:

import { awaitAllCallbacks } from "@langchain/core/callbacks/promises";

Deno.env.set("LANGCHAIN_CALLBACKS_BACKGROUND", "true");

const startTime = new Date().getTime();

await runnable.invoke({ number: "2" }, { callbacks: [customHandler] });

console.log(`Initial elapsed time: ${new Date().getTime() - startTime}ms`);

await awaitAllCallbacks();

console.log(`Final elapsed time: ${new Date().getTime() - startTime}ms`);

Initial elapsed time: 0ms
Call finished
Final elapsed time: 2098ms

Next steps

You’ve now learned how to run callbacks in the background to reduce latency.

Next, check out the other how-to guides in this section, such as how to create custom callback handlers.