Skip to content

module: allow module.register from workers - #53200

Closed
dygabo wants to merge 12 commits into
nodejs:mainfrom
dygabo:allow-hooks-thread-from-worker
Closed

dygabo wants to merge 12 commits into
nodejs:mainfrom
dygabo:allow-hooks-thread-from-worker

Conversation

@dygabo

@dygabo dygabo commented May 29, 2024 •

Copy link
Copy Markdown
Member

this is a fix for #53182

by implementing a guard on the native side, we can ensure that the internal customization hooks thread is instantiated only once but not necessarily from the main thread.
https://github.com/ShogunPanda/hooks-repro example is not hanginng

the second commit reenables the tests that were skipped because module.register() was temporarily not supported on worker threads.

One is still failing on workers. Currently analyzing.

This implements the fix based on solution number 2. from this comment

alternative to #53183

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/loaders

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels May 29, 2024
@dygabo

dygabo commented May 29, 2024

Copy link
Copy Markdown
Member Author

@nodejs/loaders @mcollina @ShogunPanda

@dygabo
dygabo marked this pull request as draft May 29, 2024 13:52
@ShogunPanda

Copy link
Copy Markdown
Contributor

I don't think you need to modify on the C++.
Can't you just use a one-size SharedArrayBuffer that is created on the main thread and forwarded to each worker?
This way you can use Atomics.Store on it.

Comment thread lib/internal/modules/esm/hooks.js Resolved
Comment thread lib/internal/modules/esm/loader.js Resolved
@dygabo
dygabo marked this pull request as ready for review May 29, 2024 14:37
@dygabo

dygabo commented May 29, 2024

Copy link
Copy Markdown
Member Author

in current form this should fix all the regressions that I was made aware of. module.register should seemlesly work as before. We have one worker for the hooks instead of N.

@ShogunPanda

ShogunPanda commented May 29, 2024 •

Copy link
Copy Markdown
Contributor

@dygabo

Have you tried the following case:

hooks.mjs

const now = Date.now();

export function initialize() {
  process._rawDebug(`[${now}] Hooks initialized ${Date.now()}.`);
}

export function resolve(specifier, context, next) {
  process._rawDebug(`[${now}] Hooks resolving ${specifier}.`);
  return next(specifier);
}

t1.js

const { register } = require("node:module");
const { pathToFileURL } = require("node:url");
const { Worker } = require("worker_threads");

register("./hooks.mjs", pathToFileURL(__filename));

new Worker("./t2.js");

t2.js

const { Worker } = require("worker_threads");

new Worker("./t3.js");

t3.js

import("../lib/local.js");

Note that there is no --import anywhere.

@dygabo

dygabo commented May 29, 2024

Copy link
Copy Markdown
Member Author

@dygabo

Have you tried the following case:

@ShogunPanda I tried that now. Works IMO exactly like v22.1 worked (and like 22.2.0 for that matter). What is the expectation?

@ShogunPanda

Copy link
Copy Markdown
Contributor

Well, that the hooks are executed. Which is not the case even on 22.1. I think this should be fixed as well as part of this.

@dygabo

dygabo commented May 29, 2024

Copy link
Copy Markdown
Member Author

Well, that the hooks are executed. Which is not the case even on 22.1. I think this should be fixed as well as part of this.

the reason for that is unrelated to the single hooks thread. If that should work I am for creating a separate issue and deal with that on its own. What do you think?