feat: ESM loader overhaul — Web/Node parity, ns:module dev surface, HTTP module loader - #1965

Dev sessions serve the app's module graph over HTTP during development,
with a mechanism-only dev-loader contract: policy stays in JS tooling,
the runtime supplies fetch/registry/invalidations. The loader is
deny-by-default — remote allowlist entries only authorize URLs on a
URL-component boundary ('/', '?', '#' or exact match), refusing
lookalike-host and lookalike-port bypasses; a specific port must be
listed explicitly. Hot-path hash containers use robin_hood maps.

Per-fetch URL logging is opt-in via the httpFetchUrlLog config flag
(volume is one line per fetch), alongside the existing
logScriptLoading-gated diagnostics. The previous HMRSupport/DevFlags
sources are replaced by HttpLoader (JNI HttpURLConnection).

The dev-loader control surface (HttpLoader) is reachable from JS as the
ns:module builtin module: NsBuiltinModules routes ns:module through
BuildNsModuleBinding — the binding builder decides build-dependent
membership — and ns-module.js (compiled in via js2c) shapes and freezes
whatever arrives. docs/ns-builtin-modules.md documents the surface.

Worker entry-script load failures now reach worker.onerror instead of
failing silently. Messages posted before the worker's entry script has
installed onmessage are no longer dropped: ConcurrentQueue::Signal
re-arms the drain source without enqueueing (a silent no-op when racing
Terminate), and WorkerWrapper retries delivery through a deferred drain,
with drainRetryPending_ preventing one stacked retry per attempt.

The ns:module surface, remote-module allowlist boundary matching, and
relative ESM dynamic-import cases exercise the async loader and the
deny-by-default HTTP gate. The on-device result harvester falls back to
run-as when adb root is unavailable (Play Store emulator images), and
-Pabis is forwarded so a single-ABI V8 tree can build and test locally.

…into ns:runtime

Live log flags (logScriptLoading, httpFetchUrlLog) move onto ns:runtime
setConfig/getConfig. Remote-module security stays boot-time
nativescript.config only. Android does not expose releasedObjectPolicy.

JNI mid-body read exceptions no longer spin the JS thread, async fetch
threads detach from the JVM, and canonicalization config is published as
an immutable snapshot so configureLoader cannot race a background fetch.

A dependency-free ServerSocket server in the test app, serving the module
fixture routes the HTTP loader specs need (MIME-gate cases, JSON modules,
graph leaves keyed by query, syntax errors, configurable delays), bound to
an ephemeral 127.0.0.1 port and started lazily from JS via
com.tns.tests.ModuleTestServer.ensureStarted(). Route set and response
bodies mirror the iOS ModuleTestServer.

Registry writes now maintain a GetIdentityHash() -> keys reverse index in
the loader state, and the two module->key lookups (resolver referrer
discovery, import.meta initialization) consult it instead of scanning the
whole registry. Buckets hold candidates because hashes collide; lookups
confirm by handle equality against the registry and prune candidates the
registry no longer backs, so a stale entry can never answer.

Writes outside the callbacks TU go through Unindex/IndexModuleForIsolate;
unindexing happens before a Reset, while the outgoing handle's hash is
still recoverable.

One trace facility for the module subsystem: a dense category enum (esm,
fetch, registry) over a relaxed-atomic bitmask, a TNS_DEBUG macro whose
arguments are never evaluated while the category is off, and a cold emit
path writing to a per-category logcat tag (TNS.esm, TNS.fetch,
TNS.registry). Compiled into release builds too - a build that cannot be
traced cannot be diagnosed. Error and lifecycle logs stay unconditional.

Enablement: the NS_DEBUG environment variable at process init (the only
way to trace boot), and ns:runtime setConfig('debug', 'esm,fetch') at
runtime - process-wide, main-isolate write, each write replacing the
whole set. getConfig('debug') returns the enabled list.

The per-flag keys logScriptLoading and httpFetchUrlLog are gone, along
with their nativescript.config plumbing through AppConfig and the JNI
seeding. UncaughtErrorPolicy's hardcoded ordinal read moves from 15 to 14
now that the two removed keys no longer precede it.

ResolveModuleCallback no longer instantiates or evaluates anything: a disk
dependency is read, compiled (ModuleInternal::CompileFileEsModule) and
registered inline, and V8 drives graph discovery from the root's
InstantiateModule. Cycles terminate through the registry via the
module-map self-insert pattern - a back-edge finds the existing entry
whatever its status. Evaluation happens once, at the root
(ModuleInternal::LoadESModule), in spec order.

Everything that compensated for resolver-order evaluation is deleted: the
resolution stack and its RAII guard, the re-entry counters/parents/primary
importers, modulesPendingReset, the moduleWaiters queue, and both HMR
fallback registries (last-known-good serving) - a failed reload after
invalidation now rejects loudly instead of silently serving a stale
module. Dynamic-import coalescing (modulesInFlight + httpDynamicWaiters)
stays.

Registry entries are reused whatever their status - only kErrored is
dropped and recompiled - and JSON modules are cached like any other
module instead of being recompiled per resolve. The root registers under
CanonicalizeRegistryKey, the same key derivation the resolver uses for
dependencies, so a module reached as a root and as a dependency share one
identity (and one import.meta.url).

The pre-instantiation walk now runs for every ES-module root - boot
entries, require(esm) roots, and dynamic import() - with per-edge
dispatch: local files are read, compiled and registered inline during
discovery; HTTP edges fetch concurrently on the async pipeline; builtins
and unmapped bare specifiers stay on the resolver's lazy path. A local
root's graph can contain HTTP edges and vice versa. Disk-only graphs
complete discovery inline and pay no pump iteration.

One shared ResolveSpecifierToPath now makes every resolution decision
(import map, HTTP referrer/origin anchoring, filesystem candidates and
extension probing, node: polyfill fallback) for both the walk and
ResolveModuleCallback - parity by construction. The resolver itself is a
thin dispatch over the classified result.

Fetch completions are delivered as nestable v8 platform tasks posted
through the isolate's own event loop instead of a raw internal-lane hop,
so background-thread import() lands on the right loop and pumps can drive
completions via RunNestableV8Tasks. Teardown quiesces in-flight loads
before the event loop shuts down, so a task the stopped loop rejects
holds only already-Reset Globals.

The synchronous fetch in LoadHttpModuleForUrl survives only as an anomaly
guard: it logs unconditionally in both builds when the walk missed an
edge, and is slated for deletion once the dev-server smoke test proves
coverage.

… module fetches

Both module-fetch transports (the resolver's synchronous fallback and the
graph walk's async pipeline) now hand their response to one
ClassifyModuleResponse: transport error, then 204/205 (no content is a
network error for a module script), then non-2xx, then missing MIME, then
JSON MIME (application/json, text/json, any +json suffix - the response
is a JSON module), then the HTML-spec JavaScript MIME essence list, then
foreign-MIME failure naming the received type. Every failure carries the
URL and the real cause to the importer's rejection, with strings
identical across both paths and both platforms.

A served JSON module now compiles as a JSON module - both HTTP paths had
been compiling JSON bodies as JavaScript, failing with an opaque
SyntaxError. An empty 2xx body with a JS MIME is a valid empty module
(type-only TS modules); an empty JSON body is a failure.

Transport truth: a bare 404 is an answer, not a connection failure - the
single retry now fires on transport error only. HttpURLConnection throws
on 4xx/5xx and may have no error stream, so a status line, once read,
marks the transport turn as answered even without a body (a truncated
2xx body still counts as transport failure). A failed response keeps a
pending cache-bust mark. LoadHttpModuleForUrl throws the classifier's
reason unconditionally in every build - an empty resolve without a
scheduled exception violated the callback contract in release builds.

Settling a promise from a plain task context runs its reactions
synchronously, and a reaction that re-imports the same URL must take the
registry-hit path - not park on a waiter list that was just flushed and
will never be settled again. ResolveHttpDynamicWaiters and
RejectHttpDynamicWaiters therefore detach the waiter vector and erase the
in-flight mark before resolving or rejecting anything. Since fetch
completions arrive as platform tasks, the settle really does run with no
JS on the stack, so the window was live.

…es async graphs

One primitive - ModuleInternal::EvaluateModuleGraph - now evaluates every
ES-module graph under a named policy:

- Sync-strict, Node's require(esm): IsGraphAsync() is refused before
  evaluation (the graph stays instantiated; import() can still load it),
  including registry hits whose evaluation promise is still pending - a
  TLA-parked module reports evaluated, and returning its namespace would
  hand out TDZ bindings. Sync graphs must settle synchronously. Global
  require() of an async-graph module now throws Node-style
  (ERR_REQUIRE_ASYNC_MODULE parity) - a deliberate breaking change.
- Sync-pumping, boot and worker entries: drive nestable platform tasks
  and microtask checkpoints until the capability promise settles or the
  deadline expires. HTTP entries get the 60s deadline and throw; local
  entries get a 1s in-place yield that returns pending rather than
  throwing.
- Async: evaluate and hand back the capability promise.

One deadline constant (kModuleEvaluateDeadlineSeconds = 60) replaces the
30s TLA spin and the scattered 60.0 literals.

Module loading also now fails identically in debug and release: the
debug-only FNV-hash/snippet/heuristic-classification compile diagnostic
is gone, and a failed compile leaves the real SyntaxError (message, line,
column) pending for the importer instead of swallowing it. A module whose
evaluation rejects is evicted from the registry rather than served again.

require() of an ES module now populates module.exports exactly as Node's
populateCJSExportsFromESM: a literal 'module.exports' named export wins
outright; a namespace with no default export, or with its own __esModule,
passes through unchanged; otherwise a synthetic facade module
(export * / export default / __esModule = true) built over the target
provides live bindings, linked through a dedicated resolve callback that
hands the target module through a pending slot no user code can reach.

Facades are cached per target in an identity-hash bucket with handle
confirmation and dropped whenever their target's registry key is
unindexed - eviction and replacement alike - so a reloaded module cannot
serve a stale facade and a replaced one cannot leak it.

…ode: shims

ns:module gains createRequire(filenameOrURL) - Node's argument contract:
absolute path, file: URL string, or URL object; TypeError otherwise;
http(s) bases refused - and createPumpingRequire(filenameOrURL, options).
Options are validated and frozen at mint time (unknown keys throw):
deadlineSeconds (positive finite, default 60), onTimeout
('throw'|'return-pending'), pumpRunLoop (default false). A minted
require's evaluation options ride the require factory as opaque
positional slots and inherit down the dependency tree; the per-directory
require cache is fingerprinted by options so a pumping require can never
be served a strict closure or poison one.

Two new builtins: node:module re-exports createRequire only (a distinct
frozen object - createPumpingRequire has no Node counterpart), and
node:url ships fileURLToPath/pathToFileURL with Node-strict semantics on
primordials-snapshotted intrinsics. The in-resolver node: polyfill
(url/module/path) is deleted: unregistered node: specifiers now fail
uniformly on every path - require, static import, dynamic import - with
'No such built-in module:', and node:path goes away with it (a future
shim candidate, not v1).

Pumping evaluation now refuses to run inside a microtask: a top-level
await resumes via a microtask, so a pump there could never drain the
queue it is running from - it throws up front instead of hanging to the
deadline.

The import map is now the full WHATWG shape - {imports, scopes} - parsed
with the engine's JSON parser and validated completely before anything is
installed: malformed JSON, non-string or empty keys and targets, a
trailing-slash key whose target lacks the slash, and unknown top-level
sections each throw a TypeError naming the offense, in both builds, and
leave the installed vocabulary untouched. The previous parser cleared the
live map before reading its input, so a bad payload emptied a running dev
session's vocabulary.

Scope keys match as plain prefixes of the importing module's canonical
registry key. Resolution cascades most-specific matching scope, then
outer matching scopes, then top-level imports, each consulted with the
same exact-then-longest-trailing-slash-prefix primitive; scopes sort once
at install. The one scoped lookup serves the resolver, the graph walk,
and dynamic import, whose referrer derives from the host-supplied
resource name.

CaptureLoaderVocabulary runs in the WorkerWrapper constructor - on the
parent's JS thread, where the parent's per-isolate state is safely
readable - and the copy rides the wrapper by value into
BackgroundLooper, where InstallLoaderVocabulary writes it into the worker
isolate after runtime init and before any module load. Zero
synchronization: each side only ever touches its own isolate's state.

A worker therefore resolves through the vocabulary its parent had at
spawn; a live worker deliberately does not observe later
reconfiguration - the dev client restarts workers when the vocabulary
changes. The vocabulary types move to the header for the by-value carry.

The runtime provides mechanism; the client supplies vocabulary. Removed:
NormalizeViteSpecifier and both of its second-chance import-map lookups
(clients register every rewritten specifier form verbatim), the
underscore-chunk bare-specifier heuristic, the import('@') empty-stub
sentinel complex on all five sites (a bare '@' now fails loudly as an
unresolvable specifier on every route, and can be mapped through the
import map like any other specifier), the default canonicalization
vocabulary (/ns/, /node_modules/.vite/, /@id/, /@fs/, the /@ng/component
preserve rule, and the import/t/v strip params), and the client-URL-shape
diagnostic labels.

Unconfigured canonicalization is purely mechanical - the fragment is
stripped and nothing else; the query is part of the module's identity
until the client teaches the runtime otherwise via
configureLoader({ canonicalization }). The two supported
configure-before-ESM-traffic bootstrap shapes are documented in the
cross-runtime contract.

…oved

The cold-boot fetch-yield pump is now armed by the runtime itself - a
thread-local RAII depth counter around entry evaluation
(SetBootEvaluationActive) - instead of staying armed from process start
until the dev client called ns:module.setDevBootComplete. The pump runs
exactly while an entry evaluates on the calling thread, a booting worker
can no longer arm the main thread's pump, and the client-visible switch
is gone (clients feature-detect, so absence is a no-op).

g_devSessionBootComplete and the __NS_HMR_BOOT_COMPLETE__ global go with
it; CleanupHttpLoaderGlobals keeps only the process-wide cache-bust
reset.

A worker's message queue now opens exactly when its entry evaluation
settles: immediately for classic and synchronous module entries, and via
a settle continuation on the entry's capability promise for a top-level
await entry - messages posted while the entry is parked buffer and
deliver after settle. A failed entry still routes through onerror first,
then dispatches into a possibly-listenerless global, as on the web.

This replaces the handler-presence probe with its detached 50ms retry
thread, which only recognized the onmessage property (addEventListener
users buffered until the ~2s budget expired and then lost messages) and
gave asynchronously-installed handlers a grace window the web does not
have.

PendingEntryEvaluation performs the probe: module status cannot answer
"is the entry still pending" - a TLA-parked module reports evaluated -
so the capability promise is re-obtained and its state read directly.

Workers get no post-load graph pump on purpose: the entry's transitive
HTTP closure is fetched before evaluation, and anything still in flight
lands on the worker's own event loop, which runWorkerLoop drives three
statements later.

…or them

An app's main entry may be an ES module, top-level await included: the
Java side hands over the resolved main path, routing dispatches on it,
and an .mjs or HTTP main takes the module route under boot evaluation
options (a 1s in-place yield locally that never throws; 60s with a throw
for HTTP entries). A CJS main is byte-identical to before. Load now
enters the context itself so both branches run with a current context
regardless of the caller.

After the entry returns, RunModule holds the process while the entry's
evaluation promise is pending or graph work is in flight, pumping
nestable tasks and microtask checkpoints, bounded at twice the module
deadline (120s). The pending-entry probe reads the capability promise -
module status cannot answer, since a TLA-parked module reports
evaluated. A rejected entry and a 2x-deadline expiry with the entry
still pending are named fatals in every build:

  Fatal: the main entry module's evaluation rejected during boot: <reason>
  Fatal: the main entry module '<path>' never settled within 120s

Graph-only stragglers at the deadline keep the previous log-and-continue
behavior. Workers get no backstop - their settle-gated message queue
covers them.

File::ReadText aborted the process on any path fopen could not open - the
FILE* was fseek'd without a null check - and the ES-module entry routes
(app main, worker main) reach CompileFileEsModule with the caller's
specifier directly, without the resolver's existence probe, so a worker
spawned with an unresolved relative .mjs path died with SIGABRT instead
of an error.

ReadText now returns null for an unreadable file (covering the
deleted-between-stat-and-open race), and CompileFileEsModule stats its
path first, throwing Cannot find module - which routes a missing worker
entry to onerror and a missing main entry to the Java exception path.

Ports the iOS loader specs onto the in-app fixture server: the module
MIME gate and JSON modules over HTTP, mixed local/HTTP graphs, import-map
scopes, canonical keys (mechanical-only unconfigured, client-supplied
vocabulary), createRequire/createPumpingRequire (argument contract,
mint-time options, the microtask re-entrancy guard), require(esm) exports
interop, node:url/node:module, import.meta referrer resolution, worker
vocabulary inheritance, and ES-module worker entries with top-level
await, plus the esm/ fixture tree backing them.

Wires in testNsModule, testNsRuntime, and testRemoteModuleSecurity,
which were added earlier but never required from mainpage.js and so
never ran; testNsRuntime now covers the debug category key and
testRemoteModuleSecurity uses Java accessors that exist. Cleartext is
permitted for 127.0.0.1/localhost only, so the fixture server is
reachable while unroutable-host specs keep failing fast.

879 -> 1013 specs, all green.

The full cross-runtime contract: createRequire/createPumpingRequire,
import maps with scopes and atomic installation, per-isolate vocabulary
with worker copy-at-spawn, the node:url and node:module shims, the debug
trace categories mapped to their logcat tags, ES-module app entries and
the boot backstop's fatal strings, and Android implementation notes
replacing the stale pre-overhaul ones. setDevBootComplete and the
node: polyfill notes are gone with the mechanisms they described.

The loader state moved into per-isolate RuntimeState slots earlier in
this series, but the access sites kept g_-named local aliases so bodies
read unchanged. Rename them to what they are - registry,
modulesInFlight, httpDynamicWaiters - matching the iOS bindings in the
counterpart functions, and drop the comment that excused the aliases.
The surviving g_ identifiers are genuinely process-wide: the cache-bust
set, the fetch-yield hook, the trace mask, the in-flight graph counter,
the shared allocator, the error-id counter, and the crash-breadcrumb
store.

Tracks iOS b4b8d8e4 (the reference restructure) and ee51d741 (the
validation contract): the per-module reference tables, import maps and
scopes with the full validation error table, registry canonicalization,
reconfiguration and workers, the require() specifier and require(esm)
sections, pumping requires, the module response contract, and app
entries and bootstraps - with the Android platform notes (logcat trace
tags, the boot backstop's fatal strings, the settle-gated worker queue)
replacing the iOS ones, and no releasedObjectPolicy key.

Android's Looper::pollInner holds a Response& into its response vector
across each fd-callback dispatch. The module pumps (evaluation,
graph-load, boot backstop, fetch yield) call ALooper_pollOnce on the
calling thread, and since fetch completions, platform tasks and worker
messages run JS from inside the EventLoop's eventfd/timerfd callbacks,
a pump reached from there re-entered pollInner, which clears and
reallocates the vector - the outer poll then resumes over freed memory.
On slower emulators the freed block was reliably reused (the tombstone
shows a module-path string overwriting the response entry, the fault
address four UTF-16 characters of "testapplication"), killing the
process in Looper::pollOnce during unrelated suites. CFRunLoopRunInMode
is re-entrancy-safe, so iOS never had the hazard.

EventLoop now tracks a thread-local dispatch depth around both fd
callbacks, and every pump consults EventLoop::IsInLooperCallback():
inside a dispatch it drains the nestable-task queue and microtasks
directly and yields, instead of polling. Reproduced on run 1 of an
API-33 emulator loop before the guard; four consecutive full-suite runs
pass after it.

The looper-callback guard substituted a fixed 1ms usleep for the skipped
ALooper_pollOnce, paying the full millisecond per iteration even when a
fetch completion or platform task had already landed. WaitForInternalWork
polls the loop's own eventfd and timerfd - the same wakeups the looper
would have delivered, without entering it - so nested pumps wake the
moment internal-lane work arrives and idle at the same 10ms cap the
un-nested path uses. The non-pumping TLA wait, which carried the same
blind 1ms spin, gets the same treatment.