Prevent path traversal via sharded-checkpoint index filenames - #47996

Fixes #47176.

Diagnosis

When from_pretrained loads a sharded checkpoint, get_checkpoint_shard_files (src/transformers/utils/hub.py) reads the shard filenames straight from the model-supplied index (model.safetensors.index.json / pytorch_model.bin.index.json) and joins each to the model directory with os.path.join and no validation:

shard_filenames = sorted(set(index["weight_map"].values()))  # attacker-controlled
...
shard_filenames = [os.path.join(pretrained_model_name_or_path, subfolder, f) for f in shard_filenames]

The weight_map values are arbitrary strings from the checkpoint. os.path.join does not neutralize .., and an absolute value discards the directory prefix entirely, so a malicious index escapes the model directory. The returned paths then flow into safe_open(...) / torch.load(...), so a plain AutoModel.from_pretrained("/untrusted/model/dir"):

• reads arbitrary files as an existence/error oracle (e.g. weight_map = {"w": "/etc/passwd"}), and
• for a safetensors-format target, fully exfiltrates its tensor bytes through the load path (e.g. another model's weights on a shared host) — weight_map = {"stolen": "../victim/private.safetensors"}.

This is the "load a model someone gave me" threat model (a downloaded repo, a shared directory on a multi-tenant host). No auth required beyond getting the victim to load the directory.

Fix

Reject any shard filename that isn't a plain filename sitting next to the index, before it is joined:

for shard_filename in shard_filenames:
    if os.path.basename(shard_filename) != shard_filename or shard_filename in (os.curdir, os.pardir):
        raise ValueError(...)

Placed right after the weight_map values are read, so it guards both the local-directory and the Hub-download paths. Legitimate index files are always produced with sibling shard filenames (model-00001-of-0000N.safetensors), confirmed against the sharded-save code in modeling_utils.py, so normal loading is unaffected.

Tests

tests/utils/test_hub_utils.py::GetCheckpointShardFilesTest:

• plain sibling filenames still resolve and load;
• ../, sub/dir/..., .., and absolute (/etc/passwd) weight_map values raise ValueError.

Verified locally: the traversal test fails on main (ValueError not raised) and passes with the fix. ruff check/ruff format clean on the changed lines; the diff is a single guard plus the test.