# HuggingFace Plugin

Model management and downloads from HuggingFace repositories.

## Overview

**Enabled by Default**: Manages HuggingFace model downloads with caching. Supports multiple repositories and models with configurable paths for weights and config files.

## Configuration

Configure models in `huggingface.yaml`:

```yaml
plugin.huggingface:
  enabled: true

huggingface:
  repositories:
    reflectorch:
      root_dir_env_var: "REFLECTOMETRY_ROOT_DIR"
      repo_id: "valentinsingularity/reflectivity"
      files:
        - source: "configs/b_mc_point_xray_conv_standard_L2_InputQ.yaml"
          target: "reflectorch/configs/b_mc_point_xray_conv_standard_L2_InputQ.yaml"
        - source: "saved_models/model_b_mc_point_xray_conv_standard_L2_InputQ.safetensors"
          target: "reflectorch/saved_models/model_b_mc_point_xray_conv_standard_L2_InputQ.safetensors"

    panpe:
      root_dir_env_var: "REFLECTOMETRY_ROOT_DIR"
      repo_id: "HW-SC/panpe-xrr-reflectometry"
      files:
        - source: "configs/panpe-2layers-xrr.yaml"
          target: "panpe/configs/panpe-2layers-xrr.yaml"
        - source: "saved_models/model_panpe-2layers-xrr.pt"
          target: "panpe/saved_models/model_panpe-2layers-xrr.pt"
```

### Configuration Fields

| Field | Type | Description |
|-------|------|-------------|
| `repositories` | `dict` | Collection of HuggingFace repositories |
| `root_dir_env_var` | `str` | Environment variable that points to the local model root |
| `repo_id` | `str` | HuggingFace repository identifier (org/repo) |
| `files` | `list` | List of files to download from the repository |
| `source` | `str` | Path to the file inside the HuggingFace repository |
| `target` | `str` | Path below the local model root |

## Model Storage

### Environment Variables

Two environment variable mechanisms control where models are stored:

**`HF_HOME`** - General HuggingFace cache directory (if unset, plugin uses a temporary directory):
```bash
export HF_HOME=${PWD}/storage/huggingface_cache
```

**`root_dir_env_var`** - Environment variable that defines the persistent local model root for a configured repository.

In the current VIPR configuration, both reflectometry repositories use the shared reflectometry root:

- `reflectorch` -> `REFLECTOMETRY_ROOT_DIR`
- `panpe` -> `REFLECTOMETRY_ROOT_DIR`

Example:
```bash
export REFLECTOMETRY_ROOT_DIR=${PWD}/storage/reflectometry
```

### Directory Structure

With `REFLECTOMETRY_ROOT_DIR` set, reflectometry models are organized as:
```
storage/reflectometry/
├── reflectorch/
│   ├── saved_models/     # Model weights (.safetensors, .pt)
│   └── configs/          # Model configuration files (.yaml)
└── panpe/
    ├── saved_models/
    └── configs/
```

Other models use the standard HuggingFace cache structure in `HF_HOME`.

## CLI Commands

### Download Models

Download configured HuggingFace model files:

```bash
vipr huggingface download
```

This downloads all configured models from HuggingFace repositories to the configured local model roots. For custom deployment additions, set `VIPR_HUGGINGFACE_CONFIG` to an additional HuggingFace YAML before running the command.

**Use Cases:**
- Pre-deployment: Download models before production
- Offline scenarios: Cache models for air-gapped environments
- CI/CD: Prepare models during build process

## Adding Custom Models

### Step 1: Identify Repository

Find your model on HuggingFace: `https://huggingface.co/organization/repository-name`

### Step 2: Add to Configuration

```yaml
huggingface:
  repositories:
    my_project:  # Unique identifier for your use
      root_dir_env_var: "MY_PROJECT_MODEL_ROOT"
      repo_id: "organization/repository-name"  # HuggingFace repo
      files:
        - source: "path/in/repo/config.yaml"
          target: "my_project/configs/config.yaml"
        - source: "path/in/repo/model.safetensors"
          target: "my_project/saved_models/model.safetensors"
```

### Step 3: Download

```bash
vipr huggingface download
```

## Supported Weight Formats

- `.safetensors` - Recommended format (faster, safer, no pickle)
- `.pt` - Standard PyTorch format
- `.bin` - Older PyTorch format (pickle-based)

## Example: Multiple Models

```yaml
huggingface:
  repositories:
    reflectorch:
      root_dir_env_var: "REFLECTOMETRY_ROOT_DIR"
      repo_id: "valentinsingularity/reflectivity"
      files:
        - source: "configs/b_mc_point_xray_conv_standard_L2_InputQ.yaml"
          target: "reflectorch/configs/b_mc_point_xray_conv_standard_L2_InputQ.yaml"
        - source: "saved_models/model_b_mc_point_xray_conv_standard_L2_InputQ.safetensors"
          target: "reflectorch/saved_models/model_b_mc_point_xray_conv_standard_L2_InputQ.safetensors"

    panpe:
      root_dir_env_var: "REFLECTOMETRY_ROOT_DIR"
      repo_id: "HW-SC/panpe-xrr-reflectometry"
      files:
        - source: "configs/panpe-2layers-xrr.yaml"
          target: "panpe/configs/panpe-2layers-xrr.yaml"
        - source: "saved_models/model_panpe-2layers-xrr.pt"
          target: "panpe/saved_models/model_panpe-2layers-xrr.pt"
```

## Download Behavior

- `vipr huggingface download` attempts to download all configured model files.
- Existing files are skipped automatically (unless forced in code).
- API startup downloads HuggingFace models by default. Set `VIPR_NO_MODEL_DOWNLOAD=true` to skip the startup download.

## See Also

- [Inference Plugin](inference.md) - Model loading and inference
- [HuggingFace Hub Documentation](https://huggingface.co/docs/hub) - Repository structure
