netapp-volumes · diff
git:20261001.b18ac39 to git:20261014.681e9c1
95 added, 469 removed. Audit A to A.
---
name: netapp-volumes
- description: Work with Domino Volumes for NetApp ONTAP - enterprise-grade, multi-terabyte storage with near-instant snapshots. Covers volume creation, snapshot versioning with commit messages, cross-project sharing, mount paths (/mnt/netapp-volumes/ or /domino/netapp-volumes/), and the NetApp Volumes REST API. Use when managing large-scale data storage, needing fast no-copy snapshots, or integrating existing NetApp ONTAP infrastructure with Domino.
+ description: Work with Domino Volumes for NetApp ONTAP (NetApp Volumes) - shared, multi-terabyte storage with near-instant snapshots. Covers creating volumes, attaching them to projects, mount paths (/mnt/netapp-volumes/ or /domino/netapp-volumes/), snapshots and tags, roles, deletion, mounting volumes in jobs, and the NetApp Volumes REST API (/remotefs/v1). Use when managing large-scale shared data, taking fast no-copy snapshots, or integrating NetApp ONTAP storage with Domino.
+ compatibility: Domino 6.3 and Domino Cloud. Requires DOMINO_API_HOST; in a run use DOMINO_API_PROXY or the localhost:8899 access-token endpoint, outside a run a Personal Access Token. An administrator must have registered a NetApp ONTAP filesystem.
---
- # Domino NetApp Volumes Skill
-
- ## Description
- This skill helps users work with Domino Volumes for NetApp ONTAP — enterprise-grade storage that mounts NetApp ONTAP filesystems directly into Domino workloads, enabling multi-terabyte scale with near-instant snapshotting.
-
- ## Activation
- Activate this skill when users want to:
- - Create or manage NetApp Volumes in Domino
- - Work with NetApp volume snapshots and versioning
- - Share large-scale data across projects or teams
- - Access existing NetApp ONTAP storage from Domino workspaces, jobs, or apps
- - Use the NetApp Volumes REST API programmatically
- - Understand mount paths for NetApp volumes
- - Choose between NetApp Volumes and Domino Datasets
-
- ## What is a Domino NetApp Volume?
+ # Domino NetApp Volumes
- A Domino NetApp Volume is:
- - **Enterprise-grade storage**: Backed by NetApp ONTAP via Kubernetes PVCs with the `netapp-storage` storage class
- - **Multi-terabyte scale**: No practical upper limit — suitable for very large datasets
- - **Near-instant snapshots**: ~3 seconds even for 100+ GB volumes using redirect-on-write (no extra storage consumed at snapshot time)
- - **Versioned with commit messages**: Snapshots support human-readable tags and commit messages, unlike Datasets
- - **Shareable**: Attach a single volume to multiple projects and teams
- - **Persistent**: Data persists across executions
+ Applies to Domino 6.3 and Domino Cloud.
- ### NetApp Volumes vs. Domino Datasets
+ Domino Volumes for NetApp ONTAP mount NetApp ONTAP filesystems directly into Domino
+ Workspaces, Jobs, Apps and Launchers. They are global entities (not scoped to a project)
+ that you attach to any project that needs them. Product docs:
+ https://docs.domino.ai/cloud/platform-capabilities/core-concepts/data/netapp-volumes.
+ Auth: [domino-api-intro](../domino-api-intro/SKILL.md#authentication); in a run the
+ `/remotefs/v1` routes answer only on `DOMINO_REMOTE_FILE_SYSTEM_HOSTPORT` with a bearer
+ ([API-NETAPP-VOLUMES.md](./API-NETAPP-VOLUMES.md#authentication-and-base-urls)).
- | Feature | NetApp Volumes | Domino Datasets |
- |---------|---------------|-----------------|
- | Storage backend | External NetApp ONTAP | Domino-managed NFS/EFS |
- | Data scale | Multi-terabyte and beyond | Up to ~1 TB |
- | Snapshot speed | ~3 seconds (any size) | Scales with data size |
- | Snapshot storage cost | No extra space (redirect-on-write) | Duplicates physical data |
- | Commit messages on snapshots | Supported | Not supported |
- | Create new volume from snapshot | Read-only clones | Can create editable dataset |
- | Cross-project sharing | Yes | Yes |
- | Admin prerequisite | Admin must register filesystems | None |
- | REST API | Full dedicated API | Via Domino Python SDK |
+ ## NetApp Volumes or Domino Datasets?
- **Use NetApp Volumes when:**
- - Data exceeds ~1 TB
- - Near-instant snapshotting is required
- - Your organization already has NetApp ONTAP infrastructure
- - You need snapshot commit messages for audit trails
- - High-performance shared storage across many teams
+ NetApp Volumes cover nearly the same ground as Datasets and are recommended wherever the
+ organization runs ONTAP. Differences:
- **Use Domino Datasets when:**
- - Data is Domino-managed with no external infrastructure dependency
- - You need to create editable new versions from snapshots
- - Data is under ~1 TB
+ - Snapshots are near-instant regardless of size (about 3 seconds) and use
+ redirect-on-write, so they consume no extra storage at creation; a Dataset snapshot
+ duplicates the data.
+ - Datasets can be cloned from a snapshot into a new Dataset; a NetApp Volume snapshot
+ cannot be cloned into a new volume (volumes can be copied for Project Templates).
+ - NetApp Volumes need ONTAP infrastructure and an administrator-registered filesystem;
+ Domino sets no volume size limit (the ONTAP volume type does). There are limits on
+ snapshots per volume (`maxSnapshots` on the volume record) and volumes per filesystem.
- ---
+ Comparison and examples: [DATA-ACCESS.md](./DATA-ACCESS.md).
## Prerequisites
- An admin must register at least one NetApp filesystem (Kubernetes PVC with the `netapp-storage` label) before users can create volumes. Contact your Domino administrator if no filesystems are available.
-
- ---
-
- ## Creating a NetApp Volume
-
- ### From the Domino Home Page
- 1. Navigate to **Data > NetApp Volumes** in the toolbar
- 2. Click **Add NetApp Volume > Create Volume**
- 3. Fill in the form:
- - **Name**: Letters, numbers, underscores, hyphens only
- - **Description**: Brief overview of the data
- - **Data Plane**: Select from available options
- - **NetApp Filesystem**: Choose a registered filesystem
- - **Capacity**: Set maximum storage allocation
- 4. Click **Next** to configure permissions
- 5. Assign users with **Reader / Editor / Owner** roles
- 6. Click **Finish**
- 7. Manually add the volume to projects afterward
-
- ### From within a Project (recommended — auto-associates)
- 1. Open a project → **Data > NetApp Volumes** (left panel)
- 2. Click **Add NetApp Volume > Create Volume**
- 3. Fill in the same form fields as above
- 4. The volume is automatically associated with the current project on creation
-
- ### Via REST API
- ```python
- import os
- import requests
-
- # Auth token from in-cluster token service
- token = requests.get("http://localhost:8899/access-token").text.strip()
- headers = {
- "Authorization": f"Bearer {token}",
- "Content-Type": "application/json"
- }
-
- api_url = os.environ["DOMINO_API_HOST"]
- remotefs_url = os.environ["DOMINO_REMOTE_FILE_SYSTEM_HOSTPORT"]
-
- # Look up your user ID (username is available in the DOMINO_USER_NAME env var)
- user_resp = requests.get(
- f"{api_url}/v4/users?userName={os.environ['DOMINO_USER_NAME']}",
- headers=headers
- ).json()
- user_id = user_resp[0]["id"]
-
- # Create a volume (capacity is in bytes; grants is required)
- response = requests.post(
- f"{remotefs_url}/remotefs/v1/volumes",
- headers=headers,
- json={
- "name": "large-training-data",
- "description": "Multi-TB training dataset for vision models",
- "filesystemId": "<filesystem-id>",
- "capacity": 5_000_000_000_000, # 5 TB in bytes
- "grants": [
- {"targetId": user_id, "targetRole": "VolumeOwner"}
- ]
- }
- )
- volume = response.json()
- print(f"Created volume ID: {volume['id']}")
- ```
-
- ---
-
- ## Adding a Volume to a Project
-
- ### Via Domino UI
- 1. Go to project → **Data > NetApp Volumes**
- 2. Click **Add NetApp Volume > Add Existing Volume**
- 3. Select the volume from the list
- 4. Configure access level for the project
-
- ### Via REST API
- ```python
- # Attach a volume to a project
- requests.post(
- f"{remotefs_url}/remotefs/v1/rpc/attach-volume-to-project",
- headers=headers,
- json={
- "volumeId": "<volume-id>",
- "projectId": "<project-id>"
- }
- )
- ```
-
- ---
-
- ## Mount Paths
-
- Mount paths depend on your **project type**. Check which exists in your execution to determine your project type.
-
- ### Git-Based Projects
-
- | Volume Type | Mount Path |
- |-------------|-----------|
- | Live volume | `/mnt/netapp-volumes/<volume-name>/` |
- | Snapshot by number | `/mnt/netapp-volumes/snapshots/<volume-name>/<snapshot-number>/` |
- | Snapshot by tag | `/mnt/netapp-volumes/snapshot-tags/<volume-name>/<tag-name>/` |
-
- ### DFS (Domino File System) Projects
-
- | Volume Type | Mount Path |
- |-------------|-----------|
- | Live volume | `/domino/netapp-volumes/<volume-name>/` |
- | Snapshot by number | `/domino/netapp-volumes/snapshots/<volume-name>/<snapshot-number>/` |
- | Snapshot by tag | `/domino/netapp-volumes/snapshot-tags/<volume-name>/<tag-name>/` |
-
- ### Snapshot Access Behavior
-
- There are two ways to access snapshots, with an important behavioral difference:
-
- - **`/snapshots/<volume-name>/<number>/`** — Accessed by snapshot number. When a new snapshot is taken while a workspace is running, the new numbered directory appears **immediately** in that workspace without a restart.
- - **`/snapshot-tags/<volume-name>/<tag-name>/`** — Accessed by tag name. Each snapshot has at most one active tag path — a symlink to its numbered snapshot directory. If you apply multiple tags to the same snapshot, only the most recently applied tag creates a path; earlier tags for that snapshot are not accessible by path. Tag paths for new snapshots are also **not** visible in a running workspace — you must **restart the workspace** for them to appear.
-
- Use the numbered path when you need to access a fresh snapshot from within a live workspace. Use the tagged path for stable, named references in reproducible runs.
-
- > **Important:** Each snapshot exposes only one tag path at a time — the most recently applied tag. Older tags on the same snapshot do not have accessible paths.
-
- > **Important:** Renaming a volume changes its mount path. Update any hardcoded paths in your code after renaming.
-
- ### Identify Your Project Type
-
- ```python
- import os
-
- if os.path.exists("/domino/netapp-volumes"):
- print("DFS Project")
- netapp_root = "/domino/netapp-volumes"
- elif os.path.exists("/mnt/netapp-volumes"):
- print("Git-Based Project")
- netapp_root = "/mnt/netapp-volumes"
- ```
-
- ### Permissions
-
- - **Owners/Editors**: Read-write access to the live volume
- - **Readers**: Read-only access
-
- ### Example: Reading Data
-
- ```python
- import pandas as pd
-
- # Git-Based Project
- df = pd.read_parquet("/mnt/netapp-volumes/large-training-data/features.parquet")
-
- # DFS Project
- df = pd.read_parquet("/domino/netapp-volumes/large-training-data/features.parquet")
-
- # Read from a specific snapshot tag
- df = pd.read_parquet("/mnt/netapp-volumes/snapshot-tags/large-training-data/v2.0/features.parquet")
- ```
-
- ### Example: Writing Data
-
- ```python
- # Write directly to the live volume
- df.to_parquet("/mnt/netapp-volumes/large-training-data/processed/output.parquet", index=False)
-
- # List files
- import os
- files = os.listdir("/mnt/netapp-volumes/large-training-data/")
- ```
-
- ---
-
- ## Snapshots and Versioning
-
- ### What is a Snapshot?
- A snapshot is a read-only, immutable record of the volume's data at a specific point in time. NetApp snapshots use redirect-on-write — no additional storage is consumed at creation time.
-
- **Advantages over Domino Dataset snapshots:**
- - Near-instant for any volume size (~3 seconds)
- - Support commit messages for documentation and audit trails
- - No storage overhead at creation time
-
- ### Create a Snapshot via UI
- 1. Go to project → **Data > NetApp Volumes**
- 2. Select the volume → click **Take Snapshot**
- 3. Add an optional **commit message** (e.g., "Added Q4 customer records")
- 4. Add an optional **tag name** (e.g., `v2.0`, `production`, `2024-Q4`)
- 5. Click **Confirm**
-
- ### Create a Snapshot via REST API
- ```python
- # Create snapshot with description and one or more tags (tagNames is an array)
- response = requests.post(
- f"{remotefs_url}/remotefs/v1/snapshots",
- headers=headers,
- json={
- "volumeId": "<volume-id>",
- "description": "Added Q4 customer records — 50M new rows",
- "tagNames": ["v2.0"]
- }
- )
- snapshot = response.json()
- print(f"Snapshot ID: {snapshot['id']}")
- ```
-
- ### Add a Tag to an Existing Snapshot
- ```python
- requests.post(
- f"{remotefs_url}/remotefs/v1/snapshots/{snapshot_id}/tags",
- headers=headers,
- json={"name": "production"}
- )
- ```
-
- ### Create Snapshot After a Job Run
- ```python
- # Snapshot tied to a specific Domino run for reproducibility
- requests.post(
- f"{remotefs_url}/remotefs/v1/rpc/create-snapshot-from-run",
- headers=headers,
- json={
- "volumeId": "<volume-id>",
- "runId": "<domino-run-id>",
- "userId": "<user-id>",
- "description": "post-training-run-42"
- }
- )
- ```
-
- ### Restore a Snapshot
- ```python
- requests.post(
- f"{remotefs_url}/remotefs/v1/rpc/restore-snapshot",
- headers=headers,
- json={"snapshotId": "<snapshot-id>"}
- )
- ```
-
- ---
-
- ## Roles and Permissions
-
- | Role | API Value | Capabilities |
- |------|-----------|-------------|
- | **Reader** | `VolumeReader` | View files and snapshots; mount as read-only |
- | **Editor** | `VolumeEditor` | All Reader capabilities + modify description, create/delete snapshots, manage shared access, manage users |
- | **Owner** | `VolumeOwner` | All Editor capabilities + update volume grants, request deletion |
-
- ### Update Permissions via UI
- 1. Go to **Data > NetApp Volumes** → select volume
- 2. Click the three-dot menu → **Edit permissions**
- 3. Add/remove users and assign roles
-
- ### Update Permissions via REST API
- ```python
- # targetRole values: "VolumeOwner", "VolumeEditor", "VolumeReader"
- requests.put(
- f"{remotefs_url}/remotefs/v1/volumes/{volume_id}/grants",
- headers=headers,
- json=[
- {"targetId": "<user-id>", "targetRole": "VolumeEditor"},
- {"targetId": "<other-user-id>", "targetRole": "VolumeReader"}
- ]
- )
- ```
-
- ---
-
- ## Using NetApp Volumes in Jobs
-
- ### Via Domino REST API
- ```python
- requests.post(
- f"{api_url}/api/jobs/v1/jobs",
- headers=headers,
- json={
- "projectId": "<project-id>",
- "runCommand": "python train.py",
- "hardwareTierId": "<hardware-tier-id>",
- "environmentId": "<environment-id>", # required — use DOMINO_ENVIRONMENT_ID env var
- "netAppVolumeIds": ["<volume-id>"],
- "snapshotNetAppVolumesOnCompletion": True # auto-snapshot mounted volumes when job finishes
- }
- )
- ```
-
- Set `snapshotNetAppVolumesOnCompletion: true` to automatically take a snapshot of all mounted NetApp volumes when the job completes. This is the recommended approach for training jobs — it captures the exact state of the volume at the end of the run without requiring a separate API call.
-
- ---
-
- ## Listing Volumes and Snapshots
-
- ```python
- # List all volumes accessible to you
- volumes = requests.get(
- f"{remotefs_url}/remotefs/v1/volumes",
- headers=headers
- ).json()
-
- for v in volumes["data"]:
- capacity_tb = v["capacity"] / 1_000_000_000_000
- print(f"{v['name']} — {capacity_tb:.1f} TB — ID: {v['id']}")
-
- # List snapshots for a volume
- snapshots = requests.get(
- f"{remotefs_url}/remotefs/v1/snapshots",
- headers=headers,
- params={"volumeId": "<volume-id>"}
- ).json()
-
- for s in snapshots["data"]:
- print(f"Snapshot {s['id']} v{s['version']} — {s.get('description', '')} — tags: {[t['name'] for t in s.get('tags', [])]}")
- ```
-
- ---
-
- ## Best Practices
-
- ### 1. Use Appropriate Storage
- | Data Type | Storage |
- |-----------|---------|
- | Large data of any file type where the entire filesystem should be versioned as a unit, with intentional snapshots | NetApp Volume |
- | Small output files — charts, reports, model binaries | Artifacts / DFS (auto-versioned per file, not suitable for large files) |
- | Code | Git / Project files |
+ An administrator registers an ONTAP filesystem (data plane, storage class, root PVC) before
+ anyone can create volumes: https://docs.domino.ai/cloud/admin/data-administration/netapp-volumes.
+ List registered filesystems with `GET /remotefs/v1/filesystems`
+ ([API-NETAPP-VOLUMES.md](./API-NETAPP-VOLUMES.md#filesystems)).
- ### 2. Snapshot Before Changes
- ```python
- # Always snapshot before modifying large volumes
- requests.post(
- f"{remotefs_url}/remotefs/v1/snapshots",
- headers=headers,
- json={
- "volumeId": "<volume-id>",
- "description": "Pre-processing baseline snapshot",
- "tagNames": ["pre-processing-2024-01"]
- }
- )
+ ## Create a volume
- # Then run your data transformation
- process_data()
- ```
+ From Domino (then add it to projects manually) or from inside a project (auto-attached):
- ### 3. Use Efficient File Formats
- ```python
- # Parquet for tabular data (faster reads, smaller storage)
- df.to_parquet("/mnt/netapp-volumes/dataset/data.parquet")
+ 1. **Data > NetApp Volumes** (toolbar, or a project's left panel).
+ 2. **Add NetApp Volume > Create Volume**.
+ 3. **Name** (letters, numbers, underscores, hyphens), **Description**, **Data Plane**,
+ **NetApp Filesystem**, **Capacity**. Click **Next**.
+ 4. Add users or organizations with a **Reader**, **Editor** or **Owner** role. **Finish**.
- # Feather for fast pandas I/O
- df.to_feather("/mnt/netapp-volumes/dataset/data.feather")
+ REST: `POST /remotefs/v1/volumes` with `name`, `filesystemId`, `capacity` (bytes) and
+ `grants` ([API-NETAPP-VOLUMES.md](./API-NETAPP-VOLUMES.md#create-a-volume)).
- # HDF5 for large numerical arrays
- import h5py
- with h5py.File("/mnt/netapp-volumes/dataset/arrays.h5", "w") as f:
- f.create_dataset("features", data=features_array)
- ```
+ ## Attach, detach, delete
- ### 4. Organize Data
- ```
- /mnt/netapp-volumes/my-volume/
- ├── raw/
- │ ├── 2024-Q1/
- │ └── 2024-Q2/
- ├── processed/
- │ ├── features.parquet
- │ └── labels.parquet
- └── metadata/
- └── schema.json
- ```
+ - **Attach**: **Data > NetApp Volumes**, three-dot menu on the volume, **Add to project**.
+ REST: `POST /remotefs/v1/rpc/attach-volume-to-project`.
+ - **Detach**: in the project, **Data > NetApp Volumes**, **Remove from project**. Only the
+ attachment goes; data and other projects are unaffected. REST:
+ `POST /remotefs/v1/rpc/detach-volume-from-project`.
+ - **Delete**: **Delete Volume** (if your admin enabled it for your role) or **Mark for
+ deletion** (requests that an admin delete it). Deletion is permanent. REST:
+ `POST /remotefs/v1/rpc/delete-volume` (no need to mark first) or
+ `/rpc/mark-volume-for-deletion` (undo: `/rpc/restore-volume`).
- ### 5. Use Snapshot Tags for Reproducibility
- Reference snapshot tags (not snapshot IDs) in your training scripts so that tagged paths remain stable across runs:
- ```python
- # Reproducible reference using a tag
- TRAINING_DATA = "/mnt/netapp-volumes/snapshot-tags/dataset/v2.0/"
- df = pd.read_parquet(f"{TRAINING_DATA}/features.parquet")
- ```
+ ## Mount paths
- ### 6. Reading Large Volumes Efficiently
- ```python
- import dask.dataframe as dd
+ Paths mirror the Dataset scheme; volumes are global, so there is no local/imported split.
- # Lazy read — no data loaded until .compute()
- df = dd.read_parquet("/mnt/netapp-volumes/dataset/large_data.parquet")
- result = df.groupby("category").mean().compute()
- ```
+ | Project type | Read-write volume | Snapshot by number | Tagged snapshot |
+ |--------------|-------------------|--------------------|-----------------|
+ | DFS | `/domino/netapp-volumes/<name>` | `/domino/netapp-volumes/snapshots/<name>/<number>` | `/domino/netapp-volumes/snapshot-tags/<name>/<tag>` |
+ | Git-based | `/mnt/netapp-volumes/<name>` | `/mnt/netapp-volumes/snapshots/<name>/<number>` | `/mnt/netapp-volumes/snapshot-tags/<name>/<tag>` |
- ---
+ `<number>` is the snapshot's `version` from the API. Owners and Editors mount read-write;
+ Readers mount read-only; Apps mount with the App owner's permission. If a snapshot has several tags, only the most
+ recently added tag is mounted. Renaming a volume changes its mount path. At launch you can
+ pin a volume to one snapshot (**Data Plane & Hardware** tab); it then mounts read-only at
+ the read-write path and `snapshots/<name>` is disabled. Details:
+ [DATA-ACCESS.md](./DATA-ACCESS.md).
- ## Troubleshooting
+ ## Snapshots and tags
- ### Volume Not Found at Mount Path
- - Verify the volume is added to the project (**Data > NetApp Volumes**)
- - Confirm the volume name matches the path exactly (case-sensitive)
- - Check that the workspace/job was started after the volume was added to the project
- - Restart the workspace to pick up newly added volumes
+ Owners and Editors take snapshots: open the volume, **Files** tab, **Take Snapshot**
+ (all files or selected files), optionally add a commit message and one or more tags.
+ Tag names: letters, numbers, underscores and hyphens only (no dots), unique within the
+ volume. The button is unavailable once the volume
+ reaches its maximum snapshot count. Deleting a snapshot is permanent and needs delete
+ permission on the volume. Docs:
+ https://docs.domino.ai/cloud/platform-capabilities/core-concepts/data/datasets/version-data-with-snapshots.
- ### Permission Denied
- - Check your role on the volume (need Editor or Owner to write)
- - Verify you have been granted access by the volume owner
- - For read-only snapshots, use the snapshot tag path instead of the live volume path
+ REST: `POST /remotefs/v1/snapshots` (`volumeId`, `description` as the commit message,
+ `tagNames`), `POST /remotefs/v1/snapshots/{id}/tags`, `PUT /remotefs/v1/snapshots/{id}`,
+ `/rpc/restore-snapshot`, `/rpc/delete-snapshot`
+ ([API-NETAPP-VOLUMES.md](./API-NETAPP-VOLUMES.md#snapshots)). `/rpc/create-snapshot-from-run`
+ is super-user only (403 otherwise); to snapshot at the end of a job use
+ `snapshotNetAppVolumesOnCompletion` where enabled, or `POST /remotefs/v1/snapshots` from
+ the job.
- ### Snapshot Tag Not Mounted
- - Each snapshot has at most **one** tag path — the most recently applied tag. Earlier tags on the same snapshot are never surfaced as directories and cannot be accessed by path.
- - Tag paths for new snapshots do **not** appear in a running workspace. The numbered snapshot directory (`/snapshots/<name>/<number>/`) appears immediately, but the tag symlink path only becomes visible after restarting the workspace.
- - Verify the tag was created successfully via the UI or API
+ ## Roles
- ### No NetApp Filesystems Available
- - Contact your Domino administrator — they must register NetApp ONTAP filesystems before volumes can be created
- - Admins can register filesystems via **Admin > NetApp Volumes**
+ | Role | API value | Capabilities |
+ |------|-----------|---------------------|
+ | Reader | `VolumeReader` | Read data and view snapshots; attach to projects read-only; can only mark a snapshot for deletion |
+ | Editor | `VolumeEditor` | Reader plus write data, edit the description, take and delete snapshots, mount, mark for deletion or restore |
+ | Owner | `VolumeOwner` | Editor plus change who has access and their roles (grants), request volume deletion |
- ### Slow Read Performance
- - Use columnar formats (Parquet, Feather) instead of CSV for tabular data
- - Read only needed columns: `pd.read_parquet(path, columns=["col1", "col2"])`
- - Use Dask or chunked reading for files larger than available RAM
+ Edit permissions: three-dot menu on the volume, **Edit permissions**, or **Share** on the
+ details page. REST: `PUT /remotefs/v1/volumes/{id}/grants`.
- ---
+ ## Volumes in Jobs
- ## Documentation Reference
+ `POST /api/jobs/v1/jobs` takes `netAppVolumeIds` (volume ids to mount) and
+ `snapshotNetAppVolumesOnCompletion: true` to snapshot every mounted volume when the job
+ ends; the latter needs a deployment feature flag and is rejected with 400 where it is
+ off. Body: [API-NETAPP-VOLUMES.md](./API-NETAPP-VOLUMES.md#mount-volumes-in-a-job).
- Before writing or verifying any API call, use the cluster swagger to confirm current endpoint paths and field names. Use public docs for workflow context and field explanations.
+ ## Reference files
- **Get the cluster base URL:** `$DOMINO_API_HOST` (injected by Domino into every workspace, job, and app).
+ - [API-NETAPP-VOLUMES.md](./API-NETAPP-VOLUMES.md): REST API with request bodies (volumes, filesystems, attach/detach, snapshots, tags, grants, deletion, listing, files, jobs)
+ - [DATA-ACCESS.md](./DATA-ACCESS.md): comparison with Datasets, mount paths, snapshot mounting, read/write examples, best practices
+ - [TROUBLESHOOTING.md](./TROUBLESHOOTING.md): missing mounts, permission errors, tag paths, no filesystems, slow reads
- Fetch the NetApp Volumes swagger spec (requires bearer token):
- ```bash
- TOKEN=$(curl -s http://localhost:8899/access-token)
- # The swagger UI is only accessible via the external cluster URL (not $DOMINO_API_HOST).
- # Derive it from the JWT iss claim — works in any workspace type.
- CLUSTER_URL=$(echo $TOKEN | cut -d'.' -f2 | python3 -c "
- import sys, base64, json, re
- p = sys.stdin.read().strip()
- p += '=' * (-len(p) % 4)
- print(re.sub(r'/auth/realms/.*', '', json.loads(base64.b64decode(p))['iss']))
- ")
- curl -H "Authorization: Bearer $TOKEN" "$CLUSTER_URL/domino-netapp-volumes/swagger/doc.json"
- # Browser UI (must be logged in): $CLUSTER_URL/domino-netapp-volumes/swagger/index.html
- ```
+ ## Documentation
- **Public docs (workflow context and field explanations):**
- - [NetApp Volumes REST API Reference](https://docs.dominodatalab.com/en/cloud/api_guide/b3b2a1/domino-netapp-volumes-api/)
- - [Work with NetApp Volumes](https://docs.dominodatalab.com/en/cloud/user_guide/06da1b/work-with-netapp-volumes/)
- - [Create NetApp Volumes](https://docs.dominodatalab.com/en/cloud/user_guide/e6887f/create-netapp-volumes-from-domino-or-a-project/)
- - [Add or Remove NetApp Volumes on Projects](https://docs.dominodatalab.com/en/cloud/user_guide/306570/add-or-remove-netapp-volumes-on-projects/)
- - [View and Edit NetApp Volumes](https://docs.dominodatalab.com/en/cloud/user_guide/93fa12/view-and-edit-netapp-volumes/)
- - [Version Data with Snapshots](https://docs.dominodatalab.com/en/cloud/user_guide/dbdbff/version-data-with-snapshots/)
- - [Access Data in Domino (comparison guide)](https://docs.dominodatalab.com/en/cloud/user_guide/16d9c1/access-data-in-domino/)
+ - https://docs.domino.ai/cloud/platform-capabilities/core-concepts/data/netapp-volumes (overview, create, add/remove, view/edit)
+ - https://docs.domino.ai/cloud/reference/api/domino-volumes-api (API served at `/remotefs/v1`)
+ - https://docs.domino.ai/cloud/admin/data-administration/netapp-volumes (administrator setup)
+ - Specs: [API-SPECS.md](../domino-api-intro/API-SPECS.md); per-deployment service document `/docs/openapi/data_remotefs_service-swagger.json`