pride-database · diff
git:20260428.81ea2d2 to git:20260522.d4a4f76
450 added, 650 removed. Audit A to A.
---
name: "pride-database"
- description: "Search PRIDE Archive REST API for proteomics datasets, peptide IDs, and MS raw files. Find experiments by organism, tissue, disease, or instrument; download RAW/mzML; retrieve peptide/PSM IDs and protein-level evidence. Use interpro-database for domains; uniprot-protein-database for sequences."
+ description: "Search the PRIDE Archive v3 REST API for proteomics datasets: discover projects by keyword + faceted filters (organism, instrument, disease, software), fetch project metadata, list and download RAW/PEAK/RESULT/FASTA files (with FTP/Aspera URLs), look up which projects mention a UniProt accession, and find similar projects. PRIDE v3 no longer exposes peptide/PSM-level identification endpoints — for spectrum-level data download the project's RESULT files. Use uniprot-protein-database for protein sequences; interpro-database for domain architecture."
license: "Apache-2.0"
---
# PRIDE Database
## Overview
- The PRIDE Archive (ProteomicsIDEntifications database) at EBI is the world's largest public repository of mass spectrometry-based proteomics data, containing 30,000+ datasets from peer-reviewed publications. The REST API v2 at `https://www.ebi.ac.uk/pride/ws/archive/v2/` provides project discovery, file listing, peptide/PSM identification retrieval, and protein-level evidence — all without authentication. Data types include RAW files, peak lists (mzML, MGF), PRIDE XML result files, and processed identification tables.
+ The PRIDE Archive (ProteomicsIDEntifications database) at EMBL-EBI is the world's largest public mass-spectrometry proteomics repository — 39,000+ projects and 3.4M+ deposited files as of 2026. Programmatic access is via a JSON REST API at `https://www.ebi.ac.uk/pride/ws/archive/v3/`. No authentication is required. The OpenAPI/Swagger spec is at `https://www.ebi.ac.uk/pride/ws/archive/v3/v3/api-docs`. PRIDE v3 returns **plain JSON arrays** for list endpoints (no HAL+JSON `_embedded` envelope) and intentionally does not expose per-peptide or per-PSM identification endpoints — for spectrum-level identifications, download the project's `RESULT` files (mzIdentML, MaxQuant txt, etc.) and parse them locally.
## When to Use
- - Finding published proteomics datasets by organism, tissue, disease keyword, or instrument type for meta-analysis or benchmarking
- - Downloading raw mass spectrometry data (RAW, mzML) or peak files (MGF) from a specific PRIDE project accession
- - Retrieving peptide identification tables with sequence, modification, and confidence score for a project
- - Querying protein-level evidence (PSMs, unique peptides) for a protein of interest across PRIDE projects
- - Checking whether a protein has experimental proteomics evidence in a specific tissue or disease context
- - Building training datasets of confident peptide-spectrum matches (PSMs) for proteomics ML applications
- - For protein domain and family classification use `interpro-database`; PRIDE provides experimental identification evidence only
+ - Finding published proteomics datasets by free-text keyword and facet filters (organism, tissue, disease, instrument, software, PTM) for meta-analysis or benchmarking
+ - Downloading raw mass-spectrometry data (RAW, mzML, MGF) or pre-processed identifications (RESULT files) from a specific PRIDE project accession
+ - Looking up which PRIDE projects mention a specific UniProt protein accession (project-level occurrence map only — no PSM/coverage counts at the API surface)
+ - Finding similar projects to one of interest for reanalysis or cross-study comparison
+ - Fetching SDRF (Sample-Data Relationship Format) files for projects so you can model the sample-to-MS-run mapping programmatically
+ - Discovering valid filter values via faceted search before constructing a structured query
- For protein sequences, Swiss-Prot annotations, and ID mapping use `uniprot-protein-database`
+ - For protein domain and family classification use `interpro-database` — PRIDE only reports project-level occurrence, not domain-level features
+ - **PRIDE v3 has no `/peptides`, `/psms`, or `/proteins?proteinAccession=` endpoints** — if you need peptide- or PSM-level data, download the RESULT files from `/projects/{accession}/files` and parse them with `pyteomics` or a search-engine-specific reader
## Prerequisites
- **Python packages**: `requests`, `pandas`, `matplotlib`
- - **Data requirements**: PRIDE project accessions (e.g., `PXD000001`) or search keywords (tissue, organism, disease)
- - **Environment**: internet connection; no API key or account required
- - **Rate limits**: ~50 requests/minute; add `time.sleep(1.2)` between sequential project or file queries to stay within limits
+ - **Data requirements**: a PRIDE project accession (`PXD######` format) or a search keyword, optionally a UniProt accession for protein-occurrence lookup
+ - **Environment**: internet connection; no API key required
+ - **Rate limits**: not formally published; keep bursts under ~5 requests/second and add `time.sleep(0.3)` in loops
```bash
pip install requests pandas matplotlib
```
## Quick Start
```python
import requests
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- def pride_get(endpoint: str, params: dict = None) -> dict:
- """Send a GET request to the PRIDE API and return parsed JSON."""
- r = requests.get(
- f"{PRIDE_BASE}/{endpoint}",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- return r.json()
+ # 1) Free-text search for cancer proteomics projects
+ projects = requests.get(f"{PRIDE}/search/projects",
+ params={"keyword": "prostate cancer", "pageSize": 5},
+ timeout=30).json()
+ print(f"Top {len(projects)} projects:")
+ for p in projects[:3]:
+ instr = ", ".join(p.get("instruments", []))[:50]
+ print(f" {p['accession']} {(p['title'] or '')[:70]} [{instr}]")
- # Quick lookup: project details for a known accession
- project = pride_get("projects/PXD000001")
- print(f"Project: {project['accession']}")
- print(f"Title: {project['title'][:80]}")
- print(f"Organisms: {[o['name'] for o in project.get('organisms', [])]}")
- print(f"Submitted: {project.get('submissionDate', 'N/A')}")
- # Project: PXD000001
- # Title: TMT spikes — iPRG2014 Study
+ # 2) Drill into one project
+ acc = projects[0]["accession"]
+ proj = requests.get(f"{PRIDE}/projects/{acc}", timeout=30).json()
+ print(f"\n{proj['accession']}: {proj['title'][:70]}")
+ print(f" Submitted: {proj.get('submissionDate')} DOI: {proj.get('doi')}")
+ print(f" Organisms: {[o['name'] for o in proj.get('organisms', [])]}")
+ print(f" Instruments: {[i['name'] for i in proj.get('instruments', [])]}")
+
+ # 3) List files and total size
+ files = requests.get(f"{PRIDE}/projects/{acc}/files/all", timeout=60).json()
+ total_mb = sum(f.get("fileSizeBytes", 0) for f in files) / 1e6
+ print(f"\n {len(files)} files, {total_mb:.0f} MB total")
```
## Core API
- ### Query 1: Project Search
+ ### Module 1: Project Search — `/search/projects`
- Search PRIDE projects by keyword, organism, tissue, disease, or instrument. Returns paginated project summaries with accession, title, and metadata.
+ Free-text search with optional facet-based filtering, pagination, and sorting. Returns a plain JSON array of project records — there is no HAL+JSON `_embedded`/`page` wrapper.
```python
- import requests
- import pandas as pd
+ import requests, pandas as pd
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- def search_projects(keyword: str = None, organism: str = None,
- tissue: str = None, disease: str = None,
- instrument: str = None,
- page_size: int = 25, page: int = 0) -> dict:
- """Search PRIDE projects by metadata fields.
+ def search_projects(keyword=None, organism=None, instrument=None,
+ disease=None, software=None,
+ page_size=25, page=0, sort_field="submission_date",
+ sort_direction="DESC"):
+ """Search PRIDE v3 for projects.
+ Filter syntax (for the `filter` arg) is `field==value, field==value` using `_facet` field names
+ that are discoverable via /facet/projects."""
+ filters = []
+ if organism: filters.append(f"organisms_facet=={organism}")
+ if instrument: filters.append(f"instruments_facet=={instrument}")
+ if disease: filters.append(f"diseases_facet=={disease}")
+ if software: filters.append(f"softwares_facet=={software}")
- Parameters
- ----------
- keyword : str
- Free-text keyword search across title and description.
- organism : str
- Organism name filter (e.g., 'Homo sapiens').
- tissue : str
- Tissue filter (e.g., 'liver', 'plasma').
- disease : str
- Disease filter (e.g., 'cancer', 'Alzheimer').
- instrument : str
- Instrument filter (e.g., 'Orbitrap').
- page_size : int
- Results per page (max 100).
- page : int
- Page number for pagination (0-indexed).
- """
- params = {"pageSize": page_size, "page": page}
- if keyword:
- params["keyword"] = keyword
- if organism:
- params["organisms"] = organism
- if tissue:
- params["tissues"] = tissue
- if disease:
- params["diseases"] = disease
- if instrument:
- params["instruments"] = instrument
+ params = {"pageSize": page_size, "page": page,
+ "sortFields": sort_field, "sortDirection": sort_direction}
+ if keyword: params["keyword"] = keyword
+ if filters: params["filter"] = ",".join(filters)
- r = requests.get(
- f"{PRIDE_BASE}/projects",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
+ r = requests.get(f"{PRIDE}/search/projects", params=params, timeout=30)
r.raise_for_status()
- return r.json()
+ return r.json() # plain list[dict]
- result = search_projects(organism="Homo sapiens", tissue="liver", page_size=10)
- projects = result.get("_embedded", {}).get("compactprojects", [])
- total = result.get("page", {}).get("totalElements", 0)
- print(f"Total liver proteomics projects: {total}")
- print(f"First page: {len(projects)} projects")
- for p in projects[:5]:
- print(f" {p['accession']} {p['title'][:60]}")
- # Total liver proteomics projects: 523
- # PXD012345 Liver proteome profiling in NAFLD
+ projects = search_projects(keyword="cancer", organism="Homo sapiens (human)",
+ instrument="Q Exactive", page_size=5)
+ df = pd.DataFrame([{
+ "accession": p["accession"],
+ "title": (p.get("title") or "")[:70],
+ "submission_date": p.get("submissionDate"),
+ "diseases": ", ".join(p.get("diseases", []))[:60],
+ "instruments": ", ".join(p.get("instruments", []))[:50],
+ } for p in projects])
+ print(df.to_string(index=False))
```
```python
- # Paginate through all results for a disease keyword
- def get_all_projects(keyword: str, page_size: int = 100) -> pd.DataFrame:
- """Retrieve all matching projects across pages."""
- records, page = [], 0
- while True:
- data = search_projects(keyword=keyword, page_size=page_size, page=page)
- batch = data.get("_embedded", {}).get("compactprojects", [])
+ # Paginate through all matches for a keyword. The API doesn't return total counts inline;
+ # walk pages until the next one is empty.
+ def search_all_projects(keyword, page_size=100, max_pages=20):
+ all_records, page = [], 0
+ while page < max_pages:
+ batch = search_projects(keyword=keyword, page_size=page_size, page=page)
if not batch:
break
- for p in batch:
- records.append({
- "accession": p.get("accession"),
- "title": p.get("title", "")[:100],
- "submission_date": p.get("submissionDate"),
- "publication_date": p.get("publicationDate"),
- "n_files": p.get("filesCount", 0),
- })
- total = data.get("page", {}).get("totalPages", 1)
+ all_records.extend(batch)
+ if len(batch) < page_size:
+ break # last page
page += 1
- if page >= total:
- break
- return pd.DataFrame(records)
+ return all_records
- df = get_all_projects("colorectal cancer", page_size=50)
- print(f"Colorectal cancer projects: {len(df)}")
- df.to_csv("pride_colorectal_projects.csv", index=False)
+ results = search_all_projects("phosphoproteomics", page_size=100, max_pages=3)
+ print(f"Phosphoproteomics projects collected (max 300): {len(results)}")
```
- ### Query 2: Project Details
+ ### Module 2: Faceted Filter Discovery — `/facet/projects`
- Retrieve complete metadata for a specific project by its PRIDE accession (PXD######).
+ Before constructing a filtered search, query the facet endpoint to see which instrument / organism / disease / software values actually exist for a given keyword, along with their counts. The response is a dict of facet groups, each mapping `{value: count}`.
```python
- import requests
-
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ import requests, pandas as pd
- def get_project(accession: str) -> dict:
- """Fetch full metadata for a PRIDE project.
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- Parameters
- ----------
- accession : str
- PRIDE accession (e.g., 'PXD000001').
- """
- r = requests.get(
- f"{PRIDE_BASE}/projects/{accession}",
- headers={"Accept": "application/json"},
- timeout=30
- )
+ def get_facets(keyword=None, facet_page_size=20):
+ """Return facet counts for projects matching `keyword`. Keys are facet groups
+ (instruments, organisms, diseases, softwares, experimentTypes, ...); values are
+ dicts of {value: count}."""
+ params = {"facetPageSize": facet_page_size}
+ if keyword: params["keyword"] = keyword
+ r = requests.get(f"{PRIDE}/facet/projects", params=params, timeout=30)
r.raise_for_status()
return r.json()
- project = get_project("PXD004131")
- print(f"Accession : {project['accession']}")
- print(f"Title : {project['title'][:80]}")
- print(f"Submission date : {project.get('submissionDate', 'N/A')}")
- print(f"Publication date: {project.get('publicationDate', 'N/A')}")
- organisms = [o["name"] for o in project.get("organisms", [])]
- print(f"Organisms : {organisms}")
- tissues = [t["name"] for t in project.get("tissues", [])]
- print(f"Tissues : {tissues}")
- instruments = [i["name"] for i in project.get("instruments", [])]
- print(f"Instruments : {instruments}")
- ptms = [m["name"] for m in project.get("ptms", [])]
- print(f"PTMs : {ptms[:5]}")
- print(f"References : {[r.get('doi') for r in project.get('references', [])[:2]]}")
+ facets = get_facets(keyword="cancer", facet_page_size=10)
+ print(f"Facet groups: {list(facets.keys())}")
+ print(f"\nTop instruments for 'cancer':")
+ for instr, n in sorted(facets.get("instruments", {}).items(), key=lambda kv: -kv[1])[:8]:
+ print(f" {instr:<35} {n}")
+ print(f"\nTop diseases:")
+ for d, n in sorted(facets.get("diseases", {}).items(), key=lambda kv: -kv[1])[:6]:
+ print(f" {d:<55} {n}")
```
- ### Query 3: Project Files
+ ### Module 3: Project Detail — `/projects/{accession}`
- List all files in a project with their types (RAW, PEAK, RESULT, FASTA, OTHER) and download URLs.
+ Full metadata for a single project: submitters, labPIs, instruments, organisms (CV-coded), diseases, experiment types, references, DOI, submission/publication dates. Lists are `CvParam`-style objects with `accession`, `cvLabel`, `name`, optionally `value`.
```python
import requests
- import pandas as pd
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- def get_project_files(accession: str, file_type: str = None,
- page_size: int = 100) -> pd.DataFrame:
- """List files available in a PRIDE project.
+ def get_project(accession):
+ r = requests.get(f"{PRIDE}/projects/{accession}", timeout=30)
+ r.raise_for_status()
+ return r.json()
- Parameters
- ----------
- accession : str
- PRIDE accession.
- file_type : str
- Filter by file type: 'RAW', 'PEAK', 'RESULT', 'FASTA', 'OTHER'.
- page_size : int
- Files per page.
- """
- params = {"pageSize": page_size, "page": 0}
- if file_type:
- params["fileType"] = file_type
+ p = get_project("PXD004131")
+ print(f"Accession : {p['accession']}")
+ print(f"Title : {p['title'][:80]}")
+ print(f"Submission : {p.get('submissionDate')}")
+ print(f"Publication : {p.get('publicationDate')}")
+ print(f"DOI : {p.get('doi')}")
+ print(f"License : {p.get('license')}")
+ print(f"Type : {p.get('submissionType')}")
+ print(f"Organisms : {[o['name'] for o in p.get('organisms', [])]}")
+ print(f"Instruments : {[i['name'] for i in p.get('instruments', [])]}")
+ print(f"Experiment : {[e['name'] for e in p.get('experimentTypes', [])]}")
+ print(f"PIs : {[pi.get('name') for pi in p.get('labPIs', [])]}")
+ print(f"References : {[r.get('doi') for r in p.get('references', [])[:3]]}")
+ ```
- records, page = [], 0
+ ### Module 4: Project Files — `/projects/{accession}/files` + `/files/all`
+
+ List the files associated with a project. Use the paginated endpoint for large projects; `/files/all` returns every file in one shot. Each file record carries `fileCategory.value` (one of `RAW`, `PEAK`, `RESULT`, `FASTA`, `OTHER`), `fileSizeBytes` (note the `Bytes` suffix — not `fileSize`), and a list of `publicFileLocations` each labeled `FTP Protocol` or `Aspera Protocol`.
+
+ ```python
+ import requests, pandas as pd
+
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
+
+ def get_project_files(accession, file_type=None, page_size=100):
+ """Walk paginated /files for a project. Optionally filter by category code
+ (RAW, PEAK, RESULT, FASTA, OTHER). Returns a DataFrame."""
+ rows, page = [], 0
while True:
- params["page"] = page
- r = requests.get(
- f"{PRIDE_BASE}/projects/{accession}/files",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
+ r = requests.get(f"{PRIDE}/projects/{accession}/files",
+ params={"pageSize": page_size, "page": page},
+ timeout=30)
r.raise_for_status()
- data = r.json()
- batch = data.get("_embedded", {}).get("files", [])
+ batch = r.json()
if not batch:
break
for f in batch:
- records.append({
+ cat = f.get("fileCategory") or {}
+ ftp = next((loc["value"] for loc in f.get("publicFileLocations", [])
+ if loc.get("name") == "FTP Protocol"), "")
+ asp = next((loc["value"] for loc in f.get("publicFileLocations", [])
+ if loc.get("name") == "Aspera Protocol"), "")
+ rows.append({
"file_name": f.get("fileName"),
- "file_type": f.get("fileCategory", {}).get("value", ""),
- "size_bytes": f.get("fileSize", 0),
- "download_url": f.get("publicFileLocations", [{}])[0].get("value", ""),
+ "category": cat.get("value"), # RAW/PEAK/RESULT/FASTA/OTHER
+ "size_mb": round((f.get("fileSizeBytes") or 0) / 1e6, 2),
+ "ftp_url": ftp,
+ "aspera_url": asp,
+ "downloads": f.get("totalDownloads"),
})
- total_pages = data.get("page", {}).get("totalPages", 1)
- page += 1
- if page >= total_pages:
+ if len(batch) < page_size:
break
+ page += 1
- df = pd.DataFrame(records)
- df["size_mb"] = (df["size_bytes"] / 1e6).round(1)
+ df = pd.DataFrame(rows)
+ if file_type:
+ df = df[df["category"] == file_type]
return df
files_df = get_project_files("PXD004131")
print(f"Total files: {len(files_df)}")
- print(files_df.groupby("file_type")["file_name"].count())
+ print(files_df.groupby("category")["size_mb"].agg(["count", "sum"]).round(1).to_string())
- # RAW files only
- raw_files = get_project_files("PXD004131", file_type="RAW")
- print(f"\nRAW files: {len(raw_files)}")
- print(f"Total size: {raw_files['size_mb'].sum():.0f} MB")
- print(raw_files[["file_name", "size_mb", "download_url"]].head(5).to_string(index=False))
+ raw_only = files_df[files_df["category"] == "RAW"]
+ print(f"\nRAW files: {len(raw_only)}; combined {raw_only['size_mb'].sum():.0f} MB")
+ print(raw_only[["file_name", "size_mb", "downloads"]].head(5).to_string(index=False))
```
- ### Query 4: Peptide Identifications
-
- Retrieve peptide identifications for a project or search by peptide sequence, modification, or protein accession.
-
```python
- import requests
- import pandas as pd
+ # /files/all returns every file in one response — convenient for small projects
+ files = requests.get(f"{PRIDE}/projects/PXD000001/files/all", timeout=60).json()
+ print(f"PXD000001 files (all): {len(files)}")
+ for f in files[:4]:
+ print(f" [{f.get('fileCategory',{}).get('value','?'):<6}] {f['fileName']} "
+ f"{f.get('fileSizeBytes',0)/1e6:.2f} MB")
+ ```
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ ### Module 5: SDRF File — `/files/sdrf/{projectAccession}`
- def get_peptides(project_accession: str = None, peptide_sequence: str = None,
- protein_accession: str = None,
- page_size: int = 100, page: int = 0) -> pd.DataFrame:
- """Retrieve peptide identifications from PRIDE.
+ PRIDE projects that follow the modern submission standard include an SDRF (Sample-Data Relationship Format) TSV that maps each MS run to its biological sample, treatment, label, fraction, etc. Pull it once, parse it as a TSV.
- Parameters
- ----------
- project_accession : str
- Filter by PRIDE project accession.
- peptide_sequence : str
- Filter by exact or partial peptide sequence.
- protein_accession : str
- Filter by UniProt protein accession.
- page_size : int
- Results per page.
- """
- params = {"pageSize": page_size, "page": page}
- if project_accession:
- params["projectAccessions"] = project_accession
- if peptide_sequence:
- params["peptideSequence"] = peptide_sequence
- if protein_accession:
- params["proteinAccession"] = protein_accession
+ ```python
+ import requests, pandas as pd, io
- r = requests.get(
- f"{PRIDE_BASE}/peptides",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- data = r.json()
- records = data.get("_embedded", {}).get("peptideevidences", [])
- rows = []
- for rec in records:
- rows.append({
- "peptide_sequence": rec.get("peptideSequence"),
- "protein_accession": rec.get("proteinAccession"),
- "project_accession": rec.get("projectAccession"),
- "ptms": str([m.get("modification") for m in rec.get("modifications", [])]),
- "num_psms": rec.get("numberPSMs", 0),
- })
- return pd.DataFrame(rows)
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- # Get peptides from a specific project
- pep_df = get_peptides(project_accession="PXD004131", page_size=50)
- print(f"Peptides retrieved: {len(pep_df)}")
- if len(pep_df) > 0:
- print(pep_df[["peptide_sequence", "protein_accession", "num_psms"]].head(8).to_string(index=False))
- ```
+ def get_sdrf(accession):
+ """Fetch the SDRF sample-to-run mapping for a project (404 if not provided)."""
+ r = requests.get(f"{PRIDE}/files/sdrf/{accession}", timeout=30)
+ if r.status_code == 404:
+ return None
+ r.raise_for_status()
+ return pd.read_csv(io.StringIO(r.text), sep="\t")
- ```python
- # Search peptides by sequence across all PRIDE
- pep_hits = get_peptides(peptide_sequence="PEPTIDER", page_size=25)
- print(f"PSM hits for 'PEPTIDER': {len(pep_hits)}")
- print(pep_hits.groupby("project_accession")["peptide_sequence"].count())
+ # Many older projects have no SDRF — newer ones typically do
+ sdrf = get_sdrf("PXD000001")
+ if sdrf is None or sdrf.empty:
+ print("No SDRF available for this project")
+ else:
+ print(f"SDRF rows: {len(sdrf)} cols: {len(sdrf.columns)}")
+ print(f"First columns: {list(sdrf.columns)[:8]}")
```
- ### Query 5: PSM (Peptide-Spectrum Match) Retrieval
+ ### Module 6: Protein → Project Mapping — `/proteins/{accession}`
- Retrieve individual PSMs with spectrum references, modifications, and confidence scores.
+ PRIDE v3's protein endpoint returns *only* the list of project accessions that contain identifications for the given UniProt accession. It does **not** return PSM counts, peptide counts, or sequence coverage — those are not exposed at the API surface in v3. For depth metrics you must download a project's `RESULT` files and parse them locally.
```python
import requests
- import pandas as pd
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
-
- def get_psms(project_accession: str = None, peptide_sequence: str = None,
- protein_accession: str = None,
- page_size: int = 100) -> pd.DataFrame:
- """Retrieve PSMs (peptide-spectrum matches) from PRIDE.
-
- Parameters
- ----------
- project_accession : str
- Filter by project accession.
- peptide_sequence : str
- Filter by peptide sequence.
- protein_accession : str
- Filter by UniProt protein accession.
- """
- params = {"pageSize": page_size, "page": 0}
- if project_accession:
- params["projectAccessions"] = project_accession
- if peptide_sequence:
- params["peptideSequence"] = peptide_sequence
- if protein_accession:
- params["proteinAccession"] = protein_accession
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- r = requests.get(
- f"{PRIDE_BASE}/psms",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
+ def get_protein_projects(uniprot_acc):
+ """Return the list of PRIDE project accessions that mention this UniProt accession.
+ No PSM/peptide/coverage counts are available at this endpoint."""
+ r = requests.get(f"{PRIDE}/proteins/{uniprot_acc}", timeout=30)
+ if r.status_code == 404:
+ return None
r.raise_for_status()
data = r.json()
- records = data.get("_embedded", {}).get("psms", [])
- rows = []
- for rec in records:
- rows.append({
- "psm_id": rec.get("psmId"),
- "peptide_sequence": rec.get("peptideSequence"),
- "protein_accession": rec.get("proteinAccession"),
- "project_accession": rec.get("projectAccession"),
- "charge": rec.get("charge"),
- "calculated_mass": rec.get("calculatedMassToCharge"),
- "experimental_mass": rec.get("experimentalMassToCharge"),
- "spectrum_id": rec.get("spectrumID"),
- })
- return pd.DataFrame(rows)
+ return data.get("projects", [])
- psm_df = get_psms(project_accession="PXD004131", page_size=50)
- print(f"PSMs retrieved: {len(psm_df)}")
- if len(psm_df) > 0:
- print(psm_df[["peptide_sequence", "protein_accession", "charge", "spectrum_id"]].head(5).to_string(index=False))
+ tp53 = get_protein_projects("P04637")
+ print(f"TP53 (P04637) is reported in {len(tp53)} PRIDE projects")
+ print(f"First 8: {tp53[:8]}")
+
+ unknown = get_protein_projects("Q99999")
+ print(f"\nQ99999 (no real protein): "
+ f"{'no PRIDE evidence' if not unknown else f'{len(unknown)} projects'}")
```
- ### Query 6: Protein Evidence
+ ### Module 7: Discovery Helpers — Similar Projects, Autocomplete
- Query protein-level identification data — unique peptides, PSM counts, and cross-project evidence for a specific UniProt accession.
+ `/projects/{accession}/similarProjects` returns projects with related metadata signatures (organism, instrument, experiment type, tags). `/search/autocomplete?keyword=...` returns project titles starting with the prefix — useful to suggest searches.
```python
- import requests
- import pandas as pd
+ import requests, pandas as pd
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- def get_protein_evidence(protein_accession: str,
- page_size: int = 50) -> pd.DataFrame:
- """Retrieve protein identification evidence across PRIDE projects.
+ # Find similar projects to one of interest
+ similar = requests.get(f"{PRIDE}/projects/PXD004131/similarProjects",
+ params={"pageSize": 5}, timeout=30).json()
+ print(f"Similar to PXD004131: {len(similar)} projects")
+ for p in similar[:5]:
+ print(f" {p['accession']} {(p.get('title') or '')[:70]}")
- Parameters
- ----------
- protein_accession : str
- UniProt accession (e.g., 'P04637' for TP53).
- page_size : int
- Results per page.
- """
- params = {
- "proteinAccession": protein_accession,
- "pageSize": page_size,
- "page": 0
- }
- r = requests.get(
- f"{PRIDE_BASE}/proteins",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- data = r.json()
- records = data.get("_embedded", {}).get("proteinevidences", [])
- rows = []
- for rec in records:
- rows.append({
- "protein_accession": rec.get("proteinAccession"),
- "project_accession": rec.get("projectAccession"),
- "num_peptides": rec.get("numberPeptides", 0),
- "num_psms": rec.get("numberPSMs", 0),
- "coverage": rec.get("sequenceCoverage"),
- })
- return pd.DataFrame(rows)
+ # Autocomplete suggestions for a project-title prefix
+ suggestions = requests.get(f"{PRIDE}/search/autocomplete",
+ params={"keyword": "tp53"}, timeout=30).json()
+ print(f"\nAutocomplete for 'tp53': {len(suggestions)} suggestions")
+ for s in suggestions[:5]:
+ print(f" {s}")
+ ```
- prot_df = get_protein_evidence("P04637") # TP53
- print(f"TP53 (P04637) evidence across PRIDE: {len(prot_df)} entries")
- if len(prot_df) > 0:
- prot_df = prot_df.sort_values("num_psms", ascending=False)
- print(prot_df[["project_accession", "num_peptides", "num_psms", "coverage"]].head(10).to_string(index=False))
- print(f"\nTotal projects with TP53 evidence: {prot_df['project_accession'].nunique()}")
+ ### Module 8: Repository-Wide Counts — `/projects/count`, `/files/count`
+
+ Get total counts across the repository — useful for status displays and sanity checks. Both endpoints return a plain integer body (no JSON object wrapper).
+
+ ```python
+ import requests
+
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
+
+ n_projects = int(requests.get(f"{PRIDE}/projects/count", timeout=30).text)
+ n_files = int(requests.get(f"{PRIDE}/files/count", timeout=30).text)
+ print(f"PRIDE Archive current scale:")
+ print(f" Projects: {n_projects:,}")
+ print(f" Files: {n_files:,}")
```
## Key Concepts
- ### PRIDE File Types
-
- Each PRIDE project can contain several file categories:
+ ### Plain JSON Arrays, No HAL Envelope
- | File type | Description | Format examples |
- |-----------|-------------|-----------------|
- | `RAW` | Unprocessed instrument output | .raw (Thermo), .d (Bruker/Agilent) |
- | `PEAK` | Centroided or deconvoluted spectra | mzML, mzXML, MGF |
- | `RESULT` | Identification results from search engine | mzIdentML, PRIDE XML, MaxQuant txt |
- | `FASTA` | Protein sequence databases used in search | .fasta |
- | `OTHER` | Supplementary files (scripts, tables) | .txt, .xlsx, .csv |
+ PRIDE v3 list endpoints return plain JSON arrays — for example `/search/projects` returns `[{...}, {...}, ...]` directly. There is no `_embedded.compactprojects`, no `page.totalElements`/`totalPages`, no `_links.next.href`. Older PRIDE v2 clients that parsed `data["_embedded"]["compactprojects"]` will silently return empty against the current API. To paginate, walk `page=0, 1, 2, ...` until you get an empty array (or a partial page shorter than `pageSize`).
- For reanalysis pipelines, start with `RESULT` files for pre-processed identifications or `PEAK` files for re-searching spectra. Use `RAW` files only when you need to re-acquire spectra from raw vendor formats.
+ ### What v3 Removed
- ### Accession Formats
+ The endpoint families below no longer exist in v3 (and v2 is now an alias for v3 internally — error messages from `/v2/peptides` literally report `path: "/pride/ws/archive/v3/peptides"`):
- PRIDE project accessions use the format `PXD######` (ProteomXchange dataset ID). The same accession is referenced in publications and indexed by ProteomXchange partner repositories (MassIVE, jPOST, iProX). Peptide IDs, PSM IDs, and protein IDs in the API responses use project-scoped internal identifiers.
+ | Removed endpoint | Status in v3 | Replacement |
+ |---|---|---|
+ | `GET /peptides?projectAccessions=X` | 404 | None — download project's RESULT files and parse |
+ | `GET /psms?projectAccessions=X` | 404 | None — download RESULT files |
+ | `GET /proteins?proteinAccession=X` (query-param style) | 404 | `GET /proteins/{accession}` (path-param) |
+ | HAL+JSON `_embedded`/`page` wrapper | Gone | Plain JSON array |
+ | `/projects?keyword=...&organisms=...&tissues=...` filters | Silently ignored | `/search/projects?keyword=...&filter=field==value` |
- ### Pagination
+ ### Filter Syntax on `/search/projects`
- All PRIDE API list endpoints are paginated. The response includes a `page` object with `totalElements`, `totalPages`, `size`, and `number` fields. Use `page` (0-indexed) and `pageSize` parameters. For large result sets, iterate until `page >= totalPages`.
+ The `filter` query parameter takes a comma-separated list of `field==value` constraints. Field names use the `_facet` suffix (the underlying Solr-style field). Discover valid field names and values via `/facet/projects` before constructing the filter:
```python
- import requests
+ # Valid filter forms
+ "organisms_facet==Homo sapiens (human)"
+ "instruments_facet==Q Exactive"
+ "diseases_facet==Prostate adenocarcinoma"
+ "softwares_facet==MaxQuant"
- def paginate_pride(endpoint: str, params: dict, result_key: str) -> list:
- """Generic paginator for any PRIDE list endpoint."""
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
- all_records, page = [], 0
- while True:
- params["page"] = page
- r = requests.get(
- f"{PRIDE_BASE}/{endpoint}",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- data = r.json()
- batch = data.get("_embedded", {}).get(result_key, [])
- all_records.extend(batch)
- total_pages = data.get("page", {}).get("totalPages", 1)
- page += 1
- if page >= total_pages or not batch:
- break
- return all_records
+ # Combine with commas
+ filter="organisms_facet==Homo sapiens (human),instruments_facet==Orbitrap Fusion Lumos"
```
+ ### File Categories
+
+ Each file in a project carries a `fileCategory` CV-param. The `.value` is a category code; the `.name` is the human-readable label:
+
+ | `value` code | Description | Common formats |
+ |---|---|---|
+ | `RAW` | Unprocessed instrument output | .raw (Thermo), .d (Bruker/Agilent), .wiff (Sciex) |
+ | `PEAK` | Centroided / deconvoluted spectra | .mzML, .mzXML, .mgf |
+ | `RESULT` | Identification results | .mzid, .mzTab, MaxQuant txt, PRIDE XML |
+ | `FASTA` | Protein sequence database used in search | .fasta |
+ | `OTHER` | Supplementary / scripts / tables | .txt, .xlsx, .csv |
+
+ For reanalysis pipelines, `RESULT` is the cheapest entry point — pre-identified peptides without re-searching spectra. `PEAK` lets you re-search with a different engine. `RAW` is only needed for full vendor-format reprocessing.
+
+ ### Accession Formats
+
+ PRIDE project accessions follow ProteomeXchange format `PXD######`. These are stable across PRIDE, MassIVE, jPOST, and iProX. File accessions inside PRIDE are SHA-256-style hashes (e.g., `5bda360133398f66021c8889e01dce921cb51300c7269e1f2b0f20368ab20af6`) — opaque identifiers; use `fileName` for human-readable filenames.
+
## Common Workflows
- ### Workflow 1: Disease Proteomics Dataset Discovery
+ ### Workflow 1: Faceted Discovery — From Disease Keyword to Filtered Project List
- **Goal**: Find all PRIDE projects for a disease, summarize available data types, and export a ranked project list for manual review.
+ **Goal**: Start from a disease keyword, see which instruments and softwares are common in matching datasets via facet counts, then pull a filtered project list using one of the top values.
```python
- import requests, time
- import pandas as pd
+ import requests, pandas as pd
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
+ disease_kw = "colorectal cancer"
- def discover_projects(disease: str, organism: str = "Homo sapiens",
- page_size: int = 50) -> pd.DataFrame:
- """Retrieve and summarize all PRIDE projects for a disease."""
- records, page = [], 0
- while True:
- params = {
- "keyword": disease,
- "organisms": organism,
- "pageSize": page_size,
- "page": page
- }
- r = requests.get(
- f"{PRIDE_BASE}/projects",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- data = r.json()
- batch = data.get("_embedded", {}).get("compactprojects", [])
- for p in batch:
- records.append({
- "accession": p.get("accession"),
- "title": p.get("title", "")[:100],
- "submission_date": p.get("submissionDate"),
- "n_files": p.get("filesCount", 0),
- "instruments": ", ".join(
- i.get("name", "") for i in p.get("instruments", [])[:2]
- ),
- "tissues": ", ".join(
- t.get("name", "") for t in p.get("tissues", [])[:2]
- ),
- })
- total_pages = data.get("page", {}).get("totalPages", 1)
- page += 1
- if page >= total_pages or not batch:
- break
- time.sleep(1.2)
+ # 1) Inspect facet counts to learn which filter values dominate
+ facets = requests.get(f"{PRIDE}/facet/projects",
+ params={"keyword": disease_kw, "facetPageSize": 10},
+ timeout=30).json()
- df = pd.DataFrame(records).sort_values("submission_date", ascending=False)
- return df
+ top_instr = sorted(facets.get("instruments", {}).items(), key=lambda kv: -kv[1])[:5]
+ top_org = sorted(facets.get("organisms", {}).items(), key=lambda kv: -kv[1])[:3]
+ print(f"Top instruments for '{disease_kw}':")
+ for k, v in top_instr: print(f" {k:<35} {v}")
+ print(f"Top organisms:")
+ for k, v in top_org: print(f" {k:<35} {v}")
- disease = "breast cancer"
- df = discover_projects(disease)
- print(f"PRIDE projects for '{disease}': {len(df)}")
- print(f"\nInstruments used:")
- instr_counts = df["instruments"].str.split(", ").explode().value_counts()
- print(instr_counts.head(8).to_string())
- df.to_csv(f"{disease.replace(' ', '_')}_pride_projects.csv", index=False)
- print(f"\nSaved {disease.replace(' ', '_')}_pride_projects.csv")
+ # 2) Build a filtered search using one top instrument
+ target_instr = top_instr[0][0]
+ projects = requests.get(f"{PRIDE}/search/projects",
+ params={"keyword": disease_kw,
+ "filter": f"organisms_facet==Homo sapiens (human),instruments_facet=={target_instr}",
+ "pageSize": 50,
+ "sortFields": "submission_date",
+ "sortDirection": "DESC"},
+ timeout=30).json()
+
+ df = pd.DataFrame([{
+ "accession": p["accession"],
+ "title": (p.get("title") or "")[:70],
+ "submission_date": p.get("submissionDate"),
+ "tissues": ", ".join(p.get("organismsPart", []))[:40],
+ "submitter": (p.get("submitters") or [""])[0] if p.get("submitters") else "",
+ } for p in projects])
+ print(f"\nFiltered projects: {len(df)} (target instrument: {target_instr})")
+ print(df.head(10).to_string(index=False))
+ df.to_csv(f"{disease_kw.replace(' ', '_')}_{target_instr.replace(' ', '_')}_projects.csv",
+ index=False)
```
- ### Workflow 2: File Download Manager for a Project
+ ### Workflow 2: File Download Manifest for One Project
- **Goal**: List, filter, and generate download commands for files in a PRIDE project — selecting specific file types and formatting for wget or aria2c batch download.
+ **Goal**: Pull the file list for a project, filter to the categories you actually want (`RAW` + `RESULT`), and emit an `aria2c`-ready URL list for parallel FTP download.
```python
- import requests
- import pandas as pd
+ import requests, pandas as pd
from pathlib import Path
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
-
- def build_download_manifest(accession: str,
- file_types: list = None,
- output_dir: str = ".") -> pd.DataFrame:
- """Build a download manifest for PRIDE project files.
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
+ accession = "PXD004131"
+ keep_categories = {"RAW", "RESULT"}
+ output_dir = Path(f"/data/pride/{accession}")
- Parameters
- ----------
- accession : str
- PRIDE accession.
- file_types : list
- List of file types to include (e.g., ['RAW', 'PEAK', 'RESULT']).
- None = include all.
- output_dir : str
- Local directory for downloaded files.
- """
- params = {"pageSize": 200, "page": 0}
- records, page = [], 0
- while True:
- params["page"] = page
- r = requests.get(
- f"{PRIDE_BASE}/projects/{accession}/files",
- params=params,
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- data = r.json()
- batch = data.get("_embedded", {}).get("files", [])
- for f in batch:
- ftype = f.get("fileCategory", {}).get("value", "OTHER")
- if file_types and ftype not in file_types:
- continue
- url = next(
- (loc["value"] for loc in f.get("publicFileLocations", [])
- if loc.get("name") == "FTP Protocol"),
- f.get("publicFileLocations", [{}])[0].get("value", "")
- )
- records.append({
- "file_name": f.get("fileName"),
- "file_type": ftype,
- "size_mb": round(f.get("fileSize", 0) / 1e6, 1),
- "url": url,
- "local_path": str(Path(output_dir) / f.get("fileName", "unknown")),
- })
- total_pages = data.get("page", {}).get("totalPages", 1)
- page += 1
- if page >= total_pages or not batch:
- break
+ files = requests.get(f"{PRIDE}/projects/{accession}/files/all", timeout=120).json()
- df = pd.DataFrame(records)
- return df
+ manifest = []
+ for f in files:
+ cat = (f.get("fileCategory") or {}).get("value")
+ if cat not in keep_categories:
+ continue
+ ftp = next((loc["value"] for loc in f.get("publicFileLocations", [])
+ if loc.get("name") == "FTP Protocol"), None)
+ if not ftp:
+ continue
+ manifest.append({
+ "file_name": f["fileName"],
+ "category": cat,
+ "size_mb": round((f.get("fileSizeBytes") or 0) / 1e6, 2),
+ "ftp": ftp,
+ })
- manifest = build_download_manifest(
- "PXD004131",
- file_types=["RAW", "RESULT"],
- output_dir="/data/pride/PXD004131"
- )
- print(f"Files to download: {len(manifest)}")
- print(f"Total size: {manifest['size_mb'].sum():.0f} MB")
- print(manifest.groupby("file_type")[["file_name", "size_mb"]].head(3).to_string())
+ mdf = pd.DataFrame(manifest).sort_values(["category", "file_name"])
+ print(f"{accession}: keeping {len(mdf)}/{len(files)} files "
+ f"({mdf['size_mb'].sum():.0f} MB total)")
+ print(mdf.groupby("category")[["size_mb"]].sum().round(0))
- # Export wget batch file
- wget_lines = [f"wget -P /data/pride/PXD004131 '{row.url}'"
- for _, row in manifest.iterrows() if row.url]
- with open(f"download_{manifest['file_type'].iloc[0] if len(manifest) else 'files'}.sh", "w") as fh:
- fh.write("\n".join(wget_lines))
- print(f"\nExported wget script with {len(wget_lines)} download commands")
+ # aria2c -i pride_dl.list -d /data/pride/PXD004131 -x 8 -j 4
+ with open("pride_dl.list", "w") as fh:
+ fh.write("\n".join(mdf["ftp"]))
+ print(f"\nWrote pride_dl.list with {len(mdf)} URLs (use aria2c -i)")
```
- ### Workflow 3: Protein Evidence Summary Across Projects
+ ### Workflow 3: Protein Cross-Project Occurrence
- **Goal**: For a list of proteins (e.g., from a differential expression analysis), retrieve proteomics evidence from PRIDE and summarize detection frequency.
+ **Goal**: For a candidate protein panel (e.g., from a differential-expression analysis), look up how many PRIDE projects mention each one and shortlist the most-evidenced proteins. Note: this is a project-count signal only — there are no PSM/peptide counts at the API surface in v3, so a high project count is breadth, not depth.
```python
- import requests, time
- import pandas as pd
- import matplotlib.pyplot as plt
-
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ import requests, time, pandas as pd, matplotlib.pyplot as plt
- def get_protein_psm_counts(uniprot_acc: str, page_size: int = 100) -> dict:
- """Return total PSMs and number of projects for a protein."""
- r = requests.get(
- f"{PRIDE_BASE}/proteins",
- params={"proteinAccession": uniprot_acc, "pageSize": page_size},
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- data = r.json()
- records = data.get("_embedded", {}).get("proteinevidences", [])
- total_psms = sum(rec.get("numberPSMs", 0) for rec in records)
- n_projects = len(set(rec.get("projectAccession") for rec in records if rec.get("projectAccession")))
- return {"uniprot": uniprot_acc, "total_psms": total_psms, "n_projects": n_projects}
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- # Candidate protein panel from a differential expression analysis
- proteins_of_interest = ["P04637", "P38398", "P31749", "P40763", "O15530"]
- # TP53, BRCA1, AKT1, STAT3, PDPK1
+ candidates = {
+ "P04637": "TP53", "P38398": "BRCA1", "P31749": "AKT1",
+ "P40763": "STAT3", "O15530": "PDPK1", "P10275": "AR",
+ }
- results = []
- for acc in proteins_of_interest:
- try:
- r = get_protein_psm_counts(acc)
- results.append(r)
- print(f" {acc}: {r['total_psms']:,} PSMs across {r['n_projects']} projects")
- except Exception as e:
- print(f" {acc} failed: {e}")
- time.sleep(1.2)
+ rows = []
+ for acc, sym in candidates.items():
+ r = requests.get(f"{PRIDE}/proteins/{acc}", timeout=30)
+ projs = r.json().get("projects", []) if r.status_code == 200 else []
+ rows.append({"uniprot": acc, "symbol": sym, "n_projects": len(projs)})
+ time.sleep(0.3)
- df = pd.DataFrame(results).sort_values("total_psms", ascending=False)
+ df = pd.DataFrame(rows).sort_values("n_projects", ascending=False)
+ print(df.to_string(index=False))
- # Bar chart
- fig, ax = plt.subplots(figsize=(8, 4))
- bars = ax.bar(df["uniprot"], df["total_psms"], color="#3182BD")
- ax.bar_label(bars, fmt="%d", fontsize=8, padding=2)
- ax.set_xlabel("UniProt Accession")
- ax.set_ylabel("Total PSMs in PRIDE")
- ax.set_title("Proteomics Evidence Depth in PRIDE Archive")
+ fig, ax = plt.subplots(figsize=(8, 3.5))
+ bars = ax.bar(df["symbol"], df["n_projects"], color="#3182BD")
+ ax.bar_label(bars, fmt="%d", fontsize=9, padding=2)
+ ax.set_ylabel("# PRIDE projects mentioning the protein")
+ ax.set_title("PRIDE project-level occurrence — candidate panel")
plt.tight_layout()
- plt.savefig("pride_protein_evidence.png", dpi=150, bbox_inches="tight")
- print(f"\nSaved pride_protein_evidence.png")
- df.to_csv("pride_protein_evidence_summary.csv", index=False)
+ plt.savefig("pride_protein_occurrence.png", dpi=150, bbox_inches="tight")
+ print("Saved pride_protein_occurrence.png")
```
## Key Parameters
| Parameter | Endpoint | Default | Range / Options | Effect |
- |-----------|----------|---------|-----------------|--------|
- | `keyword` | `GET /projects` | — | free-text string | Full-text search across title and description |
- | `organisms` | `GET /projects` | — | organism name string (e.g., `"Homo sapiens"`) | Filter projects by organism |
- | `tissues` | `GET /projects` | — | tissue name string (e.g., `"liver"`) | Filter projects by tissue |
- | `diseases` | `GET /projects` | — | disease keyword | Filter projects by disease annotation |
- | `instruments` | `GET /projects` | — | instrument name (e.g., `"Orbitrap"`) | Filter projects by MS instrument |
- | `fileType` | `GET /projects/{acc}/files` | all | `RAW`, `PEAK`, `RESULT`, `FASTA`, `OTHER` | Filter files by category |
- | `pageSize` | all list endpoints | `20` | `1`–`100` | Results per page |
- | `page` | all list endpoints | `0` | non-negative integer | 0-indexed page for pagination |
- | `projectAccessions` | `GET /peptides`, `/psms`, `/proteins` | — | `PXD######` string | Restrict identifications to a specific project |
- | `proteinAccession` | `GET /peptides`, `/psms`, `/proteins` | — | UniProt accession | Filter by protein |
- | `peptideSequence` | `GET /peptides`, `/psms` | — | amino acid sequence string | Filter by peptide sequence |
+ |---|---|---|---|---|
+ | `keyword` | `/search/projects`, `/facet/projects`, `/search/autocomplete` | — | free-text string | Full-text search across title, description, tags |
+ | `filter` | `/search/projects` | — | `field_facet==value, field_facet==value` | Server-side filter using facet field names |
+ | `pageSize` | `/search/projects`, `/projects`, `/projects/{acc}/files`, `/projects/{acc}/similarProjects` | 100 | positive integer | Results per page |
+ | `page` | same as above | 0 | 0-indexed integer | Page number (no metadata returned — walk pages until empty) |
+ | `sortFields` | `/search/projects` | `submission_date` | comma-separated field names | Sort key(s) |
+ | `sortDirection` | `/search/projects` | `DESC` | `ASC` or `DESC` | Sort order |
+ | `facetPageSize` | `/facet/projects` | 20 | positive integer | Values returned per facet group |
+ | `dateGap` | `/search/projects`, `/facet/projects` | — | e.g. `+1MONTH`, `+1YEAR` | Date-range aggregation granularity |
+ | (path) `accession` | `/projects/{acc}`, `/projects/{acc}/files`, `/projects/{acc}/similarProjects`, `/proteins/{acc}` | required | PXD###### or UniProt acc | Identifies the resource |
## Best Practices
- 1. **Start with `RESULT` files for fastest reanalysis**: mzIdentML or MaxQuant output files contain already-processed identifications and are much smaller than RAW files (MB vs GB). Use these for cross-study comparison without re-searching spectra.
+ 1. **Use `/search/projects` for searching, not `/projects`.** Plain `/projects` is a paginated listing endpoint and silently ignores keyword / organism / disease filters. Filtering only works through `/search/projects` with the `filter=field_facet==value` syntax.
- 2. **Add `time.sleep(1.2)` between project or file queries**: The PRIDE API enforces ~50 requests/minute. Batch scripts without delays will receive `HTTP 429` errors. For large surveys (100+ projects), implement exponential backoff.
+ 2. **Discover filter values via `/facet/projects` before filtering.** Facet field values must match exactly (e.g., `organisms_facet==Homo sapiens (human)`, parentheses and all). The facet endpoint tells you which values exist and how many projects each has — saves a lot of trial-and-error.
- 3. **Prefer FTP URLs for large file downloads**: The API returns both HTTPS and FTP public file locations. FTP downloads are more reliable for large RAW files (>1 GB) and can be parallelized with `aria2c -x 8`.
+ 3. **Don't try to query peptide- or PSM-level data over the API.** Those endpoints were removed in v3. Download the project's RESULT files and parse them locally with `pyteomics`, `pyOpenMS`, or a search-engine reader (MaxQuant, ProteomeDiscoverer, etc.).
- 4. **Filter by `fileType=RESULT` before downloading**: Projects often contain dozens of auxiliary files per sample. Fetching the file manifest and filtering by type avoids accidentally queuing gigabytes of RAW data when you only need identifications.
+ 4. **Prefer FTP URLs for bulk file downloads.** Each file record carries both `FTP Protocol` and `Aspera Protocol` URLs. FTP is more universally supported; pair it with `aria2c -x 8 -j 4` for parallel chunks. Use Aspera only if you have an Aspera client and need >100 Mbit transfer speeds.
- 5. **Cross-reference PTMs with UniMod**: PTM annotations in PRIDE use PSI-MOD or UniMod accessions. When parsing modification fields from peptide or PSM responses, look up accessions at `https://www.unimod.org/` to translate to modification names and masses.
+ 5. **Watch the field name `fileSizeBytes`.** The current v3 field is `fileSizeBytes`, not `fileSize` (old v2 docs may say `fileSize`). Sizes are in bytes — divide by `1e6` for MB, `1e9` for GB.
+ 6. **Filter file downloads by `fileCategory.value`.** A project can have hundreds of files spanning RAW (GB-scale) and OTHER (KB-scale). Always filter to the categories you actually need before queueing downloads — otherwise you'll easily download tens of gigabytes of vendor RAW files when you only wanted the identification tables.
+
+ 7. **Pagination has no metadata — walk until empty.** Unlike old PRIDE v2, the v3 API doesn't return `totalElements`/`totalPages`. Iterate `page=0, 1, 2, ...` and stop when a page returns an empty array, or when its length is less than `pageSize`.
+
## Common Recipes
### Recipe: Quick Project File Summary
- When to use: Given a PRIDE accession, get a rapid count of file types and total dataset size before committing to a download.
-
```python
import requests
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- def project_file_summary(accession: str) -> None:
- """Print file type breakdown and total size for a PRIDE project."""
- r = requests.get(
- f"{PRIDE_BASE}/projects/{accession}/files",
- params={"pageSize": 200},
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- files = r.json().get("_embedded", {}).get("files", [])
- type_sizes: dict = {}
+ def project_file_summary(accession):
+ files = requests.get(f"{PRIDE}/projects/{accession}/files/all", timeout=60).json()
+ by_cat = {}
for f in files:
- ftype = f.get("fileCategory", {}).get("value", "OTHER")
- size_mb = f.get("fileSize", 0) / 1e6
- type_sizes[ftype] = type_sizes.get(ftype, 0) + size_mb
+ cat = (f.get("fileCategory") or {}).get("value", "OTHER")
+ by_cat.setdefault(cat, [0, 0])
+ by_cat[cat][0] += 1
+ by_cat[cat][1] += (f.get("fileSizeBytes") or 0) / 1e6
print(f"\n{accession} file summary:")
- for ftype, total_mb in sorted(type_sizes.items()):
- print(f" {ftype:<10} {total_mb:>8.0f} MB")
- print(f" {'TOTAL':<10} {sum(type_sizes.values()):>8.0f} MB")
+ for cat, (n, mb) in sorted(by_cat.items()):
+ print(f" {cat:<8} {n:>4} file(s) {mb:>10.1f} MB")
+ total_mb = sum(mb for _, mb in by_cat.values())
+ total_n = sum(n for n, _ in by_cat.values())
+ print(f" {'TOTAL':<8} {total_n:>4} file(s) {total_mb:>10.1f} MB")
- project_file_summary("PXD004131")
+ project_file_summary("PXD000001")
```
- ### Recipe: Check If a Protein Has PRIDE Evidence
-
- When to use: Quickly validate whether a protein of interest has any experimental proteomics evidence in PRIDE before designing targeted experiments.
+ ### Recipe: Check If a Protein Has Any PRIDE Evidence
```python
import requests
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- def has_pride_evidence(uniprot_acc: str) -> tuple:
- """Return (has_evidence, n_projects, total_psms) for a UniProt accession."""
- r = requests.get(
- f"{PRIDE_BASE}/proteins",
- params={"proteinAccession": uniprot_acc, "pageSize": 10},
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- records = r.json().get("_embedded", {}).get("proteinevidences", [])
- if not records:
- return False, 0, 0
- n_projects = len(set(rec.get("projectAccession") for rec in records))
- total_psms = sum(rec.get("numberPSMs", 0) for rec in records)
- return True, n_projects, total_psms
+ def pride_evidence(uniprot_acc):
+ """Return (has_evidence, n_projects). PRIDE v3 only exposes project list, no PSM counts."""
+ r = requests.get(f"{PRIDE}/proteins/{uniprot_acc}", timeout=30)
+ if r.status_code != 200:
+ return False, 0
+ projs = r.json().get("projects", [])
+ return bool(projs), len(projs)
- for acc in ["P04637", "Q99999"]: # TP53, hypothetical unknown
- has_ev, n_proj, n_psms = has_pride_evidence(acc)
- print(f"{acc}: evidence={has_ev}, projects={n_proj}, PSMs={n_psms}")
- # P04637: evidence=True, projects=47, PSMs=12834
- # Q99999: evidence=False, projects=0, PSMs=0
+ for acc in ["P04637", "Q99999"]:
+ has, n = pride_evidence(acc)
+ print(f"{acc}: evidence={has} projects={n}")
```
- ### Recipe: Find Projects with Specific PTM Data
+ ### Recipe: Recent Submissions for a Keyword (sorted by date)
- When to use: Locate PRIDE datasets that include a specific post-translational modification (e.g., phosphorylation, ubiquitination) for a given organism.
+ ```python
+ import requests, pandas as pd
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
+
+ r = requests.get(f"{PRIDE}/search/projects",
+ params={"keyword": "single-cell proteomics",
+ "sortFields": "submission_date",
+ "sortDirection": "DESC",
+ "pageSize": 15},
+ timeout=30)
+ recent = r.json()
+ df = pd.DataFrame([{
+ "submission_date": p.get("submissionDate"),
+ "accession": p["accession"],
+ "title": (p.get("title") or "")[:80],
+ } for p in recent]).sort_values("submission_date", ascending=False)
+ print(df.to_string(index=False))
+ ```
+
+ ### Recipe: Suggest-as-You-Type via Autocomplete
+
```python
import requests
- import pandas as pd
- PRIDE_BASE = "https://www.ebi.ac.uk/pride/ws/archive/v2"
-
- def find_ptm_projects(ptm_keyword: str, organism: str = "Homo sapiens",
- page_size: int = 25) -> pd.DataFrame:
- """Search PRIDE for projects annotated with a specific PTM."""
- r = requests.get(
- f"{PRIDE_BASE}/projects",
- params={
- "keyword": ptm_keyword,
- "organisms": organism,
- "pageSize": page_size
- },
- headers={"Accept": "application/json"},
- timeout=30
- )
- r.raise_for_status()
- projects = r.json().get("_embedded", {}).get("compactprojects", [])
- rows = []
- for p in projects:
- ptm_names = [m.get("name", "") for m in p.get("ptms", [])]
- if any(ptm_keyword.lower() in name.lower() for name in ptm_names) or True:
- rows.append({
- "accession": p.get("accession"),
- "title": p.get("title", "")[:80],
- "ptms": ", ".join(ptm_names[:4]),
- "n_files": p.get("filesCount", 0),
- })
- return pd.DataFrame(rows)
+ PRIDE = "https://www.ebi.ac.uk/pride/ws/archive/v3"
- phospho_projects = find_ptm_projects("phospho")
- print(f"Phosphoproteomics projects: {len(phospho_projects)}")
- print(phospho_projects[["accession", "title", "ptms"]].head(6).to_string(index=False))
+ for prefix in ["alzheimer", "single cell", "brca"]:
+ s = requests.get(f"{PRIDE}/search/autocomplete",
+ params={"keyword": prefix}, timeout=30).json()
+ print(f"\n'{prefix}' → {len(s)} suggestions:")
+ for sug in s[:3]:
+ print(f" · {sug[:80]}")
```
## Troubleshooting
| Problem | Cause | Solution |
- |---------|-------|----------|
- | `HTTP 404` on project lookup | Accession not found or not public | Verify accession format (`PXD######`); some datasets are under embargo until publication |
- | `HTTP 429 Too Many Requests` | Exceeded ~50 req/min rate limit | Add `time.sleep(1.2)` between requests; implement exponential backoff for bursts |
- | Empty `_embedded` object | No results match the query | Broaden search terms; check organism spelling (exact match required, e.g., `"Homo sapiens"`) |
- | Empty peptide/PSM results | Project has no identification data loaded | Newer projects may not yet have identifications indexed; use `RESULT` file download instead |
- | Download URL is empty string | File not yet available on FTP | Check `publicFileLocations` list for alternative URLs; some files are HTTPS-only |
- | Very large file manifest | Project has hundreds of files | Use `fileType` filter to restrict to relevant types; build a manifest before downloading |
- | `ConnectionError` or `ReadTimeout` | Transient EBI infrastructure issue | Retry after 60 seconds; EBI services occasionally have brief maintenance windows |
+ |---|---|---|
+ | Parsing returns empty list even when `r.json()` has data | Code is doing `data["_embedded"]["compactprojects"]` — old HAL+JSON wrapper that v3 no longer returns | Parse the response directly as a list: `projects = r.json()` |
+ | `HTTP 404` on `/peptides`, `/psms`, or `/proteins?proteinAccession=X` | These endpoints were removed in v3 | For peptide/PSM data, download the project's RESULT files and parse locally. For protein lookup, use `/proteins/{accession}` (path param) |
+ | `/projects?keyword=cancer` returns the same 100 results as `/projects` with no keyword | The `/projects` endpoint only accepts `pageSize` / `page` — keyword and other filters are silently ignored | Use `/search/projects?keyword=...&filter=...` instead |
+ | `/projects/{acc}/files` shows file size 0 | Reading `fileSize` instead of `fileSizeBytes` | The v3 field is `fileSizeBytes` (bytes); compute MB via `fileSizeBytes / 1e6` |
+ | Filter has no effect | Facet value doesn't exactly match a real value | Call `/facet/projects?keyword=...` first to enumerate valid values (`Homo sapiens (human)`, not `Homo sapiens`) |
+ | `pageSize` beyond the actual result set returns an empty array | Normal pagination behavior | Stop iterating when the returned array length is < `pageSize`, or when it is empty |
+ | `findAllOrganismsCount` returns HTTP 406 Not Acceptable | The endpoint requires a non-JSON Accept header | Skip this endpoint — facet counts via `/facet/projects` cover the same need |
+ | `HTTP 429` or `ConnectionError` on bursts | Shared EBI infrastructure | Add `time.sleep(0.3)` in loops; retry on 5xx with exponential backoff |
## Related Skills
- - `interpro-database` — InterPro protein domain architecture and family classification; cross-reference proteins found in PRIDE with their structural annotations
- - `uniprot-protein-database` — UniProt protein sequences, Swiss-Prot annotations, PTM sites, and disease associations; use after retrieving protein accessions from PRIDE
- - `pdb-database` — PDB 3D structures for proteins with proteomics evidence in PRIDE
+ - `uniprot-protein-database` — UniProt sequences, Swiss-Prot annotations, ID mapping; pair with PRIDE protein lookups to enrich each UniProt accession with sequence and functional information
+ - `interpro-database` — Protein domain architecture (Pfam, SMART, PANTHER) for proteins reported in PRIDE
+ - `pdb-database` — Resolved 3D structures for proteins with PRIDE evidence
+ - `pyteomics` (off-skill Python library) — Parse mzIdentML / mzML / mzTab files downloaded from `/projects/{accession}/files`; the path for spectrum- and PSM-level analysis now that the REST API no longer exposes those
## References
- - [PRIDE Archive REST API v2](https://www.ebi.ac.uk/pride/ws/archive/v2/) — Interactive Swagger API documentation and endpoint reference
- - [Perez-Riverol et al., Nucleic Acids Research 2022](https://doi.org/10.1093/nar/gkab1038) — PRIDE 2022 update describing the repository and API
- - [PRIDE web portal](https://www.ebi.ac.uk/pride/) — Interactive dataset browser and submission guide
- - [ProteomXchange Consortium](http://www.proteomexchange.org/) — Standard accession system shared across PRIDE, MassIVE, jPOST, and iProX
+ - [PRIDE Archive v3 REST API root](https://www.ebi.ac.uk/pride/ws/archive/v3/) — base URL
+ - [PRIDE v3 OpenAPI/Swagger spec](https://www.ebi.ac.uk/pride/ws/archive/v3/v3/api-docs) — authoritative endpoint inventory
+ - [PRIDE Archive web portal](https://www.ebi.ac.uk/pride/) — interactive dataset browser
+ - [Perez-Riverol et al., *Nucleic Acids Research* 2022](https://doi.org/10.1093/nar/gkab1038) — PRIDE 2022 update describing repository and architecture
+ - [ProteomeXchange Consortium](http://www.proteomexchange.org/) — standard accession system shared across PRIDE, MassIVE, jPOST, iProX
+ - [SDRF-Proteomics specification](https://github.com/bigbio/proteomics-sample-metadata) — format used by `/files/sdrf/{projectAccession}`