etetoolkit · diff
git:20260611.1b8fae3 to v2.0
227 added, 515 removed. Audit A to A.
---
name: etetoolkit
- description: Phylogenetic tree toolkit (ETE). Tree manipulation (Newick/NHX), evolutionary event detection, orthology/paralogy, NCBI taxonomy, visualization (PDF/SVG), for phylogenomics.
- license: GPL-3.0 license
- metadata: {"version": "1.0", "skill-author": "K-Dense Inc."}
+ description: Analyze, manipulate, compare, annotate, and visualize phylogenetic or other hierarchical trees with ETE 4. Use for Newick/Nexus tree I/O, topology edits and pattern matching, Robinson-Foulds comparisons, gene-tree evolutionary events and reconciliation, NCBI/GTDB taxonomy, SmartView exploration, and publication rendering. Do not use it to infer trees from raw sequences; align sequences and infer a tree first.
+ license: GPL-3.0-or-later
+ allowed-tools:
+ - Read
+ - Write
+ - Edit
+ - Bash
+ - Python
+ compatibility: Bundled scripts require Python 3.10+ and ete4 4.4.0 (upstream ete4 supports Python >=3.7). Taxonomy setup and SmartView exploration need network access; static SmartView PNG rendering needs ete4[render-sm], and Qt PDF/SVG rendering needs ete4[treeview].
+ metadata:
+ version: "2.0"
+ skill-author: K-Dense Inc.
---
- # ETE Toolkit Skill
-
- ## Overview
-
- ETE (Environment for Tree Exploration) is a toolkit for phylogenetic and hierarchical tree analysis. Manipulate trees, analyze evolutionary events, visualize results, and integrate with biological databases for phylogenomic research and clustering analysis.
+ # ETE Toolkit 4
- ## Core Capabilities
+ ## Scope
- ### 1. Tree Manipulation and Analysis
+ Use ETE 4 to work with an existing tree:
- Load, manipulate, and analyze hierarchical tree structures with support for:
+ - Read Newick/Nexus, then inspect, annotate, transform, root, prune, and write
+ Newick trees
+ - Compare topologies and calculate phylogenetic distances
+ - Find repeated subtree topologies with `TreePattern`
+ - Analyze gene trees with `PhyloTree`
+ - Query local NCBI or GTDB taxonomy databases
+ - Explore large trees interactively with SmartView
+ - Render PNG with SmartView or PNG/PDF/SVG with the optional Qt treeview
- - **Tree I/O**: Read and write Newick, NHX, PhyloXML, and NeXML formats
- - **Tree traversal**: Navigate trees using preorder, postorder, or levelorder strategies
- - **Topology modification**: Prune, root, collapse nodes, resolve polytomies
- - **Distance calculations**: Compute branch lengths and topological distances between nodes
- - **Tree comparison**: Calculate Robinson-Foulds distances and identify topological differences
+ ETE does not replace sequence alignment or phylogenetic inference software. For
+ raw sequences, first use MAFFT or another aligner and IQ-TREE 2, FastTree, or
+ another inference tool; then load the resulting tree into ETE.
- **Common patterns:**
+ ## Current Target
- ```python
- from ete3 import Tree
+ This skill targets **ETE 4.4.0**, released September 3, 2025 and verified as the
+ current PyPI release on July 23, 2026.
- # Load tree from file
- tree = Tree("tree.nw", format=1)
+ Use `https://etetoolkit.github.io/ete/` for ETE 4 documentation. The
+ `etetoolkit.org/docs/latest` pages are legacy ETE 3 documentation despite the
+ URL name.
- # Basic statistics
- print(f"Leaves: {len(tree)}")
- print(f"Total nodes: {len(list(tree.traverse()))}")
+ Do not silently translate these examples back to ETE 3:
- # Prune to taxa of interest
- taxa_to_keep = ["species1", "species2", "species3"]
- tree.prune(taxa_to_keep, preserve_branch_length=True)
+ - Package and import: `ete4`, not `ete3`
+ - File input: pass an open file object; use strings for Newick text and do not
+ rely on path-string heuristics retained in ETE 4.4.0
+ - Newick selection: `parser=`, not `format=`
+ - Node metadata: `props`, `add_prop()`, and `add_props()`
+ - Iteration: `leaves()`, `descendants()`, and related methods return iterators
+ - Predicates: `node.is_leaf` and `node.is_root` are properties, not methods
+ - Node lookup: `tree["name"]`, not `tree & "name"`
- # Midpoint root
- midpoint = tree.get_midpoint_outgroup()
- tree.set_outgroup(midpoint)
+ For porting older code, load
+ [`references/migration-ete3-to-ete4.md`](references/migration-ete3-to-ete4.md).
- # Save modified tree
- tree.write(outfile="rooted_tree.nw")
- ```
+ ## Installation
- Use `scripts/tree_operations.py` for command-line tree manipulation:
+ Install the pinned base package:
```bash
- # Display tree statistics
- python scripts/tree_operations.py stats tree.nw
-
- # Convert format
- python scripts/tree_operations.py convert tree.nw output.nw --in-format 0 --out-format 1
-
- # Reroot tree
- python scripts/tree_operations.py reroot tree.nw rooted.nw --midpoint
-
- # Prune to specific taxa
- python scripts/tree_operations.py prune tree.nw pruned.nw --keep-taxa "sp1,sp2,sp3"
-
- # Show ASCII visualization
- python scripts/tree_operations.py ascii tree.nw
- ```
-
- ### 2. Phylogenetic Analysis
-
- Analyze gene trees with evolutionary event detection:
-
- - **Sequence alignment integration**: Link trees to multiple sequence alignments (FASTA, Phylip)
- - **Species naming**: Automatic or custom species extraction from gene names
- - **Evolutionary events**: Detect duplication and speciation events using Species Overlap or tree reconciliation
- - **Orthology detection**: Identify orthologs and paralogs based on evolutionary events
- - **Gene family analysis**: Split trees by duplications, collapse lineage-specific expansions
-
- **Workflow for gene tree analysis:**
-
- ```python
- from ete3 import PhyloTree
-
- # Load gene tree with alignment
- tree = PhyloTree("gene_tree.nw", alignment="alignment.fasta")
-
- # Set species naming function
- def get_species(gene_name):
- return gene_name.split("_")[0]
-
- tree.set_species_naming_function(get_species)
-
- # Detect evolutionary events
- events = tree.get_descendant_evol_events()
-
- # Analyze events
- for node in tree.traverse():
- if hasattr(node, "evoltype"):
- if node.evoltype == "D":
- print(f"Duplication at {node.name}")
- elif node.evoltype == "S":
- print(f"Speciation at {node.name}")
-
- # Extract ortholog groups
- ortho_groups = tree.get_speciation_trees()
- for i, ortho_tree in enumerate(ortho_groups):
- ortho_tree.write(outfile=f"ortholog_group_{i}.nw")
- ```
-
- **Finding orthologs and paralogs:**
-
- ```python
- # Find orthologs to query gene
- query = tree & "species1_gene1"
-
- orthologs = []
- paralogs = []
-
- for event in events:
- if query in event.in_seqs:
- if event.etype == "S":
- orthologs.extend([s for s in event.out_seqs if s != query])
- elif event.etype == "D":
- paralogs.extend([s for s in event.out_seqs if s != query])
- ```
-
- ### 3. NCBI Taxonomy Integration
-
- Integrate taxonomic information from NCBI Taxonomy database:
-
- - **Database access**: Automatic download and local caching of NCBI taxonomy (~300MB)
- - **Taxid/name translation**: Convert between taxonomic IDs and scientific names
- - **Lineage retrieval**: Get complete evolutionary lineages
- - **Taxonomy trees**: Build species trees connecting specified taxa
- - **Tree annotation**: Automatically annotate trees with taxonomic information
-
- **Building taxonomy-based trees:**
-
- ```python
- from ete3 import NCBITaxa
-
- ncbi = NCBITaxa()
-
- # Build tree from species names
- species = ["Homo sapiens", "Pan troglodytes", "Mus musculus"]
- name2taxid = ncbi.get_name_translator(species)
- taxids = [name2taxid[sp][0] for sp in species]
-
- # Get minimal tree connecting taxa
- tree = ncbi.get_topology(taxids)
-
- # Annotate nodes with taxonomy info
- for node in tree.traverse():
- if hasattr(node, "sci_name"):
- print(f"{node.sci_name} - Rank: {node.rank} - TaxID: {node.taxid}")
- ```
-
- **Annotating existing trees:**
-
- ```python
- # Get taxonomy info for tree leaves
- for leaf in tree:
- species = extract_species_from_name(leaf.name)
- taxid = ncbi.get_name_translator([species])[species][0]
-
- # Get lineage
- lineage = ncbi.get_lineage(taxid)
- ranks = ncbi.get_rank(lineage)
- names = ncbi.get_taxid_translator(lineage)
-
- # Add to node
- leaf.add_feature("taxid", taxid)
- leaf.add_feature("lineage", [names[t] for t in lineage])
+ uv pip install "ete4==4.4.0"
```
- ### 4. Tree Visualization
-
- Create publication-quality tree visualizations:
-
- - **Output formats**: PNG (raster), PDF, and SVG (vector) for publications
- - **Layout modes**: Rectangular and circular tree layouts
- - **Interactive GUI**: Explore trees interactively with zoom, pan, and search
- - **Custom styling**: NodeStyle for node appearance (colors, shapes, sizes)
- - **Faces**: Add graphical elements (text, images, charts, heatmaps) to nodes
- - **Layout functions**: Dynamic styling based on node properties
-
- **Basic visualization workflow:**
-
- ```python
- from ete3 import Tree, TreeStyle, NodeStyle
-
- tree = Tree("tree.nw")
-
- # Configure tree style
- ts = TreeStyle()
- ts.show_leaf_name = True
- ts.show_branch_support = True
- ts.scale = 50 # pixels per branch length unit
-
- # Style nodes
- for node in tree.traverse():
- nstyle = NodeStyle()
-
- if node.is_leaf():
- nstyle["fgcolor"] = "blue"
- nstyle["size"] = 8
- else:
- # Color by support
- if node.support > 0.9:
- nstyle["fgcolor"] = "darkgreen"
- else:
- nstyle["fgcolor"] = "red"
- nstyle["size"] = 5
+ Add only the visualization extra required by the workflow:
- node.set_style(nstyle)
+ ```bash
+ # SmartView static PNG screenshots
+ uv pip install "ete4[render-sm]==4.4.0"
- # Render to file
- tree.render("tree.pdf", tree_style=ts)
- tree.render("tree.png", w=800, h=600, units="px", dpi=300)
+ # Legacy Qt renderer for PNG, PDF, and SVG
+ uv pip install "ete4[treeview]==4.4.0"
```
- Use `scripts/quick_visualize.py` for rapid visualization:
+ Confirm the active environment:
```bash
- # Basic visualization
- python scripts/quick_visualize.py tree.nw output.pdf
-
- # Circular layout with custom styling
- python scripts/quick_visualize.py tree.nw output.pdf --mode c --color-by-support
-
- # High-resolution PNG
- python scripts/quick_visualize.py tree.nw output.png --width 1200 --height 800 --units px --dpi 300
-
- # Custom title and styling
- python scripts/quick_visualize.py tree.nw output.pdf --title "Species Phylogeny" --show-support
+ uv run --with "ete4==4.4.0" python -c "import ete4; print(ete4.__version__)"
```
- **Advanced visualization with faces:**
+ No credentials are required. NCBI and GTDB workflows download public taxonomy
+ data and can consume substantial disk space; see
+ [`references/taxonomy.md`](references/taxonomy.md) before the first update.
+ ## Quick Start
+
```python
- from ete3 import Tree, TreeStyle, TextFace, CircleFace
+ from pathlib import Path
- tree = Tree("tree.nw")
+ from ete4 import Tree
- # Add features to nodes
- for leaf in tree:
- leaf.add_feature("habitat", "marine" if "fish" in leaf.name else "land")
+ # Use an open file object for files; reserve strings for Newick text.
+ with Path("tree.nw").open(encoding="utf-8") as handle:
+ tree = Tree(handle, parser=1) # parser 1: internal node names
- # Layout function
- def layout(node):
- if node.is_leaf():
- # Add colored circle
- color = "blue" if node.habitat == "marine" else "green"
- circle = CircleFace(radius=5, color=color)
- node.add_face(circle, column=0, position="aligned")
+ print(tree.to_str(props=["name", "dist"], compact=True))
+ print("Leaves:", list(tree.leaf_names()))
- # Add label
- label = TextFace(node.name, fsize=10)
- node.add_face(label, column=1, position="aligned")
+ # Search and annotate.
+ focal = tree["species1"]
+ focal.add_props(host="human", status="focal")
- ts = TreeStyle()
- ts.layout_fn = layout
- ts.show_leaf_name = False
+ # Keep selected tips while preserving pairwise branch-length distances.
+ tree.prune(
+ ["species1", "species2", "species3"],
+ preserve_branch_length=True,
+ )
- tree.render("annotated_tree.pdf", tree_style=ts)
+ # Root and serialize explicitly.
+ tree.set_midpoint_outgroup()
+ tree.write(
+ outfile="processed.nw",
+ parser=1,
+ props=["host", "status"],
+ )
```
- ### 5. Clustering Analysis
-
- Analyze hierarchical clustering results with data integration:
+ Choose the parser deliberately. A parser mismatch is the most common cause of
+ `NewickError`, lost internal labels, or support values being read as names.
+ See [`references/api_reference.md`](references/api_reference.md).
- - **ClusterTree**: Specialized class for clustering dendrograms
- - **Data matrix linking**: Connect tree leaves to numerical profiles
- - **Cluster metrics**: Silhouette coefficient, Dunn index, inter/intra-cluster distances
- - **Validation**: Test cluster quality with different distance metrics
- - **Heatmap visualization**: Display data matrices alongside trees
+ ## Core Workflows
- **Clustering workflow:**
+ ### Inspect and transform a tree
```python
- from ete3 import ClusterTree
+ from ete4 import Tree
- # Load tree with data matrix
- matrix = """#Names\tSample1\tSample2\tSample3
- Gene1\t1.5\t2.3\t0.8
- Gene2\t0.9\t1.1\t1.8
- Gene3\t2.1\t2.5\t0.5"""
+ tree = Tree("((A:1,B:1)CladeAB:0.4,C:2)Root;", parser=1)
- tree = ClusterTree("((Gene1,Gene2),Gene3);", text_array=matrix)
+ for node in tree.traverse("preorder"):
+ label = node.name if node.name is not None else node.id
+ print(label, node.level, node.is_leaf, node.dist)
- # Evaluate cluster quality
- for node in tree.traverse():
- if not node.is_leaf():
- silhouette = node.get_silhouette()
- dunn = node.get_dunn()
+ tree["A"].add_prop("group", "case")
+ tree["B"].add_prop("group", "control")
- print(f"Cluster: {node.name}")
- print(f" Silhouette: {silhouette:.3f}")
- print(f" Dunn index: {dunn:.3f}")
+ mrca = tree.common_ancestor("A", "B")
+ print(mrca.name)
- # Visualize with heatmap
- tree.show("heatmap")
+ tree.write(
+ outfile="annotated.nhx",
+ parser=1,
+ props=["group"],
+ format_root_node=True,
+ )
```
- ### 6. Tree Comparison
-
- Quantify topological differences between trees:
-
- - **Robinson-Foulds distance**: Standard metric for tree comparison
- - **Normalized RF**: Scale-invariant distance (0.0 to 1.0)
- - **Partition analysis**: Identify unique and shared bipartitions
- - **Consensus trees**: Analyze support across multiple trees
- - **Batch comparison**: Compare multiple trees pairwise
-
- **Compare two trees:**
-
- ```python
- from ete3 import Tree
-
- tree1 = Tree("tree1.nw")
- tree2 = Tree("tree2.nw")
-
- # Calculate RF distance
- rf, max_rf, common_leaves, parts_t1, parts_t2 = tree1.robinson_foulds(tree2)
-
- print(f"RF distance: {rf}/{max_rf}")
- print(f"Normalized RF: {rf/max_rf:.3f}")
- print(f"Common leaves: {len(common_leaves)}")
-
- # Find unique partitions
- unique_t1 = parts_t1 - parts_t2
- unique_t2 = parts_t2 - parts_t1
-
- print(f"Unique to tree1: {len(unique_t1)}")
- print(f"Unique to tree2: {len(unique_t2)}")
- ```
+ Node names need not be unique. `tree["A"]` returns the first match; use
+ `list(tree.search_nodes(name="A"))` and validate the count when duplicates are
+ possible.
- **Compare multiple trees:**
+ ### Compare two topologies
```python
- import numpy as np
-
- trees = [Tree(f"tree{i}.nw") for i in range(4)]
-
- # Create distance matrix
- n = len(trees)
- dist_matrix = np.zeros((n, n))
-
- for i in range(n):
- for j in range(i+1, n):
- rf, max_rf, _, _, _ = trees[i].robinson_foulds(trees[j])
- norm_rf = rf / max_rf if max_rf > 0 else 0
- dist_matrix[i, j] = norm_rf
- dist_matrix[j, i] = norm_rf
- ```
-
- ## Installation and Setup
-
- Install ETE toolkit:
-
- ```bash
- # Basic installation
- uv pip install ete3
+ from ete4 import Tree
- # With external dependencies for rendering (optional but recommended)
- # On macOS:
- brew install qt@5
+ tree_a = Tree("((A,B),(C,D));")
+ tree_b = Tree("((A,C),(B,D));")
- # On Ubuntu/Debian:
- sudo apt-get install python3-pyqt5 python3-pyqt5.qtsvg
+ (
+ rf,
+ max_rf,
+ common_leaves,
+ edges_a,
+ edges_b,
+ discarded_a,
+ discarded_b,
+ ) = tree_a.robinson_foulds(tree_b)
- # For full features including GUI
- uv pip install ete3[gui]
+ normalized_rf = rf / max_rf if max_rf else 0.0
+ print(rf, max_rf, normalized_rf, sorted(common_leaves))
```
- **First-time NCBI Taxonomy setup:**
+ RF comparison uses shared leaf labels and requires meaningful, preferably
+ unique names. Decide explicitly whether rooted or unrooted comparison is
+ scientifically appropriate.
- The first time NCBITaxa is instantiated, it automatically downloads the NCBI taxonomy database (~300MB) to `~/.etetoolkit/taxa.sqlite`. This happens only once:
+ ### Detect duplication and speciation events
```python
- from ete3 import NCBITaxa
- ncbi = NCBITaxa() # Downloads database on first run
- ```
+ from ete4 import PhyloTree
- Update taxonomy database:
+ gene_tree = PhyloTree(
+ "((Hsa|g1,Ptr|g1),(Hsa|g2,Mmu|g1));",
+ sp_naming_function=lambda name: name.split("|", 1)[0],
+ )
- ```python
- ncbi.update_taxonomy_database() # Download latest NCBI data
+ for event in gene_tree.get_descendant_evol_events(sos_thr=0.0):
+ relationship = "speciation/orthology" if event.etype == "S" else "duplication/paralogy"
+ print(relationship, sorted(event.in_seqs), sorted(event.out_seqs))
```
- ## Common Use Cases
-
- ### Use Case 1: Phylogenomic Pipeline
+ Species-overlap calls are inferences from the supplied topology and naming
+ function, not independent evidence of orthology. Pass the naming function
+ explicitly, and use a rooted, fully bifurcating gene tree. For strict
+ reconciliation, use a curated species tree and
+ `gene_tree.reconcile(species_tree)`.
- Complete workflow from gene tree to ortholog identification:
+ ### Query taxonomy
```python
- from ete3 import PhyloTree, NCBITaxa
-
- # 1. Load gene tree with alignment
- tree = PhyloTree("gene_tree.nw", alignment="alignment.fasta")
-
- # 2. Configure species naming
- tree.set_species_naming_function(lambda x: x.split("_")[0])
-
- # 3. Detect evolutionary events
- tree.get_descendant_evol_events()
+ from ete4 import NCBITaxa
- # 4. Annotate with taxonomy
ncbi = NCBITaxa()
- for leaf in tree:
- if leaf.species in species_to_taxid:
- taxid = species_to_taxid[leaf.species]
- lineage = ncbi.get_lineage(taxid)
- leaf.add_feature("lineage", lineage)
+ names = ["Homo sapiens", "Pan troglodytes", "Mus musculus"]
+ name_to_taxids = ncbi.get_name_translator(names)
- # 5. Extract ortholog groups
- ortho_groups = tree.get_speciation_trees()
+ missing = [name for name in names if name not in name_to_taxids]
+ if missing:
+ raise ValueError(f"Names not resolved by NCBI taxonomy: {missing}")
- # 6. Save and visualize
- for i, ortho in enumerate(ortho_groups):
- ortho.write(outfile=f"ortho_{i}.nw")
+ taxids = [name_to_taxids[name][0] for name in names]
+ taxonomy_tree = ncbi.get_topology(taxids)
+ print(taxonomy_tree.to_str(props=["sci_name", "rank"]))
```
- ### Use Case 2: Tree Preprocessing and Formatting
-
- Batch process trees for analysis:
-
- ```bash
- # Convert format
- python scripts/tree_operations.py convert input.nw output.nw --in-format 0 --out-format 1
-
- # Root at midpoint
- python scripts/tree_operations.py reroot input.nw rooted.nw --midpoint
-
- # Prune to focal taxa
- python scripts/tree_operations.py prune rooted.nw pruned.nw --keep-taxa taxa_list.txt
-
- # Get statistics
- python scripts/tree_operations.py stats pruned.nw
- ```
+ ETE 4 also provides `GTDBTaxa` for genome-centric bacterial and archaeal
+ taxonomy. Do not mix NCBI numeric TaxIDs and GTDB string identifiers.
- ### Use Case 3: Publication-Quality Figures
+ ### Visualize
- Create styled visualizations:
+ Interactive SmartView:
```python
- from ete3 import Tree, TreeStyle, NodeStyle, TextFace
-
- tree = Tree("tree.nw")
-
- # Define clade colors
- clade_colors = {
- "Mammals": "red",
- "Birds": "blue",
- "Fish": "green"
- }
-
- def layout(node):
- # Highlight clades
- if node.is_leaf():
- for clade, color in clade_colors.items():
- if clade in node.name:
- nstyle = NodeStyle()
- nstyle["fgcolor"] = color
- nstyle["size"] = 8
- node.set_style(nstyle)
- else:
- # Add support values
- if node.support > 0.95:
- support = TextFace(f"{node.support:.2f}", fsize=8)
- node.add_face(support, column=0, position="branch-top")
-
- ts = TreeStyle()
- ts.layout_fn = layout
- ts.show_scale = True
+ from ete4 import Tree
- # Render for publication
- tree.render("figure.pdf", w=200, units="mm", tree_style=ts)
- tree.render("figure.svg", tree_style=ts) # Editable vector
+ tree = Tree("((A:1,B:1)90:0.2,C:1);", parser="support")
+ tree.explore()
```
- ### Use Case 4: Automated Tree Analysis
-
- Process multiple trees systematically:
+ Static SmartView screenshot:
```python
- from ete3 import Tree
- import os
-
- input_dir = "trees"
- output_dir = "processed"
-
- for filename in os.listdir(input_dir):
- if filename.endswith(".nw"):
- tree = Tree(os.path.join(input_dir, filename))
-
- # Standardize: midpoint root, resolve polytomies
- midpoint = tree.get_midpoint_outgroup()
- tree.set_outgroup(midpoint)
- tree.resolve_polytomy(recursive=True)
-
- # Filter low support branches
- for node in tree.traverse():
- if hasattr(node, 'support') and node.support < 0.5:
- if not node.is_leaf() and not node.is_root():
- node.delete()
-
- # Save processed tree
- output_file = os.path.join(output_dir, f"processed_{filename}")
- tree.write(outfile=output_file)
+ tree.render_sm("tree.png", w=1200, h=800)
```
- ## Reference Documentation
-
- For comprehensive API documentation, code examples, and detailed guides, refer to the following resources in the `references/` directory:
-
- - **`api_reference.md`**: Complete API documentation for all ETE classes and methods (Tree, PhyloTree, ClusterTree, NCBITaxa), including parameters, return types, and code examples
- - **`workflows.md`**: Common workflow patterns organized by task (tree operations, phylogenetic analysis, tree comparison, taxonomy integration, clustering analysis)
- - **`visualization.md`**: Comprehensive visualization guide covering TreeStyle, NodeStyle, Faces, layout functions, and advanced visualization techniques
-
- Load these references when detailed information is needed:
-
- ```python
- # To use API reference
- # Read references/api_reference.md for complete method signatures and parameters
-
- # To implement workflows
- # Read references/workflows.md for step-by-step workflow examples
+ `render_sm()` produces PNG screenshot data; use the Qt treeview renderer when
+ the deliverable must be vector PDF or SVG. Load
+ [`references/visualization.md`](references/visualization.md) for layouts,
+ faces, remote exploration, and renderer selection.
- # To create visualizations
- # Read references/visualization.md for styling and rendering options
- ```
+ ## Bundled Scripts
- ## Troubleshooting
+ Run from this skill directory. The commands below use a pinned, isolated ETE 4
+ runtime through `uv run --with`.
- **Import errors:**
+ ### Tree operations
```bash
- # If "ModuleNotFoundError: No module named 'ete3'"
- uv pip install ete3
-
- # For GUI and rendering issues
- uv pip install ete3[gui]
+ uv run --with "ete4==4.4.0" python scripts/tree_operations.py \
+ stats tree.nw --parser 1
+ uv run --with "ete4==4.4.0" python scripts/tree_operations.py \
+ ascii tree.nw --parser 1 --props name,dist
+ uv run --with "ete4==4.4.0" python scripts/tree_operations.py \
+ convert tree.nw output.nw \
+ --input-parser 1 --output-parser 1
+ uv run --with "ete4==4.4.0" python scripts/tree_operations.py \
+ reroot tree.nw rooted.nw \
+ --parser 1 --midpoint
+ uv run --with "ete4==4.4.0" python scripts/tree_operations.py \
+ prune tree.nw pruned.nw \
+ --parser 1 --keep species1 species2 species3
+ uv run --with "ete4==4.4.0" python scripts/tree_operations.py \
+ compare tree_a.nw tree_b.nw
```
- **Rendering issues:**
+ Use `--keep-file taxa.txt` instead of `--keep ...` for one taxon per line.
+ The script refuses ambiguous or missing requested names rather than silently
+ producing a partial tree.
- If `tree.render()` or `tree.show()` fails with Qt-related errors, install system dependencies:
+ ### Visualization
```bash
- # macOS
- brew install qt@5
-
- # Ubuntu/Debian
- sudo apt-get install python3-pyqt5 python3-pyqt5.qtsvg
- ```
-
- **NCBI Taxonomy database:**
-
- If database download fails or becomes corrupted:
-
- ```python
- from ete3 import NCBITaxa
- ncbi = NCBITaxa()
- ncbi.update_taxonomy_database() # Redownload database
- ```
-
- **Memory issues with large trees:**
-
- For very large trees (>10,000 leaves), use iterators instead of list comprehensions:
+ # Interactive SmartView
+ uv run --with "ete4==4.4.0" python scripts/quick_visualize.py \
+ tree.nw --parser 1
- ```python
- # Memory-efficient iteration
- for leaf in tree.iter_leaves():
- process(leaf)
+ # SmartView PNG (requires ete4[render-sm])
+ uv run --with "ete4[render-sm]==4.4.0" python scripts/quick_visualize.py \
+ tree.nw tree.png \
+ --parser support --mode circular --show-support --color-by-support
- # Instead of
- for leaf in tree.get_leaves(): # Loads all into memory
- process(leaf)
+ # Vector output via Qt treeview (requires ete4[treeview])
+ uv run --with "ete4[treeview]==4.4.0" python scripts/quick_visualize.py \
+ tree.nw tree.svg \
+ --parser 1 --engine treeview --title "Species phylogeny"
```
- ## Newick Format Reference
-
- ETE supports multiple Newick format specifications (0-100):
-
- - **Format 0**: Flexible with branch lengths (default)
- - **Format 1**: With internal node names
- - **Format 2**: With bootstrap/support values
- - **Format 5**: Internal node names + branch lengths
- - **Format 8**: All features (names, distances, support)
- - **Format 9**: Leaf names only
- - **Format 100**: Topology only
+ ## Quality and Interpretation Checks
- Specify format when reading/writing:
+ Before reporting a result:
- ```python
- tree = Tree("tree.nw", format=1)
- tree.write(outfile="output.nw", format=5)
- ```
+ 1. Confirm the parser preserves the intended internal names, support, and
+ branch lengths.
+ 2. Check for empty and duplicate leaf names before name-based lookup or RF
+ comparison.
+ 3. State whether the tree is treated as rooted or unrooted.
+ 4. Preserve branch lengths when pruning only if retained pairwise distances
+ should remain unchanged.
+ 5. Treat arbitrary polytomy resolution as a display/algorithmic convenience,
+ not evolutionary evidence.
+ 6. Record ETE version, parser, rooting method, pruning set, and taxonomy
+ database snapshot in reproducible analyses.
+ 7. Prefer iterators for large trees and `get_cached_content()` for repeated
+ descendant-content queries.
- NHX (New Hampshire eXtended) format preserves custom features:
+ ## Reference Map
- ```python
- tree.write(outfile="tree.nhx", features=["habitat", "temperature", "depth"])
- ```
+ Load only the reference needed for the task:
- ## Best Practices
+ - [`references/api_reference.md`](references/api_reference.md) — ETE 4 core
+ classes, parsers, properties, traversal, I/O, topology, and comparison
+ - [`references/workflows.md`](references/workflows.md) — complete analysis
+ patterns, validation, reconciliation, batching, and large-tree work
+ - [`references/visualization.md`](references/visualization.md) — SmartView,
+ layouts/faces, PNG screenshots, and Qt vector rendering
+ - [`references/taxonomy.md`](references/taxonomy.md) — NCBI and GTDB setup,
+ translation, topology, annotation, and reproducibility
+ - [`references/migration-ete3-to-ete4.md`](references/migration-ete3-to-ete4.md)
+ — breaking API changes and porting checklist
- 1. **Preserve branch lengths**: Use `preserve_branch_length=True` when pruning for phylogenetic analysis
- 2. **Cache content**: Use `get_cached_content()` for repeated access to node contents on large trees
- 3. **Use iterators**: Employ `iter_*` methods for memory-efficient processing of large trees
- 4. **Choose appropriate traversal**: Postorder for bottom-up analysis, preorder for top-down
- 5. **Validate monophyly**: Always check returned clade type (monophyletic/paraphyletic/polyphyletic)
- 6. **Vector formats for publication**: Use PDF or SVG for publication figures (scalable, editable)
- 7. **Interactive testing**: Use `tree.show()` to test visualizations before rendering to file
- 8. **PhyloTree for phylogenetics**: Use PhyloTree class for gene trees and evolutionary analysis
- 9. **Copy method selection**: "newick" for speed, "cpickle" for full fidelity, "deepcopy" for complex objects
- 10. **NCBI query caching**: Store NCBI taxonomy query results to avoid repeated database access
+ ## Authoritative Upstream Sources
+ - Documentation: https://etetoolkit.github.io/ete/
+ - ETE 3 to ETE 4 migration: https://etetoolkit.github.io/ete/3to4.html
+ - Releases: https://github.com/etetoolkit/ete/releases
+ - PyPI: https://pypi.org/project/ete4/
+ - Source: https://github.com/etetoolkit/ete
+ - Visualization gallery: https://github.com/etetoolkit/ete-gallery