data-engineering SKILL.md skillA

data-engineering from magnus919/agent-skills is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.

Install it, pinned
mdr add magnus919/agent-skills/data-engineering@git:20260914.9917fb4

Pins git:20260914.9917fb4 in mdr.lock. Pin the fingerprint instead.

Audit grade
A, 17 of 17 checks passed
Kind
SKILL.md skill
Repository
magnus919/agent-skills
Path
data-engineering/SKILL.md
Versions recorded
5
Latest version
git:20260914.9917fb4, committed Sep 14, 2026
First version recorded
Jul 21, 2026
Size
6,034 bytes
Repository stars
162
License
MIT

Design and operate data infrastructure — database operations (vector,.

Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.

What the file says

# Data Engineering Methodology

Data engineering is the operational backbone of data-driven systems. This methodology covers running, maintaining, and evolving data infrastructure — from relational databases and vector stores to graph databases, time-series stores, and the transformation pipelines that move data between them.

When a model proposes data transformations, load [the AI boundary workflow](references/ai-transformation-boundaries.md)
and use [the companion record](templates/ai-stage-contract.md).

## The Data Engineer's Domain

| You own | You don't own |
|---------|--------------|
| Database operations — schema management, indexing, backup/recovery, migration across relational, vector, graph, and time-series stores | Data modeling and schema design — that's the data architect |
| Data transformation pipelines — dbt models, ETL/ELT patterns, incremental loading, incremental strategies | Statistical analysis and experiments — that's the data scientist |
| Analytical SQL — window functions, CTEs, query optimization, execution plan analysis, star schema queries | Training infrastructure and model deployment — that's the ML engineer |
…

Read the whole file at its exact version.

How to install

The exact contents, by fingerprint
mdr add magnus919/agent-skills/data-engineering@sha256:5e06d491b7de8b13

The line at the top of this page pins the label git:20260914.9917fb4, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.

Badge

mdr badge

[![mdr](https://markdownregistry.com/badge/art_ngbwvkgpqaiqz7o7.svg)](https://markdownregistry.com/a/art_ngbwvkgpqaiqz7o7)

2 badge image fetches in 30 days. What the badge shows.

Versions

The registry has recorded 5 versions of this file, between Jul 21, 2026 and Sep 14, 2026, about one every 14 days.

versioncommittedcommitsizeaudit
git:20260914.9917fb4 latest2026-09-14 9917fb4 6,034 BA view · diff
git:20260809.4a5f18e2026-08-09 4a5f18e 5,581 BA view · diff
git:20260804.6f67a342026-08-04 6f67a34 5,323 BA view · diff
git:20260803.0223e2b2026-08-03 0223e2b 4,969 BA view · diff
git:20260721.c7c4d3b2026-07-21 c7c4d3b 4,951 BA view

Audit of the latest version

A  17 of 17 checks passed. Fixed automatic checks with no AI involved, so the same checks give the same answer every run.
  • pass: Frontmatter block present
  • pass: Frontmatter declares a name
  • pass: Frontmatter declares a description
  • pass: Size between 200 bytes and 200 KB (6034 bytes)
  • pass: No zero-width or bidi control characters
  • pass: No instruction hidden inside an HTML comment
  • pass: No link to an exfiltration or paste host
  • pass: No credential-shaped string
  • pass: No instruction to send local credentials anywhere
  • pass: No text hidden with inline styles
  • pass: No prompt-injection phrasing
  • pass: No curl or wget piped into a shell
  • pass: No recursive delete of root, home or parent
  • pass: No instruction to read or print local credentials
  • pass: No base64 blob over 200 characters
  • pass: No link to a raw IP address
  • pass: No script tag

Source

GitHub

magnus919/agent-skills · 162 stars · license MIT · pushed 2026-11-01 · branch main

API

GET https://markdownregistry.com/api/v1/artifacts/art_ngbwvkgpqaiqz7o7
GET https://markdownregistry.com/api/v1/resolve?ref=magnus919/agent-skills/data-engineering
GET https://markdownregistry.com/api/v1/blob/5e06d491b7de8b137956181f89c92d27aa6c6aaf52924c5b15622e2a5dce4aed

Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.

Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.

More from magnus919/agent-skills

AGENTS.md AGENTS.md file
magnus919/agent-skills · AGENTS.md
git:20261010.bfe1eb1 · audit A · 162 stars
actuarial-risk-modeling SKILL.md skill
magnus919/agent-skills · actuarial-risk-modeling/SKILL.md · Use when analyzing, selecting, validating, or communicating models for insurance, actuarial, financial-risk, or other…
git:20260823.077c5d0 · audit A · 162 stars
adr-authoring SKILL.md skill
magnus919/agent-skills · adr-authoring/SKILL.md · Write, review, and maintain architecture decision records with clear context, alternatives, consequences, confirmation…
git:20260906.d8f3554 · audit A · 162 stars
agent-council SKILL.md skill
magnus919/agent-skills · agent-council/SKILL.md · Run a structured multi-agent debate by spawning a panel of expert agents on any question, with convergence-aware…
git:20260823.6efee5c · audit A · 162 stars
agent-evals-and-observability SKILL.md skill
magnus919/agent-skills · agent-evals-and-observability/SKILL.md · Design, run, review, or release framework- and vendor-neutral evaluations and observability for AI agents. Use when…
git:20261020.92c327e · audit A · 162 stars
AGENTS.md@agent-production-operations AGENTS.md file
magnus919/agent-skills · agent-production-operations/AGENTS.md
git:20260814.c0c7690 · audit A · 162 stars
agent-production-operations SKILL.md skill
magnus919/agent-skills · agent-production-operations/SKILL.md · Operate an evaluated agent with tools and authority in production through a runtime control plane covering versioning…
git:20261020.03a72d6 · audit A · 162 stars
agent-skills SKILL.md skill
magnus919/agent-skills · agent-skills/SKILL.md · Use this skill when creating, reviewing, or editing Agent Skills-format skills, or when implementing skill discovery…
git:20261029.1e3aa45 · audit B · 162 stars
ai-governance SKILL.md skill
magnus919/agent-skills · ai-governance/SKILL.md · Design and operate AI governance: principles, decision rights, risk tiers, lifecycle gates, fairness, transparency…
git:20261016.06ca706 · audit A · 162 stars
ai-operating-economics SKILL.md skill
magnus919/agent-skills · ai-operating-economics/SKILL.md · Use when deciding whether an AI-enabled workflow should be adopted, scaled, constrained, redesigned, or retired, and…
git:20260821.1291f95 · audit A · 162 stars
amazon-memo SKILL.md skill
magnus919/agent-skills · amazon-memo/SKILL.md · Create, adapt, or revise Amazon-style narrative decision memos, six-pagers, PR/FAQs, and short decision briefs…
git:20261019.89945ee · audit A · 162 stars
analog-occultism SKILL.md skill
magnus919/agent-skills · analog-occultism/SKILL.md · Use this skill to create subject-neutral images and videos in an analog occultism / industrial CRT noir aesthetic…
git:20260905.3d54d7b · audit A · 162 stars

Every file in magnus919/agent-skills

Other files named data-engineering

data-engineering SKILL.md skill
cbrock84/headcount · plugins/data-analytics/skills/data-engineering/SKILL.md · Builds and operates data pipelines — ingestion, transformation, orchestration, quality testing, and reliability of data…
git:20260916.e72e404 · audit A · 2,543 stars
data-engineering SKILL.md skill
travisjneuman/.claude · skills/data-engineering/SKILL.md · ETL/ELT pipelines, data warehousing (BigQuery, Snowflake, Redshift), stream processing (Kafka, Spark Streaming)…
git:20260328.75c803a · audit A · 106 stars
data-engineering SKILL.md skill
yuecao365/offercome · src/lib/mock-interviews/skills/data-engineering/SKILL.md · 数据工程怎么面:数仓建模、离线与实时链路、调度、数据质量、湖仓。数据开发与数仓岗读。
git:20260920.a4843f5 · audit A · 24 stars
data-engineering SKILL.md skill
d-padmanabhan/agent-engineering-handbook · skills/data-engineering/SKILL.md · Data engineering workflows for designing and reviewing batch/streaming pipelines across Databricks, Snowflake…
git:20260819.d23bb73 · audit A · 20 stars
data-engineering SKILL.md skill
kina2711/data-department-agent-skills · skills/data-engineering/SKILL.md · Design, build, test, diagnose execution plans and operate batch, API, file, CDC and streaming pipelines with…
git:20260910.94a05b4 · audit A · 3 stars

Browse by kind, by grade A, or by owner.