invalid-data-cleaning SKILL.md skillA

invalid-data-cleaning from opensensenova/sensenova-skills is an agent skill: a SKILL.md folder an AI agent loads when a task calls for it.

Install it, pinned
mdr add opensensenova/sensenova-skills/invalid-data-cleaning@git:20260426.f0ab496

Pins git:20260426.f0ab496 in mdr.lock. Pin the fingerprint instead.

Audit grade
A, 17 of 17 checks passed
Kind
SKILL.md skill
Repository
opensensenova/sensenova-skills
Path
skills/sn-da-excel-workflow/capability/excel-data-cleaning/invalid-data-cleaning/SKILL.md
Versions recorded
1
Latest version
git:20260426.f0ab496, committed Apr 26, 2026
Size
2,252 bytes
Repository stars
6,031
License
MIT

用于大规模Excel数据的预处理,通过统计总行数判断是否转换为Parquet格式以提升读写效率,并使用正则表达式清洗指定文本列(如仅保留中文字符),最后导出清洗后的文件并提供下载链接。.

Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.

What the file says

# Invalid_Data_Cleaning

> This sub-skill covers one capability of the Excel workflow. For reading/counting/Parquet optimization, see the parent workflow SKILL.md.

## Skill Steps

Step1 根据总行数判断是否数据量过大,若满足条件,则将 Excel 文件转换为 Parquet 格式提升读写效率,再读取数据进行后续分析。
```python
import pandas as pd

file_path = "input_data.xlsx"
parquet_path = "temp_data.parquet"

# 读取 Excel 文件并转换为 Parquet 格式
xls = pd.ExcelFile(file_path)
dfs = []
for sheet in xls.sheet_names:
    df_sheet = pd.read_excel(xls, sheet_name=sheet)
    dfs.append(df_sheet)

# 合并所有 sheet 数据并写入 Parquet 文件
if dfs:
    df_all = pd.concat(dfs, ignore_index=True)
    df_all.to_parquet(parquet_path, engine='pyarrow', index=False)

# 读取 Parquet 文件用于后续处理
df = pd.read_parquet(parquet_path)
```

Step2 对目标文本字段中的特殊字符(如 #、-、数字)进行清洗,使用正则表达式仅保留中文字符。
```python
import pandas as pd
import re

target_col = 'target_column' # 替换为实际需要清洗的列名

# 定义清洗函数
def clean_chinese_text(text):
    if pd.isna(text):
        return text
    s = str(text)
    # 提取所有中文字符(Unicode 范围:[一-鿿])
    chinese_chars = re.findall(r'[一-鿿]', s)
    cleaned = ''.join(chinese_chars)
    return cleaned if cleaned else ''

# 应用清洗函数
if target_col in df.columns:
…

Read the whole file at its exact version.

How to install

The exact contents, by fingerprint
mdr add opensensenova/sensenova-skills/invalid-data-cleaning@sha256:25a890b17ac40316

The line at the top of this page pins the label git:20260426.f0ab496, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.

Badge

mdr badge

[![mdr](https://markdownregistry.com/badge/art_bf3uip5oq7tpslor.svg)](https://markdownregistry.com/a/art_bf3uip5oq7tpslor)

3 badge image fetches in 30 days. What the badge shows.

Versions

The registry has recorded 1 version of this file, committed on Apr 26, 2026.

versioncommittedcommitsizeaudit
git:20260426.f0ab496 latest2026-04-26 f0ab496 2,252 BA view

Audit of the latest version

A  17 of 17 checks passed. Fixed automatic checks with no AI involved, so the same checks give the same answer every run.
  • pass: Frontmatter block present
  • pass: Frontmatter declares a name
  • pass: Frontmatter declares a description
  • pass: Size between 200 bytes and 200 KB (2252 bytes)
  • pass: No zero-width or bidi control characters
  • pass: No instruction hidden inside an HTML comment
  • pass: No link to an exfiltration or paste host
  • pass: No credential-shaped string
  • pass: No instruction to send local credentials anywhere
  • pass: No text hidden with inline styles
  • pass: No prompt-injection phrasing
  • pass: No curl or wget piped into a shell
  • pass: No recursive delete of root, home or parent
  • pass: No instruction to read or print local credentials
  • pass: No base64 blob over 200 characters
  • pass: No link to a raw IP address
  • pass: No script tag

Source

GitHub

opensensenova/sensenova-skills · 6,031 stars · license MIT · pushed 2026-11-03 · branch main

API

GET https://markdownregistry.com/api/v1/artifacts/art_bf3uip5oq7tpslor
GET https://markdownregistry.com/api/v1/resolve?ref=opensensenova/sensenova-skills/invalid-data-cleaning
GET https://markdownregistry.com/api/v1/blob/25a890b17ac40316bc33d02bdc969a5da517f15af88ee1bd9ea09bd5dab5dd6c

Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.

Guides: the SKILL.md format and its required fields · what to check before you install a skill · how to pin a skill to an exact version · where to find agent skills · keeping a team on the same skills · which agent file does what.

More from opensensenova/sensenova-skills

sn-code-video SKILL.md skill
opensensenova/sensenova-skills · skills/sn-code-video/SKILL.md · Create and revise code-driven videos with Canvas, SVG, HTML/CSS, Three.js, Remotion, Manim, or p5.js, including sound…
git:20261029.dae8ea5 · audit A · 6,031 stars
sn-da-excel-workflow SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/SKILL.md · Excel 数据分析多步编排器。覆盖:(1) 读取多 Sheet Excel 文件并统计行数,(2) 大文件检测(≥10k 行自动 Parquet 优化),(3) 数据清洗(缺失值、文本标准化、无效字符),(4)…
git:20260507.197eabb · audit A · 6,031 stars
large-file-parquet-analysis-and-highlight SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-cell-coloring/category-coloring/SKILL.md · 当Excel文件总行数超过1万行时,通过转换为Parquet格式提升读取性能,提取目标指标并计算最大值,最后将结果输出为Excel并对特定行进行高亮标注。
git:20260426.f0ab496 · audit A · 6,031 stars
excel-conditional-comparison-and-large-file-processing SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-cell-coloring/duplicate-value-coloring/SKILL.md · 对比Excel多表中的特定系数并对异常值进行颜色标记。
git:20260426.f0ab496 · audit A · 6,031 stars
excel-outlier-detection-and-highlighting SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-cell-coloring/outlier-coloring/SKILL.md · 识别 Excel 中的超限数值与错误单元格并进行高亮标注。
git:20260426.f0ab496 · audit A · 6,031 stars
large-file-conditional-formatting SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-cell-coloring/threshold-cell-coloring/SKILL.md · 根据Excel总行数自动切换Parquet加速读取,计算特定维度的时间序列平均值,并使用openpyxl输出带有条件格式(如低于均值标绿)和自定义样式的分析报告。
git:20260426.f0ab496 · audit A · 6,031 stars
top-value-coloring SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-cell-coloring/top-value-coloring/SKILL.md · 根据数据规模动态选择处理策略,对多表数据进行合并、统计筛选,并利用 openpyxl 实现关键指标的自动化样式高亮与格式化导出。
git:20260426.f0ab496 · audit A · 6,031 stars
numeric-extraction-and-distribution-analysis SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-conditional-formatting/data-bar-formatting/SKILL.md · 从带单位的字符串列中提取数值并清洗,生成包含直方图、饼图、条形图和累积分布图的多维度综合分布可视化图表,用于展示数据的集中趋势与分布特征。
git:20260426.f0ab496 · audit A · 6,031 stars
categorical-comparison-analysis SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-data-analysis/comparison-analysis/SKILL.md · 对两类分类数据进行对比分析,统计数量差异与比例关系并生成可视化图表。
git:20260426.f0ab496 · audit A · 6,031 stars
group-by-analysis SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-data-analysis/group-by-analysis/SKILL.md · 对多 Sheet 的 Excel 文件进行行数统计、大文件 Parquet 转换预处理、数据清洗及分组聚合分析,并生成带样式标记的统计表与可视化图表。
git:20260426.f0ab496 · audit A · 6,031 stars
large-file-kpi-analysis SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-data-analysis/kpi-metric-analysis/SKILL.md · 根据数据量自动选择读取策略(大文件转Parquet),提取关键指标进行单位一致性验证与排序分析,并输出可下载的结果表格。
git:20260426.f0ab496 · audit A · 6,031 stars
pivot-table-cross-analysis SKILL.md skill
opensensenova/sensenova-skills · skills/sn-da-excel-workflow/capability/excel-data-analysis/pivot-table-analysis/SKILL.md · 利用交叉表与热力图对分类数据进行多维度占比分析,适用于奖项分布、绩效评估或市场占有率等结构化数据的清洗与可视化。
git:20260426.f0ab496 · audit A · 6,031 stars

Every file in opensensenova/sensenova-skills

Browse by kind, by grade A, or by owner.