llm-inference-serving skillA
llm-inference-serving is agent-read markdown (skill) from yuecao365/offercome: 推理服务深挖：投机解码、量化与算子、分布式推理与 PD 分离、MoE 服务、前缀缓存、约束解码、容量。.
Indexed from public GitHub and served as immutable, content-addressed versions. Install it pinned to an exact SHA-256 with the mdr CLI, and every file is verified against the hash recorded here before it reaches your agent. The deterministic audit below grades the latest version, and the same file always earns the same grade.
What the file says
## 面试官在意什么 这本是 ai-infra 的推理那一段往下挖：把一个模型的吞吐与延迟再抬一档要动哪些机制。面试官要算账（接受率、并行通信量、KV 传输带宽、量化后的精度差）和要故障（PD 分离后 TTFT 反而变差、MoE 专家不均、量化后某类请求崩）。应用岗读它是为了知道自己的 agent 每回合的首字延迟和缓存命中从哪来。 ## 项目 / 实习怎么深挖 简历上出现下面这类经历时从哪里切、追什么。追到候选人能说出机制、数字的来源与一次真实的故障或取舍才算实；只有框架名与结论、说不出自己那一段的，记为危险信号。通用的追问方法见 project-deep-dive。 - 简历出现投机解码 → 追草稿怎么产生、接受率、什么分布下加速消失、和 batch 的冲突 - 简历出现量化 → 追方法与量化对象、精度回归、kernel 来源、实测吞吐 - 简历出现多卡 / 多机推理 → 追并行方式、通信占比、PD 分离有没有做、掉卡表现 - 简历出现 MoE 推理 → 追专家并行与 all-to-all、负载不均、显存放法 - 简历出现前缀缓存 / 多轮 → 追命中率、驱逐策略、什么请求模式命中低 - 简历出现结构化输出 / 约束解码 → 追怎么实现、对吞吐的影响、和采样的冲突 ## 常见失守与危险信号 - 投机解码：只会"小模型猜"；说不出接受率决定加速比；不知道 batch 大时收益消失 - 量化与算子优化：量化只知道"变小变快"；不做精度回归；写 kernel 不看 profiler - 分布式推理与并行：张量并行说不出通信在哪；不知道 PD 分离解决什么、代价是什么 - 长上下文与显存：只答"加显存"；不知道 offload 与 KV 压缩的取舍 - PD 分离与 KV 传输：不知道 KV 怎么在实例间传、带宽要多少；不知道什么规模才值得分 - MoE 服务：只答"选专家"；不谈 all-to-all 与负载不均；不知道专家怎么放显存 - 前缀缓存与多轮：只答"缓存前缀"；说不出命中条件；不知道系统提示放前面的意义 - 约束解码与采样：不知道结构化输出在推理侧怎么实现；不知道它对吞吐的影响 - 容量规划：不会从 SLO 反推卡数；没有峰谷弹性方案 ## 常考主题清单 只列名字、阶梯与答实的标志，作"问到哪一层算实"的参考；问哪些、问几道由这份 JD 与这份简历定，不是配额。 ### 投机解码 - 阶梯：为什么 decode 阶段有"白算"的空间 → 草稿模型 / n-gram / 自草稿（Medusa、EAGLE 类）各怎么产生候选，验证怎么一次前向完成 → 加速比由接受率与草稿长度决定，什么分布下接受率低；batch 大时为什么收益消失 → 验证的显存与算力成本、和 continuous batching 的冲突、什么业务值得开 …
Read the whole file at its exact version.
How to install
mdr add yuecao365/offercome/llm-inference-serving@git:20260920.a4843f5mdr add yuecao365/offercome/llm-inference-serving@sha256:4e99f4872715f3cePin to a label to follow the author's releases, or to a sha256 to freeze the exact bytes forever. Either way the resolved hash is written to mdr.lock, and mdr install reproduces it on any machine.
[](https://markdownregistry.com/a/art_fsj2mvvggrkutje2)
1 badge views in 30 days
Versions
Audit of the latest version
- pass: Frontmatter block present
- pass: Frontmatter declares a name
- pass: Frontmatter declares a description
- pass: Size between 200 bytes and 200 KB (7843 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No prompt-injection phrasing
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Source
yuecao365/offercome · 23 stars · license MIT · pushed 2026-09-23 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_fsj2mvvggrkutje2 GET https://markdownregistry.com/api/v1/resolve?ref=yuecao365/offercome/llm-inference-serving GET https://markdownregistry.com/api/v1/blob/4e99f4872715f3cef6a3d5d6a9482c9522dfce3e0aa35a305bb21d69e289266d
Your agent does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.