앱에서 오픈소스로 — 검증 가능한 LLM 출력을 위한 재사용 패턴

Earlier I wrote about how LevelUp Piano generates practice songs with an LLM. That was a design story — the why and the shape of the pipeline — but there was no code you could actually run.

So I pulled the core out into an open-source tool: etude-gen. One command turns a spec into a validated, playable MIDI file, using the OpenAI API to compose and plain deterministic code to make it real.

npx tsx src/cli.ts --difficulty easy --key C --technique "left-hand arpeggios"

But a tool being runnable isn't why it deserves a second post. The reason is that once I stripped away the piano, what was left was a pattern that has nothing to do with music — and everything to do with getting trustworthy output out of a language model.

The one idea worth stealing

Let the LLM work in the symbolic vocabulary it's fluent in. Push everything that must be exactly right into deterministic code. Bridge the two with a validate-and-repair loop.

That's the whole thing. Everything below is just that sentence, made concrete.

Part 1 — Let the model compose, under a schema

The naive approach is to ask the model for the finished artifact directly: milliseconds, MIDI note numbers, the works. Do that and arithmetic mistakes pour out — "which millisecond is beat 3 at 84 BPM?" is exactly the kind of per-token math LLMs are worst at.

So the model never sees a millisecond. It writes an intermediate representation the way a human composer thinks — in measures, beats, and note names — and I enforce that shape with OpenAI Structured Outputs. The Zod schema is the contract:

const NoteSchema = z.object({
  hand: z.enum(["L", "R"]),
  pitch: z.string().describe('Scientific pitch name, e.g. "E4", "F#3"'),
  beat: z.number().describe("1-based beat position; fractions allowed"),
  len: z.number().describe("Duration in beats"),
});
 
const SongIRSchema = z.object({
  title: z.string(),
  key: z.string(),
  tempo_bpm: z.number(),
  time_signature: z.object({ beats: z.number().int(), unit: z.number().int() }),
  difficulty: z.enum(["easy", "normal", "hard"]),
  measures: z.array(z.object({ n: z.number().int(), notes: z.array(NoteSchema) })),
});
 
const completion = await client.beta.chat.completions.parse({
  model: "gpt-5.5",
  messages,
  response_format: zodResponseFormat(SongIRSchema, "song"),
});
const song = completion.choices[0].message.parsed; // typed, guaranteed-shaped

No parsing, no "please respond in JSON," no regex salvage. The response comes back already matching the type. That frees the model to spend its effort on the creative part — the melody — instead of formatting.

Part 2 — The validator is your spec, and its errors are prompts

Structured Outputs guarantees the shape, not the meaning. Nothing stops the model from writing a note that overflows its measure, or a chord no hand can reach, or a piece that wanders out of key. So every IR runs through a validator before it's trusted.

For a practice-song generator, "correct" means playable and pedagogically appropriate. That's encoded as difficulty envelopes — the teaching constraints that make output educational rather than merely musical:

Plus, for every difficulty: per-hand pitch ranges, measures that actually add up, ≤ 20% out-of-key notes, tempo in a playable band.

Here's the move that makes the whole thing converge: a validation error isn't written for a human — it's written for the model.

measure 1, note 1 (E4): beat 4.5 + len 1 overflows a 4/4 measure
measure 1: R hand plays 2 simultaneous notes, exceeding the easy limit of 1
measure 2, note 1 (H9): "H9" is not a valid pitch name

Each line names where, what, and by how much. That's not a log entry — it's an instruction. It goes straight back into the next prompt.

Part 3 — Push all the arithmetic into deterministic code

Only once an IR passes does the deterministic half run. The compiler does every calculation the model was never allowed to touch: beats become ticks, ticks become milliseconds, note names become MIDI numbers.

const ticksPerBeat = TPQN * (4 / unit);
const startTick = Math.round(((measure.n - 1) * beats + (note.beat - 1)) * ticksPerBeat);
const startMs   = Math.round(startTick * (60_000 / (tempoBpm * TPQN)));

Same IR in, same bytes out, every time. A zero-dependency Standard MIDI File encoder writes the .mid; a parallel note chart carries the millisecond timings a game engine would consume.

The loop, in one function

Structured Outputs (shape) + validator (meaning) + deterministic compiler (exactness) only become a pipeline when you wire the failure path back to the model:

for (let attempt = 1; attempt <= maxAttempts; attempt++) {
  const song = (await client.beta.chat.completions.parse({ ... })).choices[0].message.parsed;
  const errors = validate(song);
  if (errors.length === 0) return { song, attempts: attempt };
 
  messages.push(
    { role: "assistant", content: JSON.stringify(song) },
    { role: "user", content: `That song failed validation:\n${errors.map(e => `- ${e}`).join("\n")}\n\nRegenerate the complete song with every issue fixed.` },
  );
}

Generate → validate → feed failures back → regenerate. The model corrects itself against a hard oracle instead of a human.

Try it in 30 seconds

You don't even need a key to see the deterministic half work — a bundled example IR compiles offline:

git clone https://github.com/devyourown/etude-gen && cd etude-gen && npm install
npm run generate -- --from-ir examples/meadow-morning.json   # no API key needed

With a key, the full loop runs. A real run just now, --difficulty easy --key C, produced Morning Brook — 8 measures, 47 notes, a left-hand C-major broken-chord accompaniment under a stepwise right-hand melody — passing validation and compiling to a MIDI file you can drop into GarageBand.

Why this has nothing to do with music

Swap out the domain and the skeleton is unchanged. Anywhere a model has to produce something creative that must also satisfy hard constraints, the same three parts apply:

The lesson generalizes cleanly: don't ask the model to be precise — ask it to be creative in a vocabulary you can check, and let code be precise for it.

Closing

The original post was the story of solving a real product problem: don't run out of songs. This one is what was underneath it, extracted so anyone can pick it up — a small, MIT-licensed OpenAI API sample and a pattern I keep reaching for. Milliseconds to the code, melody to the model, verification to a loop that closes on its own.

Repo: github.com/devyourown/etude-gen.