As I understand, music is like language. You have this written notation that you can translate to sound with a device — vocal cords and musical instruments. Sound that your brain interprets as concepts in your mind.

Composers and writers are wizards that translate concepts in their minds to these concrete written runes that, once spoken, or played through magical instruments, convey meaning — notes and melodies. Music.

When I learned to play the keyboard, I could translate the written notation directly into the equivalent notes on the instrument and slowly learn to play a song by sheer repetition. While I could play some decent stuff, being able to do only that was still equivalent to being illiterate.

You know the alphabet and can slowly translate the letters and syllables to sounds and words; you can even memorize an entire speech. But you are not really using the language: you can’t translate the concepts in your mind into sentences and paragraphs, i.e. melodies, chord sequences, etc.

And that is lame.

While you can train the essential skill of understanding notes in the context of the key through something like the Functional Ear Trainer app, I found a big problem with that approach: it’s very boring. There’s this big skill gap where you can’t do anything interesting with the first intervals you learn. I get no joy with it. (well… this probably happens with most skills, but ok).

My gaming-programming-dopamine-flooded brain can’t keep up with learning these very basic intervals for months until I can do something interesting with them. But then, I realized that not only could we make a real game out of this, but it could be extremely fun.

We have a bunch of musical games that are rhythm-based. Guitar Hero, Osu!, Crypt of the Necrodancer, Beat Saber, you name it. While being great, they have nothing to do with the skill I’m describing (understanding the actual notes, relative pitch, etc). Rocksmith goes a step further: you play actual notes with a real instrument. But it falls for the same problem I described above: you are only mechanically translating the notes; no need to understand them.

What I’m proposing is the Real Musical Game, where you play with actual music: notes, melodies, chords, and harmonies. You start with single notes and then progress slowly, learning the small intervals and so on.

Some loose ideas about how the design could be:

  • Notes are simple attacks, and melodies are combos.
  • Chords progressions set the “mode” of action somehow.
  • Minor chords have a slightly different effect than major chords.
  • The more harmony, the more effective your actions. But a pinch of disharmony would have some interesting usable effect — perhaps a “cursed” one.
  • Specific melodies cast spells (ala Magicka) or counter specific enemies (ala Crypt of the Necrodancer).
  • Playing the sample sound of Mario mushroom spawns a magic mushroom. We can expand this concept to an entire gameplay system, ala Scribblenauts.

Ideally, we would build the base gameplay on the fundamental elements of music, and we would be free to combine them with fairly consistent rules. We form ever more intricate patterns resulting in combined and specific effects, allowing exploration and discovery. It should be in such a way that playing an entire song would make sense and just work — it would be equivalent to playing it on Very Hard mode.

While I acknowledge it may require significant amounts of computer cleverness to make this work, with the current IA revolution happening, there may be a not-that-hard way to make the game “understand” most arbitrary musical inputs from the user and procedurally generate challenges. And the result would be, sincerely, pure gold.

Games with related concepts: