We've all bought books that boasted of YEARS of playtesting on the back of the book, despite having entire rules chapters that make no sense.

I'm going to contend that playtesting is not a linear progression or a progress bar in an RPG. More isn't better. Better is better.

Let's dive in:

This goes for board games, card games, miniatures games, roleplaying games and probably all manner of other stuff too.

There's a few different things you can do that I would fit under "testing" all of which has varying amounts of value.

* Casual reading

This is the most basic level: Have other people read your game and tell you what they think.

This is good for providing a basic gut-level "does this look cool?" impression.
However, it rarely provides substantial feedback.
If you are soliciting feedback from random strangers, you have no idea if they are crazy and most will be incapable of providing the kind of mechanical insights you need.

Get a little bit of this, but don't rely on it.

* Critical reading

A line by line reading of the text with an eye towards rules that aren't explained well, references that aren't repeated, etc.

This requires a person with attention to detail and an understanding of game mechanics.

You need at least one person doing this.

* Proof reading

A thorough read aimed at catching typos, wrong words, grammatical problems etc.
This is best done by someone with experience in editing, but reasonable results can be obtained if you have a few people go through the text.

You need at least one person doing this.

* Math testing

Sitting down with dice and a calculator and crunching the numbers.
How likely is an attack to hit? How many attacks will destroy their target? If X amount of troops fire, what is the expected outcome? What are the best and worst case scenarios?

This often leaves out the "soft" factors that happen on a gaming table (for example a unit is often not able to fire in every turn of a battle) but can be invaluable in establishing the base lines for how the system is going to function.

Just remember that theoretical effectiveness is always higher than actual effectiveness (miniatures units have blocked lines of sight or are forced to redeploy, RPG characters take non-combat actions etc.)

This will save you a ton of time and ensure you have a sound basis for your game.

* Personal play test

Play testing done personally by the writer.
This is often solo or with personal friends.

This is the initial "Crash test" to figure out if the game works at all.
Beyond a few games to verify the concept, this is a diminishing return because you already know how everything is /supposed/ to work.

* Directed play test

Play testing set up by the creator (or a close associate) but with people who aren't inherently in your play group. A club or store game f.x.

Pay close attention to what rules the group internalizes quickly vs what rules they keep struggling with or flat out forgetting.
What rules do they get excited about? What parts do they protest?

Try to let the game "run itself" as much as possible.
Is your reference sheet enough to let them play through a few turns?

* Blind play test

The gold standard:

A group plays the game without you being there to tell them what to do.

This is where the real play testing happens: Your future customer will not have you present to explain exactly what that rule means. Your text needs to stand on its own feet.

To be of value of course you need a group that is willing to provide feedback in detail, isn't afraid to interpret how they think things should work if the rules aren't clear and explain that to you.

This also means you need to take it in stride. They might hate your game. They might do it wrong. They might love it BECAUSE they did it wrong.
You want to write games for people, you gotta play in the big boy/girl/robot/non-binary leagues and take your lumps.


* * * * *
So now that we know the ways you can put your design through the paces, what should you focus on?

First and foremost get feedback. Any feedback.
If you are an unknown, that can be hard. The gaming hobby is full of "idea guys" and you have to convince them that your idea is better than the 100 other guys who just re-invented D&D but the elf is called a belf this time.

It can help to put out a beta version. Make it clear it's a work in progress and make it substantial enough to be a game in its own right and put it up for download. People love the idea of a game they can help influence.

If nobody bites when you put it on a forum, put it on rpgnow and charge a dollar or make it PWYW:
The capitalist priesthood teaches us that only things that cost money have value.

Once you have people's attention, look at the feedback you get:
Some will be people who just casually read through it and will give you the equivalent of "this sucks" or "this is cool". Those are a handy barometer but aren't really that useful to go anywhere.

Look for and listen for the people who put in the effort to read it in detail. Who asks questions about that optional rule on page 34, who says "we tried this over the weekend and had some questions".
There's a lot of nerds out there who love doing this stuff and play testing is a skill just as well as writing is.

Some of the people will be crazy.
Some will convince themselves that your game would be PERFECT for [insert concept here that you have no interest in].
Don't argue, just politely thank them and move on.

Ultimately, what you want is the blind play test:
Each game played by a decent group that you are NOT a part of is worth 10 games you put on yourself.

* * * * *
So why does the book say "playtested for a decade" and you found 5 mistakes and a rule thats outright broken?

Because what they mean is usually "the author and his group played this twice a month for years and they already knew how everything is supposed to work".
The authors copy has "+2 for cover" pencilled in the margins but it never made it to the final version you just paid 50 dollars for.

Even blind playtesters can't always catch everything.
Some options don't get tested.
A particular character weapon+ability combo may simply never have come up.
RPG playtesting is notoriously difficult, because 90% of an RPG session tends to have nothing to do with the game system at all.

A group may have playtested your RPG for months and provided good feedback, but their campaign didn't involve [thing] that the problem exists with, so it never came up.

Playtesting seriously often means ensuring that situations happen that won't do so in a conventional game.
Launch a charge, just so you can see the close combat system.
Try to persuade the orc villain.
Level up in a strange way.
Try having both sides dig in their forces and shoot it out, instead of marching on the objective.

You have to be willing to push the boundaries of the system to find out if the boundaries even work at all.

* * * * *

I hope this helps a little bit with getting better testing for your games.

Let me know what you think and what your own experiences offer.

Good luck out there.