The following originally appeared on my blog, back in August, during the “RPG a Day” event. One of the prompts for that event was to mention where you go for your RPG reviews. I noticed a lot of chatter on that day about the validity of reviewing RPGs, and the way it should be approached. That prompted me to post this to my blog.

Today’s RPG A Day question, which asked where you go for your RPG reviews, sparked a lot of discussion that I wasn’t expecting. Primarily, it pointed out to me that some people that I respect a great deal either don’t think the RPG industry is large enough to support traditional critical reviews, or that reviews that do not include play experience with the game are of great value.

I’m not taking to the internet to tell anyone that they are wrong, or to defend why I do what I do. In fact, the existence of my blog or my reviews is ancillary to my personal beliefs on this topic. My only point is to explain why I do think there is value to reviews, even reviews that are written before the play experience can be factored into that review.

Let me summarize some of the (entirely valid) points that I have seen made across the internet today:

  • People would not review movies or video games based only on scripts or instructions
  • The play experience that you might envision from reading the rules may not match the play experience of running the game
  • The RPG industry is too small to be served by more academic reviews, and is better served by discussions about games

The first thing I would like to do is to say that I agree with all those statements, if they were amended to say that all those things are important, but not to the exclusion of thoughtful reviews.

People would not review movies or video games based only on scripts or instructions

Comparing a review of an RPG rulebook to a script or set of instructions misses some of the nuance of what the game book actually does, and what players of an RPG are expected to do. The rules in the book are the code that the players use to run the game. They aren’t exterior instructions, but the actual language that should be understood and engaged to make the game work. It is not the whole experience, but it is a greater part of the experience than the elemental components of other styles of entertainment.

I used this analogy in a recent review, but if you saw an impressive Lego set, and you wanted to build the model shown on the front of the box, you would likely be disappointed if the instructions were deficient in telling you how to do this. You have all the components. The Legos are no less awesome, and the final product will still be impressive, but it is important to explain to a prospective buyer that they are going to invest a significant amount of time in just analyzing the components and using their own knowledge to fill in the gaps in the instructions.

The play experience that you might envision from reading the rules may not match the play experience of running the game

The play experience will almost certain not match exactly what you envision in your mind when you read through a book. When you engaged the rules as you read, you were facing the rules one on one, directly. At the table, you will have multiple people thinking of interactions that did not occur to you when you were reading. But while I will certainly agree that the play experience will be different than you envision, I also think that it is possible to find where you, personally, will have problems engaging with the rules before it comes to the table.

The RPG industry is too small to be served by more academic reviews, and is better served by discussions about games

The RPG industry is relatively small compared to other entertainment industries. I think it is very important that there be open and communicative places for gamers to go to ask questions and posit new ideas. It is also true that some people are new to any given RPG community at any given time, and may not be comfortable engaging in conversations about an RPG. Some people, even when they have been part of a community for a long time, remain more comfortable as spectators and consumers than active participants in conversations. In fact, it is a trap that various RPG communities fall into, when they assume that only the people that are actively communicating are receiving any benefit from the existence of the community.

Because there are people that are not active conversationalists, I think it is even more imperative that reviews exist that might spell out, clearly, what the reviewer expects from a product, what the product delivers, and where the product may not be as it seems. To those people that either do not wish to engage, or just don’t wish to engage consistently, I think there is a definite value to presenting a thorough, well-reasoned review.

  • Actual play experience is always going to be a valuable piece in evaluating a game
  • Dynamic conversation is always going to create a more textured understanding of a topic than the static opinions of one reviewer

Neither of these facts invalidates the usefulness of reviews, and specifically, reviews that are based only on the product, and not the full play experience.

Why I Love Reviews

When I was younger, I loved watching Siskel and Ebert. My mother hated the show. Her opinion was that these were two people that sat in judgement of things other people might like, and told them what they should think. For some reason, despite being in my formative years, I never adopted her opinion. I would go out of my way to watch the show, especially if something I wanted to see was featured.

Yes, there were times I would get angry when something I was sure was the greatest movie ever made got panned. But I kept watching. I even watched those “boring” reviews of things like dramas and romances that I knew I was never going to like. Why couldn’t they just keep talking about action movies and sci-fi and horror? But things started to seep into my brain. They weren’t just watching these movies deciding what they liked and what they didn’t without any guidance. They compared them to other movies that had attempted the same techniques. They pointed out where some aspects of the movie were good, even when the movie, as a whole, didn’t work.

Eventually, I realized that what I liked was the analysis, not the final opinions.

By the time I started to realize how much I liked the analysis of pop culture, I started reading RPG reviews in Dragon Magazine. I started reading reviews before I ever played anything other than Dungeons and Dragons. I read reviews from people like Jim Bambra, Rick Swan, and Allen Varney, and I started to see that not all RPGs had similar rules to D&D, and that the way an RPG held together internally was more important than if it seemed like a cool way to use laser guns in a d20 level based system.

To this day, I haven’t played half the games I read reviews for, and yet, the analysis of presentation and rules in those articles helped create in me an appreciation for multiple rules that can be used to accomplish similar things in different context.

But It’s Not That Simple, Right?

 While I don’t think a reviewer needs to have played the game in question to write a valid review, I do think that a reviewer needs to have played a wide range of RPGs to give the best review. 

For a review to have value, I think there are some important elements that must properly align. While I don’t think a reviewer needs to have played the game in question to write a valid review, I do think that a reviewer needs to have played a wide range of RPGs to give the best review.

A person that has only played level based d20 games may give a decent accounting of a supplement for a game with that same base assumption, but when faced with a more narrative game, they aren’t going to be able to provide as many useful insights. I can attest to this myself. When I first read Dungeon World, I didn’t get it, and while I stated that fact on the blog, I didn’t frame it as a review. I needed to play a wider range of more open ended, narrative games before I really understood it.

It’s also very important for a reviewer to state their biases, and what they find important. No one is without bias, and knowing that a reviewer has a weakness for a certain style of adventure or genre is going to provide context for the reader. Evaluating how much the reviewer’s tendencies match the reader’s is going to be extremely valuable.

The reviewer should also call out enough important details that they provide an accurate picture of the product. No review is going to be able to explain exactly what is on every page of a book, but understanding the structure and level of detail that the product utilizes is going to help the reader weigh what level of effort has gone into different aspects of the production.

Knowing is Half the Battle

The worst mistake anyone, reviewer or consumer of reviews, can make, is to assume that the purpose of a review is to find a source to tell them if they should or shouldn’t buy a product. This may sound counter-intuitive, but this is an important bit of nuance. The purpose of a review should be to help the reader determine if the product is for them, but that determination does not need to match the reviewer’s conclusion for the review to be successful. The review should provide enough texture that the consumer can form their opinions based on facts that they have gathered, not based on the specific conclusion of the reviewer.

Why would a reviewer even come to a conclusion then, if they believe this to be the case? I can’t speak for everyone, but I know that if I don’t hold myself to some kind of metric, my reviews meander. Without that metric, it is hard for me to see if my positives have more weight than my negatives. It is a way for me to clarify my own thoughts.

Ideally, a consumer can find more than one reviewer that they find entertaining and informative, and they can contrast where one reviewer’s biases may have led them to omit details important to the consumer of the review. Even without that, the consumer can’t be passive in reading a review if they hope to gather the best results. I can’t speak for other reviewers, but my actual score is a tool to bring out the points I want to make in the review, rather than the actual point of process.

And on that note, I’m going to wrap this up before I go on a rant about how Rotten Tomatoes is killing useful movie reviews.

Thanks for joining me on this flashback WAY back to last month. If you have any thoughts on RPG reviews, or pop culture reviews in general, as well as suggestions for what you would like to see in the future, please chime in!