The two main determiners are length and rules density.
If the manuscript is short, the economics of it are such that we must keep all parts of the process short. The final work will fill relatively few pages, and the low price we can charge for that means that we have to be speedy about getting it from first draft to published product. This forces us to use fewer reviewers and give them less time. This in turn pushes the review toward professional work; a "crowd-sourcing" approach isn't really compatible with "fast and systematic." Experienced writers have a better grasp of deadlines and review processes in general, and thus are a better match to the realities imposed by filthy money.
If the manuscript is more qualitative (adventure seeds, genre analysis, GM advice, setting description, etc.) than quantitative (equipment tables, new rules, NPC character sheets, templates, etc.), then it's a candidate for peer review. This is because the main work needed is a critique of the writing – and for that, published writers are more qualified than most playtesters. Whereas if crunchy stuff dominates, it's important to have many people check the math (a task at which gamers excel) and trying the rules in play (which is the very definition of "playtest").
Which means that playtests are more for long, rules-filled items (like GURPS Martial Arts and GURPS Powers) that need lots of math checked and lengthy trials in actual play, and that represent so much auctorial and editorial work that, alongside those time budgets, the time needed for a full-fledged playtest is justifiable.
I know that at least a few playtesters will read this explanation and be insulted that we don't want them critiquing writing. So it goes. The fact is that this was never the role of playtesters, even if some appointed themselves to the task. Since before my time on staff, SJ Games' playtest instructions have spelled out that this isn't really the playtesters' job. What we really want and mostly need from playtesters is actual playtesting and stats-checking; therefore, if we think a work doesn't lend itself to the first and/or doesn't require the second, we're liable to prioritize critiquing the writing, at which point writing professionals are the more qualified candidates.
Edited to Add: In summary, it comes down to whether we're more concerned about writing or game play. Logic dictates that we have writers assess the first and gamers look at the second.
no subject
If the manuscript is short, the economics of it are such that we must keep all parts of the process short. The final work will fill relatively few pages, and the low price we can charge for that means that we have to be speedy about getting it from first draft to published product. This forces us to use fewer reviewers and give them less time. This in turn pushes the review toward professional work; a "crowd-sourcing" approach isn't really compatible with "fast and systematic." Experienced writers have a better grasp of deadlines and review processes in general, and thus are a better match to the realities imposed by filthy money.
If the manuscript is more qualitative (adventure seeds, genre analysis, GM advice, setting description, etc.) than quantitative (equipment tables, new rules, NPC character sheets, templates, etc.), then it's a candidate for peer review. This is because the main work needed is a critique of the writing – and for that, published writers are more qualified than most playtesters. Whereas if crunchy stuff dominates, it's important to have many people check the math (a task at which gamers excel) and trying the rules in play (which is the very definition of "playtest").
Which means that playtests are more for long, rules-filled items (like GURPS Martial Arts and GURPS Powers) that need lots of math checked and lengthy trials in actual play, and that represent so much auctorial and editorial work that, alongside those time budgets, the time needed for a full-fledged playtest is justifiable.
I know that at least a few playtesters will read this explanation and be insulted that we don't want them critiquing writing. So it goes. The fact is that this was never the role of playtesters, even if some appointed themselves to the task. Since before my time on staff, SJ Games' playtest instructions have spelled out that this isn't really the playtesters' job. What we really want and mostly need from playtesters is actual playtesting and stats-checking; therefore, if we think a work doesn't lend itself to the first and/or doesn't require the second, we're liable to prioritize critiquing the writing, at which point writing professionals are the more qualified candidates.
Edited to Add: In summary, it comes down to whether we're more concerned about writing or game play. Logic dictates that we have writers assess the first and gamers look at the second.