## Evaluating Practice Tests, Part Deux: A Case Study

Any given GMAT score for an individual is really a specific value that can (and should) be seen as part of a range. What this means is that when you “think about” or “talk about” GMAT scores, you should do so in the context of a GMAT score range. To better understand what we’re talking about, consider that GMAC indicates the standard margin of error for the GMAT is 40 points, meaning that a person with a certain defined ability will score within 40 points of that ability from one test to the next (assuming no additional preparation between tests).

For more, see the following link:

http://www.gmac.com/gmac/TheGMAT/GMATScores/HowtoUseGMATScores.htm

A similar situation generally presents for practice tests of any given type, or those of any given company. If you take the same testing algorithm from the same company (to balance for algorithm and question-style issues), your scores with no prep in between will most likely vary by 30 to 50 points.

Given this information, it makes little sense to consider scores on tests – both official and practice – on an individual basis. To better understand how we should be assessing our practice and official results, let’s take a look at the test results for a single person (these are real scores generated by one of our students and shared with us).

Here’s some context on how she’d originally been interpreting her results: she’d taken a course and felt good about her progress on practice tests, but scored “lower” on the real thing. After taking a break and resuming her preparation, she feared her scores were “consistently going down” during this time. She also wondered what her next steps should be to regain upward momentum with her scores.

Here are her results:

1) Initial Practice during a course (Comp X)

 Test Date Total Score Quant / %-ile Verbal / %-ile Comp X 1 10/24/09 640 37 / 52% 40 / 91% Comp X 2 11/8/09 700 41 / 64% 44 / 97% Comp X 3 11/14/09 690 46 / 79% 38 / 85%

2) Official Test after the course

 Test Date Total Score Quant / %-ile Verbal / %-ile Official GMAT 11/21/09 640 44 / 70% 33 / 66%

3) Various Practice Tests during self-study (after a 1-month break)

 Test Date Total Score Quant / %-ile Verbal / %-ile Comp X 4 2/15/10 720 44 / 78% 45 / 99% Comp Y 1 2/20/10 620 42 / 66% 35 / 77% Comp X 5 2/27/10 690 44 / 73% 39 / 89% GMAT Prep (repeat) 3/13/10 720 47 / NA 40 / NA Comp Z 3/21/10 600 42 31 Comp Y 2 4/8/10 570 35 / 42% 35 / 72%

Looking at these results, one can see how our student (we’ll call her Ms. Perplexed), could arrive at the conclusion that her scores are going down, and how she’d be confused as to how this might happen and what it means for her preparation going forward.

But … when we take a closer look at the data we see that (barring the one precipitous drop in Verbal on the real test – see point #5 of Evaluating PT, Part 1) her scores have all hovered in an unremarkable range (about 60 points). And when we account for the differing algorithms from test author to test author you actually find that the scores are in a fairly tight range.

To better see this lets dis-aggregate the data:

Comp X

 Test Source Date Total Score Quant Verbal Comp X (pre-break) 10/24/2009 640 37 40 Comp X (pre) 11/8/2009 700 41 44 Comp X (pre) 11/14/2009 690 46 38 Mean 677 41 41 Range 60 points 9 points 6 points Comp X (post-break) 2/15/2010 720 44 45 Comp X (post) 2/27/2010 690 44 39 Mean 705 44 42 Range 30 points 0 points 6 points

So since October Ms. Perplexed’s Quant score on Comp X tests ranged from 37 to 44. She seems to have made progress in terms of moving from a high-30/low-40 ability level to a solid mid-40 ability level and stagnated there. Looking at the Quant and Verbal changes it seems her Quant score has balanced out and stabilized around a 44, so unless she does something to improve her understanding her Quant score is going to remain within a few points of the 44. Her Verbal score, on the other hand, has yet to settle in. Her score range in Verbal is still pretty broad and still has the same basic range from 38 to 45, so she increased the top end of her Verbal range but not yet increased the bottom end.

The observations above seem to have been reflected in her real GMAT scores as well, and has held true in her current practice, except for on the Comp Y test, in which her score range seems to be depressed by about 5 points. So it might just be a case of Comp Y giving “hard” tests or “bad” scores.

Let’s take a closer look at the Comp Y scores to better understand them:

 Test Source Date Total Score Quant Verbal S Comp Y 2/20/10 620 42 35 Comp Y 4/08/10 570 35 35 Mean 595 38.5 35 Range 50 points 7 points 0 points

Again, her overall range is pretty narrow, however her fluctuation here is in the Quant while the Verbal is stable. We can’t be 100% sure what the root causes of this would be, but the immediate thing that comes to mind is there’s something inherent in the Comp Y tests that might have contributed significantly to the lower scores.

Let’s tackle the GMATPrep Practice test too while we’re at it:

 Test Source Date Total Score Quant Sub score Verbal Sub score GMATPrep (repeat) 3/13/10 720 47 40

When you consider the GMATPrep results (assuming repeats on the GMATPrep skewed the score to the higher end of her range), it seems to align more with the Comp X tests rather than the Comp Y tests and seems to support the contention that her Quant score is currently about a 44 and her Verbal is still a hotbed of inconsistency, ranging from 38 to 45.

As for the random practice test score:

 Test Source Date Total Score Quant Sub score Verbal Sub score Practice 3/21/10 600 42 31

This test seems to align with the Comp Y test, both in terms of its depressed results and in terms of its uncertainty with regards to the reliability of its questions and algorithm. But if we align it with Comp Y it simply confirms previous observations.

So, the bottom line:

Ms. Perplexed’s scores are not going down. Her Quant scores seem to have moved up from where she started in November, and have settled into a new range (40 to 47) while her Verbal scores have increased in range (35 to 45). While this Verbal increase is good at the top end of her range, it is mostly likely frustrating that the bottom end has not risen. And this is where, in part, she needs to look if she’s to maximize her score.

So, how does Ms. Perplexed go about maximizing her score after coming to a much clearer understanding of her score? Here are a few unadulterated thoughts …

Given her generally high score level and her fluctuations in the subsections, her best bet to improve is to work with a tutor who has the experience and knowledge to help her develop process that will help her do a couple things: 1) move her Quant into a new range (hopefully establishing 44 as the low end and 52 as the high), and 2) stabilize her Verbal (making the range smaller and higher).

To accomplish those two things she’ll likely need systematic approaches to Verbal rather than just doing what feels or sounds right. She needs to learn all rules, formulas, and approaches, because to move into the stratosphere of GMAT scores requires a mastery of content/rules and sophistication of approach. Simply having an awareness of the content/rules and a haphazard approach is not enough.

So, we hope this sheds a bit more light on how you should be going about analyzing and evaluating your practice tests.

To recap:

1) Individual scores aren’t enough, by themselves to accurately understand progress or ability.

2) All scores should be considered in the context of ranges.

3) The GMAT (and most practice tests) operate within ranges for any given individual.

4) Don’t freak out just because the scores are a) fluctuating, or b) seem to be going down. Do a more thorough assessment.

5) Base your preparation on a fuller understanding of your circumstances. In Ms. Perplexed’s case, she probably will need a skilled and experienced tutor to get her where she needs to be. For others in other score ranges there may be a need for alternative modes of instruction, guidance, and preparation.

Until next time …

The BC Team

• (646) 414-1586
CONNECT WITH US