Evaluating Practice Tests, Part Deux: A Case Study


Any given GMAT score for an individual is really a specific value that can (and should) be seen as part of a range. What this means is that when you “think about” or “talk about” GMAT scores, you should do so in the context of a GMAT score range. To better understand what we’re talking about, consider that GMAC indicates the standard margin of error for the GMAT is 40 points, meaning that a person with a certain defined ability will score within 40 points of that ability from one test to the next (assuming no additional preparation between tests).

For more, see the following link:

http://www.gmac.com/gmac/TheGMAT/GMATScores/HowtoUseGMATScores.htm

A similar situation generally presents for practice tests of any given type, or those of any given company. If you take the same testing algorithm from the same company (to balance for algorithm and question-style issues), your scores with no prep in between will most likely vary by 30 to 50 points.

Given this information, it makes little sense to consider scores on tests – both official and practice – on an individual basis. To better understand how we should be assessing our practice and official results, let’s take a look at the test results for a single person (these are real scores generated by one of our students and shared with us).

Here’s some context on how she’d originally been interpreting her results: she’d taken a course and felt good about her progress on practice tests, but scored “lower” on the real thing. After taking a break and resuming her preparation, she feared her scores were “consistently going down” during this time. She also wondered what her next steps should be to regain upward momentum with her scores.

Here are her results:

1) Initial Practice during a course (Comp X)

Test

Date

Total Score

Quant / %-ile

Verbal / %-ile

Comp X 1

10/24/09

640

37 / 52%

40 / 91%

Comp X 2

11/8/09

700

41 / 64%

44 / 97%

Comp X 3

11/14/09

690

46 / 79%

38 / 85%

2) Official Test after the course

Test

Date

Total Score

Quant / %-ile

Verbal / %-ile

Official GMAT

11/21/09

640

44 / 70%

33 / 66%

3) Various Practice Tests during self-study (after a 1-month break)

Test

Date

Total Score

Quant / %-ile

Verbal / %-ile

Comp X 4

2/15/10

720

44 / 78%

45 / 99%

Comp Y 1

2/20/10

620

42 / 66%

35 / 77%

Comp X 5

2/27/10

690

44 / 73%

39 / 89%

GMAT Prep (repeat)

3/13/10

720

47 / NA

40 / NA

Comp Z

3/21/10

600

42

31

Comp Y 2

4/8/10

570

35 / 42%

35 / 72%

Looking at these results, one can see how our student (we’ll call her Ms. Perplexed), could arrive at the conclusion that her scores are going down, and how she’d be confused as to how this might happen and what it means for her preparation going forward.

But … when we take a closer look at the data we see that (barring the one precipitous drop in Verbal on the real test – see point #5 of Evaluating PT, Part 1) her scores have all hovered in an unremarkable range (about 60 points). And when we account for the differing algorithms from test author to test author you actually find that the scores are in a fairly tight range.

To better see this lets dis-aggregate the data:

Comp X

Test Source

Date

Total Score

Quant

Verbal

Comp X (pre-break)

10/24/2009

640

37

40

Comp X (pre)

11/8/2009

700

41

44

Comp X (pre)

11/14/2009

690

46

38

Mean

677

41

41

Range

60 points

9 points

6 points

Comp X (post-break)

2/15/2010

720

44

45

Comp X (post)

2/27/2010

690

44

39

Mean

705

44

42

Range

30 points

0 points

6 points

So since October Ms. Perplexed’s Quant score on Comp X tests ranged from 37 to 44. She seems to have made progress in terms of moving from a high-30/low-40 ability level to a solid mid-40 ability level and stagnated there. Looking at the Quant and Verbal changes it seems her Quant score has balanced out and stabilized around a 44, so unless she does something to improve her understanding her Quant score is going to remain within a few points of the 44. Her Verbal score, on the other hand, has yet to settle in. Her score range in Verbal is still pretty broad and still has the same basic range from 38 to 45, so she increased the top end of her Verbal range but not yet increased the bottom end.

The observations above seem to have been reflected in her real GMAT scores as well, and has held true in her current practice, except for on the Comp Y test, in which her score range seems to be depressed by about 5 points. So it might just be a case of Comp Y giving “hard” tests or “bad” scores.

Let’s take a closer look at the Comp Y scores to better understand them:

Test Source

Date

Total Score

Quant

Verbal S

Comp Y

2/20/10

620

42

35

Comp Y

4/08/10

570

35

35

Mean

595

38.5

35

Range

50 points

7 points

0 points

Again, her overall range is pretty narrow, however her fluctuation here is in the Quant while the Verbal is stable. We can’t be 100% sure what the root causes of this would be, but the immediate thing that comes to mind is there’s something inherent in the Comp Y tests that might have contributed significantly to the lower scores.

Let’s tackle the GMATPrep Practice test too while we’re at it:

Test Source

Date

Total Score

Quant Sub score

Verbal Sub score

GMATPrep (repeat)

3/13/10

720

47

40

When you consider the GMATPrep results (assuming repeats on the GMATPrep skewed the score to the higher end of her range), it seems to align more with the Comp X tests rather than the Comp Y tests and seems to support the contention that her Quant score is currently about a 44 and her Verbal is still a hotbed of inconsistency, ranging from 38 to 45.

As for the random practice test score:

Test Source

Date

Total Score

Quant Sub score

Verbal Sub score

Practice

3/21/10

600

42

31

This test seems to align with the Comp Y test, both in terms of its depressed results and in terms of its uncertainty with regards to the reliability of its questions and algorithm. But if we align it with Comp Y it simply confirms previous observations.

So, the bottom line:

Ms. Perplexed’s scores are not going down. Her Quant scores seem to have moved up from where she started in November, and have settled into a new range (40 to 47) while her Verbal scores have increased in range (35 to 45). While this Verbal increase is good at the top end of her range, it is mostly likely frustrating that the bottom end has not risen. And this is where, in part, she needs to look if she’s to maximize her score.

So, how does Ms. Perplexed go about maximizing her score after coming to a much clearer understanding of her score? Here are a few unadulterated thoughts …

Given her generally high score level and her fluctuations in the subsections, her best bet to improve is to work with a tutor who has the experience and knowledge to help her develop process that will help her do a couple things: 1) move her Quant into a new range (hopefully establishing 44 as the low end and 52 as the high), and 2) stabilize her Verbal (making the range smaller and higher).

To accomplish those two things she’ll likely need systematic approaches to Verbal rather than just doing what feels or sounds right. She needs to learn all rules, formulas, and approaches, because to move into the stratosphere of GMAT scores requires a mastery of content/rules and sophistication of approach. Simply having an awareness of the content/rules and a haphazard approach is not enough.

So, we hope this sheds a bit more light on how you should be going about analyzing and evaluating your practice tests.

To recap:

1) Individual scores aren’t enough, by themselves to accurately understand progress or ability.

2) All scores should be considered in the context of ranges.

3) The GMAT (and most practice tests) operate within ranges for any given individual.

4) Don’t freak out just because the scores are a) fluctuating, or b) seem to be going down. Do a more thorough assessment.

5) Base your preparation on a fuller understanding of your circumstances. In Ms. Perplexed’s case, she probably will need a skilled and experienced tutor to get her where she needs to be. For others in other score ranges there may be a need for alternative modes of instruction, guidance, and preparation.

Until next time …

The BC Team

  • (877) 223-3828
CONNECT WITH US
COPYRIGHT ©2002 - 2014 BELL CURVES LLC. ALL RIGHTS RESERVED. All tests names are registered trademarks of the respective testing companies, which do not endorse and are not affiliated with Bell Curves.
BELL CURVES - 25 West 36th Street Street, 8th Floor - New York, NY 10018 Bell Curves is an educational services and test preparation company. It delivers high-quality consulting services, test preparation programs, and self-study resources to students throughout the country.
Equal Opportunity Employer - Privacy Policy - Refund Policy