AC's Top 1000: The Secret Criteria Revealed!

Ever since this year's People's Media Awards, AC's Content Producers have been recognized, congratulated, surprised, perplexed, and downright confused! In an unsurprising turn of events, AC failed to explain the qualifications for being awarded the much coveted "Top 1000" gold star badge.

Well, after much brain strain and lots of research, I have uncovered the secret! For those who can't wait to find out what it is, go ahead and skip to the last page. For the more inquisitive types, follow me through a series of deductive reasoning while we discover together the deepest, darkest secrets that AC never wanted us to know!

I was first made aware of people's perplexity over this issue in Kim Linton's article, The AC Top 1000 - What Criteria is Used to Determine the Winners? Kim got me started in this process of trying to systematically determine AC's criteria. There's also a lengthy forum thread in which many, many of AC's Top 1000 winners are similarly confused and many possible criteria are thrown around and summarily rejected. The thread is certainly entertaining enough and valid questions have been presented, but none of AC's staff have yet graced us with a reply to our pondering.

At first glance, the "Top 1000" award looks simple enough. AC originally said the winners "will be determined by total content views generated in 2007." It seems to hold up too, since plenty of the people who won the award have well over 100,000 total content views. Then again, under a little more searching, we find CP's like this one and this one who obviously have plenty of pageviews and they're somehow lacking their much-deserved badge. Some CPs don't have all that many pageviews, but they still won the award while some CPs with a few thousand more pageviews didn't.

At the same time, in AC's official PMA article, they then said, "Content Producers whose submissions finished in 2007 within the Top 1,000 'Top-Performing' pieces of content will be recognized on their Content Producer profile pages in the form of special 'PMA' marks." This led many (including myself) to speculate that the award was given to CP's who had received the highest numbers of pageviews on a single article in the year 2007. The problem with this idea is that individual article information isn't publicly available. The only way to find out is to hear directly from AC's content producers themselves.

So that's what I did. I sent out over 75 surveys to some of AC's most illustrious CP's asking a wide range of questions including the most likely (and some unlikely) determination criteria. My curiosity got the best of me and I spent the better part of my weekend compiling results, building spreadsheets, adding numbers, calculating figures, and sending out even more questions to try to get to the bottom of this. Thankfully, with responses from 50 different CP's, who are always ready to help, I feel like I've arrived at an answer.

Since I was giving my best effort, I also sent an e-mail to AC's admin address and several private messages to AC's employees. I hoped beyond hope for a response, and what do you know! - This very morning, after having already decided on the conclusion, I received the following very helpful message from AC's Michael Street completely clarifying the criteria: "I'm not sure why this is a big issue. The Top 1000 doesn't effect your offers, or PV's on AC. It's based on several factors and calculated via our internal reporting tools."

As thrilled as I was to receive a personal reply from Michael Street putting an end to my long-sought quest and revealing the source of all truth, it didn't take me long to realize that his response actually told me even less than AC's publicized responses. Well, it did tell us that AC's "Top 1000" weren't hand picked - it was the "internal reporting tools" that made the selection; no surprise there. Any mistakes can still be blamed on system glitches and that sort of thing. We're also told "it's based on several factors," so it seems we'll have to look for more than one criteria. Don't you love treasure hunts!

So I went back to my compiled data from so many of AC's very helpful Content Producers. I tried to eliminate possible criteria one by one. This investigation was as thorough as possible, so bear with me and see if we agree that there could be no other conclusion.

1. I first considered whether or not it was based on the top 1000 articles with the highest number of pageviews. I asked CPs what the highest pageviews they had for a single article was and how many articles did they have with pageviews over 3000. The results were vast and varied: one badge-winning CP had over 250,000 pageviews for a single article. At the same time, there were quite a few other winning CPs who barely had over 1000 pageviews for a single article.

While most CP's had at least one and in many cases five or more articles with well over 3000 pageviews, the fact that there were several who barely broke 1000 was a bad sign for this criteria. Several CPs who didn't win had well over 2000 pageviews for a single article and one who didn't win had over 5000 pageviews (exclusively in 2007).

Possible criteria #1 was busted. This wasn't the answer.

2. What if everyone who was showcased on AC's front page received the award? In 365 days, AC featured plenty of CPs and if AC felt articles were good enough to showcase, maybe they also felt those CPs were in their "Top 1000."

The survey results started off very good. Almost all the initial responses from winning CP's had been featured on AC's front page at least once. One winning CP had even been showcased a good 45 times or more! I also asked if they've ever been featured on AC's category pages - maybe that was enough. Again, most CP's were featured on category pages many more times than they were showcased on the main page, including some winning CPs who were featured on category pages but not the main page.

However, at least 4 winning CPs had never been featured anywhere on AC, and it wasn't like they had some other factor particularly working for them either. Not only that, but several CPs who didn't get the much-coveted "Top 1000" badge had also been featured on AC's home page, several times or more, and plenty of times on AC's category pages.

So, criteria #2 was also busted. In all honesty, AC probably doesn't keep any track of who they've showcased, just like they never inform CP's when they pick their articles to be featured.

3. Some people think it may be enough to publish in a broad range of categories on AC. So, I asked that question, too. It turns out that nearly everyone on AC publishes in a broad range of categories, winner or not. However, one winner wrote exclusively for one single category on AC. On top of that, I couldn't find any non-winners (who might've won for other reasons) who didn't also write for a broad range of categories.

I consider #3 busted. It seems too arbitrary. Anyone could do it and most people do, but plenty of people didn't win the "Top 1000" award.

4. What about AC's previously published $50 exceptional article of the day recipients? AC used to list in its official info that the most exceptional article of the day would often be rewarded with a $50 upfront payment. Since that would sound like a "top-performing CP", I decided to ask about that also. Out of ALL the "Top 1000" winners (and everyone else) who responded, there was only one CP who'd ever received the $50 exceptional article bonus. Apparently this never actually happened daily; either that or the vast majority of the "Top 1000" winner's articles weren't exceptional enough to receive it. For those of you who are interested, AC apparently took the "exceptional article of the day" possibility out of their FAQs. Now $20 is apparently the most a CP can make.

Possible criteria #4 is definitely busted. But the results were still interesting.

5. AC seems to be placing new emphasis on how many favorited by a CP has. I checked into this, but again, there was no consistency. Some winners were favorited by hundreds, but some winners were only favorited by 1 or 2. Some non-winnners were favorited by more than 50 or 60.

There's no way it could be criteria #5.

6. It could be based on clout rating. Surely those planets and rings are good for something, right? Well, everyone I've seen with a clout of 8, 9, or 10 have the "Top 1000" gold star badge. (That is except for this clout 9 and this clout 10 who shouldn't be disqualified for any of the criteria categories). For a while, everyone speculated it was clout 7 and up since so many people with clout 7 and no one with clout 6 was showing up with the award.

However, there are at least 5 or more CPs with a clout 7 and no award. I double-checked with these CPs and at least one of them also had clout 7 before the end of 2007. On top of that, there are also at least 2 to 3 CPs who currently have Clout 6 and yet they have the award. Nonetheless, most CPs who have clout 6 don't have the award and most CPs who have clout 7 do have it. We may be getting somewhere. I decided to check for actual clout results (regardless of the range they fell in) by following AC's formula. Unfortunately, the ones that caused the most discrepancies stayed that way. In at least two cases, a higher clout at the end of 2007 didn't win the award while a few lower current clouts still did.

Criteria #6, clout, is certainly not the only factor but the near-regularity of its results suggests that a related factor may be the true criteria.

So we're back to overall page views, or some factor of it. Most of this info can be found simply by looking at a CP's profile page, without even requesting the information from them. Here, again, results varied quite a bit, but there was certainly more regularity when comparing overall page views. But let me get back to that.

7. I meddled with average page views per article the whole time to see if this, should-be-important factor really played in at all. My conclusion: NO, not at all. The highest I found from a winner was an average over 3000, and that's definitely very impressive. At the same time, most people were between 200-400 average pageviews per article. Some winners were even as low as 150 average pageviews per article. Some non-winners had well over 500 average pageviews per article, which is well above the average among the "Top 1000" winner respondents, yet they still didn't win.

So #7 is definitely busted; it has very little, and probably nothing, to do with the "Top 1000" badge.

8. So we're back to overall pageviews, and this is where we'll find the answer. Obviously the winners with the highest pageviews are CPs with over a million. No one would doubt they're in the "Top 1000." So I found looking to the lowest to be more telling. The lowest number of current PVs I've found for a CP with a "Top 1000" badge is about 29,000. Quite a few other CPs with right at 30,000, 31,000 etc. pageviews also have the badge. However, a few CPs have 35,000 and more and didn't win the badge. I didn't find any current CPs with over 40,000 and no badge (besides the two continual exceptions), so I figured I might find my answer by looking at the winners with the lowest total PVs and the non-winners with the highest total PVs.

One thing I found (through personal responses) is that some people with seemingly high pageviews and a winning "Top 1000" gold star badge actually were very inactive in 2007. Though they obviously received a few pageviews in 2007, it may have been anywhere as low as 10,000 pageviews for the whole year. 10,000 pageviews in a single year obviously wasn't enough to win since many CPs had over 10,000 pageviews in 2007 and they didn't win. Nonetheless, those same CPs who were inactive in 2007 did have 25,000 or more pageviews from 2006 and prior.

This led me to the very clear conclusion that "Top 1000" in 2007 wasn't calculated only with 2007 pageviews. It also included everything before 2007. Though this was certainly not the wording that AC used to describe the award, it makes sense they did it this way. After all, AC probably only keeps updated data for the most part. The probably don't even have any of their CPs' stats from just 2006 and before and thus they'd have no way of only calculating the pageviews that were just for 2007.

Another thing I've found, that makes perfect sense and does fits AC's wording, is that the award criteria didn't include any pageviews that were received after December 31^st, 2007. And this is where we'll find the solution to (almost) all of our discrepancies.

I originally thought the minimum PVs required for the award was 30,000. That was before I found the winning CP with 29,000 current PVs and before I started questioning both the winning CPs in the lower 30,000 range as well as the non-winning CPs who had significantly more than 30,000 current PVs about how many PVs they had at the end of 2007. I had to move away from my 30,000 figure, but the results were consistent.

Believe it or not, some of the non-winning CPs with over 35,000 current PVs received 10,000 or more of them just in the last month. They've been working hard and they're growing fast and almost certainly will win "Top 1000" in next year's PMAs. But none of them had more (or possibly just barely more) than 25,000 pageviews at the end of 2007. On the other hand, out of all the "Top 1000" badge winners, the lowest at the end of 2007 still had almost 26,000 pageviews... none had less than 25,000. This avenue of search provided remarkable consistency in results compared to all other possible criteria.

CONLCUSION: What I found is that the entire award was based off a CP having a minimum of between 25,500 and 26,000 total pageviews at the end of 2007. Michael Street said it was based on "several factors," but if they somehow weighted different things like clout, average pageviews per article, or other things like that and then added those points to their total pageviews, it turns out the weighted values might equal one or two points maximum - but definitely not enough to affect the outcome. Based on the data, I'm pretty sure there are no weighted values and instead it was all based on one factor - total pageviews at the end of 2007.

This conclusion seems certain because there are several CPs who won the badge who have lower pageview averages, no featured articles on the main page or category pages, never won the $50 bonus and aren't favorited by very many people. The only criteria they fit is that they had over 25,500 page views at the end of 2007.

The only discrepancy left are this clout 9 CP and this clout 10 CP (and possibly a few others who I haven't run into yet). Why were these overlooked? Well, it's probably just another glitch in AC's software management - that happens often enough; the computer just happened to overlook them. Hopefully AC will personally look into it and award them their much-deserved badge.

So now we know. It really wasn't complicated at all. Sure, we can only see current pageviews on a person's profile page, but with enough investigative work, I've determined that the only criteria was the total pageviews that a CP had on either December 31^st, 2007 or Jan. 1^st, 2008 (who knows if there's any difference anyway). So why the secrecy on AC's part, why not just tell us? Who knows, but I think we're all used to it by now. I hope most of you can rest easy now with at least one secret revealed (and maybe another one or two in the process). I know my curiosity's finally satisfied.

Thanks again for all the help from all of our immensely wonderful and very helpful CP's here at AC! Congratulations to all the winners of 2007's PMA awards! I'm looking forward to many more of you being winners next year when AC completely changes all the criteria around again!