UPDATE: We’ve posted a public statement about this post here.
Hello there.
Way back in December we posted an update saying that we would post “about our terrible IGF experience.” Granted, we may have exaggerated a few things back then. (read: a lot of things!) But with the IGF finals at GDC coming up, there is no better time than the present to relate our experience.
First, some backstory: We are an independent developer producing an iPhone game named Kale In Dinoland:
This year the IGF decided to use TestFlight for its iPhone games. We paid the $100 customary to the festival, as over 500 other developers did for their games. Going in, we didn’t expect to get nominated for anything. The game was content-complete but wasn’t done to the level of polish we wanted. Not to mention, there were still bugs. But this isn’t about the overall quality of our build, it’s about giving every game its fair shake.
Let me explain. When the IGF chose TestFlight for iOS distribution, they made a big mistake. We were given a list of all of our judges’ email addresses, revealing their identities. We aren’t going to release those names to respect the judges, but let’s just say we had a heavy-hitter. For every judge, we could see how much they played; if they even started the game at all. How do we know this?
TestFlight records NSLogs (iPhone version of console logging) and custom “checkpoints,” uploading this data seamlessly for us to see. In addition, when a judge opened the TestFlight invitation email, downloaded and then installed the game on their iDevice, we can see all of that. I believe that the IGF organizers, who are usually lips-sealed on the judging process, did not know about this functionality. We can see exactly when a judge installed the game, when they started playing, how long they played, and how far they got.
As you can imagine, this was an opportunity for us to see what really goes on behind closed doors at the IGF. How much do games really get played? Does hype count for everything? Is it true that to be a contender in the current IGF, your game has to already be widely known in indie circles? Does this mean that most of the judges won’t end up playing your game in these circumstances regardless of the quality of the title?
Here are the statistics:
Eight (8) judges were assigned to Kale In Dinoland. Of those judges, 1 didn’t install the game or respond to any of our invitations (which we had to send multiple times before judges joined). 3 judges didn’t play the game. Of the remaining 5 judges that played the game, 3 played it very close to the IGF deadline, which was December 5th. One judge, our outlier, played the game for 53.2 minutes. Excluding the outlier, on average each judge – including the 3 that didn’t play it – played the game for almost 5 minutes’ time. Back in that build, Kale’s intro cutscene took about a minute’s time. So we’re talking almost 4 minutes for each judge of actual game time.
Granted, they could have deduced the game was absolutely terrible and didn’t deserve their time. About this time, though, we were also running a beta that was being played by anonymous iOS gamers from the community. These helpful gamers were all interested in the game, having seen it on TouchArcade and IndieGames.com. What is the influence of prior marketing? The average play time for these external beta testers was 34 minutes, accounting for that one minute of cutscene time.
So, a large group of anonymous gamers who were not required to play the game averaged about 30 minutes more play time than the the 7 judges who were required to play the game, 3 of whom did not even play the game. Is 4 minutes enough time for someone to give a fair assessment of a 2-hour-long game? How many more games were given similar treatment? Had we not taken initiative and sent multiple emails urging judges to download the game via TestFlight, how many judges would have ended up playing the game? Here’s the last email I sent out, urging the judges to accept the TF invite:
The build has been up for a while now (I sent emails via TF), but only 1 judge has installed, and 4 other judges still haven’t even signed up for TestFlight.
The sad truth is, the heads of the IGF know about all of this. They made the mistake of using TestFlight and allowing us, the developers, to see backstage. Shortly after posting the update that included negative remarks about the IGF – on this relatively unknown blog – we were mysteriously followed on Twitter by @brandonnn and received an email from none other than Simon Carless.
Hey folks,
It’s just been brought to my attention that you believe that you’ve had some issues with your IGF experience and are preparing to blog about it. My name’s Simon Carless and I head up the GDC events, including the IGF – and I’m CC-ing Brandon Boyer, the IGF chairman here.
Before you go ahead and do that, could we have a phonecall discussing your perception of what happened during judging and your impressions of what didn’t run correctly from your perspective? We _do_ actually care about individual entrants such as yourselves, and it upsets us when people don’t feel like we’re doing a good job. So let’s talk about it directly!
Myself and Brandon are available at a few times on Monday – I’m in U.S. pacific time and he’s on U.S. central time. Do you have time for a call?
Thanks,
Simon Carless
EVP, UBM TechWeb Game Network
We considered calling. At the very least, we decided not to post what we would have, and gave it some second thought. Let’s be honest: It is obvious that Brandon Boyer and friends care about the IGF as the prime outlet for indie games. We don’t doubt that. But this isn’t about handling the situation silently. If we had called and talked about our concerns, the heads of the IGF don’t have to be held accountable for a broken judging system. It’s about transparency, which is something the IGF completely lacks.
So there it is, our story about the IGF. We hope that, as a community, we can change the IGF for the better by exposing flaws in the judging system and holding those in power accountable. But until then, please hold off on marking the IGF as the be-all end-all of indie games. Instead, join protests like the IGF Pirate Kart. And if you’re still not convinced there’s something wrong with the IGF judging system, hear it from a judge herself.
The Rotting Cartridge 
We entered a Windows/Linux game and were able to track IPs for the IGF build (we check for updates at launch.) We had ~25 unique IPs access the game during the IGF judging period, and *zero completions*. The first level takes about 15 minutes to beat.
Just letting you know we saw similar less-than-par results from the judges. For our beta, about half the players completed the tutorial — much more than the *zero* that IGF managed.
Wow. And here I was hoping it was just iOS games that had trouble. You should post about it and give details! People need to be aware of this stuff. It really sucks for the developers.
i’ve heard of similar stories as this one and yours — farbs and his captain forever reported similar 5-minute averages i believe. i didn’t track results, but the judge feedback for my entries seemed to indicate they didn’t get past the tutorial / intro either. so this is probably very common
This happened to me too a few years back when IGF was doing mod submissions. It was networked and found out that there wasn’t a single judge that even logged on to play. I got an apology from Simon and a refund for my submission but that was it. I’m disappointed to hear that the IGF is still continuing this practice while claiming to represent a fair process.
A single judge? We should consider ourselves lucky then. At least they refunded your money.
Now that you’ve blogged about it, you should probably also call them and chat about it! It’s cool that they’re making an effort to try and do something about it, so if you have the opportunity to give them a more complete explanation of what happened and why you think it was unfair, you totally should. 🙂
I entered my iOS game “Foozle” into the student competition portion so it was free. My experience was far worse. After pestering the contacts I had at IGF a couple times, about a week before the *extended* student judging deadline I finally got an email asking for promo-codes(mind you, without the extended deadline I wouldn’t have even been asked for promo-codes for the judges prior to the announcement of winners). Of the 10 promo codes they asked for, exactly 1 was actually redeemed. So, in the student portion I got a single play on an iOS game that has over 200 ratings averaging 5 stars in the app store. I didn’t want to be a whiner, but if it’s widespread that’s just not acceptable.
I want to separate out a few different threads here that I think are getting conflated. I wholeheartedly agree that utilizing TestFlight for the IGF this year ended up being a mistake, when I had anticipated that it would streamline what was already a rough process when we integrated mobile games into the IGF for 2011.
I don’t think it was a mistake for the reasons you state, though: implementing the system on a wide scale (hundreds of games x hundreds of judges), and trying to educate a number of newcomers to that system (on both the developer & the judge end), was one of the biggest challenges in organizing the IGF this year, and ended up further complicating the process.
It’s the reason we ended up dropping TestFlight and reverting to our former process of directly distributing ad-hoc builds from a central location after the first round of judging, and for the Student Competition. We’ve had a lot of good feedback from our iOS devs (many of whom had additional problems where they had to open second developer accounts with Apple to accomodate all the extra judges and jurists) on how to better organize ad-hoc distribution in the future that will hopefully go two steps in the better direction of fixing a lot of this.
To your separate point about engagement with your game for the judges that did manage to get it installed, here’s the reality: all of our judges and jurists are professionals who very generously volunteer their time, year after year, to look at an ever-increasing number of entrants, for no reward other than a genuine desire to help foster and grow and reward achievements in the independent game development community.
Some have or make more time to devote to this than others — we had judges in the festival this year who went far beyond their allotted 20-ish games and took it upon themselves to dig into and leave feedback on upward of 80 some odd games, we had others who only could manage to get to their assignments (and yes, in some cases only very close to the deadline!) and we even had others who — due to interference both personal and professional — couldn’t judge all their games.
The IGF, contrary to the most creative of the conspiracy theories, is not a shadowy elite who conspire to promote their own, it is very simply a few hundred game developers, academics and journalists of every stripe, all of whom volunteer for this job solely because they are enthusiastic about independent games, but all of whom also must have to deal with new babies, and crunch-times, and fall ill and can be forgetful and distracted.
This means that there are very basic human-nature type biases at play when people look at their allotted games, and a judge can know quite quickly whether or not they wish to nominate any given game in our categories, especially when — as you admit above — a game might not be finished or polished to the degree that you as the developer want it, let alone expect it to garner a nomination.
For anyone else who might be reading this, this is an incredibly valuable lesson about engagement that extends far beyond the microcosm of the IGF (of several hundred other games demanding attention) and into the marketplace at large (of hundreds of thousands of games). It would behoove you to present the best of what you have to offer — or at least tantalize the player with something worth waiting for — up front and within minutes, or they may simply move on.
In the private feedback left by our judges on this game, many lamented that even though you directly asked that they put three hours into your game, in the time they put in they could already tell that there was little that could come that was going to change their mind on whether or not they wanted to give it a nomination (and, in your own post here, you have said you didn’t even expect that), and nominating or not nominating games is literally what our judges are assigned to do.
On the wider, unspoken point of “how many judges does it take for a game to stand out from the others”: our body of judges is diverse and opinionated enough and the sheer number of entrants so wide that it’s difficult for any game to garner many more than 4 nominations, except in extreme cases (ie. people rushing to nominate already popular games).
These nomination numbers are passed on to our juries in two forms: rote number of nominations, and “number of assigned judges who nominated”, which specifically helps temper “everyone just flocking to an already popular game that they may not be assigned to”. Our juries use both of these numbers as guidelines to consider their own nominations for finalists.
That is to say: if, for whatever reason, only four judges of eight looked at your game, and all four of those judges nominated your game for a certain category, there is enough data there to raise a flag that said game warrants further investigation amongst our juries. On top of this, we generally ask that our juries at least consider looking at any game that gets more than 2-3 nominations for any category.
All of us involved are genuinely sorry that you didn’t have the experience you expected entering the IGF this year. This is why, as you’ll recall, we took the extra time during the judging process to repeatedly contact your assigned judges — to ask if they were having trouble installing your game, and to ask that they contact us if we could help in any way.
This is also why we reached out personally to you to talk about any problems you had and look into solutions for future years, which you openly chose to ignore, and why I’ve taken the past hour or so to help shed light on the situation, both for you, and for anyone else who you’ve drawn to your blog in advance of your game’s release.
Let me know if there’s anything else we can do — I can be reached at chairman@igf.com.
Brandon Boyer
Chairman, IGF
You have no integrity. Also, you seem to be a bit of a massive jerk. Why anyone looks at the IGF as anything other than a sad joke is completely beyond me.
SO BRAVE.
Just for future reference, your insults carry no impact unless you use your real name instead of hiding behind anonymity.
Uhh yeah, nice try dude. IGF is broken as shit. What a corrupt organization.
Hi Brandon,
I’m highly curious about how my game faired in the competition, is it possible for me to contact your or someone else to find out what sort of feedback it garnered? It’s mainly for my curiosity and also because any feedback that will improve my game is always great.
Solid reply to all points Brandon. Good for you.
Also, if a game needs more than 10 minutes of playing to get a feel for what it is, well something is wrong.
Separately, Kale In Dinoland looks like quite a fun game. Congrats to the dev team on their hard work.
I think it should be made clearer to entrants in future that their games may be ignored completely.
Simply saying, that you should look your best and first impressions count really doesn’t make clear that there are judges that feel that the medium of games justifies ignorance unlike critique of any other entertainment medium.
Entrants are paying consumers. Paying consumers purchasing anything else would be entitled by law to know exactly what they are paying for.
I myself did well out of entering, my game was ignored by the IGF but got favourable reviews elsewhere. Just because the IGF ignores you, doesn’t mean you get nothing for entering. But of course I won’t get anything for entering again.
As an aside: The writer of the article as mentioned below by IGF Judge abused their knowledge of who was judging and harassed the judges. Then instead of contacting the IGF they blow up on the internet about it. That’s pretty appalling behaviour. If you’ve got a problem with someone, talk to them, not the internet.
The problem is that it wasn’t 4 Judges, it was 1 judge. 4 minutes isn’t long enough to determine what a game is like.
And 4/8 judges played it. That’s not “some had a commitment issue”, that’s a seriously problem.
I know that the people are professional volunteers, but 1 hour on 20 games is barely a weekend. IGF judging takes place over months, surely they could scrape together just 30 minutes a day for 20 games. The system is broken, and needs to be scrapped and recreated to be better.
5 judges played at least 10 minutes (don’t be confused by his average, which takes into account the people who did not play at all). That’s plenty of time to know that this game isn’t going to be nomination worthy, especially given that the author himself states that the game was buggy and unpolished.
The overall tone of Brandon’s reply seems to be: “You should just be glad we are here and grateful for any time we are able to make to evaluate your submissions.” The entrants put their good faith in the IGF and paid their money with the expectation that they would be treated fairly in the judging process. Volunteers or not, if these stories are true the judges showed an astonishing level of unprofessionalism. The misplaced paternalism of Brandon’s comments is insulting.
Basically this post makes me want to speak with my wallet, which is to say I will do my best to avoid any money that I earn ever ending up in your pockets or the pockets of anyone you support.
This guy paid you money so that his game would be judged and feedback given. What he got was this:
1 judge never installed the game
2 judges installed the game and never played it
4 judges played for an average of 4 minutes gameplay each
1 judge played for about an hour
So for his $95 entry fee, he got about 1 and 1/4 hours out of your judges. Your volunteer judges obviously dropped the ball in this case, and your attempted coverup and then condescending post here I find morally abhorrent. I especially was revolted by the following quote from the e-mail you guys sent to the company: “It’s just been brought to my attention that you believe that you’ve had some issues with your IGF experience” — they “believe” they had some issues? That wording essentially calls them liars to their faces, and you are upset that they didn’t keep your dirty secret for you?
If a judge agrees to judge games, and gets assigned 20 games, and has 3-4 months to fulfill the requirements, I do not think it’s too much to ask that each judge give each of those 20 games enough attention to actually judge it. 15-30 minutes x 20 games comes out to 5-10 hours over the course of MONTHS.
I’m sorry, but less than 5 minutes played from 4 judges, 0 minutes played by two more, completely ignoring the game from another, and only one judge who apparently actually did his job … that’s a problem. Your organization using lines like “believe you’ve had some issues” is disgusting.
It is completely indefensible. What you should have posted was “I’m very sorry that our volunteer judges did not fulfill their obligation with your game and I’d like to make that right. Let’s get in touch and make it right.”
Then you should have transparently addressed the issue on your own site to ensure this kind of crap doesn’t happen in the future.
Instead you tried to pretend like your organization is innocent and above board (when you are clearly not. You are essentially stealing money from indie developers who expect to actually have their games judged while handing out awards and publicity to your friends) and slam this guy for exposing your corruption.
And you know why I have concluded this? Because of all these other developers now coming out of the woodwork talking about how you got them to stay quiet in past years with promises you didn’t keep. When you lie and cheat and steal, eventually it comes to light. I hope you are repaid 10,000 times over for your dishonesty. You are everything that’s wrong with the game industry, in one little microcosm.
I don’t think most of us think you’re deliberately staging a corrupt competition. You just don’t seem to understand what is required for the competition to give everyone a fair shake.
This isn’t about your intentions. I think those are genuinely good. You just need to make a real effort to improve the professionalism and fairness of the judging. Just because your heart is in the right place, that doesn’t mean you’re not accidentally screwing people over.
Hi,
Just a couple of points, (mostly reiterations of what Brandon says above):
1. Your comparison of volunteer beta testers’ vs. assigned judges’ playtimes is a classic example of self-selection bias. Someone who voluntarily seeks out your game is inherently more likely to invest their time and energy into it than others.
2. As an IGF judge, I can say that myself and (according to the comments on the IGF judging website) many other judges discovered many games for the first time during the judging, downloaded and played them. I nominated more than one game that I’d never heard of before the competition began. The system Brandon outlines and discussion between the judges encourages this.
3. To reiterate Brandon’s point: the lesson here is grab the player in the first four minutes. Show them from square one that it’s worth playing half an hour, and use that half hour to show them it’s worth playing another hour, and then through to the end. If your game gets amazing after an unengaging first impression, that’s the real problem. No one is obligated to play your game; you have to prove why you’re worth their time. Every second from bootup counts.
wow, did an igf judge really just say “no one is obligated to play your game”?
brandon, what are you telling these guys? HEY, ASSHOLE. IF YOU ARE AN IGF JUDGE, GUESS WHAT? YOU ARE OBLIGATED TO PLAY THIS PERSON’S GAME. i’m pretty sure they didn’t pay that $95 entry fee just so you’d have something to wipe your ass with.
+1
Ugh, did you actually read what he said? Do you care about the context of his words at all? Come on.
To clarify:
He’s saying judges aren’t obligated to *keep* playing your game if they get bored after the first few minutes. Remember, judges are NOT game reviewers trying to give a rating out of 5, they are merely trying to pick the few *nominations* of the *very best games*.
For many games, it becomes obvious within a couple minutes of play that it’s not a contender for a nomination.
As an IGF judge, didn’t you volunteer to play whatever games you got given (provided you could run them)? If you don’t have the spare time to play a game for more than 5 minutes, then recognise that your priorities lie elsewhere and don’t be a judge.
The “four-minute-rule” point is a good one, but what about the judges who didn’t play their assigned games _at all_? Attempting to redirect blame onto the designers doesn’t work in that case.
“No one is obligated to play your game; you have to prove why you’re worth their time. Every second from bootup counts.”
This statement really saddens me. First, doesn’t the $95 entrance fee obligate the judges to play my game? Second: haven’t you ever played a game that doesn’t quite excite you at first but once you get into it (say 30 mins to an hour) suddenly it clicks and you realize it’s brilliance? It’s a shame that such a game would be quickly discarded if the judge’s attention wasn’t grabbed in the first 4 mins.
I think $95 justifies that my game should have to be played for at least 30 minutes. BTW, I have no idea how many judges played my game or for how long. It just worries me that there’s no way for me to know if it got a fair chance. I guess I’ll just have to wait to see the quality of the feedback I receive.
Why does it sadden you? It’s exactly the kind of thinking that your potential customers are going to employ.
They’re not customers, they’re judges.
“Eight (8) judges were assigned to Kale In Dinoland. Of those judges, 1 didn’t install the game or respond to any of our invitations (which we had to send multiple times before judges joined). 3 judges didn’t play the game.”
Fully half of the judges assigned to the game in question did not play it. One would assume a paying customer would run the game for at least a few minutes.
If we were to assume that the judges were paid (which several have commented on this thread that they aren’t), even then you are are asking a LOT for your $95. A half an hour or MORE per judge per game for that $95, are you KIDDING me?
Let’s do some math here. $95 split over 10 judges = $9.50 per judge per game. If you are asking for a half an hour (or more) of play time per judge plus the amount of time you expect them to write up their comments etc. then you are setting their hourly rate at around $20 per hour. To put it another way, you expect 5 hours OR MORE of work for $95. That level of time for that little money is delusional for most working professionals.
I don’t dispute that the judges should fulfill their obligation to at least give the game a shot OR stop volunteering to be judges, but they are VOLUNTEERING. It sucks that there were judges that didn’t play the games at all. The IGF should kindly deny those judges from returning as judges unless there are mitigating circumstances, and even then if it is a pattern that lasts for more than one year then that judge should stop being a judge.
Beyond that, if your games doesn’t appeal in the first 5 minutes of game play you probably aren’t ever going to appeal to that judge. That *IS* feedback. Would you prefer it was worded like this: “Your game bored me for 5 minutes. I kept playing it for another 25 and it never got better, in fact I think it got worse. Please stop making games and submitting them if this is the best you can do.”
For every game that I’ve “realized it’s brilliance after 30 minutes of play time” there are dozens that have continued to be boring or just gotten worse after not engaging me in the first 5 minutes. I don’t care how long you make me play a game from the Madden franchise… I’m not going to suddenly start liking it.
I can look back and imagine me “reviewing” my first five minutes of Minecraft.
I have to punch trees? You just punch trees? I’m all alone and there is nobody to talk to. What do I do? Oh I can punch the dirt too… woo. Now it is dark, I can’t see anything, I just blew up, Where did all my wood go? Everything is trying to kill me and I just have to punch them with wood? I died again. This game is boring. -End- Five minute review of one of the best indie games.
Notch really should have learned and started you with a full set of diamond tools fighting a dragon, you know… to hook me, let me know the game is worth playing.
Minecraft is NOT for everyone and if you don’t like it, that doesn’t mean you don’t like games / don’t like indie games. Minecraft is a terrible GAME, but an excellent WORLD BUILDER. If you try to sell it as a game to the mass market you are going to fail because that is not what most people think of when the think of games.
Minecraft really is what you make it. There are world rules and you can do pretty much anything you want inside of those world rules but there isn’t much story or plot or pacing or anything that you don’t bring into the game. You don’t “Win” at Minecraft, you construct what you want out of the tools that were given you. That appeals to certain people but not everyone.
To me, comparing Minecraft to a mass market appeal game is like comparing D&D to WoW. There are a TON of people who play WoW that would never pick up a D&D game book and try to play because the GAME of WoW is what draws them in, not the rules and WORLD BUILDING factor. They want the pre-canned quests and epic loot drops. They don’t want to have to make up the story. They want escapism and someone else to do the work of making the goals and missions and quests and reward items. They don’t want to spend hours punching trees and dirt to make a game for themselves. Does that mean they are wrong?
I think this all boils down to what you want to do. Do you want to makes games that make you money, or do you want to make games that you and your friends like and play. If you want to make money you had BETTER hook people in the first 5 minutes or less or be willing to spend a LOT of money on advertising (or get REALLY lucky with word of mouth a-la Minecraft). Have you ever heard the expression “you only get one chance to make a first impression?” that’s INCREDIBLY true in sales.
If you don’t want to make money, why are you spending $95 to enter the IGF? Just make the game and share it with whoever you want who wants to give it a try. If it is about the craft and the process, then make the game even if nobody plays it. If it is about exposure of your game, give it away. Ask John Scalzi or the guys at Penny-Arcade how well giving things away for free works out when the product is something that hooks people and draws them in. If your product is worth supporting, it will be supported.
The judges agreed to install and try the game. If they didn’t do that, then you have a legitimate complaint… unless the installer didn’t work, the game crashed as soon as they tried to launch it, they couldn’t figure out what version of the game they were supposed to download and install, etc. all of which happen ALL THE TIME to these judges.
But if they tried the game and decided in the first 5 minutes that it wasn’t for them, how long do you expect them to keep grinding for free? They volunteered to give your game a shot. If they did and didn’t like what they saw, that’s feedback. “Make the game more compelling quicker” is what they are telling you. You can do with that what you like.
Sorry guys, wasn’t clear in my reply. Obviously an IGF judge is obligated to play your game. My mistake in my phrasing. What I MEANT is, as a game developer, in general, out in the wild, you have to go forward in the MINDSET that no player owes you anything, and you have to hook them.
Forgive my lack of precision in explaining myself.
Let me see if I have this straight, and please clarify if I’m wrong:
Your comment was NOT meant to be in response to a post criticizing the process of judging games for an indie games festival which costs entrants $95.
Instead, it was actually meant to be a message toward the theoretical game developer who is trying to convince consumers to pay THEM for their product?
If that’s the case, why would you even make the comment in the first place in the context of this conversation? Not to mention you said you were “reiterating Brandon’s point.”
Was Brandon ALSO not speaking about the process of judging games for an indie festival, in response to a post about judging games for an indie festival?
seems he made a mistake with his phrasing. is it necessary to be so flagrantly pedantic? what do you hope to gain with your line of questioning? from where i’m sitting, it seems like you’re hoping to get out the pitchforks and torches and you’ll take any misstep you can find to do so.
or wait, are you just clarifying? well, then i guess all is forgiven.
These responses from @brandonnn and IGF judges are infuriating. It is *NOT OKAY*–in any universe–to simply not play the game that you are tasked with judging. These people put time and effort into their games, and they paid to have them judged.
If the game was so breathtakingly awful that the judges couldn’t play it for more than 5 minutes–well, fine, but the judges should have given feedback. If the judges were having technical difficulties, then the game developer should absolutely have gotten feedback! The game wasn’t even played!
I hope the IGF stops asking these judges to come back if they refuse to play their assigned games.
“Granted, they could have deduced the game was absolutely terrible and didn’t deserve their time.Granted, they could have deduced the game was absolutely terrible and didn’t deserve their time.”
Not quite so simple, dude. IGF is a competition for obviously brilliant games like Minecraft and Dear Esther…not for games that have an open beta and generate “some initial interest”.
Not that I particularly agree with their selections, just that that’s more or less how it works.
since when
Since they’ve been able to attract the attention of those titles. Since those titles have won prizes. Since its inception as a “best of”-style competition with strong commercial ties. Not everyone is entitled to a “fair” consideration; they are quickly tasked with weeding out the games that won’t actually make it very far upon release or generate lots of good press.
And I salute them for that. If you are entering your released game into IGF, you effectively had your shot for insane commercial success. That was the release of your game. But either you didn’t market well, or your game wasn’t cut out to make much money for whatever reason. It’s not the job of the IGF’s judges to sell peoples’ games for them. They have their own interests at heart and those of the competition they work for.
And yes, you can argue that you spent money and sank time into the game, but that just gives people free reign to question your investment choices, since clearly this one was a flop (unless it isn’t, and you’re happily making a living on this game, in which case that’s awesome, but there’s no reason to complain). Not that there’s anything wrong with failure…nobody ever got anywhere thinking that way.
I was tapped as a judge this year for the first time. It was a really great experience, but it was also really a lot more work than I expected it to be — I bit off more than I could chew.
I signed up for the mobile stuff (I was not assigned your game), but it was a mistake on my part. Dealing with the app store and whatnot is something I hate, and I don’t even have access to my iPod all the time. I barely had time to play the games I could easily install long enough to give a fair assessment of them.
If the majority of the people volunteering to beta test your game are anything like the ones testing mine, then they are mostly kids with tons of spare time. I am already working 60+ hours a week and it is hard to force myself to play a game that has many obviously bad aspects. Game journalists are probably used to trudging through games they are not enjoying, but it was a new experience for me and a lot harder than I thought it would be.
I was actually really impressed by the caliber of the judges and the discussion about many of the entries. One thing that I was really surprised about was the sheer number of really awesome games in the IGF this year. As a person who loves indie games I was overwhelmed.
Is it possible that good games got overlooked because they had bad/forgetful/busy judges? Sure, that seems possible, but it is hardly the fault of the IGF seeing as the judges that are selected are very well qualified for it. If I get asked to be a judge again I will have a lot better idea of how much work is involved, and pace myself better. I was probably not the best judge, but it was my first try at it, and I think I am starting to get the hang of it!
I would bet this is the case with many judges. They accept without know what they are getting into in the first place. Fine print, ladies and gents, fine print.
[…] have guaranteed a fair look. Brandon Boyer, IGF chairman (formerly of OffWorld; soon: VenusPatrol) responds at length in the commentsNote: I’m pretty sure that while the complainant’s website describes their submitted […]
I am a judge.
For the people mentioning the entrance fee, keep in mind that judges are not paid. We volunteer our time to look at games because, as Brandon said, we care about independent games and want to recognize the best of them on the IGF platform. I honestly don’t know where the money goes but I assume it is to fund the people who work full time to organize the program, the awards ceremony, etc.
I do think that judges should be held accountable for playing the games assigned to them. And they should also do their best to provide helpful feedback to developers. I always try to give constructive feedback because that is what I would want if I submitted a game to the IGF.
That said, I do not think it is a problem if a judge plays your game and then quits after 5-10 minutes. Judges are tasked to play many games. If your game doesn’t appeal in the first few minutes to a judge there are other games that will. That’s the reality. You can’t expect someone who is uninterested, for whatever reason, to keep playing through your game. The IGF is a competition. You are competing against what, hundreds of other games? Soon to be thousands? You win by quickly appealing to people, having unique value, and demonstrating that value as soon as you can.
As Brandon said, that’s true in both the IGF and in the larger game industry.
The stuff about having to appeal pretty pronto is generally fairly sound advice but in a case where judges haven’t installed or played the game, that becomes pretty redundant advice because they haven’t even bounced off it due to quality, the judges have simply failed to do their job.
Focusing on the bounce is pretty unfair and skirting around a far more important issue that when people pay $95 to enter a competition, the very least they can ask for is to be judged by a full complement of judges and judged fairly. Which last time I checked requires judges to install the games and play the games.
If that isn’t being achieved, then it’s a problem that needs resolving and shouldn’t be the subject of “look over there” posts or facile design advice.
Yeah, agreed. Like I said – I do think judges should be held accountable for at least installing and playing the games they are assigned, along with ideally giving constructive feedback.
We submitted an iPad game for IGF, called Crabitron. We used up 10 device slots, 7 installed the game, 3 actually played and the total playtime for all judges together was less than 30 minutes. I just assumed that they didn’t have wifi access so the data was incomplete. It kind of sucks to think they just didn’t bother.
I realise now that the IGF judging is highly political with the same names generally getting picked as finalists year after year.
As a note, our game was given positive mention on indiegames.com and praised on twitter by Jim Rossignol, so I think the lack of playtime wasn’t due to it being terrible or anything.
This is hard.
I’ve been on both sides of this. I’ve submitted a lot of games to a lot of festivals and been fortunate to have some of them selected as finalists.
In interests of impartiality, I’ve never submitted to the IGF and never juried it, so this is me commenting on the generalities, not anything about the IGF specifically.
I’ve also been on a lot of juries for games. And like several people have pointed out in this discussion, the juries I’ve been on have always been working professionals or peers who have lots of game-making of their own to do. It is a big commitment with a relatively short window to get them all played. No excuses or apologies, but it’s an ugly reality that you may not have very long to impress a juror.
Some things:
1) some jurors flake or get overwhelmed or have problems with their device (especially iOS devices or gawdhelpyou XNA games that need a depricated version of XNA framework installed to run right) – that’s why so many get assigned to one game. Three didn’t play, but you got a lot of different eyes on your game, its fate wasn’t left to a single person.
2) Brandon made a point that is true across a lot of entertainment mediums. Make sure some of your best stuff is up front, where everyone’s going to see it. If the best part of the song requires three minutes of listening to perfectly fine but bland pop to get to the hooky bridge, that’s a problem, it’s not probably going to be in the iTunes preview. If the coolest scene in your TV show is planned for the season finale, but not in the pilot, maybe move that to the pilot, when the network’s deciding if they’re gonna take it to upfronts. The first line of a poem, the first chapter of a book, they just stand a better chance of being appreciated if there’s something extra special there. You can even see this in E3 demos, regardless of whether those five minutes of demo gameplay represent the depth and awesomeness of the full forty hour game.
3) I actually agree. Both the IGF and the IGS could use some fresh blood. The names we’re maybe getting a little tired of seeing seem to grok how to make an impression. I mostly hope what comes out of this conversation isn’t a debate about whether or not a festival is bent and instead there’s a productive conversation where we all help each other present our great games in the best light we can.
[…] ya tiene su ración de polémica, servida ayer por cortesía de The Rotting Cartridge en su post What’s Wrong with the IGF. The Rotting Cartridge es la desarrolladora de Kale In Dinoland, jueguecito indi para iPhone que […]
It’s interesting to try to compare IGF with say, an indie film festival.
Sundance, for example, charges 75 dollars to admit a feature film (unless you miss the deadline, then it’s 100). After that it enters a highly competitive selection process, where judges generally plow through huge amounts of screeners. I can tell you from having known film festival screeners at SXSW and Sundance, most of the movies are obviously not going to make it, and get played through with minimal attention if they fail to grab the screener in the first few minutes. If something really stands out to a screener, they spend the duration with it, and that’s deep attention.
With a game festival, it’s more difficult to manage your time. You must separate the wheat from the chaff in your selected games, and then focus in on your top picks and spend as much time with them as possible in order to formulate your argument as to why they should be a nominee. You don’t just watch a game.
This game did get played, and there is dialogue that happens between judges on every game. How likely is it that, unable to get the test flight installs to run, the judges who didn’t launch it simply asked the judges who had played it if it was worth their time?
I’ve been a judge two years in a row, and I can tell you that no game EVER is passed over with no judges having played it. If a judge or two flakes on a game, that’s shitty. But NEVER is a game passed over completely.
Laying a carefully timed blog stink bomb to try and generate some publicity is lamer than a judge taking a shortcut and asking another judge their opinion.
“I’ve been a judge two years in a row, and I can tell you that no game EVER is passed over with no judges having played it. If a judge or two flakes on a game, that’s shitty. But NEVER is a game passed over completely.”
ummm… this claim really makes you look like this woman in the video:
GET CAUGHT by the system, and still denying FACTS. Read the post more carefully next time before you say anything, Wiley.
+1 for agreeing with me!
How can you possibly qualify this statement if you’re simply a judge; and only for two years? Never is a very long time, sir.
Only 4 out of 8 judges played it, and of those 4 only 1 played it longer than 5 minutes.
Judges get 20 games each, and even at 12 minutes a game that’s only 4 hours, plus tax. If a judge can’t spare that over several months then why are they judging?
One judge putting the judgement on one game will be too biased. 8 judges are assigned for a reason, to at least give people an unbiased chance.
This happened to me way back as well. We submitted a download link to our PC game, which was hosted on our own server. The link was unique to IGF, and it never got a single hit.
Obviously, we brought this up, only to be completely ignored. Only when this started getting a fair bit of publicity via slashdot did they contact us about it with about the same reply you had. We talked to them, and got an apology but not much else – they offered to send us some stuff. We accepted and gave them a shipping address, deciding it wasn’t worth more of our time (we had a game to work on after all).
So that was the end of the story. And I mean the end of the story, since the part where the stuff they promised actually arrives never happened either.
“I’ve been a judge two years in a row, and I can tell you that no game EVER is passed over with no judges having played it. If a judge or two flakes on a game, that’s shitty. But NEVER is a game passed over completely.”
ummm… this claim really makes you look like this woman in the video:
GET CAUGHT by the system, and still denying FACTS. Read the post more carefully next time before you say anything, Wiley.
+1 for agreeing with me!
I’m still waiting for you to qualify which game got zero plays. There’s a good enough backend to prevent this from happening and there’s quite a bit of frantic scrambling from judges to make sure of it. Obviously I can’t speak to the years before 2010, but I saw enough of the structure of the thing to see that games weren’t getting zero plays.
Obviously it’s an absurd amount of work to make a game alone or with a small team, and every game deserves eyes, but the teeth-gnashing and name calling is biting the hand of one of the only institutions set up to try and nurture indie gaming. Everyone involved with this is doing it out of love for the games and the community.
Alright, I saw Mikael’s post. Dunno, what year, what game or what the circumstances are, and I’m curious to find out. All the games I’ve seen were hosted on the IGF FTP site, pushed over testflight, or already on the app store with promo codes. I’m indignant because I judged well over my initial allotment for mobile, went voluntarily into the rest of the games, and then did final judging. I saw a lot of time being spent on unheard of games and a real effort to try to find something new and from left field, and this buddy-club bullshit that I see being bandied around was simply not the reality of the judging process.
I’m checking up on Mikael’s case and it sounds like this was long before my time or Brandon’s. This would not have been possible in the system I saw at work.
Also, I am not keen on getting ALL-CAPSed by an adult who calls himself “batman”. Wear a name if you want to get in fights with random people who are trying to help.
IGF has became a close private friends club. They are prices/promotion of games made by people and ‘judged’ by their friends. If you are not in this club, you have nothing to do. It doesn’t matter if your game is an iOS game or a PSN/XBLA game.
IGF is not only broken but completely corrupted.
I was assigned your game to judge. I struggled with it for as long as felt necessary, found it pretty unfinished and frustrating to play, and knew that there was no chance it was going to receive a nomination in any category. I imagine others had a similar experience. (In fact, from the comments I know they did.)
Also, I think it’s worth stressing how you abused having knowledge of who was judging your game, sending out direct emails to those people instructing them to play it, and complaining that they hadn’t installed it in the time you decided was appropriate. (Demanding that people begin a two-hour game over a week ahead of the deadline, only six days after you uploaded it two weeks into the judging period, is pretty ridiculous.) You wrote:
“The build has been up for a while now (I sent emails via TF), but only
1 judge has installed, and 4 other judges still haven’t even signed up
for TestFlight.”
That was massively improper, and I feel using your access to emails this way should have disqualified you. The proper method was to have IGF contact us, and they had. That didn’t influence my impression of the game, as it happens – in fact your confidence made me think I must have missed something and went back to it to try again. I hadn’t.
I am convinced that there is no chance that your game would have gone on to win anything had every judge played it for three hours. Nothing would have changed.
And I’d like to echo others here and say that I discovered games I’d never heard of through judging, that I absolutely loved, and enthusiastically voted for.
I think that the tone of this blog speaks for itself, IG isn’t complaining about not receiving a nomination or being overlooked. So I think it’s pretty disgusting that a judge would take to his/her keyboard for the sole purpose of saying “Nyeh nyeh, your game didn’t stand a chance so your point is invalid”
This is clearly about fairness and consistency.
I don’t care how busy a judge is, they should make every effort to treat every game equally. I’m pretty certain that impartiality doesn’t include dropping your assigned game or not paying it more that 4 minutes attention so that you can “discover games” and enjoy yourself. If you volunteer for something then be prepared to show some commitment, don’t piss away someone else’s hard work and $100 just because you weren’t wowed with the first 4 minutes.
One word. Ethics.
With regard to the “misuse” of email addresses… If your game was being completely overlooked I think you’d be pretty irritated and disappointed, so I don’t think it’s unreasonable to prompt people to carry out their responsibilities as a judge. If it were me, I would happily be disqualified and pay twice the entry fee to expose a broken system of judging.
In short, you clearly had 20 minutes to write up a condescending comment on a blog post you didn’t have a responsibility to read so why didn’t you spend equally as much time on something you volunteered to support? Surely impartiality and supporting EVERY indie dev is worth something substantial, right?
“To your separate point about engagement with your game for the judges that did manage to get it installed, here’s the reality: all of our judges and jurists are professionals who very generously volunteer their time, year after year, to look at an ever-increasing number of entrants, for no reward other than a genuine desire to help foster and grow and reward achievements in the independent game development community”
Then the IGF will never amount to anything more than an unprofessional, informal circle-jerk and it’s your fault it’s set up like that. It’s astonishing you bilk $95 out of start-up devs and don’t even ensure their games get properly tested. It doesn’t matter if the judges also don’t get paid you are obligated to emsure all entrants are tested. It’s expected some judges may drop out but you the IGF are expected to have contingencies in place for that, not just say it’s not your problem. We already have several examples of games getting ZERO downloads.
Scum.
Believe it or not, all award shows work like this. That’s why they’re worthless popularity contests, and why nobody but those who already have a lot of fans or followers ever wins. You don’t have to be good, just popular.
for the love of god please consider changing that background, it’s killing my eyes
same thinking here. I’m down now. IGF was a dream for me and now, I don’t want to send any games at all.
As a developer, it is very frustrating to pay $95 and not receive much, even playtesting, in return. You have paid money, you feel like you should get value.
THAT MONEY IS NOT GOING TO THE JUDGES.
Feel cheated of your value? Don’t blame them – they’re not taking your money and running with it, they are instead giving away hours of work for free.
I’m quite certain there are games that exit the IGF never being played – these are the games that had technical problems, or whose developers didn’t bother uploading them to the official depositories, or who required special hardware or multiplayer-only setups. Sorry, but an unpaid volunteer isn’t obligated to go out and buy a bunch of extra kit and then organise a local lan party for sixteen people in order to give the game the time its best presentation.
Judges install games and never play them, sometimes because the bloody thing absolutely refuses to launch (Not your game, but many games I’ve encountered.) and after you’ve tried multiple times with multiple builds and gotten nowhere…
Judges play games for about five minutes sometimes because the game is profoundly un-fun and/or hard to play. You’ve got a stack of other games to get through. If you are completely at sea with a title, you step back, ask if any other judge has insights you’re missing, read some reviews of the game if it’s out or forum posts about it if possible, again looking to see if there’s something obvious you’re missing, and if nothing is coming up you probably move on. Until and unless judges are paid to achieve minimum completion this is probably how it’s going to be.
Judges are not signing up to high-five their friends, they’re doing it because they love games and they want to discover cool ones.
I do think it would be helpful if the IGF could in some way encourage judges not to wait until the last minute. I’m not sure how to go about that, since a lot of developers wait until the last minute too, so many judges think it best to wait until the end to start looking at games because that’s when the final builds will be in.
There is a clear proof that IGF are broken: the list of entrants and the list of finalists. You can see who are the finalists and their relationship with IGF/IndieGames/Gamasutra (all of them being the same). You can see how Fez has won the same award some years (can you imagine that in movie awards?). You can see the people who has helped this game (Gamasutra has an interview with the author who talked about it). You can read about Spelunky, … And you can see in the list of entrants some good games that have been recently sucessfully released that are not even mentioned.
And don’t expect any change. As you can read in their excuses, they are happy with their private club.
It’s almost as if the best games get the most coverage?!
Who could explain this mystery?
Keep digging.
It’s more like the games with the most coverage get the most nominations. Quality doesn’t come into it.
Bit surprised this hasn’t come up (at least that I see) but there’s all this talk about judge’s lives and how they have to pop out a kid while they prepare for their astronaut physical so they can go to the moon.
Maybe uh, find different or more judges? If all these people are so damn busy, maybe they can not judge the games that year or something.
You do realise that the pile of games is so huge that the IGF was _frantically_ scrambling for more judges as it is? Seriously, it was getting down to “OH GOD do you know ANYONE else who works in games and isn’t involved in the pirate kart and is willing to get into this thing???” 🙂
Most professionals ARE ‘so damn busy’.
I suppose they could always resort to child labor…
I totally absolutely agree 100% that people that have a problem with the IGF should no longer enter or support the IGF. If you truly have a problem with it, DON’T DO IT ANY MORE.
Let me ask you something: if the game wasn’t polished, and it had bugs, and you didn’t expect to get nominated for anything, WHY DID YOU ENTER? Seriously, I’d like to know. Was it for cheap publicity? $95 gets you either a) publicity from being in the IGF, or b) the righteous indignation of having your admittedly buggy, unpolished game not be played, so you could post about how abused you were… and get publicity?
Again, if you don’t like the process, don’t be involved in it. You and all the other disgruntled people can make your own festival, where everyone is guaranteed to have their games played to completion and everyone can get an award and everyone can be sharing and caring. No one is stopping you.
Valid point.
This doesn’t explain how games such as Super Brothers: Sword & Sworcery EP won the 2009 award for ‘Achievement in Art’ well before it was released in late March of 2011.
Seeing as it wouldn’t be released for another 18+ months after it won the award. I highly doubt the entire project was polished and ready to play start to finish at the time it was entered into the competition.
This then begs the question, how did it win for ‘Achievement in Art’? Off a trailer? off of friendship? off prior knowledge of the SuperBrothers’s art style? Some other super secret reason?
Who knows.
I’m not saying that it did, or did not deserve an award. What I’m saying is, based on your criteria, I don’t see how a game can win an award before it’s feature complete.
Superbrothers: Sword & Sworcery EP was submitted as one complete polished level (Side B) as well as one nearly complete boss fight (the Boor). Both were playable on the show floor that year at GDC. Worth noting: Sworcery did not win “Achievement in Art” in the main IGF competition, it won in the “IGF Mobile” competition that no longer exists.
The IGF has set itself up as *the* competition for the entire indie community. It’s about as hard to ignore as Sundance for Movies.
And for $95, the entrants are hoping that their entry at least gets a decent chance to maybe by a finalist, even if their game stinks. They pay for the right to be judged. And that’s what they didn’t get. It doesn’t pay for publicity, it doesn’t pay for righteous indignation, it pays for the right to be judged fairly in the leading industry competition.
And the things that are stopping us from making our own festival. The cost in getting it set up. The time it would take to get sponsors and get followers. Not having GDC to back it up. Being taunted as the poor little brother of the IGF. One does not just simply create a festival out of thin air.
The true point is that if the IGF wants to be the main festival, then it should at least be professional to maintain the minimum standards of a proper competition. What we get instead is neglect and empty promises. And that’s what we are raging against.
They WERE judged. People played the game. They didn’t like it.
In a case where actually nobody plays the game then yes you should get your money back (although since many of those cases are ‘because you submitted a game that wouldn’t run’, you shouldn’t have submitted in the first place)
There are big mainstream ‘leading industry’ competitions that charge much higher entrance fees and LITERALLY do not play the games, they pick the finalists based on their marketing materials. The IGF at least gives it a good go at TRYING to make sure people play the games. They do not currently require a minimum length of time played. But if multiple people played the game, and in this case they did, you cannot say they weren’t judged. Just that they weren’t given as in-depth and personal attention as the entrant might have liked.
The whole situation is troubling. I don’t know if you blame the IGF or the individual judges, but it’s obvious that the developers were treated unfairly here. If a judge can’t verify that they gave a game a fair shot, I don’t think they should be invited to judge the next event. If the IGF can’t hold their judges to a certain standard, I don’t think the IGF should be the organization to represent indie developers.
Obviously the IGF is a good organization, but they can only remain good if they clean these sorts of things up and deal with these situations fairly.
I love when articles put the most outrageous parts in bold typeface so I know at what to be contemptuous and can largely skim the rest of the text.
Very similar to my experience with indiecade in 2010; they took my money but never even logged into the game client (it’s a client-server game; they did log into the website, once). They made some vague excuse that they would still supply useful feedback, but then later claimed their database had been corrupted and all their review files were lost.
If people are paying $95 to be judged, why aren’t the judges being paid? That sounds unfair to me.
During 30 years in the game industry, I’ve not only judged some small game competitions, I’ve spent time reviewing freelancer game submissions for publishers I’ve worked at as well. It’s surprising to me that people would think 5 minutes isn’t enough time to do an initial pass and determine which games “might be one of the top 5% in quality” and which ones definitely are not.
When you have 20 or more games to go through in a short space of time, any sane person would make that first pass, ruling out well over half of the candidates, and then go spend some extra playing time on the few that felt promising. That’s not “incompetent judging”, that’s just a sensible approach to the task at hand. I understand it’s dissapointing when you’re one of the ones that don’t make the first cut, but that’s no reason to cry foul when the volunteers are just trying to do what they can to pick out the best games.
So where do the other 4 judges who DIDN’T PLAY THE GAME AT ALL come in
Are you truly arguing that a ” sensible approach to the task at hand” is to never play the game that you were tasked to judge?
The discontent these developers express seems to stem largely from the fact that no “first pass” was ever made by fully half the judges assigned to their (paid) entry.
If you were standing in front of a Black Square by Malevich would 5 minutes be enough for you to determine whether it “might be one of the top 5% in quality” paintings?
I don’t understand why there would be so much focus on the judges who didn’t play it or didn’t play it much.
Does the $95 guarantee you eight judges at a certain amount of time each?
These are volunteers. Human beings, especially those who are volunteering for free, don’t always meet their commitments. That’s true anywhere and of anything. It should be expected, anticipated, and accounted for.
If the IGF wants to make sure that at least one judge takes a good luck at your game, and maybe a few others take a brief look, then yeah, they better assign that game to at least eight or more judges. Because they know that some of those volunteers won’t get to it.
And you shouldn’t spend 2 seconds worrying about a judge that got assigned your game but didn’t get to it or spend much time on it. For all intents and purposes, they weren’t a judge of your game. It’s not like they turned in a negative review of it. They just didn’t really judge it.
The fact is that several did play it. One for about an hour. A few more for more than 10 minutes. It was judged. By several. And none of them nominated it. That’s the judgment right there. Four judges or ten judges or a million judges that *didn’t* play the game have nothing to do with it.
If you wanted eight judges to spend an hour on your game each, then IGF probably would have had to assign your game to 64 judges. But then, of course, you’d still ignore the fact that none of them nominated your game, and instead rant about the 30 – 40 judges who didn’t play it.
“No one is obligated to play your game; you have to prove why you’re worth their time. Every second from bootup counts.”
So if the judge isn’t hooked up in your game in the first 5 minutes then the game is boring and it’s your fault as a developer? What a piece of garbage. Then apparently all RPGs are boring and stupid. Planescape Torment? Worst game ever, you just run around the mortuary and talk to zombies… What about games that have intro longer than 5 minutes?!
And to the person above who said that Minecraft is a WORLD BUILDER, not a game – brilliant! Then Skyrim is not a game too, it’s a CHARACTER BUILDER. MGS is a MOVIE, Street Fighter is a JOYPAD BENCHMARK and point & click titles are fucking BOOKS because you have to read all the time.
[…] tego było mało to pod wpisem na blogu studia pojawiają się komentarze innych developerów z ich statystykami. Po głębszym przyjrzeniu można […]
[…] *The IGF (Independent Game Festival) Judges apparently don’t judge their games! Read here! […]
[…] What’s Wrong with the IGF « The Rotting Cartridge – "Eight (8) judges were assigned to Kale In Dinoland. Of those judges, 1 didn’t install the game or respond to any of our invitations (which we had to send multiple times before judges joined). 3 judges didn’t play the game. Of the remaining 5 judges that played the game, 3 played it very close to the IGF deadline, which was December 5th. […] Excluding the outlier, on average each judge – including the 3 that didn’t play it – played the game for almost 5 minutes’ time. […] So we’re talking almost 4 minutes for each judge of actual game time." […]
[…] What’s Wrong with the IGF « The Rotting Cartridge – "Eight (8) judges were assigned to Kale In Dinoland. Of those judges, 1 didn’t install the game or respond to any of our invitations (which we had to send multiple times before judges joined). 3 judges didn’t play the game. Of the remaining 5 judges that played the game, 3 played it very close to the IGF deadline, which was December 5th. […] Excluding the outlier, on average each judge – including the 3 that didn’t play it – played the game for almost 5 minutes’ time. […] So we’re talking almost 4 minutes for each judge of actual game time." […]
[…] What’s Wrong With the IGF? […]
[…] this year’s IGF cycle generated, when the developers for iOS game Kale in Dinoland reported that nearly half the judges assigned to their work didn’t play it. Certainly there are good explanations for this, that the judges are human and technical […]
Thank you so much for this post. The naked truth about IGF at its worst.
We have a similar story: we have submitted the game Exesia: http://t.co/ZargYmEN
it is an _improved_ version of an already released Kaleidoscope, which had really good reviews from Eurogamer and pspmins:
http://t.co/roefT7Du
http://t.co/12TrSfvf
This is the first game ever in the history of video-gaming that explores synaesthesia, and I was surprised that we were not even mentioned in Nuovo category.
After reading your post I understood why. So I contacted Brandon and asked for judges feedback and information on how much time they played, because in our case this is very crucial. Below is an excerpt from pspminis.com review:
” As of this writing, it has a shockingly low 3 stars in the PS Store. Nothing against PS Plus subscribers, but they are awfully, offensively wrong. Granted, they got this game for free, so they probably didn’t want to spend time learning the game. Besides, it’s a game. Why work hard for it, right? *wink*”
An average gamer who judges games by the first 30 seconds – wouldn’t be able to get Exesia. For that reason I uploaded also a 6-page PDF concept document with the game for judges to read. (For the same reason Kandinsky and Malevich were writing whole books about their paintings. You would not be able to “get” a Black Square by looking at it for 30 seconds).
I am still awaiting his reply.
[…] Seumas McNally Grand Prize: FEZ 40:00 – Narbacular Drop 42:30 – Games Developer Conference 44:50 – The Rotting Cartridge: "What’s Wrong with the IGF" 45:10 – Testflight 46:35 – Rant-Antwort von Jenn Frank: The IGF is Just Fine – You’re the […]
[…] Stein des Anstoßes: Die Kritik am IGF von Rotting Cartridge und Jenn Franks vielfach verlinkte […]
[…] year there was an interesting event happened that someone found their game was hardly opened by the judges. Spencer had an itch that he wanted to scratch. We […]
e liquide What’s Wrong with the IGF | The Rotting Cartridge
hello
only a small scribble to let you know, i’ve shared it on facebook
Hello. Looks like the limelight’s gonna shine on you.
Due to recent developments in what is commonly now known as “GamersGate,” this is going to receive a lot of attention. Be prepared.
[…] couldnt be bothered to play it for more than 5 minutes due to deadlines of the contest. Here is the link. An interesting video that links all these people together, in some cases more than just friends, […]
“The IGF, contrary to the most creative of the conspiracy theories, is not a shadowy elite who conspire to promote their own, it is very simply a few hundred game developers, academics and journalists of every stripe, all of whom volunteer for this job solely because they are enthusiastic about independent games”
top kek
[…] IGF already had a sordid rep, as it turns out, as someone from Twitter linked to me. This indie dev, whom competed in IGF 2012, used TestFlight to deduce that of the IGF judges assigned to them, 5 […]
[…] isn’t alone either, there are a number of other developers claiming the IGF judges made almost no effort to review their games based on the tracking of their games. They’ve also stated that since […]
[…] leads us to this: remember the indie dev I linked to in the last blog? The one that chronicled lousy, lazy behavior from the IGF judges? Well, Jenn Frank quickly lept to the IGF’s rescue against this big, bad, small indie dev in […]
fake hermes bag Please Click The Next Page
What’s Wrong with the IGF | The Rotting Cartridge Hermes Medor Watch Fake
Reblogged this on The Dogg Kennel and commented:
IGF is already suspect with the behaviour of it’s judges. but here is more info on how bad this whole thing can get.
[…] Seumas McNally Grand Prize: FEZ 40:00 – Narbacular Drop 42:30 – Games Developer Conference 44:50 – The Rotting Cartridge: "What’s Wrong with the IGF" 45:10 – Testflight 46:35 – Rant-Antwort von Jenn Frank: The IGF is Just Fine – You’re the […]
Promo Kartu kredit
Whats Wrong with the IGF | The Rotting Cartridge
こんにちはちょうどあなたに素早く頭を与えたいと思いました。 |あなたはテキストの単語 記事に画面をオフに実行されているように見えるクローム。問題またはブラウザとは何か|これはフォーマット形式である場合、私はわからないの互換性が、私は考え私はあなたが知っているようにポストと思います。 設計しかし見栄え！すぐに固定 | 問題問題あなたが得る願っています。 乾杯
一部予約販売 公式直営店在庫 http://onyourleftracing.com
ありがとう最終的に話 |を好き気に入っ％blog_title％は約！
完売これで最後 中古 激安 http://drupalcookbook.ru
単に したい欲望あなたの記事のようにあると言う驚異。 公開 | 鮮明明確にするである単に クール 私 ことができる を前提としテーマに|精通し| |この専門家} あなたはプロをしています。 まあ私はへ {最新の状態に保ちます。 おかげ 1,000,000と 下さいやりがい仕事を。
一部予約販売 土日祝も発送 http://webpostdinge.nl
あなたを共有するための| おかげで、ありがとうございました。私は本当にあなたの努力に感謝し、私は午前 | ポスト 感謝をもう一度、あなたの更に次を待ちます。
一部予約販売 人気急上昇 http://www.outdoorimpact.nl
私は本当にだサイトあなたのデザインとレイアウトを楽しんで。私はここに来て、より頻繁に訪問するために|それはずっと楽しい楽しいになり目には非常に簡単です。あなたのテーマを作成するために| 開発者デザイナーあなたが雇うましたか？ 優秀働きます！
商品到着後レビュー記載でプレゼント 本物 レディース http://www.millingmachineprice.net
執筆の作品彼/彼女は維持 思考彼/彼女の中のユーザーの誰かが書き込むと、心ことをどのようにそれユーザーが知って理解することができます。 段落記事 完璧は これが理由です。ありがとう！
最も安い価格で綺麗な製品価格で提供し 一部予約販売 http://worldwiderections.com