Game theory and Summer of Code

0 Flares Twitter 0 Facebook 0 Google+ 0 Reddit 0 LinkedIn 0 Made with Flare More Info'> 0 Flares ×

During my last year at University, I spent about the whole year studying economics game theory, in order to make my thesis about friend-to-friend networks and best pratices for overlay and sharing networks. Game theory is a very interesting topic, and can lead us to choose better strategies based on the problem enviroment, rules and possible odds (or rewards) you can get from each possible strategy. Unfortunatelly, some games doesn’t have good odds for every player in the game, and when we can’t change the rules nor the environment that enforces the odds, we have to take the least worse strategy even when it is far from been optimal. Fortunatelly, sometimes we’re able to change the rules in order to create an environment capable of giving good odds for every player in the game (at least for fair players…).

That’s why the odds stands for: reward the fair player, and enforce bad ones to play fairly.

During this GSoC mid-term evaluation of students performance, I’ve been thinking about the odds of the game we have to play here, and realized some rules that could improve the competition quality and reduce project failures.

I know many mentors will feel familiarised with this situation: One of the students disapper for 2, 3 weeks and appear right before the mid-term evaluation saying that he is way behind the schedule because of problem x and y, and asks for a chance to continue working on the project. But, It is not possible for mentors to know for sure if he is going to do something
if he let Google pay him. This totally looks like a problem we can solve with game theory.

Here are the goals:
– GSoC is intended to help the open source community, by affording students and mentors to create and improve open source projects
– Mentoring orgs want that help, because money is what moves any project forward, and GSoC is a huge help
– Students want to have their names associated with Google, with their own killer open source software and get paid for that (money and t-shirt 😉

And the odds:
– If a student works hard until the midterm, he receives the 2k
grant, and continues his work
– If a student doesn’t reach the midterm eval with a good status
(behind the schedule), two choices are left to mentors:
1 – Trust on student and let google pay him, hopefully that the
student is going to make until the deadline
2 – Drop student, and avoid his midterm payment

The problem is that many students are on GSoC seeking for the money only (that’s why so many are so desperate about the payment delay), and when they reach the midterm and see that they are not going to make it, the best strategy they have is begging for another chance,
get the 2k and give-up his project. This strategy is the best for him, but the worst for Google, Organizations and other students, because:
1 – Google doesn’t want to pay for nothing. Google wants more open source. That’s the main goal of the GSoC
2 – Organizations will be left without the project and the student contributions, and the mentor will have spent his time almost for nothing (almost, because the US$500 counts a lot)
3 – To accept this student, I had to dismiss another that know would give anything to be were this student stands. The bad student wasted his chance, wasted another student chance and wasted the money and time of you guys at Google for nothing.
4 – The bad student receives 2,5k for nothing and is happy because of that

Mentors also tend to choose this strategy, because there is a little chance that student is going to make it until the end of GSoC. Actually, mentors wants so much to have the project the students were allocated, that they give that chance because the strategy of dropping him is even worse than trust him.

My sugestion, based in this analysis, is to change the rules to avoid this strategy, and still
guarantee the goals of Google and Mentoring organizations. But, how can we manage to achieve that?

At midterm eval, if a student is dismissed because he didn’t made his work, the mentoring organization receives his money in order to use it to improve the software design, usability, buy equipments for testings and improve the software, offer that money as a bounty for another person that is willing to solve the problem the bad student left behind, etc. If mentors had that choise, here are the odds:
1 – Students will be afraid of don’t be aproved at midterm, and will be encouraged of working hard before it. As they want the money, they will work harder to have it.
2 – If a student still didn’t make enough to be aproved at midterm, mentors will have another strategy that is better for Google and Organization that is dismiss the student and use his money to make better open source software for the whole community, and the Google’s money served his purpose even with student failure, and mentors will have the opportunity to give another chance to a studnet he dismissed to get the bad one.

I can think about a lot of things that the project needs and can benefit with this money, and the whole community can also benefit from it instead of wasting it with an unresponsive student that wasted his chance to change something in the open source world. With this
student’s money organizations could pay for design, books, website, servers, offer the bad student’s money as a bounty to have the problem solved and a whole load of other things that the project needs.

I don’t know if the Google folks are going to agree with my point of view, but I just wanted to share my thoughts and maybe help improve GSoC rules, reducing the amount of projects that fails. Agreeing or not, Google is still rocking a lot with Summer of Code, and I only have to say thank you for the opportunity I’ve been having these years.


0 Flares Twitter 0 Facebook 0 Google+ 0 Reddit 0 LinkedIn 0 Made with Flare More Info'> 0 Flares ×
  • ggpolo

    I don’t see how someone couldn’t agree with these suggestions.

    Furthermore, it is a real disrespect to Google and other students when someone just goes “inative” after joining the program and still tries to receive money.

    I believe these students that were dropped at midterm evalutions in previous year never came back, but it would be good to enforce permanent exclusion from any and all Google programs.

  • maurelio1234

    1) I don’t think blacklisting the failed students is a good idea, it would be better just provide this information to future mentors. Failing is part of life and just happens.

    2) I don’t think giving the money to the organization is that good:

    a) what if organization and students try to do something like dividing the money google gives to the organization? (you work just half program, and gets $3000… and the organization gets $2000)

    b) what if organization start using the extra money to pay someone that doesn’t meet the program requirements?

    I think in both cases, the final situation is good for the organization and the students and bad for Google…

    by the way, the Google’s main intent is not having “more open source code out there”, but having a page full of successful histories and a bunch of students “spreading the word”. For free.

  • Adriano Monteiro Marques

    1) I totally agree with you. I don’t mean to blacklist students, but how can you provide this information to future mentors without sounding like blacklisting them? Some students didn’t fail because that’s life, but because they weren’t able to manage their time, or they did lie on their resumes. Blacklist is not the solution, but letting mentors know about latter experiences with the failed student. This will avoid a bad student from getting into GSoC once again, when another student with a better profile could be in.

    2) Sometimes…
    a) In this case, you’ll have to trustu on student, which is not a good option. Also, the studnet will receive the money in his currency after paying some taxes to have the money converted, an than, he will have to convert it back to dollar, pay some other taxes to transfer the money to the org, and them the org will have to convert it back to its currency paying some more taxes. Doesn’t sound that good.

    b) That doesn’t sound like a problem, because the person is going to get paid outside of the program.

    Ddidn’t understand why in both cases it is worse for Google, if Google is willing to send this money out in any case, and in this case, the money is going to be paid for something useful (that even Google could make use of)

    And, I think that Google’s goal is both: Have a lot of open source (because if it won’t be having more open source, why do they require that your software must have an open source license?) and success histories *arround* open source projects spreadding the word about google for free.

    In both cases, the strategy meets Google’s goal.

    The main problem with my proposed strategy, is that Google will have to spend 100% of the GSoC money, while they usually does spend only 80~90% of the money because of students failures. If Google does count with such failures in order to set their amount of slots, my strategy isn’t good.

    BTW, the strategy is only an idea, and I posted it here in order to have it improved, and maybe help GSoC in someway. Thanks for your comment, and hope to have more feedback.

  • iGor Feghali

    Hello Adriano,

    Following maurelio point i have something more to add..

    what if the organization starts dropping students (because of a minor mistake) just to get the money ?

    I would go in another direction… The student gets US$500 when the program starts as usual, then he only gets the other US$4k at the end of the program. In the midterm evaluation the mentor should say “this student did/didn’t good progress”.

    No matter the answer the student should keep coding until the final evaluation when the mentor says “he deserves the whole $4k, he did great progress at midterm so he only deserves $2k or he didn’t do good progress at all so don’t pay him”.

    You came up with a really good discussion and your point is valid. Congratulations for the post.

  • Adriano Monteiro Marques

    Hi Igor,

    Well, I believe that dropping students to get the money can happen. What I don’t know is what pratice is worse for GSoC and the Open Source Community.

    Also, I agree about deciding how much student deserves at the end of GSoC. This will make them work harder to have their projects done until the deadline.

    Also, any evaluation, AFAIK, are subject to discussion with Google Folks. If a student disagree with his mentor evaluation, he can search Google folks, and try to show that he worked as expected and that he deserves his payment. Though, I’m afraid that this could flood the folks at Google with a lot of unhappy students that were droped because their mentors wanted their money for the organization.

    Surely, that idea would need some kind of regulation to work as expected.

    Thanks for you comment!

0 Flares Twitter 0 Facebook 0 Google+ 0 Pin It Share 0 Reddit 0 LinkedIn 0 0 Flares ×