Google Summer of Code 2013

Google Summer of Code 2013

At Twitter, we love Open Source, working with students and Google Summer of Code (GSOC)! What is GSOC? Every year, Google invites students to come up with interesting problems for their favorite open-source projects and work on them over the summer. Participants get support from the community, plus a mentor who makes sure you don't get lost and that you meet your goals. Aside from the satisfaction of solving challenging problems and contributing to the open source community, students get paid and get some sweet swag for their work! In our opinion, this is a great opportunity to get involved with open source, improve your skills and help out the community!

Information for Students

These ideas were contributed by our developers and our community, they are only meant to be a starting point. If you wish to submit a proposal based on these ideas, you may wish to contact the developers and find out more about the particular suggestion you're looking at.

Being accepted as a Google Summer of Code student is quite competitive. Accepted students typically have thoroughly researched the technologies of their proposed project and have been in frequent contact with potential mentors. Simply copying and pasting an idea here will not work. On the other hand, creating a completely new idea without first consulting potential mentors is unlikely to work out.

If there is no specific contact given you can ask questions via @TwitterOSS or via the twitter-gsoc mailing list.

Adding a Proposal

Please follow this template:

• Brief explanation:
• Expected results:
• Knowledge Prerequisite:
• Mentor:

When adding an idea to this section, please try to include the following data.

If you are not a developer but have a good idea for a proposal, get in contact with relevant developers first or @TwitterOSS.

Project Ideas

TODO

Mesos

Security and authentication

• Brief explanation: Add security and authentication support to Mesos (including integration with LDAP).
• Knowledge Prerequisite: C++
• Mentor: Benjamin Hindman (@benh) and Vinod Kone (@vinodkone)

Advanced allocators

• Brief explanation: Build advanced allocators into Mesos in order to run more complicated framework mixes
• Knowledge Prerequisite: C++
• Mentor: Benjamin Hindman (@benh) and Vinod Kone (@vinodkone)

Scalding

Scalding Twitter's library for programming in scala on Hadoop. It is approachable by new-comers with a fields/Data-frame-like API as well as a type-safe API. There is also a linear algebra API to support working with giant matrices and vectors on Hadoop.

Scalding REPL

Make a REPL to allow playing with scalding in local and remote mode with a REPL.

• Brief explanation: The challenge here is scheduling which portions of the items can be scheduled to run, and which portions are not yet ready to run. You will build a DAG and when one is materialized, you schedule the part of the job that is dependent on that output.
• Expected results: Be able to type "scalding" and be inside and interactive scalding job.
• Knowledge Prerequisite: Need to know scala, and familiarity with the scala compiler/REPL would be helpful. Knowing the basics of Hadoop would help, but not needed.
• Mentor: Oscar Boykin @posco

Matrix optimizations

• Brief explanation: How should we multiple A*B*C? Perhaps (A*B)*C takes a lot longer than A*(B*C) due to the sizes of the matrices? What about matrices with huge skew (some columns or rows are MUCH denser than others)? What about matrices that are nearly vectors (just a few columns but MANY rows, for instance). By optimizing at the Matrix API layer, we can easily reap the benefits at higher layers.

• Expected results: Matrix multiplications for a few selected benchmark matrices should be faster. May use either hinting from the user about the matrix sizes, or sampling/approximation to estimate the sizes.

• Knowledge Prerequisite: Need to know scala, linear algebra, and dynamic programming or compilers would be helpful. Knowing the basics of Hadoop would help, but not needed.

• Mentor: Oscar Boykin @posco

Integrate Algebird and Spire

• Brief explanation: Spire is a scala library modeling many algebraic concepts. Algebird is a Twitter library that is very similar and has a subset of the objects in Spire. We would like to use the type-classes of Spire in Algebird. Algebird is focused on streaming/aggregation algorithms, which are a subset of Spire's use case.

• Expected results: A branch of Algebird that has all the same functionality and all the same tests passing, but removing all code that is in common with Spire.

• Knowledge Prerequisite: Need to know scala, linear algebra. Knowing the basics of Hadoop would help, but not needed.

• Mentor: Oscar Boykin @posco

Project

Project URL

Project Idea (e.g., New Feature)

• Brief explanation:
• Expected results:
• Knowledge Prerequisite:
• Mentor:

General Proposal Requirements

Proposals will be submitted via http://socghop.appspot.com, therefore plain text is the best way to go. We expect your application to be in the range of 1000 words. Anything less than that will probably not contain enough information for us to determine whether you are the right person for the job. Your proposal should contain at least the following information, but feel free to include anything that you think is relevant:

• Please include your name and twitter handle!
• Title of your proposal
• Abstract of your proposal
• Detailed description of your idea including explanation on why is it innovative
• Description of previous work, existing solutions (links to prototypes, bibliography are more than welcome)
• Mention the details of your academic studies, any previous work, internships
• Any relevant skills that will help you to achieve the goal (programming languages, frameworks)?
• Any previous open-source projects (or even previous GSoC) you have contributed to?
• Do you plan to have any other commitments during SoC that may affect you work? Any vacations/holidays planned?
• Contact details

Good luck!