Over the past several years at Zynga I have been able to do a lot of interviews at Zynga. These have ranged from CTO’s to SDET Internships. One of my favorite questions to ask interviewers is affectionately known as “The Poker Problem”.
This became my favorite problem because it was how I was interviewed to work for Zynga. I was given a laptop computer, a problem to program a solution to and a 2 hour deadline. To do this day I still believe that if you truly want to test an engineers salt – give them a problem and a laptop and see what they code as a result.
We would provide the candidate with a laptop with really nothing installed on it. Ensuring that the candidate does not have internet access is important. Provide the user with notepad and then a command line interface along with a prompt to run their code. Ensure ahead of time that PHP or whatever language you are testing in is setup to run.
Site the candidate down with the laptop and go over the problem with them on a whiteboard. We also provided a text version of the problem so that they could re-read it to clearly understand the object.
It’s important to note that if the candidate has no experience with Poker – then this is probably not a fair test for them to take. If you provide enough instruction they should be able (as competent engineers) program any game you give to them – but during an interview you always want to remove as many blockers as possible. Or maybe not… up to you and your company.
Explain to the candidate that they are working on a team that controls a Poker game. The game is typical 5 card poker – no special variations. Provide a basic set of rules to the candidate. This includes a hand rank rule. In essence hands are ranked from highest lowest as: Royal Flush, Straight Flush, 4 of a kind, Full House, Flush, Straight, 3 of a kind, 2 pairs, 1 pair and then high card. Also explain that poker also ranks by suit in the event of a tie: Spades, Hearts, Clubs and then Diamonds.
The “Poker Problem”
Explain to the candidate that their objective is to program an algorithm that will simulate the end of a poker match and determine a winner. In specific they are to take in N number of hands, figure out what the hands are (flush, straight, nothing, etc…), rank them all and determine which player has won.
The candidate can set this up any way they want – just ensure that at the end of the algorithm a winner is provided (and it is the correct winner).
You can explain to the candidate that there are certain assumptions that they can work under.
- The candidate can dictate what the input into the algorithm can look like. Some candidates will decide to use an OOO approach and pass in objects of type “Hand” which hold 5 objects of type “Card”. Other candidates may simply pass around arrays, lists, etc… The critical part is to allow them to choose (don’t give them hints) and ensure that they can rationally explain why they choose the format they choose.
- The candidate can also dictate what the output from the algorithm should look like. Some candidates will want to return a full list of who got 2nd place, 3rd place, etc… Others will simply return an index of the winning hand, etc… Again, the critical part is to allow the candidate to choose and then ensure that they rationally explain why they made their choice.
- You are playing standard 5 card poker. There are no wilds, the 5 cards being sent in are static. There is only 1 deck of cards being used, etc…
You can explain to the candidate that there are several “extra credit” opportunities with this problem. Each of these need to be controlled within a config file so that they can be easily turned on and off. Some of these include:
- Ensuring that the code is written in a way to handle 1 WILD card. How would they handle testing for potential hands with a wild card.
- Ensuring that the code supports Texas Holdem poker which takes in 7 cards and returns the best hand of 5 cards.
- Ensuring that the code supports multiple card decks – as if this was in Vegas and 5 decks of cards were being used at once.
We give engineers 2 hours to perform this problem. The key is that we leave them alone for the 2 hours. Explain to the candidate that when the two hours is up they will be asked to show off their code and explain how it works.
2 checkpoints are offered throughout the time window. The checkpoints are designed to just be quick 1-2 min interactions to ensure that the candidate is not blocked or has any questions. The first checkpoint is done roughly 30 mins after the engineer starts. Some candidates will want to jot down notes or plan out how to program the problem. Generally during the first checkup you simply let them know that 30 mins is up and if they haven’t started programming yet – they should start. The second checkpoint is about an hour later when only 30 mins is left. This is more of a reminder checkpoint to just let them know that they only have about 30 mins left.
When the 2 hours is up it is now time to do the analysis of the written code. When I come in to do the analysis I generally give the candidate about 30 seconds to finish up whatever they were working on – but I try not to give them any more then that.
I first ask them how it went. What was easy? What was challenging? I ask them about what it was like to program in this environment and under this type of “stress”
I then ask them to explain a high level of the solution they decided to take. If a whiteboard is available I will ask them to map out the flow of the data as it passes through their algorithm. I try to ask as many questions about why they chose certain directions. I also try to ask them to explain what will happen to their code or algorithm once it is put to scale.
Next I will ask them to walk me through their code. This is more of a visual inspection on their code writing abilities. Did they use comments? Do they use shorthand code? Is it well organized with a logical flow?
Keep in mind that the questions you ask during the analysis phase will give you more insight into how that candidate will perform in your company then any other interview questions – use your time wisely and evaluate their work as if they were already a member of your team.
One thing I love about this interview process is that it offers several gotcha opportunities for candidates to either shine or fail.
One classic gotcha is whether or not the candidate gets in over their head. Often times a candidate will rat hole themselves by creating a very elaborate data structure or code framework. This candidate lost sight of the overall goal (to tell me who won) because the candidate was trying to impress me with their code breadth of knowledge. This will give you as an interviewer very good insight into how much direction and oversight the candidate will need on a regular basis with his or her programming tasks.
Another common gotcha is whether the code will be performant at scale or not. This often directly relates to the experiences that the candidate has been exposed to. Candidates with less experience tend to make solutions that have deeply nested if statements or loops. Candidates who have written production scale code often provide solutions that are modularized with clean functions or steps seperating out the various programming tasks.
The extra credit provides another gotcha. Some engineers will go after the extra credit before they finish the actual assigned task. In some cases they end up failing to deliver the result as a whole. I am always a fan of giving engineers both explicit goals as well as stretch goals to go after – but not being able to focus in on the primary task can tell you that the engineer may have trouble focusing and might need constant checkups on their assigned deliverables.
My favorite gotcha is during the explanation phase. An engineer that you cannot communicate with is not an engineer that you want to work with on a regular basis. If the candidate is unable to explain what exactly they wrote (whether it is a good solution or not) is generally a good indication of whether or not that person should be hired.
I can honestly say that using this as the SOLE interview process for a candidate has yielded some of the best Engineers I have ever worked with. We had roughly a 20% walk out rate with this interview process. We had several people who simply walked out of the building after giving up an hour or so into the problem. To be honest I was glad as they were clearly not the engineers that I would want to work with. We also had other engineers who were able to provide a slick solution who we turned around and provided job offers on the spot. To this day they are some of the top engineers that I have ever recruited.