Let's look at how we streamlined the hiring process in our Software dept.!
This year we’ve done a bit of hiring in SparkFun’s Software department, and with each new hire we’ve really been focusing on dialing in our interview process. Earlier in the year we hired three developers at various skill levels and experience, so with each round of candidates we wanted to make sure we found the best fit for the department while making the interview experience as pleasant as possible.
We settled into the following format:
Our next challenge was to hire a Junior QA Analyst internally. We decided to go internal-only on this position because:
Like normal, we first defined job requirements and a job description, keeping in mind that we wanted this position to not require any coding or web development experience. At first we only received one applicant, but after a second reminder that the position required zero coding experience, we ended up receiving 12!
Since this was an internal-only posting, we each admitted that we had certain biases towards some candidates. Some of the candidates we’ve worked more with, some we haven’t. After a recent company Lunch & Learn on Inclusion and Diversity brought up blind interviews (see why blind interviews are a thing) as a way to combat our biases, we decided to try out our own blind section for this interview process. These are the core skills that the Junior QA Analyst needed:
We also wanted to make the take-home test as work related as possible to give candidates a taste of what the job might be like. We planted several bugs (over 10) on two pages (main home page and a product page) of our public-facing site, and hosted it internally for the candidates to visit and then file “bug reports” via a Google Form. The form asked for a summary, description, steps to reproduce and “what do you think is wrong / how would you fix it?” questions.
Here are a few screenshots of some bugs, since the site was only available internally:
First we met with everyone to do an in-person phone screen and go over the take-home test. Then the candidates were given nine days (spanning two weekends) to complete it. Once the Google Form closed, HR took the responses and swapped the candidate’s name for a numbered one, e.g. “Person 1.” We had 10 candidates complete the take-home with a total of 91 reported bugs!
As a team, we went through all the responses and ranked the “quality” of each bug report from one to five. There were some bugs that were invalid or duplicates, so those didn’t get any points. Some candidates found bugs that we didn’t plant and are valid in production, so of course they got points for those. Other baselines that we established:
We chose our top three based on their total score (sum of their quality scores) and the number of fives they had. Also conveniently there was a noticeable difference between the third- and fourth-ranked applicant, so having three as our final round really made the most sense.
After notifying our top three about the next step, which was a short, in-person interview, we looked at our fourth- and fifth-ranked individuals who had done really well, but just didn’t quite compare to the top three. We realized that this grading system was biased toward those with previous development experience, but we figured that we would be able to get someone more experienced up to speed faster. Next time we open up this type of position, we may want to ensure that previous dev experience doesn’t have as much of an advantage as it did this time.
Since this was an internal-only position, we had the opportunity and felt that it was important to provide feedback to all the candidates. Our QA Engineer met with each one to discuss what they did well, what they should work on and what resources to look into.
What we really liked about making the test blind was that we only looked at what the candidate provided. We didn’t look up their past bug reports or past performances. If this had been an external candidate, we wouldn’t have known those either. We didn’t give someone that we enjoy working with higher scores. We came to a consensus much faster for our top three than we would have if we didn’t have these objective standards. We were also hoping that the candidates enjoyed the experience too, and the feedback has been positive! Our top candidate feels that they earned the new position on merit rather than biases or office politics, and hopes this type of blind exercise shows up in other job openings. Others believe that it helped to put everyone on the same playing field… which was exactly our goal of the exercise!
We’re looking forward to including more exercises like this for each open position when possible. We feel that the anonymous nature of the “blind” test helped us focus on picking the person with the best skills for the job in a timely fashion, and the candidates felt that they were being compared evenly. Win-win!
Giving feedback did take me a bit longer than I had anticipated. I wanted to make sure it was valuable to the candidates, but I also was very encouraged with the feedback I got in return from them all. While there was obviously some disappointment that they didn't get the job, it was good to hear multiple people say they felt they got a fair shake regardless of whether they knew us personally or not.
As a bit of a follow-up, we're also planning on internally providing resources and training for those interested in learning more about the Software QA/Continuous Integration process starting later this fall. Exactly how it will look is something we have yet to figure out, but to those looking for a change of pace in their careers, we're hoping to provide exactly that.
Would be a dream to work at Sparkfun, but 8 years from graduation and health issues making it so I never got an engineering job after college kind of puts me out of the running. Its cool to see some of the processes at least
I wouldn't agree that you're out of the running. Sure - it might for example be difficult to get straight into a Senior Electrical Engineer position if that's something we're hiring for, but every engineer's path at SparkFun has been a different one. There are plenty of examples of employees who have started in shipping or manual assembly, to then move on to different departments and jobs depending on skill and interest. I do believe we've had a quite a few Engineers come through Tech Support, for example.
As for SWIT, we all started off down different paths too. I think there's only a couple of people (myself included) who actually have Computer Science degrees. I'm not trying to take anything away from having a relevant education - I'm just pointing out that there are many ways to become a skilled and productive employee in a given industry if you've got your sights set on it.
I like that you took the opportunity to provide feedback to even those who did not make it all the way through. I wish that I knew what other companies that I've interviewed with that didn't hire me thought about me so I could strengthen my weaknesses as a person or even for the same company some time down the road.