When interviewing senior people, I like to get them going on war stories. "Can you talk about a project you experienced that went very badly?" is a good one. It's not necessarily that they went badly, but we've all been on death marches before (and I absolutely would not hire an architect who's never done a death march!).
The beauty of this is you can find out how they react to impossible demands and pressures. What they're proud of accomplishing in dire circumstances. Possibly what they're ashamed or embarrassed about, or mistakes/lessons learned (if they're that bold). You can learn a ton about how they talk about their colleagues! Are they full of praise, or contempt? Do they find someone to blame, or talk about how someone else saved their asses? You can also learn a lot about their actual approach to technical decisions - their taste, for lack of a better word.
You can ask probing questions here, but the important part is to get them speaking in an unguarded way. Nobody makes it to the architect level in this field without at least kind of loving the job. Find out what they love. Find out what they hate. Find out what they regret. Find out any strong technical biases they have, and why. But most of all, find out how they play with others. Because the architect's job isn't just to design, but to communicate. You don't just want a technical expert. You want someone that can value and respect the rest of the team, or they're just going to cause problems.
>> Can you talk about a project you experienced that went very badly
I'd like to offer a spin on this.
In my experience, asking a question that demands one to recall specific details about a painful event in an interview, which is typically a stressful situation, may produce poor results.
What do you think about giving the candidate an opportunity to get advance notice of that you'll be asking that question, in an interview, and ask them to prepare three to five bullet points to discuss? Whether the candidate sends the bullet points back to you before the interview, or just brings them to the interview, is beyond the scope of what I'm proposing.
Let's say your trying to assess a 3 things:
* Has Architect been through the meat grinder at all? Bullet points with little depth log a warning, as there has been plenty of time to prepare a thoughtful answer.
* Does Architect talk about failure in a calm and composed manner? Bullet points give ability to ability to recall challenges before stress of interview sets in.
Does Architect trash talk former employer or coworkers? The stress of interview can erode one's asshole filter, and cause them to be a bit of the jerk they've worked really hard to not be (speaking a bit for myself here).
To wrap this up, I like interview questions that are functional with minimal side effects, meaning the person is given every opportunity to provide the answer you are looking without external pressure or distraction. If you want to see how a person handles stress, give them a situation that is stressful in a way it'd be on the job, perhaps leading with "apologies in advance for the following situation, which is intentionally stressful..."
I actually ask "Tell me about a project that went well for you. What was it, what was your role, and what was the final outcome?" I ask follow-ups, and generally let the interviewee get comfortable, since it's a low-pressure question. However, as a follow-up, I usually ask "Tell me about a project that -didn't- go so well. Same questions, and what'd you take away from the experience?"
People tend to slide into a comfort zone and open up a bit about their successes. They then -tend- to stay open and offer up useful insights into their past behaviors under pressure. I've found that it helps to ask the "success" question first, then ask the "failure" question second, rather than directly asking about someone's failures.
As an aside, the technique outlined above is something taken from "Influence: The Psychology of Persuasion," [1] and is a useful read to recognize situations where someone might be using certain techniques to influence the outcome of a situation. :-) Also - note that the general prompt above expects someone to answer in the "STAR" technique. [2]
Yes, that's an excellent approach, and really, that's how I'd usually do it. The important thing is to get them feeling comfortable, safe, and confident.
> What do you think about giving the candidate an opportunity to get advance notice of that you'll be asking that question, in an interview, and ask them to prepare three to five bullet points to discuss?
Depends on the level you want to interview for. For a higher-level position, I don't think it's unreasonable to rely on someone's communication and presentation skills to adapt on the fly. Explicitly tell them that they can take a minute to think about it, rather than having to have an instant response.
I'm after maximal side effects here. I'm not trying to figure out what they know. I'm trying to figure out who they are. That happens when they're unguarded.
Now, I do my level best to make interviews unstressful. I don't think a stressed candidate is going to do their best, or be their best. They're going to be second-guessing their answers.
That's the point of war stories. Every experienced candidate has them. We've all done memorable things in our career, both good and bad. If it sticks with you, the details aren't what matter.
edit: This goes both ways. I expect the war stories to be an exchange, a patter. They should be learning about me and my co-interviewers as well. And, as someone who gets interviewed, if I feel the interview is completely one-sided and not revealing anything about potential colleagues and the working environment, I'm going to hesitate.
This could end up being stressful for both sides. Experienced candidates could/should counter with the same question. I doubt decision made under stress will turn out good.
If someone is so nervous and anxious that they can't just talk shop with a peer and share some war stories without feeling stressed... well, that tells me something very important about them.
Me, I'm always good for that kind of conversation. It's not going to stress me out if they're asking me things like this.
In my experience, asking a question that demands one to recall specific details about a painful event in an interview, which is typically a stressful situation, may produce poor results.
To some extent, yes. But the flip side is that for candidates that have had negative experiences in poorly managed environments (i.e. negative experience with past employers) this allows them to vent this information in a somewhat more clinical (neutral) way.
> In my experience, asking a question that demands one to recall specific details about a painful event in an interview, which is typically a stressful situation, may produce poor results.
Stressful and painful are not the same. Every engineer is going to go through stressful situations. Good ones will learn from them: to minimize the risk of them happening, and to better cope when they do.
> If you want to see how a person handles stress, give them a situation that is stressful in a way it'd be on the job
How is this any better?
> "apologies in advance for the following situation, which is intentionally stressful..."
Stressful situations don't send calendar invites.
I'm not arguing one should be an asshole during the interview. I've been interviewed by clearly hostile engineers. I didn't like it, and it surely impacted my performance. But real life is like that. Being able to react in a polite and composed manner to an unexpected situation, and recover from it, is a superb data point in an interview.
> If you want to see how a person handles stress, give them a situation that is stressful in a way it'd be on the job
Mainly because you're paying someone to do a job, not to interview. I've interviewed a lot of people, and sometimes interviewing just sucks for reasons that have nothing to do with the interview.
For example, say you're homeless, and you get your first interview in quite a while. Landing the job means housing for you. The pressure to perform is enormous, and it's not something you can prepare for. I was homeless for a bit, and in that situation.
I don't expect to get a job for which I'm not qualified, but it'd be nice if an interview process allows me to show I can meet the requirements of the position, regardless of the circumstances of my life situation outside of work.
I like screening people in for the right reasons, and not screen them out for reasons that aren't really all that important. And when you have a hard time getting any qualified applicants, can you really afford to false negatives?
> I've been interviewed by clearly hostile engineers... is a superb data point in an interview.
I agree, having someone be clearly hostile to you during an interview (which is often the first impression of what the culture at a particular company is like) is a great data point for the decision on whether to continue with the process of interviewing there.
Uh, maybe they are so good and come from an environment with good process hence no death match. As a manager, no death march here, I rather ship a few weeks late and have a superior product and developers not burnt out. A lot of hard deadlines are made up. No one on my team will ever experience, that's what agile is all about. We ship the best we can within the constraints we have.
A combination of having been fairly lucky and terrible autobiographical memory (I have trouble assembling stories from memories—basically, nothing sticks in my brain as a story and very little from my personal life ever strikes me as story worthy, which made those "what did you do over the Summer?" writing prompts in elementary school absolute hell) mean I have little choice but to pick a real event and invent a narrative for these kinds of questions. I hate them.
> Uh, maybe they are so good and come from an environment with good process hence no death match.
There are many failure modes other than "death march". I'd be shocked if an architect doesn't have at least one good answer to "talk about a project that went badly". Or, failing that, "almost went badly".
Absolutely, there are many nightmares. I know death march, I've experienced it, but I'll fight for my team never ever to, should I succeed they shouldn't be penalized for not going through the pain. I'm constantly teaching how to be pragmatic while shipping. My nightmare has been integration with external vendors. Testing blind all from unproven specs.
Agree. Even in the best environments with good processes, everyone with a few years' experience will absolutely encounter a case of "something was more difficult than we expected." Being in a good environment just means that more solutions are available (e.g. communicating with management and reducing scope).
Note that the parent post put a lot of emphasis and importance on "death march" specifically, so you are actually altering his/her point quite a lot here:
> I absolutely would not hire an architect who's never done a death march
Oh, I meant that. A candidate for a junior programming job who'd never been on a doomed, nightmare project? Sure, no problem. But an architect? What if something we're working on turns into a death march, for reasons outside of our control?
An architect role isn't just a technical role. It's a leadership role. An architect who cracks under pressure easily is a liability for the entire team. If someone has never had to work when they know they're going to lose, they have no experience with it, and don't know what to do when it happens.
Perhaps, though I'd strongly disagree. However, they phrased the actual interview question as:
> "Can you talk about a project you experienced that went very badly?"
And that question, as phrased, seems quite reasonable to ask any prospective architect. The answer tells you quite a bit about how they handled themselves, how they worked with their team, how they talk about their project and team, and how they diagnosed and addressed the situation.
Sometimes you don't have that luxury. What if your company requires an influx of new capital to be able to maintain your current staffing levels and one requirement of the funding is delivering a set of features in a week? Adopting the attitude "we'll just ship a few weeks late" may result in funding not going through, everyone losing their jobs, and the company dissolving.
I agree that death marches are counterproductive. However, if you've been around long enough you've run across situations where they are unfortunately necessary. Another example: production goes down Friday at 5pm. Do you say "we'll pick this up Monday morning"? No... it's all hands on deck to get production back up working as long as it takes, assuming your company isn't large enough to have its own dedicated 24/7 incident response team. Hopefully there's a lot of process and automation which helps to minimize the effort, and if not that's a great takeaway to improving the process down the road.
> Sometimes you don't have that luxury. What if your company requires an influx of new capital to be able to maintain your current staffing levels and one requirement of the funding is delivering a set of features in a week? Adopting the attitude "we'll just ship a few weeks late" may result in funding not going through, everyone losing their jobs, and the company dissolving.
So then the question is actually asking, "have you ever worked at a company that was two weeks away from complete failure and bankruptcy?"
> Another example: production goes down Friday at 5pm.
While highly stressful, this is not what a death march is.
If they've never worked anywhere with poor processes then they're incredibly lucky. Even places with good processes can end up in a situation where these sorts of things are necessary.
> A lot of hard deadlines are made up.
Yes, but not all of them are. A lot of my clients face legislative deadlines that don't take developer comfort or work-life balance into account. We need to help them ship X by January Y. It's rare that overtime happens but it has happened and there's little you can do to get around it when a bill is signed into law mandating you do something in 30 days or the state becomes a litigation target.
I've worked in industries with hard deadlines. My first job was in large-scale academic test scanning. The kids take their test when the calendar says to take the test. And the trucks would show up at our docks with boxes full of tests whether we were ready for them or not. We might have a four month period to do our development, but the production season rolled around and we had to be ready.
The worst death march I was ever on was against a legislative deadline. Oh god, may I never do that again.
Before I started with my current company, there was a legislative deadline that gave an agency 30 days to implement some pretty substantial changes, roughly 60 days of full-time work for the entire team. Apparently the folks writing the bill felt that 60 days was too long and the agency should just figure out how to do it.
This also happened to be during a hiring freeze, so no new contractors, no temp employees, nothing. Current team of half a dozen or so had to drop everything they were doing, switch over to this project and work ~12 hour days six days a week just to deliver a mostly-broken implementation right at the deadline and spend another week and a half at the same schedule fixing bugs and other issues.
It would have been much worse if the timelines were extended, if it was 6 months worth of work to be done in 90 days I think the team probably would have quit. They were mostly contractors so paid hourly, so they got paid for all the work, but that's still a lot of hours for that many weeks in a row.
I have been very lucky not to come across anything like that yet.
I'm not asking them to talk shit about previous employers. I'm asking them to describe a situation. You can have a difficult experience, and describe that difficult experience, without talking shit about the employer at the time.
One of my most difficult experiences was the very last day on a job, when an "Everything is going to crash in four hours if you don't fix this and you were the last person to touch the code" bug appeared (we would have run out of disk space). It turned out the bug wasn't mine - it had been latent in the code for years, caused by the difference between NULL and 0 in a C pointer. The changes I had made just made it visible. Terrifying, stressful, and I'm kind of proud I was able to resolve it under such stress.
See? Terrible situation, not a single bad thing to say about my employer, my co-workers, or my own work. And none of those were actually the root cause of the problem.
Technical detail in this case... we were creating rows with a BLOB column in Sybase. If you passed in NULL, Sybase did not allocate space for the blob. But if you passed in 0, it allocated 8k.
We had been storing data in the blobs, in the dozens-to-hundreds of k size range. Sometimes we didn't store data in a row, but mostly we did. The data got moved to the filesystem, resulting in a huge performance boost. Because we pulled the blobs out of Sybase, we massively downsized the databases, to about 10% of their original size - which was still a few times larger than what was needed for an entire production season (about three months of data).
Now, this is where I badmouth co-workers... I had argued strenuously for removing that BLOB column completely, as it smelled wrong to me to leave it. But the tech lead, a rather nervous man who was always afraid of things breaking unexpectedly, insisted we leave it there, because removing it might break something else (I wouldn't be introduced to the Lava Flow Antipattern until years later...) (also, welcome to the world of coding in C without testing frameworks).
The code worked fine in testing. Beautifully. But testing couldn't even fill 1% of a database. The wasted space from the BLOB allocation was invisible... until production started, and rows were 100 times larger than they should have been on our biggest table. Oops.
Thankfully, Sybase has excellent documentation. The gleaming row of carefully indexed, spiral-bound manuals helped me find detailed example code quicky, which got me to the latent bug. We'd never noticed the 8k being allocated for empty blobs before because they were swamped by the size of the actual blobs.
Hmmm, no, sorry. `NULL` is #defined as `((void )0)`. There is no 99% or other probability involved.
Sorry to be so pedantic as it's kind of an aside to the discussion at hand. I thought perhaps you knew something about NULL in C that I never knew and was eager to be enlightened.
I think you're conflating C pointer `NULL` with the database concept of `NULL` which is not necessarily represented the same way when interfacing with database client APIs.
There's probably a detail you glossed over whereby you're passing 0 vs NULL via separate method calls to the database client API. The distinction cannot be due to literal `NULL` vs. literal `0` purely in C programming terms.
The limitations of C require that database-NULL be represented differently from integral values in terms of data structures. Even if you do pass `NULL` and `0` in the exact same parameter location to the exact same method name, you'd have to ignore a warning about the implicit type conversion from `void ` to `int` and still the semantics do not change and you have no runtime bug as you described.
For senior positions especially, getting on to things that "went bad" right away in the beginning of the interview might be counter productive. It makes the person uncomfortable. How about seeking his advice(genuinely) on a problem that you are facing. Or telling him about a situation where you personally fucked up and ask him how would he handle it. The purpose is to build trust and to do that it is extremely important to be genuine, not just sound genuine.
The middle of the interview should be reserved for a bit of uncomfortable talk where you talk to him about something that went bad. Thrash some of his ideas, question a few approaches and let him react to it. I also try to make it point to make him work on a situation where he doesn't have control over a few variables(like the stack, or the timelines, or the people or the requirements).
I keep the last few minutes for agreement, the conversation should leave a happy memory after all.
As an ADHD sufferer, I loathe questions like this, because while I can assimilate the knowledge and lessons that came out of a failed project, I am very poor at recalling specific events on demand.
A grueling process being forced upon you by circumstances / management that requires an unhealthy amount of hours, stress, etc to achieve to a (potentially unreasonable) objective by a certain time.
Likely outcome to meet the criteria is burn-out / resignation / death of personnel, especially if repeated.
Take an actual problem that your team solved. Distill it down to essential elements. Wargame out plausible solutions to it if one didn't have your team's context on your particular situation, particularly plausible solutions which would be top-of-mind to a smart, stressed person operating under incredible time pressure. (Your actual IRL solution should probably be one of them, but don't assume it is the only solution, the most likely solution, or the only passing solution.) Take insights out of those wargamed answers (e.g. "Ahh, this is performant", "Ahh, this approach fails loudly when our infrastructure is in a degraded state", "Oof, this creates a SPOF... not desirable here") and write them in a list. Group them by theme. Try to have somewhere between 5 and 10 themes.
For each theme on the list; make 3~5 variants of things which show the insight(s) related to the theme at varying levels of quality/discernment/depth/etc. Assign an appropriate numerical weight to that tier of answer. Your list is now two-dimensional; in teaching circles it would be called "a rubric."
Grade interviews against the rubric. On security, this candidate said X, Y, and Z, which are elements which most closely match a Good answer; 7 of 10 possible points awarded.
Even better, grade written notes taken during interviews against the rubric.
Now, and this is the brutally organizationally difficult part, stop overriding the rubric when it spits out point values which imply next steps regarding a particular candidate that are clearly not what one wanted to do. Instead, fix the scenario and/or rubric.
> Wargame out plausible solutions to it if one didn't have your team's context on your particular situation,
My least favorite interview ever, was essentially "Implement this generic problem that our team had to solve, but you don't have the context of our infrastructure, and I'll criticize you every time you deviate from the actual solution we came up with because your solution wouldn't work in our context". It really felt like a "read my mind" type of problem.
No further point. Just glad to see you point out the relatively obvious point that you need to evaluate a question for someone who doesn't have your current context.
A good counter to this problem for the interviewer is to pick something that is extra-domain to your problem space. A previous employer I worked with used "Design the object model of a Chicken" as a starting point. The interviewers would then act as a potential client and we'd go back and forth with the candidate over their approach.
Definitely rough on candidates, but it does give a reasonable window into their thought processes and who the interact with other people.
We do this, but in an interactive way, to see the approach taken, additional context given, adjustments made along the way. This really shows how a candidate thinks and if they can on-their-feet. Often we will run out of alternatives before coming to a solution, with many avenues explored. Rarely we will discover a path that may have worked, possibly even better.
I was put through that once, it sucks. You're completely ignoring all the wrong turns you took to get there, the number of people involved etc. Expecting someone to just jump into your shoes and pull of the same trick is not very constructive. What if you framed the problem wrong? Maybe this is the candidate that would have helped you come up with a superior solution given enough freedom and time?
Hmm, this could be a risky approach. If you do it right, it could be a fun, interactive way to suss out a candidate's approach to solving problems, preferred architectures, etc.
However, it also feels quite artificial and stressful, which is a bad combo for a lot of people. Unless the position actually requires rapid fire system design and architecture, I'd be cautious using it as a primary filter. For most positions, designing a system architecture includes include some amount of research, reflection, iteration, etc.
In other words, you're evaluating something the job doesn't actually require (typically), and you're potentially missing out on people who might be brilliant architects given a little time to absorb the problem and think through options.
patio11's comment reads more like a method for evaluating answers. You present the candidate with a boiled down description of a real problem and then evaluate what they say against the rubric. All that to say you answer the questions by actually having expertise in the subject area of the question.
By screwing up, owning it, and fixing it. It's impossible to get real operational knowledge without actually working on it. If you're in school, design competitions are awesome. If you're working in IT already, take responsibility for some processes.
I would suggest a combination of being a point person to solve production issues (either on a pager or being a developer who fixes problems after an outage), or a more involved side project (try keeping Wordpress running on a VM when something is on HN, etc)
1) Explain to me a system you built/architected. How it worked, what it did, etc. I'm looking for a deep understanding around the requirements of the business, and how that played into technical choices.
2) Talk me through some bugs you encountered or mistakes you made, and how you course corrected. I try to make a candidate comfortable prior to asking this and usually tell a quick and horrifying war story from my own career. Here I want to understand how a candidate processes information and adapts the plan accordingly. Like mike tyson said, everyone's got a plan until they get hit.
3) If you had to build the system again today, what would you do differently? I'm looking for an awareness of how technology has evolved since they designed the system in question, as well as what didn't work well over time from an operational perspective (e.g. we didn't burn off time series data over time so table index rebuilds started to take forever, etc). I want to make sure the candidate supported the system they designed over time, and felt the pain/benefit of their decisions. I usually specifically ask the candidate at the beginning to pick a system they built and supported for an extended period of time.
I also ask about areas of technical expertise, and areas where the candidate feels less strong technically. For the former, I make sure they have the kind of expertise they claim, and for the latter, I want to see how they communicate with team members that are expert in those areas (and how they gain familiarity with new technologies).
I solve this by being very clear with candidates ahead of time (i.e. days before the interview) about generally what I'm going to ask, and I've never had a candidate tell me they couldn't recall at some some relevant details from a project. I am generally asking about experiences that were impactful and the lessons learned, not trying to test someone's recall ability.
That helps. I've had questions like that sprung on me, and I just don't have any prepared war stories to talk about.
I try to look forward and not dwell on the past. Or maybe that's how I rationalize bad memory. Either way, when asked about details of something I worked on 8 years ago, I don't have much to say.
Yeah, my mind just seems to just abstract it all just into the lessons learned decoupled from its origin. The story is something I can rarely recall. But I was always more into talking about ideas rather than trading stories.
I'm really bad about this too. I actually got to the point where I googled 'most asked Personality questions in interviews' which had like "Describe a time where you showed leadership." etc, and I have a Google Drive document that has a paragraph answer for each, and I reread that a couple of times before going into an interview so I will have a few of these stories prepared.
I still would struggle if you asked too many more details beyond that, though. I had to dig deep into my brain in order to write those paragraphs in the first place.
I do the same with technical questions. Anything I get asked (that I remember afterwards) goes into a document (with paragraph answers) that I review before future interviews. I also try to anticipate some other questions and put them in there as well.
Because the scope of Computer Science is so big I still get questions I didn't prepare for asked of me pretty much every interview, though, and I struggle through those questions.
Despite this, I do have a good memory for work and can keep entire systems in my brain. But in doing so this other crap gets shoved deeper and deeper and are more difficult to retrieve.
Lots of good answers here (and in this thread in general). I like to focus on the person's history and dig in - war stories for sure. For a software architect you should get a lot just from their CV/Resume. In the interview you get to test how valid and deep this experience is.
To your list I'd also add some human factors. Some behavioral interview questions about dealing with difficult people, entrenched opinions, people under stress.
You've also got questions such as "what is your favorite/last book on this domain and what was your take away". Or more generally what you like to read and how you stay in the loop.
As with any interview, I like to make it as personal as possible. Tell me about something you have architected and why you've architected that way. I will ask probing questions about why you chose this stack, this database tech, this integration layer, questions about security concerns, areas for improvement, what you learned and would've done differently, etc., etc. This also allows you to see the interviewee's excitement and really turns the table to put them in a comfortable position where they are the most knowledgeable person, which makes for a much smoother interview. If that goes well, I may turn to some specific tasks or problems we have faced or are facing and discuss at large how those might be addressed, again really focused more on brainstorming than any "right" answer.
If you don't have any projects or work to talk about, the interview is basically over as I don't see much value in talking about cookie cutter problems.
This is great stuff. A fun bit to add is "That thing you made, if you had more time, how would you make it better? How would you make that version better?" And, so on until it start to get a little ridiculous.
I do the same. With people who actually know something an interesting conversation will develop.
A lot of "architects" will only repeat generalities and have nothing left to go deeper. I also don't like too many buzzwords like "integration patterns" or "serverless architecture".
For me, the single largest differentiation between "senior programmer" and "software architect" is whether or not they can see the actual problem. The catch phrase is "see the forest for the trees" but in software it is recognizing you're in a forest and then understanding forest type issues. I was on a team that was nominally building a "video encoding protocol" because that is what they were asked for but the problem they were solving needed a remote procedure call protocol.
Happens all the time, a person wants to buy a waterproof computer, but really what they want is to see the results of a program while they are in a rainstorm. Or they want a "website for managing inventory" but they really want a way of tracking employee theft. So architects listen to the problem statement(s) and think about what is in play, will ask some clarifying questions and then reframe as the actual problem statement. A senior engineer will ask evaluation level questions (how fast, how much memory, etc) and come up respond with a solution to the stated problem.
A bad software architect will not be able to stop abstracting and will start with a 50,000' view and then float into orbit. A "good" software architect will add in the business or operational constraints to the problem and articulate where the ceiling is and why.
>single largest differentiation between "senior programmer" and "software architect"
I am glad that you brought this up. My question was more from a perspective of distinguishing senior engineer vs architect interview questions. There is only a limited time in the interview. Shouldn't an architect be also asked the senior engineer questions in addition to the ones discussed here. Especially my our org setup they are also hands-on and individual contributor to the code base. One solution could be to restrict the 1st couple of rounds of an architect interview to senior engineer questions only . Fair ?
If you don't mind I'm going to go a bit 'architect' on you :-) Do you need a software architect or do you need a senior engineer. Interview questions are about finding out if you should say 'no', by definition if someone is called in for an interview you've tentatively said yes to them for the job you just want to know if you should change that to a no. So the problem set is understanding if the applicant in front of you will be able to do the job you need done.
Your comment that "in our org ... they are also hands-on" is the an echo of the Joel Spolsky 'architecture astronaut' type. Someone who spends all day architecting and none of their time coding. But here is the thing, at the end of the day its about how much product the organization ships at the end of the day and whether or not that product had does the job it needs to do right? So there is some nuance there to consider.
I certainly believe that anyone who considers themselves to have software architecture skills, should also be able to do anything a senior engineer can do caveat a deep discipline/subject matter engineer[1]. So interviewing for senior engineers will also get candidates who show architecture tendencies, and you will recognize them because of the questions they ask. If you want to provoke those questions then leave some of your coding or puzzle questions a bit ambiguous to see if they let their ideas play out or if they are sticking with the task.
Here is the thing though, you can have lots of really good programmers and all they need is a bit of structure around where they are going so that together they can make something awesome. That needs an architect who is spending all of their time keeping everyone on the same page, and if changes need to be made, moving to the new page. If your programming team is fairly senior and just "gets it" and will work out amongst themselves which parts need to fit with the other parts, then you just need another senior engineer like that. In my experience though group 1 is more common than group 2.
[1] There are irreplaceable senior engineers who have become so deeply expert in a particular technology you don't want them to do anything but own that code.
>needs an architect who is spending all of their time keeping everyone on the same page
Now here is he problem, the org needs someone like that but again the expectation is to contribute equally in the implementation during every sprint as his hours are accounted for sprint velocity. In this case I can't hire someone who does not write a complex sql or merge two linked list in code etc .
Most of th programmers are so young in this part of the world that Anyone having more than 9 years of tech would expect an architect position and there are high chances that is not actively programming in his previous company and thus it becomes nesssary to eveluate coding skills
Be careful you don't set them up to fail. Measure team power not individual power. Do you really care if they spend time writing code or not if the team is getting more done?
You should make sure they've actually stayed around in their companies a few years to see how it actually behaves. I've seen a bunch of architects that regularly move on to the next hot project, before the old one is properly completed. The architect never sees all the problems that come from their grand ideas so they dont get to learn what is practical. There is a big difference between architects that know all the hottest tech and buzzwords, and an architect that can deliver a reliable and maintainable platform.
I wish I could triple up-vote this. I've worked with a few architects who were great at envisioning fantastic, pure systems that looked great on paper. But, were nearly useless at actually building these systems and making them work in the real world.
That might be ok if the role is a pure R&D role, where they're "playing around" with the latest technology and ideas. We have some of those positions at my current employer. But, it's not very good for an architect who will be embedded with a team of developers.
I have interviewed and hired software developers for many years at different companies in different countries. Which has given me the opportunity to experiment with different questions and correlate that with outcomes (how good the people I hired or recommended for hiring turned out over the next 5 years). And I have now narrowed the questions down to 4 types of questions that really filter the good developers from the not so good: (1) Ask the candidate to dive into all major groups of software development (DB/UI/Testing/OS/...) and see how deep the candidate can go (2) Ask the candidate to come up with a practical solution to a NP complete real problem (usually an optimization problem) (3) Ask the candidate how he/she would approach investigating and solving a specific real problem for a large real system (usually a mobile-server-DB system) and (4) how they would architect a system given a list of real use cases. Candidates that do a good job of answering those questions are highly likely (as in 95%) to be successful if hired. They don't have to come up with a perfect solution. What is important is how they approach the problems and what kind of possible solutions/investigations they come up with and their reasoning behind it.
An architect's role often has an economic as well as a technical part to it. So, it's important to find out whether your candidate has a strong loyalty to a particular vendor stack. Ask.
After you've had a conversation about a project in the candidate's track record, ask some pointed questions:
-- what would you lose in performance and functionality if you reworked that project to run on cheaper infrastructure? (By that I mean such things as "standard edition" rather than "enterprise edition" of DBMSs, open source vs. paid licenses, etc.) This is a good way to get a handle on the candidate's economic reasoning ability.
-- what was a significant performance bottleneck in that system? How did you address the problem?
-- with the benefit of hindsight, what parts of that system would you say you overdesigned? underdesigned?
-- what was a security vulnerability in the system? How did you plan for security? If you had to do security remediation, please describe it.
I normally go with: "If you could do it from scratch how would you implement the infrastructure of X"
X being a currently widely known service (I'm usually going with Spotify). There is no "right answer" to this question but this shows you how the candidate thinks and how well they can protect their opinion if challenged
I think this is a good question, but that it falls prey to the same conceit that befalls those who claim they could "build Twitter in a weekend". A picture-perfect sketch of a system can illustrate one's knowledge of patterns and off-the-shelf solutions, but it won't cover the things that get in the way of a team building their way towards product/market fit.
You could lead with this question and then jump in with hypothetical roadblocks to see how the candidate reacts.
Ex:
- "A network of microservices _would_ be a clean way to implement this. How would you foresee the operational burden of this impacting a small team?"
- "Imagine that you build this and it works well, but the product it powers ends up not resonating with customers. The team now wants to pivot to X. What changes would you make to the system to address this new problem space?"
> it falls prey to the same conceit that befalls those who claim they could "build Twitter in a weekend"
A good answer would account for the fact that our initial approach to big projects tends to differ quite a bit from what the the end result is. Agility of architectural design is generally underrated in my experience.
One of the fun things about these questions as an interviewer is all the answers end up a little different, so you end up learning something. While that's not the prime goal, it is nice if a candidate uses it to teach you something :)
I would think that a good answer showcases your knowledge. E.g. if you knew a lot about relational databases, this is a time to apply the knowledge.
It's also good if you show the ability to see trade-offs where they exist. E.g. the trade-offs for a relational database vs. something else might be familiarity / time to implement, vs. ease solving some type of situational performance problem.
The ORCA card system in the Seattle area is a great architecture question. You have a bus/transportation system that has:
1. RFID cards that tie into
2. Electronic wallets
3. The busses are not always internet connected
4. You need to be able to refill your card either online, or via kiosk.
5. You also want to be able to refund card balances.
What system can solve this? What drawbacks would the system have? How could the system be scammed?
Bonus questions are: "6. The LINK rail allows you go scan your ORCA card when you get on and get off the train, so you only pay for the number of stops you're on. How would you include that? 7. ORCA cards also allow for free transfers-- so if you take a second ride within 3 hours, it will be free. How would you include that?"
This is specific to my codebase I was interviewing candidates for (heavy OO). Not all of these concepts are necessary for the candidate to understand, but a few go a long way towards a great developer experience when applied correctly:
HN doesn't seem to like mine, but I've always asked "tell me about some project, can be a tiny shell script, that you did entirely by yourself: the code, the docs, the install". It has to fit through the filter of nobody asked you for help, if they sent you a thanks, that's cool, but they need to be able to install/learn/use it on their own.
If someone is claiming to be an architect and they've never done that, that's a red flag for me. Kinda like asking a building architect to talk about framing. You'd be surprised at how many of those guy have never swung a hammer and they "design" stuff that can't be built. Sort of the same with software, IMHO.
As someone else says, it depends on what you define as a "software architect". A co-worker from a former company that was a "technical fellow" and "technical director" (effectively an architect, at a 12,000 person tech company), came over to Google and just has the title of "Software Engineer". From the companies I've worked at, software architect covers a lot of different job descriptions.
At Google, more senior software engineers will have a "system design" type question as part of their interview. There are sample problems for this all over the internet[0]. For example, you could ask them "Design a URL shortening service like bit.ly.". When you ask open ended questions like these, there are so many parts of the system you can dive into that you can really test the depth and breadth of a candidate's knowledge.
If I remember correctly, the system design interview at Google is for software engineers candidates of all levels, not only the more senior ones. I was definitely asked one of these problems when I interviewed as a newgrad.
On the subject of writing code - the question is really trying to find where on a spectrum someone is from:
"Code, I don't write code, I draw UML and throw it to our offshore development partners"
through to
"I always keep all the most complex and demanding parts of the development process to myself as everyone else is an idiot and only I can realize the amazing architectural vision I have <maniacal laugh>"
[And yes, the space probably isn't one dimensional and those probably aren't the endpoints] :-)
I've known extremely successful "architects" (in terms of £££$$$) who fit both of those descriptions...
Well, I don't think interviews are always about answering something correctly. It might just be nice to have an applicant's opinion on x y and z, to spark conversation, gain some insight into their personality and what is important to them, etc.
In my opinion, discussions like that can be invaluable: sure, someone might have all the technical answers correct, and if so that is great. But humans are more complex than just being right or wrong. It's also good to see if the candidate has a similar view on the role they would be fulfilling.
3) Debugging - given a failing test fix the code so the test passes.
4) Coding - Straight forward problem to code and create.
For the peer presentation, we are expecting the candidate to be prepared. They have lots of notice about this portion of the interview. We want them to be articulate and clear about the technical problems, project management problems and unexpected issues.
System design - we come up with a sample issue that is easy to articulate quickly. For example, "we are going to design the first version of Wordpress". We then ask questions and extend the problem during the interview.
Debugging - we give them a random open source project and we break one of the unit tests. We then ask the developer to track down and fix the issue.
Coding - smaller scale than the System design question but focused on determining if the candidate can construct a small program end to end. So this is not a portion-of-a-program, but rather construct a quick command line program to do X.
For junior people we drop the peer presentation and add additional coding questions.
--------
I liked this interview the best of all the places I interviewed. There was no "gotcha" and the problems I was asked were relevant to Evernote's business.
Walking out I didn't feel like I did the best I should have, (I have high standards) but I felt the problems were incredibly fair.
Now that I am at Evernote, hands down where interviews go badly are in the debugging section.
The peer presentation shows people who have not thought about the "why" behind their coding / design decisions - and it separates the juniors from the seniors.
Lastly, we expect and can demand a far higher interview performance because we tell candidates how to prepare. If they don't prepare, it shows soooo painfully. For candidates that do prepare we can have a more interesting and engaging interview that allows us to sell Evernote as a company.
I once received the question: let's architect a web browser.
This was for an interview for backend engineering at uber, so by no means did I have any particular knowledge or speciality in browser design. Because I knew the product very well from a user's standpoint, and common knowledge about web standards, it was really fun and exciting to figure the architecture out.
Typically the questions revolve more around: "How would you design twitter/pinterest/facebook newsfeed?"
Thanks for mentioning that one - it means I get to think of a sensible answer rather than convulsing in laughter for a few moments. It did involve physical buttons for "Burger", "Beer" and "Porn".
Asking them to whiteboard out software design thoughts, risks, etc, for things we all commonly encounter can be revealing. Things like an ATM, slot machine, bank of elevators, red light cameras, and so forth.
Toy problems get toy answers, you'll learn nothing that way. I couldn't care less about traffic lights and elevators, and I can code circles around many.
- Tell me about a problem related with software architecture that you find interesting, and why. Take your time.
- Tell me some pros and cons of a library or framework that you have used recently.
- Here's an hypothetical problem (describe the problem). How would you approach solving it, from the point of view of software architecture? Please think aloud and ask questions, the idea here is getting to know how you think, not necessarily arriving to a perfect solution.
Software architecture questions are an interesting topic because unlike software implementation, you cannot simply give someone a task and ask them to demonstrate it. Architecture is not just a matter of selecting components, it's about understanding conflicting roadmaps for multiple teams and available human talent and technical resources.
My favored approach for it has been to ask people questions that mirror problems I've been forced to solve in the past at the current venue you're hiring for. I think this approach has a number of advantages:
1. You know how YOU solved the problem, so you've got at least some qualifications to discuss it with authority.
2. You can answer any and all questions the prospective architect wants to ask. These questions and their quantity are often even more telling than any hastily described solution ever could. Do they ask about infrastructure? Do they ask about expertise? Do they mention specific tooling you yourself decided on?
3. This gives them a chance to wow you in a controlled environment. If their idea for a solution is better than yours, you know you have a very high value candidate.
4. This also is a supreme test of technical communication skills. They need to rapidly get YOU to tell them things by asking questions, formulate an idea, and then communicate it to you in a way you can understand.
Of course this assumes the interviewer has done some architecture, but I think that's probably required to do this job right.
Personally, I think all this talk of "war stories" and whatnot is not a very good idea. If you're concerned about someone's personality and grace under pressure, you should treat that separately from their technical skills. That way you can weigh those attributes separately and fairly when it comes time to decide if you will hire the candidate.
Give them a sample, 'The company you thought you're interviewing for is just a smoke screen, we're actually a stealth startup for a social network for pets" and have them diagram out how it might look for a MVP launch, 2 years in, 5 years in. What technology choices, how are they going to launch internationally, scale to hundreds of millions of pets, how are they going to achieve sub-second latency for the FindMyPet API, high-availability, do operational and business intelligence monitoring (and which BI metrics would he care about), unifying code across multiple platforms, etc.
Most importantly they should light up when they get the question. Like someone just sat a piece of cake in-front of a fat kid.
These look amazing, and some really high profile projects as well (Asterisk, Audacity, Eclipse, LLVM, Mercurial, Git, GDB, MediaWiki, etc). Thanks for that! I wish there were more books like these.
You have written some code you'd like people to be able to reuse. Should you share it as a library or expose it as a service? Talk me through how you'd make this decision for a few cases and what the pros and cons are for each choice.
For architecture I prefer to just ask for input on an actual issue my org is currently facing. Perhaps abstracted a bit to make answering less of a slog through details but I don't see any advantage to using some made up scenario. By using current issue I'm in both a good position to interact on the subject since its top of mind and get a sense for the relevance of the candidate's experience and/or their creativity. Also means I don't need any better rubric for evaluating their answer than "if I'd heard that in a meeting today would I have thought 'oh there's a good idea...'"
I ask them to design a structure for a card games that uses a standard deck of playing cards, and how they would implement shuffle (not the algorithm itself) and dealing and hands. This is simple. People manage to grossly over engineer it.
My ideal response would be one struct with nothing but a suit enum and int value. A deck is just an array of these, as is a hand. But usually people start with a deck class, then try to reuse that for the hand, discard pile, etc. Things get complicated quickly, just like in real world problems, and a good architect can keep that in check while still providing useful abstractions.
Ask them about a project they did. Let them talk. Ask questions about minor details that seem interesting and (hopefully) relevant to the position you're trying to fill especially when part of their explanation is hand-wavy ("yeah, we used Kerberos because that seemed like the right thing to do." "Why?" "I tried other things. They didn't work." "Why?" etc.)
Ask a bit of trivia about the stack you're hiring for. They don't need to know the right answer, but their reasoning about it should corroborate with their experience.
I have been through a few recent interviews and the technical questions seemed like they were from a college exam.
My advice is to see how they operate... Give them a problem, even a vague one, but then solve it together, have them use google to seek out ideas. See how the person approaches it, don't expect a bunch of memorized answers off the top of their head.
The big thing is to see what questions they ask. How will you and the person collaborate in a normal working environment.
The personality and adaptability are far more important than memorizing the exact command line or function call.
Looking over their public work experience and perhaps open source contributions you will already know if this person will be a good fit.
When tasked to interview I have found that asking a question which you know the candidate will be unable to answer will show you the most-intriguing parts of their character. Something deceptively simple like a puzzle or logic problem but scales up quite quickly.
This is when you can learn things about them that they would not put to paper and thereby determine if they are for you.
My favorite way is to start with a very vague design question, with no expectation of any predefined answer. Then I try to make it a discussion, see if the candidate asks more questions, pokes around, can think in terms of abstractions - I don't expect any of it to match my own thought process.
Another question is to pick a project they've worked on, and ask them how would they do it if they were given a clean slate.
'What do you expect to be involved in as a software architect and what can you contribute?'
and take it from there. If you are interviewing for that role then surely what they do in the context of your organisation is key and open questions help you understand whether the person is a fit for that.
One of the twelve principles of the Agile Manifesto is "The best architectures, requirements, and designs
emerge from self-organizing teams." Does your style of architecture fit well with self-organizing teams?
One of my fav is to ask their preference on Composition vs Inheritance. It's a fundamental question that will give you lot of insights on how they view building applications to last.
When interviewing senior people, I like to get them going on war stories. "Can you talk about a project you experienced that went very badly?" is a good one. It's not necessarily that they went badly, but we've all been on death marches before (and I absolutely would not hire an architect who's never done a death march!).
The beauty of this is you can find out how they react to impossible demands and pressures. What they're proud of accomplishing in dire circumstances. Possibly what they're ashamed or embarrassed about, or mistakes/lessons learned (if they're that bold). You can learn a ton about how they talk about their colleagues! Are they full of praise, or contempt? Do they find someone to blame, or talk about how someone else saved their asses? You can also learn a lot about their actual approach to technical decisions - their taste, for lack of a better word.
You can ask probing questions here, but the important part is to get them speaking in an unguarded way. Nobody makes it to the architect level in this field without at least kind of loving the job. Find out what they love. Find out what they hate. Find out what they regret. Find out any strong technical biases they have, and why. But most of all, find out how they play with others. Because the architect's job isn't just to design, but to communicate. You don't just want a technical expert. You want someone that can value and respect the rest of the team, or they're just going to cause problems.