My honest feedback below from the perspective of a hirer. I'll start by saying I hate takehome stuff, for exactly reasons like this. It wastes everyone's time. They're fine as a 'last step before hire' thing, but not as a filter.
1 - Too much chatter. Part of the assignment is using judgment and working in ambiguity. I probably would have ran with what was given enough to knock out something small and local in an evening or two. Asking questions is usually fine, they even welcomed it, but seems counter to the original ask.
2 - Writing and sharing a proposal seemed like way overkill. You have to remember that these companies are now getting hundreds if not thousands if not tens of thousands of applicants, that is a lot to deal with if everyone does so. I think it's a bit of a disconnect...you feel like you're going above and beyond and being thorough, they feel like it's being a bit long winded and wasting time.
That probably explains the nonresponse.
3 - The finished product seemed functional, but seemed a bit overkill on the infra and polish. This is probably a good thing to work with you, but ended up wasting a lot of your time if not being selected, which was the case.
4 - Maybe I missed something, but the requirement asked for terminal inspired. I'm not quite sure precisely what they meant by that, but didn't see any possible interpretation of that in the result.
Anyways, hope you don't take it too negatively or personally - you obviously are a talented individual and moreso seem to really care about your work. Just wanted to play a little devil's advocate with a different perspective.
The attitude of the blog writer in their interactions also feels off. Just reading this blog post makes me think that this person is difficult to work with, requires extremely clear guidelines and instructions, and has a hard time making their own decisions. Maybe this is a good fit for a large, established company, but startups have their own needs.
"Create a terminal inspired email client so we can do an alpha test with some customers" is a reasonable ask for an engineer at an early stage startup. Of course, there would be a bit more specification, but a lot of the details would still be up to the engineer. This applicant wants more certainty than they can get.
This is illustrated by the line: "I would like to know what kind of response I could expect from Kagi if I drive it to completion." This is not a great request to make. There's no way they can answer that question, because there is no certainty available. They're probably getting a few hundred or a few thousand more submissions to evaluate.
In your comments history someone replied to a job posting "Just an FYI, if you do a takehome project for William, he won't respond to you (not even with an auto reject)."
Seriously? A candidate puts in a week of work and he can’t be guaranteed a 30 minute discussion?
Nobody is getting a few hundred or few thousand submissions to evaluate. Nobody. If you are getting 1k applicants, at best 50 are asked to do a take-home and even then, not all at once.
If by some miracle 100 people did this to completion at the same time, there should be a notice to the effect that due to high volume, blah blah blah.
…which is totally understandable… if the hiring manager had communicated that. They could’ve easily mailed back “Hi this proposal seems much more detailed than what we need for evaluation, please save yourself some time and energy.”
The author may have had issues (I personally don’t count “need clear instructions” as an issue - edit - I see they didn’t adhere to the TUI prompt), but the hiring manager definitely did.
> …which is totally understandable… if the hiring manager had communicated that.
I agree that the hiring manager could have handled it much better, but as a rule: If at any point during any hiring process you feel like you need to spend even close to a full time week of work on anything without being very explicitly told so, you are wrong.
I am a hiring manager and we do take home homeworks. I fully agree that this is a key piece of communication. I always take the time to tell candidates that although they have as much time as they want, we expect 2-3 hours of effort at most, to be respectful of their time.
Without that, take home problems would seem predatory.
Yes on this: It reminds me of when candidates ask me how they did at the end of the interview. It shows an extreme lack of decorum and empathy. What if you did terribly and I wouldn't hire you in a million years? Do you really want me to tell you that?
There's no good answer and asking a question like that shows real narcissism imo.
Conversely, if you can't handle some straightforward feedback to a candidate that took the time to interview you without violating decorum or hurting their feelings, then how can I expect you to be a good manager or supervisor? How are you possibly going to be able to handle minor personnel conflicts or provide guidance during the training period? It comes across as a complete lack of basic managerial skills.
Okay, but basic interpersonal skills are a prerequisite for anybody in a senior or team lead position, or any position that will involve code reviews.
I'm sympathetic to how awkward it can feel to provide honest feedback to a candidate, but look: we're all people here. I think we forget that sometimes when we're assembling hiring processes. As a candidate, you need some kind of feedback mechanism that allows you to improve even if you're not a good fit for a particular organization. And if you're involved in the hiring process in any way, you ought to be equipped to handle that.
> you need some kind of feedback mechanism that allows you to improve even if you're not a good fit for a particular organization
"Need". That is a strong term. I disagree. It would be nice, but it is not a need.
This topic has been discussed ad nauseam on HN. In most companies, there is specific company policy that prohibits providing feedback to candidates. There is literally no upside for these companies to provide feedback to candidates that they reject (except Fake/Feel-Good Internet Points, only redeemable on HN forums). Really: There is no way around it, no matter how many tears are spilled about it on HN.
This is simply a defense of bad policy couched in unnecessarily dehumanizing language.
There is widespread resentment of this and many other common hiring practices in the tech sector, and that is further impacting both the quality of candidates as well as employee motivation and satisfaction. The upside for companies is higher quality candidates whose first experience with the company is a hiring process that makes the candidate want to work there.
I broadly agree with this being an unfortunate outcome but you do understand that making candidates who failed your interview want to work at your company is fundamentally limited in how much it actually helps you. Yes, yes, I know some of them may come back and pass the next time, or they tell their friends about how you were super nice and gave them great feedback, but this is pretty rare. If you're doing this, you're doing it out of the goodness of your heart, not because it helps your recruiting pipeline. And, even though I agree with the idea of providing feedback, assuming that people will have positive feelings when you tell them why you didn't accept them is misguided. I have friends who I know personally that have gotten interview feedback and not taken it well. Of course I tell them to shut up and stop poisoning the well for everyone else, but the point is that this is largely not the picture you are presenting it as.
Sorry, I wasn't clear. Providing constructive feedback to a candidate is unlikely to have a direct positive impact on the relationship between that specific candidate and that specific company. It's more of ... whatever the opposite of the tragedy of the commons is. A policy that, if improved, would broadly improve the quality of many candidates for many companies.
Companies have been optimizing for candidates that are an immediate ideal cultural and technological fit. They are all competing for candidates that are the idealized developer, with perfect social skills, a brilliant CV, and deep technical experience that is an exact match for whatever the company is doing at the moment.
That's fine and rational and all, but a necessary consequence of this is that that pool is quite small and there are lots of companies competing for those people. Meanwhile, there are a lot of very good candidates who are underemployed because they aren't getting the opportunity or resources needed to become those idealized employees. This is a game theory outcome where both parties are optimizing themselves into a losing position.
I've been employed in this industry, off and on, for a long time. I assure you that companies didn't always behave this way. There has been a clear, obvious, and severe decline in the hiring experience, and these policies are hurting the entire industry.
It's generally socially frowned-upon to go on a couple of dates with someone and then ghost them. It happens, but it's not considered good practice. We recognize that it's cruel but also leads to a more cynical, detached, overall worse dating experience for everyone. Saying "I don't think this will work out, you seem nice but you're not what I'm looking for right now" is difficult and awkward, but it's also a necessary skill that needs to be maintained. Sometimes people don't react well, but that doesn't make it less necessary: it closes a feedback loop that ultimately allows earnest people who are looking for relationships to learn and grow and become better candidates for the next relationship.
> In most companies, there is specific company policy that prohibits providing feedback to candidates. There is literally no upside for these companies to provide feedback to candidates that they reject
This is the long and short of it.
In the US at least, discrimination laws are expansive. You can -very- easily end up saying something that violates this and putting your company at risk, no matter how good hearted you were attempting to be.
How do you "accidentally" end up saying something that implicates you in discrimination on the basis of legally protected characteristics - what are some examples of that?
This has always felt like an excuse used by people who who just don't want to be caught in their own lies when asked to come up with a real, non-discriminatory reason.
The other comments gave good answers. A lot of people think it means saying something horrible and racist or something, but not at all.
As one pointed out, there's a "well you said it was X, but person Y who got hired did that too. And they're a different race or gender or religion, so that leads me to believe discrimination."
There's also you trying to be helpful, saying something along the line of "well you hesitated a bit and sounded unsure in your answers", only to find out they have some disability that caused that and now have admitted you're discriminating based on it.
Maybe you'll say "well, if I had known, I wouldn't have noticed it or cared." And a lot of candidates would likely say as much up front. But they don't have to tell you about it at all. See how that creates a weird dynamic?
Is it common? Probably not. But it obviously happened or else such rules wouldn't exist. It's one of those things that the bad actors ruin it for everybody. Bigots are never going to admit their reasons - good people will. But bad people will always try to take advantage, regardless.
I think it's more of a case for legal and HR being conservative and super defensive. Not sure if you've ever handled a contract with an internal lawyer, but in my experience they often go for crazy suggestions that the other side would never accept for the sake of protecting the company as much as possible. Might be the same here - HR/legal being super protective and the hiring manager not caring enough to fight back.
> How do you "accidentally" end up saying something that implicates you in discrimination on the basis of legally protected characteristics - what are some examples of that?
Say you say it was for failure to meet a specific performance standard (because that is the documented reason); then the ex-employee has a starting point for an discrimination claim by looking for evidence that trnds to support the claim that people who differ on some protected-from-discrimination axis who failed to meet that standard were not fired. No reason given, no starting point. In theory, this policy helps make false nuisance claims more work and less likely, but a substantive reason for it is that HR knows that they cannot eliminate all prohibited acts by managers that would create liability, so making it harder to get a starting point for gathering evidence is important to prevent valid claims from materializing. HR policy does not exist to protect employees from unlawful treatment, it exists to protect the company from liability for such treatment. Sometimes thise two interests align, but when it comes to information about firing decisions they do not.
There’s similar things that can be done with other prohibited reasons for dismissal, loke retaliation; but the idea is any information you give makes it easier for them to make a case against you.
This is also, in reverse, why, as a departing employee (whether departing voluntarily or not), you should never participate in an exit interview or, if you must as a condition of some severance or other pay or benefit, never volunteer any information beyond the bare minimum necessary; one significant purpose of such interviews is to document information useful either for potential claims against you or to defend against any potential claims you might have, including those you have not yet discovered, against the company.
Part of it is, if anything can be taken slightly out of context to imply something discriminatory, there are those who will abuse the system and sue. At a large enough scale this can become a real problem. If the company policy is "never say anything" there's nothing to be taken out of context, reducing the chance of a lawsuit.
I bet you this comes back to insurance, as many things do in the corporate world. Sufficiently large companies probably have insurance coverage for discrimination lawsuits, or at least employment disputes in general. The coverage probably costs less if you have a "no feedback" policy.
Who are you trusting as a technical interviewer if you don't already trust them to give negative feedback internally?
Do you not code review? Are you a rubber stamp "LGTM" shop that should just be pushing to main but cargo culted the ceremony because github has it built in?
I had someone email me after being rejected at the final round of an interview. "Everything seemed to mesh just perfectly, and I'm at a loss to understand."
I broke it down for them. "This was nothing to do with you, and we would have had no objection to hiring you. However, the candidate who beat you out simply had more domain experience in XYZ area" and went on to say "For what it's worth, we had 500+ applications, of which we in-depth reviewed 100 resumes, had 40 first-round interviews, 15 second-round, and three final round."
They emailed me back to express appreciation and that though this didn't work out, it renewed their confidence to know they didn't "mess something up".
Since then, if we're at that point in a process and I'm rejecting you, I'll at least give you something to work with.
This is so important for people to understand, and its why I give people feedback.
People, being humans and prone to pattern seeking, assume that if they didn't get the job, it's something specific they did, or failed to do.
And sometimes, that's true. But for a lot of candidates, it just came down to another candidate being slightly better, or slightly cheaper, or some combination of value markers.
A lot of my interview feedback comes down to "I don't see any reason you wouldn't be a good fit, but we have other interviews and it's going to come down to value."
Some people will take this as me saying "Don't ask for what you're worth," or "we're gonna low-ball your salary." The reality is, we're a business, and if I can produce the same widget with person X or person Y and person X costs 10K less a year, I'm going with person X. Every time.
Yes we want to know. Framing this as an empathy issue when in reality you're just to afraid to be honest or afraid of any kind of conflict IS an empathy issue. At that point they're not a person. They're an annoyance that you want gone immediately.
Don't take this the wrong way, but I deliberately ask how I did because it helps me weed out interviewers who think like this. Not so much "how did I do?" as "now we're close to the end of the interview, do you think we're a good fit for each other?" I give my own feedback and talk honestly about points of friction.
I interview pretty well, but if I go into an interview with a company that wants hungry hustlers and I've spent the whole interview talking about kindness and team spirit, or if you think I don't know enough pl/pgsql to deal with your gnarly legacy backend, or I'm getting the vibe that none of the engineers seem to like working here, then we need to speak honestly about that.
No need. Just walk away. Remember: You are interviewing them, just as they are interviewing you. Any company worth its weight will not allow red flags to leak into the interview process, e.g., "getting the vibe that none of the engineers seem to like working here". So many times, I have reached the final round of an interview process, met the senior manager... and thought: "Barf, I don't want to work for that person. What a waste of my time."
I know I'm interviewing them. That's why we need to talk.
If they wanted to hire me enough to interview me, but at the end of a half-day of interviewing I'm going to walk away without a job, then they need to rewrite their position description so I know not to apply, deal with their morale problem, or directly ask me how much PL/pgSQL I've done. We both stand to benefit from talking about how the interview went.
But you also need to factor in their position in the situation right?
Like suppose they do hate their job. Do you expect them to speak that plainly and honestly to every candidate who asks "So how do you like working here?" and risk getting that posted to the front page of HN?
You're asking them to risk their own livelihood so you get a better signal for your own job search, that doesn't seem like a proportional trade to me.
Obviously I'm not advocating for complete opaqueness, but your interviewer is hardly ever in a good position to part with their true feelings towards questions like "How did I do compared to other candidates? How is it truly working here?"
I've basically almost always given direct and obvious non-answer to the first question: "I cannot tell you right now, because I'll need to write down and collate my thoughts. And I'm not allowed to share feedback directly, so your recruiter will be in touch with the feedback afterwards."
> What if you did terribly and I wouldn't hire you in a million years? Do you really want me to tell you that?
Yes, that sounds like extremely valuable feedback.
Why do you suppose asking a question like that shows narcissism? To me it shows a willingness to infest feedback to improve.
I will add the caveat that if someone asked me that in an interview I would likely give a non-answer because I’m not totally sure what all I’m even allowed to say.
A simple request for feedback is not evidence of narcissism or lack of empathy. Could be anxiety. Could be curiosity. Could be zeal. Could be any number of things. It's certainly not an "extreme lack of decorum" though.
It's okay to avoid giving feedback if you don't want to. I can think of a few ways to answer that question in a neutral or positive fashion to defuse the situation and legally protect the company.
Not to mention there are legal liabilities with sharing interview performance with candidates. "Oh but the interviewers told me I did extremely well on their interviews. Therefore it must be the case that I was rejected because of ${protected attribute X}."
Really?? I always appreciated candidates that would ask that at the end - being willing to step aside from the pretense of professionalism to ask a real question and listen to my answer is a signal to me that this is someone who is willing to be real with me, not pretentious or perfunctory.
I do get what you’re saying, but I disagree, there is a good answer; and as is often the case, it’s an honest one.
> asking a question like that shows real narcissism imo.
Precisely the opposite. Asking for criticism and genuinely being interested in what others think of you with the goal of taking the feedback on board and improving is the polar opposite of typical narcissistic behavior. As far as I'm aware that sort of self-reflection is inherently incompatible with NPD.
In other words, the author did a good job but failed because there was an implicit requirement to not try very hard.
> Part of the assignment is using judgment and working in ambiguity.
If trying to clarify requirements is not what they wanted here, they may as well ask candidates to pick a number between 1 and 10 and reject anyone who guesses wrong.
> overkill on the infra and polish
I know its an opinion, but hard disagree on both. Take home tests are intended to show off your ability. Without specific guidance, how can anyone guess how much showing off is expected?
Polish is only bad if it stops you from delivering. Rejecting something that was delivered for being "too polished" feels like you are saying someone did too good of a job.
---
In my opinion tests with vague requirements like this are more likely to be a different way of rejecting people who are not a "culture fit".
The author did a bunch of things that were irrelevant to the product (deploying to Fargate using Pulumi) while not delivering on the basic features that they were asked for ("terminal inspired email client"). If you watch the video it’s a super basic web app that has nothing to do with the TUI apps Kagi named you should use as inspiration. There’s no drafts. No CC or BCC field. No folders or labels (including sent or trash!). No replies. No threads. No shortcuts. There’s not even an indication of whether an email has been read or not. In terms of features it is very very very barebones, and I’m sure if you asked a few users all or most of these would be table stakes. Yes we can agree that "terminal inspired" is very open to interpretation, but spend a few minutes using the apps they gave as reference and you should be able to come up with a long list of features besides "send & receive email", which is the main thing they demonstrated. I get this is a short challenge, so pick a handful of things that are quick to implement and focus on those (like unread markers). You can even say this is a backend position so the UI shouldn’t matter, so then just implement them in the backend and make the simplest UI for it you can! They certainly "delivered" a product but I would not call it polished, and I would not confidently say this was a "culture fit" rejection - I see plenty of other reasons
The hiring manager above thought it was too polished, but you think it is not polished enough.
> There’s no drafts. No CC or BCC field. No folders or labels (including sent or trash!). No replies. No threads. No shortcuts. There’s not even an indication of whether an email has been read or not. In terms of features it is very very very barebones
None of these were asked for. Your guess as to what the true requirements are completely different from the authors, and others in this comment section.
Everyone seems to have different opinions on why the author failed. No one should have their time wasted on a take home test destined to be rejected because they guessed wrong.
And in this case, the author explicitly asked if they were on the right track before spending time on the test. Kagi had the opportunity to reject early or nudge the author back in the expected direction. Instead they wasted everyone's time.
I don't see how this can be interpreted in a positive way.
The candidate here focused on the wrong things, causing it to be overly focused in some areas while ignoring the main task which revolved around creating an interface inspired by a specifically listed set of terminal mail applications. So, yes it was both polished and unpolished simultaneously.
I think it's pretty obvious to figure out how much polish is "too much". This is a business. Engineers time is limited. Was the time you spent on that more valuable than the alternative? I'm leaving the question of what the alternative was ambiguous on purpose.
What they were looking at was not the code, but the attitude. They likely don't want an engineer with a propensity to do too much, unless it's a 100x coder who can do "too much" with a lightning speed. Usually they want he candidate to show the ability to quickly do as much as needed, and not more, and, crucially, to understand how much is needed.
So yes, this is a culture fit test as much (if not more) as a design and coding test. Some people who are great at design and coding would fail it, and it's how this filter is intended to work.
Yes, I needed a ridiculous number, a picture of the coding faculty so strong that writing twice as much code as the task really needs is hardly even noticeable.
IMHO, the author's approach of validating his ideas mirrors modern engineering workflows. Coders don't spend hours independently coding an MR and then getting feedback from prod, tech leads, QA, and UX after the feature is "finished."
Work environments that "code first, review later" have been some of the most toxic in my career. It really sucks when you spend days building a feature only to find its not wanted. Which is why explaining the feature in English and getting approvals is the industry standard for shipping projects.
This candidate followed modern software practices that healthy workplaces follow.
(I'm also a hiring manager at a 1k+ engineer company).
I've worked at large companies and I've worked at small ones (disclaimer: including Kagi, long ago). When I was at Google I wrote design docs for basically everything. The company I'm at has fewer than ten people. I rarely write design docs. This isn't because the workplace is any less healthy, but because there are fundamentally fewer people I need to communicate my changes to, and they have more context of what I'm doing.
Process is healthy at large companies, because they move slowly and communicating your changes often becomes far more important than the actual code that needs to be written. Kagi is a small company. It does not make sense to cargo-cult practices from a company a hundred times larger.
I found myself thinking the same thing - perhaps came from a bigger/more established company. Reminded me of the PRD/ERD process. And that's not necessarily bad.
But from my experience, the two types look at each other like the other is insane. If you write up an ERD at a scrappy 6 person startup, everyone is going to think you waste time.
Conversely, if you join a larger team with established processes and begin flinging code at the wall unabated, people are going to think you're reckless and possibly inexperienced.
I've worked in places with very strong product management, where every single detail must be approved by a PM. I've also worked in places where engineer has a lot of autonomy - "John from FOO dept is spending too much time wrangling the daily data updates. Can you help him, here is his email? Our higher-ups allocated two weeks for that, but we can extend this if needed." - and from then on, you are in full control of the design and implementation, subject to your team's rules (because no one wants a system with bus factor of 1).
For me, I've found that the latter places are much nicer to work in. The interview question seem to focus on latter situation too.
I don't know about everyone else, but as a dev who is currently looking for a job I simply cannot afford to devote a week of unpaid work to each application. It's not sustainable to say the least. Especially when I'm told by some people that getting rejected means I didn't spend enough time on the solution.
If the next guy puts in 10 hours, and you only put in 1 hour, assuming equal skill. Which project will be more polished?
If you are high skill and only work 1 hour on the project, but a newbie puts in 10 hours with ChatGPT's help, I'd be the newbie would have a pretty competitive project to the skilled 1 hour candidate.
And yet the over engineered solution that somehow took a week to complete fell completely flat. I'd guarantee that Kagi has hired developers through this process who only spent a couple hours on it. If I was working on this, I wouldn't have gone beyond three hours. It would have actually been a console app and I would have done something like integrate your unread mail count in your terminal prompt as some "bonus" demonstrating additional thought in the space.
I've worked with IMAP and POP libraries before so am familiar with the fundamentals for building a client and libraries in many languages make this part of the integration very straightforward. Couple that with a "modern" CLI library this should come together very quickly. And I would not have included half a dozen cloud services for a terminal like email client. The project submitted completely missed the mark and they still have no idea why.
If I wanted to create something by meticulously planning out every detail and spoon feeding them to a code monkey with no creative input I'd just use an LLM or outsource to India where you've got to spell out every little detail and still get questionable results back. I've had to do that plenty of times and I don't want to work like that. I want to work with other professionals who can run with a concept and deliver good results without constant oversight and micromanagement. That's clearly not the author of the blog post.
I think you might be missing the point of the blog post. The author is frustrated that he followed standard engineering practices (by submitting a proposal for approval). This proposal was perceived to be as accepted and then when they delivered.
The issue isn't his solution is incorrect. The issue is the company's said his solution was correct and then they rejected him anyways.
This is the equivalent of the interviewer telling you the brute force solution on a leetcode is good enough, but then rejecting you after the interview.
no, the company did not say that his solution was correct, that's just what the author thought.
The company said that _the parts he wrote in the doc_ would not negatively affect his scoring. But his doc did not contain many parts that the company cared about - read it yourself, and try to answer the questions like: Will there be a per-email indicator? Is there a "sent mail" folder? How will user be notified of incoming email?
You're not supposed to deliver a polished project, you're supposed to show what you can do and what you know. You can also document your process if you want, to let them know it only took you X hours to get to the solution you are presenting.
I had an interview once where they gave you 1.5 hours to code Tetris in ruby. I completed the core requirements and 1-2 extra requirements.
The person that got the offer coded 1-2 more extra features than I could in that time. What am I supposed to tell the interviewer?
"You're not supposed to get a polished project?"
If you're the interviewer, are you going to hire the guy that did just was asked or you are going to hire the guy that worked nights and weekends to get the perfect project delivered to you beyond what was asked?
Exactly. The position they are hiring for is not that of a coder; coding is just one of the skills the position requires, maybe not even the most important.
> (I'm also a hiring manager at a 1k+ engineer company).
I'm not a hiring manager, and I have worked at five different companies with at least 10K engineers. All of them were "code first, review later". All five companies were very profitable and technology was a core component to their commercial success.
I can’t speak for your specific jobs, but my understanding is this is standard practice at FANGA: write an RFC or Eng Spec. Get it approved. Code it up.
When I worked at unprofitable, small startups, a lot of mental energy was lost due to miscommunications before the MR review process. Eg an engineer would be tasked to complete a project, but misunderstood a critical component and only after n-days of work was this identified and corrected.
As a hirer, the kind of takehome assignment I like to give is one that:
* Can be completed in 30 minutes by a skilled programmer
* Has clear evaluation criteria, both objective and subjective
* Has multiple approaches that require making different tradeoffs
And of course, only give it to some candidates where the result will be make-or-break.
As someone who took one of these broad take-home assignments my last time looking for a job, I failed a the assignment for a job I was overqualified for because I was told I wasn't able to divine what parts of the extremely broad assignment I would be graded on.
I doubt I will be in a position where I get a job that isn't a referral for the rest of my career, but it really turned me off of these kinds of assignments, both taking and giving them.
Very curious: how do you deal with AI answers for those?
While writing my questions (and testing in my teammates), I found that "can be completed in 30 minutes by a skilled programmer" very often means "can be completed almost automatically by AI", and that AI will give explanations too, that interviewer could repeat during code review phase.
Every step in the process is a filter. You can ask them not to use AI and trust, you can ask them what tools they used (including AI) and for what part, you can ask the candidate to screen record them programming their solution and forbid using AI, you can ask them about their solution in a followup interview, etc.
It's kind of up to what you're filtering for, and how much you trust the candidate at that part of the process, and how you follow up after hiring.
I think the huge proposal (after several rounds of questions) was what sunk him. The instructions did not request a proposal, and submitting one, especially one that detailed, conveyed “OP cannot take the initiative and proceed with work without seeking approval beforehand.” The project was about asking a few questions, listing a few assumptions, and giving yourself approval to confidently dive in and build something ambiguous. Maybe I’m wrong, but I don’t think the code mattered after that email was sent. The company should have just ended it there without him wasting his time building the assignment.
That's the blogger's point. The hiring manager saw the proposal, knew that wasn't what he wanted, and then proceeded to waste an entire week of the candidate's time while he was looking for income.
Just to be clear, this wasn't my advice, it was written into the task description -
This project tests your ability not only to code, but to deal with ambiguity and open-endedness
I'm not sure how I feel about that to be honest. On one hand I get what they're shooting for in general by saying that. On the other, they're going to have some preconceived notion of what they want, and it's a bit of luck if you come close to that.
> And don't hesitate to tell us if you have any questions!
I personally have no problem dealing with "ambiguity and open-endedness" (and, in fact, enjoy it!), but my solution to this problem at every job that has had this issue is: talk to people and understand the problem.
Attempting to "mind-read" is the worst solution to ambiguity, and, in practice, nearly always leads to disaster.
The best developers I've worked with have an uncanny knack for reading people's minds. (Of course, they are actually just really good at communicating and predicting what people want. But if you do it well enough it sure seems like mind reading.)
It's not really culture fit. I've used the open ended test before and it was a good filter for "can they walk the walk". Just some basic text processing, but only few people cared to mention error checking, tested Unicode behaviour, included any documentation, etc. It's "will you do the usual things without explicit prompting". (The assignment explicitly said you can call them out as something to do without fully implementing to save time)
>This is two business days later than the two-week mark since you sent me your first email.
With no accompanying personal reason for the delay, it's already a rejection in my books. The objective was to design an appropriate solution within the deadline.
This is the problem. Your guess is different from the author's guess, because nothing is explicitly stated.
For others who did not click the link, the explicit requirements are:
> - Email client can either be in the terminal (i.e. a TUI) or a web app
> - Should have basic email viewing + sending functionality
> - Can use a fake backend (DB, in-memory, etc) or real IMAP/POP/JMAP/etc backend
> - Does not have to handle rich text messages, just plaintext
> Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.
It should feel fast and intuitive, and you can choose which email flows you'd like to implement.
The word "keyboard" does not appear. Inspiration from X, Y, and Z is entirely subjective, and you should not be punished for not reading the interviewer's mind.
Is it not clear that every single one of the example apps the spec cites are text-based UIs, as in curses-style, monospace text, and (yes) keyboard-driven?
My takeaway from that is that if you don't make an app that is (quote) "in the terminal (i.e. a TUI)" but choose to make a web app, then it should look at act like a TUI.
What I got from that is "fast and intuitive" is the must have. When you have a choice (backend bit), justify you decision. I'd try to use that to stand out or an additional cool feature. When I have to do such homework, I do the requirements to what I think an above average candidate would, they've seen this a million times, then add 1-2 bits of flair/coolness only a handful of people would do to pique their curiosity and want to ask about the next round
All these things are easy to say in hindsight. However, the same outcome could have happened if the opposite strategy were taken (Don't clarify requirements, do bare minimum); someone could just as easily post a response saying the candidate should have clarified requirements and gone beyond the bare minimum.
My first though on looking at the code, and then on demo video was: wow, it took that person a solid week to write a web app with two pages.. all while missing the most basic email features, like an ability to opening a message (and not just show full text of mail bodies on one page).
Later on I read the requirements... This person applied to a position named "Email Backend Engineer" but they actually used a third-party product (postmark + turso) for email backend! They also clearly don't think about email too much - the basic stuff, like plaintext email formatting, viewing, and folders (at least inbox + outbox) are simply missing; while there are optional features, like login screen, admin page and backend framework. And backend database does not even containing a email headers map!
That person might be a great engineer, but I don't think they would be a good fit for that specific position. I'd reject them as well.
(A separate question is that hiring manager's second response... We don't do take-home interviews, but I imagine I'd be stumped by a proposal like that, it seems so irregular in a take-home assignment. Still, I can see how the candidate took that response for an approval... perhaps a next candidate would get an extra sentence perhaps something like "actual problem grading would depend on the code quality, number of features implemented, and how close those are to requirements and job description")
Update: read original job description and not just the except from the blog. That person surely omitted some stuff in their blog! the original title was:
"The project is to build a minimal, terminal-inspired email client"
it gave examples:
"Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya."
wow! that's 100% no hire, serious inability to read the requirements.
- Email client can either be in the terminal (i.e. a TUI) or a web app
- Should have basic email viewing + sending functionality
Seems like both were more than met to me.
Inspiration from existing tools is subjective. I would have assumed that spending time on the UI for a backend position was not the best way to show off my skills in a take home test.
The 2nd line of assignment, "The project is to build a minimal, terminal-inspired email client".
I agree that "inspiration" can be subjective, but in this particular case the solution was so much off that it'd be hard to argue otherwise. There is nothing terminal-like in it, and the tool is not usable as an email client either. Instead of doing login screens and user management, author should have made a page to view individual email or added folder (just 3, inbox/sent/trash, would already be a great improvement)
And if you want to ignore the requirements and work off position description... the position is titled "Email Backend Engineer", and the candidate's solution offloads actual email operations (storage, transfer) to third-party services. I suspect that even if candidate provided the same UX, but would have backed it with SMTP, IMAP, TCP/IP (not via http library), DNS MX lookup, resilient storage etc... then they might have passed.
But author failed to do either serious front-end work (required by question) or serious back-end work (required by job posting), so result was entirely predictable.
For every comment in this thread that says the author did too much, there is another saying the author clearly did not do enough.
Plenty of comments here have strong, but conflicting, opinions about which set of features are obviously in or out of scope. That alone indicates to me that this is a badly worded take home test.
Even ignoring that, the author told Kagi exactly what they were going to do in advance.
Applicant: I'm going to spend a week building X
Kagi: Sounds great!
Applicant: After a week, I have built X
Kagi: Not at all what we are looking for. Rejected.
I don't see any way to interpret this interaction positively.
I don't see much disagreement in the comments here? However there seems to be some people who only read the blogpost and did not actually follow the link to the original assignment [0].. you can tell their response by the lack of mention of "terminal inspired" features. Sadly this happens during discussions sometimes, especially when original author did not mention all the relevant facts in the post.
anyway, the exchange went like this:
Kagi: "please build terminal-inspired email client, you can choose which email flows you'd like to implement."
Applicant: I am going to spend a week building it, using golang, AWS, TMPL etc... I am also going to fit it onto 2 html pages.
Kagi: Sounds great!
Applicant: After a week, I have built something using golang, AWS, TMPL etc.. It has nothing to do with terminal, and does implement any of the email flows completely.
Kagi: Not at all what we are looking for. Rejected.
I don't see any way to blame Kagi for that exchange. Who would have thought the candidate decided to ignore half of the requirements?
Still, as a candidate, it sucks that they worked for hours and only got a rejection with a templated message. At least tell me the reason why you don't like my work, so I can learn something from it!
I get it that it's not always realistic to do, especially if the hiring manage reviews hundreds of applications for a hot role. But that's why take home assignment sucks. Some candidates may waste hours of their lives for nothing. Both sides need to be respectful of other people's time.
I understand that their responses were very minimal and not helpful, but it clearly says in the requirements to make a "terminal-inspired" email client. However, in the video you shared [1] I see a somewhat generic email web app, with nothing particularly "terminal-inspired". Considering the fact that they wanted either a real TUI or a webapp, I'm pretty sure that the web app should've also been "terminal-inspired". Am I missing something? I'm not trying to necessarily criticize you, perhaps I just misunderstood the requirements.
Edit: I've actually checked the full requirements page, and they explicitly say this in "Inspiration":
> Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.
Yeah, a clearer explanation in the rejection email would have been nice, or perhaps even a warning in response to the proposal (though it's not 100% clear from the proposal alone that OP would be going down a path that ignores that part of the assignment.) But the prompt explicitly lists other terminal clients as the inspiration.
Also, the prompt for a terminal client really changes what they're testing! Email web UIs have been a known quantity for years. But the UX of a terminal client is still something that's not "solved." I suspect the rubric for the question has large sections about how they decide to make a terminal client that OP's submission doesn't address at all
Also they seem to put emphasis on Go programming, which takes < 20 lines to get a cli running so not sure why they chose a browser GUI, which is arguably much more work.
Take home assessments can be a valuable part of an interview process but they absolutely need a time limit. I think 2-3 hours is going to give you all the information you need, unless what you're selecting for is new grads with no dependents, hobbies, or responsibilities.
If this had been limited to 3 hours then the worst case is that the candidate would have lost 3 hours, but far more likely is that they would have come up with an entirely different proposal and/or solution that was appropriate for that timeline, and that extra information would have made it clearer what the company was looking for.
The other point I'd always encourage applicants to confirm is: are you looking for any answer, or are you looking for a good answer. Some take-home tests are purely about passing a test suite and how you manage it doesn't matter, some would prefer that you meet 80% of the requirements but write better code. I've seen applicants do the wrong thing on both sides of this.
To me, as someone who hires and gives candidates take home assignments, Kagi’s assignment is huge and disrespectful of the candidate’s time. Surely they’re (probably unintentionally) filtering for candidates that have a lot of time to waste on projects like this — while people who are busy (the kind of people you want) will pass.
Surely there must be other ways to have an idea on a candidate’s skills.
In our case, hiring people with data engineering skills, we just ask of them a simple ETL challenge (pull data from zip file, transform it, insert into any database). We leave ambiguities in there and there are some Easter eggs in the dataset (eg null values where you would not expect it, incorrectly formatted CSV) that we use to evaluate how well a candidate can perform.
We timebox it at 4 hours, but don’t give guidance in case they run out of time, that’s a good suggestion.
During a follow up call, we review the code together and we’ll ask them questions on how to improve the code (“what if the dataset doesn’t fit in memory”, etc), which is what the actual technical assessment is. At that point they should already feel somewhat comfortable with the problem domain, and you can assess their real skills.
I agree about Kagi, but about your process wanted to ask: do you really feel like having them write the code as a take-home task adds any signal to the process?
At my current gig we do a 1-hour pair programming kind of thing, where we video-chat and watch the candidate work on a small, straightforward task. And as the interviewer I watch how they use the tools, where they go for docs, do they read the requirements, how do they test, etc. By the end I always feel like I have a strong picture of their ability, and the whole thing is capped at 90 minutes for both sides (adding time for their questions).
If the candidate's code was written offline beforehand, I'd have no idea whether it was theirs, whether it came from a friend or chatGPT, whether it took ten minutes or ten hours, etc. Sure I could try to suss out those things during discussion, but isn't it better to observe directly?
Leaving aside the non-trivial question of candidate nervousness in live coding, I do find the "ask questions about take-home" process very illuminating.
If it came from a friend or chatGPT, you'll learn that quickly because you can ask them stuff like "show me how you're implementing FOO. OK, if the requirements change, and now we need to BAR, what do you need to change across the app?" If they wrote the code and understood the task, they should understand what needs to change, how to do it, etc, and you can do that part with them live. Maybe someday LLMs will be fast enough that you can't catch someone constantly waiting for the AI to help them, at least as of May 2025 we're not there yet.
This is a data engineer’s fizzbuzz and you would be surprised at how many people actually fail it.
If they didn’t write it themselves, they will absolutely fail in the follow-up session where the real evaluation happens and I ask them to scale the code, handle crash safety without using transactions, etc.
Some people are nervous during live coding challenges, which is why we decided to allow this to be taken home, and the candidate becoming familiar with the problem domain in their own time, rather than "live" under pressure.
Would you think doing this live in a screen sharing session adds more value? How would you prepare the candidate for this?
> Would you think doing this live in a screen sharing session adds more value?
In my experience, hugely yes. Above all else I really like that the time commitment is fixed on both sides - no homework for the candidate, and we're not asking them to invest any more time than we are.
Also doing things interactively gives a lot of leeway for adjusting and avoiding wasted time. If somebody's got one section basically solved, I might cut in and say "yeah that bit is great, let's call it finished and look at this other bit". If they're obviously having a lot of trouble, I might ask about their current work and maybe switch tasks if there's a better fit. Or if they're obviously coasting easily I might cut in with "how would you scale this?" questions that give more signal than watching them finish up the loose ends.
Another thing I like is that we can keep the task specification simple, like what you'd get from a PM, and it's up to the candidate whether they want to ask questions or jump in. I imagine that with take-home tasks you either need to give out pretty specific requirements, or else have people interpreting the task a variety of different ways.
> How would you prepare the candidate for this?
We explain at the beginning that the goal is write code for an hour and watch how they work, and there's no hard requirement for what needs to be finished by the end. And they're welcome to search the web, use AI, or anything else they'd typically do while working.
And I've only done a few dozen of these, but personally I've never seen anybody get particularly nervous. I think nerves are a bigger issue with HR style "pass all the tests within the time limit" tests, where the danger is that somebody can hit a wall and never get past it. That doesn't happen with my format, because if somebody is stuck I just give them hints until they're unstuck (since I get no signal from watching somebody scratch their head).
But if you're unable to determine if someone has this knowledge via a phone call, instead of 4 hours of their work without you even being present, then that failing is on you, not them. If you can't judge someone's knowledge by asking questions, then you don't know how to come up with the right questions. Again that's on you. The only thing a 4-hr take home test will filter for is desperation. You'll get the most desperate candidates, and you'll do so by wasting days and days of people's time once you add it all up. It's just utterly disrespectful to demand these silly homework problems. I always take it that way. I take it personally, tbh. I find it absolutely insulting to even be asked and I simply refuse.
I'm surprised this seemed to be voted down. I've been a hiring manager for over 30 years, and I never do "technical tests" -- no take-home, no live coding, none of it.
I have a map of topics and questions, and I get the candidate chatting about their past projects, their approach, and what they liked/disliked about past projects and various technology they've used.
It takes a maximum of one hour and usually close to about thirty minutes to make a yes/no decision on a candidate (sometimes it only takes ten minutes to make a no decision, and then it's a matter of trying to politely end the interview).
I've interviewed hundreds of candidates this way over the years, and everyone I've hired has been capable of doing the job. Not once have I ever had to let someone go for lack of technical ability.
Part of the problem is that we don't train people how to conduct interviews, and another part is "this is how I was interviewed, so this is how I'm going to interview other candidates" -- pure inertia.
As an industry, we really need to do better.
As for the OP, _if_ I had been administering a vague take-home project that had a 1-week delivery deadline, and a candidate peppered me with Qs and then presented a full proposal for the project for approval, prior to working on it... I would have rejected them. But I'm pretty certain I would have decided to reject them in my regular 30-60 "chat" interview, and I would not have moved on to the take-home project and wasted their time like that. So, again, I fault the interviewer(s) for not being able to filter candidates efficiently.
I had an interview once to work for IPFS, with a guy who already knew my skill level well (decades of experience), because I was active on their web forum solving everyone's problems, for about a year. Even during the interview he mentioned the technical part of the interview wasn't even necessary, because he had total confidence in me, and didn't even ask technical questions at all. He spent the ENTIRE hour selling me on the job. I hardly had time to speak. Everything went perfectly fine. Then he asked me to do the take-home project at the very end, just because "everybody has to", and I politely told him sorry I don't do those. So that ended the opportunity. So you're exactly right it's absolutely an "inertia" thing.
The solution to this problem ultimately needs to be a regulatory one.
People should be paid for the time they spend in interviews.
You make that the law and this ambiguous hoop jumping bullshit goes away real fast. Companies will optimize for cost instead of dumping cost onto the prospective employees.
This is good for the economy because it forces companies to innovate and optimize the interview process and it saves hundreds hours of totally economically unproductive time on the part of candidates.
That's basically my feeling too. No one would ask for a maid to clean their house free for 4 hours, in order to decide if they want to hire her long term. And that's precisely what hiring managers have been getting away with for decades, because no one is pushing back. The managers are essentially abusing their power. It's all about abuse of power and utter arrogance.
I don't know what ETL means (I could look it up) - regardless I can do that ETL challenge. Realtek SDR dongles and Python loved messing up CSV. The only question I'd have is if a null or and "incorrectly formatted" row was invalid or not.
So I have to wonder, is this a junior position, or did too much hadoop rub off on me a decade and a half ago?
This is for a medior solution, but it’s more intended as a data engineer’s “fizzbuzz” and then in the follow-up call I’ll ask them how they would distribute the workload over multiple clusters, make it idempotent, etc — that’s where the real technical evaluation happens.
How do you know that interviewees aren't spending more time on it?
Because you can't guarantee all candidates are spending the same amount of time, it becomes a game theory problem where the candidates will typically lose in some form. In many cases, the right answer is to spend extra time making a really polished (but not too polished!) solution and pretend like you stayed in the time limit. And every candidate is either a) doing that, or at least b) worried that their competition is doing that.
Even if we ignore that dynamic, 3 hours is a long ass time for a candidate to spend when they're not even sure they'll get to talk to another human about it.
In a 1-hour interview, you can run a candidate through a programming exercise and be guaranteed they're not wasting extra time on it. And if they happen to prefer doing take home assessments, you can always let them send you an updated answer later. (But often by the time a candidate asks me if they can do that, I've already developed a favorable view of their skills and can tell them, "go for it if you want, but you've already 'passed' my test.")
By keeping the candidate-interviewer time investment the same, you guarantee that you're respecting the candidate's time as you would your own (because you're sitting there with them.) I can help them skip over the parts I'm not interested in (e.g., by feeding them info they'd be able to find via search or telling them not to worry about certain details.)
If a hiring manager doesn't respect their candidates' time, how likely are they to respect their employees' time?
Yep, this is what I am taking from this thread: Next time I am given a take-home, I am going to ask them to promise, that I will get to talk to a human about it. They can of course straigh-ass lie about it, and I am sure I will run into such abysmal behavior at some point.
> How do you know that interviewees aren't spending more time on it?
In some cases we roughly timed it, scheduling an email for a time the candidate wanted and asked them to return it 3 hours later. In some cases we just treated it as an honour system. We made it clear that the task was intended to take about that much time and that spending more time was not allowed/encouraged.
In reality, we found that good candidates took ~1-2h, and in some cases where candidates spent a lot longer and owned up to it, we found no improvements. In one case a candidate submitted at 3h and then again at 8, and we marked the 8h version 1 mark lower.
I agree with the author that live code reviews are much better than live coding, but if companies insist on a live coding exercise: let me bring my laptop and mouse leave the room for 45 minutes and come back when we can talk about what I built.
I did a recent round of interviews and this experience closely mirrors mine (at multiple places). Delivered an exceptional solution to the problem only to be rejected without a discussion about the delivered project. I have been on the hiring end of take-homes multiple times and firmly believe that if you ask someone to do a take-home assignment you must follow up with a chat about the code. Apparently that's not the case at most places.
If you're asked to do a take home, I highly recommend confirming that there will be a follow up regardless of the assessment, and if they do not agree to this, do not do the "home work". The honest truth is that most of the teams hiring are of pretty low quality and therefore implementing a good solution is a negative because the team hiring is not at your level (which is a frustrating reason for being rejected).
I'm an early adopter of Kagi and am now planning on cancelling my account over of this. Completely unacceptable. If you don't have time to chat with a candidate about their work, don't ask them to do the work in the first place.
> I have been on the hiring end of take-homes multiple times and firmly believe that if you ask someone to do a take-home assignment you must follow up with a chat about the code.
That’s actually a very valid point. Take-home assignments not only require a significant amount of effort from the person administering them but also from the interviewer (or rather, the hiring team). After investing the time and effort in reviewing a project, it’s reasonable to expect feedback/a response back if requested.
However, we must also acknowledge certain realities. If there are 20 solid candidates for a single open position, many capable individuals will be rejected. This doesn’t imply that they were inadequate or even “failed.” It’s simply a reality of life.
Now to be asked to justify why a hiring manager picked A and not B, legally speaking, cannot be that I had to pick one out of the amazing candidates, so I picked one. The legally correct response (unfortunately) is to get nit picky and find faults where none really exists. At least, that has been my observation in how corporate America likes to operate. And last time I checked, Kagi is based out of Palo Alto.
> if there are 20 excellent responses for a single open position, many capable individuals will be rejected.
I would never ask 20 people to do a take-home assignment. There are so many better ways to test team fit before asking someone to commit serious, unpaid, time to a project. Historically speaking a 30 minute chat with someone provides a surprisingly good amount of information to anyone experienced in hiring.
But if you want to commit 20 people to take-home assignments, then block off 20 hours next week for 1-on-1s to chat with them about those assignments. If that sounds like too much, then don't find a better way to filter down the number of people doing homework until it feels manageable.
> If there are 20 solid candidates for a single open position, many capable individuals will be rejected.
And yet, the role is still advertised 1.5 months after my rejection, which means they are still getting emails from new candidates applying for the role. Do you suppose they get so many competent candidates that they can't make a decision on whom to hire?
At this point we can only speculate about their intentions, because the intentions are definitely not transparent.
I agree. The company should have either published clear guidelines on how the project would be assessed or provided some feedback on what could have been done better. It's okay to fail an assignment, but it's not acceptable to waste someone's time without offering anything in return. There are many companies, including pop startups, that handle this well.
did they really deliver "exceptional solution" though?
Their solution seems to be nothing like "a minimal, terminal-inspired email client", and OP completely ignored the references to tools they were told to "take inspiration from"
When there is such bad misunderstanding of the requirements, I'd not waste time on the discussion either.
There was able opportunity for the hiring manager to point that out if that were the case. They specify questions where encouraged and then completely ignored the questions. If the HM doesn't have time to respond, then maybe take-home as first filter is not a good decision.
Were there? The question clearly said, "build a minimal, terminal-inspired email client" and "take inspiration from existing terminal email tools like aerc, mutt". Candidate completely ignored that part and built a tool which is nothing like terminal email at all.
Would any of their emails give a hiring manager idea that the candidate failed to understand the assignment in such a gross way?
Email 1, "What kind of extra feature do you value highly"
Interviewer thinks: "UX improvement could be VI-like megacommands, "pretty UI" must mean creative use of colors and font attributes, privacy-related must be encryption at rest... It's all good, we are happy to look at all of those!"
Email 2, "A webapp with golang accessible online, deployed through AWS using ECS Fargate, with SSL (https), integrated with an email sender provider, with authentication through a login screen, with the ability to send emails through a form, and displaying incoming emails in the user interface."
Interviewer thinks: "OK, that's a lot of implementation details.. Not sure why candidate feels the need to confirm that, but nothing in the list above will be graded as a negative. The important things we care about are that it's inspired by terminal apps like mutt, and that "it should feel fast and intuitive" and one can totally do that using technologies above"
Without seeing the screenshots/mockups, how would one even guess that the candidate was so off?
> Interviewer thinks: "OK, that's a lot of implementation details.. Not sure why candidate feels the need to confirm that"
Really? My email states explicitly why I want to confirm that:
> I would like to know what kind of response I could expect from Kagi if I drive it to completion.
Are you forgetting that the entire transaction implies a lot of work for the candidate?
Are you forgetting that candidate is doing this to land a job?
Do you think the candidate is writing a design document just for fun?
Your subtext implies that I should have guessed the grading, which is quite mind-blowing.
Remember, the instructions mention a web app as an option, the role is for a web-based company, and if my design document did not meet any of the key aspects they could have brought it up and rejected my proposal.
If you read the assignment, you can see that the majority of tasks is about UI and user-facing features ("Should have basic email viewing + sending functionality", "Does not have to handle rich text messages, just plaintext", "take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.", "It should feel fast and intuitive")
The assignment had exactly 1 line about actual back-endy stuff: "Can use a fake backend (DB, in-memory, etc) or real ..."
Now, what did your email asked for? There was a whole bunch of things about back-endy stuff, and exactly 1 (one) sentence about the user-visible things, the thing they cared about the most:
"The UI will be kept simple, showing pagination for sent and received emails. In addition to the requirements of the assessment, there will be a login screen and two accounts"
So maybe you intended this to be a complete design document, but it actually was not. This is because it did not actually did contain any parts the interviewer cared about. And that's one of the reasons why you got no feedback - you could do _anything_ for the backend and they would likely still accepted that, they simply did not care if you used Pulumi or terraform or "start.sh" or "docker compose up"
And no, you don't need to guess the grading, you just need to read the assignment carefully. What do they seem to want? What kinds of things do they mention a lot? What kind of things do they mention in passing?
The HM answered, it was just not what the author wanted:
> That is part of the assessment itself, see what kind of extra features you can come up if any
It seems like the author wanted to hear something more like "ok, here’s a list of things we want you to implement, and here’s mock-ups for the UI, and here’s the API spec you can follow". How I’m reading the HM’s response is - I should spend a few minutes trying out the email clients they mentioned or watching videos on them, and come up with a list of their features. Apply my own judgement to say which ones are most important and roughly estimate the time it will take to implement a PoC for the assignment. At this point you could take that to the HM and say "hey, I think I can do X, Y, Z features within the timeframe. I’d like to have W, U, V, features in the client but I think they’re less important and I won’t have time. Since this is a backend position I will focus on the API and the client will be a thin wrapper around that. Does that sound good?". They will probably say yes and maybe throw in a "when you do X feature think about users that have 100k unread emails in their inbox", and you’ll get a gold star for "dealing with ambiguity". If I was the HM I would expect some amount of back and forth like this. Since the guidelines are so broad you can focus in on the areas you are most familiar with and keep the rest super simple. As far as we know the author sent the one email on March 18 including a lot of details about what dependencies they were going to use but no product or clarifying questions after that, which the HM might have happily answered. They also both went past the deadline set by the company AND their own self-imposed later deadline from their own proposal.
At the end of the day this is a startup and so it makes sense that what they’re looking for is someone who can work independently. They won’t have a PM to come up with the spec and list of requirements for you every time, and an architect who hands you the perfect software design, and they need you to be able to apply your own judgement and look ahead so that if you are designing their backend and 6 months down the line users ask for drafts support, you don’t come back with "sorry I didn’t account for drafts in my data model because no one told me, we can’t support that without redoing the whole thing".
It sounds like this actually worked fine as a filter for both the author and the company, just that the author realized too late in the process that this was not the work environment they were looking for. A phone screen or whatever else would've just been additional time spent for both the company and the author. I also don't think it's the company's fault that the author put in a "full work week" of hours, as far as we can tell that was never the expectation.
This is a classic case of misreading the subtext. They want you to be independent and able to create your own work and goals. The ambiguity of the take home task is not there for you to interrogate over multiple emails, rather its an open palette for you to show how you take a relatively broad task and produce your own interrogation of the problem space.
As someone who works at a university this is very reminiscent of students who don't understand the assignment then complain when they get an unexpected mark.
I completely agree. The assigment seems to be looking for someone who will make as minimaly clever functional solution as possible. Someone who will dare to “be lazy” in interview where they want to impress.
This is not that common trait. But if your company has more prototype driven approach with quick “experiment” releases some people are just very bad fit and thats bad for everyone.
There is probably a candidate that thought for 10 minutes - what best can be done in 60min. They probably leveraged some http auth and forms, run that through tiny go backend, sent it with brief email. And they most likely looked a lot better in the process.
That would maybe be a valid point if the take home set any context beyond "it's an R&D project" and "minimal".
What do you want to achieve here, is this a throwaway prototype or would a user ever see this? Am I going to get picked on for imperfect UX or do you just want to see something. It becomes an exercise in guessing the reviewers sensibilities.
Guessing the sensibilities of people is a large part of experimental software design and research. So I would say that the interview task is working as intended :)
Broadly, sure. You probably know who your audience is. But how'd I know who my recruiter is, individually. Is stalking their social media presence part of the exercise?
... and for every such Choose Your Own Adventure assignment there will be many people rejected because they "misread the subtext" and did not keep pushing for requirements, which is a very important virtue in engineers!
Life isn't fair, but apparently we expect more from hiring managers. And since life isn't fair we continue to get disappointed by hiring managers.
> As someone who works at a university this is very reminiscent of students who don't understand the assignment then complain when they get an unexpected mark.
This is such a lazy and convenient conclusion for someone that is giving out instructions.
Another valid reason that the students might misunderstand an assignment is because they cannot read minds and the instructions are poorly written.
Good teachers are the ones that make themselves understood by the largest amount of students. Bad teachers are the opposite.
With my best teachers, I never had to question their grading, because it was transparent from the start. So if you are getting a lot of confused students, you might be the common denominator.
Buddhist monks learn by solving riddles and guessing the messages of cryptic messages called Koans. Lets agree that students at a faculty, literally paying for tuition, should not be subject to the same practice.
Since the author himself seems to have posted this, I figure I might as well try to help contextualize the response he got. I worked at Kagi (though not on their search product) and went through a similar take-home test. Obviously, a lot of disclaimers: this was a while ago, I am obviously biased, I am not at Kagi anymore and this is my perspective. I'm not even sure how they would feel about this but they're more than welcome to say hi if not reply to the actual content of this comment.
When I went through the process the company was very small and Vlad personally reviewed applicants, and he used take home projects like this one. It seems like the company has grown larger so now he isn't looking at them himself anymore, but the essence seems the same. If you ever talk to him you'll find that he's basically the Hacker News commenter stereotype and his interview is really a vibe check to see if you seem cool. To him, submitting a long design doc where you go "I am going to use Galactor. I will make sure the project is florp-ready. I will fleem." is the exact opposite of cool. When the assignment says "I want a terminal-inspired email client" the project that passes with flying colors has keyboard shortcuts for everything and comes with a paragraph explaining how it never takes more than 2ms to draw a frame. I suspect the interviewer thinks you are too "enterprise-brained" to work at Kagi.
Now, is this actually a good way to screen people? A bunch of other people here have opinions on it, so I won't rehash it too much. There are obvious problems with it. Vlad basically wants people who think like him. This kind of interview has a lot of problems that have been discussed and studied at length. But if you understand that and are privileged enough to go through that process, the task will actually make sense to you. I think Kagi could do a better job at communicating this. It's unfortunate that you had to waste a bunch of your time to get to this point but I hope you can find something else that matches your working style better.
A lot of companies seem to want to hire folks who "think like me". IMO, that's a mistake: you need diversity of thought to provide new insights into problems. If you all think the same way, and one of you gets stuck, then it's likely you'll all get stuck. This seems to be a common problem in startups, IME, and perhaps an indicator of why 9 out of 10 startups fail.
Hiring someone who wants detailed requirements is not a good idea when you don't have a structure in place that can provide these detailed requirements.
If you rely on people to work independently, you just can't hire someone who wants feedback everyday to see if they are still on the right track.
Depending on the level of detail, "detailed requirements" can be the bare minimum for any level of cohesive, cross-functional collaboration.
You need to have discussions to talk about things like:
* common UX language
* common API expectations
* programming language choice
(because you will not be the only person to work on it)
* in-app / domain language choice
(because your users need to know what a word means.)
* library choice (because you will not be the only person to work on it)
* storage strategy (because there may be legal requirements here)
Ignoring all that and letting all the engineers just run wild is a great way to make a very painful company to work with and cross-collaborate on.
My impression is that Kagi aims to hire people who are already aligned on most of these. Not all of them, of course, but I read the interview as an attempt to be able to go "do this" and have candidates understand what it means.
I don't disagree, but again: Kagi is Vlad's pet project; he's already rich. I think he would rather it fail than it become something that he doesn't feel aligned with.
What seems objectionable to me, is that in order for Vlad to find people that are "cool" he's wasting everyone else's time.
This isn't exactly the same as if I give people a test on their ability. It's different. In here you pass if you did something that scores well in a metric that you're not told about. It's not very different than if I gave you a take home assignment which is just literally "code something - you have to be good at ambiguity!" Joking, you just have to guess the thing I had in mind. It's very inconsiderate to people, and doesn't tell me anything good about this Vlad person.
I think it's very reasonable to expect that there should be better communication here about this actually being a combination culture-fit-and-coding assessment. But I don't actually think it's fair to say that nobody is told about the metric. It's pretty clear to me. I think a lot of other people here also understood it (which makes sense, because they're all Hacker News brained). I agree that this sucks if you don't "get" it but I'm not entirely sure if there is a good way to convey this. Maybe they shouldn't be looking for this at all but that's a separate discussion.
> To him, submitting a long design doc where you go "I am going to use Galactor. I will make sure the project is florp-ready. I will fleem." is the exact opposite of cool.
Am I supposed to guess this much?
Do they expect me to creep on their social media to figure this out or something?
Finally, I made it abundantly clear that I am uncool, so giving me the "go ahead", simply to satisfy their curiosity... I hope they are treated the same as they treat other people.
No ill will against you, if anything you are giving me a plausible explanation where I have none. I just wish I had come across a post like mine earlier, and I hope that we, as a collective, start categorizing any job with a "take-home" as a job for the "cool kids". That way, all of us boring and experienced engineers may force companies to adapt if they actually want our labor. And those that do not adapt, we can infer the type of company that "offers" the role.
// Create a context variable that inherits from a parent, and sets the value "test".
// Create a context key for the theme.
ctx, err := getEmails(pb, e)
First line is very weird, unrelated to the task. Maybe copy-paste from a sample in a blog post? Anyway not paying attention to leave it in.
Wording in the second line is not consistent with third.
I’d stop my review there. Lack of attention to detail. Author does not demonstrate an ability to think clearly.
The problem in the story is not that the candidate was rejected but that the process is largely disrespectful by asking for a lot of unpaid time to implement some project without clear guidelines or giving valuable feedback.
The author wasted a significant amount of his own time, and the hiring manager’s time with all the questions, the proposal, etc.
Even after asking for all this clarity, he failed to do what was originally asked. If asking for all those details, you have to at least do the basics of what was asked.
He failed on multiple fronts here, and even wrote an entire blog post without realizing it. It seems like Kagi did the right thing by not hiring him, if we’re being brutally honest. If he would re-read the original ask with a fresh set of eyes, then look at his final product and all the communication, the feedback should go without saying. When someone misses the mark by that much, it’s takes a lot of effort to try and say it nicely to soften the blow, as a company would want to do. Multiply this by however many people applied, and there just aren’t enough hours in the day.
Now that he’s written this blog post, I wonder if he’ll have more trouble even getting past the first gate of hiring processes, as other companies won’t want to sign on to have a blog post trying to drag them through the mud over a rejection. That shows more questionable judgement.
> The author wasted a significant amount of his own time, and the hiring manager’s time with all the questions, the proposal, etc.
Maybe if the hiring manager didn't want their time wasted they shouldn't be using a stupid take-home prompt? Frankly everything I've seen here indicates that I would never work for Kagi and would actively encourage people to never work there.
Yes, I agree. You cannot implement arduous processes and then complain about how arduous they are.
A week long take home exam with no discussion after the fact is absolutely unbelievable. It's so disrespectful to the candidates time.
I don't care if the exam wasn't supposed to take a week, or if they get 1 million exams, or if their hiring manager only works 1 hour a day. If that's the case, then they shouldn't be doing this type of test. If you can't handle the heat, get out of the kitchen.
I don’t see where the hiring manager was unprofessional. He responded to the questions/proposal without giving the author an unfair advantage, and he responded to the message after getting rejection. The author wasn’t ghosted or ignored, and has closure, which is more than a lot of people get.
Had the hiring manager given detailed feedback, it would have ended up in the blog post, and then they would need to revamp their process, as anyone reading would get additional information they weren’t intended to have.
I’ve also seen situations like this where the person never stops responding and seems to try and turn a rejection into a long term mentorship opportunity. The author seems like they might have those tendencies and keeping to standard boilerplate responses helps avoid that. It’s not overly personal, but I don’t think I’d call it unprofessional. The author simply misunderstood the assignment at a foundational level and is trying to shift the blame for that onto the hiring manager… when that was exactly what they were testing for.
I deployed the app to my own domain and the functionality was without flaws. It took me forever. Included extra features typical of a backend role such as authentication and Infrastructure as Code. My efforts led to an empty rejection email.
And you are telling me that I should have spent extra time on my code comments?
As an engineering interviewer I feel the urge to comment. I like neither leet code, nor home assignments. They are both time consuming and provide you with a little information.
But having said this I would have likely rejected the author too. Kagi Search is a startup, these folks are typically looking for fast moving, pragmatic optimists who can strike the breadth - depth balance.
We used to have a colleague. They would collect the inputs, close themselves for a few days, come up with a solution only to learn that reqs changed in the meantime. It was not pleasant experience for anyone.
I had a similar experience with DuckDuckGo several years ago, but the difference was that I got paid as I progressed through the various stages.
I had to submit a short written design proposal and was told to cap it at a few hours and was paid for it. It was accepted and then I spent time implementing a solution. The pay again was capped at a certain amount of hours, but I ended up going past the recommendation because I was actually having fun trying to figure it out.
I submitted the solution and after a while I got a response with basically, "it was a difficult decision, but sorry we're going to pass" and they wouldn't provide additional detail. It's been over 5 years, but I still wonder why they passed.
I can only assume they had a lot of candidates and they may have had other very strong submissions that were better than mine. However, it took a lot out of me emotionally and affected me for quite some time... more than any other interview in my ~20 year career. I feel for the OP.
When I see instructions that say “This project tests your ability not only to code, but to deal with ambiguity and open-endedness”, it actually means either a) this exercise has been so poorly conceived that we didn’t bother with a rubric or b) the work environment is so chaotic, we never clearly define specs or requirements for anything.
That, and the instruction to simultaneously “show off your skills” and deliver a pragmatic functional solution are completely at odds. Good simple solutions aren’t flashy.
To me, take-homes that have UI or API integrations are a bit of a red flag, because UI code (for most roles) is relatively boring, and API integrations are a lot of faff with not much signal. Cool you can make an HTTP request, cool you've got a basic CRUD editing setup. It's a lot of code that takes a lot of time and tells me almost nothing about how you code, hell, AI tools will happily generate these things in no time and at a pretty good quality.
What I want to see is an engineer implement an awkward bit of business logic. Does it become a million nested if statements and a "here be dragons" comment at the top, or do they identify the right patterns and build something that I can reason about when reading the code? This is far more valuable in the job, more signal in the interview, and much harder for AI to get right. It also takes less time.
I’ve seen a few that let you fork a repo with all that boilerplate set up and you just have a few stubbed methods to fill out, and that seems reasonable. But going from 0->1 on an app is so much grunt work and I doubt the reviewers would even look into all of it
I would say that's reasonable, although it again sorta depends on the stubbed methods and what you're trying to figure out from them, and I'd question the necessity of the boilerplate - can't you just implement the methods?
Can I ask you and everyone else - why do AI is so good at UI/CRUD apps and terrible at business logic?
I've been caught with this few times now. Spend ages trying to coerce AI to solve logic problem and end up just manually solving it myself. Whereas UIs are so good and usually near perfect from first prompt. I suspect it's the weak prompt. I need to learn and solve this before my brain completely atrophies (there must be Anthropic joke here somewhere hehe).
I doubt it's prompting. I think the issue is more likely that UI code is often similar, has a lot of examples online, and often doesn't require understanding data flow. This is why LLMs are great at "boring" React components, because they don't actually understand the flow of the data, but they don't need to.
Business logic on the other hand is much more likely to be novel in some way, there are likely fewer similar examples for rules to be learnt from, etc.
Obviously this is all gradations, LLMs can manage some business logic and mess up some UIs (they can't "see" the UI which doesn't help!), but this is my experience of them and fits with my understanding of the technology.
It's a start-up with a lot of different projects. It is expected that the work environment would be chaotic. People who look for a no-surprises type of work week should probably not apply to such a kind of workplace.
> Good simple solutions aren’t flashy.
Maybe showing off your skills is making a good and simple solution?
Sure, the work is chaotic. But why would the interview process need to be chaotic? You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants
> Sure, the work is chaotic. But why would the interview process need to be chaotic?
Because the interview is meant to measure how well someone would perform on the work?
> You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants
Well sure, all else being equal, but if the cost of consistency is an assignment that doesn't reflect the actual work environment then that may outweigh the benefit.
But why conflate “technical skill assessment” and “ambiguity handling” into one big stressful test?
Doing two small tests toy can measure each skill independently
A clear coding test to get technical skill and then some in person requirements gathering exercise or something to measure ambiguity handling.
You’ll get better insight into their abilities with less effort for the interviewee AND the interviewer (if the startup is really that chaotic, do you really think they are doing a careful code review on a zillion different bespoke takehomes?)
> we never clearly define specs or requirements for anything.
I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements? I consider that a big part of what it means to be a senior software engineer.
Maybe this exact format isn’t the best way to test for it, but I don’t think “we want to see if someone can deal with ambiguity” means all the work is ambiguous. It might just mean all work needs to go through a process to take it from “ambiguous requirements” to “clearly defined specs”
> I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements?
Then I'd expect the interview process to focus on that process, not on the final deliverable. As it stands, the candidate tried to interact with the interviewer by trying to ask for clarification on the requirements, then making a detailed proposal, and got no actionable feedback from either.
Could one do a "terminal inspired, mutt-like email client" that completely satisfies OP's proposal (web-based, TEMPL, pocketbase, pulumi, etc...)? Sure, it would be possible. None of those are _required_ for the submission but you can totally use them if you wanted to. So the interviewer probably thought, "hey this person is going all out", and was absolutely honest in saying "Looking forward to receiving your submission"
I cannot really blame the interviewer for not reminding the candidate that they should actually follow the requirements, in addition to all the optional stuff they mentioned in their spec.
I have to be blunt, you have no idea what you are talking about. I am just going to say this: A terminal app is completely at odds with an online web client. If the interviewer is as clueless as you are, that explains two things:
- I was never going to pass the interview in the first place
- I was never going to get a reasonable conversation from this person, no amount of communication is going to solve incompetence from the interviewer's side
Corollary:
- Gross negligence from the company for assigning this interviewer
> what if part of the role that’s being hired for is to help people clearly define the specs and requirements?
Then they should have responded to his questions in email?
I personally love working on projects that are vaguely defined because it means getting to interact with people and understanding the heart of the problem. My favorite roles have involved reaching out to non-technical people and figuring out how to solve their problem. Often it's not the solution they even initially asked for.
But if your role is to guess about vague specs with no communication, then you're going to fail no matter how senior you are.
I just read half of the article and this is painful to read. They’re asking for something relative simple as a take home, that just requires a little bit of imagination to fill the gaps. Why does the candidate have to be so fussy about details? Do the best you can, fast, and don’t announce you’re gonna be 2 days late for a take home!
To work in a startup you have to embrace the "just do it" mindset, you have a brain to fill the gaps, and you can always iterate later. I’d be more interested in getting the result of the take home quickly, obviously in a working state, but with a documentation explaining trade offs because of lack of time, ideas for future improvements, design choices, stuff not included (ie. a pretty UI) etc.
My take: they’re literally asking for a CRUD (if you decouple well, you can easily replace the DB with SMTP/IMAP later) with basic email features: send (to, cc, bcc), fetch, reply, reply all, transfer. That’s it, you can do it in 2 hours. Add folders, signatures, "send later", authentication, or whatever if you want to show off.
--
Additionally as an experimented Go developer (8 years working with Go as my main language), the code is not great and tbh I would have rejected it if I received this technical test where I work.
In 5 minutes of reading the code:
- lots of commented out test stuff, don't leave that in a take home or I'm gonna assume you'll do the same in your commits
- lots of println but no logging
- relative imports (eg "mymail/app/router"), don't do this
- weird choice of relatively big dependencies (turso? pocketbase? negroni?)
- no architecture or decoupling, everything is in the router
- bad use of the context (getEmails)
- no tests (no need to test everything for a take home but at least some things can be tested)
People say languages are interchangeable and you're never a "xxx" dev, but that's not true. They asked for "Proficiency in Go" and the candidate is not proficient in Go, that would be the number 1 reason for rejection.
I highly doubt you can produce something that would get you an offer in 2 hours for this task, given its breadth and human nature to worry and attempt perfection to impress. You are likely competing with much more engineered solutions.
Damn now I’m curious what I could do in 2 hours, this is the kind of things where you can easily get carried away so 4 hours feel like 2. I don’t have 2 hours (or 4) to waste to do this but it itches a little bit.
Anyway in 2 hours I’m pretty sure I can write some Go code that would get me an offer. This is not about the amount of features, it’s about giving them what they want.
> Anyway in 2 hours I’m pretty sure I can write some Go code that would get me an offer. This is not about the amount of features, it’s about giving them what they want.
Please do it, get the offer that I didn't, maybe you could even write a blog post about how easy it is and how wrong I am! ;)
The thing is I did a few interviews for Go positions last year and I have a lot of experience with Go so yeah, I’m confident in my skills. I’ve also been on the other side of the process many times and I know exactly what they look for. I’m not saying this test is easy, I’m saying I’m good at it and prepared for these kinds of take homes.
I don’t agree that it’s a bad technical test though, and reading your article I can say your approach was suboptimal. If you keep interviewing for startups you need to put yourself in their shoes.
To quote the test:
> This project tests your ability not only to code, but to deal with ambiguity and open-endedness
Would you mind explaining what you mean by relative imports and what the correct approach would be? I'm somewhat confused since I think of relative imports as something like "../.." which you can't do anyway.
I'm currently learning Go, so I apologize in advance if I'm missing something.
Building a concrete, working, minimum-viable solution from ambiguous requirements: that's the point of the exercise. That's what hiring managers want in candidates because those candidates end up being good at building a concrete, working solutions from ambiguous requirements. Which is every software project ever. Although the AI Age has unquestionably changed the efficacy of this kind of candidate screening, that is orthogonal to this discussion. For many years it has been one of the most-effective ways to screen for the ability to build concrete, working solutions from ambiguous requirements. Which is every software project ever. So it's no surprise it persists.
Yes, software is full of ambiguities but there are methods we use to handle them. OP emailed an outline wanting feedback, as any team player would do to iron out ambiguities, and received a meaningless reply. I think it's safe to say companies don't want their engineers going into a corner never to be seen again for 2 weeks, which is what this interview process recreates.
OP's proposal was only describing irrelevant stuff (the backend technologies) and was completely silent on on stuff that mattered (demonstrating how actual RFC822 email works, mutt-inspired UI). It was therefore accepted without comments, as there were no "substance" to comment on.
That is often a problem with proposals/design docs in general. In the real job, if proposal is actually required, it would be sent it back with "please add details on UX and how you are going to store email headers". In this case, the proposal was explicitly _not_ required though, so interviewers did not want to ask for more details on the optional document. They checked what was written there, found no problems, and approved it.
I think what has happened is the author has no idea what "email backend" was, so he just decided to ignore that part and build the only backend he knew, web-app backend. And those terms are pretty different. The "email backend" is the service which actually stores and transfers email, in the author's case it was turso + postman.
So from the interviewer's standpoint, author was asking about few details of implementation, like "can I use third-party service for email storage?"; and the response was of course "yes, you can" (because assignment was pretty clear that backend does not need to be advanced or even present, and that it's UI that matters)
I guess the question worked as intended, and filtered out candidate who cannot even read the simple requirements.
(The amount of effort was disproportional though, but I am not sure how to solve this in take-home context without discriminating against people who have busy schedules and/or work slowly)
OP didn't take into account the (great) asymmetry between themselves and the hiring manager, then built an entire lament on that. Dealing with this job req is likely just one of many day-to-day responsibilities the HM has and frankly I'm impressed they responded with three whole sentences. One method we can use to handle such ambiguity is to "make your best judgement" based on your skills, knowledge and experience (things that are tested for in the hiring process, incidentally), because often you may not get the answer you want or expect if any at all.
They explicitly asked for a minimal, terminal-inspired email client for their Email Backend Engineer role. OP built a ton of infrastructure, created a generic web app that has no semblance of terminal inspiration as far as I can tell, and outsourced the backend to third party APIs.
Concrete and working? Technically, yes. This would have been an excellent submission for a different assignment and role. But it doesn't seem to suit the specifications for this one.
Not what was expected of them. I included "minimum-viable" in my original reply for a specific reason: to counter OP's lament that they lost out to a simpler solution.
If you are asked to implement X and instead you take extra time and come back with X+Y+Z, what have you done? Wasted time e.g. money. Companies really hate wasting money.
So if in your candidate project you demonstrate a propensity for bike-shedding any task, that's gonna be a big red flag.
I worked with a lot of engineers in my 40+ year engineering career. One thing that most engineers don't like is ambiguity. I didn't like it either, until mid career when I was thrown into an environment where the customers didn't really know what they wanted, and it was impossible to explore all of the solution space within time and budget constraints. I ended up thriving in this environment, but I watched as many qualified and experienced engineers floundered and over-stressed themselves. The attrition rate in this environment was about 80%. To survive, I had to use intuition, speculation, and innovation -- things I had not used much in my previous engineering roles. I sadly watched while a lot of really good engineers chose to move on, or were assigned to less relevant roles.
In this case, I perceive that the author (Jose) was looking for firm requirements, and did what he could to elicit them, but the Kagi folks were just looking for innovation and a demonstration of competency. They didn't care as much about the details of the submittal as they did about seeing what the candidate would do without specific requirements.
> They didn't care as much about the details of the submittal as they did about seeing what the candidate would do without specific requirements.
This would be acceptable in a world where time is infinite. The reality is that they are going to churn through -who knows- how many candidates, as if it was worthless, as if it was a game.
If they really wanted people to spend little time on the solution, they could include that bit of info in their instructions. Clearly, they omit that bit on purpose. Obviously this is not their first rodeo, they must have done this for a long amount of time now.
As I said above, I did a lot of this when time and money constraints severely limited exploration of the solution space, yet I succeeded most of the time.
Guessing is a skill, and some people are better at it than others. An "educated guess" is based upon knowledge and experience, but little more.
I don't really know that this is what Kagi was looking for though. Perhaps they wanted something more than a guess but less than a full solution. Maybe just enough to gauge the confidence of the candidate.
The "new business" environment often involves a lot of guesswork.
Man this sounds rough. I had a somewhat similar experience a while back but instead of going back and forth trying to nail down requirements, I went down this spiral of wondering if the lack of clarity wwas part of some kind of meta-test.
"Do they want to see if I can build something with as little guidance as possible?"
"Do they want me to push for more requirements like I would on a real project?"
"If I build something cool but totally off from what they expected will that make me look better or worse?"
"Or are they just trying to weed out the people that can't code at all?"
In the end I didn't get a follow-up interview and they refused to give me any feedback on how I could have done better on the take-home assessment.
Back to the OP:
One such example would be to do a live code review. This could be done asynchronously or synchronously. It could allow actually talking through topics and issues that relate to the challenges in a real software project. This would allow to surface much more of the knowledge in an experienced software engineer.
Yeah, I had a very similar experience recently. I refactored some things because the code was weirdly circuitous. Should I not have? I wrote a lot of explanatory notes. Was that a bad move? Ultimately I spent several hours on it and my response was basically "Didn't work." I realized that they had some automation set up that would spin up the thing, send some HTTP requests, then shut down. No human ever actually looked at my work, so the time I spent wondering whether I should or should not refactor was probably moot. Didn't feel great.
> One such example would be to do a live code review. This could be done asynchronously or synchronously. It could allow actually talking through topics and issues that relate to the challenges in a real software project. This would allow to surface much more of the knowledge in an experienced software engineer.
From my understanding of the post, this was the initial screening phase as it was in response to OP's application. In other words, this is what every candidate who passes the application screen (the weakest one) is sent.
Let's say they have 100 candidates for this role. A proper code review here should take ~45 minutes to an hour. Even 15% of the candidates requesting a full code review - regardless of synchronicity - represents a 11.25-to-15 hour time commitment from the hiring team. For the initial screen. That is asinine. No proper organization would accept such a large time sink for so few candidates at this phase.
As I've said already multiple times in this thread, OP very clearly does not understand the asynchronous relationship at play here, and then based much of their interactions & interpretations on this misunderstanding.
Exactly! You are actually considering all the possible ways in which your approach might be wrong and in which the time would be wasted. You are assessing risk before you take action. You are analyzing the subtleties in communication, you determined that the wording relied on a lot of subjective vocabulary and you know that this leads towards misunderstandings. This is, in my humble opinion, the professional way of making decisions as a software engineer.
From reading the other comments, it seems like there are a lot of mind-readers among us. ¯\_(ツ)_/¯
I think it’s telling that the comments here are full of people noting mistakes that you made in your interpretation and/or execution, and rather than trying to learn from the experience, you appear to just be doubling down on your insistence that you’re entirely in the right.
Yes, there were flaws in the assignment and communication with the company, but there were also flaws in your approach. You’d be better off to try to take away some lessons here rather than blaming everyone but yourself.
Frankly, if I were you I’d consider deleting both your comments here and the blog post. I don’t think they reflect well on you as a candidate for future roles.
Well said. The author of the blogpost is taking this all too personal.
While also seemingly ignoring their own shortcomings in their approach.
I'd see it more as a learning opportunity because there was clearly a huge
gap in the intended interpretation of the take home and their own interpretation.
> Frankly, if I were you I’d consider deleting both your comments here and the blog post. I don’t think they reflect well on you as a candidate for future roles.
That would be a wise move at this point. I keep wondering what the point of
the post was?
I have done this dance a few times. My learning about interview assignments is review the written grading rubric before reading anything else. If the grading criteria lacks specificity then don’t do the assignment. If there is no provided grading rubric in advance then don’t do the assessment.
I was sick of working with this guy halfway through his blog post. To me it was pretty clear what they wanted, it was a "show us what you can do" kind of assignment while he wanted handholding and constant validation, already making excuses for being late and avoiding IMAP/POP due to "complexity" when applying for a backend email engineer position.
Yes, they should've told you not to bother after you sent that proposal. But they were completely right to ultimately reject you. You're a lot of work.
I just can’t get past the fact that he didn’t even deliver on the core deliverable of “ The project is to build a minimal, terminal-inspired email client”. Nothing about the demo I watched on YouTube looked “terminal-inspired”
And they failed for it, because they didn't follow the instructions. Nit picking about the requirements and being all "well, _technically_" is both a common personality trait among programmers and _THE WRONG WAY TO THINK_ when interacting with humans.
Lot of people are very spicy in the comments, but for a mid-level position mind-reading shouldn't be required.
Not to mention that the whole problem is that the fucking hiring manager was too slimy to actually go to the technical team and ask them what they think about the proposal, and too lazy to take the time and energy and answer like a normal sentient being, and instead sent this autoresponder-level bullshit reply.
a wrong kind of backend though.. The position is "Email Backend Engineer" and the backend OP built was "web backend"
(OP outsourced the actual "email backend", that is the thing that stores and sends/receives emails, to a third-party products. Probably not the wisest solution if you are trying to get hired to work on email backends :) )
I think rejecting a candidate for almost any reason that those making the decision feel as important is okay, but wasting people's time is not.
Just reading the thread is obvious that the requirements were sufficiently vague -- which, again, is not necessarily a problem -- but going about this in a very disrespectful way is not.
(And yes, of course, the hiring manager always can say that "oh I did not want to reject them before they sent the finished assignment, because they could have surprised us with their code!" -- which, while technically true, but I think simply showcases the absolute uncaring laziness that we see from many companies.)
Well, you are a hiring manager, and you get an email filled with irrelevant decisions (i.e. the parts which do not affect scoring much), and the only part relevant to assignment text [0] is:
> The UI will be kept simple, showing pagination for sent and received emails. In addition to the requirements of the assessment, there will be a login screen ...
How would you reply to that?
One option is to tell them that the document looks fine, but also add that it does not actually describe the parts they are going to grade on (such as: "which part of aerc/mutt is does author imitate?"). But this is a bad idea, because the likely reaction of candidate is to spend even more time on spec... and the assignment is not about writing specs, the specs are not in the rubric, and you don't want to waste candidate's time on asking for docs you don't care about.
Another option is to tell them: "This spec is so bad, I don't think you can possibly pass. Bye-bye!". This is even worse, as it can potentially reject good candidates who are just bad at writing documents - and there are plenty of them.
So I think responding: "Looking forward to receiving your submission." is the pretty OK answer.
(I suppose the in the non-interview settings, if this was a junior engineer, I'd might also add something like "And don't forget to make the pages you write be terminal-inspired, as the requirements say, see videos of people using [aerc], [mutt] and [himalaya]" - but I can see why this was not said in the interview. After all, testing that the candidate can read and comprehend a tiny requirements doc is a part of the interview)
Like I said, it's okay to reject the candidate for submitting something that's implies they are not a good fit.
In my interpretation the "righteous indignation" part is that the candidate tried to make sure that they are not ending up wasting their time and not chasing some pipe-dream (which is bad for both parties after all, leaves a sour taste in the candidate's mouth, and ... apparently does not further the good reputation of Kagi), yet ... that is exactly what ended up happening, because the fucking ~~money lenders in the temple~~ hiring mangers in the corposphere are not just useless individually, but are actively harmful as a cohort.
(Of course, the people deciding to hire hiring managers usually have good reasons to do so. And companies are not expected to babysit every candidate, but since almost always the power dynamics favors them they ought to be the more generous and proactive in the transaction. But at this point in the analysis we are just a few steps from declaring that it's cough late-stage capitalism's cough fault.)
Imagine yourself in the hiring manager's place. Assume all you know nothing about the candidate, and all you have is this email he posted in the blog.. How would you respond to it? Keep in mind you have no idea that the candidate is going to ignore all the requirements, you have not seen the product yet.
I do (and I'm very happy that I got one more insightful and detailed reply), but I think many people here dismiss how easy (and thus common) this kind of fuckup is.
And I expect more from professional hiring facilitators.
I would tell them to write more about the UI/UX, do a mockup whatever. Emphasize that the people doing the evaluation looove TUIs so anything web starts with a bit of a handicap, etc.
I think your response will select a different kind of people that Kagi wanted do.
"I would tell them to write more about the UI/UX, do a mockup whatever." is effectively changing the take-home task from "write code", to "write spec, including mockup, get it approved, and then write code". This can be a valid skill, depending on position - but it also means that applicants waste their time on the writing documents that are not part of grading rubric.
And this brings us to a second part: "Emphasize that the people doing the evaluation looove TUIs..." - this is giving a really big hint. It would be a valid response _if the goal was to get this particular candidate_, say because unethical recruiter is getting paid for each candidate placed, and does not care about Kagi's own needs. But does Kagi (or anyone else, really?) want the kind of candidates that could not even parse out the simple doc? Because they were not subtle about loving TUIs in the requirements, mentioning them multiple times and giving examples.
So I am pretty sure that these kinds of strong hints would be against the interview rules, and there is no "fuckup" in this regard.
If I were to write those replies... I don't actually know their policy about how much help can they give to candidate over email.
If reviewing intermediate docs is too much, I'd have to refuse the candidate firmly: "Reviewing documents mid-assignment is against a spirit of this question. Please write the code using the best understanding of the assignment, and I am looking forward to receiving your submission."
If reviewing those docs is OK, I'd say something mildly encouraging - as the document as sent is not wrong, it's just incomplete. "This document contains no glaring defects"? "I can imagine a passing submission that would follow the plan outlined in this document"? "The results will depend on quality of implementation and how well you've followed the assignment, but so far I see no blockers"? "I cannot promise anything until I see the code, but that can potentially lead to passing assignment"?
Some of those would be better than "This is all very exciting. Thanks for keeping me updated" -- but looking at the candidate, would this make a difference? Judging by the tone of the post, and how he decided to proceed because "this is somewhat of a positive answer", even those harsher responses would end up the same way.
I understand the frustration here. I assume Kagi has a lot of candidates and is trying to identify the best ones for handling ambiguity. While I can appreciate the interviewee’s desire for more information or a rubric, it might potentially hurt the effectiveness of the take-home assignment.
During my college days, I had an on-campus interview with Microsoft that served as an assessment for whether I would be eligible for more in-person interviews in Seattle. The interview question was open-ended: “I would like you to design an alarm clock.” It seemed like an unusual open-ended question that prompted me to consider several critical factors, such as the intended audience, physicality, visibility, and audibility. By considering these factors and not immediately presenting a solution, the interview team informed me that I had passed the test. If they had provided me with more specific instructions or guidelines before the interview, they might have missed a signal that was important to them.
Unfortunately, I did not advance to the subsequent rounds at Microsoft. I interviewed during the fall semester. While I made it thru the full process, I was informed that I lacked confidence, but wasn’t a no. I was encouraged to reapply in the spring. The initial experience had left me drained and disheartened, so I decided not to pursue a second attempt. I now deeply regret this decision and wish I had given it another chance. If anyone is in a similar situation during their college years, I strongly advise them to avoid making my mistake. Working at a FAANG company could have changed my entire life and career trajectory.
> The initial experience had left me drained and disheartened, so I decided not to pursue a second attempt. I now deeply regret this decision and wish I had given it another chance. If anyone is in a similar situation during their college years, I strongly advise them to avoid making my mistake. Working at a FAANG company could have changed my entire life and career trajectory.
If this applies to you, this is probably the best advice anyone can give to you. I was granted an in-person interview with Microsoft as a freshman back in the early 2000s on account of my stellar GPA. They were the company to work at back then. Unfortunately, I didn't solve the brain teaser. Disheartened, I refused to try again even when they invited me to try again the next few years, and I went with an old dinosaur company instead.
I realized my mistake shortly after starting to work at the dinosaur, and clawed my way tooth and nail into a job at Microsoft. It took me 2 years, but it was the single best thing I have done for my career, and changed the course of my life for the better.
The uncertainty of the assignment and the lack of responsiveness during this week-long unpaid endeavor are inexcusable. Your anecdote strikes me as a totally different process as Kagi's in every possible way.
The problem is not the lack of a "rubric", it is the sheer amount of effort expected from the candidate, especially related to the unprofessionalism of the response. This process is set up to waste a maximum amount of time from candidates without any sort of feedback, and no proof that a reward even existed.
How many dozens of people completed this assignment and got rejected? How many of those assignments were even reviewed by the manager? Did anyone actually get hired? "Giving details would make this less effective for the manager" doesn't begin to excuse this behavior.
Honestly this hiring manager casts a negative light on the entire organization. Do they treat everyone this poorly? Business partners, employees?
I built an email client, it sent emails, it also received emails, it had an intuitive and speedy user interface, and it touched on fundamental topics for a web-application with users, quintessential topics for a back-end engineering role.
Somehow the only criticism I got is that it was not simple enough.
Perhaps the moral of the story is that many managers have no idea what the word "simple" implies in terms of software.
This is a classic situation:
Engineer gets asked to implement the "do what I mean" button by some manager, this is a magic button that when pressed will do whatever the manager wants at that moment. The manager acts shocked when they are told that this is not a simple request to fulfill. The manager thinks: "I am being tricked! I merely asked for a single button!"
> I understand the frustration here. I assume Kagi has a lot of candidates and is trying to identify the best ones for handling ambiguity. While I can appreciate the interviewee’s desire for more information or a rubric, it might potentially hurt the effectiveness of the take-home assignment.
I mean, potentially they failed the real test by asking all the questions - Kagi specifically say they're looking for people who can deal with an open-ended project, and instead of just deciding to do something cool, they seem to spend a lot of time demonstrating that they can't deal with open-endedness by asking a lot of questions and coming up with a detailed spec and asking for approval before starting.
That's not to say what Kagi is doing is good, but it's probably for the best for the candidate that they didn't get the job because it sounds like they wouldn't flourish in that kind of environment (which is better to know before starting a job, instead of starting a job you're going to hate and then burning out or failing probation)
nope, they asked the clarifying question, and got the response that there are no business requirements: "That is part of the assessment itself, see what kind of extra features you can come up if any."
They were failed because they could not deal with the answer, they were not failed for act of asking the question itself.
Actually happens sometimes in my work... Q: "Are there any requirements on the database used?" A: "Nope, just make sure the final system will be able to handle the required load"
I feel like Kagi was just asking "Impress us" and OP misunderstood the assignment by actually building a solid project and handling edge cases that no one cared about.
Anyways thanks for writing this up OP, it was an interesting read and I am a Kagi customer so I liked learning about them.
they actually did not... the original requirement said "a minimal, terminal-inspired email client" and "take inspiration from existing terminal email tools like aerc, mutt". This is nowhere near them, not even close.
Not sure why OP makes no mention of this in their blog post - perhaps they thought those parts were unimportant? In which case I am not surprised at the results.
The project was well made, but my read on it was that they wanted to be shown something interesting. Even if it wasn't as well made of polished, I got the impression they would have preferred something "fun" or imaginative.
This article perfectly highlights the perils of unbounded take home tests.
A timeboxed test for a small group (say, no more than 10) of final candidates can be an excellent differentiator.
The problem here is Kagi have omitted the timebox, so less confident and/or experienced candidates go to the ends of the earth to deliver something that was never going to meet the mark.
Had Kagi said “spend no more than 4 hours on this” - and arguably reduced some of the requirements accordingly - then your man here wouldn’t have wasted a week (!) of his time and Kagi’s.
It’s a real shame that so much time was wasted by the candidate, but the requirements are very clear that “This project tests your ability … to deal with ambiguity and open-endedness”. He’d already dug a hole for himself before he’d started, unfortunately.
It's a tough lesson, but a good one to learn early: never waste time with interview take home assignments. This is especially true for broad and ambiguous assignments that you will need to sink lots of time into.
What if there are 300 other applicants? What's to disincentivize giving all of them the assignment even when there's only one open position? There is no guarantee that anybody will even clone your repository.
In a live coding interview, the company is committing valuable engineering resources to evaluate you. You have some assurances that they actually want to hire someone and not just get free labor or waste your time.
I don't mind take home assignments per se, but it seems like there should be a reasonable "this should take this long" time boxing of the thing so people don't waste a ton of time. I've done take-home things a couple of times and they always had "this should take 12 hours" or whatever time amount.
The "simpler" solution part stood out to me though. Given that Kagi was putting little effort into the thing, I wonder if his verbosity/try-hardness might have been a turnoff? Afterall, part of the thing is determining if you want to work with a person, my impression reading his blog post was.. mildly turned off? There was just a bit of desperation to it (which might be from real factors!) but still. That sounds mean and I don't like saying it, that's just kind of the vibe that hit me.
Yeah the company has to give a timebox. It's true that candidates could abuse that, but open ended "impress us" is asking for trouble for both parties. And honestly you should be able to tell of someone spends 20h or something you said to take 4h on.
My take on the proposal: the hiring contact is not the person who wrote or will review the prompt. The candidate sent this long message and I doubt the hirer even read it. That's some nativety on the candidate's part, but the company absolutely should indicate "you're overthinking this"
I think this is a classic case of over delivering.
The requirements to spin up and obtain live services (postmark, torso), is a bit much, especially for a take home assignment.
Personally, If your app can’t be spun up and tested with a simple "docker compose up —build", then you are doing something wrong.
My rule of thumb for take home assignments: if I spend more than a few hours on it. Then I am probably over delivering and need to rethink approach and remove shit I don’t need. There’s no way in hell I’m spending a full working day, let alone a _week_ of free labor for these assignments.
Note: also not a fan of "take home assignments" either. Of the few companies that ask for these, I consequently ask to be compensated for my time. Only 1 company agreed to compensation, but others declined and I moved on.
While I would never do a take-home assignment for a job, I think your expectations are out of line with the current buyers market that has hundreds or thousands of applications for every position.
Your solution is the same old thing thousands of others could produce. They want more than a programmer that knows some languages, tools and libraries.
I think they might give you 10 more minutes of their consideration if you simply produced a command-line program in Go that used sockets to talk protocols to servers.
I feel Kagi's entire point of the take home assignment is to see how people handled assignments if left to their own devices with little instructions to go off.
When good companies are trying to hire engineers, they don't hire based on skill alone. They also try to hire based on how a person performs given certain constraints like time and ambiguity.
Sharing a proposal and asking for more time was probably the polar opposite what they were looking for
I think a lot of product managers prefer to see simple, fast solutions to a take home assignment. Understandable and turned in on time.
Feels harsh to jump on them for it, but it seems like they forgot the job spec when doing the take home. From that list Kagi wants to confirm:
- Proficiency in Go
- Strong understanding of scaling and maintaining backend systems
- Solid understanding of containerization technologies like Docker
Nailing the take home would require a project that demonstrates all three. Of those I think they maybe do okay on the second one, though offloading to paid services is maybe cheating a little there. I don't think they do well at either of the other two though.
From the detailed spec, one might expect the project to demonstrate those things, but the code as submitted just doesn't.
You might not know this, but implementing Templ with PocketBase is non-trivial. It is actually impossible to do so unless you can read the source code for PocketBase yourself. Pocketbase is a golang framework that doubles as a library, PocketBase implements an entire backend, it is a very real project by an extremely adept author. You would be lucky to work on a Golang repo as featureful as PocketBase.
The author didn't really ask for a code review, though a lot of people imagining themselves in the position of technical hiring manager are already taking care of that.
Anyway: I don't think homework assignments are a valuable mechanism for filtering out candidates. Setting aside shoulds and shouldnots and the ethics of it and everything else, it simply provides a really poor signal for the value of a potential candidate.
Especially today, when something like this particular assignment could just be pasted into any of the popular LLMs and returned as completed within a couple hours.
If an organization wants to hire developers that can take vague project descriptions and convert them into code that may or may not do whatever was in the heads of the people that wrote the project description, then (a) that's a bad practice, but (b) it would be better for everyone involved if you handled this over a video call. If your staff are too busy to do a small pile of video calls with potential candidates over a couple of weeks, then they are too busy to properly onboard a new candidate and you should probably just pack it in. (i.e., they are not too busy to do this.)
If the goal is to hire somebody that knows IMAP, SMTP, POP, etc. inside-and-out and can crank out RFC-compliant code without using any of the third-party libraries that already exist for this sort of thing, then the assignment description should have asked for that.
Homework assignments are an unfortunately common practice, but worst of all, most of them are carelessly designed.
> Especially today, when something like this particular assignment could just be pasted into any of the popular LLMs and returned as completed within a couple hours.
That would have been significantly more in line with what the interviewer was looking for and would have produced better results. It’s a take home test. Use the tools at your disposal.
I'd think it's more along the lines of "candidate can work with ambiguity and can self direct and use tools at their disposal to accomplish something". Things they explicitly asked for which the candidate completely failed to deliver on.
Seems ultimately the applicant and company weren't going to be a good fit. An interview is a process where both parties evaluate each other. This applicant set the bar too low for this company and wasted time when the signals were there from the start.
Although I agree with you regarding leetcode and the lack of feedback on rejections, I honestly don't understand the dislike for take-home assignments. If you truly enjoy coding, you could see them as an excuse to work on a new hobby project.
My brother on Earth, if I wanted to work on a new hobby... unpaid coding for a company wouldn't be on the list even if I had to think of a thousand hobbies.
You had the right idea to begin with - never do take home assignments.
Second - this is the wrong approach. Nothing wrong with writing a small doc. Write it, then implement the necessary features (terminal inspired client that can read and send email), and test it. Literally a cli for read and send, tack on curses or use imgui. Do IMAP/POP/SMTP or emulate it. The job position is for a backend role in the email team.
From this post, it seems like the hiring manager is either not the real hiring manager (maybe just an HR agent with this lofty title) or was himself hired without any screening or on the job evaluation for reading and responding to emails (which is a big part, IMO, of any manager) or has multiple roles to play and has no time to devote to hiring.
I feel sorry for the author to have spent more time on actually doing the assignment after that reply.
I went through something very similar — spent several days on a take-home assignment, took it seriously, carefully followed all the instructions and delivered everything.
Then I waited for two weeks, and all I got was a clearly templated response.
The worst part isn’t even the rejection — it’s the vague replies, or no reply at all.It just makes you feel like your time doesn’t matter.
Thank you for writing this. We really do need more people to speak up about these things.
Very insightful idea here. If you work in software, requirements engineering is a key task. Figuring out “would this deliver for you?” is a key question you hash out with clients.
But the absurdity of take home work like this is the company feels obliged to keep their requirements secret. Thus, by doing the task not knowing if it will be up to spec, you compromise on your most important skill!
Failing an assignment might cost you one job opportunity, but complaining about it publicly can harm your reputation and limit your chances with a lot of employers.
In the long run, venting online can be far more damaging than the initial setback - and a huge waste of time!
I see your point, but I think it depends on the situation. In this case, he misunderstood the task, and instead of reflecting on his mistakes, he complained about it online. Not a great look.
Honestly, you lost it with your first email asking for more direction, and every email after that just made it worse.
This was part of the assignment:
> This project tests your ability not only to code, but to deal with ambiguity and open-endedness, which is essential in R&D projects like Kagi Labs.
It was vague on purpose, they wanted to see how your decision making process goes with what you fill in the blanks with, without needing hand holding. You seemed to ignore this critical piece and went further and further against it with each message.
I read so many responses (the replies here to this post and to so many other similar posts) of the form
"Oh, I simply refuse to do interviews [of format xyz]. It's just not worth my time..."
I feel like such a worthless piece of crap when I read these. I apply to hundreds of places, and when someone comes along that even wants to talk to me or a miracle happens and I get to any sort of technical round, I simply DON'T HAVE THE OPTION to be turning them away for any reason.
Like, these companies have all the leverage right now, and lots of us have none. Responses like "just tell them no" seem so tone deaf. I guess some of you must be seriously baller, but it seems otherworldly to me.
This is close to how I feel when I read posts on Linkedin "demanding" that companies dispense with some unethical hiring practice (e.g. keeping up job postings for which they are not hiring, rejecting candidates without feedback, endless interview rounds, etc). I'm like, "Dude! The people who are doing this do not give a rat's ass about your preachy Linkedin post. We have no leverage here. Spare me your clickbait."
Without wanting to be unkind, this author misunderstands the purpose of the exercise, which is to demonstrate to another human being that you can write code that meets the company's standards (as in, would pass PR review), not to ship a fully-functional product solo. The initial email makes it explicit that they don't care if the product even works ("Can use a fake backend") and that the specifications are deliberately open-ended. A little intuition and empathy can tell you that they probably are not going to spend many hours reviewing a complex submission, so the project should optimize for demonstrability of code quality, not completeness.
"I would like to know what kind of response I could expect..." - This is also established from the beginning: you can expect either a "Looks good, let's do some interviews" or a "Sorry, not interested," based on the code you submit. They can't narrow down the choices prior to your submission, because they're grading your submission, not your proposal document with an extensive list of details that they already told you they're mostly ambivalent to.
"So it is funny that my project is so weak, yet it made them update the guidelines to something stricter." - Main character syndrome. As someone who has been on the other side of these kinds of reviews, the far more likely explanation is that they kept getting submissions where the build instructions didn't work (which is not disqualifying by itself; the authors may not be on the same OS as the reviewers) and they got tired of spending time dealing with it.
Ultimately, the failure here was not a technical one but a social one. The author tried very hard to do the thing that it seemed like they were asking for, not the thing they actually wanted. The hiring manager's unwillingness to engage with the proposal doc was itself a form of communication that they were not interested in this level of detail, it was just an implicit one.
It's common for engineer types to want all that kind of communication to be explicit, and I have a lot of sympathy for those folks, but the reality is that teamwork is a skill, and the ability to suss out what a stakeholder actually wants, but isn't saying due to incomplete information and/or office politics, is a reasonable thing to select for. The ambiguity of the prompt is a feature, not a bug: it's kept the author away from a company whose communication style they're not compatible with.
(that said, this project does sound excessively complex for a take-home)
> A little intuition and empathy can tell you that they probably are not going to spend many hours reviewing a complex submission, so the project should optimize for demonstrability of code quality, not completeness.
> "I would like to know what kind of response I could expect..." - This is also established from the beginning: you can expect either a "Looks good, let's do some interviews" or a "Sorry, not interested," based on the code you submit. They can't narrow down the choices prior to your submission, because they're grading your submission, not your proposal document with an extensive list of details that they already told you they're mostly ambivalent to.
Absolutely spot on, and you ID it later with "Main character syndrome", but it is so very clear from this post's tone & content that OP expected a symmetric outlay of effort & focus from the company's side. They thought they were the main character.
That's a fundamental misunderstanding that seems to have predicated a lot of their ultimate response: they feel as if they were entitled to much more effort from the company than they received. Such is often the case with strong entitlement, it's nearly impossible for the person suffering it to see it.
> suss out what a stakeholder actually wants, but isn't saying due to incomplete information and/or office politics
What you are saying sounds a lot like this:
"it is common for engineers to communicate properly, but people above them prefer to be vague, because plausible deniability is a political advantage to them at the detriment of the engineer"
EDIT: I originally had an escalatory reply here but have thought better of it. Trying again but without being an asshole: yeah, what you describe is one way miscommunications can happen. They can also happen in other ways that may be the engineer's fault, or nobody's at all. They may not happen every day, just like how you won't be deploying a new service every day, but anyone who works in an office is going to need to be able to deal with them regardless. Highly functional organizations are so because they can recover from mistakes, not because they don't make them.
I want to make a lot of points about this, so many that I sort of don't know where to start. Here they are in no particular order. I feel pretty strongly that all of these things are true at once.
1) An open-ended take-home assignment with no expected time limit and extremely little guidance is a very poor choice for an initial assessment of a candidate
2) A week of work is way too long to spend on such an assignment unless it explicitly says it expects that much time (and even then, I'd decline)
3) It doesn't seem like what OP produced in that week was terribly strong; it seems heavy on documentation and low on usable code. I see why he was passed.
4) Kagi as a company has always seemed to have an extremely fire-from-the-hip, whatever-Vlad-thinks-is-cool mentality, and I don't honestly trust it much as a result; it also seems like it therefore would not hire someone who works the way OP does.
5) Companies implicitly hire people who are similar to the ones they currently have unless they work strenuously to avoid doing so (in a personality and technical knowledge sense; this is not a comment about "DEI" or something)
6) Asking someone to put in a lot of time and effort and then providing essentially no feedback is an extremely common dick move
7) Take-home assignments should be compensated, if only some token amount, to express gratitude to the candidate for their effort. The better long-form take-home assignments do this already (Automattic is one example of a company that does this, and their other behavior aside, I think it's a good move)
All in all, nobody in this article comes off particularly well. I think OP managed to demonstrate why Kagi has this assessment while also producing an effective indictment of the assessment as a poor choice of hiring tool. Even so, I'm glad OP posted this, credit to him for showing what the hiring world is like right now. It's rough out there.
I'm fascinated to see how modern interviews especially those with take-home cope with developments like LLMS. But one thing that probably makes sense is
introducing more ambiguity in the requirements for the candidate. That's what
I'll do because we are no longer just testing for technical skills but also
judgement, autonomy and good taste. Todays developers are more like builders
and to a large extent product developers.
The actual software engineering and coding is now a lot less valuable as compared to
building something useful within constraints. Anyways, here are my thoughts
1. The interviewee failed to truly appreciate the need for ambiguity. Some people
felt the interview was over after the submission of the proposal and I agree with
them. Coming up with your own requirements was necessary. And this requires your
judgement in determining what was essential or not. This skill is surprisingly rare.
2. Good taste is also now more important than ever. It is also hard to test but the
interview questions were really good in that regard. They gave you nudges towards
the ideal state and left the door open for you to determine the best "path" towards
that goal. This tends to reveal a lot about a developer that they might not even be
aware of - like over engineering. For a take home like this, I felt the proposal and
the half a dozen AWS services was not in good taste. It was probably more complicated than it needed to be.
3. Deadline and deliverables. Exceeding the deadline is a negative. The final
solution is not terminal inspired and those alone may qualify as basis of rejection.
4. Of course the hiring manager should probably have rejected the proposal but will that be fair on other candidates? I also feel giving you access to a hiring manager
is a stroke of genius. They can assess you via the kind of questions you asked. And I doubt this candidate asked the "right questions". Right here means that they understood the point of the take-home and also what needed to be built. I didn't
get that impression from the candidate.
5. I truly believe take-homes like this should be paid. I just don't know how it
might work in the real world or whether they are sustainable. The payment here is
just a consolation for those that never make it so that their valuable time will
be respected. Even more so because I feel a lot of people will probably diverge from the spirit of the exercise.
What I'm saying is that it should be able to interact with all the email services out there, not rely on a single 3rd party service for the functionality.
Sure, for the demo they can choose postmark as a mockup server. But he should keep in mind that postmark is not a hard requirement, eventually they need to switch to something more controllable/cost effective.
Failing to knowledge that is already a considered failure.
The requirements for even senior software engineers now are absurd (to say nothing of entry/mid level, which I think may not even exist anymore at most shops).
Companies are abusing their position and we should remember the most egregious of them and avoid them in the future.
I don't like the "take-home" interview tasks, but this one seems reasonable and actually interesting (maybe because my relatively recently found love for CLIs). If the language was C# I would apply.
Do I get some bonus points for being born in the same country as the founder?
I tell everyone up front (recruiters and companies) I absolutely don't do homework assignments as part of a job interview. If they're unable to judge my skill level from an interview, that speaks volumes about them, not me.
Furthermore, if they are even the kind of company that thinks they're so special they can or should even ask for that, then it proves they're the kind of people I want to avoid like the plague to begin with. So when I'm asked to do a homework assignment it's actually a huge time saver for me, because it tells me right up front to avoid them.
All developers should refuse this. But there are always enough that are so desperate they'll do pretty much anything.
I forgot my own rule: if the test is to filter out candidates that is ok, if it is to filter in, then it is nok. Filter out: small app with bugs, find the bug or add a small feature.
Take home assignments I feel have the second worst signal for any hiring workflow (behind automated leetcode), you give someone free reign to spend too much time working on something useless without seeing how they actually think or approach problems, you only get a solution at the end, and the solution, especially in open ended problems like these has very loose objective measures. If you want a long term hiring process, I'd just stick to a contract project, though personally just do monitored interviews; I know it takes up more of the hiring managers time, but the amount of signal you get from actually talking and interfacing with a dev is pretty useful
Assuming the recruiting manager is non technical, he should have referred to the team in need when he received the "specification" email, something like "what should I do with this candidate?". A 5min chat with engineers would have clarified the situation and 5min to send an answer to the candidate. Same for the feedback request.
Seems like he did none of it, and in this company people do not talk and everyone is left guessing what is work consists in, which resonates strangely with the take home assignment directives. What an horrible workplace it would be.
To me, it is difficult to concoct a plausible explanation for the behavior from this recruiting manager. Assuming that the manager is not a technical professional and is therefore clueless about my emails is one of the few explanations that make sense. Thanks for bringing it up.
This is the same thing that happened to me with an Expensify interview back in 2011. After that, I decided to only do interviews in person, or where the person was on a call with me. Take home tests are not worth the risk of this crap.
It's totally unreasonable to ask somebody to do so much free work, and the outputs are hard to judge, and likely not the candidate's own work anyhow, so almost entirely pointless.
The instructions seemed clear. They deliberately set an open ended task, and they want to see how well you independently add valuable features (to a well known format).
I agree that this is an obscene amount of unpaid work but if you're going to do it, you can't ask your hiring contact how to do it. It demonstrates a lack of independence, and that's something they told you they wanted for this role.
Independence in choosing an implementation is never ever going to give you good results at a job where you have to report to someone else.
I have been at those jobs.
Trust me (or not): even if you are given this kind of freedom at first, it comes back to bite. You want to get the person in charge to agree with your design for any project that takes a week or longer. Otherwise, the moment that your solution does not create the expected happiness in the stakeholders, you are in a position to become the scapegoat.
This sounds cynical at first, but it's just reality. Besides, imagine if you were the boss of someone... It would be of immense value to get a specific plan for a large project before it even begins. There is nothing good about the uncertainty of not having a plan.
Remember this: I did not ask for the plan to be given to me, I created the plan and offered it as a proposal. I am the one doing all of the difficult pieces, and serving it on a golden platter to the person in charge.
It’s pretty disrespectful in my opinion. It would be better if they took 90% of their applications and secretly threw them in the garbage than to waste applicants’ time evaluating them on vague criteria.
I have been lucky enough to be able to reject take homes. Next time, I might offer to do them for a flat rate of $200/hour but working for free is something I will avoid.
If it’s going to be used as a screening process, at least give clear guidelines and evaluate it pass/fail.
if you are asking candidates to complete a take home you should provide at least a partial rubric of how the project will be evaluated.
if you can't provide that then i think you haven't created one, and if you haven't created one i'm guaranteed to be wasting my time because you're just doing vibe interviewing.
“but to deal with ambiguity and open-endedness, which is essential in R&D projects like Kagi Labs”…. Also based on the nonsense responses by the hiring manager, it sounds like one of 2 things
Either someone had a vision and is saying ‘Read my mind’
The alternative explanation seems to be, there is no vision, and the interviewee needs to define it.
In either scenario, the amount of communication, feedback, specificity, lack of respect for the power dynamics is appalling.
Wow you got a lot of responses! I was given a take-home assignment from Kagi to build a RSS scraper where Vlad responded to only one of my four questions. I asked how long I should spend on it, and he responded with a single sentence saying as long as I thought necessary. After four days of work to get an MVP complete, I submitted the project. I received a single sentence rejection a week later.
Love the service. Very disappointed in the process. Four days for a take-home with absolutely minimal feedback is really bad. I decided to jump through the hoops because it seemed like a really cool company to work for.
I am sorry to hear that. It is high time that us engineers learn the lesson and refuse to work with any company that asks for unpaid work. There is no other way, but this only works if we are brave enough to act as activists, so that other engineers become aware of the abuse and circumvent it successfully.
The few that benefit from the abuse also benefit from keeping it hush hush. This is no different from obscuring the pay-bands for employees so that they can get away with underpaying the most vulnerable workers.
Same thing happened to me at Zipline, they give you a basic problem to solve in a take home assignment and then ask you to do something you’re “proud of” or to really show off your skills for the rest of it.
Never heard back after submitting, spent probably 10 hours on it
As someone who struggles with whiteboard and leet-code style interviews, I'm not against take-home assignments.
Just two things:
- If the candidate already has a portfolio, whether personal projects on GitHub, open source contributions, or projects they're willing to share directly, why do you need to make them implement some specific project? The whole point of a take-home assignment is the discussion that follows it where you talk about the code, which trade-offs they made and why, and you try to get a general feel for their understanding and enthusiasm for the domain. Whether they implemented something specific is irrelevant.
I get that this is often done to make lives easier for the interviewer, and to be able to relatively compare implementations between candidates, but every candidate is different, and since you want to judge their thinking the project should be relatively open-ended as well. So just spend some time to review their portfolio instead of asking them to implement the same cookie-cutter project.
- All take-home assignments should be paid. Period. Don't ask candidates to work for free. Agree on a time estimate, and pay them a fair hourly rate. The initial offer can be some average rate for the specific role, but if the candidate has a higher rate, then negotiate based on that. It's disrespectful to expect them to work for free, only to reject them afterwards. This way at least it wouldn't be a complete waste of their time.
I'm disappointed that Kagi's process doesn't take these two points into consideration.
That said, my favorite interview style for programmers is the code review. Show them a piece of intentionally buggy and incomplete code, ask them to mention any issues they find, and to fix them. This could be open ended and include performance issues, testing, etc. Then you work on it together, the interviewer almost taking a co-driver seat in a pair programming session, and the discussion is almost always relaxed and informative. I've been on both sides of this type of interview and it's always been a positive experience, and most candidates enjoy it as well. It's much better than a stressful "here's a blank whiteboard, implement this from scratch", a leet-code style puzzle, and much less of a time-investment than a take-home assignment.
Both are wrong. OP should have just done the assignment and not send proposals, deploy it all on server less deployments etc. Kagi should not have implied that the more the better
I know "minimal terminal email client in golang" sounds like a small feat, but it is not simpler than a webapp, it would be about equivalent, at least.
TUI libraries are more of a PITA than HTML. Also you gotta learn the entire API of a specific TUI library to make it work, this learning will never be useful to you again. With HTML at least you know it is a standard.
And remember, they told me:
> We always have a lot of candidates, some they do the basic, some provide a lot of extra features [...] From hiring point of view, we prefer stronger assessments.
and the instructions mention:
> Do the project in a way that shows off your skills as a developer
The instructions also mention "web app" as a valid solution, literally.
So why should I conclude that making the minimal effort would give me a better chance as an applicant?
There is simply no rhyme or reason to the entire process and communication from the company.
> We receive a lot of interest and applications for each position at Kagi which makes selection processes is extremely competitive.
This is the real take away. Don't put in outsized effort at companies that are highly subscribed. Everyone prides themselves on being "extremely competitive" and unless it's a FAANG (or well known quiet money fountain like Valve), it's probably not actually worth the effort.
If it's hot on HN, it's highly subscribed -- you just can't expect effort to be valued the same way when there are 10 applicants.
> We normally don’t provide feedback at this stage. We have had other submissions that were simpler and stronger, so we decided to continue forward with these candidates.
This would have been a great place to push and ask for a little more feedback (while being nice as possible since they have nearly no incentive to share more)... How could the other solutions have been simpler and "stronger" (whatever that means? more robust?) -- a few details on what you missed that they were looking for (tests? documentation? CI?) could at least add some value.
All that said -- finding a job is really hard right now. Wish you the best.
Probably Kagi is using candidates as "chatgpt" answers on what they want to further work on for free. I have already suspect this kind of shenigans 2 years ago and stopped my Kagi subscription. As for adverts on positions, ghost jobs likely. That is the current trends. As for Jose, don't be surprised say 3 years from now they contact you. Some companies pool potential candidates like backup boyfriends. I usually just flat out ban this kind of companies and actively spread these companies to my circle to avoid. The lost is really on these companies without further access to specific job pool (and even customers) and of course bad evangelism on their part for decades to come even if they have changed.
This is very speculative so I left it out of the blog post. But... 'market research' is typical endeavour for companies breaking into an unknown space, the decent ones will carry out focus groups with experienced professionals and pay for people's time.
This kind of "do something for X but we won't tell you how!" looks a whole lot like market research disguised as an interview, that way they can do the research for free.
So, as someone who has applied to a lot of jobs and has had a lot of jobs, I'm going to be a bit more critical here. I think the amount of information they gave is sufficient for a take-home.
Make an email client, email view+send, fake backend or real imap, handle plaintext.
At this stage, for a take-home, I'd start working and write down assumptions I made as I went along. I'm probably the opposite of the author, as a take-home (unless it's the last stage or something) is, in my view, a tester to see what a person can do within a few hours of work. I've had several take-home exercises during my time as a software engineer and they varied from "we have provided all details that a stakeholder would provide" to "if you have any further clarifying questions, please get back to us".
The most recent one I took a couple of years ago came with an internal library the company used. They said, "use this library to make a web app that takes advantage of these methods inside of it; create an app that simulates behavios using those methods. Do not spend more than 10 hours on the assignment."
I started coding, I threw something together that worked in about 6-7 hours and I was writing down assumptions as I worked as well as trade-offs from those assumptions. "I assume the user would not be bothered by a failure here, if reliability would be important, what would we want to do in the case of a failure? Retry? Back off retries? etc etc". I then provided the code and provided a list of improvements "Add unit tests to these 3 components, add integration test to ensure this functionality works end-to-end if its essential, improve UI, clean up code base, refactor these services, use a framework/library for the UI instead of hard-coded JS to make these few calls. I wrapped it up because I think I was 70-80% there."
During the next interview with the architect of the company, he said that the solution I provided worked fine, we discussed some things I did and he asked why I did them, but specifically, and this is why I'm commenting here, he mentioned that he appreciated the assumptions document, the future improvements and the "I stopped here because I'd rather get feedback on this and refine, as opposed to keep building something I imagine you want". He said that the ability to work up to a point where you hit the majority of the story and then get feedback on that incomplete project is better than having someone dissapear into a cave for a month to try and come back with the absolute finished product in their oppinion and then having to make changes to get it in line with what was actually wanted.
So I guess, become comfortable with uncertainty. If nothing else, ask only "hey, I assume you want something along these lines that I can bang out in 5-6-10-20-40 hours. If you're not happy with what you get for 5 hours, it might not be a good fit in general or if you think I prioritized the wrong thing in those 5 hours, then we can chat about that too". I am also saying this, because my current role is a lot more in line with what the author is looking for - they spend weeks refining requirements, they write documentation, they create mocks and they have meetings over meetings before I even know what I'm supposed to do and within a day or two they start realizing that what they described is about 10-20% off what they wanted or what is possible and the whole cycle of meetings starts again. Instead, and I've been pushing for this each sprint, I'm asking them to accept a certain level of unknown in a given story, we work on it, we see it behave and get used and refine it based off of that.
The author seems to want waterfall, but agile exists for a reason. Hell, let's not even call it agile or whatever else, in a real situation, you start doing and you learn as you move along. You refine based on feedback, you refine based on new experiences and based on new requirements. You work in the murky areas of someone elses mind. Or not, I don't know, but expecting a series of jira tickets with screenshots and deliverables from a company that just wants to see how you think, how you work with uncertainty and how you deal with unknowns feels... wrong.
This (and other terrible interviewing processes) seem like the epitome of a problem being tackled from the wrong end: Trying to filter things coming OUT of the hiring funnel instead of filtering the things going IN to the hiring funnel.
Case-in-point: Modern job sites (cough LinkedIn). Every job listing has HUNDREDS of applicants within an hour of the job being posted. It's ridiculous. It should not be _that_ easy to apply for a job.
The outcome is what we are seeing today: A company posts a job, is inundated with 100s/1,000s of applications. In order to filter out the 80% of applicants who aren't deeply interested in the role, the company deliberately assigns busywork/road blocks to slow down the process.
The other 20% of applicants will then spend days/weeks/months in the hiring process on intro videos, take home challenges, etc. Basically anything that can be throw at the applicant that isn't time with a human.
The takeaway for each can be broken down into:
- 80% of applicants: didn't spend anything, didn't get anything, don't care
- 19% of applicants: spent time doing some/all of the busywork, _aren't_ hired, end up very frustrated at the amount of time/energy/resources that was spent only to be discarded
- The 1%: spent time doing some/all of the busywork, _are_ hired, feel great!
I'm not sure what the exact solution is, but I know that:
a) it's a race to the bottom (with the bottom being full automation on BOTH sides of the hiring process),
b) I'd much rather spend a fair bit of time putting together applications for 5 jobs and be seriously considered than spend very little time on 50 jobs only to be immediately rejected or handed an assignment before I even talk to a real human being
If you post a JS position, you will get 1,000 or more applicants, so it is a huge amount of work behind the scenes to filter this down to try to find the vaguely worthwhile candidates to interview.
If you post, for example, a Clojure position, you will get 10s or maybe even a 100 candidates tops. And they tend to be uniformly more qualified because niche tech tends to self-select folks who want to explore outside the mainstream.
Of course, a lot of businesses want to use mainstream tech because "the hiring pool is much larger", but the flip side is "the hiring process is a lot more work", because of the volume. So, we get a crappy hiring process because they can't scale up a good hiring process :(
A bit OT, but I'm wondering how any fresh grad is supposed to pass today's hiring bar.
And, with the market flooded with experienced devs, why should any employer take a chance on a 100% green dev, when they can pick and choose from among 5+ YoE SWEs, who are desperate for jobs? As Shawn's post from yesterday showed, many of us are willing to work at much lower salaries, so even financially, it does not make sense to hire a fresher.
If there was enough competition that experienced people were willing to work then at the same rate as college grads now, then college grads then would be even cheaper.
Asking candidates to put in disproportionate amounts of time and effort is abusive. The hiring manager here seems to have taken all of 5 minutes to reject the candidate's multiple hours of work.
edit: commenters here are also missing the forest for the trees. If the candidate had done X, Y, Z to make a more compelling entry then it would be some other poor guy staring at a two line rejection of tens of hours of work. The process is exploitative.
the test asks for a very significant amount of work, and makes it clear that they are looking for more effort than the minimum ("Do the project in a way that shows off your skills as a developer", "see what kind of extra features you can come up with").
Yes. Because it's a vague unbounded take-home exam and when companies pull stunts like this, they're usually looking for candidates to go 'above and beyond'.
I've done take-homes before and have been rejected because I followed the prompt to a tee and only did the exact amount of effort they wanted, and was quite literally told that they expected me to do more (outside of the prompt). If you've done enough of this game then everything in the OP will be extremely familiar to you.
It seems like half of the commenters here believe they could implement a Golang TUI that sends and receives emails in less than 2 hours. Clearly they have never tried to build a TUI, let alone consider any kind of the issues that might occur between different terminal emulators (OS, Terminal emulator, Shell). The web is king for a reason.
The only explanation I can presume is that these people are mistaking TUI for a CLI app, which would be a fatal mistake.
No offense but even the readme is way too long and complicated. And the project itself seems like too much.
I often see people applying make this same mistake with their CV. They'll have a 5 page long incredibly overfilled CV with whatever they ever worked on. Instead, just make it one single clear and concise page so that the recruiter can quickly assess your selling point relevant to the position. They don't have time for all this so why would you?
I apologize if the tone in my comment was not the nicest.
You should know your worth as an employee. If a company asks for too much you must have the ability to stop yourself from working for an entire week with no compensation.
> kind' meaning something like 'UX improvement', or 'pretty UI',
They literally said they want a simple TUI email app
> If he wanted a simpler solution, he could have told me on March 18, when I submit my proposal.
When he said your emails could be stored in a fake database or loaded in memory it was obvious they didn’t want something over complicated.
No one likes over engineering, it can come off as egotistical, indulgent and more importantly a sign of technical debt lurking ahead.
Your proposal is long winded, you’re not the only applicant these people are looking at my guy.
I’m not sure what Fargate is (nor care) or even why you suggested AWS and all that other jazz but you clearly went overboard. No one wants a developer where you ask them for A and they return A B and a half finished C.
Yes, you’re allowed to ask questions but the ones you asked showed signs of overthinking and lack of initiative.
That thread is hilarious... The dude bootstraps his own search/browser company, an incredibly difficult space, becomes profitable, and sends his customers t-shirts they didn't want. A bunch of HN users who have probably never even applied at a startup are criticizing him for how he runs his profitable business that he funded himself. If he wants to shovel every dollar of profit into a furnace, who cares? It's their money.
Yikes, I love Kagi but if I'd applied for this position and they gave me this assignment, I'd have told them to take a hike. This isn't an afternoon project, it's a full weekend. Maybe two. And are you supposed to be writing tests and stuff for this?
Moreover, this exercise tells you so little about the candidate and their experience. Sure, you know they can code (or did they just use Cursor?) but you have no idea whether they are good and addressing prompts and not at making wise choices. Who cares if the program can send email? What matters is that the client doesn't choke after 10k messages are received or that unicode is handled well: that's the sort of actual challenges you'd be doing at a company like Kagi, not making enormous toy projects.
Edit: Despite that, it seems like a bold choice for the author to build a web app instead of a TUI like the instructions lay out. One could say "terminal inspired" could include web applications but I think that's a stretch.
To be fair, they do explicitly say "build a minimal, terminal-inspired email client" and then "Email client can either be in the terminal (i.e. a TUI) or a web app" below, so even if it's a web app, they expected it to be terminal inspired, especially from the next paragraph in "Inspiration":
> Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.
You can totally do terminal-inspired web app - http://www.coboloncogs.org/ is an extreme example, but for this assignment, even fixed-width font and compact layout would likely be enough.
Of course the OP in this post did nothing even close to that.
No, my assumption is that actually reading and following the requirements would increase the chance of the interview. It's like that Van Halen brown M&M Story - something that is simple to implement, but shows that the candidate actually read the requirements. Because after all, who wants to work with a person who just ignores half of the requirements they were given?
While we are talking, I was actually always wondered about candidate responses like that...
There is that spec, which starts with "minimal, terminal-inspired email client", and contains specific references to the products you want to imitate. Line 3 even explicitly says "Can use a fake backend", and to me, this shows pretty clearly that they don't care about backend, they need UI.
And then there is your submissions which basically has no UI to speak of. It's not even "minimal", as the basic features (like new mail indicator) are missing. Also, in no possible world you can call this "terminal inspired", it's 100% generic web app.
How on earth did you expect this submission to work, given the assignment they sent you?
Did you just skip over the original page and then forgot about it immediately?
Or did you read it, decided that "terminal-inspired web pages cannot possibly exist" and proceeded to implement completely different thing? In that case, why didn't you mention this in email ("and btw, I am going to ignore the part where you said I should build terminal-inspired email client because I think it's stupid.")
Or maybe you read the requirements, and randomly declared half of them "useless fluff" and decided not to implement them?
The (interview) game has changed and it will get worse because of more layoffs this year.
To standout, you have to be a bit creative and you must sustain yourself with your own company / startup. (4 basic instructions here) [0]
Companies do not care about you or your take home solution(s) unless you've built something that threatens their existence with a competing product that makes money and steals their users.
Stop playing by their rules with these interview 'games' unless you have lots of time. Spend your time elsewhere. You now have AI, and you should build a startup instead.
My honest feedback below from the perspective of a hirer. I'll start by saying I hate takehome stuff, for exactly reasons like this. It wastes everyone's time. They're fine as a 'last step before hire' thing, but not as a filter.
1 - Too much chatter. Part of the assignment is using judgment and working in ambiguity. I probably would have ran with what was given enough to knock out something small and local in an evening or two. Asking questions is usually fine, they even welcomed it, but seems counter to the original ask.
2 - Writing and sharing a proposal seemed like way overkill. You have to remember that these companies are now getting hundreds if not thousands if not tens of thousands of applicants, that is a lot to deal with if everyone does so. I think it's a bit of a disconnect...you feel like you're going above and beyond and being thorough, they feel like it's being a bit long winded and wasting time. That probably explains the nonresponse.
3 - The finished product seemed functional, but seemed a bit overkill on the infra and polish. This is probably a good thing to work with you, but ended up wasting a lot of your time if not being selected, which was the case.
4 - Maybe I missed something, but the requirement asked for terminal inspired. I'm not quite sure precisely what they meant by that, but didn't see any possible interpretation of that in the result.
Anyways, hope you don't take it too negatively or personally - you obviously are a talented individual and moreso seem to really care about your work. Just wanted to play a little devil's advocate with a different perspective.
The attitude of the blog writer in their interactions also feels off. Just reading this blog post makes me think that this person is difficult to work with, requires extremely clear guidelines and instructions, and has a hard time making their own decisions. Maybe this is a good fit for a large, established company, but startups have their own needs.
"Create a terminal inspired email client so we can do an alpha test with some customers" is a reasonable ask for an engineer at an early stage startup. Of course, there would be a bit more specification, but a lot of the details would still be up to the engineer. This applicant wants more certainty than they can get.
This is illustrated by the line: "I would like to know what kind of response I could expect from Kagi if I drive it to completion." This is not a great request to make. There's no way they can answer that question, because there is no certainty available. They're probably getting a few hundred or a few thousand more submissions to evaluate.
In your comments history someone replied to a job posting "Just an FYI, if you do a takehome project for William, he won't respond to you (not even with an auto reject)."
Seriously? A candidate puts in a week of work and he can’t be guaranteed a 30 minute discussion?
Nobody is getting a few hundred or few thousand submissions to evaluate. Nobody. If you are getting 1k applicants, at best 50 are asked to do a take-home and even then, not all at once.
If by some miracle 100 people did this to completion at the same time, there should be a notice to the effect that due to high volume, blah blah blah.
I don't really think it was intended to take a week of work.
…which is totally understandable… if the hiring manager had communicated that. They could’ve easily mailed back “Hi this proposal seems much more detailed than what we need for evaluation, please save yourself some time and energy.”
The author may have had issues (I personally don’t count “need clear instructions” as an issue - edit - I see they didn’t adhere to the TUI prompt), but the hiring manager definitely did.
> …which is totally understandable… if the hiring manager had communicated that.
I agree that the hiring manager could have handled it much better, but as a rule: If at any point during any hiring process you feel like you need to spend even close to a full time week of work on anything without being very explicitly told so, you are wrong.
I am a hiring manager and we do take home homeworks. I fully agree that this is a key piece of communication. I always take the time to tell candidates that although they have as much time as they want, we expect 2-3 hours of effort at most, to be respectful of their time. Without that, take home problems would seem predatory.
Interviews check for fit first of all, and especially in a buyers market the point is not to solve the assignment, it's to blow it out of the water.
It's meant for the person for whom it's ideally an afterthought. Tough reality.
Yes on this: It reminds me of when candidates ask me how they did at the end of the interview. It shows an extreme lack of decorum and empathy. What if you did terribly and I wouldn't hire you in a million years? Do you really want me to tell you that?
There's no good answer and asking a question like that shows real narcissism imo.
Conversely, if you can't handle some straightforward feedback to a candidate that took the time to interview you without violating decorum or hurting their feelings, then how can I expect you to be a good manager or supervisor? How are you possibly going to be able to handle minor personnel conflicts or provide guidance during the training period? It comes across as a complete lack of basic managerial skills.
Supervisors/manages don't usually do coding interviews though, especially in bigger orgs.
There is usually a separate interview stage with some sort of manager, and those usually have no coding.
Okay, but basic interpersonal skills are a prerequisite for anybody in a senior or team lead position, or any position that will involve code reviews.
I'm sympathetic to how awkward it can feel to provide honest feedback to a candidate, but look: we're all people here. I think we forget that sometimes when we're assembling hiring processes. As a candidate, you need some kind of feedback mechanism that allows you to improve even if you're not a good fit for a particular organization. And if you're involved in the hiring process in any way, you ought to be equipped to handle that.
This topic has been discussed ad nauseam on HN. In most companies, there is specific company policy that prohibits providing feedback to candidates. There is literally no upside for these companies to provide feedback to candidates that they reject (except Fake/Feel-Good Internet Points, only redeemable on HN forums). Really: There is no way around it, no matter how many tears are spilled about it on HN.
This is simply a defense of bad policy couched in unnecessarily dehumanizing language.
There is widespread resentment of this and many other common hiring practices in the tech sector, and that is further impacting both the quality of candidates as well as employee motivation and satisfaction. The upside for companies is higher quality candidates whose first experience with the company is a hiring process that makes the candidate want to work there.
I broadly agree with this being an unfortunate outcome but you do understand that making candidates who failed your interview want to work at your company is fundamentally limited in how much it actually helps you. Yes, yes, I know some of them may come back and pass the next time, or they tell their friends about how you were super nice and gave them great feedback, but this is pretty rare. If you're doing this, you're doing it out of the goodness of your heart, not because it helps your recruiting pipeline. And, even though I agree with the idea of providing feedback, assuming that people will have positive feelings when you tell them why you didn't accept them is misguided. I have friends who I know personally that have gotten interview feedback and not taken it well. Of course I tell them to shut up and stop poisoning the well for everyone else, but the point is that this is largely not the picture you are presenting it as.
Sorry, I wasn't clear. Providing constructive feedback to a candidate is unlikely to have a direct positive impact on the relationship between that specific candidate and that specific company. It's more of ... whatever the opposite of the tragedy of the commons is. A policy that, if improved, would broadly improve the quality of many candidates for many companies.
Companies have been optimizing for candidates that are an immediate ideal cultural and technological fit. They are all competing for candidates that are the idealized developer, with perfect social skills, a brilliant CV, and deep technical experience that is an exact match for whatever the company is doing at the moment.
That's fine and rational and all, but a necessary consequence of this is that that pool is quite small and there are lots of companies competing for those people. Meanwhile, there are a lot of very good candidates who are underemployed because they aren't getting the opportunity or resources needed to become those idealized employees. This is a game theory outcome where both parties are optimizing themselves into a losing position.
I've been employed in this industry, off and on, for a long time. I assure you that companies didn't always behave this way. There has been a clear, obvious, and severe decline in the hiring experience, and these policies are hurting the entire industry.
It's generally socially frowned-upon to go on a couple of dates with someone and then ghost them. It happens, but it's not considered good practice. We recognize that it's cruel but also leads to a more cynical, detached, overall worse dating experience for everyone. Saying "I don't think this will work out, you seem nice but you're not what I'm looking for right now" is difficult and awkward, but it's also a necessary skill that needs to be maintained. Sometimes people don't react well, but that doesn't make it less necessary: it closes a feedback loop that ultimately allows earnest people who are looking for relationships to learn and grow and become better candidates for the next relationship.
> In most companies, there is specific company policy that prohibits providing feedback to candidates. There is literally no upside for these companies to provide feedback to candidates that they reject
This is the long and short of it.
In the US at least, discrimination laws are expansive. You can -very- easily end up saying something that violates this and putting your company at risk, no matter how good hearted you were attempting to be.
How do you "accidentally" end up saying something that implicates you in discrimination on the basis of legally protected characteristics - what are some examples of that?
This has always felt like an excuse used by people who who just don't want to be caught in their own lies when asked to come up with a real, non-discriminatory reason.
The other comments gave good answers. A lot of people think it means saying something horrible and racist or something, but not at all.
As one pointed out, there's a "well you said it was X, but person Y who got hired did that too. And they're a different race or gender or religion, so that leads me to believe discrimination."
There's also you trying to be helpful, saying something along the line of "well you hesitated a bit and sounded unsure in your answers", only to find out they have some disability that caused that and now have admitted you're discriminating based on it.
Maybe you'll say "well, if I had known, I wouldn't have noticed it or cared." And a lot of candidates would likely say as much up front. But they don't have to tell you about it at all. See how that creates a weird dynamic?
Is it common? Probably not. But it obviously happened or else such rules wouldn't exist. It's one of those things that the bad actors ruin it for everybody. Bigots are never going to admit their reasons - good people will. But bad people will always try to take advantage, regardless.
I think it's more of a case for legal and HR being conservative and super defensive. Not sure if you've ever handled a contract with an internal lawyer, but in my experience they often go for crazy suggestions that the other side would never accept for the sake of protecting the company as much as possible. Might be the same here - HR/legal being super protective and the hiring manager not caring enough to fight back.
> How do you "accidentally" end up saying something that implicates you in discrimination on the basis of legally protected characteristics - what are some examples of that?
Say you say it was for failure to meet a specific performance standard (because that is the documented reason); then the ex-employee has a starting point for an discrimination claim by looking for evidence that trnds to support the claim that people who differ on some protected-from-discrimination axis who failed to meet that standard were not fired. No reason given, no starting point. In theory, this policy helps make false nuisance claims more work and less likely, but a substantive reason for it is that HR knows that they cannot eliminate all prohibited acts by managers that would create liability, so making it harder to get a starting point for gathering evidence is important to prevent valid claims from materializing. HR policy does not exist to protect employees from unlawful treatment, it exists to protect the company from liability for such treatment. Sometimes thise two interests align, but when it comes to information about firing decisions they do not.
There’s similar things that can be done with other prohibited reasons for dismissal, loke retaliation; but the idea is any information you give makes it easier for them to make a case against you.
This is also, in reverse, why, as a departing employee (whether departing voluntarily or not), you should never participate in an exit interview or, if you must as a condition of some severance or other pay or benefit, never volunteer any information beyond the bare minimum necessary; one significant purpose of such interviews is to document information useful either for potential claims against you or to defend against any potential claims you might have, including those you have not yet discovered, against the company.
Part of it is, if anything can be taken slightly out of context to imply something discriminatory, there are those who will abuse the system and sue. At a large enough scale this can become a real problem. If the company policy is "never say anything" there's nothing to be taken out of context, reducing the chance of a lawsuit.
I bet you this comes back to insurance, as many things do in the corporate world. Sufficiently large companies probably have insurance coverage for discrimination lawsuits, or at least employment disputes in general. The coverage probably costs less if you have a "no feedback" policy.
Who are you trusting as a technical interviewer if you don't already trust them to give negative feedback internally?
Do you not code review? Are you a rubber stamp "LGTM" shop that should just be pushing to main but cargo culted the ceremony because github has it built in?
I always tell people how they did. What went right, what went wrong, whether I think they're a good fit and if not, why not.
Because I see what happens to my wife when she interviews, and goddamn its brutal.
I had someone email me after being rejected at the final round of an interview. "Everything seemed to mesh just perfectly, and I'm at a loss to understand."
I broke it down for them. "This was nothing to do with you, and we would have had no objection to hiring you. However, the candidate who beat you out simply had more domain experience in XYZ area" and went on to say "For what it's worth, we had 500+ applications, of which we in-depth reviewed 100 resumes, had 40 first-round interviews, 15 second-round, and three final round."
They emailed me back to express appreciation and that though this didn't work out, it renewed their confidence to know they didn't "mess something up".
Since then, if we're at that point in a process and I'm rejecting you, I'll at least give you something to work with.
This is so important for people to understand, and its why I give people feedback.
People, being humans and prone to pattern seeking, assume that if they didn't get the job, it's something specific they did, or failed to do.
And sometimes, that's true. But for a lot of candidates, it just came down to another candidate being slightly better, or slightly cheaper, or some combination of value markers.
A lot of my interview feedback comes down to "I don't see any reason you wouldn't be a good fit, but we have other interviews and it's going to come down to value."
Some people will take this as me saying "Don't ask for what you're worth," or "we're gonna low-ball your salary." The reality is, we're a business, and if I can produce the same widget with person X or person Y and person X costs 10K less a year, I'm going with person X. Every time.
Yes we want to know. Framing this as an empathy issue when in reality you're just to afraid to be honest or afraid of any kind of conflict IS an empathy issue. At that point they're not a person. They're an annoyance that you want gone immediately.
Don't take this the wrong way, but I deliberately ask how I did because it helps me weed out interviewers who think like this. Not so much "how did I do?" as "now we're close to the end of the interview, do you think we're a good fit for each other?" I give my own feedback and talk honestly about points of friction.
I interview pretty well, but if I go into an interview with a company that wants hungry hustlers and I've spent the whole interview talking about kindness and team spirit, or if you think I don't know enough pl/pgsql to deal with your gnarly legacy backend, or I'm getting the vibe that none of the engineers seem to like working here, then we need to speak honestly about that.
I know I'm interviewing them. That's why we need to talk.
If they wanted to hire me enough to interview me, but at the end of a half-day of interviewing I'm going to walk away without a job, then they need to rewrite their position description so I know not to apply, deal with their morale problem, or directly ask me how much PL/pgSQL I've done. We both stand to benefit from talking about how the interview went.
But you also need to factor in their position in the situation right?
Like suppose they do hate their job. Do you expect them to speak that plainly and honestly to every candidate who asks "So how do you like working here?" and risk getting that posted to the front page of HN?
You're asking them to risk their own livelihood so you get a better signal for your own job search, that doesn't seem like a proportional trade to me.
Obviously I'm not advocating for complete opaqueness, but your interviewer is hardly ever in a good position to part with their true feelings towards questions like "How did I do compared to other candidates? How is it truly working here?"
I've basically almost always given direct and obvious non-answer to the first question: "I cannot tell you right now, because I'll need to write down and collate my thoughts. And I'm not allowed to share feedback directly, so your recruiter will be in touch with the feedback afterwards."
> What if you did terribly and I wouldn't hire you in a million years? Do you really want me to tell you that?
Yes, that sounds like extremely valuable feedback.
Why do you suppose asking a question like that shows narcissism? To me it shows a willingness to infest feedback to improve.
I will add the caveat that if someone asked me that in an interview I would likely give a non-answer because I’m not totally sure what all I’m even allowed to say.
Not everyone will take feedback the same. It's not worth the reputational risk.
A simple request for feedback is not evidence of narcissism or lack of empathy. Could be anxiety. Could be curiosity. Could be zeal. Could be any number of things. It's certainly not an "extreme lack of decorum" though.
It's okay to avoid giving feedback if you don't want to. I can think of a few ways to answer that question in a neutral or positive fashion to defuse the situation and legally protect the company.
Not to mention there are legal liabilities with sharing interview performance with candidates. "Oh but the interviewers told me I did extremely well on their interviews. Therefore it must be the case that I was rejected because of ${protected attribute X}."
> Yes on this: It reminds me of when candidates ask me how they did at the end of the interview. It shows an extreme lack of decorum and empathy.
If my interviewer stumbled over this it would be a red flag.
Really?? I always appreciated candidates that would ask that at the end - being willing to step aside from the pretense of professionalism to ask a real question and listen to my answer is a signal to me that this is someone who is willing to be real with me, not pretentious or perfunctory.
I do get what you’re saying, but I disagree, there is a good answer; and as is often the case, it’s an honest one.
I would because accept honest, authentic feedback that would support my efforts.
> asking a question like that shows real narcissism imo.
Precisely the opposite. Asking for criticism and genuinely being interested in what others think of you with the goal of taking the feedback on board and improving is the polar opposite of typical narcissistic behavior. As far as I'm aware that sort of self-reflection is inherently incompatible with NPD.
In other words, the author did a good job but failed because there was an implicit requirement to not try very hard.
> Part of the assignment is using judgment and working in ambiguity.
If trying to clarify requirements is not what they wanted here, they may as well ask candidates to pick a number between 1 and 10 and reject anyone who guesses wrong.
> overkill on the infra and polish
I know its an opinion, but hard disagree on both. Take home tests are intended to show off your ability. Without specific guidance, how can anyone guess how much showing off is expected?
Polish is only bad if it stops you from delivering. Rejecting something that was delivered for being "too polished" feels like you are saying someone did too good of a job.
---
In my opinion tests with vague requirements like this are more likely to be a different way of rejecting people who are not a "culture fit".
The author did a bunch of things that were irrelevant to the product (deploying to Fargate using Pulumi) while not delivering on the basic features that they were asked for ("terminal inspired email client"). If you watch the video it’s a super basic web app that has nothing to do with the TUI apps Kagi named you should use as inspiration. There’s no drafts. No CC or BCC field. No folders or labels (including sent or trash!). No replies. No threads. No shortcuts. There’s not even an indication of whether an email has been read or not. In terms of features it is very very very barebones, and I’m sure if you asked a few users all or most of these would be table stakes. Yes we can agree that "terminal inspired" is very open to interpretation, but spend a few minutes using the apps they gave as reference and you should be able to come up with a long list of features besides "send & receive email", which is the main thing they demonstrated. I get this is a short challenge, so pick a handful of things that are quick to implement and focus on those (like unread markers). You can even say this is a backend position so the UI shouldn’t matter, so then just implement them in the backend and make the simplest UI for it you can! They certainly "delivered" a product but I would not call it polished, and I would not confidently say this was a "culture fit" rejection - I see plenty of other reasons
> but I would not call it polished
The hiring manager above thought it was too polished, but you think it is not polished enough.
> There’s no drafts. No CC or BCC field. No folders or labels (including sent or trash!). No replies. No threads. No shortcuts. There’s not even an indication of whether an email has been read or not. In terms of features it is very very very barebones
None of these were asked for. Your guess as to what the true requirements are completely different from the authors, and others in this comment section.
Everyone seems to have different opinions on why the author failed. No one should have their time wasted on a take home test destined to be rejected because they guessed wrong.
And in this case, the author explicitly asked if they were on the right track before spending time on the test. Kagi had the opportunity to reject early or nudge the author back in the expected direction. Instead they wasted everyone's time.
I don't see how this can be interpreted in a positive way.
The candidate here focused on the wrong things, causing it to be overly focused in some areas while ignoring the main task which revolved around creating an interface inspired by a specifically listed set of terminal mail applications. So, yes it was both polished and unpolished simultaneously.
I think it's pretty obvious to figure out how much polish is "too much". This is a business. Engineers time is limited. Was the time you spent on that more valuable than the alternative? I'm leaving the question of what the alternative was ambiguous on purpose.
> Was the time you spent on that more valuable than the alternative?
In this case the alternative was applying to better companies, so I guess in a way the author really did fail the test.
What they were looking at was not the code, but the attitude. They likely don't want an engineer with a propensity to do too much, unless it's a 100x coder who can do "too much" with a lightning speed. Usually they want he candidate to show the ability to quickly do as much as needed, and not more, and, crucially, to understand how much is needed.
So yes, this is a culture fit test as much (if not more) as a design and coding test. Some people who are great at design and coding would fail it, and it's how this filter is intended to work.
Sorry for the humorous reply (I know this isn't Reddit), but this part gave me a good laugh:
I see you there. Raising the bar from 10x to 100x!?Yes, I needed a ridiculous number, a picture of the coding faculty so strong that writing twice as much code as the task really needs is hardly even noticeable.
IMHO, the author's approach of validating his ideas mirrors modern engineering workflows. Coders don't spend hours independently coding an MR and then getting feedback from prod, tech leads, QA, and UX after the feature is "finished."
Work environments that "code first, review later" have been some of the most toxic in my career. It really sucks when you spend days building a feature only to find its not wanted. Which is why explaining the feature in English and getting approvals is the industry standard for shipping projects.
This candidate followed modern software practices that healthy workplaces follow.
(I'm also a hiring manager at a 1k+ engineer company).
I've worked at large companies and I've worked at small ones (disclaimer: including Kagi, long ago). When I was at Google I wrote design docs for basically everything. The company I'm at has fewer than ten people. I rarely write design docs. This isn't because the workplace is any less healthy, but because there are fundamentally fewer people I need to communicate my changes to, and they have more context of what I'm doing.
Process is healthy at large companies, because they move slowly and communicating your changes often becomes far more important than the actual code that needs to be written. Kagi is a small company. It does not make sense to cargo-cult practices from a company a hundred times larger.
I found myself thinking the same thing - perhaps came from a bigger/more established company. Reminded me of the PRD/ERD process. And that's not necessarily bad.
But from my experience, the two types look at each other like the other is insane. If you write up an ERD at a scrappy 6 person startup, everyone is going to think you waste time.
Conversely, if you join a larger team with established processes and begin flinging code at the wall unabated, people are going to think you're reckless and possibly inexperienced.
Depends on a place.
I've worked in places with very strong product management, where every single detail must be approved by a PM. I've also worked in places where engineer has a lot of autonomy - "John from FOO dept is spending too much time wrangling the daily data updates. Can you help him, here is his email? Our higher-ups allocated two weeks for that, but we can extend this if needed." - and from then on, you are in full control of the design and implementation, subject to your team's rules (because no one wants a system with bus factor of 1).
For me, I've found that the latter places are much nicer to work in. The interview question seem to focus on latter situation too.
I don't know about everyone else, but as a dev who is currently looking for a job I simply cannot afford to devote a week of unpaid work to each application. It's not sustainable to say the least. Especially when I'm told by some people that getting rejected means I didn't spend enough time on the solution.
You (and others) have misunderstood it completely. The deadline is one week, this doesn't necessarily mean you should work on it for one week.
It's game theory.
If the next guy puts in 10 hours, and you only put in 1 hour, assuming equal skill. Which project will be more polished?
If you are high skill and only work 1 hour on the project, but a newbie puts in 10 hours with ChatGPT's help, I'd be the newbie would have a pretty competitive project to the skilled 1 hour candidate.
And yet the over engineered solution that somehow took a week to complete fell completely flat. I'd guarantee that Kagi has hired developers through this process who only spent a couple hours on it. If I was working on this, I wouldn't have gone beyond three hours. It would have actually been a console app and I would have done something like integrate your unread mail count in your terminal prompt as some "bonus" demonstrating additional thought in the space.
I've worked with IMAP and POP libraries before so am familiar with the fundamentals for building a client and libraries in many languages make this part of the integration very straightforward. Couple that with a "modern" CLI library this should come together very quickly. And I would not have included half a dozen cloud services for a terminal like email client. The project submitted completely missed the mark and they still have no idea why.
If I wanted to create something by meticulously planning out every detail and spoon feeding them to a code monkey with no creative input I'd just use an LLM or outsource to India where you've got to spell out every little detail and still get questionable results back. I've had to do that plenty of times and I don't want to work like that. I want to work with other professionals who can run with a concept and deliver good results without constant oversight and micromanagement. That's clearly not the author of the blog post.
I think you might be missing the point of the blog post. The author is frustrated that he followed standard engineering practices (by submitting a proposal for approval). This proposal was perceived to be as accepted and then when they delivered.
The issue isn't his solution is incorrect. The issue is the company's said his solution was correct and then they rejected him anyways.
This is the equivalent of the interviewer telling you the brute force solution on a leetcode is good enough, but then rejecting you after the interview.
no, the company did not say that his solution was correct, that's just what the author thought.
The company said that _the parts he wrote in the doc_ would not negatively affect his scoring. But his doc did not contain many parts that the company cared about - read it yourself, and try to answer the questions like: Will there be a per-email indicator? Is there a "sent mail" folder? How will user be notified of incoming email?
More on this here: https://news.ycombinator.com/item?id=43996105
You're not supposed to deliver a polished project, you're supposed to show what you can do and what you know. You can also document your process if you want, to let them know it only took you X hours to get to the solution you are presenting.
I had an interview once where they gave you 1.5 hours to code Tetris in ruby. I completed the core requirements and 1-2 extra requirements.
The person that got the offer coded 1-2 more extra features than I could in that time. What am I supposed to tell the interviewer?
"You're not supposed to get a polished project?"
If you're the interviewer, are you going to hire the guy that did just was asked or you are going to hire the guy that worked nights and weekends to get the perfect project delivered to you beyond what was asked?
If you're the interviewer you will invite both people to the next round of interviews.
If you have 2 equally qualified candidates, then who do you pick?
See my comment above.
> Coders don't spend hours independently coding
Exactly. The position they are hiring for is not that of a coder; coding is just one of the skills the position requires, maybe not even the most important.
I can’t speak for your specific jobs, but my understanding is this is standard practice at FANGA: write an RFC or Eng Spec. Get it approved. Code it up.
When I worked at unprofitable, small startups, a lot of mental energy was lost due to miscommunications before the MR review process. Eg an engineer would be tasked to complete a project, but misunderstood a critical component and only after n-days of work was this identified and corrected.
> FANGA
Only ever heard it referred to as FAANG before
> when you spend days building a feature only to find its not wanted.
lol, this makes me very old school but I suffered jobs where that was the outcome of months if not years of my work!
As a hirer, the kind of takehome assignment I like to give is one that:
* Can be completed in 30 minutes by a skilled programmer
* Has clear evaluation criteria, both objective and subjective
* Has multiple approaches that require making different tradeoffs
And of course, only give it to some candidates where the result will be make-or-break.
As someone who took one of these broad take-home assignments my last time looking for a job, I failed a the assignment for a job I was overqualified for because I was told I wasn't able to divine what parts of the extremely broad assignment I would be graded on.
I doubt I will be in a position where I get a job that isn't a referral for the rest of my career, but it really turned me off of these kinds of assignments, both taking and giving them.
Very curious: how do you deal with AI answers for those?
While writing my questions (and testing in my teammates), I found that "can be completed in 30 minutes by a skilled programmer" very often means "can be completed almost automatically by AI", and that AI will give explanations too, that interviewer could repeat during code review phase.
Every step in the process is a filter. You can ask them not to use AI and trust, you can ask them what tools they used (including AI) and for what part, you can ask the candidate to screen record them programming their solution and forbid using AI, you can ask them about their solution in a followup interview, etc.
It's kind of up to what you're filtering for, and how much you trust the candidate at that part of the process, and how you follow up after hiring.
Holy crap. That was exactly my same reaction. Only thirty mins to complete? Sheesh: Vibe code that!
I think the huge proposal (after several rounds of questions) was what sunk him. The instructions did not request a proposal, and submitting one, especially one that detailed, conveyed “OP cannot take the initiative and proceed with work without seeking approval beforehand.” The project was about asking a few questions, listing a few assumptions, and giving yourself approval to confidently dive in and build something ambiguous. Maybe I’m wrong, but I don’t think the code mattered after that email was sent. The company should have just ended it there without him wasting his time building the assignment.
That's the blogger's point. The hiring manager saw the proposal, knew that wasn't what he wanted, and then proceeded to waste an entire week of the candidate's time while he was looking for income.
What do you want the HM to say? "Actually, don't bother anymore, you've already failed"?
yes. "Please don't spend a week implementing this proposal". Early reject or give some indication that the scope should be much smaller.
that's what the OP wanted.
I also read that proposal and thought: "it's over".
The proposal itself was almost certainly written by an LLM, rather than by someone familiar with the tools that ChatGTP came up with for him.
> Part of the assignment is using judgment and working in ambiguity.
I love that the industry has become so poor at gathering requirements that devs are now effectively filtered for their ability to mind read.
Just to be clear, this wasn't my advice, it was written into the task description -
I'm not sure how I feel about that to be honest. On one hand I get what they're shooting for in general by saying that. On the other, they're going to have some preconceived notion of what they want, and it's a bit of luck if you come close to that.What about:
> And don't hesitate to tell us if you have any questions!
I personally have no problem dealing with "ambiguity and open-endedness" (and, in fact, enjoy it!), but my solution to this problem at every job that has had this issue is: talk to people and understand the problem.
Attempting to "mind-read" is the worst solution to ambiguity, and, in practice, nearly always leads to disaster.
Within the article, he details his attempts to get clarity/feedback and the response of the hiring manager is as vague as it gets
The way to deal with ambiguity and open endedness is by limiting scope and sticking to the unambiguous.
When the client can play with that, the scope can be expanded and additional features can be described.
What they want is someone they can work with. Take home tests are about cultural fit too.
That’s also where it helps to be a mind reader. :)
The best developers I've worked with have an uncanny knack for reading people's minds. (Of course, they are actually just really good at communicating and predicting what people want. But if you do it well enough it sure seems like mind reading.)
I agree, it's very fluffy. I think "testing with ambiguity" is the new version of "culture fit."
It's not really culture fit. I've used the open ended test before and it was a good filter for "can they walk the walk". Just some basic text processing, but only few people cared to mention error checking, tested Unicode behaviour, included any documentation, etc. It's "will you do the usual things without explicit prompting". (The assignment explicitly said you can call them out as something to do without fully implementing to save time)
So many poor interviews I've seen are what xkcd described once as "communicating poorly then acting smug when you're misunderstood".
Also, he missed the deadline.
>Delivery date: Sunday EOD March 30,
>This is two business days later than the two-week mark since you sent me your first email.
With no accompanying personal reason for the delay, it's already a rejection in my books. The objective was to design an appropriate solution within the deadline.
Regarding 4 - in the full version shared by the author at https://archive.md/A95Ju there is info on what they meant by "terminal-inspired"
It seems they wanted "keyboard driven", "fast/light", "quickly interactive"
The youtube video provided by the OP seems more "web-app", "click driven".
For contrast:
* The OP's submission: https://www.youtube.com/watch?v=yY1sVXMkP_o
* Someone interacting with aerc: https://youtu.be/kpAwwgnZUxg?t=308
* Someone interacting with mutt: https://youtu.be/C35NRp42bEQ
> It seems they wanted
This is the problem. Your guess is different from the author's guess, because nothing is explicitly stated.
For others who did not click the link, the explicit requirements are:
> - Email client can either be in the terminal (i.e. a TUI) or a web app
> - Should have basic email viewing + sending functionality
> - Can use a fake backend (DB, in-memory, etc) or real IMAP/POP/JMAP/etc backend
> - Does not have to handle rich text messages, just plaintext
> Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya. It should feel fast and intuitive, and you can choose which email flows you'd like to implement.
The word "keyboard" does not appear. Inspiration from X, Y, and Z is entirely subjective, and you should not be punished for not reading the interviewer's mind.
Is it not clear that every single one of the example apps the spec cites are text-based UIs, as in curses-style, monospace text, and (yes) keyboard-driven?
My takeaway from that is that if you don't make an app that is (quote) "in the terminal (i.e. a TUI)" but choose to make a web app, then it should look at act like a TUI.
What I got from that is "fast and intuitive" is the must have. When you have a choice (backend bit), justify you decision. I'd try to use that to stand out or an additional cool feature. When I have to do such homework, I do the requirements to what I think an above average candidate would, they've seen this a million times, then add 1-2 bits of flair/coolness only a handful of people would do to pique their curiosity and want to ask about the next round
Reading the brief, the simplest thing that might work is:
1. Spin up an EC2.
2. Install Himalaya.
3. Do some configuration.
Code quality and a sound engineering approach are there.
Personal touches aren’t. Though they look for them, absent domain knowledge they probably don’t want to see them.
Himalaya has documentation and a Github already.
No need to invent the wheel.
Won’t take an unreasonable amount of time.
And looks like a first iteration of a tool.
All these things are easy to say in hindsight. However, the same outcome could have happened if the opposite strategy were taken (Don't clarify requirements, do bare minimum); someone could just as easily post a response saying the candidate should have clarified requirements and gone beyond the bare minimum.
My first though on looking at the code, and then on demo video was: wow, it took that person a solid week to write a web app with two pages.. all while missing the most basic email features, like an ability to opening a message (and not just show full text of mail bodies on one page).
Later on I read the requirements... This person applied to a position named "Email Backend Engineer" but they actually used a third-party product (postmark + turso) for email backend! They also clearly don't think about email too much - the basic stuff, like plaintext email formatting, viewing, and folders (at least inbox + outbox) are simply missing; while there are optional features, like login screen, admin page and backend framework. And backend database does not even containing a email headers map!
That person might be a great engineer, but I don't think they would be a good fit for that specific position. I'd reject them as well.
(A separate question is that hiring manager's second response... We don't do take-home interviews, but I imagine I'd be stumped by a proposal like that, it seems so irregular in a take-home assignment. Still, I can see how the candidate took that response for an approval... perhaps a next candidate would get an extra sentence perhaps something like "actual problem grading would depend on the code quality, number of features implemented, and how close those are to requirements and job description")
Update: read original job description and not just the except from the blog. That person surely omitted some stuff in their blog! the original title was:
"The project is to build a minimal, terminal-inspired email client"
it gave examples:
"Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya."
wow! that's 100% no hire, serious inability to read the requirements.
> serious inability to read the requirements.
Which requirements were not met?
- Email client can either be in the terminal (i.e. a TUI) or a web app
- Should have basic email viewing + sending functionality
Seems like both were more than met to me.
Inspiration from existing tools is subjective. I would have assumed that spending time on the UI for a backend position was not the best way to show off my skills in a take home test.
The 2nd line of assignment, "The project is to build a minimal, terminal-inspired email client".
I agree that "inspiration" can be subjective, but in this particular case the solution was so much off that it'd be hard to argue otherwise. There is nothing terminal-like in it, and the tool is not usable as an email client either. Instead of doing login screens and user management, author should have made a page to view individual email or added folder (just 3, inbox/sent/trash, would already be a great improvement)
And if you want to ignore the requirements and work off position description... the position is titled "Email Backend Engineer", and the candidate's solution offloads actual email operations (storage, transfer) to third-party services. I suspect that even if candidate provided the same UX, but would have backed it with SMTP, IMAP, TCP/IP (not via http library), DNS MX lookup, resilient storage etc... then they might have passed.
But author failed to do either serious front-end work (required by question) or serious back-end work (required by job posting), so result was entirely predictable.
For every comment in this thread that says the author did too much, there is another saying the author clearly did not do enough.
Plenty of comments here have strong, but conflicting, opinions about which set of features are obviously in or out of scope. That alone indicates to me that this is a badly worded take home test.
Even ignoring that, the author told Kagi exactly what they were going to do in advance.
Applicant: I'm going to spend a week building X
Kagi: Sounds great!
Applicant: After a week, I have built X
Kagi: Not at all what we are looking for. Rejected.
I don't see any way to interpret this interaction positively.
I don't see much disagreement in the comments here? However there seems to be some people who only read the blogpost and did not actually follow the link to the original assignment [0].. you can tell their response by the lack of mention of "terminal inspired" features. Sadly this happens during discussions sometimes, especially when original author did not mention all the relevant facts in the post.
anyway, the exchange went like this:
Kagi: "please build terminal-inspired email client, you can choose which email flows you'd like to implement."
Applicant: I am going to spend a week building it, using golang, AWS, TMPL etc... I am also going to fit it onto 2 html pages.
Kagi: Sounds great!
Applicant: After a week, I have built something using golang, AWS, TMPL etc.. It has nothing to do with terminal, and does implement any of the email flows completely.
Kagi: Not at all what we are looking for. Rejected.
I don't see any way to blame Kagi for that exchange. Who would have thought the candidate decided to ignore half of the requirements?
[0] https://archive.md/A95Ju#selection-549.5-549.63
> For every comment in this thread that says the author did too much, there is another saying the author clearly did not do enough.
Both can be true (they arguably did too much extraneous stuff, but didn't do what they were actually asked for).
Mind you, this is a pretty ridiculous take-home question, and I'm sure they lose plenty of good candidates to "life is too short" at this point.
Still, as a candidate, it sucks that they worked for hours and only got a rejection with a templated message. At least tell me the reason why you don't like my work, so I can learn something from it!
I get it that it's not always realistic to do, especially if the hiring manage reviews hundreds of applications for a hot role. But that's why take home assignment sucks. Some candidates may waste hours of their lives for nothing. Both sides need to be respectful of other people's time.
"Email client can either be in the terminal (i.e. a TUI) or a web app"
I understand that their responses were very minimal and not helpful, but it clearly says in the requirements to make a "terminal-inspired" email client. However, in the video you shared [1] I see a somewhat generic email web app, with nothing particularly "terminal-inspired". Considering the fact that they wanted either a real TUI or a webapp, I'm pretty sure that the web app should've also been "terminal-inspired". Am I missing something? I'm not trying to necessarily criticize you, perhaps I just misunderstood the requirements.
Edit: I've actually checked the full requirements page, and they explicitly say this in "Inspiration":
> Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.
[1] https://www.youtube.com/watch?v=yY1sVXMkP_o
Yeah, a clearer explanation in the rejection email would have been nice, or perhaps even a warning in response to the proposal (though it's not 100% clear from the proposal alone that OP would be going down a path that ignores that part of the assignment.) But the prompt explicitly lists other terminal clients as the inspiration.
Also, the prompt for a terminal client really changes what they're testing! Email web UIs have been a known quantity for years. But the UX of a terminal client is still something that's not "solved." I suspect the rubric for the question has large sections about how they decide to make a terminal client that OP's submission doesn't address at all
so, the hiring manager should spent more time on formulating the rejection than the candidate spent on reading the actual requirements?
Also they seem to put emphasis on Go programming, which takes < 20 lines to get a cli running so not sure why they chose a browser GUI, which is arguably much more work.
but now they want something "deployed"
"Email client can either be in the terminal (i.e. a TUI) or a web app"
Take home assessments can be a valuable part of an interview process but they absolutely need a time limit. I think 2-3 hours is going to give you all the information you need, unless what you're selecting for is new grads with no dependents, hobbies, or responsibilities.
If this had been limited to 3 hours then the worst case is that the candidate would have lost 3 hours, but far more likely is that they would have come up with an entirely different proposal and/or solution that was appropriate for that timeline, and that extra information would have made it clearer what the company was looking for.
The other point I'd always encourage applicants to confirm is: are you looking for any answer, or are you looking for a good answer. Some take-home tests are purely about passing a test suite and how you manage it doesn't matter, some would prefer that you meet 80% of the requirements but write better code. I've seen applicants do the wrong thing on both sides of this.
To me, as someone who hires and gives candidates take home assignments, Kagi’s assignment is huge and disrespectful of the candidate’s time. Surely they’re (probably unintentionally) filtering for candidates that have a lot of time to waste on projects like this — while people who are busy (the kind of people you want) will pass.
Surely there must be other ways to have an idea on a candidate’s skills.
In our case, hiring people with data engineering skills, we just ask of them a simple ETL challenge (pull data from zip file, transform it, insert into any database). We leave ambiguities in there and there are some Easter eggs in the dataset (eg null values where you would not expect it, incorrectly formatted CSV) that we use to evaluate how well a candidate can perform.
We timebox it at 4 hours, but don’t give guidance in case they run out of time, that’s a good suggestion.
During a follow up call, we review the code together and we’ll ask them questions on how to improve the code (“what if the dataset doesn’t fit in memory”, etc), which is what the actual technical assessment is. At that point they should already feel somewhat comfortable with the problem domain, and you can assess their real skills.
I agree about Kagi, but about your process wanted to ask: do you really feel like having them write the code as a take-home task adds any signal to the process?
At my current gig we do a 1-hour pair programming kind of thing, where we video-chat and watch the candidate work on a small, straightforward task. And as the interviewer I watch how they use the tools, where they go for docs, do they read the requirements, how do they test, etc. By the end I always feel like I have a strong picture of their ability, and the whole thing is capped at 90 minutes for both sides (adding time for their questions).
If the candidate's code was written offline beforehand, I'd have no idea whether it was theirs, whether it came from a friend or chatGPT, whether it took ten minutes or ten hours, etc. Sure I could try to suss out those things during discussion, but isn't it better to observe directly?
Leaving aside the non-trivial question of candidate nervousness in live coding, I do find the "ask questions about take-home" process very illuminating.
If it came from a friend or chatGPT, you'll learn that quickly because you can ask them stuff like "show me how you're implementing FOO. OK, if the requirements change, and now we need to BAR, what do you need to change across the app?" If they wrote the code and understood the task, they should understand what needs to change, how to do it, etc, and you can do that part with them live. Maybe someday LLMs will be fast enough that you can't catch someone constantly waiting for the AI to help them, at least as of May 2025 we're not there yet.
Respectfully, you've only answered the thing I pre-replied to in my final paragraph. My question was why the take-home part adds signal?
This is a data engineer’s fizzbuzz and you would be surprised at how many people actually fail it.
If they didn’t write it themselves, they will absolutely fail in the follow-up session where the real evaluation happens and I ask them to scale the code, handle crash safety without using transactions, etc.
Okay, but what signal is added by administering it as a take-home task?
(Even moreso if it's a fizzbuzz level task, I'd have thought - actual fizzbuzz being famous as a coding interview task, after all.)
Some people are nervous during live coding challenges, which is why we decided to allow this to be taken home, and the candidate becoming familiar with the problem domain in their own time, rather than "live" under pressure.
Would you think doing this live in a screen sharing session adds more value? How would you prepare the candidate for this?
I see, thanks for clarifying!
> Would you think doing this live in a screen sharing session adds more value?
In my experience, hugely yes. Above all else I really like that the time commitment is fixed on both sides - no homework for the candidate, and we're not asking them to invest any more time than we are.
Also doing things interactively gives a lot of leeway for adjusting and avoiding wasted time. If somebody's got one section basically solved, I might cut in and say "yeah that bit is great, let's call it finished and look at this other bit". If they're obviously having a lot of trouble, I might ask about their current work and maybe switch tasks if there's a better fit. Or if they're obviously coasting easily I might cut in with "how would you scale this?" questions that give more signal than watching them finish up the loose ends.
Another thing I like is that we can keep the task specification simple, like what you'd get from a PM, and it's up to the candidate whether they want to ask questions or jump in. I imagine that with take-home tasks you either need to give out pretty specific requirements, or else have people interpreting the task a variety of different ways.
> How would you prepare the candidate for this?
We explain at the beginning that the goal is write code for an hour and watch how they work, and there's no hard requirement for what needs to be finished by the end. And they're welcome to search the web, use AI, or anything else they'd typically do while working.
And I've only done a few dozen of these, but personally I've never seen anybody get particularly nervous. I think nerves are a bigger issue with HR style "pass all the tests within the time limit" tests, where the danger is that somebody can hit a wall and never get past it. That doesn't happen with my format, because if somebody is stuck I just give them hints until they're unstuck (since I get no signal from watching somebody scratch their head).
But if you're unable to determine if someone has this knowledge via a phone call, instead of 4 hours of their work without you even being present, then that failing is on you, not them. If you can't judge someone's knowledge by asking questions, then you don't know how to come up with the right questions. Again that's on you. The only thing a 4-hr take home test will filter for is desperation. You'll get the most desperate candidates, and you'll do so by wasting days and days of people's time once you add it all up. It's just utterly disrespectful to demand these silly homework problems. I always take it that way. I take it personally, tbh. I find it absolutely insulting to even be asked and I simply refuse.
I'm surprised this seemed to be voted down. I've been a hiring manager for over 30 years, and I never do "technical tests" -- no take-home, no live coding, none of it.
I have a map of topics and questions, and I get the candidate chatting about their past projects, their approach, and what they liked/disliked about past projects and various technology they've used.
It takes a maximum of one hour and usually close to about thirty minutes to make a yes/no decision on a candidate (sometimes it only takes ten minutes to make a no decision, and then it's a matter of trying to politely end the interview).
I've interviewed hundreds of candidates this way over the years, and everyone I've hired has been capable of doing the job. Not once have I ever had to let someone go for lack of technical ability.
Part of the problem is that we don't train people how to conduct interviews, and another part is "this is how I was interviewed, so this is how I'm going to interview other candidates" -- pure inertia.
As an industry, we really need to do better.
As for the OP, _if_ I had been administering a vague take-home project that had a 1-week delivery deadline, and a candidate peppered me with Qs and then presented a full proposal for the project for approval, prior to working on it... I would have rejected them. But I'm pretty certain I would have decided to reject them in my regular 30-60 "chat" interview, and I would not have moved on to the take-home project and wasted their time like that. So, again, I fault the interviewer(s) for not being able to filter candidates efficiently.
I had an interview once to work for IPFS, with a guy who already knew my skill level well (decades of experience), because I was active on their web forum solving everyone's problems, for about a year. Even during the interview he mentioned the technical part of the interview wasn't even necessary, because he had total confidence in me, and didn't even ask technical questions at all. He spent the ENTIRE hour selling me on the job. I hardly had time to speak. Everything went perfectly fine. Then he asked me to do the take-home project at the very end, just because "everybody has to", and I politely told him sorry I don't do those. So that ended the opportunity. So you're exactly right it's absolutely an "inertia" thing.
The solution to this problem ultimately needs to be a regulatory one.
People should be paid for the time they spend in interviews.
You make that the law and this ambiguous hoop jumping bullshit goes away real fast. Companies will optimize for cost instead of dumping cost onto the prospective employees.
This is good for the economy because it forces companies to innovate and optimize the interview process and it saves hundreds hours of totally economically unproductive time on the part of candidates.
Yes. It would also force them to screen resumes and do better profile research or they wouldn’t spend the money.
That's basically my feeling too. No one would ask for a maid to clean their house free for 4 hours, in order to decide if they want to hire her long term. And that's precisely what hiring managers have been getting away with for decades, because no one is pushing back. The managers are essentially abusing their power. It's all about abuse of power and utter arrogance.
I don't know what ETL means (I could look it up) - regardless I can do that ETL challenge. Realtek SDR dongles and Python loved messing up CSV. The only question I'd have is if a null or and "incorrectly formatted" row was invalid or not.
So I have to wonder, is this a junior position, or did too much hadoop rub off on me a decade and a half ago?
This is for a medior solution, but it’s more intended as a data engineer’s “fizzbuzz” and then in the follow-up call I’ll ask them how they would distribute the workload over multiple clusters, make it idempotent, etc — that’s where the real technical evaluation happens.
"extract, transform, load" - it's an acronym often used to describe typical data wrangling from one source to another
How do you know that interviewees aren't spending more time on it?
Because you can't guarantee all candidates are spending the same amount of time, it becomes a game theory problem where the candidates will typically lose in some form. In many cases, the right answer is to spend extra time making a really polished (but not too polished!) solution and pretend like you stayed in the time limit. And every candidate is either a) doing that, or at least b) worried that their competition is doing that.
Even if we ignore that dynamic, 3 hours is a long ass time for a candidate to spend when they're not even sure they'll get to talk to another human about it.
In a 1-hour interview, you can run a candidate through a programming exercise and be guaranteed they're not wasting extra time on it. And if they happen to prefer doing take home assessments, you can always let them send you an updated answer later. (But often by the time a candidate asks me if they can do that, I've already developed a favorable view of their skills and can tell them, "go for it if you want, but you've already 'passed' my test.")
By keeping the candidate-interviewer time investment the same, you guarantee that you're respecting the candidate's time as you would your own (because you're sitting there with them.) I can help them skip over the parts I'm not interested in (e.g., by feeding them info they'd be able to find via search or telling them not to worry about certain details.)
If a hiring manager doesn't respect their candidates' time, how likely are they to respect their employees' time?
Yep, this is what I am taking from this thread: Next time I am given a take-home, I am going to ask them to promise, that I will get to talk to a human about it. They can of course straigh-ass lie about it, and I am sure I will run into such abysmal behavior at some point.
> How do you know that interviewees aren't spending more time on it?
In some cases we roughly timed it, scheduling an email for a time the candidate wanted and asked them to return it 3 hours later. In some cases we just treated it as an honour system. We made it clear that the task was intended to take about that much time and that spending more time was not allowed/encouraged.
In reality, we found that good candidates took ~1-2h, and in some cases where candidates spent a lot longer and owned up to it, we found no improvements. In one case a candidate submitted at 3h and then again at 8, and we marked the 8h version 1 mark lower.
I agree with the author that live code reviews are much better than live coding, but if companies insist on a live coding exercise: let me bring my laptop and mouse leave the room for 45 minutes and come back when we can talk about what I built.
I did a recent round of interviews and this experience closely mirrors mine (at multiple places). Delivered an exceptional solution to the problem only to be rejected without a discussion about the delivered project. I have been on the hiring end of take-homes multiple times and firmly believe that if you ask someone to do a take-home assignment you must follow up with a chat about the code. Apparently that's not the case at most places.
If you're asked to do a take home, I highly recommend confirming that there will be a follow up regardless of the assessment, and if they do not agree to this, do not do the "home work". The honest truth is that most of the teams hiring are of pretty low quality and therefore implementing a good solution is a negative because the team hiring is not at your level (which is a frustrating reason for being rejected).
I'm an early adopter of Kagi and am now planning on cancelling my account over of this. Completely unacceptable. If you don't have time to chat with a candidate about their work, don't ask them to do the work in the first place.
> I have been on the hiring end of take-homes multiple times and firmly believe that if you ask someone to do a take-home assignment you must follow up with a chat about the code.
That’s actually a very valid point. Take-home assignments not only require a significant amount of effort from the person administering them but also from the interviewer (or rather, the hiring team). After investing the time and effort in reviewing a project, it’s reasonable to expect feedback/a response back if requested.
However, we must also acknowledge certain realities. If there are 20 solid candidates for a single open position, many capable individuals will be rejected. This doesn’t imply that they were inadequate or even “failed.” It’s simply a reality of life.
Now to be asked to justify why a hiring manager picked A and not B, legally speaking, cannot be that I had to pick one out of the amazing candidates, so I picked one. The legally correct response (unfortunately) is to get nit picky and find faults where none really exists. At least, that has been my observation in how corporate America likes to operate. And last time I checked, Kagi is based out of Palo Alto.
> if there are 20 excellent responses for a single open position, many capable individuals will be rejected.
I would never ask 20 people to do a take-home assignment. There are so many better ways to test team fit before asking someone to commit serious, unpaid, time to a project. Historically speaking a 30 minute chat with someone provides a surprisingly good amount of information to anyone experienced in hiring.
But if you want to commit 20 people to take-home assignments, then block off 20 hours next week for 1-on-1s to chat with them about those assignments. If that sounds like too much, then don't find a better way to filter down the number of people doing homework until it feels manageable.
to justify why a hiring manager picked A and not B, legally speaking, cannot be that I had to pick one out of the amazing candidates, so I picked one.
Why not? Does this phrasing implies any form of illegal discrimination?
> If there are 20 solid candidates for a single open position, many capable individuals will be rejected.
And yet, the role is still advertised 1.5 months after my rejection, which means they are still getting emails from new candidates applying for the role. Do you suppose they get so many competent candidates that they can't make a decision on whom to hire?
At this point we can only speculate about their intentions, because the intentions are definitely not transparent.
I agree. The company should have either published clear guidelines on how the project would be assessed or provided some feedback on what could have been done better. It's okay to fail an assignment, but it's not acceptable to waste someone's time without offering anything in return. There are many companies, including pop startups, that handle this well.
did they really deliver "exceptional solution" though?
Their solution seems to be nothing like "a minimal, terminal-inspired email client", and OP completely ignored the references to tools they were told to "take inspiration from"
When there is such bad misunderstanding of the requirements, I'd not waste time on the discussion either.
There was able opportunity for the hiring manager to point that out if that were the case. They specify questions where encouraged and then completely ignored the questions. If the HM doesn't have time to respond, then maybe take-home as first filter is not a good decision.
Were there? The question clearly said, "build a minimal, terminal-inspired email client" and "take inspiration from existing terminal email tools like aerc, mutt". Candidate completely ignored that part and built a tool which is nothing like terminal email at all.
Would any of their emails give a hiring manager idea that the candidate failed to understand the assignment in such a gross way?
Email 1, "What kind of extra feature do you value highly"
Interviewer thinks: "UX improvement could be VI-like megacommands, "pretty UI" must mean creative use of colors and font attributes, privacy-related must be encryption at rest... It's all good, we are happy to look at all of those!"
Email 2, "A webapp with golang accessible online, deployed through AWS using ECS Fargate, with SSL (https), integrated with an email sender provider, with authentication through a login screen, with the ability to send emails through a form, and displaying incoming emails in the user interface."
Interviewer thinks: "OK, that's a lot of implementation details.. Not sure why candidate feels the need to confirm that, but nothing in the list above will be graded as a negative. The important things we care about are that it's inspired by terminal apps like mutt, and that "it should feel fast and intuitive" and one can totally do that using technologies above"
Without seeing the screenshots/mockups, how would one even guess that the candidate was so off?
> Interviewer thinks: "OK, that's a lot of implementation details.. Not sure why candidate feels the need to confirm that"
Really? My email states explicitly why I want to confirm that:
> I would like to know what kind of response I could expect from Kagi if I drive it to completion.
Are you forgetting that the entire transaction implies a lot of work for the candidate?
Are you forgetting that candidate is doing this to land a job?
Do you think the candidate is writing a design document just for fun?
Your subtext implies that I should have guessed the grading, which is quite mind-blowing.
Remember, the instructions mention a web app as an option, the role is for a web-based company, and if my design document did not meet any of the key aspects they could have brought it up and rejected my proposal.
If you read the assignment, you can see that the majority of tasks is about UI and user-facing features ("Should have basic email viewing + sending functionality", "Does not have to handle rich text messages, just plaintext", "take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.", "It should feel fast and intuitive")
The assignment had exactly 1 line about actual back-endy stuff: "Can use a fake backend (DB, in-memory, etc) or real ..."
Now, what did your email asked for? There was a whole bunch of things about back-endy stuff, and exactly 1 (one) sentence about the user-visible things, the thing they cared about the most:
"The UI will be kept simple, showing pagination for sent and received emails. In addition to the requirements of the assessment, there will be a login screen and two accounts"
So maybe you intended this to be a complete design document, but it actually was not. This is because it did not actually did contain any parts the interviewer cared about. And that's one of the reasons why you got no feedback - you could do _anything_ for the backend and they would likely still accepted that, they simply did not care if you used Pulumi or terraform or "start.sh" or "docker compose up"
And no, you don't need to guess the grading, you just need to read the assignment carefully. What do they seem to want? What kinds of things do they mention a lot? What kind of things do they mention in passing?
The HM answered, it was just not what the author wanted:
> That is part of the assessment itself, see what kind of extra features you can come up if any
It seems like the author wanted to hear something more like "ok, here’s a list of things we want you to implement, and here’s mock-ups for the UI, and here’s the API spec you can follow". How I’m reading the HM’s response is - I should spend a few minutes trying out the email clients they mentioned or watching videos on them, and come up with a list of their features. Apply my own judgement to say which ones are most important and roughly estimate the time it will take to implement a PoC for the assignment. At this point you could take that to the HM and say "hey, I think I can do X, Y, Z features within the timeframe. I’d like to have W, U, V, features in the client but I think they’re less important and I won’t have time. Since this is a backend position I will focus on the API and the client will be a thin wrapper around that. Does that sound good?". They will probably say yes and maybe throw in a "when you do X feature think about users that have 100k unread emails in their inbox", and you’ll get a gold star for "dealing with ambiguity". If I was the HM I would expect some amount of back and forth like this. Since the guidelines are so broad you can focus in on the areas you are most familiar with and keep the rest super simple. As far as we know the author sent the one email on March 18 including a lot of details about what dependencies they were going to use but no product or clarifying questions after that, which the HM might have happily answered. They also both went past the deadline set by the company AND their own self-imposed later deadline from their own proposal.
At the end of the day this is a startup and so it makes sense that what they’re looking for is someone who can work independently. They won’t have a PM to come up with the spec and list of requirements for you every time, and an architect who hands you the perfect software design, and they need you to be able to apply your own judgement and look ahead so that if you are designing their backend and 6 months down the line users ask for drafts support, you don’t come back with "sorry I didn’t account for drafts in my data model because no one told me, we can’t support that without redoing the whole thing".
It sounds like this actually worked fine as a filter for both the author and the company, just that the author realized too late in the process that this was not the work environment they were looking for. A phone screen or whatever else would've just been additional time spent for both the company and the author. I also don't think it's the company's fault that the author put in a "full work week" of hours, as far as we can tell that was never the expectation.
This is a classic case of misreading the subtext. They want you to be independent and able to create your own work and goals. The ambiguity of the take home task is not there for you to interrogate over multiple emails, rather its an open palette for you to show how you take a relatively broad task and produce your own interrogation of the problem space.
As someone who works at a university this is very reminiscent of students who don't understand the assignment then complain when they get an unexpected mark.
I completely agree. The assigment seems to be looking for someone who will make as minimaly clever functional solution as possible. Someone who will dare to “be lazy” in interview where they want to impress.
This is not that common trait. But if your company has more prototype driven approach with quick “experiment” releases some people are just very bad fit and thats bad for everyone.
There is probably a candidate that thought for 10 minutes - what best can be done in 60min. They probably leveraged some http auth and forms, run that through tiny go backend, sent it with brief email. And they most likely looked a lot better in the process.
> misreading the subtext
I mean it's right there in the text itself.
That would maybe be a valid point if the take home set any context beyond "it's an R&D project" and "minimal".
What do you want to achieve here, is this a throwaway prototype or would a user ever see this? Am I going to get picked on for imperfect UX or do you just want to see something. It becomes an exercise in guessing the reviewers sensibilities.
Guessing the sensibilities of people is a large part of experimental software design and research. So I would say that the interview task is working as intended :)
Broadly, sure. You probably know who your audience is. But how'd I know who my recruiter is, individually. Is stalking their social media presence part of the exercise?
... and for every such Choose Your Own Adventure assignment there will be many people rejected because they "misread the subtext" and did not keep pushing for requirements, which is a very important virtue in engineers!
Life isn't fair, but apparently we expect more from hiring managers. And since life isn't fair we continue to get disappointed by hiring managers.
Should we feel bad about the hiring managers?
Hiring manager:
- Gets paid to perform the interviews
- Spends a 15 minutes answering emails and reviewing the solutions
Candidate(s):
- Does not get paid for their time
- Spends anywhere between 4 to 20 hours (or more) creating the solution
> As someone who works at a university this is very reminiscent of students who don't understand the assignment then complain when they get an unexpected mark.
This is such a lazy and convenient conclusion for someone that is giving out instructions.
Another valid reason that the students might misunderstand an assignment is because they cannot read minds and the instructions are poorly written.
Good teachers are the ones that make themselves understood by the largest amount of students. Bad teachers are the opposite.
With my best teachers, I never had to question their grading, because it was transparent from the start. So if you are getting a lot of confused students, you might be the common denominator.
Buddhist monks learn by solving riddles and guessing the messages of cryptic messages called Koans. Lets agree that students at a faculty, literally paying for tuition, should not be subject to the same practice.
Since the author himself seems to have posted this, I figure I might as well try to help contextualize the response he got. I worked at Kagi (though not on their search product) and went through a similar take-home test. Obviously, a lot of disclaimers: this was a while ago, I am obviously biased, I am not at Kagi anymore and this is my perspective. I'm not even sure how they would feel about this but they're more than welcome to say hi if not reply to the actual content of this comment.
When I went through the process the company was very small and Vlad personally reviewed applicants, and he used take home projects like this one. It seems like the company has grown larger so now he isn't looking at them himself anymore, but the essence seems the same. If you ever talk to him you'll find that he's basically the Hacker News commenter stereotype and his interview is really a vibe check to see if you seem cool. To him, submitting a long design doc where you go "I am going to use Galactor. I will make sure the project is florp-ready. I will fleem." is the exact opposite of cool. When the assignment says "I want a terminal-inspired email client" the project that passes with flying colors has keyboard shortcuts for everything and comes with a paragraph explaining how it never takes more than 2ms to draw a frame. I suspect the interviewer thinks you are too "enterprise-brained" to work at Kagi.
Now, is this actually a good way to screen people? A bunch of other people here have opinions on it, so I won't rehash it too much. There are obvious problems with it. Vlad basically wants people who think like him. This kind of interview has a lot of problems that have been discussed and studied at length. But if you understand that and are privileged enough to go through that process, the task will actually make sense to you. I think Kagi could do a better job at communicating this. It's unfortunate that you had to waste a bunch of your time to get to this point but I hope you can find something else that matches your working style better.
A lot of companies seem to want to hire folks who "think like me". IMO, that's a mistake: you need diversity of thought to provide new insights into problems. If you all think the same way, and one of you gets stuck, then it's likely you'll all get stuck. This seems to be a common problem in startups, IME, and perhaps an indicator of why 9 out of 10 startups fail.
Hiring someone who wants detailed requirements is not a good idea when you don't have a structure in place that can provide these detailed requirements.
If you rely on people to work independently, you just can't hire someone who wants feedback everyday to see if they are still on the right track.
Depending on the level of detail, "detailed requirements" can be the bare minimum for any level of cohesive, cross-functional collaboration.
You need to have discussions to talk about things like:
Ignoring all that and letting all the engineers just run wild is a great way to make a very painful company to work with and cross-collaborate on.My impression is that Kagi aims to hire people who are already aligned on most of these. Not all of them, of course, but I read the interview as an attempt to be able to go "do this" and have candidates understand what it means.
I don't disagree, but again: Kagi is Vlad's pet project; he's already rich. I think he would rather it fail than it become something that he doesn't feel aligned with.
What seems objectionable to me, is that in order for Vlad to find people that are "cool" he's wasting everyone else's time.
This isn't exactly the same as if I give people a test on their ability. It's different. In here you pass if you did something that scores well in a metric that you're not told about. It's not very different than if I gave you a take home assignment which is just literally "code something - you have to be good at ambiguity!" Joking, you just have to guess the thing I had in mind. It's very inconsiderate to people, and doesn't tell me anything good about this Vlad person.
I think it's very reasonable to expect that there should be better communication here about this actually being a combination culture-fit-and-coding assessment. But I don't actually think it's fair to say that nobody is told about the metric. It's pretty clear to me. I think a lot of other people here also understood it (which makes sense, because they're all Hacker News brained). I agree that this sucks if you don't "get" it but I'm not entirely sure if there is a good way to convey this. Maybe they shouldn't be looking for this at all but that's a separate discussion.
hi :)
> To him, submitting a long design doc where you go "I am going to use Galactor. I will make sure the project is florp-ready. I will fleem." is the exact opposite of cool.
Am I supposed to guess this much?
Do they expect me to creep on their social media to figure this out or something?
Finally, I made it abundantly clear that I am uncool, so giving me the "go ahead", simply to satisfy their curiosity... I hope they are treated the same as they treat other people.
No ill will against you, if anything you are giving me a plausible explanation where I have none. I just wish I had come across a post like mine earlier, and I hope that we, as a collective, start categorizing any job with a "take-home" as a job for the "cool kids". That way, all of us boring and experienced engineers may force companies to adapt if they actually want our labor. And those that do not adapt, we can infer the type of company that "offers" the role.
I'm curious what you would have done differently if you had come across a post like yours?
I took a look at the code, to see how I would react as a reviewer.
In the first file I opened, I got as far as here: https://github.com/Sleepful/mymail/blob/main/app/router/page...
First line is very weird, unrelated to the task. Maybe copy-paste from a sample in a blog post? Anyway not paying attention to leave it in.Wording in the second line is not consistent with third.
I’d stop my review there. Lack of attention to detail. Author does not demonstrate an ability to think clearly.
The problem in the story is not that the candidate was rejected but that the process is largely disrespectful by asking for a lot of unpaid time to implement some project without clear guidelines or giving valuable feedback.
The author wasted a significant amount of his own time, and the hiring manager’s time with all the questions, the proposal, etc.
Even after asking for all this clarity, he failed to do what was originally asked. If asking for all those details, you have to at least do the basics of what was asked.
He failed on multiple fronts here, and even wrote an entire blog post without realizing it. It seems like Kagi did the right thing by not hiring him, if we’re being brutally honest. If he would re-read the original ask with a fresh set of eyes, then look at his final product and all the communication, the feedback should go without saying. When someone misses the mark by that much, it’s takes a lot of effort to try and say it nicely to soften the blow, as a company would want to do. Multiply this by however many people applied, and there just aren’t enough hours in the day.
Now that he’s written this blog post, I wonder if he’ll have more trouble even getting past the first gate of hiring processes, as other companies won’t want to sign on to have a blog post trying to drag them through the mud over a rejection. That shows more questionable judgement.
> The author wasted a significant amount of his own time, and the hiring manager’s time with all the questions, the proposal, etc.
Maybe if the hiring manager didn't want their time wasted they shouldn't be using a stupid take-home prompt? Frankly everything I've seen here indicates that I would never work for Kagi and would actively encourage people to never work there.
Yes, I agree. You cannot implement arduous processes and then complain about how arduous they are.
A week long take home exam with no discussion after the fact is absolutely unbelievable. It's so disrespectful to the candidates time.
I don't care if the exam wasn't supposed to take a week, or if they get 1 million exams, or if their hiring manager only works 1 hour a day. If that's the case, then they shouldn't be doing this type of test. If you can't handle the heat, get out of the kitchen.
The author may be wrong here but the hiring manger wasn’t particularly professional either. It’s possible for multiple people to be wrong.
I don’t see where the hiring manager was unprofessional. He responded to the questions/proposal without giving the author an unfair advantage, and he responded to the message after getting rejection. The author wasn’t ghosted or ignored, and has closure, which is more than a lot of people get.
Had the hiring manager given detailed feedback, it would have ended up in the blog post, and then they would need to revamp their process, as anyone reading would get additional information they weren’t intended to have.
I’ve also seen situations like this where the person never stops responding and seems to try and turn a rejection into a long term mentorship opportunity. The author seems like they might have those tendencies and keeping to standard boilerplate responses helps avoid that. It’s not overly personal, but I don’t think I’d call it unprofessional. The author simply misunderstood the assignment at a foundational level and is trying to shift the blame for that onto the hiring manager… when that was exactly what they were testing for.
I deployed the app to my own domain and the functionality was without flaws. It took me forever. Included extra features typical of a backend role such as authentication and Infrastructure as Code. My efforts led to an empty rejection email.
And you are telling me that I should have spent extra time on my code comments?
I have to disagree with your point.
yes? you either copied the code from somewhere and didn't bother fixing the comments or you write nonsensical comments. neither is great.
As an engineering interviewer I feel the urge to comment. I like neither leet code, nor home assignments. They are both time consuming and provide you with a little information.
But having said this I would have likely rejected the author too. Kagi Search is a startup, these folks are typically looking for fast moving, pragmatic optimists who can strike the breadth - depth balance.
We used to have a colleague. They would collect the inputs, close themselves for a few days, come up with a solution only to learn that reqs changed in the meantime. It was not pleasant experience for anyone.
I had a similar experience with DuckDuckGo several years ago, but the difference was that I got paid as I progressed through the various stages.
I had to submit a short written design proposal and was told to cap it at a few hours and was paid for it. It was accepted and then I spent time implementing a solution. The pay again was capped at a certain amount of hours, but I ended up going past the recommendation because I was actually having fun trying to figure it out.
I submitted the solution and after a while I got a response with basically, "it was a difficult decision, but sorry we're going to pass" and they wouldn't provide additional detail. It's been over 5 years, but I still wonder why they passed.
I can only assume they had a lot of candidates and they may have had other very strong submissions that were better than mine. However, it took a lot out of me emotionally and affected me for quite some time... more than any other interview in my ~20 year career. I feel for the OP.
Man, I hate take-homes.
When I see instructions that say “This project tests your ability not only to code, but to deal with ambiguity and open-endedness”, it actually means either a) this exercise has been so poorly conceived that we didn’t bother with a rubric or b) the work environment is so chaotic, we never clearly define specs or requirements for anything.
That, and the instruction to simultaneously “show off your skills” and deliver a pragmatic functional solution are completely at odds. Good simple solutions aren’t flashy.
To me, take-homes that have UI or API integrations are a bit of a red flag, because UI code (for most roles) is relatively boring, and API integrations are a lot of faff with not much signal. Cool you can make an HTTP request, cool you've got a basic CRUD editing setup. It's a lot of code that takes a lot of time and tells me almost nothing about how you code, hell, AI tools will happily generate these things in no time and at a pretty good quality.
What I want to see is an engineer implement an awkward bit of business logic. Does it become a million nested if statements and a "here be dragons" comment at the top, or do they identify the right patterns and build something that I can reason about when reading the code? This is far more valuable in the job, more signal in the interview, and much harder for AI to get right. It also takes less time.
I’ve seen a few that let you fork a repo with all that boilerplate set up and you just have a few stubbed methods to fill out, and that seems reasonable. But going from 0->1 on an app is so much grunt work and I doubt the reviewers would even look into all of it
I believe they don't. I think its a filter that they want people they can push into too much work to apply.
I would say that's reasonable, although it again sorta depends on the stubbed methods and what you're trying to figure out from them, and I'd question the necessity of the boilerplate - can't you just implement the methods?
Can I ask you and everyone else - why do AI is so good at UI/CRUD apps and terrible at business logic?
I've been caught with this few times now. Spend ages trying to coerce AI to solve logic problem and end up just manually solving it myself. Whereas UIs are so good and usually near perfect from first prompt. I suspect it's the weak prompt. I need to learn and solve this before my brain completely atrophies (there must be Anthropic joke here somewhere hehe).
I doubt it's prompting. I think the issue is more likely that UI code is often similar, has a lot of examples online, and often doesn't require understanding data flow. This is why LLMs are great at "boring" React components, because they don't actually understand the flow of the data, but they don't need to.
Business logic on the other hand is much more likely to be novel in some way, there are likely fewer similar examples for rules to be learnt from, etc.
Obviously this is all gradations, LLMs can manage some business logic and mess up some UIs (they can't "see" the UI which doesn't help!), but this is my experience of them and fits with my understanding of the technology.
It's a start-up with a lot of different projects. It is expected that the work environment would be chaotic. People who look for a no-surprises type of work week should probably not apply to such a kind of workplace.
> Good simple solutions aren’t flashy.
Maybe showing off your skills is making a good and simple solution?
Sure, the work is chaotic. But why would the interview process need to be chaotic? You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants
> Sure, the work is chaotic. But why would the interview process need to be chaotic?
Because the interview is meant to measure how well someone would perform on the work?
> You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants
Well sure, all else being equal, but if the cost of consistency is an assignment that doesn't reflect the actual work environment then that may outweigh the benefit.
But why conflate “technical skill assessment” and “ambiguity handling” into one big stressful test?
Doing two small tests toy can measure each skill independently
A clear coding test to get technical skill and then some in person requirements gathering exercise or something to measure ambiguity handling.
You’ll get better insight into their abilities with less effort for the interviewee AND the interviewer (if the startup is really that chaotic, do you really think they are doing a careful code review on a zillion different bespoke takehomes?)
> But why conflate “technical skill assessment” and “ambiguity handling” into one big stressful test?
Because the combination is what matters for the hiring decision, and a single test is a lot easier for everyone than two tests.
> You’ll get better insight into their abilities with less effort for the interviewee AND the interviewer
Disagree. It would always end up being approximately twice as much effort for both.
> we never clearly define specs or requirements for anything.
I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements? I consider that a big part of what it means to be a senior software engineer.
Maybe this exact format isn’t the best way to test for it, but I don’t think “we want to see if someone can deal with ambiguity” means all the work is ambiguous. It might just mean all work needs to go through a process to take it from “ambiguous requirements” to “clearly defined specs”
> I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements?
Then I'd expect the interview process to focus on that process, not on the final deliverable. As it stands, the candidate tried to interact with the interviewer by trying to ask for clarification on the requirements, then making a detailed proposal, and got no actionable feedback from either.
Could one do a "terminal inspired, mutt-like email client" that completely satisfies OP's proposal (web-based, TEMPL, pocketbase, pulumi, etc...)? Sure, it would be possible. None of those are _required_ for the submission but you can totally use them if you wanted to. So the interviewer probably thought, "hey this person is going all out", and was absolutely honest in saying "Looking forward to receiving your submission"
I cannot really blame the interviewer for not reminding the candidate that they should actually follow the requirements, in addition to all the optional stuff they mentioned in their spec.
> Sure, it would be possible
I have to be blunt, you have no idea what you are talking about. I am just going to say this: A terminal app is completely at odds with an online web client. If the interviewer is as clueless as you are, that explains two things:
- I was never going to pass the interview in the first place
- I was never going to get a reasonable conversation from this person, no amount of communication is going to solve incompetence from the interviewer's side
Corollary:
- Gross negligence from the company for assigning this interviewer
have you ever seen http://www.coboloncogs.org/ ? Or https://term.m4tt72.com/ ?
Nothing "at odds" here.. You can absolutely have fixed-size font, character grid based layout, and keyboard-based navigation.
> what if part of the role that’s being hired for is to help people clearly define the specs and requirements?
Then they should have responded to his questions in email?
I personally love working on projects that are vaguely defined because it means getting to interact with people and understanding the heart of the problem. My favorite roles have involved reaching out to non-technical people and figuring out how to solve their problem. Often it's not the solution they even initially asked for.
But if your role is to guess about vague specs with no communication, then you're going to fail no matter how senior you are.
I just read half of the article and this is painful to read. They’re asking for something relative simple as a take home, that just requires a little bit of imagination to fill the gaps. Why does the candidate have to be so fussy about details? Do the best you can, fast, and don’t announce you’re gonna be 2 days late for a take home!
To work in a startup you have to embrace the "just do it" mindset, you have a brain to fill the gaps, and you can always iterate later. I’d be more interested in getting the result of the take home quickly, obviously in a working state, but with a documentation explaining trade offs because of lack of time, ideas for future improvements, design choices, stuff not included (ie. a pretty UI) etc.
My take: they’re literally asking for a CRUD (if you decouple well, you can easily replace the DB with SMTP/IMAP later) with basic email features: send (to, cc, bcc), fetch, reply, reply all, transfer. That’s it, you can do it in 2 hours. Add folders, signatures, "send later", authentication, or whatever if you want to show off.
--
Additionally as an experimented Go developer (8 years working with Go as my main language), the code is not great and tbh I would have rejected it if I received this technical test where I work.
In 5 minutes of reading the code:
- lots of commented out test stuff, don't leave that in a take home or I'm gonna assume you'll do the same in your commits
- lots of println but no logging
- relative imports (eg "mymail/app/router"), don't do this
- weird choice of relatively big dependencies (turso? pocketbase? negroni?)
- no architecture or decoupling, everything is in the router
- bad use of the context (getEmails)
- no tests (no need to test everything for a take home but at least some things can be tested)
People say languages are interchangeable and you're never a "xxx" dev, but that's not true. They asked for "Proficiency in Go" and the candidate is not proficient in Go, that would be the number 1 reason for rejection.
I highly doubt you can produce something that would get you an offer in 2 hours for this task, given its breadth and human nature to worry and attempt perfection to impress. You are likely competing with much more engineered solutions.
Damn now I’m curious what I could do in 2 hours, this is the kind of things where you can easily get carried away so 4 hours feel like 2. I don’t have 2 hours (or 4) to waste to do this but it itches a little bit.
Anyway in 2 hours I’m pretty sure I can write some Go code that would get me an offer. This is not about the amount of features, it’s about giving them what they want.
> Anyway in 2 hours I’m pretty sure I can write some Go code that would get me an offer. This is not about the amount of features, it’s about giving them what they want.
Please do it, get the offer that I didn't, maybe you could even write a blog post about how easy it is and how wrong I am! ;)
I’m sorry I didn’t want to offend you.
The thing is I did a few interviews for Go positions last year and I have a lot of experience with Go so yeah, I’m confident in my skills. I’ve also been on the other side of the process many times and I know exactly what they look for. I’m not saying this test is easy, I’m saying I’m good at it and prepared for these kinds of take homes.
I don’t agree that it’s a bad technical test though, and reading your article I can say your approach was suboptimal. If you keep interviewing for startups you need to put yourself in their shoes.
To quote the test:
> This project tests your ability not only to code, but to deal with ambiguity and open-endedness
Would you mind explaining what you mean by relative imports and what the correct approach would be? I'm somewhat confused since I think of relative imports as something like "../.." which you can't do anyway. I'm currently learning Go, so I apologize in advance if I'm missing something.
Building a concrete, working, minimum-viable solution from ambiguous requirements: that's the point of the exercise. That's what hiring managers want in candidates because those candidates end up being good at building a concrete, working solutions from ambiguous requirements. Which is every software project ever. Although the AI Age has unquestionably changed the efficacy of this kind of candidate screening, that is orthogonal to this discussion. For many years it has been one of the most-effective ways to screen for the ability to build concrete, working solutions from ambiguous requirements. Which is every software project ever. So it's no surprise it persists.
Yes, software is full of ambiguities but there are methods we use to handle them. OP emailed an outline wanting feedback, as any team player would do to iron out ambiguities, and received a meaningless reply. I think it's safe to say companies don't want their engineers going into a corner never to be seen again for 2 weeks, which is what this interview process recreates.
OP's proposal was only describing irrelevant stuff (the backend technologies) and was completely silent on on stuff that mattered (demonstrating how actual RFC822 email works, mutt-inspired UI). It was therefore accepted without comments, as there were no "substance" to comment on.
That is often a problem with proposals/design docs in general. In the real job, if proposal is actually required, it would be sent it back with "please add details on UX and how you are going to store email headers". In this case, the proposal was explicitly _not_ required though, so interviewers did not want to ask for more details on the optional document. They checked what was written there, found no problems, and approved it.
That position is called Email BACKEND Specialist. "We’re looking for a Backend Engineer to help build and maintain our brand new email service. "(c) https://kagi.peopleforce.io/careers/v/108008-email-backend-e...
No wonder he focused on backend part.
I think what has happened is the author has no idea what "email backend" was, so he just decided to ignore that part and build the only backend he knew, web-app backend. And those terms are pretty different. The "email backend" is the service which actually stores and transfers email, in the author's case it was turso + postman.
So from the interviewer's standpoint, author was asking about few details of implementation, like "can I use third-party service for email storage?"; and the response was of course "yes, you can" (because assignment was pretty clear that backend does not need to be advanced or even present, and that it's UI that matters)
I guess the question worked as intended, and filtered out candidate who cannot even read the simple requirements.
(The amount of effort was disproportional though, but I am not sure how to solve this in take-home context without discriminating against people who have busy schedules and/or work slowly)
Yeah, that's likely what happened.
OP didn't take into account the (great) asymmetry between themselves and the hiring manager, then built an entire lament on that. Dealing with this job req is likely just one of many day-to-day responsibilities the HM has and frankly I'm impressed they responded with three whole sentences. One method we can use to handle such ambiguity is to "make your best judgement" based on your skills, knowledge and experience (things that are tested for in the hiring process, incidentally), because often you may not get the answer you want or expect if any at all.
You don't think the author built a concrete, working solution?
They explicitly asked for a minimal, terminal-inspired email client for their Email Backend Engineer role. OP built a ton of infrastructure, created a generic web app that has no semblance of terminal inspiration as far as I can tell, and outsourced the backend to third party APIs.
Concrete and working? Technically, yes. This would have been an excellent submission for a different assignment and role. But it doesn't seem to suit the specifications for this one.
Not what was expected of them. I included "minimum-viable" in my original reply for a specific reason: to counter OP's lament that they lost out to a simpler solution.
If you are asked to implement X and instead you take extra time and come back with X+Y+Z, what have you done? Wasted time e.g. money. Companies really hate wasting money.
So if in your candidate project you demonstrate a propensity for bike-shedding any task, that's gonna be a big red flag.
It was expected that the candidate filled the blanks. He specifically mentioned he would fill those blanks with Y+Z, before he spent time on them.
In the real world, if the time he spent was deemed wasted time, that's management's fault.
I worked with a lot of engineers in my 40+ year engineering career. One thing that most engineers don't like is ambiguity. I didn't like it either, until mid career when I was thrown into an environment where the customers didn't really know what they wanted, and it was impossible to explore all of the solution space within time and budget constraints. I ended up thriving in this environment, but I watched as many qualified and experienced engineers floundered and over-stressed themselves. The attrition rate in this environment was about 80%. To survive, I had to use intuition, speculation, and innovation -- things I had not used much in my previous engineering roles. I sadly watched while a lot of really good engineers chose to move on, or were assigned to less relevant roles.
In this case, I perceive that the author (Jose) was looking for firm requirements, and did what he could to elicit them, but the Kagi folks were just looking for innovation and a demonstration of competency. They didn't care as much about the details of the submittal as they did about seeing what the candidate would do without specific requirements.
> They didn't care as much about the details of the submittal as they did about seeing what the candidate would do without specific requirements.
This would be acceptable in a world where time is infinite. The reality is that they are going to churn through -who knows- how many candidates, as if it was worthless, as if it was a game.
If they really wanted people to spend little time on the solution, they could include that bit of info in their instructions. Clearly, they omit that bit on purpose. Obviously this is not their first rodeo, they must have done this for a long amount of time now.
As I said above, I did a lot of this when time and money constraints severely limited exploration of the solution space, yet I succeeded most of the time.
Guessing is a skill, and some people are better at it than others. An "educated guess" is based upon knowledge and experience, but little more.
I don't really know that this is what Kagi was looking for though. Perhaps they wanted something more than a guess but less than a full solution. Maybe just enough to gauge the confidence of the candidate.
The "new business" environment often involves a lot of guesswork.
Man this sounds rough. I had a somewhat similar experience a while back but instead of going back and forth trying to nail down requirements, I went down this spiral of wondering if the lack of clarity wwas part of some kind of meta-test.
"Do they want to see if I can build something with as little guidance as possible?"
"Do they want me to push for more requirements like I would on a real project?"
"If I build something cool but totally off from what they expected will that make me look better or worse?"
"Or are they just trying to weed out the people that can't code at all?"
In the end I didn't get a follow-up interview and they refused to give me any feedback on how I could have done better on the take-home assessment.
Back to the OP: One such example would be to do a live code review. This could be done asynchronously or synchronously. It could allow actually talking through topics and issues that relate to the challenges in a real software project. This would allow to surface much more of the knowledge in an experienced software engineer.
I like this idea a lot.
Yeah, I had a very similar experience recently. I refactored some things because the code was weirdly circuitous. Should I not have? I wrote a lot of explanatory notes. Was that a bad move? Ultimately I spent several hours on it and my response was basically "Didn't work." I realized that they had some automation set up that would spin up the thing, send some HTTP requests, then shut down. No human ever actually looked at my work, so the time I spent wondering whether I should or should not refactor was probably moot. Didn't feel great.
> One such example would be to do a live code review. This could be done asynchronously or synchronously. It could allow actually talking through topics and issues that relate to the challenges in a real software project. This would allow to surface much more of the knowledge in an experienced software engineer.
From my understanding of the post, this was the initial screening phase as it was in response to OP's application. In other words, this is what every candidate who passes the application screen (the weakest one) is sent.
Let's say they have 100 candidates for this role. A proper code review here should take ~45 minutes to an hour. Even 15% of the candidates requesting a full code review - regardless of synchronicity - represents a 11.25-to-15 hour time commitment from the hiring team. For the initial screen. That is asinine. No proper organization would accept such a large time sink for so few candidates at this phase.
As I've said already multiple times in this thread, OP very clearly does not understand the asynchronous relationship at play here, and then based much of their interactions & interpretations on this misunderstanding.
Exactly! You are actually considering all the possible ways in which your approach might be wrong and in which the time would be wasted. You are assessing risk before you take action. You are analyzing the subtleties in communication, you determined that the wording relied on a lot of subjective vocabulary and you know that this leads towards misunderstandings. This is, in my humble opinion, the professional way of making decisions as a software engineer.
From reading the other comments, it seems like there are a lot of mind-readers among us. ¯\_(ツ)_/¯
I think it’s telling that the comments here are full of people noting mistakes that you made in your interpretation and/or execution, and rather than trying to learn from the experience, you appear to just be doubling down on your insistence that you’re entirely in the right.
Yes, there were flaws in the assignment and communication with the company, but there were also flaws in your approach. You’d be better off to try to take away some lessons here rather than blaming everyone but yourself.
Frankly, if I were you I’d consider deleting both your comments here and the blog post. I don’t think they reflect well on you as a candidate for future roles.
Well said. The author of the blogpost is taking this all too personal. While also seemingly ignoring their own shortcomings in their approach. I'd see it more as a learning opportunity because there was clearly a huge gap in the intended interpretation of the take home and their own interpretation.
> Frankly, if I were you I’d consider deleting both your comments here and the blog post. I don’t think they reflect well on you as a candidate for future roles.
That would be a wise move at this point. I keep wondering what the point of the post was?
I have done this dance a few times. My learning about interview assignments is review the written grading rubric before reading anything else. If the grading criteria lacks specificity then don’t do the assignment. If there is no provided grading rubric in advance then don’t do the assessment.
I was sick of working with this guy halfway through his blog post. To me it was pretty clear what they wanted, it was a "show us what you can do" kind of assignment while he wanted handholding and constant validation, already making excuses for being late and avoiding IMAP/POP due to "complexity" when applying for a backend email engineer position.
Yes, they should've told you not to bother after you sent that proposal. But they were completely right to ultimately reject you. You're a lot of work.
Reading your comment felt more tedious than writing my blog post.
I just can’t get past the fact that he didn’t even deliver on the core deliverable of “ The project is to build a minimal, terminal-inspired email client”. Nothing about the demo I watched on YouTube looked “terminal-inspired”
It's advertised as a backend position, so OP built a backend.
And they failed for it, because they didn't follow the instructions. Nit picking about the requirements and being all "well, _technically_" is both a common personality trait among programmers and _THE WRONG WAY TO THINK_ when interacting with humans.
Lot of people are very spicy in the comments, but for a mid-level position mind-reading shouldn't be required.
Not to mention that the whole problem is that the fucking hiring manager was too slimy to actually go to the technical team and ask them what they think about the proposal, and too lazy to take the time and energy and answer like a normal sentient being, and instead sent this autoresponder-level bullshit reply.
a wrong kind of backend though.. The position is "Email Backend Engineer" and the backend OP built was "web backend"
(OP outsourced the actual "email backend", that is the thing that stores and sends/receives emails, to a third-party products. Probably not the wisest solution if you are trying to get hired to work on email backends :) )
I think rejecting a candidate for almost any reason that those making the decision feel as important is okay, but wasting people's time is not.
Just reading the thread is obvious that the requirements were sufficiently vague -- which, again, is not necessarily a problem -- but going about this in a very disrespectful way is not.
(And yes, of course, the hiring manager always can say that "oh I did not want to reject them before they sent the finished assignment, because they could have surprised us with their code!" -- which, while technically true, but I think simply showcases the absolute uncaring laziness that we see from many companies.)
Well, you are a hiring manager, and you get an email filled with irrelevant decisions (i.e. the parts which do not affect scoring much), and the only part relevant to assignment text [0] is:
> The UI will be kept simple, showing pagination for sent and received emails. In addition to the requirements of the assessment, there will be a login screen ...
How would you reply to that?
One option is to tell them that the document looks fine, but also add that it does not actually describe the parts they are going to grade on (such as: "which part of aerc/mutt is does author imitate?"). But this is a bad idea, because the likely reaction of candidate is to spend even more time on spec... and the assignment is not about writing specs, the specs are not in the rubric, and you don't want to waste candidate's time on asking for docs you don't care about.
Another option is to tell them: "This spec is so bad, I don't think you can possibly pass. Bye-bye!". This is even worse, as it can potentially reject good candidates who are just bad at writing documents - and there are plenty of them.
So I think responding: "Looking forward to receiving your submission." is the pretty OK answer.
(I suppose the in the non-interview settings, if this was a junior engineer, I'd might also add something like "And don't forget to make the pages you write be terminal-inspired, as the requirements say, see videos of people using [aerc], [mutt] and [himalaya]" - but I can see why this was not said in the interview. After all, testing that the candidate can read and comprehend a tiny requirements doc is a part of the interview)
[0] https://archive.md/A95Ju#selection-529.0-545.9
Like I said, it's okay to reject the candidate for submitting something that's implies they are not a good fit.
In my interpretation the "righteous indignation" part is that the candidate tried to make sure that they are not ending up wasting their time and not chasing some pipe-dream (which is bad for both parties after all, leaves a sour taste in the candidate's mouth, and ... apparently does not further the good reputation of Kagi), yet ... that is exactly what ended up happening, because the fucking ~~money lenders in the temple~~ hiring mangers in the corposphere are not just useless individually, but are actively harmful as a cohort.
(Of course, the people deciding to hire hiring managers usually have good reasons to do so. And companies are not expected to babysit every candidate, but since almost always the power dynamics favors them they ought to be the more generous and proactive in the transaction. But at this point in the analysis we are just a few steps from declaring that it's cough late-stage capitalism's cough fault.)
I don't think you are hearing my arguments?
Imagine yourself in the hiring manager's place. Assume all you know nothing about the candidate, and all you have is this email he posted in the blog.. How would you respond to it? Keep in mind you have no idea that the candidate is going to ignore all the requirements, you have not seen the product yet.
I do (and I'm very happy that I got one more insightful and detailed reply), but I think many people here dismiss how easy (and thus common) this kind of fuckup is.
And I expect more from professional hiring facilitators.
I would tell them to write more about the UI/UX, do a mockup whatever. Emphasize that the people doing the evaluation looove TUIs so anything web starts with a bit of a handicap, etc.
I think your response will select a different kind of people that Kagi wanted do.
"I would tell them to write more about the UI/UX, do a mockup whatever." is effectively changing the take-home task from "write code", to "write spec, including mockup, get it approved, and then write code". This can be a valid skill, depending on position - but it also means that applicants waste their time on the writing documents that are not part of grading rubric.
And this brings us to a second part: "Emphasize that the people doing the evaluation looove TUIs..." - this is giving a really big hint. It would be a valid response _if the goal was to get this particular candidate_, say because unethical recruiter is getting paid for each candidate placed, and does not care about Kagi's own needs. But does Kagi (or anyone else, really?) want the kind of candidates that could not even parse out the simple doc? Because they were not subtle about loving TUIs in the requirements, mentioning them multiple times and giving examples.
So I am pretty sure that these kinds of strong hints would be against the interview rules, and there is no "fuckup" in this regard.
If I were to write those replies... I don't actually know their policy about how much help can they give to candidate over email.
If reviewing intermediate docs is too much, I'd have to refuse the candidate firmly: "Reviewing documents mid-assignment is against a spirit of this question. Please write the code using the best understanding of the assignment, and I am looking forward to receiving your submission."
If reviewing those docs is OK, I'd say something mildly encouraging - as the document as sent is not wrong, it's just incomplete. "This document contains no glaring defects"? "I can imagine a passing submission that would follow the plan outlined in this document"? "The results will depend on quality of implementation and how well you've followed the assignment, but so far I see no blockers"? "I cannot promise anything until I see the code, but that can potentially lead to passing assignment"?
Some of those would be better than "This is all very exciting. Thanks for keeping me updated" -- but looking at the candidate, would this make a difference? Judging by the tone of the post, and how he decided to proceed because "this is somewhat of a positive answer", even those harsher responses would end up the same way.
"Email client can either be in the terminal (i.e. a TUI) or a web app"
I understand the frustration here. I assume Kagi has a lot of candidates and is trying to identify the best ones for handling ambiguity. While I can appreciate the interviewee’s desire for more information or a rubric, it might potentially hurt the effectiveness of the take-home assignment.
During my college days, I had an on-campus interview with Microsoft that served as an assessment for whether I would be eligible for more in-person interviews in Seattle. The interview question was open-ended: “I would like you to design an alarm clock.” It seemed like an unusual open-ended question that prompted me to consider several critical factors, such as the intended audience, physicality, visibility, and audibility. By considering these factors and not immediately presenting a solution, the interview team informed me that I had passed the test. If they had provided me with more specific instructions or guidelines before the interview, they might have missed a signal that was important to them.
Unfortunately, I did not advance to the subsequent rounds at Microsoft. I interviewed during the fall semester. While I made it thru the full process, I was informed that I lacked confidence, but wasn’t a no. I was encouraged to reapply in the spring. The initial experience had left me drained and disheartened, so I decided not to pursue a second attempt. I now deeply regret this decision and wish I had given it another chance. If anyone is in a similar situation during their college years, I strongly advise them to avoid making my mistake. Working at a FAANG company could have changed my entire life and career trajectory.
> The initial experience had left me drained and disheartened, so I decided not to pursue a second attempt. I now deeply regret this decision and wish I had given it another chance. If anyone is in a similar situation during their college years, I strongly advise them to avoid making my mistake. Working at a FAANG company could have changed my entire life and career trajectory.
If this applies to you, this is probably the best advice anyone can give to you. I was granted an in-person interview with Microsoft as a freshman back in the early 2000s on account of my stellar GPA. They were the company to work at back then. Unfortunately, I didn't solve the brain teaser. Disheartened, I refused to try again even when they invited me to try again the next few years, and I went with an old dinosaur company instead.
I realized my mistake shortly after starting to work at the dinosaur, and clawed my way tooth and nail into a job at Microsoft. It took me 2 years, but it was the single best thing I have done for my career, and changed the course of my life for the better.
The uncertainty of the assignment and the lack of responsiveness during this week-long unpaid endeavor are inexcusable. Your anecdote strikes me as a totally different process as Kagi's in every possible way.
The problem is not the lack of a "rubric", it is the sheer amount of effort expected from the candidate, especially related to the unprofessionalism of the response. This process is set up to waste a maximum amount of time from candidates without any sort of feedback, and no proof that a reward even existed.
How many dozens of people completed this assignment and got rejected? How many of those assignments were even reviewed by the manager? Did anyone actually get hired? "Giving details would make this less effective for the manager" doesn't begin to excuse this behavior.
Honestly this hiring manager casts a negative light on the entire organization. Do they treat everyone this poorly? Business partners, employees?
I built an email client, it sent emails, it also received emails, it had an intuitive and speedy user interface, and it touched on fundamental topics for a web-application with users, quintessential topics for a back-end engineering role.
Somehow the only criticism I got is that it was not simple enough.
Perhaps the moral of the story is that many managers have no idea what the word "simple" implies in terms of software.
This is a classic situation:
Engineer gets asked to implement the "do what I mean" button by some manager, this is a magic button that when pressed will do whatever the manager wants at that moment. The manager acts shocked when they are told that this is not a simple request to fulfill. The manager thinks: "I am being tricked! I merely asked for a single button!"
> I understand the frustration here. I assume Kagi has a lot of candidates and is trying to identify the best ones for handling ambiguity. While I can appreciate the interviewee’s desire for more information or a rubric, it might potentially hurt the effectiveness of the take-home assignment.
I mean, potentially they failed the real test by asking all the questions - Kagi specifically say they're looking for people who can deal with an open-ended project, and instead of just deciding to do something cool, they seem to spend a lot of time demonstrating that they can't deal with open-endedness by asking a lot of questions and coming up with a detailed spec and asking for approval before starting.
That's not to say what Kagi is doing is good, but it's probably for the best for the candidate that they didn't get the job because it sounds like they wouldn't flourish in that kind of environment (which is better to know before starting a job, instead of starting a job you're going to hate and then burning out or failing probation)
* Ask clarifying questions: Get failed for not being able to handle ambiguity
* Don't ask clarifying questions: Get failed for making assumptions and not identifying business requirements before implementation
nope, they asked the clarifying question, and got the response that there are no business requirements: "That is part of the assessment itself, see what kind of extra features you can come up if any."
They were failed because they could not deal with the answer, they were not failed for act of asking the question itself.
Actually happens sometimes in my work... Q: "Are there any requirements on the database used?" A: "Nope, just make sure the final system will be able to handle the required load"
I feel like Kagi was just asking "Impress us" and OP misunderstood the assignment by actually building a solid project and handling edge cases that no one cared about.
Anyways thanks for writing this up OP, it was an interesting read and I am a Kagi customer so I liked learning about them.
they actually did not... the original requirement said "a minimal, terminal-inspired email client" and "take inspiration from existing terminal email tools like aerc, mutt". This is nowhere near them, not even close.
Not sure why OP makes no mention of this in their blog post - perhaps they thought those parts were unimportant? In which case I am not surprised at the results.
"Email client can either be in the terminal (i.e. a TUI) or a web app"
yes? I am not sure how this contradicts to what I said?
You can absolutely have terminal-inspired web apps, see http://www.coboloncogs.org/
If Kagi was asking to impress them, then the OP understood perfectly because that is not what they said.
I agree on all points.
The project was well made, but my read on it was that they wanted to be shown something interesting. Even if it wasn't as well made of polished, I got the impression they would have preferred something "fun" or imaginative.
Imagine your Jira card at the job:
- Title: Make something fun for the customer.
Then your performance review:
- Manager: We cannot tell you why, but you failed at your job.
This article perfectly highlights the perils of unbounded take home tests.
A timeboxed test for a small group (say, no more than 10) of final candidates can be an excellent differentiator.
The problem here is Kagi have omitted the timebox, so less confident and/or experienced candidates go to the ends of the earth to deliver something that was never going to meet the mark.
Had Kagi said “spend no more than 4 hours on this” - and arguably reduced some of the requirements accordingly - then your man here wouldn’t have wasted a week (!) of his time and Kagi’s.
It’s a real shame that so much time was wasted by the candidate, but the requirements are very clear that “This project tests your ability … to deal with ambiguity and open-endedness”. He’d already dug a hole for himself before he’d started, unfortunately.
It's a tough lesson, but a good one to learn early: never waste time with interview take home assignments. This is especially true for broad and ambiguous assignments that you will need to sink lots of time into.
What if there are 300 other applicants? What's to disincentivize giving all of them the assignment even when there's only one open position? There is no guarantee that anybody will even clone your repository.
In a live coding interview, the company is committing valuable engineering resources to evaluate you. You have some assurances that they actually want to hire someone and not just get free labor or waste your time.
I don't mind take home assignments per se, but it seems like there should be a reasonable "this should take this long" time boxing of the thing so people don't waste a ton of time. I've done take-home things a couple of times and they always had "this should take 12 hours" or whatever time amount.
The "simpler" solution part stood out to me though. Given that Kagi was putting little effort into the thing, I wonder if his verbosity/try-hardness might have been a turnoff? Afterall, part of the thing is determining if you want to work with a person, my impression reading his blog post was.. mildly turned off? There was just a bit of desperation to it (which might be from real factors!) but still. That sounds mean and I don't like saying it, that's just kind of the vibe that hit me.
Yeah the company has to give a timebox. It's true that candidates could abuse that, but open ended "impress us" is asking for trouble for both parties. And honestly you should be able to tell of someone spends 20h or something you said to take 4h on.
My take on the proposal: the hiring contact is not the person who wrote or will review the prompt. The candidate sent this long message and I doubt the hirer even read it. That's some nativety on the candidate's part, but the company absolutely should indicate "you're overthinking this"
I think this is a classic case of over delivering.
The requirements to spin up and obtain live services (postmark, torso), is a bit much, especially for a take home assignment.
Personally, If your app can’t be spun up and tested with a simple "docker compose up —build", then you are doing something wrong.
My rule of thumb for take home assignments: if I spend more than a few hours on it. Then I am probably over delivering and need to rethink approach and remove shit I don’t need. There’s no way in hell I’m spending a full working day, let alone a _week_ of free labor for these assignments.
Note: also not a fan of "take home assignments" either. Of the few companies that ask for these, I consequently ask to be compensated for my time. Only 1 company agreed to compensation, but others declined and I moved on.
> If your app can’t be spun up and tested with a simple "docker compose up —build", then you are doing something wrong.
It can be spun with a simple "make live", I mention this in the first line of the README. ¯\_(ツ)_/¯
While I would never do a take-home assignment for a job, I think your expectations are out of line with the current buyers market that has hundreds or thousands of applications for every position.
Your solution is the same old thing thousands of others could produce. They want more than a programmer that knows some languages, tools and libraries.
I think they might give you 10 more minutes of their consideration if you simply produced a command-line program in Go that used sockets to talk protocols to servers.
I feel Kagi's entire point of the take home assignment is to see how people handled assignments if left to their own devices with little instructions to go off.
When good companies are trying to hire engineers, they don't hire based on skill alone. They also try to hire based on how a person performs given certain constraints like time and ambiguity.
Sharing a proposal and asking for more time was probably the polar opposite what they were looking for
I think a lot of product managers prefer to see simple, fast solutions to a take home assignment. Understandable and turned in on time.
It sounds like Kagi asked for something simple but creative, and the author overengineered it.
Feels harsh to jump on them for it, but it seems like they forgot the job spec when doing the take home. From that list Kagi wants to confirm:
- Proficiency in Go - Strong understanding of scaling and maintaining backend systems - Solid understanding of containerization technologies like Docker
Nailing the take home would require a project that demonstrates all three. Of those I think they maybe do okay on the second one, though offloading to paid services is maybe cheating a little there. I don't think they do well at either of the other two though.
From the detailed spec, one might expect the project to demonstrate those things, but the code as submitted just doesn't.
You might not know this, but implementing Templ with PocketBase is non-trivial. It is actually impossible to do so unless you can read the source code for PocketBase yourself. Pocketbase is a golang framework that doubles as a library, PocketBase implements an entire backend, it is a very real project by an extremely adept author. You would be lucky to work on a Golang repo as featureful as PocketBase.
So it sounds like using PocketBase and Templ wasn't a good decision then?
The author didn't really ask for a code review, though a lot of people imagining themselves in the position of technical hiring manager are already taking care of that.
Anyway: I don't think homework assignments are a valuable mechanism for filtering out candidates. Setting aside shoulds and shouldnots and the ethics of it and everything else, it simply provides a really poor signal for the value of a potential candidate.
Especially today, when something like this particular assignment could just be pasted into any of the popular LLMs and returned as completed within a couple hours.
If an organization wants to hire developers that can take vague project descriptions and convert them into code that may or may not do whatever was in the heads of the people that wrote the project description, then (a) that's a bad practice, but (b) it would be better for everyone involved if you handled this over a video call. If your staff are too busy to do a small pile of video calls with potential candidates over a couple of weeks, then they are too busy to properly onboard a new candidate and you should probably just pack it in. (i.e., they are not too busy to do this.)
If the goal is to hire somebody that knows IMAP, SMTP, POP, etc. inside-and-out and can crank out RFC-compliant code without using any of the third-party libraries that already exist for this sort of thing, then the assignment description should have asked for that.
Homework assignments are an unfortunately common practice, but worst of all, most of them are carelessly designed.
> didn't really ask for a code review
Posting a blog article with a link to the code that they felt should have got them the job isn't asking for a code review?
> Especially today, when something like this particular assignment could just be pasted into any of the popular LLMs and returned as completed within a couple hours.
That would have been significantly more in line with what the interviewer was looking for and would have produced better results. It’s a take home test. Use the tools at your disposal.
That would mean Kagi's selection criteria at this stage is "candidate knows what an LLM is".
I'd think it's more along the lines of "candidate can work with ambiguity and can self direct and use tools at their disposal to accomplish something". Things they explicitly asked for which the candidate completely failed to deliver on.
Seems ultimately the applicant and company weren't going to be a good fit. An interview is a process where both parties evaluate each other. This applicant set the bar too low for this company and wasted time when the signals were there from the start.
Although I agree with you regarding leetcode and the lack of feedback on rejections, I honestly don't understand the dislike for take-home assignments. If you truly enjoy coding, you could see them as an excuse to work on a new hobby project.
My brother on Earth, if I wanted to work on a new hobby... unpaid coding for a company wouldn't be on the list even if I had to think of a thousand hobbies.
You had the right idea to begin with - never do take home assignments.
Second - this is the wrong approach. Nothing wrong with writing a small doc. Write it, then implement the necessary features (terminal inspired client that can read and send email), and test it. Literally a cli for read and send, tack on curses or use imgui. Do IMAP/POP/SMTP or emulate it. The job position is for a backend role in the email team.
Tack on bells and whistles later.
From this post, it seems like the hiring manager is either not the real hiring manager (maybe just an HR agent with this lofty title) or was himself hired without any screening or on the job evaluation for reading and responding to emails (which is a big part, IMO, of any manager) or has multiple roles to play and has no time to devote to hiring.
I feel sorry for the author to have spent more time on actually doing the assignment after that reply.
I went through something very similar — spent several days on a take-home assignment, took it seriously, carefully followed all the instructions and delivered everything.
Then I waited for two weeks, and all I got was a clearly templated response.
The worst part isn’t even the rejection — it’s the vague replies, or no reply at all.It just makes you feel like your time doesn’t matter.
Thank you for writing this. We really do need more people to speak up about these things.
Very insightful idea here. If you work in software, requirements engineering is a key task. Figuring out “would this deliver for you?” is a key question you hash out with clients.
But the absurdity of take home work like this is the company feels obliged to keep their requirements secret. Thus, by doing the task not knowing if it will be up to spec, you compromise on your most important skill!
Failing an assignment might cost you one job opportunity, but complaining about it publicly can harm your reputation and limit your chances with a lot of employers.
In the long run, venting online can be far more damaging than the initial setback - and a huge waste of time!
Same for companies. Also, not writing stuff like this is how people lose power.
I see your point, but I think it depends on the situation. In this case, he misunderstood the task, and instead of reflecting on his mistakes, he complained about it online. Not a great look.
How he misunderstood the task? is it a task or a test? what's the purpose of the test, what do they test? the ability to code or something else?
The fact that there are plenty of hints on this same page as to what he did wrong, does not work in your favour.
Learn what an argument is.
Honestly, you lost it with your first email asking for more direction, and every email after that just made it worse.
This was part of the assignment:
> This project tests your ability not only to code, but to deal with ambiguity and open-endedness, which is essential in R&D projects like Kagi Labs.
It was vague on purpose, they wanted to see how your decision making process goes with what you fill in the blanks with, without needing hand holding. You seemed to ignore this critical piece and went further and further against it with each message.
I had to scroll way too far to see this.
I cringed hard at the “proposal” he sent them (whyyy) + confusing the hiring manager as some sort of counselor + “will I get the job if I build this?”
Huge misread on OP’s part long before he wrote a line of code.
I read so many responses (the replies here to this post and to so many other similar posts) of the form
"Oh, I simply refuse to do interviews [of format xyz]. It's just not worth my time..."
I feel like such a worthless piece of crap when I read these. I apply to hundreds of places, and when someone comes along that even wants to talk to me or a miracle happens and I get to any sort of technical round, I simply DON'T HAVE THE OPTION to be turning them away for any reason.
Like, these companies have all the leverage right now, and lots of us have none. Responses like "just tell them no" seem so tone deaf. I guess some of you must be seriously baller, but it seems otherworldly to me.
This is close to how I feel when I read posts on Linkedin "demanding" that companies dispense with some unethical hiring practice (e.g. keeping up job postings for which they are not hiring, rejecting candidates without feedback, endless interview rounds, etc). I'm like, "Dude! The people who are doing this do not give a rat's ass about your preachy Linkedin post. We have no leverage here. Spare me your clickbait."
End of rant.
The people who are saying "just refuse" are humblebragging that they can afford to do so.
Without wanting to be unkind, this author misunderstands the purpose of the exercise, which is to demonstrate to another human being that you can write code that meets the company's standards (as in, would pass PR review), not to ship a fully-functional product solo. The initial email makes it explicit that they don't care if the product even works ("Can use a fake backend") and that the specifications are deliberately open-ended. A little intuition and empathy can tell you that they probably are not going to spend many hours reviewing a complex submission, so the project should optimize for demonstrability of code quality, not completeness.
"I would like to know what kind of response I could expect..." - This is also established from the beginning: you can expect either a "Looks good, let's do some interviews" or a "Sorry, not interested," based on the code you submit. They can't narrow down the choices prior to your submission, because they're grading your submission, not your proposal document with an extensive list of details that they already told you they're mostly ambivalent to.
"So it is funny that my project is so weak, yet it made them update the guidelines to something stricter." - Main character syndrome. As someone who has been on the other side of these kinds of reviews, the far more likely explanation is that they kept getting submissions where the build instructions didn't work (which is not disqualifying by itself; the authors may not be on the same OS as the reviewers) and they got tired of spending time dealing with it.
Ultimately, the failure here was not a technical one but a social one. The author tried very hard to do the thing that it seemed like they were asking for, not the thing they actually wanted. The hiring manager's unwillingness to engage with the proposal doc was itself a form of communication that they were not interested in this level of detail, it was just an implicit one.
It's common for engineer types to want all that kind of communication to be explicit, and I have a lot of sympathy for those folks, but the reality is that teamwork is a skill, and the ability to suss out what a stakeholder actually wants, but isn't saying due to incomplete information and/or office politics, is a reasonable thing to select for. The ambiguity of the prompt is a feature, not a bug: it's kept the author away from a company whose communication style they're not compatible with.
(that said, this project does sound excessively complex for a take-home)
> A little intuition and empathy can tell you that they probably are not going to spend many hours reviewing a complex submission, so the project should optimize for demonstrability of code quality, not completeness.
> "I would like to know what kind of response I could expect..." - This is also established from the beginning: you can expect either a "Looks good, let's do some interviews" or a "Sorry, not interested," based on the code you submit. They can't narrow down the choices prior to your submission, because they're grading your submission, not your proposal document with an extensive list of details that they already told you they're mostly ambivalent to.
Absolutely spot on, and you ID it later with "Main character syndrome", but it is so very clear from this post's tone & content that OP expected a symmetric outlay of effort & focus from the company's side. They thought they were the main character.
That's a fundamental misunderstanding that seems to have predicated a lot of their ultimate response: they feel as if they were entitled to much more effort from the company than they received. Such is often the case with strong entitlement, it's nearly impossible for the person suffering it to see it.
> OP expected a symmetric outlay of effort & focus from the company's side
And you don't? Do you enjoy being walked all over?
“If you're not the hero of your own novel, then what kind of novel is it?”
> suss out what a stakeholder actually wants, but isn't saying due to incomplete information and/or office politics
What you are saying sounds a lot like this:
"it is common for engineers to communicate properly, but people above them prefer to be vague, because plausible deniability is a political advantage to them at the detriment of the engineer"
EDIT: I originally had an escalatory reply here but have thought better of it. Trying again but without being an asshole: yeah, what you describe is one way miscommunications can happen. They can also happen in other ways that may be the engineer's fault, or nobody's at all. They may not happen every day, just like how you won't be deploying a new service every day, but anyone who works in an office is going to need to be able to deal with them regardless. Highly functional organizations are so because they can recover from mistakes, not because they don't make them.
You probably triggered LLM blindness with your proposal. Never use formal style speech in markdown with bullet points.
I want to make a lot of points about this, so many that I sort of don't know where to start. Here they are in no particular order. I feel pretty strongly that all of these things are true at once.
1) An open-ended take-home assignment with no expected time limit and extremely little guidance is a very poor choice for an initial assessment of a candidate
2) A week of work is way too long to spend on such an assignment unless it explicitly says it expects that much time (and even then, I'd decline)
3) It doesn't seem like what OP produced in that week was terribly strong; it seems heavy on documentation and low on usable code. I see why he was passed.
4) Kagi as a company has always seemed to have an extremely fire-from-the-hip, whatever-Vlad-thinks-is-cool mentality, and I don't honestly trust it much as a result; it also seems like it therefore would not hire someone who works the way OP does.
5) Companies implicitly hire people who are similar to the ones they currently have unless they work strenuously to avoid doing so (in a personality and technical knowledge sense; this is not a comment about "DEI" or something)
6) Asking someone to put in a lot of time and effort and then providing essentially no feedback is an extremely common dick move
7) Take-home assignments should be compensated, if only some token amount, to express gratitude to the candidate for their effort. The better long-form take-home assignments do this already (Automattic is one example of a company that does this, and their other behavior aside, I think it's a good move)
All in all, nobody in this article comes off particularly well. I think OP managed to demonstrate why Kagi has this assessment while also producing an effective indictment of the assessment as a poor choice of hiring tool. Even so, I'm glad OP posted this, credit to him for showing what the hiring world is like right now. It's rough out there.
I did one once for Barclays for a Haskell role in London. A whole evening of work and they never even sent me a rejection email.
I'm fascinated to see how modern interviews especially those with take-home cope with developments like LLMS. But one thing that probably makes sense is introducing more ambiguity in the requirements for the candidate. That's what I'll do because we are no longer just testing for technical skills but also judgement, autonomy and good taste. Todays developers are more like builders and to a large extent product developers.
The actual software engineering and coding is now a lot less valuable as compared to building something useful within constraints. Anyways, here are my thoughts
1. The interviewee failed to truly appreciate the need for ambiguity. Some people felt the interview was over after the submission of the proposal and I agree with them. Coming up with your own requirements was necessary. And this requires your judgement in determining what was essential or not. This skill is surprisingly rare.
2. Good taste is also now more important than ever. It is also hard to test but the interview questions were really good in that regard. They gave you nudges towards the ideal state and left the door open for you to determine the best "path" towards that goal. This tends to reveal a lot about a developer that they might not even be aware of - like over engineering. For a take home like this, I felt the proposal and the half a dozen AWS services was not in good taste. It was probably more complicated than it needed to be.
3. Deadline and deliverables. Exceeding the deadline is a negative. The final solution is not terminal inspired and those alone may qualify as basis of rejection.
4. Of course the hiring manager should probably have rejected the proposal but will that be fair on other candidates? I also feel giving you access to a hiring manager is a stroke of genius. They can assess you via the kind of questions you asked. And I doubt this candidate asked the "right questions". Right here means that they understood the point of the take-home and also what needed to be built. I didn't get that impression from the candidate.
5. I truly believe take-homes like this should be paid. I just don't know how it might work in the real world or whether they are sustainable. The payment here is just a consolation for those that never make it so that their valuable time will be respected. Even more so because I feel a lot of people will probably diverge from the spirit of the exercise.
Is it just me or he missed the point completely?
Look at all the third party services he uses! Nobody sane would pick that solution. That would be a completely maintainment nightmare.
He did not write an email client, he wrote a wrapper to some email client, which handles all the heavy lifting.
At the very least, write something that interact with a real (or fake) SMTP server. Other things are just cherry on top.
Postmark is a real SMTP provider, what are you saying?
What I'm saying is that it should be able to interact with all the email services out there, not rely on a single 3rd party service for the functionality.
Sure, for the demo they can choose postmark as a mockup server. But he should keep in mind that postmark is not a hard requirement, eventually they need to switch to something more controllable/cost effective.
Failing to knowledge that is already a considered failure.
The requirements for even senior software engineers now are absurd (to say nothing of entry/mid level, which I think may not even exist anymore at most shops).
Companies are abusing their position and we should remember the most egregious of them and avoid them in the future.
I don't like the "take-home" interview tasks, but this one seems reasonable and actually interesting (maybe because my relatively recently found love for CLIs). If the language was C# I would apply.
Do I get some bonus points for being born in the same country as the founder?
I tell everyone up front (recruiters and companies) I absolutely don't do homework assignments as part of a job interview. If they're unable to judge my skill level from an interview, that speaks volumes about them, not me.
Furthermore, if they are even the kind of company that thinks they're so special they can or should even ask for that, then it proves they're the kind of people I want to avoid like the plague to begin with. So when I'm asked to do a homework assignment it's actually a huge time saver for me, because it tells me right up front to avoid them.
All developers should refuse this. But there are always enough that are so desperate they'll do pretty much anything.
I forgot my own rule: if the test is to filter out candidates that is ok, if it is to filter in, then it is nok. Filter out: small app with bugs, find the bug or add a small feature.
Take home assignments I feel have the second worst signal for any hiring workflow (behind automated leetcode), you give someone free reign to spend too much time working on something useless without seeing how they actually think or approach problems, you only get a solution at the end, and the solution, especially in open ended problems like these has very loose objective measures. If you want a long term hiring process, I'd just stick to a contract project, though personally just do monitored interviews; I know it takes up more of the hiring managers time, but the amount of signal you get from actually talking and interfacing with a dev is pretty useful
Assuming the recruiting manager is non technical, he should have referred to the team in need when he received the "specification" email, something like "what should I do with this candidate?". A 5min chat with engineers would have clarified the situation and 5min to send an answer to the candidate. Same for the feedback request.
Seems like he did none of it, and in this company people do not talk and everyone is left guessing what is work consists in, which resonates strangely with the take home assignment directives. What an horrible workplace it would be.
To me, it is difficult to concoct a plausible explanation for the behavior from this recruiting manager. Assuming that the manager is not a technical professional and is therefore clueless about my emails is one of the few explanations that make sense. Thanks for bringing it up.
This is the same thing that happened to me with an Expensify interview back in 2011. After that, I decided to only do interviews in person, or where the person was on a call with me. Take home tests are not worth the risk of this crap.
It's totally unreasonable to ask somebody to do so much free work, and the outputs are hard to judge, and likely not the candidate's own work anyhow, so almost entirely pointless.
Remember kids "just say no"
You asked way too many questions.
The instructions seemed clear. They deliberately set an open ended task, and they want to see how well you independently add valuable features (to a well known format).
I agree that this is an obscene amount of unpaid work but if you're going to do it, you can't ask your hiring contact how to do it. It demonstrates a lack of independence, and that's something they told you they wanted for this role.
Independence in choosing an implementation is never ever going to give you good results at a job where you have to report to someone else.
I have been at those jobs.
Trust me (or not): even if you are given this kind of freedom at first, it comes back to bite. You want to get the person in charge to agree with your design for any project that takes a week or longer. Otherwise, the moment that your solution does not create the expected happiness in the stakeholders, you are in a position to become the scapegoat.
This sounds cynical at first, but it's just reality. Besides, imagine if you were the boss of someone... It would be of immense value to get a specific plan for a large project before it even begins. There is nothing good about the uncertainty of not having a plan.
Remember this: I did not ask for the plan to be given to me, I created the plan and offered it as a proposal. I am the one doing all of the difficult pieces, and serving it on a golden platter to the person in charge.
At least now you understand why you were not a good fit.
It’s pretty disrespectful in my opinion. It would be better if they took 90% of their applications and secretly threw them in the garbage than to waste applicants’ time evaluating them on vague criteria.
I have been lucky enough to be able to reject take homes. Next time, I might offer to do them for a flat rate of $200/hour but working for free is something I will avoid.
If it’s going to be used as a screening process, at least give clear guidelines and evaluate it pass/fail.
if you are asking candidates to complete a take home you should provide at least a partial rubric of how the project will be evaluated.
if you can't provide that then i think you haven't created one, and if you haven't created one i'm guaranteed to be wasting my time because you're just doing vibe interviewing.
I hadn't thought about this, but you are 100% correct.
“but to deal with ambiguity and open-endedness, which is essential in R&D projects like Kagi Labs”…. Also based on the nonsense responses by the hiring manager, it sounds like one of 2 things
Either someone had a vision and is saying ‘Read my mind’
The alternative explanation seems to be, there is no vision, and the interviewee needs to define it.
In either scenario, the amount of communication, feedback, specificity, lack of respect for the power dynamics is appalling.
> lack of respect for the power dynamics is appalling.
Yes! That's exactly it.
It is mighty convenient for interviewers to feign ignorance of the power dynamics at play.
Wow you got a lot of responses! I was given a take-home assignment from Kagi to build a RSS scraper where Vlad responded to only one of my four questions. I asked how long I should spend on it, and he responded with a single sentence saying as long as I thought necessary. After four days of work to get an MVP complete, I submitted the project. I received a single sentence rejection a week later.
Love the service. Very disappointed in the process. Four days for a take-home with absolutely minimal feedback is really bad. I decided to jump through the hoops because it seemed like a really cool company to work for.
I am sorry to hear that. It is high time that us engineers learn the lesson and refuse to work with any company that asks for unpaid work. There is no other way, but this only works if we are brave enough to act as activists, so that other engineers become aware of the abuse and circumvent it successfully.
The few that benefit from the abuse also benefit from keeping it hush hush. This is no different from obscuring the pay-bands for employees so that they can get away with underpaying the most vulnerable workers.
Same thing happened to me at Zipline, they give you a basic problem to solve in a take home assignment and then ask you to do something you’re “proud of” or to really show off your skills for the rest of it.
Never heard back after submitting, spent probably 10 hours on it
As someone who struggles with whiteboard and leet-code style interviews, I'm not against take-home assignments.
Just two things:
- If the candidate already has a portfolio, whether personal projects on GitHub, open source contributions, or projects they're willing to share directly, why do you need to make them implement some specific project? The whole point of a take-home assignment is the discussion that follows it where you talk about the code, which trade-offs they made and why, and you try to get a general feel for their understanding and enthusiasm for the domain. Whether they implemented something specific is irrelevant.
I get that this is often done to make lives easier for the interviewer, and to be able to relatively compare implementations between candidates, but every candidate is different, and since you want to judge their thinking the project should be relatively open-ended as well. So just spend some time to review their portfolio instead of asking them to implement the same cookie-cutter project.
- All take-home assignments should be paid. Period. Don't ask candidates to work for free. Agree on a time estimate, and pay them a fair hourly rate. The initial offer can be some average rate for the specific role, but if the candidate has a higher rate, then negotiate based on that. It's disrespectful to expect them to work for free, only to reject them afterwards. This way at least it wouldn't be a complete waste of their time.
I'm disappointed that Kagi's process doesn't take these two points into consideration.
That said, my favorite interview style for programmers is the code review. Show them a piece of intentionally buggy and incomplete code, ask them to mention any issues they find, and to fix them. This could be open ended and include performance issues, testing, etc. Then you work on it together, the interviewer almost taking a co-driver seat in a pair programming session, and the discussion is almost always relaxed and informative. I've been on both sides of this type of interview and it's always been a positive experience, and most candidates enjoy it as well. It's much better than a stressful "here's a blank whiteboard, implement this from scratch", a leet-code style puzzle, and much less of a time-investment than a take-home assignment.
Thank you
Both are wrong. OP should have just done the assignment and not send proposals, deploy it all on server less deployments etc. Kagi should not have implied that the more the better
> Kagi should not have implied that the more the better
Did they, though? From my reading of it, it seems like they wanted a minimal terminal inspired client akin to mutt with a fake backend and plain text.
I know "minimal terminal email client in golang" sounds like a small feat, but it is not simpler than a webapp, it would be about equivalent, at least.
TUI libraries are more of a PITA than HTML. Also you gotta learn the entire API of a specific TUI library to make it work, this learning will never be useful to you again. With HTML at least you know it is a standard.
And remember, they told me:
> We always have a lot of candidates, some they do the basic, some provide a lot of extra features [...] From hiring point of view, we prefer stronger assessments.
and the instructions mention:
> Do the project in a way that shows off your skills as a developer
The instructions also mention "web app" as a valid solution, literally.
So why should I conclude that making the minimal effort would give me a better chance as an applicant?
There is simply no rhyme or reason to the entire process and communication from the company.
> We receive a lot of interest and applications for each position at Kagi which makes selection processes is extremely competitive.
This is the real take away. Don't put in outsized effort at companies that are highly subscribed. Everyone prides themselves on being "extremely competitive" and unless it's a FAANG (or well known quiet money fountain like Valve), it's probably not actually worth the effort.
If it's hot on HN, it's highly subscribed -- you just can't expect effort to be valued the same way when there are 10 applicants.
> We normally don’t provide feedback at this stage. We have had other submissions that were simpler and stronger, so we decided to continue forward with these candidates.
This would have been a great place to push and ask for a little more feedback (while being nice as possible since they have nearly no incentive to share more)... How could the other solutions have been simpler and "stronger" (whatever that means? more robust?) -- a few details on what you missed that they were looking for (tests? documentation? CI?) could at least add some value.
All that said -- finding a job is really hard right now. Wish you the best.
Probably Kagi is using candidates as "chatgpt" answers on what they want to further work on for free. I have already suspect this kind of shenigans 2 years ago and stopped my Kagi subscription. As for adverts on positions, ghost jobs likely. That is the current trends. As for Jose, don't be surprised say 3 years from now they contact you. Some companies pool potential candidates like backup boyfriends. I usually just flat out ban this kind of companies and actively spread these companies to my circle to avoid. The lost is really on these companies without further access to specific job pool (and even customers) and of course bad evangelism on their part for decades to come even if they have changed.
This is very speculative so I left it out of the blog post. But... 'market research' is typical endeavour for companies breaking into an unknown space, the decent ones will carry out focus groups with experienced professionals and pay for people's time.
This kind of "do something for X but we won't tell you how!" looks a whole lot like market research disguised as an interview, that way they can do the research for free.
They provided several example apps to imitate. I hardly think they’re expecting to get novel work to crib here.
So, as someone who has applied to a lot of jobs and has had a lot of jobs, I'm going to be a bit more critical here. I think the amount of information they gave is sufficient for a take-home.
Make an email client, email view+send, fake backend or real imap, handle plaintext.
At this stage, for a take-home, I'd start working and write down assumptions I made as I went along. I'm probably the opposite of the author, as a take-home (unless it's the last stage or something) is, in my view, a tester to see what a person can do within a few hours of work. I've had several take-home exercises during my time as a software engineer and they varied from "we have provided all details that a stakeholder would provide" to "if you have any further clarifying questions, please get back to us".
The most recent one I took a couple of years ago came with an internal library the company used. They said, "use this library to make a web app that takes advantage of these methods inside of it; create an app that simulates behavios using those methods. Do not spend more than 10 hours on the assignment."
I started coding, I threw something together that worked in about 6-7 hours and I was writing down assumptions as I worked as well as trade-offs from those assumptions. "I assume the user would not be bothered by a failure here, if reliability would be important, what would we want to do in the case of a failure? Retry? Back off retries? etc etc". I then provided the code and provided a list of improvements "Add unit tests to these 3 components, add integration test to ensure this functionality works end-to-end if its essential, improve UI, clean up code base, refactor these services, use a framework/library for the UI instead of hard-coded JS to make these few calls. I wrapped it up because I think I was 70-80% there."
During the next interview with the architect of the company, he said that the solution I provided worked fine, we discussed some things I did and he asked why I did them, but specifically, and this is why I'm commenting here, he mentioned that he appreciated the assumptions document, the future improvements and the "I stopped here because I'd rather get feedback on this and refine, as opposed to keep building something I imagine you want". He said that the ability to work up to a point where you hit the majority of the story and then get feedback on that incomplete project is better than having someone dissapear into a cave for a month to try and come back with the absolute finished product in their oppinion and then having to make changes to get it in line with what was actually wanted.
So I guess, become comfortable with uncertainty. If nothing else, ask only "hey, I assume you want something along these lines that I can bang out in 5-6-10-20-40 hours. If you're not happy with what you get for 5 hours, it might not be a good fit in general or if you think I prioritized the wrong thing in those 5 hours, then we can chat about that too". I am also saying this, because my current role is a lot more in line with what the author is looking for - they spend weeks refining requirements, they write documentation, they create mocks and they have meetings over meetings before I even know what I'm supposed to do and within a day or two they start realizing that what they described is about 10-20% off what they wanted or what is possible and the whole cycle of meetings starts again. Instead, and I've been pushing for this each sprint, I'm asking them to accept a certain level of unknown in a given story, we work on it, we see it behave and get used and refine it based off of that.
The author seems to want waterfall, but agile exists for a reason. Hell, let's not even call it agile or whatever else, in a real situation, you start doing and you learn as you move along. You refine based on feedback, you refine based on new experiences and based on new requirements. You work in the murky areas of someone elses mind. Or not, I don't know, but expecting a series of jira tickets with screenshots and deliverables from a company that just wants to see how you think, how you work with uncertainty and how you deal with unknowns feels... wrong.
This (and other terrible interviewing processes) seem like the epitome of a problem being tackled from the wrong end: Trying to filter things coming OUT of the hiring funnel instead of filtering the things going IN to the hiring funnel.
Case-in-point: Modern job sites (cough LinkedIn). Every job listing has HUNDREDS of applicants within an hour of the job being posted. It's ridiculous. It should not be _that_ easy to apply for a job.
The outcome is what we are seeing today: A company posts a job, is inundated with 100s/1,000s of applications. In order to filter out the 80% of applicants who aren't deeply interested in the role, the company deliberately assigns busywork/road blocks to slow down the process.
The other 20% of applicants will then spend days/weeks/months in the hiring process on intro videos, take home challenges, etc. Basically anything that can be throw at the applicant that isn't time with a human.
The takeaway for each can be broken down into:
- 80% of applicants: didn't spend anything, didn't get anything, don't care
- 19% of applicants: spent time doing some/all of the busywork, _aren't_ hired, end up very frustrated at the amount of time/energy/resources that was spent only to be discarded
- The 1%: spent time doing some/all of the busywork, _are_ hired, feel great!
I'm not sure what the exact solution is, but I know that:
a) it's a race to the bottom (with the bottom being full automation on BOTH sides of the hiring process),
b) I'd much rather spend a fair bit of time putting together applications for 5 jobs and be seriously considered than spend very little time on 50 jobs only to be immediately rejected or handed an assignment before I even talk to a real human being
c) hiring teams hate the status quo, too.
Absolutely this, yes!
If you post a JS position, you will get 1,000 or more applicants, so it is a huge amount of work behind the scenes to filter this down to try to find the vaguely worthwhile candidates to interview.
If you post, for example, a Clojure position, you will get 10s or maybe even a 100 candidates tops. And they tend to be uniformly more qualified because niche tech tends to self-select folks who want to explore outside the mainstream.
Of course, a lot of businesses want to use mainstream tech because "the hiring pool is much larger", but the flip side is "the hiring process is a lot more work", because of the volume. So, we get a crappy hiring process because they can't scale up a good hiring process :(
A bit OT, but I'm wondering how any fresh grad is supposed to pass today's hiring bar.
And, with the market flooded with experienced devs, why should any employer take a chance on a 100% green dev, when they can pick and choose from among 5+ YoE SWEs, who are desperate for jobs? As Shawn's post from yesterday showed, many of us are willing to work at much lower salaries, so even financially, it does not make sense to hire a fresher.
Anyone care to engage with this topic?
Isn't the point of inexperienced people that you can pay them less? That's the answer to your question.
Hmmm, what if there's enough competition that experienced people are willing to work at the same entry-level wages as fresh college grads?
If there was enough competition that experienced people were willing to work then at the same rate as college grads now, then college grads then would be even cheaper.
Asking candidates to put in disproportionate amounts of time and effort is abusive. The hiring manager here seems to have taken all of 5 minutes to reject the candidate's multiple hours of work.
edit: commenters here are also missing the forest for the trees. If the candidate had done X, Y, Z to make a more compelling entry then it would be some other poor guy staring at a two line rejection of tens of hours of work. The process is exploitative.
They're not asking for a disproportionate amount of time, the candidate just gave that amount of time, whilst still missing the brief.
the test asks for a very significant amount of work, and makes it clear that they are looking for more effort than the minimum ("Do the project in a way that shows off your skills as a developer", "see what kind of extra features you can come up with").
Define "very significant", you read that brief and you think "an entire week of full-time effort"?
Yes. Because it's a vague unbounded take-home exam and when companies pull stunts like this, they're usually looking for candidates to go 'above and beyond'.
I've done take-homes before and have been rejected because I followed the prompt to a tee and only did the exact amount of effort they wanted, and was quite literally told that they expected me to do more (outside of the prompt). If you've done enough of this game then everything in the OP will be extremely familiar to you.
It seems like half of the commenters here believe they could implement a Golang TUI that sends and receives emails in less than 2 hours. Clearly they have never tried to build a TUI, let alone consider any kind of the issues that might occur between different terminal emulators (OS, Terminal emulator, Shell). The web is king for a reason.
The only explanation I can presume is that these people are mistaking TUI for a CLI app, which would be a fatal mistake.
Ever heard of charm.sh?
No offense but even the readme is way too long and complicated. And the project itself seems like too much.
I often see people applying make this same mistake with their CV. They'll have a 5 page long incredibly overfilled CV with whatever they ever worked on. Instead, just make it one single clear and concise page so that the recruiter can quickly assess your selling point relevant to the position. They don't have time for all this so why would you?
They asked for too much, and I am at fault for delivering? C'mon.
I apologize if the tone in my comment was not the nicest.
You should know your worth as an employee. If a company asks for too much you must have the ability to stop yourself from working for an entire week with no compensation.
> kind' meaning something like 'UX improvement', or 'pretty UI',
They literally said they want a simple TUI email app
> If he wanted a simpler solution, he could have told me on March 18, when I submit my proposal.
When he said your emails could be stored in a fake database or loaded in memory it was obvious they didn’t want something over complicated.
No one likes over engineering, it can come off as egotistical, indulgent and more importantly a sign of technical debt lurking ahead.
Your proposal is long winded, you’re not the only applicant these people are looking at my guy.
I’m not sure what Fargate is (nor care) or even why you suggested AWS and all that other jazz but you clearly went overboard. No one wants a developer where you ask them for A and they return A B and a half finished C.
Yes, you’re allowed to ask questions but the ones you asked showed signs of overthinking and lack of initiative.
This basically screens for desperation.
did the take home assignment precede a chat with the team?
Yes, no one took the time to talk with me on a call.
I love Kagi as a product. I do not love Kagi as a company (based on reports of founder, posted here: https://news.ycombinator.com/item?id=40011314)
That thread is hilarious... The dude bootstraps his own search/browser company, an incredibly difficult space, becomes profitable, and sends his customers t-shirts they didn't want. A bunch of HN users who have probably never even applied at a startup are criticizing him for how he runs his profitable business that he funded himself. If he wants to shovel every dollar of profit into a furnace, who cares? It's their money.
just use cursor lol
Yikes, I love Kagi but if I'd applied for this position and they gave me this assignment, I'd have told them to take a hike. This isn't an afternoon project, it's a full weekend. Maybe two. And are you supposed to be writing tests and stuff for this?
Moreover, this exercise tells you so little about the candidate and their experience. Sure, you know they can code (or did they just use Cursor?) but you have no idea whether they are good and addressing prompts and not at making wise choices. Who cares if the program can send email? What matters is that the client doesn't choke after 10k messages are received or that unicode is handled well: that's the sort of actual challenges you'd be doing at a company like Kagi, not making enormous toy projects.
Edit: Despite that, it seems like a bold choice for the author to build a web app instead of a TUI like the instructions lay out. One could say "terminal inspired" could include web applications but I think that's a stretch.
To be fair, they do explicitly say "build a minimal, terminal-inspired email client" and then "Email client can either be in the terminal (i.e. a TUI) or a web app" below, so even if it's a web app, they expected it to be terminal inspired, especially from the next paragraph in "Inspiration":
> Your implementation should take inspiration from existing terminal email tools like aerc, mutt, or even something like himalaya.
You can totally do terminal-inspired web app - http://www.coboloncogs.org/ is an extreme example, but for this assignment, even fixed-width font and compact layout would likely be enough.
Of course the OP in this post did nothing even close to that.
> even fixed-width font and compact layout would likely be enough
Your assumption is that a better font-choice and some CSS would have made the difference for a back-end role.
No, my assumption is that actually reading and following the requirements would increase the chance of the interview. It's like that Van Halen brown M&M Story - something that is simple to implement, but shows that the candidate actually read the requirements. Because after all, who wants to work with a person who just ignores half of the requirements they were given?
While we are talking, I was actually always wondered about candidate responses like that...
There is that spec, which starts with "minimal, terminal-inspired email client", and contains specific references to the products you want to imitate. Line 3 even explicitly says "Can use a fake backend", and to me, this shows pretty clearly that they don't care about backend, they need UI.
And then there is your submissions which basically has no UI to speak of. It's not even "minimal", as the basic features (like new mail indicator) are missing. Also, in no possible world you can call this "terminal inspired", it's 100% generic web app.
How on earth did you expect this submission to work, given the assignment they sent you?
Did you just skip over the original page and then forgot about it immediately?
Or did you read it, decided that "terminal-inspired web pages cannot possibly exist" and proceeded to implement completely different thing? In that case, why didn't you mention this in email ("and btw, I am going to ignore the part where you said I should build terminal-inspired email client because I think it's stupid.")
Or maybe you read the requirements, and randomly declared half of them "useless fluff" and decided not to implement them?
What was your thought process?
I did someting like this for a hobby project of mine: https:/nbittich.github.io/adana/
I will not be applying to Kagi!
The (interview) game has changed and it will get worse because of more layoffs this year.
To standout, you have to be a bit creative and you must sustain yourself with your own company / startup. (4 basic instructions here) [0]
Companies do not care about you or your take home solution(s) unless you've built something that threatens their existence with a competing product that makes money and steals their users.
Stop playing by their rules with these interview 'games' unless you have lots of time. Spend your time elsewhere. You now have AI, and you should build a startup instead.
[0] https://news.ycombinator.com/item?id=43212438
I like to validate advice. In that line, What startup have you successfully cloned with AI, and gotten traction on, per your own advice.