• My First Week Learning vLLM

    My First Week Learning vLLM

    When I first read about KV caching in AI Engineering by Chip Huyen, I remember wondering why I was reading this. I was never going to use it. This week I spent quite a lot of time looking at KV cache charts. I’ve been learning deep learning for a while now, mostly through books and by…

  • I got my hand in Apple’s cookie jar

    I got my hand in Apple’s cookie jar

    When I was building Dashman, I hit a wall that Apple’s paid developer support told me was unsolvable. Then I solved it by reading the binaries in assembly until I understood them well enough to swap them out. A note up front: this happened a long time ago and I’m writing it from memory. The…

  • When failure was an option

    When failure was an option

    A few years ago, at Wifinity, around 4 in the morning, I made one of the hardest decisions of my career: I rolled back a migration we’d spent months preparing for. We were migrating the production infrastructure of an ISP serving 200k+ customers. Traffic was already flowing through the new environment in read-only mode, the…

  • Your competitive advantage is your agency

    Your competitive advantage is your agency

    I see a lot of grandiose statements floating around: You will not be replaced by AI, you’ll be replaced by someone using AI. AI won’t replace your creativity. AI can’t do THIS or THAT. AI will never be able to do THIS or THAT. And many more. My take: maybe. Who knows! LLMs are getting…

  • Culture goals should be achievable

    Culture goals should be achievable

    A company’s overall goals can shape its culture. Take the popular “Employee of the Month” idea, when employees see it as meaningful, it becomes ingrained in the culture and can motivate higher performance. But these goals need to be achievable, and sometimes making them achievable is a small matter of phrasing. Let me show you…

  • A better monetization model for LLM applications

    A better monetization model for LLM applications

    Bring your own keys into an affiliate relationship I’m starting to observe a problem where a lot of LLM-enhanced apps are starting to pop up. For coding you have Cursor, but now there’s also a terminal called Warp and it costs $15/month. For individuals, consultants, small and even medium sized companies, this isn’t a workable…

  • The Feedback Loop

    The Feedback Loop

    I work at a pretty amazing company, Canva, that has a culture of feedback. I think I have given and received more feedback in the two years I’ve been here than in the rest of my professional life put together. It has taught me something critical about the importance of a growth mindset. Managing several…

  • The role of CTO should have no job description

    The role of CTO should have no job description

    I’ve been thinking about this one for a while. Imagine you are the CEO of a company and your competitors are getting ahead of you because they started to use ChatGPT to build their tech. You ask the CTO what’s going on and the CTO says “ChatGPT was not on my job description, so I…