My First System Design Interview FAILED
44sRelatable failure story of a common interview question (TinyURL) that hooks viewers with a personal struggle.
▶ Play Clip"Delivers a solid, in-depth conversation on learning system design, though the title is a bit generic."
In this episode of Hello Interviews, the host talks with Jordan, the creator of the 'Jordan Has No Life' YouTube channel, about his journey learning system design, his approach to studying complex technologies, and his insights on the evolution of the field. Jordan shares his personal story of being inspired to start his channel after struggling with a system design interview, and discusses the importance of building a strong foundation in fundamental concepts before tackling specific problems.
Jordan is the creator of the 'Jordan Has No Life' YouTube channel, which focuses on system design deep dives and breakdowns of common problems. The host respects his content and wants to discuss his learning journey.
Jordan's motivation for studying system design came from a failed internship interview with HubSpot where he was asked to design TinyURL. He realized he didn't understand how databases work, including locking techniques and not modifying rows in place.
Jordan read 'Designing Data-Intensive Applications' by Martin Kleppmann twice and took extensive notes. He found the book dense and decided to share his notes online, which became the foundation of his YouTube channel.
Jordan noticed that many system design interview videos on YouTube lacked depth and were based on arbitrary reasoning rather than solid logic from books like DDIA. He wanted to provide more grounded explanations.
Jordan uses a breadth-first approach to learning, starting with reading books like DDIA to identify topics he doesn't understand, then researching each one. He uses official documentation, YouTube summaries, white papers, and ChatGPT to gain understanding.
Jordan acknowledges that while he would love to do depth-first learning, the sheer volume of content in distributed systems makes it impractical. He balances breadth to stay pragmatic and useful in his career.
Jordan shifted from designing systems to deep dives into specific technologies because he realized that real-world implementation involves many small engineering choices that are glossed over in interview prep. Reading white papers like DynamoDB revealed the complexity of issues like hot partitions.
Jordan advises beginners to avoid jumping straight into common interview problems. Instead, they should first learn fundamental concepts like partitioning and replication from books like DDIA, then tackle specific problems with a solid foundation.
Jordan emphasizes building personal projects to ground system design concepts in real experience. He suggests starting with simple deployments to understand how databases and servers work, and even manufacturing data problems to test solutions.
Jordan discusses trends such as the rise of stream processing (e.g., Flink, Spark Structured Streaming), the separation of storage and compute in cloud databases, the decline of NoSQL in favor of SQL, and the impact of open table formats like Iceberg and the Arrow project.
Jordan avoids naming a top database, emphasizing that the best choice depends on the situation, organizational context, and operational know-how. He mentions Spanner as great for transactional workloads but expensive, and stresses that trade-offs are everywhere.
Jordan highlights the Kafka and Flink papers as accessible and well-written, and is excited about the Arrow ecosystem, which standardizes columnar data representation in memory and over the network, reducing serialization overhead.
Jordan shares how his understanding of data locality, co-located joins, and indices helped him at Google, and how at his current prop trading job he can identify areas for better architectural decisions, leading to interesting opportunities.
Jordan advises reminding yourself to be lazy and avoid over-engineering. He notes that encountering others' over-engineered code can be frustrating and encourages simplicity for maintainability.
Jordan finds AI useful for parsing documentation but not yet for making high-level design decisions, as those require organizational context and practical benchmarking. He believes AI can't replace the synthesis of technical and interpersonal factors.
Jordan's journey illustrates that effective system design learning requires a balance of breadth and depth, grounded in practical experience. He emphasizes the importance of understanding fundamentals, building personal projects, and recognizing that real-world decisions are full of trade-offs that require contextual awareness.
Designing Data-Intensive Applications
book
Martin Kleppmann
person
ChatGPT
tool
Apache Flink
tool
Apache Kafka
tool
Apache Iceberg
tool
Apache Arrow
tool
Amazon DynamoDB
tool
HDFS
tool
Spark
tool
Trino
tool
Cassandra
tool
VoltDB
tool
MongoDB
tool
Google Spanner
tool
Polars
tool
Arrow Flight
tool
Jordan Has No Life YouTube channel
link
What was Jordan's initial motivation to study system design?
He failed an internship interview with HubSpot where he was asked to design TinyURL, realizing he didn't understand how databases work.
01:00
What book did Jordan read twice and take notes on?
Designing Data-Intensive Applications by Martin Kleppmann.
02:10
What learning approach does Jordan use?
Breadth-first search, balancing breadth and depth to stay pragmatic.
05:40
What is the main lesson from the Amazon DynamoDB paper?
It focuses on dealing with hot partitions and efficiently moving partitions.
08:42
What advice does Jordan give to beginners in system design?
Learn fundamental concepts like partitioning and replication before tackling specific interview problems.
09:51
What is the Arrow project?
A standardized memory format for representing columnar data, enabling efficient data transfer without serialization.
24:43
What is the benefit of open table formats like Iceberg?
They allow transactional metadata updates and avoid vendor lock-in, enabling switching between query executors.
18:03
What is Jordan's view on AI for system design?
AI is useful for parsing documentation but not for making high-level design decisions due to lack of organizational context.
31:11
What is the key to avoiding over-engineering?
Remind yourself to be lazy and prefer simple solutions that are maintainable.
29:32
HubSpot interview failure
Shows a concrete, relatable starting point for learning system design.
01:00Breadth-first learning
Provides a practical strategy for tackling vast topics like distributed systems.
05:40Hot partitions in DynamoDB
Illustrates the gap between interview-level understanding and real-world complexity.
08:42Open table formats
Highlights a significant trend in data engineering that reduces vendor lock-in.
18:03Arrow project potential
Explains a technology that could dramatically improve data transfer efficiency.
24:43Avoid over-engineering
Offers a practical principle for engineers to focus on business value.
29:32[00:01] welcome to another edition of Hello Interviews. This is our chance to talk with leading personalities and experts in the industry. Today, I've got the pleasure of talking with Jordan of Jordan Has No Life YouTube channel. If
[00:16] you should definitely go check it out. I'll include a link in the description. But, Jordan has a huge catalog of both technologies that he's done deep dives on, as well as breakdowns of common problems. But, I really wanted to have
[00:31] him on the channel because A, I I really respect the content and I I think that things. But, also I wanted to talk to him a bit about his journey in learning all of the different topics that he's come into. Jordan, welcome to the
[00:45] a pleasure. I've been following Hello Interview for a while and uh you guys to be on here. Well, I I want to start off with kind of an intro. What made you decide to start a system design YouTube channel? What was it that that prompted
[01:00] this? Sure. So, I guess I have it kind of in my channel description, where uh the short form is at some point I was studying systems design myself. Uh the reason being, so back in my sophomore year of college, I had an interview with
[01:12] with HubSpot for an internship. And that was the first time I was uh was ever know, it was a pretty simple one. It was design TinyURL and I I just was unable really understand how a database worked. I didn't really understand uh what
[01:27] like that. Uh I didn't really understand the concept of, you know, certain uh locking techniques in databases or, you know, not modifying rows in place, but adding new rows to like do things like counting. And I came out of that
[01:40] eventually. So, eventually I'm going to have to get to this." So, I I finished and didn't have anything to do before right, well, I think uh yeah, it was it was a great time, but, you know, besides
[01:54] I think uh I spent a lot of time trying to read DDIA or Designing Data Intensive Highly recommend it by Martin Kleppmann. But as I was going through that, it was really, really long time to get through. And so I was trying to supplement that.
[02:10] that twice now and I've taken notes on it and I think that's really how I got this whole thing started is I read that, I took notes on that, and then I put know what? This was so dense for me. Maybe I can help some other people by
[02:23] internet were were pretty appreciative of that. I was also going on YouTube at actual system design interview questions. And I felt that a lot of the choices that people were making in the videos weren't really based in any you
[02:36] know, logic from that book or anything like that, but more so just kind of arbitrary reasoning. From someone who pretty much all the programming I'd done YouTube tutorials where you're building some website and you're using a Google
[02:49] to spin up. You don't really have to make any decisions. You're not really need to make those decisions. And they're all these pet projects. So I and getting that type of perspective really changed the way that I thought
[03:02] about programming. And then going on YouTube and seeing kind of the the in the past to try and explain those decisions, I was like there's clearly a decisions, I was like there's clearly a pretty big lack of depth here
[03:15] what I see on the internet. You know, at first I made all these shameless videos just completely ripping off DDIA to try and help explain that. re-implement a lot of the problems that I saw in popular interview books like
[03:28] Rocking the System Design Interview and and trying to use the logic that I saw in DDIA to to help those again. Very cool. Well, I one of the things that I love about your channel is you go into a lot of depth about various
[03:40] technologies. I'd love to actually understand how you learn these topics sense being able to teach something is is almost a higher bar than than being able to apply it because you need to
[03:52] really build a good mental model around it. What's your process for learning? So, I think it kind of started out, like I was saying, with just reading the book nice thing about that was right off the bat, that identified
[04:06] tens, if not hundreds, of topics that I knew I wasn't getting enough information then from there, it was just a matter of, like, okay, I heard about this system. You offhandedly mentioned a system like Cassandra or VoltDB or Kafka
[04:19] And it just became a matter of, okay, well, I understand this system exists. I it. Uh let me go ahead and try and look that up a bit more. So, I think when it comes to a new piece of technology, what I
[04:32] find is that when I go to a lot of their official documentation websites, it's really easy to get lost in how many buzzwords they're using. And cuz at the technology, even if it's open source. And so, I I think the actual purpose of
[04:45] find that actually these like 10-minute simple YouTube summaries of a given start. And then after that, once you really understand why it exists in the documentation a little bit more. Or, you know, if they have white papers, that's
[05:00] more recently, cuz I find that to be very helpful. And, you know, just dig ChatGPT is also very much your friend in these areas. If someone's written a good technology, it's really good at parsing that and summarizing it for you. So, and
[05:14] But it's good about lying to it. Exactly. So, yeah, I mean, I I feel like, you know, just using the the internet and really good if you're, you know, if you're not willing to to Google and and
[05:26] actually works internally. I feel like you're going to have a rough time work. Definitely. Now, this process that you topics that you don't understand, you go deeper on them. Would you say you use
[05:40] search as you're going through this? How do you decide when you've gone deep I think definitely a breadth-first search. As much as I would love to do a a depth-first search, I never would have gotten to really cover anything cuz I
[05:56] distributed systems or anything like that, there's just so much content that's been written about it from literally multiple decades. Like or the '80s with uh with databases coming out at IBM. And so at the end of
[06:09] the day, I feel like I could really, really try and dig deep into that, but I'm also trying to balance it with being pragmatic and and learning about stuff think getting a bit more breadth has been nice as well because like you
[06:21] mentioned, I'm still fairly early in my career. It's not like I've, you know, area. And so I think having a lot of breadth has been useful because I can working on. It can just give me a lot of high-level ideas of, you know, solutions
[06:35] learn about them, right? So if it works, someone says, "Hey, you know, we want to do it with stream processing." Rather than being like, "Okay, well, I know a ton about one particular technology and only rely on that." I think it's better
[06:49] to have a a high-level understanding of a few different ones and then use that have to take for a specific problem. Got it. So it sounds like it's focus on things that are going to be practical for your future career
[07:03] opportunities. Is that accurate? Yeah, absolutely. Like at work, I'm I recently team doing software development there. And I think that a lot of the videos the things that I'm interested in learning about. Uh definitely definitely
[07:18] is benefiting my YouTube channel, my YouTube channel's benefiting work, and it's it's a nice symbiosis to have. Cool. So you've taken a step back bit about white papers where you're going to focus more on like deep dives
[07:32] rather than design this system. What motivated that transition and have you learned in in some sense differently now that you're going deeper than kind of this breadth-oriented solution design that you've been doing previously.
[07:47] When I was just trying to build like systems videos for them. At the end of the day, those are tailored towards a 40-minute systems design interview. And even within my videos, frankly, like I don't
[08:02] that in an a 40-minute interview. But like when you really try and implement details and so many small engineering choices that I I glossed over. And I the papers, the more that I came to understand that. And one I think good
[08:16] example of it would be uh partitioning and like dealing with hot partitions. Uh so if you're taking a database and you have to shard it out, so you have representing a subset of the key range, certain percentages of that database are
[08:30] others. And I feel like in most of my systems design interviews, I'm like, partitions, but just make a bunch of small ones and moving them around as And the more that I read these papers, the more I come to understand how
[08:42] is in practice. Like much of the Amazon DynamoDB paper, for example, is about dealing with hot partitions. And and similarly, the original Amazon Dynamo uh paper is also based around how are we going to move partitions efficiently?
[08:56] for our partitions? Things like that. So I think there's a there's a lot of understand through studying all these very abstract problems. But when you build these things, uh you know, it gets a lot more complicated than that. And I
[09:11] like actually come to make some of those decisions myself made me realize that. deeper at times here because I'm just not thinking of things with enough complexity." Got it. So I guess through your career,
[09:24] the depth required for you personally actually increased. And as a necessity, brought to the table. Well, Jordan, I I think you covered a lot of ground over the last several years. And it sounds like you've got a
[09:38] you started and where you you're you're at now. Looking back on all your learning that you you've done, what advice would you give to people who are relatively new to system design? We have a lot of folks who come to Hello
[09:51] design interview coming up and they've never designed a system at scale. They depth already. What would you suggest for them? Yeah, I I don't know if I can give like a be-all end-all best way to learn, but I think I could definitely
[10:06] give a a way that many people do attempt to learn and say it's not the best one, which is to just be like, "Okay, I've got a systems design interview coming up really done this before, so let me go look up all the common problems and try
[10:20] that right off the bat, if you're fairly new problems, the the logic and the reasoning that people give in these videos, if if there's any at all, is not really going to reflect. It might be
[10:33] lost on you. So, if if I choose a particular database and then I give some certain properties of it that make it better for like a lot of writes or a lot don't have more prerequisite knowledge there, then I think it could be really
[10:45] that video. And if your interviewer challenges you, I think you might be in a bit more trouble. So, I feel like it helps a lot to go through the the quote-unquote like fundamental concepts or building blocks of learning that
[10:59] aren't necessarily specific to any one of these particular problems, but are helpful for all of them. Really try and get those down or those fundamentals down and then from there try and tackle interviews. So, I've kind of formulate
[11:12] because at the end of the day, everything that I'm doing on my channel learning. I'm I'm not a master of any of these things and and you know, I'm trying to learn it. I felt that it helped me a lot to first be like, "Okay,
[11:26] concepts down of what's in DDIA plus like a little bit more." I think I made some videos on some MIT distributed systems class content as well. And then a lot a lot more prepared to actually go and like tackle interview questions in
[11:41] particular. So, it could be DDIA that you read. Maybe that's going to be too summary of it. Maybe there's another book that I'm not sure of. I there are there are a few different books that people talk about where DDIA
[11:54] the the most popular one, but you know, just some some book that really explains partitioning is, what replication is, uh stuff like that. Yeah, it seems like there there's kind of like this top-down approach, which is
[12:07] go and find questions that you're likely to answer and go and see if you can get very shallow and and brittle. I think it about the best that you can do in the time. And then you're talking about this bottoms-up approach where you learn all
[12:20] of the concepts and gradually build. It sounds like actually your own approach has been kind of this middle-out, if I can abuse this open valley association, where you're kind of finding some of the fundamentals, but they're attached to
[12:33] things that you want to learn or that you are trying to solve for. And at effective cuz I I don't know that you can approach system design learning from pure first principles. Obviously, without those foundations, you're not
[12:47] It's also it needs to be grounded in something that you understand and you're part is so important. And I'm glad to hear that that's also been a common I totally agree and then I I think if I had to add anything else, it would just
[13:00] be, you know, to to keep building personal projects on the side because understand and you can kind of tie those into the systems design a little bit. So, you know, you'll start to understand why you need a database or why you need
[13:12] something built a certain way, why you need a couple of application servers know. I I felt that things like that to the whole thing rather than just like learning a bunch of vocabulary terms.
[13:25] personal projects that you've attempted outside of work that you felt were were valuable and taught you a lot? Nothing nothing too crazy since college for example, but literally
[13:37] so much as like being, you know, I'm going to deploy a website and I'm going a server to literally understand like, okay, this thing is persistent. This thing is what's answering my application requests. Like for a lot of people when
[13:49] and you're writing code, it's just going to be like data structures for the most part or operating systems and the jump from doing that to actually making like a a user-facing application is a lot less trivial than looking back it it
[14:03] seems at times. Like, you know, the first time that I went on YouTube and I React and have a back end and let people log in? Like that's that's actually a most people are working on in college and takes a lot of work to get from from
[14:17] point A to point B there. Totally. I think one of the challenges with personal projects for a lot of people is just getting to that level of interesting. So certainly if you've never deployed a website, by all means.
[14:29] you're trying to figure out how to deal with hot shards as an example or how to scale and distribute load, you oftentimes don't have that problem have it everywhere, but as an individual
[14:42] earliest personal projects were actually web scrapers and my whole goal was I'm going to assemble a ton of data so that I can try all these cool approaches. I I basically need the big data so I can can have problems that I can go and test my
[14:56] solutions on. I'm not sure if you had anything similar, but it very much feels that is kind of the bottleneck. That's where you'll start to suffer if you don't have a job already where you can test those those concepts. That's a
[15:09] that's actually a really great idea on your end is to basically just manufacture them for yourself and mess around. So I like that a lot. Well, you've touched on a lot of old tech and some modern stuff.
[15:22] I love the the TikTok video. ML system design is one of my sweet spots. I'm As well as, you know, some ancient ancient, but I remember some of the content that you had around there. I'd
[15:35] someone who's been exposed to, you know, a lot of the history of system design. What are some trends that you're seeing either things that people used to be doing and are no longer doing and so, for instance, candidates should try to
[15:49] avoid or things that you think are hot and are going to be really likely setting the stage for the next a few iterations of system design. caveat this statement with saying who knows because, you know, despite there
[16:03] being trends, you know, I think there are tons of companies that are old-fashioned way of doing them. So, for example, my company is still running underlying object store for all of our data. So
[16:16] Wait, HDFS or what is this? HDFS, yes. Oh, HDFS is great. Battle-tested. Battle-tested, exactly. The first one I'm going to say maybe is um consumer technologies. And to be clear, like this isn't something that arose
[16:30] for a while, but I I feel like just the number of people who are like, "I've never heard of anyone using like Spark structured streaming or Flink very seems pretty cool to me and it definitely seems like it's being used
[16:42] opposed to just like building your own consumer and then, you know, building your own replay mechanism of persisting state to some data store on your own. So, I think that's maybe one of them. Again, not super new, but just has to be
[16:55] kind of focused on the analytics space cuz I think that's where I'm finding a just, you know, like columnar storage as opposed to row-based stuff. You know, you'd be surprised, but like Spark operates on row to row and that's funny
[17:08] because, you know, it's a big data tool and so it should be operating on, you time to speed things up. So, there are a lot of uh you know, projects going on to to make that happen. We kind of touched upon this already but just the concept
[17:21] of the separate scaling between storage and compute in cloud databases putting all these files in S3 and then having a separate compute layer because network so polling a bunch of parquet files out of S3 is not too bad. I think
[17:35] this one is more of an ode to some of the the the OG database guys but I guess maybe this is less true in practice. I feel like no sequel is losing some steam maybe I'm wrong about that. Mhm. And I feel like everyone's just
[17:49] like okay yeah sequel's good again but maybe I'm wrong there like I've got very this off what I see at my current company and the vibes of the people that you you're hearing? I I think so. I know that okay
[18:03] like three to four years as opposed to like last 10 years I would say something like iceberg you know open table formats iceberg hoodie these hive tables just saves you a lot of time in terms of listing all the you
[18:17] you to update your metadata transactionally which is very very useful. A lot of benefits there. I think the arrow project is insanely impactful or at least has the potential to be having one standardized format to
[18:30] and over the network is massively science with python or like polar it's great for sending data over the network another for more processing.
[18:43] in general. And then the last one I'm going to give you your flowers on because I think you really opened up my mind more to this is all the time maybe CPUs are just getting good enough and you can run on you
[18:57] know one node and that's okay. And so yeah Exactly and vertical scaling is back. Tons of cool projects going on out machine learning space but I think a lot of these are starting to be integrated
[19:10] just going to be a big big big data bottleneck as well. You hit on a lot of cool stuff there. I love the stateful streaming approaches. practices that have been accumulated over the years and put them in nice
[19:23] frameworks that you can use. I think Flink is a great example of this. And given so far are Apache projects. It's been crazy to watch how effective Apache has been at incubating good ideas and then giving them standard status that
[19:37] can be used. I think Iceberg is the perfect example of this, where you've got so many vendors in the big data space, but having an open source, reasonably open standard that everyone can integrate with means that, you know,
[19:49] the floor raises. And that's probably how we should expect technology to evolve for, you know, decades to come. It's great to see everything just said, because you know, the the less vendor lock-in that you have, the
[20:01] Iceberg as your metadata layer, you can switch between types of query executors that you're using. So you're not necessarily locked into to Trino or Spark or Flink or anything like that, which is great.
[20:13] Perfect. So I want to kind of put you on the spot. I'd love to hear about your opinion on databases. And let me just kind of set the stage. Like as a generalist software engineer who might be put on kind of any problem space, and
[20:25] you want to be able to be effective in any of them. What would you say are like your top four databases? Maybe the top four categories and then who's the Oh, top four categories is tough. Oh man. I was going to say I'm I'm almost
[20:37] like a little bothered by this question because I think that uh more so just No, you have to answer. answer. This is this is my This is my channel.
[20:50] kick me off. It's going to be like Zelensky yesterday, but anyway, point is necessarily a top database. I think there's a a top database for a given situation, and it's not even necessarily based off performance.
[21:03] standards. There's operational know-how that's built up between the people working there. And uh sometimes the best database for the task is going to be the one that you already know how to use or have tools built around. And so, I think
[21:15] bunch of databases that I personally am really interested in and I think are great. You know, I'll toss in whatever. Spanner is always going to be great. transactional stuff, but it's probably expensive. So, everything has pros and
[21:28] cons and I think that looking at your specific organizational structure, what you're able to spend, how realistic things are to deploy is just as important as like random performance characteristics of a given database. And
[21:40] that's that's another thing is, you know, as much as you can make all of systems design interview where you have an idealized problem where, you know, you can pick whatever database and and just be like, "We're going to use this
[21:53] you know, what your organization is comfortable with, maybe the cloud with or things like that are also just going to take major precedent when it comes to to making a data store choice. Yeah, definitely. Tons of trade-offs to
[22:07] are completely outside your control. There's not really an optimal in some sense. This is actually a good lesson in general for people who are entering system design interviews is that there isn't a right answer to almost any
[22:20] question. I think there definitely some of our audience got the impression that there is some optimal solution and the reality is there's not. There's just trade-offs all the way down and there's obviously a Pareto frontier. You can
[22:32] make a bad choice but the best one is often times ill-defined. So, I I actually love the fact that you you differed on the the question here. thing is you just you don't want to build and put engineering hours into
[22:45] like you mentioned, everything could possibly just be handled with one one and you don't need to make all these crazy decisions. So, it's really only when you're at the frontier of of optimizing stuff that things get super
[22:58] yeah, sometimes life can be pretty simple, which is nice. friend as a software engineer. Of interview for that talent that's going to lead the charge when they are at that
[23:12] that new problem that no one solved before. But, I think for the reality of most engineers, they're solving problems that are reasonably common and and there out there. So, choose some boring stuff cuz it's going to work out pretty well
[23:26] To that end, I'm going to maybe flip a bit and and be less practical, but I'd love to hear what has been the most interesting system that you've covered recently or the most interesting technology. I think, you know, there's
[23:38] technology, some that have very broad appeal, but just from a pure learning or new concepts that are being brought to the table, what would you say maybe over covered that you would say tops the list?
[23:52] Sure. Well, I think I guess I would maybe split this into two categories. I think there were like the the best reads in terms of papers because they're, you understand. I think, you know, most engineers, regardless of how much
[24:05] systems, could understand them. So, for that, you know, I think I would give the the Kafka and the Flink paper a pretty big shout-out just because uh there's no like crazy compiler optimizations that they're doing or at
[24:18] them. All the terms are pretty easy to look up and figure out what's going on. They're not super long papers, super common technologies as well, and I I feel so like evangelized by by both of those technologies. It's just like
[24:31] really nice papers and and easy to consume as a as a reader and understand are using them. In terms of just like projects I'm excited about, I'm just getting out this one a bit, but the whole Arrow ecosystem in general,
[24:43] I guess if I if I dive into it for like a minute, basically it is a standardized memory format for representing columnar data. So, rather than representing your data in row by row and basically having A, B, C next to one another where A, B,
[24:57] same row. You would basically have A1, A2, A3, B1, B2, B3 where those are for you using tactics like dictionary and run-length encoding. And you know,
[25:09] format in memory, which is really nice for passing data from one process to also got Arrow Flight, which does the same exact thing except it's passing data over the network without doing any extra serialization or deserialization
[25:24] really efficient. A lot of tooling is getting built around it, and I think it's going to be very exciting in the big data space in the next few years. copying data. I don't think people realize until you actually go and look
[25:37] technologies how many copies are created, especially during network biggest bottleneck for the performance of a lot of these systems. You know, plenty of plenty of benchmarking went into that. And now
[25:52] Got it. I want to talk a little bit about career and how your learning has real job. I think there's this impression from some engineers that, you large-scale systems all the time. Or maybe my work is mostly code and bug
[26:08] fixes and on-call issues as opposed to design, which I might do once in a blue moon. Have you found that to be true for you personally? And then where have you accrued both as the creation of your channel and just your personal curiosity
[26:24] come to be kind of most effective in you doing your job? I guess uh lane for a second. I guess I'll outline what I've done in work. So, I started out my career working at Google again as a software engineer on like a data
[26:38] team, so building a lot of pipelines. Right off the bat there, I think when in environment that's handling a lot of data and you're trying to write having an understanding of what things like data locality and co-located joins
[26:52] and indices really do for you is hugely impactful. I was really fortunate that Google gave us a great set of profiling tools to determine what was making a the bottlenecks were in a given piece of code that I had wrote. And I feel like,
[27:06] fundamentals of just how big data and databases works in general, why it helps you to sort certain pieces of data, why it helps you to partition your data on a certain key, massively helped me to build things that were more efficient.
[27:19] And then also I even had people who were higher levels on on my team that you areas as well. And I was able to help them out with it as well. So I I I felt big impact off the bat and I was able to present to my team. I think there were a
[27:32] lot of huge benefits there. At at my current job in in prop trading, I think it's more of a situation where it's like if you want to seek out roles where stuff, you certainly can. I think there are a lot of areas where it's more so
[27:46] just, you know, about writing clean code and and functional code than it is about same time, I think because there are a lot of people who have spent more time focusing on that, it's also something that has at time
[27:59] been neglected. And so I've been able to identify certain areas of our code base better architectural decisions in terms of which technology we use or in terms that. And I feel like it's allowed me to pursue some really interesting
[28:12] opportunities in my company and then jump around different teams as a result passions. So in general, maybe it's not always at, you know, these fan companies or whatever is going to be working on
[28:26] know, mobile designers, front-end designers, all this other stuff and just like normal back-ends as well that aren't huge tools. But I also think that at these companies and you can totally seek them out and work on them if you
[28:39] want to and you find them interesting. Yeah, I like the contrast there also and you kind of making this part of your job versus this being implicitly part of your job. I'm kind of curious how you avoid this trap that I find at least
[28:52] from a career perspective some engineers find themselves in where they're in some sense making uh they're creating problems and and solutions that maybe don't need to exist. I've certainly been guilty of
[29:04] bunch of interesting code to solve problems that probably weren't the most important things to be solved by the business. How do you keep yourself the problems that you actually need to solve in order to be effective and to
[29:18] make your team successful while also applying interesting things where they're going to be most valuable. Yeah, that's a great question. I haven't, right? And uh you know, I've tried to really over-engineer something
[29:32] excited. And I think that's when uh someone who's more junior it's really experience on the team who can be like, "Listen, interesting and I think it'll work well, but you know, the the bottom line is
[29:45] any money." And that's fair and I can respect that. I think as time goes on I'm starting to get a bit more of a sense for this myself. Uh and to be it in my head is just like reminding myself that I'm lazy and being like,
[30:00] my own developer hours on this if I can get it done with a lot less code or a simple solution to this problem." And also I know it's one thing when you're own because you want to, but when you're the yeah, now I wouldn't say victim's
[30:15] encounter someone else's over-engineering I think it really Yeah. like, "Wait a second. Huh, maybe this makes me re-evaluate uh have done a little bit more simply such that other engineers come along and have
[30:28] maintaining it." There you go. Yeah, you look at the Git have been really bored in 2023. I wish her boss would have given her something more interesting to work on." Exactly. So, you know, you find
[30:41] just like, "Geez, this is really impressive, but at the same time, why'd you do this?" I kind of hate you for this, yeah. It's It's definitely a bit of a mixed bag. Awesome. Well, I've got one more question for you and you you
[30:55] you'd mentioned ChatGPT earlier and using it in in an educational capacity, but I'm curious to get your perspective on the impact that AI will have on software engineer's role into the future. Do you find yourself being able
[31:11] higher-level questions that you encounter practically? And do you find it to be useful in answering deeper, more reasoning-heavier, trade-off-heavy deep dives? Right now, at least in my like
[31:25] day-to-day workflow, AI has been really, really nice in terms of just being like you know, parsing existing documentation. I think it's not gotten to the point, at least in my experience using, I don't know, ChatGPT-4.0.
[31:39] I don't think it's gotten to the point that I would just be like, "Hey, what's the best design to go implement this problem?" Because, like I was, you know, alluding to before, that is very much not a question
[31:51] that can be answered by someone without any actual organizational and practical context within your company. Like, you know, you have to go talk to other know, situation is, what your relationships are with certain companies
[32:05] actually like pay for, what you can't pay for, what you have to deploy open source, or how many resources you have available, or how much other departments implement this service. And I mean, in theory, yeah, maybe you could pipe all
[32:17] that information into ChatGPT, but just like practically, you know, it can't do it on its own with a quick question. You have to do a ton of of speaking to other people and really flesh out practical requirements in order to to build real
[32:30] systems as opposed to, you know, asking at high-level trade-offs. I also think that even in terms of something like you know, oh, in theory like take a database like Cassandra, that's
[32:43] really nice for write speeds because it uses an LSM tree. Like I think it does I feel like it may have trouble synthesizing like, well, how does that make it compared to like MongoDB if we were to choose that instead for this
[32:56] problem? Maybe if someone like really formally tried to outline these things in like a very easy to parse way on one document, it could do so and then like spit that logic out. But again, that's that's all conceptual. It's easy to say
[33:09] really mean anything until you actually go and like benchmark stuff. You know, all this like high-level kind of hand-wavy stuff that I like to say in my matter because, you know, maybe someone just wrote better code than someone
[33:22] engine type, you know, that database is slower. So at know, maybe you can be skeptical of things that it says, but you're just of the day. And you also have to make practical decisions to to get systems
[33:35] that actually work for your for your user base and you know, within your I think there's some practical takeaways for that as well. You you mentioned both there not usually being an optimal answer, a bunch of tradeoffs, but also
[33:48] important to the decisions that you're making. And if you think about that as you're adding value is that you're internalizing all of those pieces. And that has always been important, but it seems increasingly important as the
[34:02] tooling changes. Yeah, and it's something also that I I feel like I've become more privy to, especially the the older I get and you know, the more years of the day, you probably have stakeholders and and making sure they're
[34:15] doing is also very important. So you know, maybe ChatGPT can go and suggest stakeholders have never heard of it and they're not interested in it, you know, there there are definitely a lot of practical concerns that come up when
[34:30] think that like you mentioned synthesis of all these distinct sets of information and navigating not only like the technical challenges here, but also the interpersonal challenges are very real. Well, Jordan, this has been
[34:42] incredibly insightful. I like how we covered both your career journey and then how a lot of your learning you were able to basically broadcast this outward, but you're able to integrate this into the career path that you you
[34:54] thinking that you had and how it's evolved over the depth that you're covering, but also their practical solutions and there's a lot of trade-offs and part of your job as an
[35:06] engineer is to build content. Some of those things are important for system design interviews like especially at the staff level. These are things that your Do you recognize that these are contextual? And some of them will only
[35:19] the job just because you're right, a 40-minute interview necessarily can't therein. So, super grateful for you spending the time with us and yeah, I will drop Jordan's channel into
[35:33] the description. If you haven't subscribed, I'd highly recommend it. weekly basis? You seem to be coming them out pretty frequently. I try to do it weekly. I will see it may may drive me insane soon, but we're we're still doing
[35:46] weekly for the moment. That's impressive. Awesome. Thanks man. I appreciate it. Yeah, thanks again. Talk to you later. Yeah, thanks again. Talk to you later. Yeah, thanks for having me.
⚡ Saved you 0h 35m reading this? Transcribe any YouTube video for free — no signup needed.