Discover more from The Engineering Value Chain Revolution
EVC #1: Uber's move to RAMEN, Apple M1's architecture and RISC-V, an interview with DHH
If you’re reading this, the idea for this Substack didn’t end up in my pile of abandoned side projects! My goal with this newsletter is to keep track of the “Engineering Value Chain Revolution”: how are engineers changing their way of working? What are some new architectures they are experimenting with? What products are getting a lot of love from developers? These are all questions I’m always been very interested in answering, both as an investor and as an engineer. The newsletter will usually include links to interesting technical articles, as well as a lighting round with a founder of a startup in the developer tools, infrastructure software, or security space. If you have suggestions, please do reach out! Hope you will enjoy it :)
Uber’s Real-Time Push Platform: deep architectural dive on Uber’s RAMEN system (Realtime Asynchronous MEssaging Network), and how they transitioned from polling to push messaging.
Apple M1 foreshadows RISC-V: interesting read on the M1 architecture and how similar approaches could drive adoption of the open source ISA RISC-V in coprocessors. If you are not familiar with RISC-V, InfoQ has a good overview.
Patrick Woods from Orbit interviews DHH on the Developer Love podcast: DHH is in it, so you know it will be an entertaining one. Touches on a lot of topics ranging from building tools that devs love, to the “merchants of complexity”.
Slides for Julien Le Dem’s talk “Observability for data pipelines with OpenLineage": OpenLineage helps data engineering teams streamline their metadata and lineage collection standards. You can find OpenLineage’s spec on GitHub.
Steam’s login method is kinda interesting: a technical analysis on how Steam handles logins, which gives you an insight into how many systems still run on old infrastructure. Steam’s login page still uses jQuery 1.8.3, which was released in 2013.
Pavel Tiunov is answering our lighting round today. Pavel is the co-founder and CTO of Cube Dev, the company behind the Cube.js open source project. He has a Ph.D. in computer science and has been doing some very cool work at the database level; one of the most technically interesting pieces of their OSS project is Cube Store (more on that later).
What’s one productivity hack that you really love? (Automation, dotfiles tweaks, etc)
Integration and functional tests are my favorite solution to the test time optimization problem. These are tests that don’t rely on code internals so doesn’t change even if code change and doesn’t rely on UI as end-to-end tests.
What’s the latest product your engineering team has adopted at work?
What technology/architecture are you the most excited about trying? What about getting rid of?
Distributed partition wise join using micro partitioning in Cube Store.
[Editorial note: Cube Store is a materialized OLAP cache store designed to store and serve rollup tables at scale. You can follow the development Epic on GitHub.]
What’s one side project you started but never finished?
Many of them. The most recent is to fix integration tests memory pressure errors on CI. My team picked up this one.