Data lineage is a problem because most companies have several tables and queries before humans consume it! This has well known challenges: changes do not propagate downstream from the source, and reliable (fresh, updated or complete) data is not always available. What does Minerva do? I was expecting Minerva to a database (collection of tables), but it turns out that Minerva is what I’ll call: Data Transformation Manager. It overlaps quite a bit with dbt but it’s not a pure execution layer.

This one’s gonna be brief and echoes 2 Less Obvious Ideas to the younger me. I am assuming that you already know the hygiene factors: Make few promises. Keep most of them and exceed few of them atleast. Get to like the top 5% in the skill of effort estimation for your own work at the very least. And so on. Contribute to Developer Ecosystem Improving any part of the developer ecosystem is useful and visible at the same time.

This is primarily for my future self. These are observations based on my own experience of 2 years at Verloop.io and helping a few companies hire for similar roles. Do Seniormost hire first: Start by hiring the senior most person you’re going to hire. E.g. start by hiring the ML Lead (assuming you already have a CTO) Have a means to tell that your investment in data science is working out or not Closest to User first: Hire the person who will consume the data to build for the user first Sourcing: Begin sourcing early and over-emphasise two channels: Referrals and Portfolios Typically, in India - expect: ~2 months to close a full time role at early career (0-3 years) and ~3 months to close a mid career (3-7 years) and 6+ months to close a senior hire If a developer has open source code contributions in the last 2-3 years, consider waving off the coding or algorithmic challenge to speed up the interview process Pay above market cash salaries In 12-18 months from now, when your ML Engineer will have internalised all the requirements, company culture and built a bunch of important tooling - she would get an offer which is 2-3x of today.

You learn a well-paying skill and years later - it comes back to hurt you in unexpected ways. That’s an Anti Skill. Consider this hypothetical: You start your software engineering career and build a reputation as someone who is good at iOS development. Each year, the money you make keeps improving as you keep getting better at it. The downside? You’ll find it hard to get job offers outside of iOS development [1]

Bulk of this is borrowed from notes made my teammate and friend at Verloop.io’s NLP/ML team of our conversations. I’ve taken the liberty to remove our internal slang and some boring stuff. I want to build a community around me on NLP. How can I get discovered by others? Broadly speaking, the aim in forming connections can be split into Long Term and Short term. A short term aim would be where you can receive something immediate out of the connections or a particular connection itself.

This quick entry is partially inspired by a similar entry from Manas Saloi. Ofc, other folks have offered their opinions on Basecamp and earlier Coinbase. The woke mob is shitting on Jason and DHH, but I strongly agree with Basecamp’s new policies. IMHO a company is supposed to create an environment where people come and do their best work without having to align to one side of the political spectrum.