A Hopeful Restart

Well… it turns out that I was caught up in learning on the job as well as the Covid-19 pandemic, and neglected to actually write what I wanted to document when I set out to create this blog. I ended up keeping it around because it was useful for testing web stuff, but so many years later, I’m less than a month away from another life milestone… perhaps it is time to revisit this one.

During my time at Vend, it was acquired by Lightspeed, a Canadian company. The timing was rather unfortunate as I missed out on the vesting of my Vend options, though to be fair I did not place much stock in it to begin with (heheh).

What I did learn at Vend and then Lightspeed was managing a data lake with PostgreSQL, a data warehouse with Redshift, a data pipeline with Python and Prefect, and then migrating from Redshift to BigQuery. Being the key data engineer to get Prefect into production with AWS was perhaps the highlight of my time at Vend/Lightspeed: quite painful, but rewarding. As a bonus, I contributed code fixes and bug reports to Prefect such that they roped me into their Club 42 community ambassador programme.

Eventually, I moved on to Marketplacer, an Australian company that had setup a branch in New Zealand. Over there I was introduced to Snowflake and dbt. I also was involved in more attempts to Terraform our way into managing data infrastructure, something I first experienced with AWS and Prefect, though at Vend I had more support from the SRE team. Oh, and we used the AWS integration with Airflow instead of Prefect, so I felt like such a traitor, lol.

I think that I’ll continue future posts by elaborating on these learnings and adventures, as well as updating y’all on where I am now!

Photo by Nadir Čajić