🇬🇧🇺🇸 Rebooting Amethyst Platform

As the year is almost complete, I am yet to come up with an MVP for a personal venture I want to pursue. Over the course of 2020 I tried various ideas, explored different markets and wrote plenty of code. But unfortunately, at the end of the year, I don't really have anything concrete to show for it.

But that's fine. I learned a lot too and that offsets the "wasted" time and per total, I came out ahead.

For the next year though, I plan to be more focused on a single personal project and see how far I can take it. The project of choice is a project that I rebooted for a few times already for different reasons: Amethyst Platform .

The why

Over the past few years, deploying applications was a chore. For each one, I had to get some servers, prepare them and then install the application on it to run. Basically a bunch of repeated steps that I had to redo every time I had a new mini-project I wanted to deploy.

At some point, with Ansible I was able to bootstrap these repetitive processes but there were still some tasks hard to automate with it: launching servers in some cloud and updating the DNS records from Cloudflare (although there is an ansible module for that, it required giving access to the whole Cloduflare account so that it could make the necessary changes, a security risk that I wasn't too comfortable with).

After a while, I got into Kubernetes. I loved that I could keep the configuration near the code and deploy the application as containers wherever I want since most clouds are now offering managed Kuberentes clusters. So no overhead cost, I have only to provision a cluster and then run a command from the application, and everything works.

But still, there were some issues with it: Kubernetes by nature works with very granular and highly configurable resources (such as Pods , Deployments , ReplicaSets , Ingress , PodNetworkPolicy , Service , Role , RoleBinding , etc) which are hard to write from scratch and are very verbose. Creating a new service would often take some googling for the exact syntax and YAML structures I needed.

And it surely was a chore.

Another issue was related to auxiliary software that every production ready app should have: logging, monitoring and alerting. Setting them up in a Kubernetes cluster means fiddling with Graphana, Prometheus and other open source software which have the same issues as Kubernetes: they are complex, very powerful and flexible but no sane defaults, so you have to configure everything from scratch. And it is very complex and time consuming.

So, I wanted a solution that combines the flexibility and automation of Kubernetes with a more straight forward language to specify exactly what I need for my application. In 90% of the cases all I need are some services accessible through HTTP, some cron jobs and the necessary logging and monitoring.

But out there, I could not find a suitiable solution that does what I needed. So I decided to build it.

The what

What I want Amethyst Platform to be is a fully automated solution for application deployment.

Ideally, how I envision the workflow to be, is that the owner(s) of some project would need to write a bunch of YAML files placed inside the project, that will specify what kinds of resources they need: web services and/or cron jobs, configuration and secret management.

The language used by the YAML configuration file would be as bare minimum as possible with sane defaults, so that the user can get their app to a production environment as soon as possible.

I really like what Vercel did with static apps, and I want to achieve something similar for any kind of app. Just connect your project, and from then on every change will automatically test, build and deploy your app, without any kind of human interaction.

The how

The technologies I am going to use are the ones I am most comfortable with, because I plan to build an MVP and launch as soon as possible, ideally by the end of March 2021.

Those technologies are:

• React and Bootstrap for the front end part
• NextJs (basically still React and Bootstrap) for the website: https://amethystplatform.com
• Python + Django with GraphQL for the backend part (I use GraphQL extensively at vuuh.com) so I'm pretty familiar and I find it easier to iterate and build on than REST).
• Kubernetes - The actual workloads will be deployed in a Kubernetes cluster, since I am already pretty familiar with its workings, and I can easily integrate the backend with its API.

Past reboots

As I was saying before, I rebooted this project many times before, for various reasons.

At first, when I first had this idea, it had a completely other shape: it was meant to be a competitor for Kubernetes, and that's how I started the first few iterations: building a distributed agent-controller system that can deploy Docker containers on different servers.

I was basically re-inventing the wheel big time and was trying to compete with a 95 thousands commits giant that is around for quite some years and continuously evolved since then (plus that it was kickstarted at Google and they poured a lot of resources into getting it off the ground).

Not very smart of me, now that I think about it.

But back then I was still in the developer mindset: I liked to build things, but by building all I could understand was writing code. I also was eager to learn and exercise my skills, so as a learning exercise, it was actually not that insane.

From a startup/product point of view, it was definitely over-ambitious, bordering insanity.

I struggled a lot fo get out of this mind trap, and start focusing on how I can actually build something people want to use (and eventually pay for). I am still struggling with it, but I only spent three weeks writing code for the last project before doing some actual market research compared to the months of work I was putting in before that, so it's still some progress.

After a while, I scraped the initial version where I was basically rewriting Kubernetes because I was starting to put in more and more effort but the progress slowed down quite a lot. I was spending a lot of time testing (testing, identifying bugs and replicated them reliably is a whole another beast in distributed environments) and I wasn't spending that much time coding anymore.

I grew more and more frustrated by it because the initial motivation I started the project with was to write code, not test and debug (the frustrated know-it-all developer syndrome when real-life doesn't match the fairy land expectations I had initially).

Then I rebooted it, this time embracing Kubernetes building on top of it. But I dumped the project again when I became CTO at vuuh.com because I had no more time and energy at the end of the day to put into personal projects.

After all, taking care of yourself and your mental and physical health yields better results in the long term than anything. The whole experience was new for me, and I needed to adjust to the new environment and way of work. Over time, I managed to find that work-life balance with my job, and now I have the energy to work on side projects from time to time.

What now?

From now on, I plan to put into Amethyst Platform somewhere around one hour everyday and strive to not distract myself with meaningless busy-work, such as writing code and features for the sake of it.

I'll try to be more systematic about my approach this time around, and build based on feedback. I'll have to identify the potential clients, ask for their needs, build a quick-and-dirty MVP to prove the demand for it and if everything works out fine, I'll continue to grow it until it becomes a full-fledged business. If not, it won't be a tragedy, as long as I come out of it with a bit more knowledge than I had before.

Thank you a lot for reading to the end :)