– 2 teams composed of 9 developers work on the back-end and front-end

– Motivation for choosing Golang for the back-end came down to scaling requirements

– Tens of thousands of clients are connected to 1 point of control (the Golang server)

– Balancing operational scale with developer programming speed

– Their dev team has lots of programming experience and decided Golang was solid

– The client / server architecture of how their load balancer is installed

– The “cloud” component which is the managed web UI to configure your load balancer

– The web UI is written in PHP using Laravel

– It wasn’t really a matter of using Laravel, it was “should we even use a framework?”

– Motivation for picking Laravel for the web interface

– Picking a web framework where hiring isn’t a problem and the documentation is great

– The Laravel app isn’t a monolithic app, many things run on Kubernetes and Serverless

– As an end user, if you click a button on the site it ultimately leads to a job running

– Docker and Vagrant are heavily being used in development

– This isn’t a typical web app, there’s lots of moving parts to deal with in development

– Vagrant makes it really easy to network together your VMs to other systems

– The value in spending the time in your dev tooling to spin new devs up quickly

– InfluxDB is being used as a time series database and what problems it solves

– After only 4 months of being around, we’re writing 33,000+ metrics per second

– Nova operates at massive scale but if you’re not, maybe stick with a SQL database

– Their load balancer is the single source that your clients (web visitors) connect to

– Even if Nova happens to have growing pains, it won’t affect your site’s up-time at all

– What makes Nova different than a load balancer on AWS, GCP or anywhere else?

– It’s an ADC with availability, security, caching and observability

– Nova is more than load balancing and there’s also multi-cloud hosting to think about

– For example, Nova is currently hosted on both AWS and DigitalOcean

– It’s difficult to rely on cloud specific services (ELB, ALB, Firewalls, etc.)

– Nova is replicated between AWS and DigitalOcean for redundancy

– (40) $20 / month servers on DigitalOcean running in Kubernetes

– And another (100) servers for their testing environment to perform load tests

– About (20) servers are running on AWS

– For the Nova load balancers, they are running on $5 / month DigitalOcean droplets

– Everything is running Ubuntu 18.04 LTS, except for a few servers running 19.x

– On AWS, those 20 servers range from $40-60 / month

– 2-4 CPU cores is the sweet spot for their work load, more cores doesn’t help much

– They run their own load balancer to manage their own infrastructure

– Most of their servers are a piece of a Kubernetes cluster

– The rest of the servers are template driven but we’re not using Ansible, etc.

– Those development changes were great because it makes things easier to scale

– Kubernetes is nice but it took a lot of changes in development to make it work

– There is no magic Kubernetes button to scale, it takes a lot of preparation

– Nova supports many different deployment environments, not just Kubernetes

– For example, you can load balance (2) DigitalOcean droplets, here’s how it works

– Doing things like rolling restarts is all handled by Nova

– Using Kubernetes is hard, especially for larger organizations

– What would the deploy process look like for an end user load balancing 2 servers?

– Performing an automated rolling restart with Kubernetes

– The dangers of a fully automated rolling restart without extra “smarts”

– Nova’s deploy process for their own infrastructure (Golang server and client)

– Their CI / CD environment runs on CircleCI but the deploy script is custom

– Secrets are managed mostly with environment variables

– Being cloud neutral is a trend right now (AKA, not being locked into a vendor)

– Moving the data, replication and keeping things in sync are the hardest parts

– End to end, a new web UI deploy for Nova could be done in under 10 seconds

– What about deploying the client / server component? It’s quite a bit different

– Shell scripts can go a long ways, especially for gluing together deployment code

– A lot of monitoring and reporting is kept in-house for performance reasons

– For error reporting on the web UI with Laravel, it goes to Sentry

– Then it’s all integrated with an agent-less DataDog and Slack notifications

– It’s nice using 3rd party tools that integrate easily like Laravel and Sentry

– Not using DigitalOcean’s alerting mainly due to working with a cluster of servers

– Being able to switch cloud hosting providers without a huge fuss

– One of Nova’s clients spend 5 million US dollars a month on cloud hosting fees

– Another client has 500+ load balances deployed across 5,000+ servers

– Spending that kind of money on hosting fees is a whole different level

– Nowadays a small team can be responsible for a huge amount of infrastructure

– WhatsApp and Instagram are great examples of a few devs with lots of end users

– We live in an interesting time where 1 developer can do so much

– Nova’s test environment has 1 million connected clients

– Their databases are run across multiple data centers and are auto-backed up

– Their business is redundancy and up-time, so good disaster plans are necessary

– Working with high traffic clients helped define best practices

– Nova is great for small sites too and you get 5 nodes for free (no strings attached)

– It’s not a pay to win system, the free tier has everything you would need at that scale

– Stripe is being used to process payments and it uses the new Payments Intents API

– Nova bills you per hour instead of per month and Stripe makes this really easy

– SendGrid is being used to send transactional emails to end users

– They send less than a 1,000 emails out a day (mostly for notifications and alerts)

– Their load balancer deals with handling SSL certificates using Let’s Encrypt

– End users of the service don’t need to worry about issuing their own SSL certs

– Push button, receive bacon

– If you’re in the public cloud you can also get encryption end to end

– Dealing with SSL at the load balancer level can save a lot of headaches

– It comes back to setting up your app using best practices in order to scale

– Best tips? Create a code contract to help keep your code sane for years to come

– It’s sort of like Heroku’s 12 factor guidelines but not really

– It’s more like 8 things to do to avoid an angry meme in your pull request

– It’s 8 core concepts (but it doesn’t need to be 8) to define your system’s purpose

– One mistake that was corrected was under estimating time series databases

– snapt