– Working nights and weekends until it made enough to replace consulting work

– Being your own boss changes how you think about writing software

– Thousands of developers a day use TrackJS resulting in thousands of errors per minute

– Installing TrackJS is painless, just drop the JS snippet into your site’s HTML

– Debugging client side JavaScript can be difficult for a number of reasons

– Motivation for using .NET / C#

– Why it’s a good idea to avoid shiny new tech when you’re building a new product

– Why TrackJS is mostly a monolithic application instead of microservices

– But there are bits that are broken into their own service when it makes sense

– Creating a pipeline to efficiently capture and process a ton of incoming data

– Leveraging nginx to quickly create logs for requests that are processed later

– For data that is more time sensitive they wrote a .NET service that uses Redis

– If TrackJS gets slammed, it will never effect page load speeds for their customers

– nginx was configured to write out JSON formatted logs

– The processor service ingests those log files and figures out what to do next

– Then there’s the web front-end service that developers use to browse their errors

– Elasticsearch is used to store the errors to create very fine grained reports and filtering

– A quick recap of the technologies used so far

– ASP.NET is similar to Rails, it’s server rendered templates but they use React too

– pjax is used to make the app feel very fast even with server rendered templates

– pjax / Turbolinks is one of the best bangs for your buck to make your site feel fast

– Making the most of your tech stack with a small team of developers

– Elasticsearch needs a bit of tuning if you’re using it as your primary database

– Writing their own .NET class to interface with the Redis backed queue

– IIS (Microsoft’s web server) serves the app without nginx sitting in front of it

– Load balancing is done over DNS with a round-robin strategy across 3 servers

– All 3 web servers get restarted at once during updates because IIS is great like that

– Everything is hosted on dedicated hardware with OVHCloud after moving off Azure

– Poor support and opaque downtime resolutions is why they moved off the cloud

– After thinking about, using Ansible to set up machines seemed like a good idea

– They landed on using OVH after doing a bunch of research

– $180 / month for a high end Xeon 8 CPU core server w/ 64 GB of RAM + 2 TB of SSDs

– It was more work to set up but it’s A LOT faster and costs were dropped in half

– When something goes wrong, it’s obvious on what went wrong when it will be fixed

– Even while running at 10% capacity, they do capacity planning every quarter

– $180 / month is an average figure, they have smaller servers doing different things

– They run about 12x Elasticsearch servers that are pretty beefy ($240 / month)

– Overall they have about 30 servers that they have to manage

– Some servers run Ubuntu LTS, and the web servers run Windows Server 2016

– Managing Windows servers is kind of a pain in the butt

– Ansible is used to configure both the Windows and Linux servers

– It takes about 48 hours to get new hardware from OVH, but that’s not a problem

– Using Team City to help get code from development to production

– The test environment gets real production data synced every hour

– Their “dev” environment is really a test environment

– It gets pushed to production manually through a Team City job by choice

– But every time they git push code, a new test environment is set up automatically

– They use their own service to help monitor JavaScript errors and it helps

– They built their own back-end monitoring tools too due to lack of choices

– Todd has opinions on back-end monitoring in general

– Real exceptions get sent to their primary Slack chat channel

– Payments are handled using Stripe but it doesn’t use SCA

– Monitis is used to monitor their infrastructure load and website up-time

– They would still use rented hardware but maybe use .NET Core today

– Depending on well tested and mature tools allows you to use them years later

– Best tips? Don’t build something in new tech just to use new tech

– When it comes to billing code, try to deal with it early on (it’s tricky)

– It’s hard to test webhooks and other external interactions in an automated way

– You can find Todd on Twitter @toddhgardner and check out his new monitoring service at https://requestmetrics.com