– Over 7 billion log events are being handled per month

– What are CloudFlare apps?

– In the future native language libraries will have more support

– Currently there’s support in Elixir for the Logflare library

– Elixir and Phoenix is powering the web front-end for Logflare

– Motivation for choosing Elixir and Phoenix

– Phoenix allowed Chase to get things up and running in a few months

– The web UI is using server side templates with a touch of Live View

– Live View is mostly used for counters but it powers a search page too

– A monolithic / mono-repo Phoenix app ingests all of the logs

– Phoenix contexts are being used to break up the domain a bit

– Docker isn’t being used in dev, but it is in production on Google Cloud

– Google’s managed instance groups are being used to host the app

– These managed instance groups help with doing rolling deploys

– Motivation for choosing GCP came down to free hosting credits mostly

– (6) 16 CPU core / 32 GB of memory instances power the Phoenix app

– Each instance has an identical copy of the Phoenix app

– Google’s container-optimized OS is being used

– PostgreSQL stores a bit of user data, but the logs are in BigQuery

– BigQuery does not work with Ecto, but it’s a fairly simple set up

– For development, Chase connects to a dev BigQuery database on GCP

– Cowboy is in front of Google’s load balancer (nginx isn’t being used)

– SSL certificates are issued and signed by CloudFlare

– A step by step walk through of how code goes from development to production

– It takes 5 minutes for each server to get drained, but it’s configurable

– Having manual components of your deploy can be beneficial

– Logflare is very stateful, so a 5 minute time-out is necessary

– Chase uses his own tool and there are public charts to look at online

– Scratching your own itch is a great way to build a useful service

– Sign ups for free right now, but a billing system is coming soon

– Getting $3,000, $17,000 and then $80,000 in GCP hosting credits for free

– A majority of the cost is processing the log events on the web servers

– Server costs could probably be cut down but Chase wanted to learn / experiment

– Initially everything was on a single 32 CPU core server (3 billion reqs / month)

– Rate limiting is cluster-wide and uses Phoenix Tracker

– Using Redis vs rolling your own Elixir based set up for cluster state

– Best tips? Stop over-thinking things because just starting helps a lot

– Chase is on Twitter @chasers and Logflare is open source on GitHub