Cribl Edge & Search Homelab Part 1 - Setup Minio

In my last post, I talked about how I’d be blogging a lot about my lessons from startup life. I’ve obviously failed at that over the last year, but now I’m back to describe how I’m running two of Cribl’s latest products in my homelab. There really is no practical purpose to doing this, as it would be more reliable and easier to forward this data to an actual AWS S3 bucket.

Cribl Edge & Search Homelab Part 2 - Configuring Edge

We now have Minio up and running, it’s time to configure Cribl Edge to collect some data. First, sign up for a Cribl.Cloud account. It’s free for up to 1TB/Day and 10 search executors. Signing up should be pretty straightforward so I’ll not document it here. I sign up with my Google ID, as it’s the simplest way to authenticate for me. Note, you can also run the Cribl leader yourself if you like, also free up to 10TB a day, but you won’t have access to Search as it is Cloud only.

Cribl Edge & Search Homelab Part 3 - Exposing Minio to the Internet

Next, we need to get Minio exposed to the Internet so that Cribl’s search workers, running in AWS Lambda, can access the S3 compatible API exposed by Minio. In my last post, I mentioned Tailscale as a way to bridge the Internet to my Kubernetes cluster. Now, they’ve provided a new alpha level feature which makes it even easier, Tailscale Funnels. Tailscale provides several advantages for my home setup: Universal reachability, like my laptop is on the LAN from anywhere Central DNS for all nodes Central SSH key management for all nodes Tunneling from the Internet, including managing TLS certicates automatically If you haven’t setup Tailscale at all, check out the Quickstart.

Cribl Edge & Search Homelab Part 4 - Search

Now that Minio is internet accessible, let’s configure Cribl Search to read that data. In Search, there’s two concepts that are relevant to our configuration: Datasets and Dataset Providers. We will configure a Dataset Provider to access our Minio endpoint on the internet, and a Dataset to query our particular bucket in Minio. Go to Cribl Search From Edge or Search, select the product hamburger at the top and select Search From the Cloud main navigation, click Search Click Data Click Dataset Providers at the left Click the New Dataset Provider button Under Dataset Provider Type, click Cribl Edge then select Amazon S3 Click Advanced Settings to expand more options Fill out the form like this: Add an ID Set Access Key and Secret Key to values you setup in Part 1 Set Endpoint to the https:// URL you got in Part 3 Click Save You now have a Dataset Provider which is pointing to your Minio.

Self-Hosted Blog Part 1 - Overview

Recently I’ve been thinking of sharing some of my harder won lessons from the last few years of starting a company. There’s a number of topics in my backlog, like what is a basic sales process and how to run it? How do you learn to lead other sales people? How should you think about marketing and marketing team leadership? How do you find the right price for your product? Is your sales play a replacement play or a complementary play?

Self-Hosted Blog Part 2 - The Hardware

In part 1, we established this project with a requirement to run Kubernetes on bare metal. I considered a number of potential options: Get a beefy desktop and run Minikube on a single machine Run a hypervisor and spin up a series of VMs on a beefy machine Buy a small number of not-as-powerful machines, like Raspberry Pis From a requirements perspective, all could easily meet the requirement of running kubernetes on-prem.

Self-Hosted Blog Part 3 - OS & Kubernetes

In part 2, we got our hardware ordered and assembled. Now, we need to turn it into a functioning cluster. Rather than re-invent the wheel, I recommend you follow the Ubuntu tutorial for installing Ubuntu on Raspberry Pis. A few things to note as you go through that tutorial. First, I strongly recommend using Ethernet rather than WiFi, and I didn’t put WiFi in the build of materials. Secondly, I recommend assigning Static IPs to make discovery easier.

Self-Hosted Blog Part 4 - The Blog

In part 3 we got the infrastructure up and running, now it’s time to get a blog created, building, and containerized. First, in the requirements, I had already decided I was going to build a static site to keep things simple. For this tutorial, I looked at a number of options, but you can substitute anything that builds a static site. There’s been a recent revolution in static sites with a number of options for how to build them.

Self-Hosted Blog Part 5 - Container

In part 4 we picked a blog framework and built a static site. Now that we can build our static site, we need to create a container which contains bits for hosting that site. For simplicity’s sake, I decided to build the static site and build the contents of the static site into the container. Then, in Kubernetes, everything we need to host the site is distributed by pulling the Docker image.

Self-Hosted Blog Part 6 - Build

In part 5 we got our static site into a container. Now, we’re going to replicate the build process I did on my local machine in my CI/CD pipeline on every git push. I’m hosting the source for the site in GitHub. For this, I’m going to use GitHub Actions, because it’s free and integrated right into GitHub. GitHub Actions defines workflows in your repository, and then executes those workflows on various triggers.