Cribl Edge & Search Homelab Part 4 - Search

Now that Minio is internet accessible, let’s configure Cribl Search to read that data. In Search, there’s two concepts that are relevant to our configuration: Datasets and Dataset Providers. We will configure a Dataset Provider to access our Minio endpoint on the internet, and a Dataset to query our particular bucket in Minio.

  1. Go to Cribl Search
    • From Edge or Search, select the product hamburger at the top and select Search
    • From the Cloud main navigation, click Search
  2. Click Data
  3. Click Dataset Providers at the left
  4. Click the New Dataset Provider button
  5. Under Dataset Provider Type, click Cribl Edge then select Amazon S3
  6. Click Advanced Settings to expand more options
  7. Fill out the form like this:

  1. Add an ID
  2. Set Access Key and Secret Key to values you setup in Part 1
  3. Set Endpoint to the https:// URL you got in Part 3
  4. Click Save

You now have a Dataset Provider which is pointing to your Minio. Next, we need to configure a Dataset which uses that provider.

  1. Click Data
  2. Click New Dataset at the right
  3. For ID enter something that matters to you, you’ll be using in search
  4. Click Provider and select the provider you added before
  5. For bucket, enter data/${_time:%Y}/${_time:%m}/${_time:%d}/${_time:%H}/${host}
    • If you named your bucket something other than data make sure to replace that in this string
  6. For region, select US East 1
  7. Your form should look like this:

We now have a dataset which understands the partitioning structure we created in the S3 output. Now we should be able to search our data.

  1. Click Search next to your dataset

You should see a screen that looks like this:

There’s lots we can do to explore the data we’ve been collecting, but that’s for another post. Check out our documentation for more information on how to craft searches and get value out of your data.

Hope this was helpful and fun! More content coming as I work to explore the data being generated out of my home network.

Posts in this Series