DEV Community: Armando C. Santisbon

Cloud native homelab, Part 2: Kubernetes + Ceph

Armando C. Santisbon — Wed, 25 Oct 2023 01:57:08 +0000

In Part 1 of this series we went over the hardware you'll need to get started on your cloud native homelab, where hopefully I've convinced you to choose Raspberry Pi boards and a cooling solution.

For the best performance we'll use SSDs for the boot partition as well as the storage cluster. This will give us a huge advantage over microSD cards in both speed and reliability.

In this example we'll use:

2 Raspberry Pi 4B boards. I used an 8GB model for the control plane node and a 4GB model for a worker node.
2 SATA SSDs with USB adapters (ASMedia chipset).
1 microSD card to flash the USB bootloader to the Pis.
Cluster case with fans.

Here's what my homelab looks like. It also has a Raspberry Pi 2 that serves as a retro gaming console but for this post we'll be focusing on the Kubernetes cluster.

Our cluster will have a lightweight distribution of both Kubernetes and Ceph, a unified storage service with block, file, and object interfaces.

I've documented how to set up a Raspberry Pi and SSD in an automated, cloud native way in my notes here:

Raspberry Pi

Follow those instructions to:

Set up your Pi to boot from SSD.
Get the OS image (Ubuntu for Raspberry Pi).
Following the instructions on the setup section, flash the image on the SSD.
Verify that the automatic setup worked correctly. Do not continue without first fixing any errors found.

Repeat the process for each Raspberry Pi you have. Once you're done you'll have your Pi devices with MicroK8s installed, all software updated, and relevant kernel modules added.

Now it's time to configure our MicroK8s and MicroCeph cluster. I documented the process in my notes here:

MicroK8s

Follow those instructions to set up:

MicroCeph cluster.
MicroK8s cluster.
Connect MicroK8s to the external MicroCeph cluster.
(Optional) MicroK8s dashboard.

🎉 You're done! 🍾

Here's what a Ceph cluster looks like with 2 nodes and 3 (virtual) disks on each node/SSD:

Come back soon for part 3 where we'll deploy some workloads to our cluster.

Finding the right macOS `defaults` command

Armando C. Santisbon — Sun, 01 Oct 2023 01:40:25 +0000

Automating the setup of a clean macOS install is a lot easier when you know which keys to edit via the defaults command. Since there's no official listing of all the keys, there are places online that give you the most commonly used ones and try to stay up to date with the names of the keys in the latest release of macOS, with varying degrees of success. The best way is to grab the correct names directly from your Mac.

Note: First, make sure your terminal has Full Disk Access in System Settings > Privacy and Security.

The format to edit your macOS preferences is
defaults write $DOMAIN $KEY -$TYPE $VALUE

And you can list all the domains with
defaults domains | tr ', ' '\n'

You can figure out the correct domain and key names by comparing the output of defaults read before and after manually changing a setting in the UI. You can also specify a domain to narrow down the output but it's not required so don't worry if you're not sure which domain is affected by the setting you're interested in. Remember to kill any affected apps to apply the changes.

Example:

defaults read com.apple.finder > before.txt
# Change Finder Settings in the UI
# to clear "Show warning before changing an extension" 
defaults read com.apple.finder > after.txt
diff before.txt after.txt
# <     FXEnableExtensionChangeWarning = 1;
# ---
# >     FXEnableExtensionChangeWarning = 0;

Now check the type and you got a command you can add to any automation you wish.

defaults read-type com.apple.finder FXEnableExtensionChangeWarning
# Type is boolean
defaults write com.apple.finder FXEnableExtensionChangeWarning -bool false
killall Finder 2> /dev/null

If you can't find a key where you expected it, remove the domain from your query or check the global domain. This one affects Finder but it's in the global domain instead of com.apple.finder.

defaults read "Apple Global Domain" AppleShowAllExtensions
# 1
defaults read-type "Apple Global Domain" AppleShowAllExtensions
# Type is boolean

Other commands to try:

defaults help
defaults find $WORD
defaults read com.apple.dock | grep wvous # Hot Corners

Want an example of a setup script? Check out mine.

Cloud native homelab, Part 1: Hardware

Armando C. Santisbon — Sun, 10 Sep 2023 03:29:43 +0000

In this first installment of the series on building a cloud native homelab, we go over selecting your hardware.

The emphasis here is on cost optimization so we're not getting a cluster board, a bunch of compute modules (general purpose or machine learning), servers, a rack mount, or anything like that. We'll start out with one or two single-board computers (SBCs) that we can set up easily without a big capital expense. This will allow us to:

Deploy new nodes to our homelab in an automated fashion, just like on a public cloud with cloud-init.
Set up a Kubernetes cluster and deploy workloads to it using cloud native technologies for container orchestration, storage, and more.
Interact with the real world through sensors, cameras, motors, and other hardware if we choose to.

The most popular SBC is Raspberry Pi but finding one during a shortage may be hard. Fortunately at the time of this writing the shortage is over and you can now easily find one at a reasonable price. There are cheaper options, though, and you might be tempted to buy something like the Orange Pi. Let's go over the differences to show exactly what you're getting with each.

TL;DR Get the Raspberry Pi if you want a cloud native homelab

	Raspberry Pi 4B	Orange Pi 3B
Origin	United Kingdom	China
RAM	8GB LPDDR4-3200 SDRAM	8GB LPDDR4/4x
Storage	Micro-SD card slot	Micro SD slot. Supports an eMMC module: 16GB - 256GB. SPI Flash: 16MB/32MB. M.2 M-KEY slot for SATA3 or PCIe 2.0 NVMe SSD.
Price (USD)	$75 + shipping	$45 + shipping
OS	Raspberry Pi OS, Ubuntu, RetroPie, Recalbox, OctoPi, Home Assistant, others	Orange Pi OS [1], Ubuntu, Debian, Android, others
Official images from Ubuntu and Debian?	Yes	No. Custom Ubuntu and Debian images provided by Orange Pi.
cloud-init included?	Yes (at least on Ubuntu)	No
Default Ubuntu package repo	ubuntu.com	huaweicloud.com
Linux kernel release [4]	5.15.0-1037-raspi (kernel extra modules available)	5.10.160-rockchip-rk356x (kernel extra modules not available)
CPU	Broadcom BCM2711 Quad core Cortex-A72 (ARM v8) 64-bit SoC 1.8GHz	Rockchip RK3566 Quad-core Cortex-A55 (ARM v8) 64-bit SoC 1.8GHz
Graphics Processing Unit (GPU)	Integrated	ARM Mali G52 2EE (Bitfrost) [2]
Neural Processing Unit (NPU)	-	Rockchip NPU AI accelerator with 0.8Tops@INT8 performance. RKNN model type supports conversion of Caffe / TensorFlow / TFLite / ONNX / PyTorch / Keras / Darknet models
Vision Processing Unit (VPU)	-	Yes
Graphics & Video	OpenGL ES 3.1, Vulkan 1.0. H.265 (4K 60fps decode). H.264 (1080p 60fps decode, 1080p 30fps encode).	GPU OpenGL ES 3.2/2.0/1.1, Vulkan 1.1, OpenCL 2.0. Embedded high performance 2D acceleration hardware. VPU H.265/H.264/VP9 (4K 60fps decode). H.265 (1080p 100fps encode). H.264 (1080p 60fps encode).
Ethernet	Gigabit	Gigabit
Wireless	Wi-Fi 5 (802.11ac) Bluetooth 5.0, BLE	Wi-Fi 5 (802.11ac) Bluetooth 5.0, BLE Wired antenna (not shown). Useful when building into a metal housing.
USB	2 USB 3.0 ports 2 USB 2.0 ports	1 USB 3.0 HOST 2 USB 2.0 HOST 1 USB 2.0 Device or HOST mode
General Purpose Input/Output (GPIO)	40 pin GPIO header	40 pin header supporting interface types: GPIO, UART, I2C, SPI, PWM
Video output	2 × micro-HDMI up to 4K 60fps. 1 × 2-lane MIPI DSI display port.	1 × HDMI TX 2.0 up to 4K 60fps. 1 × 2-lane MIPI DSI display port. eDP1.3
Camera	1 × 2-lane MIPI CSI camera port	1 × 2-lane MIPI CSI camera port
Audio	4-pole stereo audio and composite video port	3.5mm headphone jack audio in/out
Power	5V DC via USB-C (minimum 3A) [3]. 5V DC via GPIO header (minimum 3A) [3] Power over Ethernet (PoE) enabled (requires separate PoE HAT).	5V DC via USB-C (3A)
Button	-	1 MaskROM key 1 RESET key 1 POWER key
Fan	Use the GPIO pins (3.3V or 5V) 2.54mm pin pitch	2-Pin 1.25mm pin pitch 5V fan connector
Real-time clock (RTC)	Use the GPIO pins (3.3V) 2.54mm pin pitch	2-Pin 1.25mm pin pitch battery backup connector
Dimensions	85mm x 56mm x 11mm 46g	85mm x 56mm x 17mm 49g

[1] Orange Pi OS doesn't support the GPU, NPU, or VPU. But Debian, Ubuntu, and Android support them.

[2] arm.com, Rockchip GitHub

[3] A good quality 2.5A power supply can be used if downstream USB peripherals consume less than 500mA in total.

[4] Linux kernel extra modules `nvme_fabrics` and `nvme_tcp` are required for cloud native storage like OpenEBS.

You'll also need a 5V 3A power supply for each board if you're not setting them up with Power over Ethernet (PoE).
You don't need a mouse and keyboard for the boards because you'll manage them over SSH from your laptop. Even if you installed a graphical desktop, you could still access it remotely with a VNC viewer.
If you're not setting them up with Wi-Fi, you'll need an ethernet cable (CAT 5e or better) for each board.
For many applications you can use the Pi by itself without heat sinks or a fan and monitor the temperature in millidegree Celsius with cat /sys/class/thermal/thermal_zone*/temp. But we're going to be running some resource-intensive software like Kubernetes. For our cloud native homelab I recommend getting heat sinks, a fan, and a case to house everything from the start. The Raspberry Pi ARM cores and GPU will start throttling when they reach around 80°C.
Speaking of cases, I recommend getting a cluster case. It's way cheaper than a cluster board and it houses your Pis in a stackable way so it's great for expanding your homelab in the future. This one from the UK is particularly nice and made with Perspex (a high-quality acrylic) but there are other options on Amazon with faster shipping.
For storing large amounts of data you'll need external storage like an SSD drive. I recommend a SATA SSD although if you already have an NVMe SSD you can use that as well, just know that your board won't take full advantage of its speeds. This one is a great option along with this cable. Just make sure the cable/adapter has an ASMedia chipset.

With Raspberry Pi it's easy to automate the setup with cloud-init. You're also able to have cloud native replicated storage with OpenEBS thanks to the extra kernel modules for NVMe over Fabrics. If you want your homelab to be cloud native, Raspberry Pi is the way to go.

For Orange Pi, cloud-init is not included. On top of that, the images provided by Orange Pi can't be mounted on macOS so you can't inject the user-data anyway (and you have to use the images they provide, others won't work). That means you'll have to set it up manually. There's also the issue of not being able to use cloud native storage due to a lack of extra modules for that kernel release. Having said that, Orange Pi's AI capabilities might be a good addition to your homelab if you need a low-cost AI-accelerated IoT device and don't need any cloud native features like an automated setup or replicated storage.

If you have a 32-bit SBC like the Raspberry Pi 2, keep in mind that it can't run some Kubernetes distributions like MicroK8s so whatever SBCs you choose make sure they're 64-bit. If you have a Pi 2 it's great as a dedicated retro gaming console with Recalbox.

In the next part of this series, we'll set up our nodes as a Kubernetes cluster.

A Mastodon bot to find EV charging stations

Armando C. Santisbon — Thu, 31 Aug 2023 21:38:05 +0000

I have a new addition to my homelab. I wanted to do a project that was fun to build but also let me apply a few different technical concepts. Read along if you're interested in any of the following:

Homelabs based on Raspberry Pi.
Mastodon and the wider Fediverse.
Civic tech.
Electric vehicles.
Loosely-coupled microservices.
Message brokers with long-running producers and consumers.
Caching.
Container orchestration.
Secrets management.
Handling server-sent events from a streaming API.
Consuming REST APIs.

The goal was simple: a bot that, whenever you mention it with a location, will reply with a list of nearby EV charging stations. It must also display the type of connectors available and link to the location of each station using Apple or Google Maps, depending on the user's device.

Once you've set up your Raspberry Pi and lightweight Kubernetes distribution, it's time to think about our architecture.

Our solution has:

A watcher that keeps track of Mastodon notifications for mentions of our bot account and prepares a request for the task to be done.
A worker that fulfills the task of putting together the response and sending it to the user.
A message broker that the watcher and the worker will use to communicate with each other.
A cache to store the responses from the external service that helps us translate user requests into a geographic location.
A Mastodon instance hosting our bot account and offering a streaming API with notifications and a web API for posting.
A third-party API with a geocoding service.
A third-party API with charging station locations.

Here's the high-level architecture

We'll take advantage of Docker container images to package our two microservices (watcher and worker) and their dependencies. We'll also use Kubernetes for container orchestration.

Each microservice consists of:

App container.
Configuration and secrets.
Horizontal autoscaling to meet an increase in demand.
Init container to delay the creation of the app container until the message broker is available.
In the case of the worker, a cache to avoid unnecessary calls to the geocoding API as required by its terms of service.

Dive into the instructions, code, a detailed diagram of the Kubernetes objects, and documentation here:

santisbon / evstations

Mastodon bot to get electric vehicle charging stations.

EV Stations

🧪 Experimental

@ev@botsin.space is a Mastodon bot to get electric vehicle (EV) charging stations near a location in the United States or Canada.

Highlights

Mention it with just a requested location and it will reply with some charging stations near it.
The links provided in the results (even though they point to Apple Maps) will automatically open either Apple or Google Maps depending on your device.
If the response doesn't fit in one post due to Mastodon's character limit it will be split into a couple of posts.
The response includes symbols for the available connector types at each station
- Ⓣ Tesla aka North American Charging Standard (NACS)
- Ⓙ J1772
- ⒸⒸⓈ Combined Charging System (CCS) aka J1772 Combo
- Ⓝ⑭-㊿ NEMA 14-50
- Ⓝ⑤-⑮ NEMA 5-15
- Ⓝ⑤-⑳ NEMA 5-20
- ⒸⒽⒶ CHAdeMO

Usage

Post this from your Fediverse account:

@ev@botsin.space Brickell, Miami

Query examples:

60601
Wrigley Field
chinatown, chicago
financial…

View on GitHub

Monitor internet speeds with Raspberry Pi

Armando C. Santisbon — Sun, 13 Aug 2023 20:55:13 +0000

A great addition to any homelab is being able to track your internet speeds over time and have a nice dashboard to visualize the data for download/upload speeds, latency, and jitter.

We'll use the speedtest.net API for the test itself, a time series database to store our test results, and a dashboard to visualize them from other machines on the network.

To make the deployment easier we'll take advantage of Docker containers to package our apps, Kubernetes for orchestration, and Helm to deploy everything.

You'll need a Raspberry Pi or some other low-cost, low-energy consumption computer you'd like to leave on to do the speed monitoring. You won't need a graphical desktop on it since you can ssh into it to carry out all tasks.

Let's get started! All the details are here:

santisbon / speedtest

Keep track of your internet connection speeds over time with a Raspberry Pi.

Internet Speed Monitor

Keep track of your internet connection speeds over time with a Raspberry Pi.

This Helm chart deploys:

InfluxDB 2.x database for time series data.
Grafana dashboard.
Python app to run the speed test on a schedule.

Prerequisites

Set up your Raspberry Pi.
Install and configure MicroK8s (or another lightweight Kubernetes distribution with DNS and Helm addons) on your Pi.

Note: MicroK8s by default uses Dqlite as its storage backend instead of etcd. Further securing of Secret objects with encryption at rest for either storage backend is outside the scope of this project.

Install

Prepare your Raspberry Pi's local storage for both the database and dashboard

# On your Pi
sudo mkdir -p /var/lib/influxdb2
sudo mkdir -p /etc/influxdb2
sudo mkdir -p /var/lib/grafana

If installing from the repository

# On your Pi
helm repo add santisbon https://santisbon.github.io/charts/
# or 
helm repo update
CHART="santisbon/speedtest"

Or if…

View on GitHub

Homelab or cloud? Where should your apps and data go?

Armando C. Santisbon — Fri, 11 Aug 2023 06:54:59 +0000

You may have a lot of personal digital assets: web applications, a blog, productivity and collaboration software, email, calendars, photos, video, documents, music, social networking bots or servers, monitoring and security tools, game servers, content management systems, home automation software. All of these have to live somewhere and sometimes it's not easy to decide what the best place is for each.

The suitability of your homelab for a particular workload will, of course, depend in part on how sophisticated your homelab is. But there are a few rules of thumb that can help you decide.

Your workload may be well suited for the public cloud if:

It has extremely high durability and availability requirements e.g. storage with 99.999999999% durability and 99.99% availability of objects over a given year.
It exposes services to the outside world and demand is unpredictable so elasticity is important.
It needs specialized hardware like GPU-based instances or FPGAs.
You want someone else to take care of administrative overhead, power, cooling, physical security, and other undifferentiated heavy lifting.

Your workload may be best on your homelab if:

It only provides services to your home network and doesn't need to expose services to the outside.
Its demand is predictable.
In the event that your house burns down or is flooded, the loss of apps and data is not your biggest concern.
It doesn't need specialized hardware.
You want local control of your data and the infrastructure that hosts it.
You want to learn more about clusters, networking, containers, or orchestration with a hands-on approach.

Some apps that are great for a homelab include a network-wide ad blocker like Pi-hole, a Mastodon bot, an internet speed monitor, open source home automation like Home Assistant, a media server like Plex, a private cloud for your family like Nextcloud, and pet projects/experiments for learning new technologies.

Set up a new dev machine

Armando C. Santisbon — Wed, 09 Aug 2023 20:01:28 +0000

I keep my notes on setting up a new development machine as a MkDocs site on Github Pages. You can find all the material here:

santisbon.github.io

The Sleeping Beauty Problem

Armando C. Santisbon — Mon, 17 Jul 2023 17:20:12 +0000

Sleeping Beauty agrees to be the subject of an experiment and is informed of the procedure:

On Sunday night, she will be put to sleep.
Then, a fair coin will be flipped.
If it comes up Heads, she will be awakened on Monday and put back to sleep.
If it comes up Tails, she will also be awakened on Monday and put back to sleep, but then she'll be awakened on Tuesday as well and put back to sleep.
Each time that she is put back to sleep she will forget that she was ever awakened.
Any time she's awake, she will be told no information but she'll be asked one question: What do you believe is the probability that the coin came up heads?

Image credit: Veritasium

How should she answer?

You might be tempted to say $50%50\%$ because that's the probability of a fair coin coming up Heads, but you would be mistaken. The correct answer is $1/3$ . To understand why, we need to realize that the question that is really being asked is this: What is the probability that the coin came up Heads given that you've been awakened?

This is a case where we need to incorporate observed data into our prior odds to come up with new odds that account for the new data. To do that, we need to calculate the probability of an event happening given that another event has already happened. We can use Bayes' Theorem for this.

ℹ️ Bayes' Theorem

$\over P(B)}$

But wait, where did that come from?

💡 It can be derived from the conditional probability that tells us in how many cases do both events $A$ and $B$ happen out of the cases where $B$ happens:

$\cap B) \over P(B)}$

Similarly, $\cap B) \over P(A)}$

which means

$\cap B)=P(B|A)P(A)$

and substituting in the expression for conditional probability yields Bayes' Theorem:

$\over P(B)}$

Now that we understand where Bayes' Theorem comes from, there's another way to look at it: As a way to understand two competing hypotheses $H_1$ and $H_2$ . We start by having some prior belief about the odds of those two. After observing some data, what are the new odds of the two hypotheses given the new data? In other words, how do we update our belief based on new evidence?

ℹ️
Let's start by specifying what we mean by odds. We mean how large is one probability relative to the other. Let's put it as a ratio. For example, if prior to seeing any data one hypothesis has $3/4$ probability of being true and the other has $1/4$ :
$P(H1)P(H2)=3/41/4=31\frac {P(H_1)} {P(H_2)}=\frac {3/4} {1/4}=\frac 3 1$
Meaning the odds of $H_1$ to $H_2$ are $3$ to $1$ .

So how do we incorporate the new data $D$ into our belief?

💡 By calculating the new probability of each hypothesis given the new data.
$P(H1∣D)P(H2∣D)=P(H1∩D)P(D)P(H2∩D)P(D)=P(H1∩D)P(H2∩D)\frac {P(H_1|D)} {P(H_2|D)}=\frac {\frac {P(H_1 \cap D)} {P(D)}} {\frac {P(H_2 \cap D)} {P(D)}}=\frac {P(H_1 \cap D)} {P(H_2 \cap D)}$
Now we'll use a mathematical trick of multiplying and dividing by the same number to get the same value but expressed in terms of the probability of seeing the observed data given each hypothesis.
$=P(H1∩D)P(H2∩D)⋅P(H2)P(H1)⋅P(H1)P(H2)=\frac {P(H_1 \cap D)} {P(H_2 \cap D)} \cdot \frac {P(H_2)} {P(H_1)} \cdot \frac {P(H_1)} {P(H_2)}$ $=P(H1∩D)P(H1)P(H2∩D)P(H2)⋅P(H1)P(H2)=\frac {\frac {P(H_1 \cap D)} {P(H_1)}} {\frac {P(H_2 \cap D)} {P(H_2)}} \cdot \frac {P(H_1)} {P(H_2)}$
So,

$P(H1∣D)P(H2∣D)=P(D∣H1)P(D∣H2)⋅P(H1)P(H2)\frac {P(H_1|D)} {P(H_2|D)}=\frac {P(D|H_1)} {P(D|H_2)} \cdot \frac {P(H_1)} {P(H_2)}$
And so we have that as a ratio, the posterior odds equal the prior odds times the probabilities of generating the new data.

Now let's apply this to our Sleeping Beauty problem. Let
$H_1=$ The coin came up Heads
$H_2=$ The coin came up Tails
$D =$ You've been woken up

P(H1∣D)P(H2∣D)=1/32/3⋅1/21/2=3/6=1/2\frac {P(H_1|D)} {P(H_2|D)}=\frac {1/3} {2/3} \cdot \frac {1/2} {1/2}=3/6=1/2

Meaning $H_2$ is twice as likely as $H_1$ given the new data. If you want to convert that to probabilities:

P(H1)=11+2=1/3P(H_1)=\frac 1 {1+2}=1/3

P(H2)=21+2=2/3P(H_2)=\frac 2 {1+2}=2/3

Therefore Sleeping Beauty should answer that the coin came up Tails as the probabilities are $1/3$ for Heads and $2/3$ for Tails.

We can give another explanation that is more informal but more intuitive. Imagine she's still woken up once if it's Heads, but on $100$ consecutive days if it's Tails. She's awake. She has no idea which of those $101$ scenarios she's in, but effectively she's being asked if she thinks she's in the $1$ scenario where it was Heads or in one of the $100$ scenarios where it was Tails. The answer should be clear.

Document your project with MkDocs and GitHub Pages

Armando C. Santisbon — Fri, 17 Mar 2023 22:13:38 +0000

Most project documentation starts with a single README.md file. But for medium and large projects this quickly becomes a really long and hard to manage file. Searching for the content you want, navigation, and styling become a mess.

Enter MkDocs. It can generate a static site that can be hosted on any web server. We'll set it up on GitHub Pages with a configuration that not only makes it look great but allows rich content like math expressions and simplifies deployment.

First install the needed software and create the file structure we'll use.

pip install mkdocs python-markdown-math mkdocs-material
mkdocs new [dir-name]

This will create this default layout

mkdocs.yml    # The configuration file.
docs/
    index.md  # The documentation homepage.
    ...       # Other markdown pages, images and other files.

GitHub Pages gives you the option to host your static site on any branch in either the root of your project or in a /docs folder so we'll modify the default layout to make our lives easier. Rename /docs to /src/:

mv ./docs/ ./src/

and move the whole dir with mkdocs.yaml and /src/ to your project's directory.

Now edit your mkdocs.yml file to look like this. Replace the placeholders with your GitHub username and repo.

site_name: My Site
site_url: https://<user>.github.io/<repo>/
repo_url: https://github.com/<user>/<repo>
nav:
  - Home: 'index.md'
markdown_extensions:
  - toc:
    permalink: "#"
theme:
  name: material
  features:
    - navigation.expand
    - navigation.top
    - content.code.copy
  palette:
    # Palette toggle for light mode
    - media: "(prefers-color-scheme: light)"
      scheme: default
      toggle:
        icon: material/brightness-5
        name: Switch to dark mode
    # Palette toggle for dark mode
    - media: "(prefers-color-scheme: dark)"
      scheme: slate
      toggle:
        icon: material/brightness-3
        name: Switch to light mode
extra_javascript: 
  - https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML
markdown_extensions:
  - mdx_math
  - admonition
  - sane_lists
  - pymdownx.highlight:
      auto_title: false
  - pymdownx.superfences
  - pymdownx.keys
docs_dir: src
edit_uri: edit/main/src/
site_dir: docs

Notice we set docs_dir: src to hold our .md files and site_dir: docs to automatically place the generated site in the folder that GitHub uses to serve our GitHub Pages site.

We're also using the Material theme which is clean, elegant, and has great customization options.

A few extensions we added

Admonition

Pencil: attention, caution, error, hint, important,note.
Fire: tip.
Exclamation mark: warning.
Lightning bolt: danger.

The text must be indented.

!!! attention
    Text goes here.

Keys

++command+shift+n++

MathJax

When \(a \ne 0\), there are two solutions to \(ax^2 + bx + c = 0\) and they are
$$x = {-b \pm \sqrt{b^2-4ac} \over 2a}$$

Now generate the site. You can use the live-reloading site during development and when you're ready to deploy, build the production site.

mkdocs serve - Start the live-reloading docs server.
mkdocs build - Build the documentation site.

Almost ready!

Enable GitHub Pages for your project. Make sure you set it to use the main branch and the /docs folder.
Commit your changes and push them to GitHub.

This will kick off a GitHub Action to serve your site. After about a minute it will be available at username.github.io/repo.

Go ahead and create stunning technical documentation for your project!

Block ads with Pi-hole and Kubernetes

Armando C. Santisbon — Tue, 14 Mar 2023 21:49:35 +0000

While there are browser extensions and apps you can use to block ads on your desktop and mobile, they require installing them individually on each of your devices. And that's assuming there's an app available for the device and OS in question.

Wouldn't it be better to also block ads at the router level? All those smart TVs, set-top boxes, game consoles, VR headsets, smart speakers, smart thermostats, robot vacuums, security cameras, doorbells, and other IoT devices love chatting with metrics/ad services. Let's put a stop to that by deploying a DNS server that will block requests to known ad servers.

Raspberry Pi is a tiny computer you can use as a desktop, smart home hub, robot brains, media center, and more. If you have been thinking about setting up a homelab, a Raspberry Pi is a great place to start at an extremely low cost. They are in high demand but this site can help you find one.

MicroK8s is a lightweight distribution of Kubernetes (K8s) for container orchestration. We'll also use Helm, a package manager for K8s, for easy deployment.

Pi-hole is a DNS sinkhole that protects your devices from unwanted content, without installing any client-side software. It's installed in one place and blocks ads for your entire home network.

When you're done you'll have an admin interface for all your K8s workloads:

And another for your Pi-hole stats:

Let's get started! All the details are here:

santisbon / pi-hole-k8s

Deploy Pi-hole to a Kubernetes cluster on a Raspberry Pi.

Network-wide ad blocking with Pi-hole, Kubernetes, and Raspberry Pi

Prerequisites

Set up your Raspberry Pi with a static IP.
Install and configure MicroK8s (or any other lightweight Kubernetes distribution) on your Pi.

Install

Prepare your Pi's storage

# On your Pi
sudo mkdir -p /etc/pihole
sudo mkdir -p /etc/dnsmasq.d

If installing from source, grab the code

# On your Pi
CHART="./piholechart"
git clone https://github.com/santisbon/pi-hole-k8s.git && cd pi-hole-k8s
nano $CHART/values.yaml
# edit the values

Or if installing from the repository

# On your Pi
helm repo add santisbon https://santisbon.github.io/charts/
CHART="santisbon/pihole"

Install the Helm chart which will enforce the installation order. Replace parameters with a secure password and the static IP of your Pi if you didn't do it through the values.yaml file. If using MicroK8s type the commands as microk8s helm and microk8s kubectl
```
# On your Pi
RELEASE=pihole
NAMESPACE=pihole-n
helm install 
```
…

View on GitHub

Service Worker lifecycle

Armando C. Santisbon — Sun, 05 Mar 2023 22:13:04 +0000

Getting started with Progressive Web Apps (PWAs)? Here's a succinct explanation of the service worker (SW) lifecycle and update process.

SW lifecycle:

The SW is registered with

 if ('serviceWorker' in navigator) {
    navigator.serviceWorker.register("/serviceworker.js");
 }

The browser attempts to download and parse the SW file as JavaScript.
If it succeeds the install event is fired (it only fires once). The SW is now installed but not yet in control of its clients like the PWA.
By default, the SW will not take control until the next time you navigate to the page by reloading it or re-opening the PWA.

Updating a SW

You can find all the details here and can be summarized like this:

Browser detects that the SW currently controlling the client and the new (from the server) version of the same file are byte-different.
After the new SW is installed it will wait to activate until the old SW no longer controls any clients. This way only 1 version of the SW runs at a time. Refreshing the page or reopening the PWA won't make the new SW take control.
The user closes or navigates away from all tabs and windows using the curernt SW and then navigates back. The new SW takes control.

Serving a React Router app through CloudFront

Armando C. Santisbon — Fri, 03 Mar 2023 05:44:12 +0000

Single Page Apps (SPAs) are great. They can be hosted as a static site on object storage like an S3 bucket and served through a CDN like CloudFront to cache the content close to your users wherever they are in the world. All without having to manage server infrastructure.

There are however, things to keep in mind to make this setup work. The different frameworks these apps are built on have their own way of dealing with URLs. For example, React Router can intercept the request and handle it instead of sending it to the server.

This client-side routing means that React Router knows how to handle a request for www.example.com/products/12 while clearly there's no such resource as <bucket-name>/products/12 on the server. This all works fine as long as navigation happens within your app where React can intercept those requests. The only request the server should be handling is for www.example.com or any other resource that actually lives on the server.

But what happens when you type www.example.com/products/12 in your browser's address bar and hit enter? That request is going to the server where it will make no sense because there's no resource by that URI. You might get something like this in your browser's console:

Refused to apply style from 'https://www.example.com/assets/index-12345abc.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.

This and other errors have the same underlying cause: the resource your browser asked for is just not there. Depending on your setup, the server might be replying with the default document or something else.

We need to run some code at the edge locations that can intercept the request, rewrite the URI, and hand it over to CloudFront for further processing. If the requested resource is something the client should handle, set the URI to / so the server will respond as expected and let the client figure out how to handle the request. If the requested URI is for a server resource like /img/cat.jpg then don't change it. Fortunately, CloudFront Functions let us do precisely that. You write them in Javascript, with some restrictions. Here [1] is an example showing you how to create and test a function that looks like this:

function handler(event) {
  var request = event.request;
  var uri = request.uri;
  var paths = ['assets', 'img', 'robots.txt', 'app.webmanifest']
  var isServerPath = (path) => uri.includes(path);

  if (!paths.some(isServerPath)) {
    request.uri = '/';
  }

  return request;
}

In this example, assets/ is the folder where your bundler e.g. Vite places your app's Javascript and CSS files for production.

Once you have your CloudFront function working, it's time to associate it with your CloudFront distribution. This [2] will show you how to deploy your SPA as a secure static site with a CloudFront function handling requests as needed by React Router.

[1] https://github.com/santisbon/amazon-cloudfront-functions/tree/main/url-rewrite-single-page-apps

[2] https://github.com/santisbon/static-site

DEV Community: Armando C. Santisbon

Cloud native homelab, Part 2: Kubernetes + Ceph

Finding the right macOS `defaults` command

Cloud native homelab, Part 1: Hardware

[1] Orange Pi OS doesn't support the GPU, NPU, or VPU. But Debian, Ubuntu, and Android support them.

[2] arm.com, Rockchip GitHub

[3] A good quality 2.5A power supply can be used if downstream USB peripherals consume less than 500mA in total.

[4] Linux kernel extra modules nvme_fabrics and nvme_tcp are required for cloud native storage like OpenEBS.

A Mastodon bot to find EV charging stations

santisbon / evstations

Mastodon bot to get electric vehicle charging stations.

Monitor internet speeds with Raspberry Pi

santisbon / speedtest

Keep track of your internet connection speeds over time with a Raspberry Pi.

Homelab or cloud? Where should your apps and data go?

Set up a new dev machine

The Sleeping Beauty Problem

Image credit: Veritasium

Document your project with MkDocs and GitHub Pages

MathJax

Block ads with Pi-hole and Kubernetes

santisbon / pi-hole-k8s

Deploy Pi-hole to a Kubernetes cluster on a Raspberry Pi.

Service Worker lifecycle

SW lifecycle:

Updating a SW

Serving a React Router app through CloudFront

[4] Linux kernel extra modules `nvme_fabrics` and `nvme_tcp` are required for cloud native storage like OpenEBS.