cluster: deploy NixOS-based ceph
First pass at a non-rook-managed Ceph cluster. We call it k0 instead of
ceph-waw4, as we pretty much are sure now that we will always have a
one-kube-cluster-to-one-ceph-cluster correspondence, with different Ceph
pools for different media kinds (if at all).
For now this has one mon and spinning rust OSDs. This can be iterated on
to make it less terrible with time.
See b/6 for more details.
Change-Id: Ie502a232c700af93f33fcad9fa1c57058161aa11
diff --git a/ops/machines.nix b/ops/machines.nix
index 5401e30..9a54c56 100644
--- a/ops/machines.nix
+++ b/ops/machines.nix
@@ -19,7 +19,28 @@
repo = "nixpkgs-channels";
rev = "44ad80ab1036c5cc83ada4bfa451dac9939f2a10";
sha256 = "1b61nzvy0d46cspy07szkc0rggacxiqg9v1py27pkqpj7rvawfsk";
- }) {};
+ }) {
+ overlays = [
+ (self: super: rec {
+ # Use a newer version of Ceph (16, Pacific, EOL 2023-06-01) than in
+ # this nixpkgs (15, Octopus, EOL 2022-06-01).
+ #
+ # This is to:
+ # 1. Fix a bug in which ceph-volume lvm create fails due to a rocksdb
+ # mismatch (https://tracker.ceph.com/issues/49815)
+ # 2. At the time of deployment not start out with an ancient version
+ # of Ceph.
+ #
+ # Once we unpin nixpkgsCluster past a version that contains this Ceph,
+ # this can be unoverlayed.
+ inherit (super.callPackages ./ceph {
+ boost = super.boost17x.override { enablePython = true; python = super.python3; };
+ lua = super.lua5_4;
+ }) ceph ceph-client;
+ ceph-lib = ceph.lib;
+ })
+ ];
+ };
# edge01 still lives on an old nixpkgs checkout.
#
@@ -44,6 +65,7 @@
imports = [
../cluster/nix/modules/base.nix
../cluster/nix/modules/kubernetes.nix
+ ../cluster/nix/modules/ceph.nix
];
})];