diff --git a/.gitignore b/.gitignore index a859aa9dd1..0622f1a26b 100644 --- a/.gitignore +++ b/.gitignore @@ -13,5 +13,7 @@ acceptance-tests/syslog-release acceptance-tests/os-conf-release ci/docker/os-image-stemcell-builder/VMware-ovftool-*.bundle +ci/docker/VMware-ovftool-*.bundle +ci/docker/*/VMware-ovftool-*.bundle -tmp/ \ No newline at end of file +tmp/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b2022eece4..42bfe7bc2c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,3 +11,4 @@ an "Ubuntu SHORT_NAME" based stemcell will be on the branch: As of `2026-04-03` the following stemcell lines / branches are supported: - Ubuntu Jammy / `ubuntu-jammy` - Ubuntu Noble / `ubuntu-noble` +- Ubuntu Resolute / `ubuntu-resolute` diff --git a/Gemfile b/Gemfile index 5158c13c83..e566b69695 100644 --- a/Gemfile +++ b/Gemfile @@ -1,4 +1,4 @@ -source 'https://rubygems.org' +source "https://rubygems.org" -gem 'rake' -gem 'bosh-stemcell', path: 'bosh-stemcell' +gem "rake" +gem "bosh-stemcell", path: "bosh-stemcell" diff --git a/README.md b/README.md index c3a385a246..8aa403f740 100644 --- a/README.md +++ b/README.md @@ -3,10 +3,14 @@ This repo contains tools for creating BOSH stemcells. A stemcell is a bootable disk image that is used as a template by a BOSH Director to create VMs. +This branch builds stemcells for **Ubuntu 26.04 LTS (Resolute)**. For other +Ubuntu releases, switch to the appropriate branch (for example `ubuntu-noble` +for 24.04). + ## Quick Start: Building a Stemcell Locally ```bash -export short_name="noble" +export short_name="resolute" git clone git@github.com:cloudfoundry/bosh-linux-stemcell-builder.git cd bosh-linux-stemcell-builder @@ -14,7 +18,15 @@ git checkout ubuntu-${short_name} mkdir -p tmp docker build \ --platform linux/amd64 \ - --build-arg SYFT_VERSION=v1.42.3 \ + --build-arg BASE_IMAGE="ubuntu:${short_name}" \ + --build-arg META4_CLI_URL="https://github.com/dpb587/metalink/releases/download/v0.5.0/meta4-0.5.0-linux-amd64" \ + --build-arg SYFT_CLI_URL="https://github.com/anchore/syft/releases/download/v1.42.3/syft_1.42.3_linux_amd64.tar.gz" \ + --build-arg YQ_CLI_URL="https://github.com/mikefarah/yq/releases/download/v4.52.5/yq_linux_amd64" \ + --build-arg RUBY_INSTALL_URL="https://github.com/postmodern/ruby-install/releases/download/v0.10.2/ruby-install-0.10.2.tar.gz" \ + --build-arg RUBY_VERSION="$(cat .ruby-version)" \ + --build-arg GEM_HOME="/usr/local/bundle" \ + --build-arg OVF_TOOL_INSTALLER="VMware-ovftool-4.4.3-18663434-lin.x86_64.bundle" \ + --build-arg OVF_TOOL_INSTALLER_SHA1="6c24e473be49c961cfc3bb16774b52b48e822991" \ -t bosh/os-image-stemcell-builder:${short_name} \ ci/docker/os-image-stemcell-builder/ docker run \ @@ -32,10 +44,13 @@ gem install bundler bundle install # build OS image -bundle exec rake stemcell:build_os_image[ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz] +bundle exec rake stemcell:build_os_image[ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image_${short_name}.tgz] # build vSphere stemcell -bundle exec rake stemcell:build[vsphere,esxi,ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz] +bundle exec rake stemcell:build[vsphere,esxi,ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image_${short_name}.tgz,9.000] + + # build warden (BOSH Lite) stemcell +bundle exec rake stemcell:build_with_local_os_image[warden,warden,ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image_${short_name}.tgz,9.000] ``` When building a vSphere stemcell, you must download `VMware-ovftool-*.bundle` @@ -56,7 +71,7 @@ installed in the operating system or when making changes to the configuration of those packages. ```bash -export short_name="noble" +export short_name="resolute" bundle exec rake stemcell:build_os_image[ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz] ``` @@ -78,7 +93,7 @@ The arguments to the `stemcell:build_os_image` rake task follow: Rebuild the stemcell when you are making and testing BOSH-specific changes such as a new BOSH agent. ```bash -export short_name="noble" +export short_name="resolute" export build_number="0.0.8" bundle exec rake stemcell:build[vsphere,esxi,ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz,${build_number}] @@ -95,6 +110,7 @@ The arguments to `stemcell:build` are: - `google` → `kvm` - `openstack` → `kvm` - `vsphere` → `esxi` + - `warden` → `warden` 3. `operating_system_name` (`ubuntu`): Type of OS. Same as `stemcell:build_os_image`. 4. `operating_system_version` (``): OS release. Same as @@ -114,7 +130,7 @@ the stemcell would be at upload the stemcell to a vSphere BOSH Director: ```bash -export short_name="noble" +export short_name="resolute" bosh upload-stemcell tmp/bosh-stemcell-0.0.8-vsphere-esxi-ubuntu-${short_name}-go_agent.tgz ``` @@ -130,15 +146,14 @@ the rake task the first time you create your docker container, but everytime after, as long as you do not destroy the container, you should be able to run the specific tests. -To run the `ubuntu_${short_name}_spec.rb` tests (**assuming you've already built +To run the OS image tests in `spec/os_image/ubuntu_spec.rb` (**assuming you've already built the OS image** at the `tmp/ubuntu_base_image.tgz` and you're within the Docker container): ```shell - export short_name="noble" cd /opt/bosh/bosh-stemcell bundle install - OS_IMAGE=/opt/bosh/tmp/ubuntu_base_image.tgz bundle exec rspec -fd spec/os_image/ubuntu_${short_name}_spec.rb + OS_IMAGE=/opt/bosh/tmp/ubuntu_base_image.tgz bundle exec rspec -fd spec/os_image/ubuntu_spec.rb ``` ### How to Run Tests for Stemcell @@ -208,7 +223,7 @@ If you find yourself debugging any of the above processes, here is what you need Example usage: ```shell - export short_name="noble" + export short_name="resolute" bundle exec rake stemcell:build_os_image[ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz] resume_from=rsyslog_config ``` @@ -219,8 +234,8 @@ If you find yourself debugging any of the above processes, here is what you need modifications you can rerun the tests (without rebuilding OS image). Details in section `How to run tests for OS Images` * If the Stemcell has been built, and you are only updating tests, you do not - need to re-build the stemcell. You can simply rerun the tests - without - rebuilding Stemcell. Details in section `How to run tests for Stemcell` + need to re-build the stemcell. You can simply rerun the tests (without + rebuilding Stemcell). Details in section `How to run tests for Stemcell` * It's possible to verify OS/Stemcell changes without making a deployment using the stemcell. For a vSphere-specific Ubuntu stemcell, the filesystem is available at `/mnt/stemcells/vsphere/esxi/ubuntu/work/work/chroot` @@ -249,7 +264,7 @@ You will need the ovftool installer present in Rebuild the container with: ```shell -export short_name="noble" +export short_name="resolute" docker build \ --platform linux/amd64 \ @@ -280,7 +295,7 @@ gsutil cp MY_OVFTOOL_FILE gs://bosh-vmware-ovftool/MY_OS/ Example: ```shell -export short_name="noble" +export short_name="resolute" gsutil cp VMware-ovftool-4.4.3-18663434-lin.x86_64.bundle gs://bosh-vmware-ovftool/${short_name}/ ``` @@ -387,4 +402,3 @@ When switching from the old pipeline to the new one, don't forget to: whatever the public bucket should be * update the tasks YAML to point to tasks in the `os-images` directory * rename this directory from `new` - diff --git a/Rakefile b/Rakefile index 287b1ae269..781c251a90 100644 --- a/Rakefile +++ b/Rakefile @@ -1,111 +1,114 @@ -require 'json' +require "json" namespace :stemcell do - desc 'Build a base OS image for use in stemcells' + desc "Build a base OS image for use in stemcells" task :build_os_image, [:operating_system_name, :operating_system_version, :os_image_path] do |_, args| - begin - require 'bosh/stemcell/archive_handler' - require 'bosh/stemcell/build_environment' - require 'bosh/stemcell/definition' - require 'bosh/stemcell/os_image_builder' - require 'bosh/stemcell/stage_collection' - require 'bosh/stemcell/stage_runner' - - definition = Bosh::Stemcell::Definition.for('null', 'null', args.operating_system_name, args.operating_system_version) - environment = Bosh::Stemcell::BuildEnvironment.new( - ENV.to_hash, - definition, - '', - args.os_image_path, - ) - collection = Bosh::Stemcell::StageCollection.new(definition) - runner = Bosh::Stemcell::StageRunner.new( - build_path: environment.build_path, - command_env: environment.command_env, - settings_file: environment.settings_path, - work_path: environment.work_path, - ) - archive_handler = Bosh::Stemcell::ArchiveHandler.new - - builder = Bosh::Stemcell::OsImageBuilder.new( - environment: environment, - collection: collection, - runner: runner, - archive_handler: archive_handler, - ) - builder.build(args.os_image_path) - - sh(environment.os_image_rspec_command) - rescue RuntimeError => e - print_help - raise e + require "bosh/stemcell/archive_handler" + require "bosh/stemcell/build_environment" + require "bosh/stemcell/definition" + require "bosh/stemcell/os_image_builder" + require "bosh/stemcell/stage_collection" + require "bosh/stemcell/stage_runner" + + os_image_path = File.expand_path(args.os_image_path) + if args.operating_system_version.to_s.strip.empty? + raise "stemcell:build_os_image: operating_system_version (2nd argument) is empty. " \ + "Set it to the Ubuntu release codename (for example export short_name=resolute per README), " \ + "or pass it literally: rake stemcell:build_os_image[ubuntu,resolute,tmp/os.tgz]" end + definition = Bosh::Stemcell::Definition.for("null", "null", args.operating_system_name, args.operating_system_version) + environment = Bosh::Stemcell::BuildEnvironment.new( + ENV.to_hash, + definition, + "", + os_image_path + ) + collection = Bosh::Stemcell::StageCollection.new(definition) + runner = Bosh::Stemcell::StageRunner.new( + build_path: environment.build_path, + command_env: environment.command_env, + settings_file: environment.settings_path, + work_path: environment.work_path + ) + archive_handler = Bosh::Stemcell::ArchiveHandler.new + + builder = Bosh::Stemcell::OsImageBuilder.new( + environment: environment, + collection: collection, + runner: runner, + archive_handler: archive_handler + ) + builder.build(os_image_path) + + sh(environment.os_image_rspec_command) + rescue RuntimeError => e + print_help + raise e end - desc 'Build a stemcell, requires `os_image_path` pointing at an image created via `stemcell:build_os_image`' + desc "Build a stemcell, requires `os_image_path` pointing at an image created via `stemcell:build_os_image`" task :build, [:infrastructure_name, :hypervisor_name, :operating_system_name, :operating_system_version, :os_image_path, :build_number] do |_, args| - begin - require 'bosh/stemcell/build_environment' - require 'bosh/stemcell/definition' - require 'bosh/stemcell/stage_collection' - require 'bosh/stemcell/stage_runner' - require 'bosh/stemcell/stemcell_packager' - require 'bosh/stemcell/stemcell_builder' - - args.with_defaults(build_number: '0000') - - definition = Bosh::Stemcell::Definition.for(args.infrastructure_name, args.hypervisor_name, args.operating_system_name, args.operating_system_version) - environment = Bosh::Stemcell::BuildEnvironment.new( - ENV.to_hash, - definition, - args.build_number, - args.os_image_path, - ) - - sh(environment.os_image_rspec_command) - - puts "Working from #{environment.work_path}..." - puts "########################################" - runner = Bosh::Stemcell::StageRunner.new( - build_path: environment.build_path, - command_env: environment.command_env, - settings_file: environment.settings_path, - work_path: environment.work_path, - ) - - stemcell_building_stages = Bosh::Stemcell::StageCollection.new(definition) - - builder = Bosh::Stemcell::StemcellBuilder.new( - environment: environment, - runner: runner, - definition: definition, - collection: stemcell_building_stages - ) - - packager = Bosh::Stemcell::StemcellPackager.new( - definition: definition, - version: environment.version, - work_path: environment.work_path, - tarball_path: environment.stemcell_tarball_path, - disk_size: environment.stemcell_disk_size, - runner: runner, - collection: stemcell_building_stages, - ) - - builder.build - - mkdir_p('tmp') - definition.disk_formats.each do |disk_format| - puts "Packaging #{disk_format}..." - stemcell_tarball = packager.package(disk_format) - cp(stemcell_tarball, 'tmp') - end - - sh(environment.stemcell_rspec_command) - rescue RuntimeError => e - print_help - raise e + require "bosh/stemcell/build_environment" + require "bosh/stemcell/definition" + require "bosh/stemcell/stage_collection" + require "bosh/stemcell/stage_runner" + require "bosh/stemcell/stemcell_packager" + require "bosh/stemcell/stemcell_builder" + + args.with_defaults(build_number: "0000") + + os_image_path = File.expand_path(args.os_image_path) + definition = Bosh::Stemcell::Definition.for(args.infrastructure_name, args.hypervisor_name, args.operating_system_name, args.operating_system_version) + environment = Bosh::Stemcell::BuildEnvironment.new( + ENV.to_hash, + definition, + args.build_number, + os_image_path + ) + + sh(environment.os_image_rspec_command) + + puts "Working from #{environment.work_path}..." + puts "########################################" + runner = Bosh::Stemcell::StageRunner.new( + build_path: environment.build_path, + command_env: environment.command_env, + settings_file: environment.settings_path, + work_path: environment.work_path + ) + + stemcell_building_stages = Bosh::Stemcell::StageCollection.new(definition) + + builder = Bosh::Stemcell::StemcellBuilder.new( + environment: environment, + runner: runner, + definition: definition, + collection: stemcell_building_stages + ) + + packager = Bosh::Stemcell::StemcellPackager.new( + definition: definition, + version: environment.version, + work_path: environment.work_path, + tarball_path: environment.stemcell_tarball_path, + disk_size: environment.stemcell_disk_size, + runner: runner, + collection: stemcell_building_stages + ) + + builder.build + + mkdir_p("tmp") + definition.disk_formats.each do |disk_format| + puts "Packaging #{disk_format}..." + stemcell_tarball = packager.package(disk_format) + cp(stemcell_tarball, "tmp") end + + sh(environment.stemcell_rspec_command) + rescue RuntimeError => e + print_help + raise e end def print_help diff --git a/bosh-stemcell/image-metalinks/ubuntu-resolute.meta4 b/bosh-stemcell/image-metalinks/ubuntu-resolute.meta4 new file mode 100644 index 0000000000..b5538e45e5 --- /dev/null +++ b/bosh-stemcell/image-metalinks/ubuntu-resolute.meta4 @@ -0,0 +1,19 @@ + + + placeholder-update-when-resolute-os-image-is-published + placeholder + placeholder + 0 + https://storage.googleapis.com/bosh-os-images/ubuntu-resolute/ubuntu-resolute.tgz + 0.0.0 + + + placeholder + placeholder + 1 + https://storage.googleapis.com/bosh-os-images/ubuntu-resolute/usn-log.json + 0.0.0 + + metalink-repository-resource/0.0.0 + 1970-01-01T00:00:00Z + diff --git a/bosh-stemcell/lib/bosh/stemcell/stage_collection.rb b/bosh-stemcell/lib/bosh/stemcell/stage_collection.rb index 4b64f86f0e..5ed64a21dd 100644 --- a/bosh-stemcell/lib/bosh/stemcell/stage_collection.rb +++ b/bosh-stemcell/lib/bosh/stemcell/stage_collection.rb @@ -198,7 +198,7 @@ def google_stages end def warden_stages - [ + stages = [ :system_parameters, :base_warden, :bosh_clean, @@ -211,6 +211,8 @@ def warden_stages :image_install_grub, :sbom_create ] + stages.insert(2, :base_ubuntu_warden_rosetta) if operating_system.variant == "rosetta" + stages end def azure_stages diff --git a/bosh-stemcell/spec/assets/dpkg-list-ubuntu-kernel.txt b/bosh-stemcell/spec/assets/dpkg-list-ubuntu-kernel.txt index 71ced9ee1e..e2f7edd49c 100644 --- a/bosh-stemcell/spec/assets/dpkg-list-ubuntu-kernel.txt +++ b/bosh-stemcell/spec/assets/dpkg-list-ubuntu-kernel.txt @@ -1,8 +1,8 @@ linux-generic -linux-headers-6.8 -linux-headers-6.8-generic +linux-headers-7.0 +linux-headers-7.0-generic linux-headers-generic -linux-image-6.8-generic +linux-image-7.0-generic linux-image-generic -linux-modules-6.8-generic -linux-modules-extra-6.8-generic +linux-main-modules-zfs-7.0-generic +linux-modules-7.0-generic diff --git a/bosh-stemcell/spec/assets/dpkg-list-ubuntu-vsphere-additions.txt b/bosh-stemcell/spec/assets/dpkg-list-ubuntu-vsphere-additions.txt index dd16ea5e55..94e21bd484 100644 --- a/bosh-stemcell/spec/assets/dpkg-list-ubuntu-vsphere-additions.txt +++ b/bosh-stemcell/spec/assets/dpkg-list-ubuntu-vsphere-additions.txt @@ -2,8 +2,8 @@ libdrm-common libdrm2:amd64 libmspack0t64:amd64 libpci3:amd64 -libxmlsec1t64:amd64 -libxmlsec1t64-openssl:amd64 +libxmlsec1-1:amd64 +libxmlsec1-openssl1:amd64 open-vm-tools pciutils pci.ids diff --git a/bosh-stemcell/spec/assets/dpkg-list-ubuntu.txt b/bosh-stemcell/spec/assets/dpkg-list-ubuntu.txt index f9c3bc0e3b..4a3135842b 100644 --- a/bosh-stemcell/spec/assets/dpkg-list-ubuntu.txt +++ b/bosh-stemcell/spec/assets/dpkg-list-ubuntu.txt @@ -1,10 +1,10 @@ +3cpio adduser amd64-microcode anacron apparmor apparmor-utils apt -apt-utils auditd autoconf automake @@ -30,13 +30,14 @@ chrony cloud-guest-utils cmake cmake-data +comerr-dev:amd64 console-setup console-setup-linux coreutils -cpio +coreutils-from-uutils cpp -cpp-13 -cpp-13-x86-64-linux-gnu +cpp-15 +cpp-15-x86-64-linux-gnu cpp-x86-64-linux-gnu cron cron-daemon-common @@ -61,7 +62,6 @@ distro-info distro-info-data dmidecode dmsetup -dnsutils dpkg dpkg-dev dracut-install @@ -74,39 +74,36 @@ file findutils flex g++ -g++-13 -g++-13-x86-64-linux-gnu +g++-15 +g++-15-x86-64-linux-gnu g++-x86-64-linux-gnu gcc -gcc-13 -gcc-13-base:amd64 -gcc-13-x86-64-linux-gnu -gcc-14-base:amd64 +gcc-15 +gcc-15-base:amd64 +gcc-15-x86-64-linux-gnu +gcc-16-base:amd64 gcc-x86-64-linux-gnu gdb gdisk gettext gettext-base -gir1.2-girepository-2.0:amd64 +gir1.2-girepository-3.0:amd64 gir1.2-glib-2.0:amd64 gir1.2-packagekitglib-1.0 +gnu-coreutils gpg gpg-agent gpgconf gpgv grep groff-base -grub-common grub-efi-amd64-bin -grub-gfxpayload-lists -grub-pc +grub-efi-amd64-unsigned grub-pc-bin -grub2 grub2-common gzip hostname htop -icu-devtools ifupdown init init-system-helpers @@ -126,33 +123,35 @@ keyboard-configuration klibc-utils kmod krb5-locales +krb5-multidev:amd64 less libacl1:amd64 libaio1t64:amd64 libapparmor1:amd64 libappstream5:amd64 -libapt-pkg6.0t64:amd64 +libapt-pkg7.0:amd64 libarchive-zip-perl libarchive13t64:amd64 -libargon2-1:amd64 libasan8:amd64 -libassuan0:amd64 -libatm1t64:amd64 +libassuan9:amd64 libatomic1:amd64 libattr1:amd64 libaudit-common libaudit1:amd64 libauparse0t64:amd64 +libauplugin1:amd64 libbabeltrace1:amd64 libbinutils:amd64 libblkid1:amd64 libbpf1:amd64 +libbrotli-dev:amd64 libbrotli1:amd64 libbsd0:amd64 libbz2-1.0:amd64 libbz2-dev:amd64 libc-bin libc-dev-bin +libc-gconv-modules-extra:amd64 libc6-dev:amd64 libc6:amd64 libcap-dev:amd64 @@ -186,6 +185,7 @@ libefiboot1t64:amd64 libefivar1t64:amd64 libelf1t64:amd64 libestr0:amd64 +libevent-2.1-7t64:amd64 libexpat1:amd64 libext2fs2t64:amd64 libfastjson4:amd64 @@ -195,31 +195,39 @@ libfido2-1:amd64 libfile-stripnondeterminism-perl libfreetype6:amd64 libfribidi0:amd64 -libfuse3-3:amd64 -libgcc-13-dev:amd64 +libfuse3-4:amd64 +libfyaml0:amd64 +libgcc-15-dev:amd64 libgcc-s1:amd64 -libgcrypt20-dev +libgcrypt20-dev:amd64 libgcrypt20:amd64 libgdbm-compat4t64:amd64 libgdbm6t64:amd64 -libgirepository-1.0-1:amd64 +libgirepository-2.0-0:amd64 libglib2.0-0t64:amd64 libglib2.0-bin libglib2.0-data +libgmp-dev:amd64 libgmp10:amd64 +libgmpxx4ldbl:amd64 +libgnutls-dane0t64:amd64 +libgnutls-openssl27t64:amd64 +libgnutls28-dev:amd64 libgnutls30t64:amd64 libgomp1:amd64 -libgpg-error-dev +libgpg-error-dev:amd64 +libgpg-error-l10n libgpg-error0:amd64 libgprofng0:amd64 libgssapi-krb5-2:amd64 +libgssrpc4t64:amd64 libgstreamer1.0-0:amd64 libhogweed6t64:amd64 libhwasan0:amd64 libibverbs1:amd64 -libicu-dev:amd64 -libicu74:amd64 +libicu78:amd64 libidn2-0:amd64 +libidn2-dev:amd64 libinih1:amd64 libip4tc2:amd64 libip6tc2:amd64 @@ -227,19 +235,30 @@ libipt2 libisl23:amd64 libitm1:amd64 libjansson4:amd64 +libjemalloc2:amd64 libjson-c5:amd64 -libjsoncpp25:amd64 +libjsoncpp26:amd64 libk5crypto3:amd64 +libkadm5clnt-mit12:amd64 +libkadm5srv-mit12:amd64 +libkdb5-10t64:amd64 libkeyutils1:amd64 libklibc:amd64 libkmod2:amd64 libkrb5-3:amd64 +libkrb5-dev:amd64 libkrb5support0:amd64 +libksba8:amd64 +liblastlog2-2:amd64 +libldap-common +libldap-dev:amd64 libldap2:amd64 liblmdb0:amd64 liblocale-gettext-perl liblsan0:amd64 +liblsof0 liblz4-1:amd64 +liblzma-dev:amd64 liblzma5:amd64 libmagic-mgc libmagic1t64:amd64 @@ -260,15 +279,18 @@ libnfnetlink0:amd64 libnftables1:amd64 libnftnl11:amd64 libnghttp2-14:amd64 +libnghttp2-dev:amd64 libnl-3-200:amd64 libnl-genl-3-200:amd64 libnl-route-3-200:amd64 libnpth0t64:amd64 libnss-systemd:amd64 -libnvme1t64 +libnvme1t64:amd64 +libp11-kit-dev:amd64 libp11-kit0:amd64 libpackagekit-glib2-18:amd64 libpam-cap:amd64 +libpam-lastlog2:amd64 libpam-modules-bin libpam-modules:amd64 libpam-pwquality:amd64 @@ -278,25 +300,28 @@ libpam0g:amd64 libparted2t64:amd64 libpcap0.8t64:amd64 libpcre2-8-0:amd64 -libperl5.38t64:amd64 +libperl5.40:amd64 libpipeline1:amd64 +libpkgconf7:amd64 libpng16-16t64:amd64 libpolkit-agent-1-0:amd64 libpolkit-gobject-1-0:amd64 libpopt0:amd64 libproc2-0:amd64 +libpsl-dev:amd64 libpsl5t64:amd64 libpwquality-common libpwquality1:amd64 libpython3-stdlib:amd64 -libpython3.12-minimal:amd64 -libpython3.12-stdlib:amd64 -libpython3.12t64:amd64 +libpython3.14-minimal:amd64 +libpython3.14-stdlib:amd64 +libpython3.14:amd64 libquadmath0:amd64 libreadline-dev:amd64 libreadline8t64:amd64 librelp0:amd64 -librhash0:amd64 +librhash1:amd64 +librtmp-dev:amd64 librtmp1:amd64 libsasl2-2:amd64 libsasl2-modules-db:amd64 @@ -307,22 +332,23 @@ libsemanage2:amd64 libsensors-config libsensors5:amd64 libsepol2:amd64 -libsframe1:amd64 +libsframe3:amd64 libslang2:amd64 libsmartcols1:amd64 libsource-highlight-common libsource-highlight4t64:amd64 libsqlite3-0:amd64 libss2:amd64 -libssh-4:amd64 +libssh2-1-dev:amd64 +libssh2-1t64:amd64 libssl-dev:amd64 libssl3t64:amd64 -libstdc++-13-dev:amd64 +libstdc++-15-dev:amd64 libstdc++6:amd64 libstemmer0d:amd64 -libsub-override-perl libsystemd-shared:amd64 libsystemd0:amd64 +libtasn1-6-dev:amd64 libtasn1-6:amd64 libtext-charwidth-perl:amd64 libtext-iconv-perl:amd64 @@ -335,26 +361,30 @@ libtsan2:amd64 libubsan1:amd64 libuchardet0:amd64 libudev1:amd64 +libunbound8:amd64 libunistring5:amd64 libunwind8:amd64 liburcu8t64:amd64 libuuid1:amd64 libuv1t64:amd64 libwrap0:amd64 +libxml2-16:amd64 libxml2-dev:amd64 -libxml2:amd64 libxmlb2:amd64 libxslt1-dev:amd64 libxslt1.1:amd64 libxtables12:amd64 libxxhash0:amd64 libyaml-0-2:amd64 +libzstd-dev:amd64 libzstd1:amd64 linux-base -linux-firmware +linux-firmware-minimal linux-libc-dev:amd64 +linux-sysctl-defaults locales login +login.defs logrotate logsave lsb-release @@ -365,7 +395,6 @@ make man-db mawk media-types -module-assistant mount ncurses-base ncurses-bin @@ -373,6 +402,7 @@ net-tools netbase netcat-openbsd netplan-generator +nettle-dev:amd64 networkd-dispatcher nftables nvme-cli @@ -380,14 +410,17 @@ openssh-client openssh-server openssh-sftp-server openssl +openssl-provider-legacy packagekit parted passwd patch perl perl-base -perl-modules-5.38 +perl-modules-5.40 pinentry-curses +pkgconf-bin +pkgconf:amd64 po-debconf polkitd procps @@ -396,6 +429,8 @@ python-apt-common python3 python3-apparmor python3-apt +python3-autocommand +python3-bcrypt python3-blinker python3-cffi-backend:amd64 python3-cryptography @@ -403,38 +438,44 @@ python3-dbus python3-distro python3-gi python3-httplib2 +python3-inflect +python3-jaraco.context +python3-jaraco.functools +python3-jaraco.text python3-jwt python3-launchpadlib python3-lazr.restfulclient python3-lazr.uri python3-libapparmor +python3-linkify-it python3-markdown-it python3-mdurl python3-minimal -python3-netifaces:amd64 +python3-more-itertools python3-netplan python3-oauthlib python3-pkg-resources python3-pygments python3-pyparsing python3-rich -python3-six +python3-setuptools python3-software-properties +python3-typeguard python3-typing-extensions +python3-uc-micro python3-wadllib python3-yaml -python3.12 -python3.12-minimal +python3-zipp +python3.14 +python3.14-minimal quota readline-common -rng-tools-debian rpcsvc-proto rsync rsyslog rsyslog-gnutls -rsyslog-openssl rsyslog-relp -runit +rust-coreutils sed sensible-utils sgml-base @@ -442,20 +483,18 @@ shared-mime-info software-properties-common strace sudo +sudo-common +sudo-rs sysstat systemd -systemd-dev +systemd-cryptsetup systemd-hwe-hwdb systemd-resolved systemd-sysv -systemd-timesyncd -sysuser-helper sysvinit-utils tar tcpdump -traceroute tzdata -tzdata-legacy ubuntu-keyring ubuntu-pro-client ubuntu-pro-client-l10n diff --git a/bosh-stemcell/spec/bosh/stemcell/stemcell_packager_spec.rb b/bosh-stemcell/spec/bosh/stemcell/stemcell_packager_spec.rb index 9476dcedf6..058d499cbb 100644 --- a/bosh-stemcell/spec/bosh/stemcell/stemcell_packager_spec.rb +++ b/bosh-stemcell/spec/bosh/stemcell/stemcell_packager_spec.rb @@ -205,6 +205,20 @@ def additional_cloud_properties end end + context "when packaging Ubuntu Resolute (26.04)" do + let(:operating_system) { Bosh::Stemcell::OperatingSystem.for("ubuntu", "resolute") } + + it "writes stemcell.MF with ubuntu-resolute and returns resolute tarball path" do + expect(packager.package("raw")).to eq( + File.join(tarball_dir, "bosh-stemcell-1234-fake_infra-fake_hypervisor-ubuntu-resolute-raw.tgz") + ) + + actual_manifest = YAML.load_file(File.join(work_dir, "stemcell/stemcell.MF")) + expect(actual_manifest["name"]).to eq("bosh-fake_infra-fake_hypervisor-ubuntu-resolute-raw") + expect(actual_manifest["operating_system"]).to eq("ubuntu-resolute") + end + end + context "when packaging a non standard os variant" do let(:operating_system) { Bosh::Stemcell::OperatingSystem.for("ubuntu", "FOO-fips") } diff --git a/bosh-stemcell/spec/os_image/ubuntu_spec.rb b/bosh-stemcell/spec/os_image/ubuntu_spec.rb index 0b4b40d428..35ec872c5f 100644 --- a/bosh-stemcell/spec/os_image/ubuntu_spec.rb +++ b/bosh-stemcell/spec/os_image/ubuntu_spec.rb @@ -2,7 +2,7 @@ require "spec_helper" require "shellout_types/file" -describe "Ubuntu 24.04 OS image", os_image: true do +describe "Ubuntu 26.04 OS image", os_image: true do it_behaves_like "every OS image" do let(:syslog_config) { file("/etc/audit/plugins.d/syslog.conf") } end @@ -32,9 +32,9 @@ describe "base_apt" do describe file("/etc/apt/sources.list") do - its(:content) { should match 'deb http:\/\/(archive|snapshot).ubuntu.com\/ubuntu(|\/\d*T\d*Z) noble main universe multiverse' } - its(:content) { should match 'deb http:\/\/(archive|snapshot).ubuntu.com\/ubuntu(|\/\d*T\d*Z) noble-updates main universe multiverse' } - its(:content) { should match 'deb http:\/\/(security|snapshot).ubuntu.com\/ubuntu(|\/\d*T\d*Z) noble-security main universe multiverse' } + its(:content) { should match 'deb http:\/\/(archive|snapshot).ubuntu.com\/ubuntu(|\/\d*T\d*Z) resolute main universe multiverse' } + its(:content) { should match 'deb http:\/\/(archive|snapshot).ubuntu.com\/ubuntu(|\/\d*T\d*Z) resolute-updates main universe multiverse' } + its(:content) { should match 'deb http:\/\/(security|snapshot).ubuntu.com\/ubuntu(|\/\d*T\d*Z) resolute-security main universe multiverse' } end describe file("/lib/systemd/system/monit.service") do @@ -86,16 +86,20 @@ context "installed by system_grub" do %w[ - grub2 + grub-pc-bin + grub-efi-amd64-bin ].each do |pkg| describe package(pkg) do it { should be_installed } end end - %w[unicode.pf2 menu.lst gfxblacklist.txt].each do |grub_stage| - describe file("/boot/grub/#{grub_stage}") do - it { should be_file } - end + # ubuntu-noble tested unicode.pf2 and gfxblacklist.txt here, which were + # installed by the grub2 meta-package (via grub-common). Resolute installs + # only grub-pc-bin and grub-efi-amd64-bin (the bare binaries), which do not + # include those files. They are written to /boot/grub/ during grub-install at + # stemcell build time, after this OS-image phase. + describe file("/boot/grub/menu.lst") do + it { should be_file } end end @@ -149,14 +153,14 @@ end context "package signature verification (stig: V-38462) (stig: V-38483)" do - # verify default behavior was not changed describe command("grep -R AllowUnauthenticated /etc/apt/apt.conf.d/") do its(:stdout) { should eq("") } end end context "official Ubuntu gpg key is installed (stig: V-38476)" do - describe command("apt-key list") do + # apt-key is deprecated in Resolute; keys live in /usr/share/keyrings as binary GPG files. + describe command("gpg --no-default-keyring --keyring /usr/share/keyrings/ubuntu-archive-keyring.gpg --list-keys 2>/dev/null") do its(:stdout) { should include("Ubuntu Archive Automatic Signing Key") } end end @@ -193,6 +197,12 @@ end end + context "display the number of unsuccessful logon/access attempts since the last successful logon/access (stig: V-51875)" do + describe file("/etc/pam.d/common-password") do + its(:content) { should match(/^session\toptional\t\t\tpam_lastlog2\.so showfailed/) } + end + end + # V-38498 and V-38495 are the package defaults and cannot be configured context "ensure auditd is installed (stig: V-38498) (stig: V-38495)" do describe package("auditd") do @@ -201,15 +211,16 @@ end context "ensure auditd file permissions and ownership (stig: V-38663) (stig: V-38664) (stig: V-38665)" do + # auvirt (/usr/bin/auvirt) and autrace (/sbin/autrace) were removed from the + # auditd package in Ubuntu 26.04. auvirt was deprecated upstream; autrace was + # dropped when audit transitioned away from the legacy ptrace-based tracing model. [[0o644, "/usr/share/lintian/overrides/auditd"], - [0o755, "/usr/bin/auvirt"], [0o755, "/usr/bin/ausyscall"], [0o755, "/usr/bin/aulastlog"], [0o755, "/usr/bin/aulast"], [0o750, "/var/log/audit"], [0o755, "/sbin/aureport"], [0o755, "/sbin/auditd"], - [0o755, "/sbin/autrace"], [0o755, "/sbin/ausearch"], [0o755, "/sbin/augenrules"], [0o755, "/sbin/auditctl"], @@ -290,12 +301,6 @@ end end - context "display the number of unsuccessful logon/access attempts since the last successful logon/access (stig: V-51875)" do - describe file("/etc/pam.d/common-password") do - its(:content) { should match(/session\toptional\t\t\tpam_lastlog2\.so showfailed/) } - end - end - context "ensure whoopsie and apport are not installed (CIS-4.1)" do describe package("apport") do it { should_not be_installed } @@ -309,9 +314,15 @@ describe command('grep "^\s*auth\s*required\s*pam_wheel.so\s*use_uid" /etc/pam.d/su') do it("exits 0") { expect(subject.exit_status).to eq(0) } end + describe command("getent group wheel") do + its(:exit_status) { should eq(0) } + its(:stdout) { should match(/root/) } + its(:stdout) { should match(/vcap/) } + end describe user("vcap") do it { should exist } it { should be_in_group "sudo" } + it { should be_in_group "wheel" } end end @@ -352,17 +363,15 @@ _apt:x:42:65534::/nonexistent:/usr/sbin/nologin nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin systemd-network:x:998:998:systemd Network Management:/:/usr/sbin/nologin - systemd-timesync:x:996:996:systemd Time Synchronization:/:/usr/sbin/nologin - dhcpcd:x:100:65534:DHCP Client Daemon,,,:/usr/lib/dhcpcd:/bin/false - messagebus:x:101:101::/nonexistent:/usr/sbin/nologin - syslog:x:102:102::/nonexistent:/usr/sbin/nologin - systemd-resolve:x:991:991:systemd Resolver:/:/usr/sbin/nologin - uuidd:x:103:104::/run/uuidd:/usr/sbin/nologin - _chrony:x:104:106:Chrony daemon,,,:/var/lib/chrony:/usr/sbin/nologin - _runit-log:x:999:990:Created by dh-sysuser for runit:/nonexistent:/usr/sbin/nologin - sshd:x:105:65534::/run/sshd:/usr/sbin/nologin - tcpdump:x:106:108::/nonexistent:/usr/sbin/nologin - polkitd:x:989:989:User for polkitd:/:/usr/sbin/nologin + dhcpcd:x:996:996:DHCP Client Daemon:/usr/lib/dhcpcd:/bin/false + messagebus:x:995:995:System Message Bus:/nonexistent:/usr/sbin/nologin + syslog:x:100:101::/nonexistent:/usr/sbin/nologin + systemd-resolve:x:989:989:systemd Resolver:/:/usr/sbin/nologin + _chrony:x:988:988:Chrony Daemon:/var/lib/chrony:/usr/sbin/nologin + uuidd:x:101:103::/run/uuidd:/usr/sbin/nologin + sshd:x:987:65534:sshd user:/run/sshd:/usr/sbin/nologin + tcpdump:x:986:986:tcpdump:/nonexistent:/usr/sbin/nologin + polkitd:x:985:985:User for polkitd:/:/usr/sbin/nologin vcap:x:1000:1000:BOSH System User:/home/vcap:/bin/bash HERE end @@ -387,17 +396,15 @@ irc:\*:(\d{5}):0:99999:7::: _apt:\*:(\d{5}):0:99999:7::: nobody:\*:(\d{5}):0:99999:7::: - systemd-network:!\*:(\d{5}):::::: - systemd-timesync:!\*:(\d{5}):::::: - dhcpcd:!:(\d{5}):::::: - messagebus:!:(\d{5}):::::: + systemd-network:!\*:(\d{5}):::::1: + dhcpcd:!\*:(\d{5}):::::1: + messagebus:!\*:(\d{5}):::::: syslog:!:(\d{5}):::::: - systemd-resolve:!\*:(\d{5}):::::: + systemd-resolve:!\*:(\d{5}):::::1: + _chrony:!\*:(\d{5}):::::: uuidd:!:(\d{5}):::::: - _chrony:!:(\d{5}):::::: - _runit-log:!:(\d{5}):::::: - sshd:!:(\d{5}):::::: - tcpdump:!:(\d{5}):::::: + sshd:!\*:(\d{5}):::::: + tcpdump:!\*:(\d{5}):::::1: polkitd:!\*:(\d{5}):::::: vcap:(.+):(\d{5}):1:99999:7::: END_SHADOW @@ -448,26 +455,27 @@ systemd-journal:x:999: systemd-network:x:998: crontab:x:997: - systemd-timesync:x:996: - input:x:995: - sgx:x:994: - kvm:x:993: - render:x:992: - messagebus:x:101: - syslog:x:102: - systemd-resolve:x:991: - netdev:x:103: - uuidd:x:104: - _ssh:x:105: - _chrony:x:106: - _runit-log:x:990: - rdma:x:107: - tcpdump:x:108: - polkitd:x:989: - admin:x:988:vcap + dhcpcd:x:996: + messagebus:x:995: + syslog:x:101: + input:x:994: + sgx:x:993: + clock:x:992: + kvm:x:991: + render:x:990: + systemd-resolve:x:989: + _chrony:x:988: + netdev:x:102: + uuidd:x:103: + _ssh:x:104: + rdma:x:105: + tcpdump:x:986: + polkitd:x:985: + admin:x:987:vcap vcap:x:1000:syslog bosh_sshers:x:1001:vcap bosh_sudoers:x:1002: + wheel:x:1003:root,vcap HERE end @@ -514,30 +522,60 @@ systemd-journal:!*:: systemd-network:!*:: crontab:!*:: - systemd-timesync:!*:: + dhcpcd:!*:: + messagebus:!*:: + syslog:!:: input:!*:: sgx:!*:: + clock:!*:: kvm:!*:: render:!*:: - messagebus:!:: - syslog:!:: systemd-resolve:!*:: + _chrony:!*:: netdev:!:: uuidd:!:: _ssh:!:: - _chrony:!:: - _runit-log:!:: rdma:!:: - tcpdump:!:: + tcpdump:!*:: polkitd:!*:: admin:!::vcap vcap:!::syslog bosh_sshers:!::vcap bosh_sudoers:!:: + wheel:!::root,vcap HERE end end + context "runit removed (Resolute Raccoon: no chpst)" do + # Per Resolute RFC #1498, the runit package is removed from the stemcell. + # Releases must migrate off chpst (use BPM, su, runuser, or setpriv instead). + describe package("runit") do + it { should_not be_installed } + end + + describe file("/usr/bin/chpst") do + it { should_not be_file } + end + + describe file("/usr/bin/runsv") do + it { should_not be_file } + end + + describe file("/usr/sbin/runit") do + it { should_not be_file } + end + end + + context "tmp.mount masked (systemd 259)" do + # systemd 259 introduced a static tmp.mount unit that mounts /tmp as a tmpfs. + # bosh already configures the VM's /tmp (as a tmpfs, sized smaller than + # systemd's default). Mask tmp.mount so systemd cannot override that setup. + describe file("/etc/systemd/system/tmp.mount") do + it { should be_linked_to File::NULL } + end + end + describe "systemd-resolved modifications" do describe file("/lib/systemd/system/create-systemd-resolved-listener-address.service") do its(:content) { should match(/ip addr add 169\.254\.0\.53 dev lo/) } diff --git a/bosh-stemcell/spec/stemcells/rosetta_spec.rb b/bosh-stemcell/spec/stemcells/rosetta_spec.rb new file mode 100644 index 0000000000..86a8a25fbb --- /dev/null +++ b/bosh-stemcell/spec/stemcells/rosetta_spec.rb @@ -0,0 +1,107 @@ +require "spec_helper" + +describe "Rosetta Warden Stemcell", stemcell_image: true do + context "arm64 systemd binaries (Rosetta pidfd fix)" do + # On Apple Silicon (arm64 kernel + Rosetta x86_64 emulation), systemd v256+ + # calls pidfd_open and pidfd_send_signal which Rosetta does not translate for + # x86_64 processes, causing ENOSYS. These assertions verify that all key + # systemd binaries have been replaced with arm64 ELF equivalents so they run + # natively on the arm64 kernel without Rosetta translation. + + # Only service daemons are replaced with arm64; user-facing CLI tools in + # /usr/bin/ (systemctl, journalctl, etc.) remain x86-64 because they + # communicate with PID1 over D-Bus and never call pidfd themselves. + %w[ + /usr/lib/systemd/systemd + /usr/lib/systemd/systemd-executor + /usr/lib/systemd/systemd-journald + /usr/lib/systemd/systemd-logind + /usr/lib/systemd/systemd-networkd + /usr/lib/systemd/systemd-resolved + /usr/lib/systemd/systemd-shutdown + ].each do |bin| + describe command("file #{bin}") do + its(:stdout) { should match(/ELF 64-bit.*ARM aarch64/) } + end + end + + # arm64 runtime libraries land in /lib/aarch64-linux-gnu/ via Ubuntu multiarch. + # On Ubuntu 22.04+ (UsrMerge), /lib is a symlink to usr/lib; the real file is + # at /lib/aarch64-linux-gnu/libc.so.6 (resolves via the UsrMerge symlink). + describe file("/lib/aarch64-linux-gnu/libc.so.6") do + it { should be_file } + end + + # The arm64 ELF interpreter is at the multiarch path; /lib/ld-linux-aarch64.so.1 + # is a symlink to it but ShelloutTypes::File cannot check symlink targets reliably. + describe file("/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1") do + it { should be_file } + end + + describe file("/usr/lib/aarch64-linux-gnu/systemd/libsystemd-shared-259.so") do + it { should be_file } + end + end + + context "Rosetta x86_64 emulation compatibility for Apple Silicon" do + # These systemd drop-in overrides disable security features that conflict + # with Rosetta's JIT compilation on Apple Silicon Macs. + + rosetta_services = %w[ + systemd-journald + systemd-resolved + systemd-networkd + systemd-logind + systemd-timesyncd + auditd + ] + + rosetta_services.each do |service| + describe file("/etc/systemd/system/#{service}.service.d/rosetta-compat.conf") do + it { should be_file } + its(:content) { should include("MemoryDenyWriteExecute=no") } + its(:content) { should include("LockPersonality=no") } + its(:content) { should include("NoNewPrivileges=no") } + end + end + + describe file("/etc/systemd/system/systemd-binfmt.service") do + it { should be_linked_to File::NULL } + end + end + + context "SSH without socket activation (Rosetta/Colima ENOSYS)" do + describe file("/etc/systemd/system/ssh.socket") do + it { should be_linked_to File::NULL } + end + + describe file("/etc/systemd/system/ssh.service.d/warden-no-socket-activation.conf") do + it { should be_file } + its(:content) { should include("RefuseManualStart=no") } + end + end + + context "auditd foreground (Rosetta/Colima Docker pidfd ENOSYS)" do + # Under Docker/Colima with Rosetta emulation, systemd cannot create pidfd + # references or cgroup entries for processes started with Type=forking + PIDFile. + # Running auditd with -n (no-fork / foreground) avoids the fork-and-PIDFile + # lifecycle entirely, so systemd tracks the process directly without pidfd. + describe file("/etc/systemd/system/auditd.service.d/warden-auditd-foreground.conf") do + it { should be_file } + its(:content) { should include("Type=simple") } + its(:content) { should include("ExecStart=/usr/sbin/auditd -n") } + end + end + + context "restrict access to the su command CIS-9.5 (Rosetta PAM override)" do + # The Rosetta stemcell replaces /etc/pam.d/su with a minimal config that + # avoids unix-chkpwd (which AppArmor blocks under Lima/Rosetta, causing every + # su invocation to fail with "Authentication failure" even for root). + # pam_wheel.so use_uid is kept in the replacement config so that the CIS-9.5 + # requirement — only wheel-group members may use su — is still enforced. + # This test verifies that the override did not inadvertently remove the wheel check. + describe command('grep "^\s*auth\s*required\s*pam_wheel.so\s*use_uid" /etc/pam.d/su') do + it("exits 0") { expect(subject.exit_status).to eq(0) } + end + end +end diff --git a/bosh-stemcell/spec/stemcells/ubuntu_spec.rb b/bosh-stemcell/spec/stemcells/ubuntu_spec.rb index dd705c9cc7..b807d1875d 100644 --- a/bosh-stemcell/spec/stemcells/ubuntu_spec.rb +++ b/bosh-stemcell/spec/stemcells/ubuntu_spec.rb @@ -1,11 +1,10 @@ require "spec_helper" -describe "Ubuntu 24.04 stemcell image", stemcell_image: true do +describe "Ubuntu 26.04 stemcell image", stemcell_image: true do it_behaves_like "All Stemcells" it_behaves_like "a Linux kernel based OS image" it_behaves_like "a Linux kernel module configured OS image" - # linux_version_regex = '/linux-(.+)-([0-9]+.+)/d' linux_version_regex = 's/linux-(.+)-([0-9]+).([0-9]+).([0-9]+)-([0-9]+)/linux-\1-\2.\3/' context "installed by image_install_grub", { @@ -119,8 +118,13 @@ subject { command("find -L / -xdev -perm /ug=s -type f") } it("includes the correct binaries") do - # expect(subject.stdout.split).to match_array(%w(/bin/su /usr/bin/sudo /usr/bin/sudoedit)) - expect(subject.stdout.split).to match_array(%w[/bin/su /bin/sudo /bin/sudoedit /usr/bin/su /usr/bin/sudo /usr/bin/sudoedit]) + expect(subject.stdout.split).to match_array(%w[ + /bin/su /bin/su-rs /bin/sudo /bin/sudo-rs /bin/sudoedit /bin/sudoedit-rs + /etc/alternatives/sudo /etc/alternatives/sudoedit + /lib/cargo/bin/su /lib/cargo/bin/sudo + /usr/bin/su /usr/bin/su-rs /usr/bin/sudo /usr/bin/sudo-rs /usr/bin/sudoedit /usr/bin/sudoedit-rs + /usr/lib/cargo/bin/su /usr/lib/cargo/bin/sudo + ]) end end end @@ -342,9 +346,16 @@ end end + context "on non-warden stemcells", { + exclude_on_warden: true + } do + describe file("/etc/sysctl.d/20-disable-apparmor-restrict.conf") do + it { should_not be_file } + end + end + describe "installed packages" do dpkg_list_packages = "dpkg --get-selections | cut -f1 | sed -E '#{linux_version_regex}'" - # TODO: maby we can use awk "dpkg --get-selections | awk '!/linux-(.+)-([0-9]+.+)/&&/linux/{print $1}'" let(:dpkg_list_ubuntu) { File.readlines(spec_asset("dpkg-list-ubuntu.txt")).map(&:chop) } let(:dpkg_list_kernel_ubuntu) { File.readlines(spec_asset("dpkg-list-ubuntu-kernel.txt")).map(&:chop) } @@ -358,7 +369,8 @@ exclude_on_cloudstack: true, exclude_on_google: true, exclude_on_vsphere: true, - exclude_on_azure: true + exclude_on_azure: true, + exclude_on_rosetta: true } do it "contains only the base set of packages for alicloud, aws, openstack, warden" do expect(subject.stdout.split("\n")).to match_array(dpkg_list_ubuntu.concat(dpkg_list_kernel_ubuntu)) @@ -427,7 +439,7 @@ end end -describe "Ubuntu 24.04 stemcell tarball", stemcell_tarball: true do +describe "Ubuntu 26.04 stemcell tarball", stemcell_tarball: true do context "installed by bosh_dpkg_list stage" do describe file("#{ENV["STEMCELL_WORKDIR"]}/stemcell/packages.txt", ShelloutTypes::Chroot.new("/")) do it { should be_file } diff --git a/bosh-stemcell/spec/stemcells/warden_spec.rb b/bosh-stemcell/spec/stemcells/warden_spec.rb index 13099230de..6fe3b9543b 100644 --- a/bosh-stemcell/spec/stemcells/warden_spec.rb +++ b/bosh-stemcell/spec/stemcells/warden_spec.rb @@ -21,30 +21,12 @@ end end - context "Rosetta x86_64 emulation compatibility for Apple Silicon" do - # These systemd drop-in overrides disable security features that conflict - # with Rosetta's JIT compilation on Apple Silicon Macs - - rosetta_services = %w[ - systemd-journald - systemd-resolved - systemd-networkd - systemd-logind - systemd-timesyncd - auditd - ] - - rosetta_services.each do |service| - describe file("/etc/systemd/system/#{service}.service.d/rosetta-compat.conf") do - it { should be_file } - its(:content) { should include("MemoryDenyWriteExecute=no") } - its(:content) { should include("LockPersonality=no") } - its(:content) { should include("NoNewPrivileges=no") } - end - end - - describe file("/etc/systemd/system/systemd-binfmt.service") do - it { should be_linked_to File::NULL } + context "installed by base_warden" do + describe file("/etc/sysctl.d/20-disable-apparmor-restrict.conf") do + it { should be_file } + its(:mode) { should eq(0o644) } + its(:content) { should match(/^kernel\.apparmor_restrict_unprivileged_userns = 0$/) } + its(:content) { should match(/^kernel\.apparmor_restrict_unprivileged_unconfined = 0$/) } end end end diff --git a/bosh-stemcell/spec/support/os_image_shared_examples.rb b/bosh-stemcell/spec/support/os_image_shared_examples.rb index d919e57bb5..109f891655 100644 --- a/bosh-stemcell/spec/support/os_image_shared_examples.rb +++ b/bosh-stemcell/spec/support/os_image_shared_examples.rb @@ -150,7 +150,7 @@ describe file("/etc/rsyslog.conf") do it { should be_file } - its(:content) { should match(/^module\( load="omrelp" tls.tlslib="openssl" \)$/) } + its(:content) { should match(/^module\( load="omrelp" tls.tlslib="gnutls" \)$/) } its(:content) { should match '\$FileGroup syslog' } # stig: V-38519 its(:content) { should match '\$FileOwner syslog' } # stig: V-38518 its(:content) { should match '\$FileCreateMode 0600' } # stig: V-38623 @@ -590,7 +590,7 @@ describe "events that modify system date and time must be recorded (CIS-8.1.4)" do its(:content) { should match(/^-a always,exit -F arch=b64 -S adjtimex -S settimeofday -k time-change$/) } - its(:content) { should match(/^-a always,exit -F arch=b32 -S adjtimex -S settimeofday -S stime -k time-change$/) } + its(:content) { should match(/^-a always,exit -F arch=b32 -S adjtimex -S settimeofday -k time-change$/) } its(:content) { should match(/^-a always,exit -F arch=b64 -S clock_settime -k time-change$/) } its(:content) { should match(/^-a always,exit -F arch=b32 -S clock_settime -k time-change$/) } its(:content) { should match(/^-w \/etc\/localtime -p wa -k time-change$/) } @@ -634,8 +634,8 @@ end describe "record events that modify system network environment (CIS-4.1.6)" do - its(:content) { should match(/^-a exit,always -F arch=b64 -S sethostname -S setdomainname -k system-locale$/) } - its(:content) { should match(/^-a exit,always -F arch=b32 -S sethostname -S setdomainname -k system-locale$/) } + its(:content) { should match(/^-a always,exit -F arch=b64 -S sethostname -S setdomainname -k system-locale$/) } + its(:content) { should match(/^-a always,exit -F arch=b32 -S sethostname -S setdomainname -k system-locale$/) } its(:content) { should match(/^-w \/etc\/issue -p wa -k system-locale$/) } its(:content) { should match(/^-w \/etc\/issue\.net -p wa -k system-locale$/) } its(:content) { should match(/^-w \/etc\/hosts -p wa -k system-locale$/) } diff --git a/ci/docker/os-image-stemcell-builder/.gitignore b/ci/docker/os-image-stemcell-builder/.gitignore index 9761a2e21f..c93103d575 100644 --- a/ci/docker/os-image-stemcell-builder/.gitignore +++ b/ci/docker/os-image-stemcell-builder/.gitignore @@ -1 +1,2 @@ /*.bundle +/ovftool/ diff --git a/ci/docker/os-image-stemcell-builder/Dockerfile b/ci/docker/os-image-stemcell-builder/Dockerfile index 40fc8e6f1a..dd2a3b6a29 100644 --- a/ci/docker/os-image-stemcell-builder/Dockerfile +++ b/ci/docker/os-image-stemcell-builder/Dockerfile @@ -4,6 +4,9 @@ FROM $BASE_IMAGE LABEL maintainer="cf-bosh@lists.cloudfoundry.org" +ARG USER_ID=1000 +ARG GROUP_ID=1000 + # BUILD_ARGs ARG META4_CLI_URL ARG SYFT_CLI_URL @@ -67,7 +70,15 @@ RUN apt-get update \ xvfb \ && locale-gen ${LANG} -RUN echo 'ubuntu ALL=NOPASSWD:ALL' >> /etc/sudoers +# AppArmor's unix-chkpwd profile can block the Rosetta translator under +# docker run --privileged on Apple Silicon; use a distinct helper name so PAM still works. +RUN cp /usr/sbin/unix_chkpwd /usr/sbin/unix_chkpwd_rosetta \ + && chmod 4755 /usr/sbin/unix_chkpwd_rosetta \ + && ln -sf unix_chkpwd_rosetta /usr/sbin/unix_chkpwd + +RUN (id -u ubuntu &>/dev/null || useradd -u ${USER_ID} -g ${GROUP_ID} -m ubuntu) \ + && usermod -p '*' ubuntu \ + && echo 'ubuntu ALL=NOPASSWD:ALL' >> /etc/sudoers RUN temp_dir="/mnt/tmp" \ && mkdir -p "${temp_dir}" \ diff --git a/ci/docker/os-image-stemcell-builder/Vagrantfile b/ci/docker/os-image-stemcell-builder/Vagrantfile deleted file mode 100644 index b01294254a..0000000000 --- a/ci/docker/os-image-stemcell-builder/Vagrantfile +++ /dev/null @@ -1,54 +0,0 @@ -VAGRANTFILE_API_VERSION = '2'.freeze - -Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| - required_plugins = %w[vagrant-vbguest vagrant-disksize] - should_retry = false - required_plugins.each do |plugin| - unless Vagrant.has_plugin? plugin - system "vagrant plugin install #{plugin}" - should_retry = true - end - end - - exec 'vagrant ' + ARGV.join(' ') if should_retry - - config.disksize.size = '50GB' - - config.vm.box = 'ubuntu/bionic64' - - config.vm.provider(:virtualbox) do |v| - v.name = 'bosh-os-image-stemcell-docker-builder' - v.customize ['modifyvm', :id, '--cpus', '8'] - v.customize ['modifyvm', :id, '--memory', '8192'] - end - - config.vm.synced_folder('../../../', '/opt/bosh') - - if Vagrant.has_plugin?('vagrant-proxyconf') - if ENV['http_proxy'] - config.proxy.http = ENV['http_proxy'] - config.apt_proxy.http = ENV['http_proxy'] - end - if ENV['https_proxy'] - config.proxy.https = ENV['https_proxy'] - config.apt_proxy.https = ENV['https_proxy'] - end - config.proxy.no_proxy = ENV['no_proxy'] if ENV['no_proxy'] - end - - config.vm.provision('docker') - - config.vm.provision :shell do |shell| - shell.inline = <<-BASH - # disable annoying ctrl-p docker binding - mkdir -p ~vagrant/.docker - echo '{ "detachKeys": "ctrl-q,q" }' > ~vagrant/.docker/config.json - sudo chown -R vagrant ~vagrant/.docker - - # the centos build process runs into an error if we use the default aufs; devicemapper seems to work - # see https://github.com/docker/docker/issues/6980 - echo 'DOCKER_OPTS="${DOCKER_OPTS:-} -s devicemapper "' >> /etc/default/docker - service docker restart - BASH - end -end diff --git a/ci/tasks/light-aws/merge-builds b/ci/tasks/light-aws/merge-builds index e309dfad1e..c3129b6706 100755 --- a/ci/tasks/light-aws/merge-builds +++ b/ci/tasks/light-aws/merge-builds @@ -5,23 +5,23 @@ # from different regions, by concatenating the 'ami' field in # the manifest -require 'yaml' +require "yaml" require "tmpdir" -output_dir=File.expand_path("light-stemcell") +output_dir = File.expand_path("light-stemcell") -merged_amis = {} +merged_amis = {} def run_command(cmd) - result = `#{cmd}` + `#{cmd}` if $?.exitstatus > 0 abort("Failed to execute #{cmd}") end end -Dir['*-stemcell'].each do | tarball_dir | - next if tarball_dir == 'light-stemcell' - Dir.mktmpdir do | temp_dir | +Dir["*-stemcell"].each do |tarball_dir| + next if tarball_dir == "light-stemcell" + Dir.mktmpdir do |temp_dir| matches = Dir.glob("#{tarball_dir}/*.tgz") if matches.length == 0 abort("Empty glob for '#{tarball_dir}/*.tgz'") @@ -32,11 +32,11 @@ Dir['*-stemcell'].each do | tarball_dir | run_command("tar -xvf #{tarball_path} -C #{temp_dir}") yaml = YAML.load_file("#{temp_dir}/stemcell.MF") - ami = yaml['cloud_properties']['ami'] + ami = yaml["cloud_properties"]["ami"] merged_amis = ami.merge(merged_amis) - yaml['cloud_properties']['ami'] = merged_amis + yaml["cloud_properties"]["ami"] = merged_amis - File.open("#{temp_dir}/stemcell.MF",'w') do |s| + File.open("#{temp_dir}/stemcell.MF", "w") do |s| s.puts yaml.to_yaml end diff --git a/stemcell_builder/lib/helpers.sh b/stemcell_builder/lib/helpers.sh index 2ca4bbdaaa..3f1a0e7f4e 100644 --- a/stemcell_builder/lib/helpers.sh +++ b/stemcell_builder/lib/helpers.sh @@ -24,18 +24,37 @@ function run_in_chroot { disable $chroot/sbin/initctl disable $chroot/usr/sbin/invoke-rc.d - # `unshare -f -p` to prevent `kill -HUP 1` from causing `init` to exit; - unshare -f -p -m $SHELL < ../../bin/udevadm) are +# skipped by -type f so they continue to point at their existing targets. +run_in_chroot $chroot " + # All regular-file daemons from the main systemd package (includes PID1). + # On Ubuntu 22.04+ (UsrMerge), /lib -> usr/lib, so the deb ships binaries + # at usr/lib/systemd/ rather than lib/systemd/. + find /tmp/arm64-debs/systemd/usr/lib/systemd/ -maxdepth 1 -type f \ + | xargs -I{} cp {} /usr/lib/systemd/ + + # systemd-resolved is packaged separately on Ubuntu. + cp /tmp/arm64-debs/systemd-resolved/usr/lib/systemd/systemd-resolved \ + /usr/lib/systemd/systemd-resolved +" + +# Clean up staging area. +run_in_chroot $chroot "rm -rf /tmp/arm64-debs" + +# --------------------------------------------------------------------------- +# Part 2: Rosetta compatibility overrides +# --------------------------------------------------------------------------- + +# Apply systemd drop-ins that disable security hardening features conflicting +# with Rosetta's JIT compilation (which requires writable+executable memory). +rosetta_services=( + systemd-journald + systemd-resolved + systemd-networkd + systemd-logind + systemd-timesyncd + systemd-udevd + logrotate + auditd +) + +for service in "${rosetta_services[@]}"; do + mkdir -p "$chroot/etc/systemd/system/${service}.service.d" + cp "$assets_dir/rosetta-compat.conf" "$chroot/etc/systemd/system/${service}.service.d/rosetta-compat.conf" +done + +# Mask systemd-binfmt.service which fails under Rosetta emulation. +run_in_chroot "$chroot" "systemctl mask systemd-binfmt.service" + +# auditd: systemd can return ENOSYS when creating pidfd/cgroup references from +# PIDFile under Docker/Colima. Foreground auditd avoids forking + PIDFile quirks. +mkdir -p "$chroot/etc/systemd/system/auditd.service.d" +cat > "$chroot/etc/systemd/system/auditd.service.d/warden-auditd-foreground.conf" <<'UNIT' +[Service] +Type=simple +PIDFile= +ExecStart= +ExecStart=/usr/sbin/auditd -n +UNIT + +# Ubuntu enables OpenSSH via ssh.socket (socket activation). systemd's listener +# stub fork can fail with ENOSYS under Docker/Colima with Rosetta x86_64 +# emulation. Use the traditional ssh.service so sshd binds port 22 itself. +mkdir -p "$chroot/etc/systemd/system/ssh.service.d" +cat > "$chroot/etc/systemd/system/ssh.service.d/warden-no-socket-activation.conf" <<'UNIT' +[Unit] +# When ssh.socket is masked, sshd must start via ssh.service; drop RefuseManualStart from the vendor unit. +RefuseManualStart=no +UNIT +run_in_chroot "$chroot" "systemctl mask ssh.socket" +run_in_chroot "$chroot" "systemctl enable ssh.service" + +# Fix `su` PAM authentication failures under Colima/Lima (Apple Silicon). +# +# Under Lima's Rosetta x86_64 emulation, AppArmor blocks unix-chkpwd +# (an x86_64 binary) from accessing the Rosetta runtime path +# /mnt/lima-rosetta/rosetta, causing every `su` invocation to fail with +# "Authentication failure" — even for root. The standard PAM config in +# /etc/pam.d/su includes common-auth, which chains pam_faillock → pam_unix, +# and pam_unix forks unix-chkpwd to verify passwords. That fork triggers +# the AppArmor denial. +# +# pam_rootok.so already handles "root switching to any user" correctly on +# its own, but because pam_unix / pam_faillock come after it in common-auth +# they still run (pam_rootok is "sufficient" only when it *succeeds* at the +# auth step level, not at the include level). Using pam_permit for the +# remaining auth/account rules means root can always su without unix-chkpwd. +# +# pam_wheel.so use_uid is retained (CIS-9.5): it only checks group membership +# and never forks unix-chkpwd, so it is safe under Rosetta/AppArmor. +# pam_rootok is "sufficient", so root bypasses the wheel check entirely; +# non-root users must be in the wheel group (vcap is added by restrict_su_command). +# +# This override is safe for warden containers: the host provides isolation, +# and the container root user is already fully privileged. +cat > "$chroot/etc/pam.d/su" <<'PAMEOF' +# PAM configuration for su - warden Rosetta stemcell override. +# AppArmor blocks unix-chkpwd under Lima/Rosetta causing su to fail even for root. +# pam_rootok handles legitimate root → any-user su; pam_permit covers the rest. +# pam_wheel.so use_uid is kept for CIS-9.5 compliance (does not invoke unix-chkpwd). +auth sufficient pam_rootok.so +auth required pam_wheel.so use_uid +auth required pam_permit.so +account required pam_permit.so +session required pam_env.so readenv=1 +session required pam_limits.so +PAMEOF +chmod 0644 "$chroot/etc/pam.d/su" diff --git a/stemcell_builder/stages/base_warden/assets/rosetta-compat.conf b/stemcell_builder/stages/base_ubuntu_warden_rosetta/assets/rosetta-compat.conf similarity index 100% rename from stemcell_builder/stages/base_warden/assets/rosetta-compat.conf rename to stemcell_builder/stages/base_ubuntu_warden_rosetta/assets/rosetta-compat.conf diff --git a/stemcell_builder/stages/base_warden/apply.sh b/stemcell_builder/stages/base_warden/apply.sh index a576d0b22f..ce367daded 100755 --- a/stemcell_builder/stages/base_warden/apply.sh +++ b/stemcell_builder/stages/base_warden/apply.sh @@ -16,6 +16,19 @@ sed -i 's/^local_events = yes$/local_events = no/g' $chroot/etc/audit/auditd.con # restart limit of 5 restarts in 5 seconds sed -i 's/^#DefaultStartLimitBurst=5$/DefaultStartLimitBurst=500/g' $chroot/etc/systemd/system.conf +# Override the upstream AppArmor sysctl restrictions that ship in +# /usr/lib/sysctl.d/10-apparmor.conf on Ubuntu 26.04+. When systemd-sysctl +# runs inside a privileged container it writes to the *host* /proc/sys, +# resetting kernel.apparmor_restrict_unprivileged_userns to 1 and breaking +# any host process that relies on unprivileged user namespaces. +if [ -f "$chroot/usr/lib/sysctl.d/10-apparmor.conf" ]; then + cat > "$chroot/etc/sysctl.d/20-disable-apparmor-restrict.conf" < $chroot/var/vcap/bosh/bin/restart_networking < $chroot/var/vcap/bosh/agent.json <> "$chroot/etc/audit/rules.d/audit.rules" -w /sbin/insmod -p x -k modules -w /sbin/rmmod -p x -k modules -w /sbin/modprobe -p x -k modules @@ -20,7 +20,7 @@ function write_shared_audit_rules { # Record events that modify system date and time -a always,exit -F arch=b64 -S adjtimex -S settimeofday -k time-change --a always,exit -F arch=b32 -S adjtimex -S settimeofday -S stime -k time-change +-a always,exit -F arch=b32 -S adjtimex -S settimeofday -k time-change -a always,exit -F arch=b64 -S clock_settime -k time-change -a always,exit -F arch=b32 -S clock_settime -k time-change -w /etc/localtime -p wa -k time-change @@ -52,8 +52,8 @@ function write_shared_audit_rules { -w /etc/security/opasswd -p wa -k identity # Record events that modify system network environment --a exit,always -F arch=b64 -S sethostname -S setdomainname -k system-locale --a exit,always -F arch=b32 -S sethostname -S setdomainname -k system-locale +-a always,exit -F arch=b64 -S sethostname -S setdomainname -k system-locale +-a always,exit -F arch=b32 -S sethostname -S setdomainname -k system-locale -w /etc/issue -p wa -k system-locale -w /etc/issue.net -p wa -k system-locale -w /etc/hosts -p wa -k system-locale @@ -135,7 +135,7 @@ function write_shared_audit_rules { # Recorde execution of unix_update -a always,exit -F path=/sbin/unix_update -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged-unix-update -' >> $chroot/etc/audit/rules.d/audit.rules +AUDIT_RULES } function override_default_audit_variables { @@ -154,7 +154,6 @@ function override_default_audit_variables { } function record_use_of_privileged_binaries { - echo ' -# Record use of privileged commands' >> $chroot/etc/audit/rules.d/audit.rules + echo '# Record use of privileged commands' >> $chroot/etc/audit/rules.d/audit.rules find $chroot/bin $chroot/sbin $chroot/usr/bin $chroot/usr/sbin $chroot/boot -xdev \( -perm -4000 -o -perm -2000 \) -type f | sed -e s:^${chroot}:: | awk '{print "-a always,exit -F path=" $1 " -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged" }' >> $chroot/etc/audit/rules.d/audit.rules } diff --git a/stemcell_builder/stages/bosh_audit_ubuntu/apply.sh b/stemcell_builder/stages/bosh_audit_ubuntu/apply.sh index 5727fffe30..b14babc66e 100755 --- a/stemcell_builder/stages/bosh_audit_ubuntu/apply.sh +++ b/stemcell_builder/stages/bosh_audit_ubuntu/apply.sh @@ -8,11 +8,14 @@ source $base_dir/lib/prelude_bosh.bash pkg_mgr install auditd +# Ensure the rules directory exists before writing rules. On Ubuntu 26.04+, +# auditd no longer pre-creates this directory during package installation. +mkdir -p $chroot/etc/audit/rules.d + # Without this, auditd will read from /etc/audit/audit.rules instead # of /etc/audit/rules.d/*. cp $chroot/lib/systemd/system/auditd.service $chroot/etc/systemd/system/auditd.service -sed -i '/#ExecStartPost=-\/sbin\/augenrules --load/s/^#//g' $chroot/etc/systemd/system/auditd.service -sed -i '/ExecStartPost=-\/sbin\/auditctl -R \/etc\/audit\/audit.rules/s/^/#/g' $chroot/etc/systemd/system/auditd.service +sed -i '/^ExecStart=.*auditd/a ExecStartPost=-/sbin/augenrules --load' $chroot/etc/systemd/system/auditd.service run_in_bosh_chroot $chroot "systemctl disable auditd.service" run_in_bosh_chroot $chroot "chown root:root /var/log/audit" # (stig: V-38663) (stig: V-38664) (stig: V-38665) diff --git a/stemcell_builder/stages/bosh_monit/apply.sh b/stemcell_builder/stages/bosh_monit/apply.sh index 961a9e4fb2..e8ed76903c 100755 --- a/stemcell_builder/stages/bosh_monit/apply.sh +++ b/stemcell_builder/stages/bosh_monit/apply.sh @@ -12,13 +12,13 @@ monit_archive=$monit_basename.tar.gz mkdir -p $chroot/$bosh_dir/src cp -r $dir/assets/$monit_archive $chroot/$bosh_dir/src -pkg_mgr install "zlib1g-dev" +pkg_mgr install "zlib1g-dev libcrypt-dev" run_in_bosh_chroot $chroot " cd src tar zxvf $monit_archive cd $monit_basename -./configure --prefix=$bosh_dir --without-ssl CFLAGS="-fcommon" +./configure --prefix=$bosh_dir --without-ssl CFLAGS='-fcommon' LIBS='-lcrypt' make -j4 && make install " diff --git a/stemcell_builder/stages/bosh_systemd/apply.sh b/stemcell_builder/stages/bosh_systemd/apply.sh index 9edd672da8..13be0d1f40 100755 --- a/stemcell_builder/stages/bosh_systemd/apply.sh +++ b/stemcell_builder/stages/bosh_systemd/apply.sh @@ -12,3 +12,13 @@ source $base_dir/lib/prelude_bosh.bash run_in_chroot $chroot " echo 'RemoveIPC=no' >> /etc/systemd/logind.conf " + +# systemd 259 (Ubuntu 26.04) mounts /tmp as a tmpfs by default via the static +# tmp.mount unit, making /tmp RAM-backed and size-limited rather than the +# disk-backed directory BOSH stemcells have historically shipped. That can +# surprise jobs that write large temp files to /tmp (it competes with VM RAM). +# Mask tmp.mount so /tmp stays a regular directory on the root filesystem, +# preserving the pre-systemd-259 behaviour. (/tmp keeps the conventional 1777 +# permissions applied by systemd-tmpfiles; jobs should still use +# /var/vcap/data/tmp for scratch space.) +run_in_chroot $chroot "systemctl mask tmp.mount" diff --git a/stemcell_builder/stages/bosh_users/apply.sh b/stemcell_builder/stages/bosh_users/apply.sh index 6afe426c05..5fb51f9617 100755 --- a/stemcell_builder/stages/bosh_users/apply.sh +++ b/stemcell_builder/stages/bosh_users/apply.sh @@ -9,6 +9,9 @@ source $base_dir/lib/prelude_bosh.bash # Set up users/groups vcap_user_groups='admin,adm,audio,cdrom,dialout,floppy,video,bosh_sshers,dip,plugdev' +# Ensure cracklib dictionary exists before chpasswd (avoids "error loading dictionary" warnings). +run_in_chroot $chroot "cracklib-update" + run_in_chroot $chroot " groupadd --system admin groupadd -f vcap diff --git a/stemcell_builder/stages/bosh_users/assets/sudoers b/stemcell_builder/stages/bosh_users/assets/sudoers index d2f7795cc6..e7fdcd0694 100644 --- a/stemcell_builder/stages/bosh_users/assets/sudoers +++ b/stemcell_builder/stages/bosh_users/assets/sudoers @@ -3,7 +3,7 @@ # See the man page for details on how to write a sudoers file. # Defaults -Defaults !lecture,tty_tickets,!fqdn +Defaults !lecture,!fqdn # Uncomment to allow members of group sudo to not need a password # %sudo ALL=NOPASSWD: ALL diff --git a/stemcell_builder/stages/password_policies/apply.sh b/stemcell_builder/stages/password_policies/apply.sh index bf9753b841..1a5717dd8e 100755 --- a/stemcell_builder/stages/password_policies/apply.sh +++ b/stemcell_builder/stages/password_policies/apply.sh @@ -38,4 +38,6 @@ patch $chroot/etc/pam.d/login < $assets_dir/ubuntu/login.patch # /etc/login.defs are only effective for new users sed -i -r 's/^PASS_MIN_DAYS.+/PASS_MIN_DAYS 1/' $chroot/etc/login.defs +# Ensure ENCRYPT_METHOD matches the sha512 used in pam_unix.so (Resolute defaults to YESCRYPT) +sed -i -r 's/^ENCRYPT_METHOD .+/ENCRYPT_METHOD SHA512/' $chroot/etc/login.defs run_in_chroot $chroot "chage --mindays 1 vcap" diff --git a/stemcell_builder/stages/password_policies/assets/ubuntu/common-auth.patch b/stemcell_builder/stages/password_policies/assets/ubuntu/common-auth.patch index 80fe028b9b..dd1b02b10a 100644 --- a/stemcell_builder/stages/password_policies/assets/ubuntu/common-auth.patch +++ b/stemcell_builder/stages/password_policies/assets/ubuntu/common-auth.patch @@ -8,4 +8,4 @@ +auth sufficient pam_faillock.so authsucc audit deny=3 unlock_time=604800 fail_interval=900 # here's the fallback if no module succeeds auth requisite pam_deny.so - # prime the stack with a positive return value if there isn't one already; \ No newline at end of file + # prime the stack with a positive return value if there isn't one already; diff --git a/stemcell_builder/stages/password_policies/assets/ubuntu/common-password.patch b/stemcell_builder/stages/password_policies/assets/ubuntu/common-password.patch index d1bc57e2ca..c74257f2e0 100644 --- a/stemcell_builder/stages/password_policies/assets/ubuntu/common-password.patch +++ b/stemcell_builder/stages/password_policies/assets/ubuntu/common-password.patch @@ -13,5 +13,5 @@ # since the modules above will each just jump around password required pam_permit.so # and here are more per-package modules (the "Additional" block) -+#session optional pam_lastlog2.so showfailed #NOBLE_TODO: this will only work if util-linux =>2.40 which provide pam_lastlog2.so or if users will install it manually ++session optional pam_lastlog2.so showfailed # end of pam-auth-update config diff --git a/stemcell_builder/stages/restrict_su_command/apply.sh b/stemcell_builder/stages/restrict_su_command/apply.sh index 287350f2de..208fcd859e 100755 --- a/stemcell_builder/stages/restrict_su_command/apply.sh +++ b/stemcell_builder/stages/restrict_su_command/apply.sh @@ -5,9 +5,12 @@ set -ex base_dir=$(readlink -nf $(dirname $0)/../..) source $base_dir/lib/prelude_apply.bash -# restrict Access to the su Command add the following line to the /etc/pam.d/su file. -# auth required pam_wheel.so use_uid add vcap user to 'root' group +# CIS-9.5: restrict access to the su command via pam_wheel. +# pam_wheel.so use_uid requires the calling user to be in the wheel group. +# Ubuntu does not ship a wheel group, so we create it and add root + vcap. run_in_chroot $chroot " - sudo echo 'auth required pam_wheel.so use_uid' >> /etc/pam.d/su - sudo usermod -aG sudo vcap + groupadd -f wheel + usermod -aG wheel root + usermod -aG wheel,sudo vcap + echo 'auth required pam_wheel.so use_uid' >> /etc/pam.d/su " diff --git a/stemcell_builder/stages/rsyslog_config/assets/rsyslog.conf b/stemcell_builder/stages/rsyslog_config/assets/rsyslog.conf index cd6afe802a..f6f3710982 100644 --- a/stemcell_builder/stages/rsyslog_config/assets/rsyslog.conf +++ b/stemcell_builder/stages/rsyslog_config/assets/rsyslog.conf @@ -15,7 +15,7 @@ $ModLoad imuxsock # provides support for local system logging $SystemLogSocketName /run/systemd/journal/syslog $ModLoad imklog # provides kernel logging support (previously done by rklogd) -module( load="omrelp" tls.tlslib="openssl" ) +module( load="omrelp" tls.tlslib="gnutls" ) #$ModLoad immark # provides --MARK-- message capability # provides UDP syslog reception diff --git a/stemcell_builder/stages/static_libraries_config/apply.sh b/stemcell_builder/stages/static_libraries_config/apply.sh index a7ce78e70b..a48241110d 100755 --- a/stemcell_builder/stages/static_libraries_config/apply.sh +++ b/stemcell_builder/stages/static_libraries_config/apply.sh @@ -10,7 +10,7 @@ source $base_dir/etc/settings.bash cp -p "${assets_dir}/static_libraries_list.txt" $chroot/var/vcap/bosh/etc/static_libraries_list kernel_suffix="-generic" -major_kernel_version="6.8" +major_kernel_version="7.0" if [[ "${stemcell_operating_system_variant}" == 'fips' ]]; then # TODO use iaas specific kernel diff --git a/stemcell_builder/stages/static_libraries_config/assets/static_libraries_list.txt b/stemcell_builder/stages/static_libraries_config/assets/static_libraries_list.txt index a7185b4774..92ec0497db 100644 --- a/stemcell_builder/stages/static_libraries_config/assets/static_libraries_list.txt +++ b/stemcell_builder/stages/static_libraries_config/assets/static_libraries_list.txt @@ -19,14 +19,36 @@ /usr/lib/gcc/x86_64-linux-gnu/13/libsupc++.a /usr/lib/gcc/x86_64-linux-gnu/13/libtsan.a /usr/lib/gcc/x86_64-linux-gnu/13/libubsan.a +/usr/lib/gcc/x86_64-linux-gnu/15/libasan.a +/usr/lib/gcc/x86_64-linux-gnu/15/libatomic.a +/usr/lib/gcc/x86_64-linux-gnu/15/libbacktrace.a +/usr/lib/gcc/x86_64-linux-gnu/15/libgcc.a +/usr/lib/gcc/x86_64-linux-gnu/15/libgcc_eh.a +/usr/lib/gcc/x86_64-linux-gnu/15/libgcov.a +/usr/lib/gcc/x86_64-linux-gnu/15/libgomp.a +/usr/lib/gcc/x86_64-linux-gnu/15/libhwasan.a +/usr/lib/gcc/x86_64-linux-gnu/15/libitm.a +/usr/lib/gcc/x86_64-linux-gnu/15/liblsan.a +/usr/lib/gcc/x86_64-linux-gnu/15/libquadmath.a +/usr/lib/gcc/x86_64-linux-gnu/15/libssp_nonshared.a +/usr/lib/gcc/x86_64-linux-gnu/15/libstdc++.a +/usr/lib/gcc/x86_64-linux-gnu/15/libstdc++exp.a +/usr/lib/gcc/x86_64-linux-gnu/15/libstdc++fs.a +/usr/lib/gcc/x86_64-linux-gnu/15/libsupc++.a +/usr/lib/gcc/x86_64-linux-gnu/15/libtsan.a +/usr/lib/gcc/x86_64-linux-gnu/15/libubsan.a /usr/lib/libsupp.a /usr/lib/x86_64-linux-gnu/gprofng/libgp-collectorAPI.a /usr/lib/x86_64-linux-gnu/libBrokenLocale.a /usr/lib/x86_64-linux-gnu/libanl.a +/usr/lib/x86_64-linux-gnu/libbrotlicommon.a +/usr/lib/x86_64-linux-gnu/libbrotlidec.a +/usr/lib/x86_64-linux-gnu/libbrotlienc.a /usr/lib/x86_64-linux-gnu/libbz2.a /usr/lib/x86_64-linux-gnu/libc.a /usr/lib/x86_64-linux-gnu/libc_nonshared.a /usr/lib/x86_64-linux-gnu/libcap.a +/usr/lib/x86_64-linux-gnu/libcom_err.a /usr/lib/x86_64-linux-gnu/libcrypt.a /usr/lib/x86_64-linux-gnu/libcrypto.a /usr/lib/x86_64-linux-gnu/libcurl.a @@ -36,22 +58,37 @@ /usr/lib/x86_64-linux-gnu/libformw.a /usr/lib/x86_64-linux-gnu/libg.a /usr/lib/x86_64-linux-gnu/libgcrypt.a +/usr/lib/x86_64-linux-gnu/libgmp.a +/usr/lib/x86_64-linux-gnu/libgmpxx.a +/usr/lib/x86_64-linux-gnu/libgnutls.a +/usr/lib/x86_64-linux-gnu/libgnutls-dane.a +/usr/lib/x86_64-linux-gnu/libgnutls-openssl.a /usr/lib/x86_64-linux-gnu/libgpg-error.a /usr/lib/x86_64-linux-gnu/libhistory.a +/usr/lib/x86_64-linux-gnu/libhogweed.a /usr/lib/x86_64-linux-gnu/libicudata.a /usr/lib/x86_64-linux-gnu/libicui18n.a /usr/lib/x86_64-linux-gnu/libicuio.a /usr/lib/x86_64-linux-gnu/libicutest.a /usr/lib/x86_64-linux-gnu/libicutu.a /usr/lib/x86_64-linux-gnu/libicuuc.a +/usr/lib/x86_64-linux-gnu/libidn2.a +/usr/lib/x86_64-linux-gnu/liblber.a +/usr/lib/x86_64-linux-gnu/libldap.a +/usr/lib/x86_64-linux-gnu/libldap_r.a +/usr/lib/x86_64-linux-gnu/liblzma.a /usr/lib/x86_64-linux-gnu/libm-2.35.a /usr/lib/x86_64-linux-gnu/libm-2.38.a /usr/lib/x86_64-linux-gnu/libm-2.39.a +/usr/lib/x86_64-linux-gnu/libm-2.42.a +/usr/lib/x86_64-linux-gnu/libm-2.43.a /usr/lib/x86_64-linux-gnu/libm.a /usr/lib/x86_64-linux-gnu/libmcheck.a /usr/lib/x86_64-linux-gnu/libmenu.a /usr/lib/x86_64-linux-gnu/libmenuw.a /usr/lib/x86_64-linux-gnu/libmvec.a +/usr/lib/x86_64-linux-gnu/libnettle.a +/usr/lib/x86_64-linux-gnu/libnghttp2.a /usr/lib/x86_64-linux-gnu/libncurses++.a /usr/lib/x86_64-linux-gnu/libncurses++w.a /usr/lib/x86_64-linux-gnu/libncurses.a @@ -59,21 +96,26 @@ /usr/lib/x86_64-linux-gnu/libnsl.a /usr/lib/x86_64-linux-gnu/libpanel.a /usr/lib/x86_64-linux-gnu/libpanelw.a +/usr/lib/x86_64-linux-gnu/libpsl.a /usr/lib/x86_64-linux-gnu/libpsx.a /usr/lib/x86_64-linux-gnu/libpthread.a /usr/lib/x86_64-linux-gnu/libpthread_nonshared.a /usr/lib/x86_64-linux-gnu/libreadline.a /usr/lib/x86_64-linux-gnu/libresolv.a /usr/lib/x86_64-linux-gnu/librt.a +/usr/lib/x86_64-linux-gnu/librtmp.a +/usr/lib/x86_64-linux-gnu/libssh2.a /usr/lib/x86_64-linux-gnu/libssl.a /usr/lib/x86_64-linux-gnu/libtermcap.a /usr/lib/x86_64-linux-gnu/libtic.a /usr/lib/x86_64-linux-gnu/libtinfo.a +/usr/lib/x86_64-linux-gnu/libtasn1.a /usr/lib/x86_64-linux-gnu/libtirpc.a /usr/lib/x86_64-linux-gnu/libutil.a /usr/lib/x86_64-linux-gnu/libuuid.a /usr/lib/x86_64-linux-gnu/libxml2.a /usr/lib/x86_64-linux-gnu/libz.a +/usr/lib/x86_64-linux-gnu/libzstd.a /usr/src/linux-headers-__KERNEL_VERSION__/tools/bpf/resolve_btfids/libbpf/libbpf.a /usr/src/linux-headers-__KERNEL_VERSION__/tools/bpf/resolve_btfids/libsubcmd/libsubcmd.a /usr/src/linux-headers-__KERNEL_VERSION__/tools/objtool/libsubcmd/libsubcmd.a diff --git a/stemcell_builder/stages/system_grub/apply.sh b/stemcell_builder/stages/system_grub/apply.sh index a544963995..7657c2fcf8 100755 --- a/stemcell_builder/stages/system_grub/apply.sh +++ b/stemcell_builder/stages/system_grub/apply.sh @@ -5,7 +5,7 @@ set -e base_dir=$(readlink -nf $(dirname $0)/../..) source $base_dir/lib/prelude_apply.bash -pkg_mgr install grub2 grub-efi-amd64-bin +pkg_mgr install grub-pc-bin grub-efi-amd64-bin # When a kernel is installed, update-grub is run per /etc/kernel-img.conf. # It complains when /boot/grub/menu.lst doesn't exist, so create it. diff --git a/stemcell_builder/stages/system_kernel/apply.sh b/stemcell_builder/stages/system_kernel/apply.sh index 9790b92192..3ed180aa2e 100755 --- a/stemcell_builder/stages/system_kernel/apply.sh +++ b/stemcell_builder/stages/system_kernel/apply.sh @@ -16,4 +16,8 @@ mkdir -p $chroot/tmp ## and later), don't switch to the HWE kernels unless you have a good reason and ## have discussed it with the rest of the team, or whoever is currently responsible ## for the Linux Stemcell. -pkg_mgr install initramfs-tools linux-generic +# +# linux-firmware-minimal Provides/Conflicts linux-firmware so we avoid the full +# vendor firmware meta-package while keeping stock archive packages for microcode +# and wireless-regdb (see docs/resolute-dev.md). +pkg_mgr install initramfs-tools linux-firmware-minimal linux-generic