Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,7 @@ acceptance-tests/syslog-release
acceptance-tests/os-conf-release

ci/docker/os-image-stemcell-builder/VMware-ovftool-*.bundle
ci/docker/VMware-ovftool-*.bundle
ci/docker/*/VMware-ovftool-*.bundle

tmp/
tmp/
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ an "Ubuntu SHORT_NAME" based stemcell will be on the branch:
As of `2026-04-03` the following stemcell lines / branches are supported:
- Ubuntu Jammy / `ubuntu-jammy`
- Ubuntu Noble / `ubuntu-noble`
- Ubuntu Resolute / `ubuntu-resolute`
6 changes: 3 additions & 3 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
source 'https://rubygems.org'
source "https://rubygems.org"

gem 'rake'
gem 'bosh-stemcell', path: 'bosh-stemcell'
gem "rake"
gem "bosh-stemcell", path: "bosh-stemcell"
46 changes: 30 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,30 @@
This repo contains tools for creating BOSH stemcells. A stemcell is a bootable
disk image that is used as a template by a BOSH Director to create VMs.

This branch builds stemcells for **Ubuntu 26.04 LTS (Resolute)**. For other
Ubuntu releases, switch to the appropriate branch (for example `ubuntu-noble`
for 24.04).

## Quick Start: Building a Stemcell Locally

```bash
export short_name="noble"
export short_name="resolute"

git clone git@github.com:cloudfoundry/bosh-linux-stemcell-builder.git
cd bosh-linux-stemcell-builder
git checkout ubuntu-${short_name}
mkdir -p tmp
docker build \
--platform linux/amd64 \
--build-arg SYFT_VERSION=v1.42.3 \
--build-arg BASE_IMAGE="ubuntu:${short_name}" \
--build-arg META4_CLI_URL="https://github.com/dpb587/metalink/releases/download/v0.5.0/meta4-0.5.0-linux-amd64" \
--build-arg SYFT_CLI_URL="https://github.com/anchore/syft/releases/download/v1.42.3/syft_1.42.3_linux_amd64.tar.gz" \
--build-arg YQ_CLI_URL="https://github.com/mikefarah/yq/releases/download/v4.52.5/yq_linux_amd64" \
--build-arg RUBY_INSTALL_URL="https://github.com/postmodern/ruby-install/releases/download/v0.10.2/ruby-install-0.10.2.tar.gz" \
--build-arg RUBY_VERSION="$(cat .ruby-version)" \
--build-arg GEM_HOME="/usr/local/bundle" \
--build-arg OVF_TOOL_INSTALLER="VMware-ovftool-4.4.3-18663434-lin.x86_64.bundle" \
--build-arg OVF_TOOL_INSTALLER_SHA1="6c24e473be49c961cfc3bb16774b52b48e822991" \
-t bosh/os-image-stemcell-builder:${short_name} \
ci/docker/os-image-stemcell-builder/
docker run \
Expand All @@ -32,10 +44,13 @@ gem install bundler
bundle install

# build OS image
bundle exec rake stemcell:build_os_image[ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz]
bundle exec rake stemcell:build_os_image[ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image_${short_name}.tgz]

# build vSphere stemcell
bundle exec rake stemcell:build[vsphere,esxi,ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz]
bundle exec rake stemcell:build[vsphere,esxi,ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image_${short_name}.tgz,9.000]

# build warden (BOSH Lite) stemcell
bundle exec rake stemcell:build_with_local_os_image[warden,warden,ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image_${short_name}.tgz,9.000]
```

When building a vSphere stemcell, you must download `VMware-ovftool-*.bundle`
Expand All @@ -56,7 +71,7 @@ installed in the operating system or when making changes to the configuration
of those packages.

```bash
export short_name="noble"
export short_name="resolute"

bundle exec rake stemcell:build_os_image[ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz]
```
Expand All @@ -78,7 +93,7 @@ The arguments to the `stemcell:build_os_image` rake task follow:
Rebuild the stemcell when you are making and testing BOSH-specific changes such as a new BOSH agent.

```bash
export short_name="noble"
export short_name="resolute"
export build_number="0.0.8"

bundle exec rake stemcell:build[vsphere,esxi,ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz,${build_number}]
Expand All @@ -95,6 +110,7 @@ The arguments to `stemcell:build` are:
- `google` → `kvm`
- `openstack` → `kvm`
- `vsphere` → `esxi`
- `warden` → `warden`
3. `operating_system_name` (`ubuntu`): Type of OS. Same as
`stemcell:build_os_image`.
4. `operating_system_version` (`<short_name>`): OS release. Same as
Expand All @@ -114,7 +130,7 @@ the stemcell would be at
upload the stemcell to a vSphere BOSH Director:

```bash
export short_name="noble"
export short_name="resolute"

bosh upload-stemcell tmp/bosh-stemcell-0.0.8-vsphere-esxi-ubuntu-${short_name}-go_agent.tgz
```
Expand All @@ -130,15 +146,14 @@ the rake task the first time you create your docker container, but everytime
after, as long as you do not destroy the container, you should be able to run
the specific tests.

To run the `ubuntu_${short_name}_spec.rb` tests (**assuming you've already built
To run the OS image tests in `spec/os_image/ubuntu_spec.rb` (**assuming you've already built
the OS image** at the `tmp/ubuntu_base_image.tgz` and you're within the Docker
container):

```shell
export short_name="noble"
cd /opt/bosh/bosh-stemcell
bundle install
OS_IMAGE=/opt/bosh/tmp/ubuntu_base_image.tgz bundle exec rspec -fd spec/os_image/ubuntu_${short_name}_spec.rb
OS_IMAGE=/opt/bosh/tmp/ubuntu_base_image.tgz bundle exec rspec -fd spec/os_image/ubuntu_spec.rb
```

### How to Run Tests for Stemcell
Expand Down Expand Up @@ -208,7 +223,7 @@ If you find yourself debugging any of the above processes, here is what you need
Example usage:

```shell
export short_name="noble"
export short_name="resolute"

bundle exec rake stemcell:build_os_image[ubuntu,${short_name},${PWD}/tmp/ubuntu_base_image.tgz] resume_from=rsyslog_config
```
Expand All @@ -219,8 +234,8 @@ If you find yourself debugging any of the above processes, here is what you need
modifications you can rerun the tests (without rebuilding OS image). Details
in section `How to run tests for OS Images`
* If the Stemcell has been built, and you are only updating tests, you do not
need to re-build the stemcell. You can simply rerun the tests - without
rebuilding Stemcell. Details in section `How to run tests for Stemcell`
need to re-build the stemcell. You can simply rerun the tests (without
rebuilding Stemcell). Details in section `How to run tests for Stemcell`
* It's possible to verify OS/Stemcell changes without making a deployment using
the stemcell. For a vSphere-specific Ubuntu stemcell, the filesystem is
available at `/mnt/stemcells/vsphere/esxi/ubuntu/work/work/chroot`
Expand Down Expand Up @@ -249,7 +264,7 @@ You will need the ovftool installer present in
Rebuild the container with:

```shell
export short_name="noble"
export short_name="resolute"

docker build \
--platform linux/amd64 \
Expand Down Expand Up @@ -280,7 +295,7 @@ gsutil cp MY_OVFTOOL_FILE gs://bosh-vmware-ovftool/MY_OS/
Example:

```shell
export short_name="noble"
export short_name="resolute"

gsutil cp VMware-ovftool-4.4.3-18663434-lin.x86_64.bundle gs://bosh-vmware-ovftool/${short_name}/
```
Expand Down Expand Up @@ -387,4 +402,3 @@ When switching from the old pipeline to the new one, don't forget to:
whatever the public bucket should be
* update the tasks YAML to point to tasks in the `os-images` directory
* rename this directory from `new`

203 changes: 103 additions & 100 deletions Rakefile
Original file line number Diff line number Diff line change
@@ -1,111 +1,114 @@
require 'json'
require "json"

namespace :stemcell do
desc 'Build a base OS image for use in stemcells'
desc "Build a base OS image for use in stemcells"
task :build_os_image, [:operating_system_name, :operating_system_version, :os_image_path] do |_, args|
begin
require 'bosh/stemcell/archive_handler'
require 'bosh/stemcell/build_environment'
require 'bosh/stemcell/definition'
require 'bosh/stemcell/os_image_builder'
require 'bosh/stemcell/stage_collection'
require 'bosh/stemcell/stage_runner'

definition = Bosh::Stemcell::Definition.for('null', 'null', args.operating_system_name, args.operating_system_version)
environment = Bosh::Stemcell::BuildEnvironment.new(
ENV.to_hash,
definition,
'',
args.os_image_path,
)
collection = Bosh::Stemcell::StageCollection.new(definition)
runner = Bosh::Stemcell::StageRunner.new(
build_path: environment.build_path,
command_env: environment.command_env,
settings_file: environment.settings_path,
work_path: environment.work_path,
)
archive_handler = Bosh::Stemcell::ArchiveHandler.new

builder = Bosh::Stemcell::OsImageBuilder.new(
environment: environment,
collection: collection,
runner: runner,
archive_handler: archive_handler,
)
builder.build(args.os_image_path)

sh(environment.os_image_rspec_command)
rescue RuntimeError => e
print_help
raise e
require "bosh/stemcell/archive_handler"
require "bosh/stemcell/build_environment"
require "bosh/stemcell/definition"
require "bosh/stemcell/os_image_builder"
require "bosh/stemcell/stage_collection"
require "bosh/stemcell/stage_runner"

os_image_path = File.expand_path(args.os_image_path)
if args.operating_system_version.to_s.strip.empty?
raise "stemcell:build_os_image: operating_system_version (2nd argument) is empty. " \
"Set it to the Ubuntu release codename (for example export short_name=resolute per README), " \
"or pass it literally: rake stemcell:build_os_image[ubuntu,resolute,tmp/os.tgz]"
end
definition = Bosh::Stemcell::Definition.for("null", "null", args.operating_system_name, args.operating_system_version)
environment = Bosh::Stemcell::BuildEnvironment.new(
ENV.to_hash,
definition,
"",
os_image_path
)
collection = Bosh::Stemcell::StageCollection.new(definition)
runner = Bosh::Stemcell::StageRunner.new(
build_path: environment.build_path,
command_env: environment.command_env,
settings_file: environment.settings_path,
work_path: environment.work_path
)
archive_handler = Bosh::Stemcell::ArchiveHandler.new

builder = Bosh::Stemcell::OsImageBuilder.new(
environment: environment,
collection: collection,
runner: runner,
archive_handler: archive_handler
)
builder.build(os_image_path)

sh(environment.os_image_rspec_command)
rescue RuntimeError => e
print_help
raise e
end

desc 'Build a stemcell, requires `os_image_path` pointing at an image created via `stemcell:build_os_image`'
desc "Build a stemcell, requires `os_image_path` pointing at an image created via `stemcell:build_os_image`"
task :build, [:infrastructure_name, :hypervisor_name, :operating_system_name, :operating_system_version, :os_image_path, :build_number] do |_, args|
begin
require 'bosh/stemcell/build_environment'
require 'bosh/stemcell/definition'
require 'bosh/stemcell/stage_collection'
require 'bosh/stemcell/stage_runner'
require 'bosh/stemcell/stemcell_packager'
require 'bosh/stemcell/stemcell_builder'

args.with_defaults(build_number: '0000')

definition = Bosh::Stemcell::Definition.for(args.infrastructure_name, args.hypervisor_name, args.operating_system_name, args.operating_system_version)
environment = Bosh::Stemcell::BuildEnvironment.new(
ENV.to_hash,
definition,
args.build_number,
args.os_image_path,
)

sh(environment.os_image_rspec_command)

puts "Working from #{environment.work_path}..."
puts "########################################"
runner = Bosh::Stemcell::StageRunner.new(
build_path: environment.build_path,
command_env: environment.command_env,
settings_file: environment.settings_path,
work_path: environment.work_path,
)

stemcell_building_stages = Bosh::Stemcell::StageCollection.new(definition)

builder = Bosh::Stemcell::StemcellBuilder.new(
environment: environment,
runner: runner,
definition: definition,
collection: stemcell_building_stages
)

packager = Bosh::Stemcell::StemcellPackager.new(
definition: definition,
version: environment.version,
work_path: environment.work_path,
tarball_path: environment.stemcell_tarball_path,
disk_size: environment.stemcell_disk_size,
runner: runner,
collection: stemcell_building_stages,
)

builder.build

mkdir_p('tmp')
definition.disk_formats.each do |disk_format|
puts "Packaging #{disk_format}..."
stemcell_tarball = packager.package(disk_format)
cp(stemcell_tarball, 'tmp')
end

sh(environment.stemcell_rspec_command)
rescue RuntimeError => e
print_help
raise e
require "bosh/stemcell/build_environment"
require "bosh/stemcell/definition"
require "bosh/stemcell/stage_collection"
require "bosh/stemcell/stage_runner"
require "bosh/stemcell/stemcell_packager"
require "bosh/stemcell/stemcell_builder"

args.with_defaults(build_number: "0000")

os_image_path = File.expand_path(args.os_image_path)
definition = Bosh::Stemcell::Definition.for(args.infrastructure_name, args.hypervisor_name, args.operating_system_name, args.operating_system_version)
environment = Bosh::Stemcell::BuildEnvironment.new(
ENV.to_hash,
definition,
args.build_number,
os_image_path
)

sh(environment.os_image_rspec_command)

puts "Working from #{environment.work_path}..."
puts "########################################"
runner = Bosh::Stemcell::StageRunner.new(
build_path: environment.build_path,
command_env: environment.command_env,
settings_file: environment.settings_path,
work_path: environment.work_path
)

stemcell_building_stages = Bosh::Stemcell::StageCollection.new(definition)

builder = Bosh::Stemcell::StemcellBuilder.new(
environment: environment,
runner: runner,
definition: definition,
collection: stemcell_building_stages
)

packager = Bosh::Stemcell::StemcellPackager.new(
definition: definition,
version: environment.version,
work_path: environment.work_path,
tarball_path: environment.stemcell_tarball_path,
disk_size: environment.stemcell_disk_size,
runner: runner,
collection: stemcell_building_stages
)

builder.build

mkdir_p("tmp")
definition.disk_formats.each do |disk_format|
puts "Packaging #{disk_format}..."
stemcell_tarball = packager.package(disk_format)
cp(stemcell_tarball, "tmp")
end

sh(environment.stemcell_rspec_command)
rescue RuntimeError => e
print_help
raise e
end

def print_help
Expand Down
Loading
Loading