add type3 spec

[Samueru-sama]

I saw TheAssassin made this repo but nothing has been added, so instead of waiting here is my two cents of what a type3 AppImage must be.

Note: This is all already supported by the uruntime minus the ARG0 change which is something I want to get upstreamed first.

Rationale behind some of these changes:

---

• The mountpoint SHOULD be a unique hash of the AppImage and not random every time.

The reason is that this allows reusing already running AppImages instead of creating multiple mountpoints which wastes resources. Fixes AppImage/AppImageKit#1100

By making it a hash of the AppImage we avoid any possible conflict with other AppImages and make sure the mounpoint is reused by the same application only.

---

• MUST set the following enviroment variables:
  • APPIMAGE Which points to the location of the AppImage with symlinks followed
  • ARG0 Which points to the location of the AppImage without following symlinks
  • OWD Path to the working directory when the AppImage was executed

The reason I have changed ARGV0 for ARG0 is becasue ARGV0 causes a ton of problems with zsh to the point that I've seen projects code workarounds for this in their code instead of just unsetting the variable manually in the AppRun which that on its own is a terrible hack.

Even I that I'm aware of this problem have been bitten by it multiple times, the most recent one was this, which was fixed previously...

---

• The AppImage MUST also preserve the original values of HOME, XDG_DATA_HOME, XDG_CONFIG_HOME and XDG_CACHE_HOME by setting the following envrioment variables:
  * REAL_HOME To the value of HOME before chaging it
  * REAL_XDG_DATA_HOME To the value of XDG_DATA_HOME before chaging it
  * REAL_XDG_CONFIG_HOME To the value of XDG_CONFIG_HOME before changing it
  * REAL_XDG_CACHA_HOME To the value of XDG_CACHE_HOME before changing it
  • If the REAL_* enviroment variables are already set, the AppImage MUST NOT change their value
  • If XDG enviroment variables are empty, the AppImage MUST set the REAL_* variables to the defaults specificed in the XDG Base directory specification

By keeping the original values of these enviroment variables, the application can restore them before executing child processes. This fixes the issue of other apps using the portable directories of the AppImage.

Since the uruntime does this already, we have this xdg-open wrapper with all of our sharun AppImages that restores the enviroment variables. For applications like terminal emulators I preload a library that does the same.

snap also does something similar already btw

---

Now onto things I don't think need to be changed.

I don't think it is a good idea to mandate a specific filesystem or compression algo, because maybe in 5 years from now some new format that is much better comes out and we are not able to use it.

Instead it is better to mandate specific flags into the AppImage runtime, that way tools like libappimage can instead falllback to using --appimage-extract for example, and this can be done easily in a sandbox

This is also why --appimage-extract must extract to an AppDir directory instead of squashfs-root, since the filesystem may be completely different, the AppImage can instead make a squashfs-root symlink to keep compatiblity with existing scripts/tools that expect a squashfs-root directory.

I also don't think we need to change the magic bytes, since most of these changes do not break backwards compatibility with existing tools, for example appimageupdatetool has no problem updating an AppImage that follows this type3 spec.

[probonopd]

This requires a more detailed discussion. Just some key points:

• We do need to move the magic bytes since the current location is known to cause real life issues (blame me for choosing the current location, which turns out to be suboptimal)
• We do not want to adopt even more from XDG/fd.o - to the contrary, I'd like to decrease the use of XDG/fd.o specs, which I consider to be inelegant
• Sooner or later (in 3 or 4) we will want to specify the exact filesystem/compression, in order to allow e.g., libraries and tools to extract files without having to execute AppImages

[Samueru-sama]

• We do need to move the magic bytes since the current location is known to cause real life issues (blame me for choosing the current location, which turns out to be suboptimal)

What are the issues exactly?

We do not want to adopt even more from XDG/fd.o - to the contrary, I'd like to decrease the use of XDG/fd.o specs, which I consider to be inelegant

You already set XDG_CONFIG_HOME and HOME in the type2-runtime, this change just adds the rest of variables and preserves them.

I also don't know what you mean by "being inelegant", thanks to variables like XDG_DATA_DIRS you don't have a ton of applications hardcoded to look into /usr/share and are easy to package as result.

Sooner or later (in 3 or 4) we will want to specify the exact filesystem/compression, in order to allow e.g., libraries and tools to extract files without having to execute AppImages

Alright but not adding fallback to --appimage-extract and not mandating the flag sounds like a terrible idea.

[probonopd]

What are the issues exactly?

For example, AppImages with these magic bytes don't run in certain environments. See the many magic-related issues.

You already set XDG_CONFIG_HOME and HOME in the type2-runtime

$HOME is a proper Unix standard. I was thinking about removing the XDG one, as it is rarely ever used and is confusing for users.

I also don't know what you mean by "being inelegant"

"Thanks" to XDG specifications, we have to deal with inelegant solutions like copying around desktop files and icon files. Neither of which is necessary on Windows nor the Mac.

thanks to variables like XDG_DATA_DIRS you don't have a ton of applications hardcoded to look into /usr/share and are easy to package as result.

If all apps actually really worked that way, maybe. But currently few do.

Alright but not adding fallback to --appimage-extract and not mandating the flag sounds like a terrible idea.

Well, just looking at an AppImage in the file manager should never lead to the AppImage being executed. After all, it could be really anything that gets executed.

[Samueru-sama]

$HOME is a proper Unix standard. I was thinking about removing the XDG one, as it is rarely ever used and is confusing for users.

No it is not confusing to users. The variable even says it in its name, it is for configuration files.

It is also not rarely used, I use it everyday and we had to include the other XDG variables because of requests. 1 2

If all apps actually really worked that way, maybe. But currently few do.

• Qt

• Glib (which is used by GTK apps)

• SDL they just check XDG_DATA_HOME instead.

• Vulkan-loader (used by mesa to load the vulkan gpu drivers).

"Thanks" to XDG specifications, we have to deal with inelegant solutions like copying around desktop files and icon files. Neither of which is necessary on Windows nor the Mac.

That's the Desktop entry spec, different thing. The XDG Base dir spec is also not that different from what MacOS does.

After all, it could be really anything that gets executed.

Do it in a sandbox then, it is going to be 15 lines of POSIX shell.

Nautilus actually sandboxes all thumbnailers for this reason, and more recently GNOME is working on something called glycin which sandboxes its binaries with bwrap to do svg rendering because with svg anything could be executed.

[probonopd]

If we ever do a type 3 spec, we should probably store icons and similar metadata out of the compressed filesystem, so that AppImages could use any file format for that and still be compliant, with file managers still being able to access icons etc.

[Samueru-sama]

If we ever do a type 3 spec, we should probably store icons and similar metadata out of the compressed filesystem, so that AppImages could use any file format for that and still be compliant, with file managers still being able to access icons etc.

there is no need to do this since the appimage runtime supports the --appimage-extract flag with simple globbing.

Just run --appimage-extract *.desktop and --appimage-extract .DirIcon as a fallback automatically, it is a few lines of shell, and this can be done safely inside a sandbox via bwrap or directly with a feature of the kernel like landlock if you don't like bwrap, though it is not longer simple if you go this route.

I will also add I don't have energy to be arguing about this anymore, @probonopd likes to disappear for a while and then come back forgetting everything that has been talked about before.

[probonopd]

File managers must be able to extract e.g., the icon and other metadata without executing code from the AppImage for security reasons.

[Samueru-sama]

File managers must be able to extract e.g., the icon and other metadata without executing code from the AppImage for security reasons.

For security reasons you shouldn't even be parsing any random media from the appimage to begin with, a while back openai got hacked due to a security issue in libheif, sending a perfectly crafted image allowed for remote code execution.

For the 3rd time, this must be done in a very restrictive sandbox instead.

#!/bin/sh

set -e

APP=$1
APP_NAME=${1##*/}
SBX_DIR="$PWD"/."$APP_NAME".temp.extract.dir.$$
TMP_APP="$SBX_DIR"/"$APP_NAME"
mkdir -p "$SBX_DIR"
shift
bwrap \
	--unshare-all                 \
	--die-with-parent             \
	--tmpfs /tmp                  \
	--bind "$SBX_DIR" "$SBX_DIR"  \
	--bind "$APP" "$TMP_APP"      \
	--chdir "$SBX_DIR"            \
	--                            \
	"$TMP_APP" "$@"
cp -rLT "$SBX_DIR"/squashfs-root "$PWD"/AppDir
rm -rf "$SBX_DIR"

---

To give you an idea of how restrictive this sandbox is, /proc is not even mounted anymore (it used to be when I originally shared the script) the uruntime was recently updated to work on systems without /proc, meanwhile the type2-runtime dies without /proc:

[probonopd]

Lets not have a sandbox as a dependency.

Scientific tests started at https://github.com/probonopd/AppImageTests/.

[Samueru-sama]

Lets not have a sandbox as a dependency.

Why not? bwrap is tiny and can be compiled statically. Not using it literary goes againts any security argument you've been making.

Scientific tests started at https://github.com/probonopd/AppImageTests/.

We use mkdwarfs with --order=path --set-owner 0 --set-group 0 --no-history --no-create-timestamp zstd:level=22 -S26 -B6

[TheAssassin]

Putting in bubblewrap into the runtime that you can't trust by definition doesn't solve the security concerns.

A while back, I suggested sandwiching a .zip archive in between the ELF header and the actual payload. This is straightforward to parse (zip implementations just skip data until they find a zip header, also zip is widely available in most programming languages).

[TheAssassin]

Reading @Samueru-sama's statement again, I think you meant that desktops should use a sandboxed process to parse data. Yes, that would be good. Still the overhead of running an executable is suboptimal. And the runtime parameters were never standardized.

[Samueru-sama]

Putting in bubblewrap into the runtime that you can't trust by definition doesn't solve the security concerns.

bublewrap would not be in the runtime, that makes no sense.

bwrap would be used by the tool that wants to get information from the appimage, that is the appimage thubmnailer would run the example script I just pasted above.

[Samueru-sama]

Still the overhead of running an executable is suboptimal. And the runtime parameters were never standardized.

Well --appimag-extract + simple globbing are 100% followed by the uruntime to be similar to the type2-runtime. I believe the only difference is that the uruntime follows symlinks while the type2-runtime does not.

Worst case scenario nothing happens, the application is not able to execute at all in the sandbox I shared where /proc /lib /usr and /bin are not mounted.

Note that this practice isn't odd, glycin works this way already, it runs every image decoder inside a sandbox via bwrap.

[TheAssassin]

Arguably, for a type 3 runtime, these parameters could be specified. What do you think of the .zip approach, though?

[Samueru-sama]

What do you think of the .zip approach, though?

I don't see it as necessary. It is a breaking change without a good compeling reason.

For example appimageupdate works regardless of the filesystem/compression-algo, it doesn't need to read the payload. People are also able to execute the appimage regardless of any of this which is the most important part.

thumbnailers can just run --appimage-extract .DirIcon automatically as a fallback inside a sandbox, no need to put the icon inside a different archive.

[TheAssassin]

It's hardly a breaking change. --appimage-extract would still work.

[Samueru-sama]

It's hardly a breaking change. --appimage-extract would still work.

Well it depends, we actually had trouble with the uruntime before because it used to be an UPXed binary prepended to the payload. This confused tools when finding the offset

The uruntime is no longer UPXed.

gluing a different archive to the payload will likely break tools that would have worked otherwise.

[TheAssassin]

I'd solve this problem and add a reliability feature: just add an ELF header whose value is the offset. The current "Measure ELF header size" is anyway a bit of guesswork and time consuming. Reading the section header is just a few lines of Rust code, for instance.