Table Of Contents

Previous topic

CDE streaming mode (experimental)

This Page

Frequently asked questions

PACKAGE PORTABILITY

How portable are CDE packages?

The short answer: CDE packages should run fine on x86 machines with a Linux distro that’s less than 5 years old.

The longer answer: First, the hardware architecture of the source and target machines must be compatible: e.g., you can run x86-64 (64-bit) CDE packages on other x86-64 machines, and you can run i386 (32-bit) packages on both i386 and x86-64 machines.

A good predictor for the likelihood that your package will work on another machine is the age difference between your respective Linux installations. The greater the age gap, the more likely that some library within the package will be incompatible with the other machine’s kernel.

Furthermore, it’s always easier to create a CDE package on an older machine and have it work on a newer machine, since developers try to maintain backwards compatibility. That being said, though, I’ve created packages on 2010-era Linux 2.6 distros and had them work fine on 2006-era Linux 2.6 distros.

From my experience, the absolute limit on portability seems to be the major kernel version: I can’t seem to get programs ported from Linux 2.6 to 2.4, probably because the system call interface differs too much (I can sometimes go from 2.4 to 2.6, though). However, this hasn’t been a problem in practice, since most Linux distros have been using 2.6 kernels since 2005.

CDE package portability is most likely limited by binary incompatibilities with the target machine’s kernel (see Why am I getting a “FATAL: kernel too old” error?).

[Update on 2010-11-18: A user just informed me that by setting the LD_ASSUME_KERNEL environment variable, he was able to create a CDE package on a recent Linux 2.6 machine and execute that package on an old Linux 2.4.9 machine (he used the setting of LD_ASSUME_KERNEL=2.4.1). YMMV, though!]

Why am I getting a “FATAL: kernel too old” error?

You’re getting this error when executing a CDE package because some executable or library within your package requires a kernel version that’s newer than the kernel on your machine. This could happen because the person who created the CDE package did so on a newer machine with libraries that use newer kernel features that your kernel doesn’t support.

To be concrete, let’s say that your friend created a package on her Fedora Core 14 machine, running a Linux 2.6.35.6-45 kernel. If we run the file command on the standard C library (libc) on her machine, we get the following output:

$ file /lib64/libc-2.12.90.so
/lib64/libc-2.12.90.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux),
dynamically linked (uses shared libs), for GNU/Linux 2.6.32, not stripped

Focus on the “for GNU/Linux 2.6.32” substring, which indicates that her C library will not work on any machine with a kernel older than 2.6.32. Thus, any CDE packages she creates on her machine can only be run on machines with at least a Linux 2.6.32 kernel (nearly all packages include libc).

Sadly, I haven’t yet found a way around this hard limit on CDE package portability aside from full-on OS virtualization!

Why am I getting an “illegal instruction” error?

CDE does not perform any sort of CPU emulation, so if the target machine’s x86 processor cannot execute some of the instructions in binaries within your package, then you’re out of luck. This might happen when porting software from, say, an i686 machine with cmov instructions to an older i586 machine that lacks these instructions.

Sometimes these sorts of CPU incompatibility errors manifest in more subtle ways. For example, on some machines libc is installed in the following directory:

/lib/tls/i686/cmov/libc.so.6

This indicates that libc has been compiled for i686 machines with cmov instructions and tls (thread-local storage). When that file is copied into a CDE package, it will be located in:

cde-package/cde-root/lib/tls/i686/cmov/libc.so.6

Now if you move this package to an older i586 machine, then when cde-exec fires up the dynamic linker and tries to look for libc, the following problem arises: The linker will make a system call to find the machine type, and since CDE doesn’t spoof architecture-related syscalls (things break if it tries!), the linker will start looking in sub-directories named i586 to find libc. However, the libc in the package is located within a sub-directory named i686, which will never get inspected on the target machine. Note that even if you moved libc to another directory within the package (e.g., cde-package/cde-root/lib/), you still won’t be able to run its code on an i586 machine due to those pesky “illegal instruction” errors.

What about the portability of cde-exec itself?

Good question! I’ve designed cde-exec to have the minimal dependencies of any Linux program; all it depends on is an old version of libc. In short, cde-exec should be more portable than the program you’re trying to package. But in case it isn’t, then I would suggest to compile cde-exec on the target machine and link against its libc.

For the curious, here are the gcc flags I used to compile a cde-exec binary that links again the oldest possible version of libc on Linux 2.6 (so that it can run on distros from as old as 2006):

-fno-stack-protector -U_FORTIFY_SOURCE -D_GNU_SOURCE -Wl,--hash-style=both

If you run “objdump -x cde-exec | grep GLIBC”, you’ll see the exact version of libc that this binary depends on.

Why am I getting mysterious segfaults on a 2.6.18-238.el5 kernel?

I’ve noticed that some CDE packages created on newer distros (e.g., those with libc-2.11.X) mysteriously segfault when run on a distro like CentOS 5.6 or Red Hat Enterprise Linux (RHEL) 5.6 with a 2.6.18-238.el5 kernel. However, those packages run just fine on the immediately previous release, RHEL/CentOS 5.5 (2.6.18-194.el5 kernel). I can reproduce the segfault by packaging up a program that simply calls the libc-2.11 mkstemp() function, passing in a stack array as its argument.

Unfortunately, I haven’t been able to discover the root cause of this bug, so the only workaround I can suggest is to create your packages on a distro with a kernel/libc version that’s numerically closer to the one running on your Linux 2.6.18-238.el5 machine.

[Update on 2011-07-11: This same problem manifests even when running programs natively without CDE. If you take certain libraries and the dynamic linker from some distros (e.g., Ubuntu 11.04) and move them to, say, CentOS 5.6, then attempting to execute the dynamic linker with the target program and libraries will lead to mysterious segfaults. Since the cause of this problem lies outside of CDE, there is no easy fix that I can implement a fix.]

Can I run 32-bit CDE packages on 64-bit machines?

Yes, you should be able to run 32-bit (i386) CDE packages on reasonably-modern 64-bit (x86-64) Linux machines. Here’s the full run-down on 32-bit and 64-bit compatibility:

  • You can use a 32-bit cde-exec to run 32-bit ELF binaries on either a 32-bit or 64-bit Linux machine. If you created your CDE package on a 32-bit machine, then the cde-exec in your package is probably 32-bit (run file cde-exec to check).
  • You can use a 64-bit cde-exec to run 32-bit and 64-bit ELF binaries on a 64-bit Linux machine.

Let’s draw a table just for fun:

  32-bit cde-exec 64-bit cde-exec
32-bit machine Runs 32-bit binaries Doesn’t run!
64-bit machine Runs 32-bit binaries Runs 32- and 64-bit binaries

PACKAGE COMPLETENESS

How far can CDE execution diverge from the code path(s) originally executed to create that package?

It depends on the structure and complexity of your program. When running within a CDE package, you can modify the program’s behavior to safely diverge from the original run’s code path until it accesses a file that is not in the package.

Programs often load most (or all) of their libraries at start-up time, so you should be able to get the program’s behavior to diverge quite a lot before it tries to load a library or other auxiliary file that it did not load in the original run(s).

What happens if I run cde multiple times on different programs?

Each time you run cde, it inserts files into the cde-package/ sub-directory of your current working directory. This means that if you run cde multiple times on different programs, your package will contain all the files required to run all of those programs (as long as you remain in the same current working directory).

How can I make my CDE packages more complete?

CDE packages are almost always incomplete since CDE can only package up the files that the target program accesses during execution. One way to make packages more complete is to execute the program additional times with different inputs. Another way is to manually copy the appropriate files into the cde-package/cde-root/ directory.

In my experience, copying entire groups of relevant sub-directories into the package using the okapi deep copying tool that comes with CDE works pretty well. See okapi-ing an entire directory for more details.

(Note that rsync -a or similar tools that copy symlinks verbatim might lead to improper behavior when copying symlinks that refer to absolute paths. For example, if a particular symlink refers to /lib/libc.so.6, then when it’s rsync’ed into the CDE package, it will still refer to /lib/libc.so.6, which is incorrect since that’s the target matchine’s libc! That symlink should actually refer to the copy of libc.so.6 within the CDE package, which might be located at a relative path like ../../../lib/libc.so.6. The technique described in okapi-ing an entire directory does not suffer from these limitations.)

Can CDE track dependencies in spawned sub-processes?

Yes, if you run cde on a program that spawns sub-processes (e.g., make spawns sub-processes to do compilation, linking, etc.), CDE will track dependencies in all forked and cloned sub-processes.

Can CDE track dependencies on programs and files accessed via IPC?

Unfortunately, CDE will not track dependencies on (non-child) processes that your program communicates with via IPC mechanisms such as pipes, shared memory, or sockets. For example, if Program A communicates with Program B via a local socket, then running CDE only on Program A will collect the dependencies required to re-run Program A, but not those required to re-run Program B. In order to properly package up a set of programs that communicate via IPC, you must start all of them separately under CDE supervision.

One concrete example of this limitation is that CDE does not track dependencies on custom libraries that your machine’s X Server accesses. One user reported that CDE was unable to track the MESA lib version, but his app still ran fine. Hopefully the interface between GUI apps and the X Server should remain fairly stable across different distros.

What happens if CDE can’t find some shared libraries within the package?

In rare cases, cde-exec won’t be able to find some shared library that clearly exists within your CDE package. For example, your program might give an error like:

libGL.so.1: cannot open shared object file: No such file or directory

even though the file libGL.so.1 exists within your CDE package in, say, cde-package/cde-root/usr/lib/mesa. The problem is that /usr/lib/mesa hasn’t been properly added to the search path on the target machine by ldconfig, so the dynamic linker doesn’t know to try to look in there for libraries.

An easy workaround is to use the LD_LIBRARY_PATH environment variable to explicitly tell CDE where your library resides:

LD_LIBRARY_PATH=/usr/lib/mesa/ [command within cde-package/ to launch]

Note that CDE will redirect this path within cde-root/, so you need to specify an absolute path, not one within cde-root/.

Why is my zipped CDE package not working?

Be wary of methods of compression/transmission that don’t preserve symlinks, since they might result in broken CDE packages. One user reported that when he tried to to zip up a CDE package directory using the KDE GUI (right-click on directory icon -> compress -> zip/tar), it didn’t properly follow symlinks (plain old tar has always worked for me, though).

Also, if you use scp to copy a directory, that might not respect symlinks (rsync has always worked for me, though). The Linux zip program does not preserve symlinks by default; use zip -ry to force it to preserve symlinks.</p>

SECURITY AND PRIVACY

Does CDE provide any security guarantees?

Absolutely not. CDE is not designed to provide a secure sandbox for running untrusted code. CDE runs packaged programs in a sandbox similar to a chroot jail, but truly malicious applications can circumvent this sort of isolation attempt. You should only run CDE packages you receive from trusted sources. Of course, this same advice applies to all software you run on your computer.

That said, though, one way in which CDE packages can be more secure than traditional software is that root access is not required to run CDE packages. In contrast, you might need to run a traditional software installer as root, which could be a security threat.

Does CDE provide any privacy guarantees?

Absolutely not. CDE blindly copies all files accessed by your target program into the package, without regard for privacy. Thus, it might copy some of your super-secret private files into the package. It’s your responsibility to vet the package (running find cde-package/ is a good starting point) and remove any files that you don’t want to share with others.

You can also specify “ignore path” options in the cde.options file (see CDE options) to instruct CDE not to copy certain files into the package.

Finally, all of your environment variables (even super-secret private ones!) are stashed away in cde-package/cde.full-environment, so please also take a look at that file to make sure it’s not spilling any secrets!

How does CDE deal with software licenses?

It doesn’t! It’s your responsibility to ensure that you have permission to re-distribute the binaries, libraries, and other files included in your CDE packages.

SPECIFIC APPLICATION PROBLEMS

Why am I getting mysterious segfaults when running BASH within a CDE package?

There is a known bug in some Ubuntu distros (unrelated to CDE) that causes the BASH shell (e.g., /bin/bash, /bin/sh) to sometimes mysteriously crash when invoked with a custom dynamic linker. My suggested workaround is to tell CDE to ignore the entire BASH process and instead load the default version of BASH on the target machine. See Process-specific ignore paths for instructions on how to do so.

Why might some Java apps not run from within a CDE package?

You might be encountering a problem where a Java VM file like client/libjvm.so is not included in the package. One workaround is to find that file yourself and explicitly copy it into the proper location within cde-package/.

A user reported that if you explicitly activate the -client option when running Java (java -client), that will help CDE pick up vital dependencies like client/libjvm.so (that was the case on his Gentoo machine, at least).

Why do some X Window GUI apps fail with “No protocol specified” or “X11 connection rejected because of wrong authentication” errors?

This is probably because your XAUTHORITY environment variable is not set. Set it to the following value and try running the program again from within your CDE package:

export XAUTHORITY=$HOME/.Xauthority

(If there is enough demand, then I’ll add code to CDE to do this automatically.)

Also, on distros like openSUSE, you might need to run the “xhost +” command before X Window apps will work.

This blog post on .Xauthority files might also be informative.

Why do X Window applications not work on SUSE distros?

I’ve noticed that on SUSE-based distros like openSUSE or SUSE Linux Enterprise Server, you might need to run this command:

xhost +

before X Window apps will work. I doubt that this is the optimal solution, but it’s a hack that seems to work for now.

This blog post on .Xauthority files might also be informative.

Why am I getting a “cannot restore segment prot after reloc: Permission denied” error?

Some distros are running SELinux for security, so you might need to disable SELinux before some CDE-packaged applications will work. You can disable SELinux by running sudo /usr/sbin/setenforce 0.

Alternatively, you can try running chcon -t texrel_shlib_t on certain .so files, as suggested by this forum post.

This problem seems to manifest whenever you are transporting executables and libraries to distros running SELinux, so it’s not a CDE-specific problem.

How does CDE interact with ccache?

If you’re trying to use CDE to package up a compilation job and ccache is running on your machine, then it might be a good idea to disable ccache before running, say, cde make. Otherwise, CDE might simply package up ccache and the cached object files in, say, ~/.ccache/ and not actually package up the compiler, linker, or other dependencies required to compile.

This problem occurs since the whole point of ccache is to <i>skip</i> the invocation of the compiler toolchain when the appropriate files are in its cache. Here is an easy way to temporarily disable ccache by setting an environment variable:

export CCACHE_DISABLE=1

MISCELLANEOUS

Why is my CDE-packaged application unable to find my home directory?

Suppose that Alice created a package for an application that needs to read files from her home directory, /home/alice, perhaps within a dot directory like /home/alice/.gimp. When she creates the package, the copy of her dot directory within the package will be located in cde-package/cde-root/home/alice/.gimp.

Now when she gives her package to Bob, and Bob tries to run her application from within the package, it might use Bob’s username to generate a path to the dot directory. The path that it generates might be /home/bob/.gimp, which cde-exec redirects into the package as cde-package/cde-root/home/bob/.gimp. Clearly this directory doesn’t exist within the package, thus resulting in a cryptic error message.

The fundamental problem here is that cde-exec doesn’t spoof user IDs (trust me, doing so would lead to even more problems), so when the application asks for the user ID, it will get Bob’s ID, and then it looks in /etc/passwd to find the home directory corresponding to that UID (/home/bob).

One workaround to this problem is to simply rename the home directory within the package to match the current user on the target machine. In our example, that would mean renaming cde-package/cde-root/home/alice/ to cde-package/cde-root/home/bob/. This isn’t a perfect solution, but it seems to work alright in practice.

Note that sometimes this problem doesn’t arise because if the application tries to find the home directory by reading the $HOME (or $USER) environment variable, then cde-exec will use the version stored in cde-package/cde.full-environment, which will correctly return /home/alice (or alice, respectively) no matter which target machine it is running on.

How can I run CDE packages natively without using cde-exec?

Executing a package with cde-exec has several limitations, most notably a run-time slowdown (due to system call redirection) and suffering from the limitations of ptrace. Here are some ideas for running CDE packages without cde-exec:

All user-level dependencies that your program requires are located in cde-package/cde-root/, so you can simply use a regular chroot command to directly execute programs within that directory.

Of course, chroot requires root permissions to execute (whereas cde-exec doesn’t) and also some intricate setup work (e.g., bind mounting the /dev and /proc filesystems and copying /etc/resolv.conf into cde-root/). In exchange, though, your programs run at native speed without any of the limitations associated with the ptrace monitoring that CDE uses.

You can bind mount your home directory inside of cde-root/ in order to allow your CDE-packaged program to access files on the native filesystem (similar to running in seamless execution mode).

If you’re feeling even more adventurous, you can use a union mount (either a kernel module or FUSE implementation) to combine your package’s cde-root/ with portions of your own filesystem and then chroot into the union mount. Both bind and union mounting will allow your CDE-packaged program to access and manipulate arbitrary files on your machine rather than being isolated in a sandbox.