A picture is worth a thousand words:
A slightly longer story follows, resuming from the previous episode, where Gtk seemed to be working.
Making sure Gtk actually works
Since the udebs are quite fat already, I decided to put the contents
of the gtk2.0-examples package into a new revision of the new
libgtk-x11-udeb package. gtk-demo was running fine (if one forgets
that this demo application is really easier to deal with if one uses a
window manager), although image handling was broken. Julien and I
spent quite some time double-checking that the
libpixbufloader-png.so plugin was present; that both previous
DirectFB-based udeb and new x11-based udeb were very similar;
that configuration was generated properly; we went so far as to include
strace in the netboot image, and to check the configuration file
was open()’d, but the plugin was never to be even looked for. All
this resulting in a Unrecognized file format message every time.
Josselin, our Messiah, enlightened us: although Gtk was previously
using some configuration files to determine which pixbufloader
plugin to use, it still continues to access those files… but relies on
a MIME database in the end. Installing the host’s
/usr/share/mime/mime.cache database did the trick (we have a proper
fix pending), and all of a sudden, we got image support (which is
needed for the logo_installer.png displayed on top of every screen,
as well as for some icons during the installation, like the final
one).
Tweaking d-i itself
Julien took care of rootskel-gtk. It was mostly about:
- Dropping some
DirectFBstart-up scripts. - Dropping a related configuration file.
Adding a start-up script doing trivial things:
Xorg -retro -noreset & export DISPLAY=:0
Another easy step was cdebconf. Mostly: search and destroy on the
few bits using DirectFB. An interesting issue we had was that even
though a proper cursor theme was installed and made the default, we
only had a black cross as X pointer. That’s where we learnt it’s
needed to manually set a cursor for the root window, which can be done
this way:
GdkCursor *cursor = gdk_cursor_new(GDK_LEFT_PTR);
gdk_window_set_cursor(gdk_get_default_root_window(), cursor);
gdk_cursor_unref(cursor);
Tweaking gtk2-engines was trivial, just a matter of building against
the x11 flavour of Gtk instead of the directfb one. Some
substitutions in debian/control and debian/rules, and that was
done.
Julien also took care of tweaking console-setup so that
keyboard-configuration’s and console-setup-udeb’s postinst scripts
call setxkbmap when $DISPLAY is set. Meaning we were able to
replace kbd-chooser with console-setup-udeb, leading to a correct
setting within X right after having selected the layout!
Reverting d-i’s r61645 (which disabled Gtk-based d-i images) was
trivial (although while looking back, it was probably unnecessary),
and after having messed around with the packages lists (there’s a lot
of local udebs at this point), especially to add some fonts, we
managed to get an image which seems to work fine (at least in my
VirtualBox environment). I tried Chinese, Japanese, Arabic, and French
installations, and all of them seem to work fine (installed systems
were bootable after an installation in the selected language) while
issues were limited to some rendering or translation glitches, which
didn’t seem critical at all.
Next steps
Since there were some iterations, and since the “trial-and-error”
approach was chosen, the first goal is to generate clean patches
against all modified packages, and make sure a single walk through all
packages in a clean chroot is sufficient to generate a working d-i
image again.
Once that done, which should probably considered (at least, I hope!)
quite a good base already, the idea is to look into the various
udebs to see whether there are some parts that could be disabled
(maybe introducing a trimmed-down flavour especially for the udeb in
some non-flavoured-yet libraries), so as to reduce the size of the
resulting image.
In the meanwhile, one can fetch the current hand-built netboot image
by browsing http://people.debian.org/~kibi/di-x11/. It’s only
available for amd64, and weights 23MB. There’s a sha256sum as
well as a detached GPG signature in the same directory.
Last night was the third one spent on playing around with X in a
d-i environment.
Making sure X actually works
So far, running X
was nice already, but is it really possible to run some applications
inside? Bad news is that many trivial applications (xeyes, xclock,
…) pull more dependencies than
previously packaged.
Finally, xev was chosen, and it indeed works fine (provided one
makes sure it gets the focus).
X was crashing when the (only|last) client was exiting, and that
wasn’t normal. Thankfully the bug was reproducible outside d-i and
Julien
sent a patch
a few minutes later.
In the meanwhile, using -noreset did the trick to keep X running.
Another way to run X applications there, is to point the host
environment to the guest environment using DISPLAY=$GUEST:0
xterm. With VirtualBox, I followed those steps:
- Forward the host’s
6000/tcpto the guest’s6000/tcp. - Answer the language-related questions in
debconfso that network configuration happens, which is needed for the next steps to work (just gettinglocalhostset up is probably sufficient, but let’s be lazy and let the installer take care of it for now). Run
Xin the guest, disabling access control, with:X -retro -noreset -acStart
xtermfrom the host with:DISPLAY=localhost:0 xterm
Moving forward: cairo, pango, gtk
The next step was to rework some udebs. Legend:
- Red box:
udebwe no longer want to depend on. - Blue boxes:
udebsto be tweaked first. - Green boxes:
udebsto be tweaked afterwards. - Ellipse:
udebprovided by the one pointed to (magenta edge).
Current situation, keeping only the relationships between udebs of
interest:
I took care of gtk+2.0 while Julien hacked pango1.0 and
cairo. Some more libraries were needed, but being X libraries,
tweaking the packaging was as trivial as for the first X libraries,
and it only took a couple of minutes: libxcomposite, libxdamage,
and libxinerama.
Once those added to localudebs/ and to pkg-lists/local, and once
the image rebuilt, I prepared a tiny “Hello world”-style application:
- Grab the source code from the GTK+ 2.0 Tutorial.
Built it against the
x11flavour:gcc -o hello $(pkg-config --cflags gtk+-2.0) hello.c -lgtk-x11-2.0Grab it from within
d-i, and make it executable.Start
Xas previously explained, then:DISPLAY=:0 ./hello
Pango wasn’t quite happy at first (message wrapped):
(hello:1991): Pango-WARNING **: failed to choose a font, expect
ugly output. engine-type='PangoRenderFc', script='latin'
Indeed, since no fonts were included, there was a tiny issue. No big
deal, adding ttf-dejavu-udeb did the trick. I wanted to play around
with a non-Latin language, so I asked for an example (thanks,
Theppitak!), and once ttf-thai-tlwg-udeb added as well, one gets:
Of course, there are many more features to test, but that sounds like we’re going in the right direction (and this is not a joke about RTL).
Next step
Have a look at the green boxes, which may require to have a closer
look at g-i’s integration within d-i (it might be needed to see
how it’s supposed to be started, that might help testing the tweaked
packages).
Getting to X
After
having prepared a bunch of libraries,
the idea was to get X in shape. I’d like to thank Julien Cristau
very much for his support all along the way, since I knew nothing
about X.
Basically the plan was to get:
xserver-xorg-core-udeb:Xitself.xserver-xorg-input-evdev-udeb: for input, as you probably guessed.xserver-xorg-video-fbdev-udeb: for output, as you also probably guessed.
Not much to say about the former, it only pulled two additional
libraries compared to what was previously udeb-ified: libaudit0
and libselinux1. Since security modules aren’t the primary concerns
in this exercise, --enable-xselinux quickly became
--disable-xselinux and those dependencies went away. Both drivers
were trivial to tweak, nothing really worth noting there.
Once those 3 additional packages added to localudebs/ and to
pkg-lists/local, the mklibs tool really wasn’t happy with them
during the rebuild of the netboot image, since the drivers are X
plugins, meaning they need symbols which aren’t defined in other
libraries. Example:
No library provides non-weak EventSwapVector
Since the idea is to get something working rather than getting
something perfect at once, d-i’s packages/mklibs/src/mklibs.py
(installed as /usr/bin/mklibs) was told to stop raising an
exception, and to print a warning message instead. Rebuilding an image
was then working again.
It took some time, but finally it got noticed those packages were
actually all installed in the initramfs, so there was no need to try
and load/install them manually during the installation process,
everything was there already, and it was sufficient to switch to VT2
to see the /usr/bin/Xorg binary.
Some packages were still needed: xkb-data, and x11-xkb-utils. The
latter also pulled libxkbfile1, but well, an additional library
doesn’t sound very scary.
At this point, one has to try and run Xorg. The result was a black
screen, and no means of getting back to another terminal, be it inside
Qemu or VirtualBox.
Troubleshooting X
First idea: adding the openssh-server-udeb package to the image, and
start sshd from there. But then one has to go through the missing
config file, tweaking privilege separation, generating a host key,
setting a password, etc. And for whatever reasons, it wasn’t possible
to access it. That’s where the
DebianInstaller/NetworkConsole
page on the Debian Wiki came into play: it
includes a few steps to get a working sshd within d-i,
painlessly.
So, X was actually starting properly, but getting a black screen is
now the default. One wants X -retro to get the well-known old
behaviour.
From udevadm info --export-db, it looked like input stuff were
missing, and indeed, udev-udeb doesn’t include /lib/dev/input_id
and /lib/udev/rules.d/60-persistent-input.rules. Since scp seems
broken, wget from the host did the trick followed by chmod +x
for the input_id file, otherwise that doesn’t work so well.
But still, there were some missing evdev stuff. And indeed evdev
wasn’t to be seen in lsmod, nor available to be modprobe’d.
That’s where it became a little dirtier: wget and udpkg on
mouse-modules-2.6.30-2-amd64-di, followed by depmod to make sure,
and then modprobe was OK. A little udevadm trigger run to ensure
udev was fully aware of the new situation, and restarting X… led
to a moving cursor there, as well as a working keyboard, since it was
possible to switch between terminals!
Prove it!
Alright, I’m proudly providing you with a screenshot:
(Really, I’ve never been so happy to see this fracking black-and-grey-grid screen. I kid you not!)
If we forget about the very few network-console-related questions
answered to get an sshd running, please note that the installation
process is still just starting, the very first debconf question is
still unanswered.
Now, patching udev again so that udev-udeb includes the two
mentioned files was almost easy (details aren’t important anyway),
while pulling evdev.ko module automatically was trivial, just a
matter of adding such a line to a packages list (I picked
pkg-lists/netboot/amd64.cfg):
mouse-modules-${kernel:Version}
Which means one can now, out of the box:
- Boot the ISO, select Install.
- Wait for the first question to appear.
- Switch to another terminal, and activate it.
- Run
Xorg -retrothere.
Next step
Rework the Pango/Cairo/Gtk stack to use X11 instead of DirectFB.
Producing udebs
How to patch
That’s actually quite trivial. Most packages from Josselin’s 1st and 2nd lists are libraries, and generating udebs for libraries is easy. Basically it boils down to declaring a new binary package:
- Duplicate the
libfooNentry indebian/control. - Add the
-udebsuffix to thePackageline. - Add a
Package-Type: udebline. - Make sure
Sectionisdebian-installer. - (Maybe) make sure
Priorityis onlyoptional.
Now, populating it might depend on the considered source package, but
thankfully, pkg-x11 packages are quite sane, using debhelper’s
dh_install. So duplicating debian/libfooN.install into
debian/libfooN-udeb.install is usually sufficient. It’s a bit
trickier when dh_movefiles is used, but calling install and cp
-r after the dh_movefiles call should be sufficient.
Next step, tweak shlibs for libfooN. Without that step, a
libbarM package built against libfooN would get libfooN in
Depends, which is not appropriate. The idea is to get libfooN-udeb
instead. Trivial: add --add-udeb=libfooN-udeb to the dh_makeshlibs
call.
Possible issues at that step:
debian/shlibsordebian/libfooN.shlibsis used instead ofdh_makeshlibs. Trivial fix: feed its contents todh_makeshlibsthrough its-Vargument, and delete that file. Caveat: when it comes to applying a patch removing that file, make sure it’s actually removed, and not just empty. I’m going to file a bug about that, the default behaviour with empty files seems a bit buggy.- Several libraries are shipped: one can’t pass several
--add-udebarguments. Solution: use severaldh_makeshlibs -p$pkg -V$depends --add-udeb=$pkg-udebcall. (I thought it was needed forlibx11but I can’t read, actually:libxcb1≠libx11-xcb1.)
Various checks to perform:
Make sure
debian/libfooN/DEBIAN/shlibsis correct: it should have audebline. Example:libfoo N libfooN (>= some-version) udeb: libfoo N libfooN (>= some-version)Make sure the generated udebs have no non-udeb packages mentioned in
Depends. Since I’m still at the proof-of-concept stage, I’ve iterated when I encountered such additional packages, without checking what was actually needed. The idea is to get rid of extra stuff once the final needs figured out. Of course, one has to track down the dependencies to get the packages incrementally built and installed (for the updatedshlibsfiles).
Results
(These files might be moved around during the following steps, but this post shall be kept up-to-date and point to them at all time.)
Legend:
- Red box: package needing a patch.
- Green box: package already udeb-aware.
- Label with
[n]: package patched at stepN. - Label with a
*: warning when patching,shlibsfile to be removed.
Graphs are scaled down to 75%, click for full size.
Step 1:
- Patches: udebs-1/diffs/
- Udebs: udebs-1/udebs/
- Notes: as explained above,
libx11-xcb1is probably not needed. - Dependency graph:

Step 2:
- Patches: udebs-2/diffs/
- Udebs: udebs-2/udebs/
- Dependency graph:

Adding udebs to a d-i image
Basic d-i image
General guidelines (see d-i documentation for detailed installation,
needed packages, etc.):
svn co svn://svn.debian.org/svn/d-i/trunk/ debian-installer-trunk
cd debian-installer-trunk/installer/build
make build_netboot
If that breaks (e.g. due to missing packages), make sure there’s a
similar entry in sources.list.udeb:
deb http://yourmirror/debian squeeze main/debian-installer
If everything goes well, the generated image should be available as
dest/netboot/mini.iso, and tmp/netboot/udeb.list should contain
the list of embedded udebs.
To give it a try:
qemu-system-x86_64 -cdrom dest/netboot/mini.iso
Actually adding udebs
Drop all additional udebs under localudebs/. Needed Packages and
Packages.gz should be generated on the fly (but beware, secure apt
is then disabled, even for non-local udebs).
Now, tell d-i we want some more packages. If all of them are wanted,
one can do that:
for i in localudebs/*.udeb; do \
dpkg-deb --showformat='${Package}\n' -W $i; \
done > pkg-lists/local
(Update: Thanks to Guillem for pointing out a better and cleaner way
than dpkg --info $i|grep Package:|awk '{print $2}')
Generate a new image (note the rebuild part of the target):
make rebuild_netboot
Make sure they got added by looking in tmp/netboot/udeb.list and
profit!
What’s next
Future steps include:
- Making sure those udebs are actually usable.
- Reworking Pango/Cairo/Gtk udebs.
- Taking a look into X itself.
Stay tuned.
(Disclaimer: this post is not about G.I.)
Graphical installer
Josselin posted his thoughts about g-i, and given at least the first parts sound doable, I’m tempted to have a look at dealing with the needed udebs. Never touched anything like this before, but oh well, one learns every day. Time to learn a bit more about X11 as well.
It looks like generating a new mini.iso (using d-i’s trunk) wasn’t
that difficult, so hopefully once I’m done with a few udebs, it should
be possible to add them to this image and make sure those are usable,
at least when loaded manually.
Once that finished, gluing everything together is probably going to be another story, but hopefully that won’t be impossible.
Unrelated stories
Thanks to DSA (especially weasel),
kfreebsd-*now have DSA-admin’dexperimentalbuildds. Aurélien took care of setting them up; they managed to go through allexperimentalpackages, and bugs were filed accordingly. Some were just marked asfailed, usually with “Needs porting.”, especially when there were no regressions with respect to what was previously built on those architectures (as a reminder:out-of-datemeansserious,uncompiledmeansimportant).On a related note, this buildd.d.o form makes it possible to browse the build logs for various suites. See the dda@ announce for details.
It looks like I’m going to stop dealing with Java at all: #557230. Thanks to Gabriele Giacone for taking care!
It also looks like I’m done with Python packages as well.
A new module-assistant package got uploaded, which should fix issues with newer kernel versions (ya can haz detailz). I’m wondering whether backporting it through backports.org from time to time would be of some help to anyone? It is (or should be) directly installable in a
lennyenvironment, but maybe people would like it to be directly available once thelenny-backportsrepository enabled? Mail/wishlist bug welcome if you want it.If you received some signatures from me lately… no, I wasn’t pretending I met you at LCA or whatever, but that was from Cáceres (Hi Penny!
;)). Key rollover being thought of.
With the current move of X.Org to unstable, a few tips might help
xserver-xorg-video-intel’s users.
Current issues involve screen turning to black from time to time,
including while typing, or after a few seconds of inactivity, or after
a few minutes. Depending on the applied tips, it’s sometimes possible
to switch back to a console, trigger a suspend to RAM, and then get
back to an un-black screen. (The funny think being that using s2ram
may break the Intel wireless driver, which can be fixed by then using
s2disk…)
Restarting the X server is usually insufficient. Logs (be it X.Org’s or kernel’s) don’t show anything. Given those issues showed up with the previous X.Org stack too, with recent kernels, it might just be a bug there.
Various stuff that can be tried:
Disable powersaving by passing
i915.powersave=0on the kernel command-line.grub2users can add it toGRUB_CMDLINE_LINUXin/etc/default/grub, and runupdate-grub2. Once the system is started, it can be checked by checking the contents of the/sys/module/i915/parameters/powersavefile.Disable DPMS in
/etc/X11/xorg.conf: SpecifyOption "DPMS" "False"in theMonitorsection.DPMS enabledor so should vanish from/var/log/Xorg.0.log.
With that setup (apparently, the first tip is particularly
interesting, the second didn’t seem to help much), turning to black
seems to only happen when using MPlayer or VLC, while it was
previously happening while e.g. reading a few lines of IRC backlog…
Passing various -vo flags to MPlayer, like -vo x11, doesn’t seem
to help.
Have to try with some 2.6.33-rcX before reporting this bug, just
wanted to let others know about current findings…
After a while playing around with the build system, with python 3.1
(which is required), etc., I finally got around to upload a Blender
package for 2.50 alpha 0, just uploaded to experimental
(hopefully) given python 3.1 is only available there.
Potential users really want to read the NEWS file.
One can also check the upstream release logs.
For those wondering about the version number, it had to be >= 2.49,
and upstream doesn’t seem to be sure about the versioning, since it
can be 2.5 on the website, 2.50 as SVN tag, etc. I’m not going to
introduce an epoch right now. ;)
It certainly has some rough edges, but testing and reporting are more than welcome.
I really liked doing stuff like Zack’s RCBW in the past, but nowadays I’m busy with some buildds, reporting RC bugs because some maintainers don’t test their uploads, as well as posting patches to fix FTBFS on GNU/kFreeBSD.
But from time to time, it’s also fun to close RC bugs. Here’s a new
concept (that I probably won’t reproduce as regularly as Zack and his
minions, though): RCBN for Release-Critical Bugs of the Night.
Those are now closed:
- #552862:
postman: FTBFS: cgi-lib.c:238: error: conflicting types for 'getline' - #545621:
regina-normal: FTBFS: problem linking with boost - #540427:
jconv: Build-Depends on libc6-dev (>= 2.7) - #540669:
kflickr: FTBFS: '.libs/libkflickrpart.so': No such file or directory - #554638:
jconv_0.8.1-1(ia64/unstable): FTBFS: build-depnds libc6-dev - #527732:
orpie: FTBFS: (.data+0x2c0): undefined reference to 'ml_gsl_vector_float_minmaxindex' - #520862:
libkarma - FTBFS: /usr/bin/csc: Command not found
Many thanks to Peter Green, to Ruben Molina, and to Ilya Barygin for their patches. Special thanks to Richard Darst for orpie’s patches: he even managed to get rid of embedded code copies while addressing the FTBFS. Nice!
Unfortunately, in the same time, I had to open some FTBFS/RC bugs:
- #559766:
java3d: FTBFS on non-Linux ports - #559767:
java3d: FTBFS on armel: install: cannot stat 'j3d-core/build/default/…' - #559768:
java3d: FTBFS on hppa: java compilation issues - #559771:
evolution-dev: Might be missing a Depends on libgtkhtml*-dev - #559781:
ktoon: FTBFS with current ffmpeg - #559794:
libfvm-dev: *LOTS* of undefined references
So, what’s next? Get back to work!
Some uploads this week:
There’s a
2.49.2~dfsg-1package for Blender sitting inunstable. Despite its version string, it’s actually 2.49b (release logs). Looks like there’s a good reason for~dfsgvs.+dfsg.There’s a pending package for Blender
2.5 alpha 0, but some things still need ironing out. As stated in #558083, it’ll probably go toexperimentalfirst, given its dependency on Python3.1.Speaking of which, some dozens of X.Org drivers (
xserver-xorg-input-*andxserver-xorg-video-*) were meant to be uploaded toexperimentallast night, but due to sbuild’s #559659 and lintian’s #542747, they ended up inunstableinstead. If you see packages with aDepends:on X.Org stuff fromexperimental, that’s normal, wait untilxorg-server1.7moves tounstable, go and fix some bugs instead of reaching forreportbug.
Recipe for a smile:
Notice digikam's FTBFS on GNU/kFreeBSD.
Work on a patch, send it to the BTS.
Receive the ACK from the BTS.
Receive the upstream commit notification a dozen of minutes later.
Denightening lights or enlightening nights or nightening delights…
Kudos, Luka.



