Syllable Desktop Change Log
Syllable Desktop 0.6.7Kaj de Vos, 12 April 2012
This release is dedicated to the memory of Jack Tramiel, who passed away when I was making the finishing touches to this document. He was dealt hard blows in life and always recovered. I will never forget how he saved Atari from bankruptcy and orchestrated their comeback.
We are happy and proud to present this new Syllable version. We thank all of the volunteers who contributed to this release for their generous efforts. Most of the work that went into this version was done by Kristian Van Der Vliet, Anthony Morphett and Kaj de Vos. Other contributors are mentioned with their contributions.
The highlights of this release include:
- Improved documentation.
- Improved installation, with support for specific configurations such as the Asus EeePC.
- The system is much more usable with a keyboard. Installation and log-in can now be completed without a mouse.
- Auto-login: you can set a user account to log in automatically without entering a password.
- POSIX interfaces for memory mapped and async I/O implemented in the kernel.
- Improved support for Intel eepro100 network driver and Intellipoint PS/2 mice.
- Improved native applications. The web browser was fixed to remember cookies again. An FTP client was added.
- Improved support for SDL programs. Almost hundred SDL programs that “Bogomips” ported can now be used without extending the system:
- Upgraded multimedia support. Flash videos can now be played.
- First 3D support implemented, through porting of Mesa3D and PicoGL.
- QEmu now included for running Virtual Machines, with improved performance due to memory mapped and async I/O.
- SaMBa now included for Windows compatible networking.
- The server part of OpenSSH now works for remote access to Syllable.
- SQLite now included and Fossil version control system ported.
- System restructuring to make it more user friendly.
- REBOL 3 and Boron now included for programming.
This release has been a long time in the making. Of course it is good to have regular releases, as Syllable had in the past, but we are a volunteer organisation, so we need to make do with limited means. A lot of work is needed to make a release, and we need to fit that into our lifes. Along the way, we lost our release engineer and build infrastructure. This is an intermediate release in the process of replacing them.
To make it easier for us to make this release, its scope is limited. It is not based on the latest source code in CVS, but on the previous development build, which had been tested for a long time, with many fixes and additions made to it. Building from the latest source would have meant a series of extra issues to fix and test. Due to this decision, no separate source package is available for this release. It can be constructed from Syllable CVS, but we currently don't have the resources to do it. The next release will have a separate source package again. At the time of writing, there is no package to upgrade from Syllable 0.6.6. The Virtual Machine image, the live CD and the API documentation have not been updated, but will probably be released later, as time permits. Sadly, Lulu.com has stopped producing CD's, so there is no Premium CD anymore, either. We are looking into replacing it.
Syllable 0.6.6 was a good release, so there wasn't an urgent need to replace it. We are making this release now to prepare for several future developments. The system is in a transition state in several ways. Syllable 0.6.6 was a stable base to work on, so we need to move to a new stable state, with as little disruption as possible. In the past, new Syllable releases suffered from too many regressions. We want to minimise those. We made a new development release two months before this release and tested it quite heavily. No problems were reported and we fixed all the regressions compared to Syllable 0.6.6 that we could find.
The structure of the system and software packages are being reorganised, to make them easier to understand. During this migration, both the old and new package structures need to be supported. Resource packages built according to the new structure on Syllable 0.6.7 will not be compatible with older releases. Further, the system now supports the POSIX
We also need this release to prepare for the upcoming high-level Red programming language. Its memory allocator uses
DocumentationThe base documentation for the system was overhauled, updated and cleaned up. Several documents were converted from plain text to HTML format, including these release notes and the installation manual. This fixes Unicode encoding issues when displaying the Russian and German versions, particularly from our web sites. A new style was applied to them that is easier to read, and being HTML, they now have clickable tables of contents and other links. These release notes are now included in the system.
Some documents are still in transition to this new format, particularly the translated versions.
InstallationAnthony has implemented keyboard navigation in Disk Manager. This means you will be able to install Syllable without using a mouse, in case the installation CD has problems with it. To do this, Anthony had to make fixes to keyboard handling in LibSyllable Views. This will also improve keyboard use in other applications.
The start-up menu was further extended to help with specific installation environments and incompatible hardware.
The options for IDE and USB CD players were merged, so that the troubleshooting options can now also be tried with a USB CD player. A specific installation option was added for ASUS EeePC netbooks. The EeePC requires compensation for its shifting of drive positions, which is now performed by the installer. This was tested for us by Hans Rood on the Summer SylCon. There are now more safe mode options for troubleshooting, such as an option to fully remove the USB 2 driver, which is buggy on some systems. The original Acer Aspire One netbook was tested to work for us by Ruud Kuin, when disabling ACPI.
Added extra boot menu option to remove the buggy EHCI USB 2 driver. The installer also removes it for EeePC.
Added extra boot menu option and boot parameter for SATA configurations that shift drive numbers.
Extended installer to compensate for drive number swapping and shifting on EeePC and some SATA configurations.
Experimental installation options were added for installing from and to USB memory sticks. These don't work yet, because Syllable doesn't start from USB devices other than CD players yet, but this is being worked on. Apart from Syllable's own AFS format, there's an option for starting from USB memory formatted with Linux's Ext2FS file system. Installing to USB is still unreliable on Syllable, so the Ext2FS option allows to experiment with creating a USB installation medium from Linux.
Work on the boot menu:
- Added option without AGP.
- Added USB drivers to most options, so they can be used with both ATAPI and USB CD drives.
Left a failsafe option without early loading of USB drivers.
- Split off Virtual Machine options and failsafe options into separate submenus.
- Added experimental option to boot from Ext2FS formatted USB media.
- Clarified descriptions of several options.
SecurityThe super-user group was renamed from the traditional Unix name “root” to “system”. In this release, the super-user account is still named “root” but in a future release this will also be renamed to “system”. This is more meaningful to common people and also increases security somewhat by protecting against attacks by the user name “root” instead of user ID 0.
A default user account named “administrator” was created to signal to the user that this is the account for doing system administration. This way, the roles of the super-user account for running system processes and for logging in to do administration tasks are clearly separated. The default password of the administrator account is
To match the administrator user account, there is a new group named “administrators”.
Desktop EnvironmentThe system is now preconfigured with multiple screen resolutions, which can be switched at a key press by going to different virtual desktops. The default resolution on the first desktop is more usable than before.
The Blue window decorator, by John Aspras and Arno Klenke, is now set as default. Its larger window border and buttons work better on the higher resolution displays that are now common, especially when the physical screen is smaller, such as on mobile devices.
Definitions for many common file types were added, so you can click such files to open them in their associated application.
Made the log-in app fully keyboard navigable.
Anthony has implemented auto-login. You can now go into the Users & Groups preferences and set a user account to log in automatically, without going through the password dialog.
Implemented auto-login in the login app and Users & Groups preferences.
Added GUI for auto-login. UI improvements for auto-login preferences.
Cosmetic fixes for
New settings location for login app settings and auto-login configuration. Made login app save more settings including per-user keymaps.
Added filter to hide system users and groups in Users & Groups preferences.
Added Show system users/groups checkboxes to Users & Groups preferences.
Command Line EnvironmentThe quality of terminal emulation in the Terminal application has suffered from regressions over the years.
Changed terminal type to
Graphical ApplicationsApplications were moved to more functional names. While this is a tad boring, it reduces confusion for new users trying to learn the system and the function of applications. Only the launch names were changed, not the display names of applications once they are started.
The Webster web browser that was new in Syllable 0.6.6 fails to remember cookies, because it tries to store them in the location for the old ABrowse browser that it succeeded. This release has a newer version that fixes this, which was already on the Syllable 0.6.6 live CD. You need it for example to make web sites remember your log-in status.
The collection of default bookmarks for the web browser was updated.
File Transferrer was added, our FTP client application. Anthony has made a number of fixes to the functionality and the appearance, and has integrated work from contributors. This is originally our community project. The application is not done yet, but it is becoming usable for your downloading pleasure, and your uploads if you are so inclined.
Several applications from the sIDE development collection that are useful outside of programming were added to the base system. Code Editor (Sourcery) as an advanced text editor, Translation Editor for editing program translations and Event Viewer for showing available system events.
Several classic Syllable demos were added: Double Buffering (moving objects without flickering), Eyes (eyes following the mouse cursor) and Stars (scrolling star field).
The new PicoGL package includes several classic 3D animation demos: Cube (spinning solid cube with texture), Gears (spinning gears), Mech (walking mechanical robot), Spin (spinning wire-frame box) and Texture (spinning textures).
Made the disk partitioner fully keyboard navigable.
Patches for Settings Editor from John Aspras.
Resource PackagesMost packages in the system were upgraded, including key components such as Packager, FFMPEG, FreeType, BASh and NCurses. Other important packages such as OpenSSH and Transmission were also updated.
OpenSSL, ReadLine and LibJPEG are now shared libraries included in separate packages, instead of statically compiled into the programs that use them.
The collection of well-known root certificates from Certification Authorities was upgraded from the Debian CA Certificates collection to the LFS Mozilla CA Bundle collection. CA-Certificates has too much Debian specific machinery to get the certificates in the needed form.
XZ-Utils was ported to Syllable, providing the same LZMA compression as in 7-Zip, but in a different format that is becoming popular, and is better integrated with POSIX systems. TAr was upgraded and this version has support for XZ-Utils. Compression of the development files pack was changed from 7-Zip to XZ format.
QEmu was added, so you can run other operating systems or test versions of Syllable as Virtual Machines out of the box.
SaMBa was added, so you can connect to Windows machines over the network, or to other machines running a Windows network. You can use the
Fossil and the latest version of SQLite were ported to Syllable. Recent SQLite versions work again thanks to file locking fixes in the Syllable kernel. Some stability problems were observed in the Fossil port, so it is still experimental.
MultimediaThe FFMPEG codecs collection was updated to version 0.5.0. The Syllable media framework needed to be adapted to match. As a result, the Media Player can now play Flash videos.
“Bogomips” found that the SDL subsystem didn't support PNG images yet. We tracked the problem to the SDL-Image component and found TIFF image support missing, as well. Oddly, even though JPEG (and GIF) images already worked, the fix required making LibJPEG a shared library. Fixed versions of SDL-Image and LibJPEG are included.
SDL-Image configure didn't detect LibPNG and LibTIFF, so it didn't have support for PNG and TIFF images. The configuration has been extended to fix this. The only way I got the new combination to work was by making the LibJPEG library dynamic. There are some combinations left that could be tested, for example in case the problem is with LibTIFF, that is now being used and uses LibJPEG itself. However, this debugging takes a lot of time, and I have always thought that forcing the popular image libraries in the system to be static libraries may be a bridge too far. LibJPEG is used in several places in the base system, so making it dynamic is the most straigthforward solution, and also makes the system more efficient.
Kaj ported the latest version of Mesa3D. Kaj and “Bogomips” ported PicoGL. This means the first 3D functionality is now available in Syllable. Previous attempts at porting Mesa3D long ago were never completed. It has a new graphics architecture now, so a Syllable driver for these old versions does not work anymore. Therefore, Mesa3D currently only provides off-screen rendering. However, PicoGL is built on SDL, so programs written for PicoGL can render directly to the screen. Several demos are included.
Server FunctionalityConfigurations, including initialisation scripts, were added for the OpenSSH remote access server, the SaMBa Windows-compatible file server and the Fossil version-control file server. To activate one of the servers, edit its start-up script in
After many years of trying by several people, Kaj collected their hints and got the server part of OpenSSH to work. The integration into Syllable Desktop is based on earlier integration into Syllable Server of the system configuration needed for the OpenSSH server. Kristian implemented the
Unique encryption keys for OpenSSH that identify your system were already generated at the first start-up of your Syllable installation.
Keyboard navigation improvements for compound views such as
KernelPOSIX interfaces for memory mapped and asynchronous input/output were implemented. Bugs in file locking and random number generation were fixed.
Kristian has implemented asynchronous input/output. This has been tested with QEmu, which shows increased performance both due to this and the also new implementation of memory-mapped files.
Async I/O is implemented as a native POSIX interface. The available functions are
Originally, Syllable relied solely on pervasive multithreading for its exceptional responsiveness. Several years ago, a new scheduler was introduced that is more suitable to desktop use than the original round-robin scheduler. Async I/O further enhances responsiveness in ported applications that use it.
Implemented async I/O support:
Kristian has implemented the POSIX interface for memory-mapped input/output. Ever more third-party software requires this functionality. Both the interfaces for memory-backed files and anonymous memory maps are supported. Syllable has always used memory-mapped files for its in-kernel ELF loader, so the POSIX interface is implemented on top of this. Several components in the system make use of
There is a tendency in much open source software in recent years to require advanced features of the host platform, that are only available on the major systems; sometimes only on Linux. This is problematic for alternative operating systems and undermines the portability of open source and the diversity and choice of platforms. Through the addition of the
- Both anonymous and file-backed maps are supported by
DriversFixed support for Intellipoint mice by completing the switching to IMPS2 protocol. Patch by Evgeniy Philippov for Defender PS/2 mouse. This was probably a regression because IMPS2 support was added long ago.
Added another OEM PCI ID to the
Added an extra PCI ID to the
Added chip-specific controller drivers for the AGP bus driver, for Nvidia, Ali and Sis chips, from Dee Sharpe. The Via driver is waiting for a fix to memory management in the AGP bus manager.
- Added the
Removed the old
System StructureSyllable is an operating system with a high degree of compatibility with the POSIX standards and de facto conventions in Unix systems and Linux distributions. This is a double edged sword. It makes it much easier to get existing third-party software to run on Syllable, but it also burdens us with ancient methods from the beginning of computing that have partly lost their meaning. One example is the layout and naming of files in the system and in additional software packages. We have always tried to strike a balance between following conventions and inventing more modern and user friendly methods. With a decade of experience, we have gathered the confidence to make a number of extra steps towards more user friendly structures, with as little disruption to compatibility as possible.
Instead of trying to shield people from the workings of their system, we want them to be able to go into the directories of the system or a separate package just with a filer, and be able to make some sense out of the files there and find what they are looking for. To that end, we are introducing a series of new measures to rename and move subdirectories that often form the starting point for searches for information.
The system tools Packager and Builder were further developed to support these restructurings, among other things to improve the use of symlinking for package registration. Packager is included in the system, but Builder needs to be installed as part of the Developer's Delight package, if you want to compile software yourself. When Builder is installed on Syllable 0.6.6, an older version is used that is still compatible with the older system structures. The
Graphical programs that, besides the desktop menu, also need to be available from the command line are now kept in the search path locations and symlinked from
Packages are in the process of moving to clearer names for their main package directory. Instead of the traditional, short, lowercase names that are confusing to common people, common typographical standards are now being used. Resource packages have project names, which are proper names, so they are capitalised. Generic terms remain lowercased. When names consist of multiple words, CamelCase is used to avoid spaces. When uppercase abbreviations would lead to words of only capitals being concatenated, hyphens are used for more clarity. Abbreviations are expanded when they don't take too much space; roughly twenty characters maximum. Exceptions can be made to keep names recognisable to people who already know the short names, and to keep them roughly in the same place in alphabetical order. Around half the resource packages in the system are currently renamed.
The internal structure of resource packages is in the process of being heavily reorganised, doing away with the traditional Unix naming that is confusing and has lost its meaning over time. The structure of the symlink pools has changed accordingly. Subdirectories are being moved and renamed thus:
etc -> settings bin -> programs sbin -> system-programs libexec -> framework/executables lib -> framework/libraries include -> framework/headers share/aclocal -> framework/AutoConfigure share/pkgconfig -> framework/PackageConfigure lib/pkgconfig -> framework/PackageConfigure share -> data share/doc -> documentation man -> manuals share/man -> manuals info -> manuals/info share/info -> manuals/infoSubdirectories that are not immediately meaningful to common users are tucked away in the
Since there is no standard for the location of REBOL and Boron libraries, they can go into
The Syllable-specific initialisation scripts were consolidated into one subdirectory. This leaves room for adding the common Unix script classes for
early-init -> tasks/setup init -> tasks/start
An extra subdirectory
Support in the menu needs to be implemented separately.
A few packages need extra, internal compatibility symlinks because they don't fully support this reorganisation. On the other hand, such links can also be used to solve interdependencies between packages.
The symlink pool
Most common people would probably associate “bin” with a dustbin, and wonder why every package has its own trashcan. We'd have to explain that it means “binaries”, then explain what binaries are and what a binary number system is, and then explain that
The “root” super user in Syllable Server has already been renamed to “system”. This is also planned for Desktop.
Libel? Liberation? Makes it clear what we're talking about. It's still not the same as a public library, but at least it's clear that it belongs to the framework of a resource package, and libraries is the term used everywhere in system and programming documentation, so it would be confusing for techies to change it more than this.
Include which in what? We're talking about compiled language programming headers for the libraries, and that's still a very technical term, so we tuck it away in
This contains M4 macros for the GNU AutoTools, AutoConfigure and AutoMake, that many packages use for source code configuration. Very technical and applies to the libraries, so it's part of the framework. Besides, M4 macros are scripts, so it's not even proper
This is for another software configuration system named pkg-config, and contains data files describing properties of the package. Again, this is mostly meant for other packages to use the libraries in this resource package, so it's part of the framework. We use the clear CamelCase version of the name of the configuration system. Packages in the wild are confused about whether this is data or belongs with the libraries. Consolidating those two locations into one simplifies our search path for PackageConfigure files.
Programming languages tend to have their own hierarchy under
There is no standard place for REBOL and Boron libraries, so we are free to change this.
Having it immediately under
Note that eventually, we shouldn't have changing data in program directories, so in most cases we direct this settings directory to the global and unfortunately standard
Unlike earlier restructurings, the locations of these directories are critical to the operation of the software packages. Many components have accompanying search paths, that I have adapted in different places in the system. However, packages don't always use these and may try to find components in fixed locations or relative to themselves. Many packages configured with the GNU AutoTools support build options to change these locations. As much as possible, Builder now uses those options to perform these moves, so the packages are aware of them. However, many package configurations are partly broken, especially regarding this because few systems are interested in changing these locations. Such packages may need to be repaired by adding compatibility symlinks from the traditional subdirectories to the new ones.
This can actually be turned into an advantage. Quite a few resource packages act as platforms for other packages. Those other packages contain extensions in the same location as in the platform package. In other systems, these files would end up in the very same directories. In Syllable, the platform packages can't find those modules because packages are built into their own, separate directories. However, they are registered in our symlink pools to create the same view as in other systems. By using the traditional subdirectories in platform packages to symlink into the symlink pools, they can be pointed to the extensions in other packages.
This is the subdirectory relative to a program executable, that can contain private dynamic libaries for the program. Changed it to match the other renaming.
Some of these changes still have the old counterparts in the system, so that most old resource packages and applications will still work on systems with the new structure, until they are obsolete.
Reorganised the init scripts.
We've been porting more Unix server packages over time. They may need setup and teardown procedures that are usually categorised in five types:
Created a new (optional) subdirectory
This name also better reflects the fact that setup scripts are called at every shell start, not just at system start-up or user log-in.
Future scripts can be added under
Reorganised, unified and integrated the documentation in ported packages.
Third-party packages have at least six locations for documentation. We collect loose documentation files in a
This is a mess for users. We want them to be able to go into a package directory just with a filer, and find documentation they are looking for. We were already moving
Any files in
Remaining files are moved afterwards.
The GCC libraries and headers installed in the system now have priority over a separately installed full GCC package. This protects system integrity while allowing to install versions of GCC older than the system libraries. This is a cleaner way than manipulating the GCC package, which was done before.
There are several legitimate reasons for installing a GCC package that is not in sync with the system libraries. The GCC package may be newer than the system release. It may be older, or an old GCC may be required to compile some program. If the versions diverge too much, this can disrupt the working of the system.
Originally, I removed the shared libraries from the GCC packages, but that may not be enough. Then I removed the complete
What's really needed is to give the system libraries priority over GCC, while letting GCC keep its own versions for when it needs those internally. I chose to do that through the system paths. Other options are installing GCC in a separate package pool, or making an exception for it when symlinking in Packager.
Note that other packages in
The symlink pools were moved from
Implemented, in Packager, extra symlinking in the symlink pools to point to the main version of installed resource packages. This allows to point to packages in versioned pools without mentioning the version. The package symlink points to the main/currently selected version.
The links are made in the symlink pools, not in the installed packages. This allows to have multiple symlink pools making different selections of the same packages, for example for different user accounts or software build environments.
Symlinks that are made from the system into resource packages (usually for compatibility reasons) can be completed by Builder. So far, Builder added the complete path of the specific resource package, including the version for packages in the system area. This is not robust against upgrades of a versioned package or a change of the package name, because these symlinks are not included in single binary packages.
Now, Builder completes incomplete paths in a recipe based on the symlink pool of the package. Thus, the symlink does not contain the package name and version anymore and remains valid when those change.
ProgrammingThe situation of the REBOL family of programming languages is currently in a transition phase. We try to select only one of each class of software sub-systems in Syllable, but we make an exception for REBOL because of its innovative qualities. As we predicted many years ago, there are now several open source alternatives, but they haven't yet reached the level where they can fully replace REBOL. In particular, we need REBOL because our in-house content management system still runs on it. We won't make the Syllable base platform dependent on closed source components, so we used ORCA to implement some system tools. Boron is now available as its successor, but we expect that both REBOL and Boron will eventually be replaced by the new Red language.
Boron, the successor to the ORCA programming language, was added. Syllable's system tools (such as Packager) haven't been migrated yet, so ORCA is still included as well.
Here is a Boron presentation we did for Software Freedom Day. We also made a web site where you can try out and compare REBOL and Boron on-line.
REBOL 3 was added, and a cURL binding for it, written by us.
REBOL 3 has been split into a core interpreter library and an open source host environment for integration with as many systems as possible. Kaj has got this version to work on Syllable Desktop. This is the first time that a version of REBOL can run there (besides emulating a complete operating system), so this is a major step towards our REBOL cross-platform strategy.
Interestingly, this R3 version is not fully native to Syllable yet and is not fully ported. The open host environment has been compiled on Syllable, but is using the binary Linux version of the core interpreter library. Normally, this is impossible, but it works in this case through a trick. Both Linux and Syllable use the GNU GLibC C library, but the porting layer and the internal library versions are different. Providing symlinks from the Linux library versions to the Syllable versions makes this work. This is exceptional and is a testament both to Syllable's compatibility with Linux and to the excellent isolation the host environment achieves between REBOL 3 and the host platform.
These compatibility symlinks could potentially be used to run other binary Linux libraries.
The current version of REBOL 3 for Syllable interfaces with the system at the POSIX level. Beyond that, you can write your own bindings with system and user libraries. Bindings are written as REBOL 3 extensions. We have created a binding with the popular cURL library, which is included in Syllable as its networking library.
Just like the core REBOL 3 interpreter library, Syllable Desktop can run the Linux binary of the binding unchanged, so that the same binary can be shared between Syllable Desktop and Syllable Server. This is possible because the interfaces between REBOL 3 and its extensions are designed for maximal portability, to avoid dependency hell. Documentation for the cURL binding is available here.
A massive overhaul of the Syllable system headers for C and C++ development. The intention is to stop kernel only types and structures “leaking” into user-space, where they can cause problems.
The changes should not impact user-space code except in cases where the code is not explicitly including headers it should be and relying on the side-effect of other system headers including them (e.g. using
Another important change has been to deprecate the
The changes have added a new
From this point forward new code should include headers from the
There is still work to be done to clean up the headers to ensure the licence and copyright information are correct for all headers, move anything which is in the wrong place (a lot of stuff in the
These changes can and will be made gradually over time.