LEGEEK BLOG

CREATED 20131201

close source game middleware, GPU optimization, and open source

Opengl and similar high level abstraction 3D APIs are not made for high performance, there are mainly around for compatibility. Have a look at mantle, the AMD attempt to get a better 3D API getting rid of many industry standard shortcomings of existing 3D APIs. A game engine middleware, nowadays it means a 3D engine, usually gets optimizations for selected GPUs. Here, selected, ranges from unfair GPU vendor agreements to the ones the game developers like the most, or my guess, the hidden ones in close source drivers. You can expect some games to be optimized for nvidia GPUs, not AMD, and the other way around. Moreover, far in the game life cycle, newer GPUs do not get optimizations and may end up performing barely as good as the GPUs being around at time of the release of the game which got optimizations. To make things worse, such APIs evolve and backward compatibility is a heavy technical burden. John Carmack (with who I disagree totally on his support of c++) found the ultimate answer to those issues: he open sources his older games and protects them with a GNU GPL license. We can see the results, the games get upgraded and optimized for latest GPU generations and 3D APIs. For games which are not going to be opened sourced, the best compromise would be to use an open source 3D engine. The 3D engine has higher semantics than opengl-like 3D APIs and then allows more flexibily regarding new GPU optimizations and support for newer 3D APIs.


CREATED 20131130

open source software is on a death slop

Nowadays, many open source developers do not realise that they should not use all the pieces of software available on mainstream GNU/Linux distros. Indeed, mainstream distros are work of lots of integrators. Namely, many open source developers make their SDK (Software Development Kit) depend on such huge distros. Since such huge/mainstream distros are out of reach of an individual or a small group of individuals, those mainstream huge distros are becoming kind of mandatory in order to be able to do some development work on more and more open source pieces of software. An another way to put things is it's near unreasonnable to setup your custom distro for development work on more and more of open source piece of software. For instance, the glib SDK dependency tree is technically massive! The build system is based on the GNU autotools (insane shell scripts, makefiles, disgusting perl5), and many code generators (perl5 and python2 both with additional modules).


last edit 20131016
CREATED 20131014

mandatory javascript dependency is evil

A www browser with proper javascript support is a massive quantic jump from a www browser without javascript support. For security reasons, people are righteous to disable javascript in javascript supported www browsers. Hence, to avoid to discriminate light www browsers (i.e 0-javascript lynx), respect the efforts of people worried about the integrity of their www browser, and most of all, respect the basic rules which made the web what it is nowadays, always have a fully featured 0-javascript www portal on the side of your main www portal or have your main www portal being able to fully work without javascript.

  • A www browser with javascript support is the mother of all ultra-heavy clients.
  • A www browser without javascript support is a light client.

CREATED 20131016

C99 partial port of harfbuzz

GTK+ guys told me on IRC they don't care adding a dependency which makes the graphic toolkit pulls a c++ compiler and runtime (through pango). They justified that move with thread safety. Well, I started a partial port of harfbuzz using the C99 language. I discovered at the same time that the original code is actually not thread safe and racy.


CREATED 20131014

gnome shell extension deployment system seems wrong

To be able to customize your gnome shell with user extensions is right and sharing them even more. To force a user system to have one high level language to write those extensions is plain wrong. It should propose any languages. Glib and gobject were made for this. While sharing the extensions on the www, the deployment system should be able to query the user system to build a list of available languages for the extensions. The only required language for user extensions would be C (then the deployment system should be aware of the user machine code, or to be able to compile on the fly the extension which is expected on open source systems), C is the core/alpha/omega language. Never ever expect a user system to have more than a light C compiler (C99 max) with a small C runtime (with the gnome extension APIs). Respect those who are willing to pay the cost of a dependency on python2/python3/ruby/lua/guy/javascript/c++/ocaml/java/D.... and those who only want to be dependent on the core language, C. The only exception to this are close source components and OSes: they just can go to hell.


CREATED 20131010

open source API documentation

Open source APIs tend to do the same thing than close source APIs regarding their documentation. This is clumsy. When I work with open source API, I'm not interested in a detailed list of methods and variables (each method and variable get a description). I'm interested in a bare list of methods and variables (most of the time the name is self explanatory), and working sample code, namely patterns of use. I'll get the details of a variable or method when looking at the code. I can safely presume that most of the open source code documentation tools are mostly not pertinent and overkill.


last edit 20130913
CREATED 20130912

software is a fraud, for most of it

I have been busy building my own software stack from top to bottom, and make that stack be minimal. I did already that in a far past, but I catch up today. It's a fraud. The base software components are being kludgified and their dependencies are looking like a ball a fur for no significant gain. To say the technical cost is astountingly more expensive than several years ago, even for a base system. Let's try to enumerate some clear-cut issues.

The script languages war: you can see clearly the push of the supporters of each script language trying hard to become mandatory on base system. The main open source build system, the GNU autotools is mainly a set of perl5 scripts (with some crazy command shell scripts). Many software packages expect many of their source files to be generated... of course generators are most of the time perl5 and python2 or python3 scripts. Basically they offload their cost of portability to a script language... that would be interesting if only one script language were used, but it's far from being the case. You need *tons* of script languages to be able to set up a source level base system. When I wanted to source compile myself libsndfile, I discovered that some of it's source was generated with autogen... which is guile based (another script language). The glib has some perl5 and python code generators. Aside of the build system, all script languages supporters are trying hard to be mandatory on a base system. For ruby, they got the grub2, the mighty system boot manager.

The obvious forcing of c++: I'm not going to enter deep in the subject, but my opinion on that language/runtime is clear cut: insane. The technical cost of this thing is hardly bearable for anybody with some sanity left. Programmers tend to create with such complex syntax, massive intellectual (say "object oriented) brain fuckages. Some of them try to maintain a good hygene of coding, by not using too much of the language syntax features... well they should do that in C, since the it's a way less costly language. As you all know, gcc cannot anymore be compiled with a simple C89 compiler with a minimal runtime, it needs a full fledged c++ compiler with its runtime. c++ is one of the most selfish computer language out there. I do really think that even the C99 language syntax is too rich... One day perhaps, somebody will design a C- language. Another stunning example of this complexity rampage is man... indeed, nowadays, if you want to properly read man page on your terminal, your need groff, a massive prehistoric document processor, which is a object oriented c++ kludge. Yes, just to process the man page syntax. Bare insanity.

Now, it's time to pay the "price" of that kind of folly: look at tizen, the upcomming mobile operating system from the linux fondation: The software toolchain/build system has a massive technical cost: it's bigger that the operating system itself. All that for the wrong reasons. Some shell command scripts with verbose and simple makefiles should be sufficient (can have a drop of C coded helpers), or something is wrong in your build system design. One additionnal though of how to make all thoses ignorants or idiots (I concur on what Linus T. says) work together: It's near impossible because most of those programmers forgets a C API and/or a strongly defined network protocol, and most of all, forgets to have the decency of packaging their costly script language or c++/d/java... in their build systems (yes it matters not to rely to much on GNU/Linux distros integrators, for those who dare to ask the question). I don't say all the dependencies should be internally packages, but those who depart too much from a basic C toolchain/C runtime should, hence any script language or, for instance, a c++ layer for a big piece of sotfware (i.e. the www renderer).

Their is some evil doing in software development. For instance, some developers will want to max out the complexity of software to mainly make their little person essential. Tons of layers and ultra high level of generalization/factorization are BAD code practices. If the software developer who is over zealous on the previous matter, is perfectly aware of those problems, it means he/she is perfectly dishonnest/sleazy. Some of them can be honestly lacking some experience to be able to apprehend the big picture, but they can be bare idiots too. I'm optimistic, I do really want to think the main cause is a lack of experience and can be cured. Without obvious hints, I will default to that opinion. Well... it's harder every day... sometimes I ask myself if there is a group of powerfull evil people impeding hardcore any attempt to have a clean/minimal software stack... whatever the reasons, those cannot be rigtheous.


CREATED 20130827

weston backend and asynchronous page flipping

I used the linux frame buffer weston backend as a template for my radeon SI driver. That was not a very good idea. Indeed, the rendering timings of weston are based on asynchronous events. I coded a quick and dirty synchronous page flipping mecanism... then I'm now switching to an asynchronous mecanism for page flipping.


CREATED 20130820

radeon APU (Accelerated Processing Unit) memory sharing and DCE (Display Controller Engine)

I had a deeper look into memory sharing between the CPU and the GPU on radeon APUs: it's a mess. Actually it's more complex than a discret GPU, namely a GPU with its own memory. With APUs the memory is split at boot by the bios: one slice of memory will be "local ram", namely fast and for the DCE (the frame buffers), the other slice is for "system ram". GPU (not the DCE) and CPU can share the 2 slices, but with the addition of cachable system memory, the programmer has to be very carefull, because some data paths (the way to read/write data in memory) can be a huge performance penalty. Actually, better see the GPU of an APU as a discret GPU and forget about most data paths. The playstation 4, based on what I got from the net, will have an unified memory access, namely no more slices. Full speed for the GPU, the CPU will do its best with the high latency from GDDR RAM. If this happens to be what we really going to get into the ps4, well... better wait for PC APUs based on the same architecture and stick with mobile discret GPUs. I do give all my condolences to the official radeon developers... because coping with such brain damaged hardware must be really irritating. Of course, for PCs, switching to an unified memory access means the DCE must get VM (Virtual Memory) to overcome the issue of getting continuous memory buffers for frame buffers.


CREATED 20130818

youtube mobile or youtube zero java/ecma script

In France, in Paris at least, it seems youtube is killing its mobile portal. I get a http redirection on the main portal. Many functionalities in the main portal requires a browser with a full java/ecma script engine... which is super evil when not providing a zero javascript www portal. Additionnaly, when I tested the HTML5 support with a kludgy/expensive browser (gecko/webkit/c++ based), it got really bad: nearly 100% of the videos with a webm file available forced the use of the flash player. I kept a cool head, since my internet access is probably not a real one with a transparent proxy (and certainly worse), from my IAP (Internet Access Provider) or from the french intelligence services (I do wander in computer security circles), proxy which modifies my view of the www.


CREATED 20130818

Elysium

From a scify point of view, I got exactly the same feeling than with After Earth: many very good design ideas, but as many very bad design ideas (I'm bad mouthing, After Earth felt better on that aspect). I mean many scify elements aren't... they are magical elements. But still, it's a good basic action movie with the usual too much and the little drop of incoherence, even in the rule set of its scify/magical universe. I would not be surprised to see this movie having some bit of success among the teenage geek population. The main "message" of the movie is too easy and cheesy, I expected it to be less obvious with more sophistication, something a bit more subtle (see Miyasaky animation movies). Entertaining and efficient, average plus.


CREATED 20130802

custom radeon SI driver progress and official dpm (dynamic power management) upstream being stabilized

Erf... I finished a first version of the code for different kinds of linux bus mappings. The radeon dma engines worried me a bit since their fence command must be used (I was not expecting that) in order to function properly. Good news from upstream, maybe my powerplay table bug fix did something, but radeon dynamic power management is in. It's done using a integrated microcontroller called smc. It's still in testing, use at your own risk (really low though... :) ). agd5f, the main radeon open source developer, wrote a very nice description of radeon dynamic power management.


CREATED 20130725

passing pointers on a reference counted object between parallel tasks

The mmapping races made me think about races when 2 parallel tasks are exchanging references on an object which will be freed from memory. It's tough, because if a task tries to access the object reference count which is gone from memory, ouch! Basically, basing the lifetime in memory of objects on a reference count is easy while the processing of that object is mono-tasked. If several tasks want to share that object, the reference counting must be synchronized with a read/write primitive not in the object memory. For instance, a task would do a final (reference count dropped to 0) put() on the object, and then would free the latest from memory (with the reference count!!), while in parallel another task would try to get() (increase the reference count) of the object which is not anymore in memory. Then, in the use case of this type of objects, the atomic type for the object reference count... is nearly useless... just there to ensure you can "parallel" get() the object on all architectures since a read lock will not block the "getting the object pointer" parallel tasks.


CREATED 20130725

linux mmaping

This is related to my radeon Southern Islands GPU driver. I'm coding the mmaping of a bus aperture, and I realized that the proper way to do it was difficult to grasp. The inner linux callbacks gave me troubles with races. Indeed, existing code for this is huge and complex, then understanding how to handle properly those callbacks in a multi-processor context is a matter of days of reverse engineering tough code. I had to write some code to create races and see if I must do multi-processor sync myself or it's already done by the kernel code paths, and I'm not finished yet. Those callbacks are struct file_operations and struct vm_operations_struct.

I'm going to generate a race that close() for good a struct vm_area_struct (last reference) while a fork() will open the mmaping (in theory). If the sync is done by kernel code paths, the fork() should be stalled by the final close().

All that made me think about races when 2 parallel tasks are exchanging references on object which can be freed. It' tough, because if a task tries to access the object ref count which is gone from memory, ouch. Basically, basing the lifetime in memory of objects on a reference count is easy while the processing of that object is mono-tasked. If several tasks are shared that objets, the reference counting must be synchronized with a primitive not in object memory. For instance, o


CREATED 20130725

The Wolverine

Well, this is Wolverine in Japan. Or I should say, Wolverine in an japanese (asian) animation. It's that japanese side of Pacific Rim, but 10 folds. I felt like watching a live action drama of a anime, really! The mutant super power magic (means defies laws of physics) is well suited here, where you can cut chains with a saber, several times. This movie was done properly. 3D useless as usual. Average plus.


CREATED 20130725

Pacific Rim

Well, I'm not disappointed with this movie. It gave me what I was expected: braindead totally crazy action filled with tons of testosterone, done properly. I did notice the japanese (asian) side (animations and movies like gamera/godzilla), which may look ridiculous for those who are not used to such things, but hey... that movie does celebrate "ridiculous" and flashy colors, alien slime and so on. Oh, and as usual, the 3D is useless, even unpleasant in fast paced action sequences, because you need to adapt your eyes to a new depth most of the time each time. Average plus.


last edit 20130725
CREATED 20130716

Monsters University

As a lot of you may know, this movie is the prequel of Monsters INC, still from Pixar Sudios. Well, it's a very pleasant surprise. I went to the movie theatre with a certain amount of pre-conception about it. Actually, I did not like most of the trailers, maybe that's the main reason. This Pixar animation is really good, with a lot of references to the previous movie. This piece of entertainment is definitively worth the money. Most of the trailers are not parts of the movie. Good.


last edit 20130725
CREATED 20130706

Despicable Me 2

It felt less good than the first one. It is really for children only. As an adult, it's not interesting to go to see it. Average plus.


CREATED 20130701

the google movie

Way too chatty and way too long (good job though)... but hey, trends: www companies must have their movies.


last edit 20130625
CREATED 20121224

wayland implementation

I'm able to program the 3D pipeline of the radeon Southern Island from a standard state. Now, I'm into a wayland implementation attempt. If successful, it will be minimal, namely based as much as possible on linux syscalls. Of course, it's protected with a GNU GPLv3 license.

I'm really on and off... hard to keep the motivation, since I cannot forecast my life more than a few weeks in the future. I don't know why, but it's very hard to stay focused, like being way off from my comfort zone. But still, I do add some code to my wayland implementation, and expect to switch using it someday in a future not that far on my systems.


CREATED 20130625

Superman

This movie is also AAA among action movies, like Star Trek Into Darkness. This is not "superman", it's more the story of a very advanced alien on our planet. The tone of the movie is very different from what was done before, and as far as I'm concerned, I'm satisfied fairly enough. A "tour de force" was to hide as much as possible the "magical" aspect of superman powers with a kind of super science.


CREATED 20130625

Star Trek Into Darkness

This movie is AAA among action movies. This is a visual firework and firmly coherent in the Trek universe. A really good job. A special note on the klingon reboot, amazing.


CREATED 20130611

After Earth

This movie contains a lot of good ideas. But some of those ideas are too much. The main issue with this movie, it feels flat and empty: I got bored, it's a miss.


CREATED 20130302

GCN (Graphic Core Next)assembler/desassembler

To avoid to write GCN machine code binary by hand, I did write an assembler/desassembler. Both are partial, but will grow based on needs. The first needs will be those of my wl_shell wayland compositor.


CREATED 20121224

wayland implementation

I'm able to program the 3D pipeline of the radeon Southern Island from a standard state (still some upstream fixes need to get into my code). Now, I'm into a wayland implementation attempt. If successful, it will be minimal, namely based as much as possible on linux syscalls. Of course, it's protected with a GNU GPLv3 license.


CREATED 20121224

udev fork:mudev

It seems the main developer of udev, who is the one of systemd kludge, forced the recent udev version to be dependent on the later kludge. Since udev is a bit brain dead, I allowed myself a custom fork. The build system is free from the GNU autotools (hence the toxic perl5 dependency from automake).


CREATED 20121224

custom linux early userspace

Configuration of gentoo genkernel with openrc is a pain and way overkill for most needs (... mine). Then I started my own implementation. It is minimal, direct on linux syscalls. Of course, it's protected with a GNU GPLv3 license.


CREATED 20120813

AMD GPU hardware programming patterns

Like "design patterns", I started to implement basic patterns for AMD GPU programming. Namely, you would have raw hardware commands and raw buffers to perform very basic functions. Of course, I'm into trying to render a basic triangle primitive from a standard state.


CREATED 20120803

Linux AMD GPU driver programming continued

I am still refactoring the official DRM driver for AMD GPUs. I recently upgraded my hardware from an evergreen based board to a southern island based board. Those are GPU generation names. In the southern island generation I have a tahiti codename GPU. From a marketing point of view it is a HD7970 board from gigabyte manufacturer.

I stick to my plan, to follow latest hardware, and once enough features went into the driver then keep code. The evergreen code is usable anyway. I still do not want to do the hdmi code since it's royalties based. If one day it happens, it will be backed up by a company with a solid legal department.

Now, I can say this: it is really a pain to code a driver without understanding the different silicium blocks and the way they relate to each other for a complex piece of hardware, and that even with open source code!I don't know why AMD is not releasing such information: the official engineers *do* have this information. It's not fair :(. The bright side of this: you can ask them for such information on IRC, they do answer most of the time.


last edit 20120603
CREATED 20120601

deep into linux address space and memory management

  • linux core management of virtual address space is done with vm_area structures.
  • the mapping of the virtual address space to real address space uses 3 or 4 page table levels: pgd in linux code means PaGe Directory which is the top level, pud means Page Upper Directory which is the level following the pgd level. pmd is the Page Middle Directory, the last level before the page entries. pte the Page Table Entries, the last level where the physical address to virtual space mapping happens. Till those tables are not filled in the CPU and the TLB is not synchronized, the mapping does not happen.
  • it seems you don't deal with vm_area structures directly. One subsystem to deal with vm_area structures can be the vmalloc subsystem. The vmalloc subsystem tracks the vm_area structures it's using with vm_struct structures. The vmalloc subsystem has its slice of the global virtual address space between VMALLOC_START and VMALLOC_END, which depends on the architecture.
  • when you want to map pages into a kernel contiguous virtual address space, you can use the vmalloc subsystem. You would use the vmap/vunmap function pair. vmap/vunmap flags are those of vm_struct structure. Careful to grep those flags into the linux code to be sure to set them properly for the code paths you will use (don't worry, usually VM_MAP is enough). This pair of function does contain the page protection flags which will be set in the page table entry: those are very architecture dependant and mostly pre-processor based, then care is to be taken for the support of your target architectures (usually, PAGE_KERNEL only).
  • do consider using a linux lxr to navigate through kernel code.
  • the kmalloc subsystem is not using the vmalloc subsystem. It is different as it allocates a continuous physical space of memory. Since significantly sized continuous space of physical memory tends to become a scarse resource as the memory system gets fragmented, the kmalloc subsystem targets small memory allocation.

CREATED 20120126

Youtube and worldwide royalty free codec, webm, and HTML5 video

WOW! Youtube HTML5 video player got some love, and webm (high quality video, worldwide royalty free, codec) transcoding server farms seem to work quite faster than before.


It really means, you just need an HTML5 video browser with webm support to browse Youtube... The competition is sent near pluto.


last edit 20111004
CREATED 20110917

little break on driver refactoring, forking OpenBox to lboxwm

I need a little break from drm code refactoring for AMD GPUs. I always wanted a window manager which I have control over, then I selected the closest to my need, OpenBox, and started refactor its code as lboxwm or the Linux Box Window Manager. Openbox has a nice code. I think this user land break as a vacation from kernel land... that's a geeky kind of joke...

Actually, it's a massive amount of complex code. I was surprised by its complexity. I don't know what I will do, it will take a good deal of time to refactor it to my liking. Moreover, coding the tuning I would need may be quite harder than expected, due to the mangling of all its features.


CREATED 20110831

youtube and webm

I don't know what youtube is doing with webm/html5. Indeed, all the videos I would like to watch are available in webm. So why I still need most of the time to install *the evil plugin that shall not be named*?? Are your streamers that *hard* to upgrade? Are you generating hate on purpose?


last edit 20110515
CREATED 20110206

scouting by forking/refactoring

Ok, I'm into a refactoring of the Linux DRM for AMD GPUs. The primary goal is to explore in the real what was done. But goals are quite different from official DRM ones. Then, it may end up like as a sibling project.

I get deeper and deeper in DRM drivers for the AMD(ATI) GPU refactoring. I did really different design choices from the DRM ones:

  • The DRM drivers have a kernel abstraction layer. My choice is to hardwire the driver on Linux APIs.
  • The AMD(ATI) DRM drivers are factorized over all GPUs generations and families. My choice is to factorize the code over one generation only. And if a family gets too different in its generation, it would get its own driver.
  • The refactored driver would provide only native linux APIs. Generalized APIs would be done somewhere else (kernel space and/or user space)

last edit 20110322
CREATED 20110309

ffmpeg-mt switch

I switched from ffmpeg to ffmpeg-mt. High bandwidth 1080p is now mine. To do it on Gentoo GNU/Linux, use this report to set it up with minimal trouble.

Well... ffmpeg-mt was merged in ffmpeg. Now the thing is to wait and see how the development will take place. Like Linux? Namely, code will arrive in ffmpeg-mt then will be merged in ffmpeg?


CREATED 20110303

µpdf

It is a PDF reader. Actually it's a library and a set of tools to render PDF data. Some would say, "oh no! one more!"... but that one is coded using the C language, protected by a GNU GPLv3 and has an experimental source versionning using GIT.


last edit 20110218
CREATED 20100607

chromium and gconf/orbit

Chromium depends on gconf then orbit. Since those latest pieces of software will be soon phased out from gnome for gsettings/dconf, I had a look at chromium source code to see if in the meantime it's reasonnable to disable the dependency... Well I had a surprise, gconf is used for 2 things:

  • positions of the chromium title bar buttons (minimize/maximize/close)
  • one of the backends to store proxy settings

Well, I asked on chromium devel mailing list if it was desirable to provide a patch in order to disable that dependency, a "proprietary OS" traitor in google said "no".


good news, gconf dependency is gone from chromium. Waiting for GTK+ 3 support now... and nss removal... better use GNUTLS.


CREATED 20110218

OpenGL 4.1 core profile

I started to read the OpenGL 4.1 core profile specifications. A friend of mine told me that it's way fat free that OpenGL 2.1. I'll try to make some notes. Better have the opengl 4.1 ways to see 3D in mind while dealing with AMD GPUs.


CREATED 20101229

MeeGo, clutter and qt

I ran into something quite weird today. I checked mutter today, the new gnome window manager, and then clutter which is a kind of sibling for mutter. Clutter is presented as a core component of MeeGo, basically it would mean it's the toolkit of the MeeGo platform, and it would not be qt. Clutter and mutter are ways more welcome than qt as a toolkit/wm for a mobile platform. The rub is, what is that mess? What does it mean?


CREATED 20100916

yaperperl

At work I was granted the right to publish this C coded TCP/IPv4 server for perl5, protected by the GNU AGPLv3. The nice thing about it, it allowed me to lay down the base of glibc by-passing code to deal directly with the Linux syscalls (Bernstein would be happy).


CREATED 20100704

Linux rt sigaction

I'm currently coding directly on Linux syscalls. Then I expose myself to all architecture dependent code in user space. Futex was one with the atomic operations you must perform in user space and now... signal handlers. Indeed ABI (Application Binay Interface) of x86_64 for the stack is kind of a nightmare. Then you must have special code to deal with the return of a signal handler, namely restoring the stack. And for that code to execute without messing your stack, it will have to be a few lines of assembly. And then GDB lose the ability to see your signal handler.


CREATED 20100607

GNU libc and Linux futex

Erf... most of the time I try to use directly the GNU libc (glibc) Linux syscall wrappers... well for futex to work, user space architecture dependent code is needed. For now then, it's more reasonnable to use pthread. Latter maybe, a good idea would be to create a "pthread-like" lib for Linux based on GNU libc code.


CREATED 20100607

mysql/mariadb port to C for GNU/Linux

At work, I started to port mariadb to C. I have been looking for a C coded SQL engine (mysql compatible) with a lean and clean storage plugin interface. The closest is mysql/mariadb, then I used mariadb as a base for that port. For the moment, I do remove all the code tricks related to portability... namely mariadb is on a code diet.

With that, people wanting mysql-like access to my data stores will, and still, I will have my programs dealing directly to my data stores.


CREATED 20100607

AMD GPU

Finished to read the AMD GPU published documentation. Now need to fork kernel space ATOM bios parser and EDID parser to make them kernel modules and remove kernel abstraction code... if all that is requiered.


CREATED 20100306

upgrade to nouveau driver for GPU (Graphics Processing Unit) support on my workstation

It works. I had to downgrade the libdrm to a pre 2.4.18, but it works with nouveau kernel DRM (Direct Rendering Manager) from 2.6.33 and a G7x hardware. The API breakage is not a big deal. Still there is this kludgy kernel abstraction layer and that dangerous BSD licence. I don't have brain time to deal with it, and if one day I do, I'll probably start with the AMD radeon stack since they are publishing the hardware programming manuals. Meaning only those GPUs are worth any driver improvement and cleanup work.


CREATED 20100306

upgrade Shuttle P37 to latest BIOS using flashrom

Shuttle, you are smelly, don't be pranks and support GNU/Linux.

flashrom guys, you rock, but where is the donate button??


CREATED 20100302

upgrade MSI K9A2 Platinum motherboard BIOS to v1.9 using flashrom

MSI, you are smelly, don't be pranks and support GNU/Linux.

flashrom guys, you rock, but where is the donate button??


CREATED 20100302

OCZ Agility EX firmware 1.5 upgrade, Linux 2.6.33 and discard file system option

Now I have a full TRIM supported SSD system disk in my home station.

  1. Upgrade the disk firmware. For this I went to OCZ support forums where you can download an ISO file to burn on a CD/DVD which will upgrade your firmware when you boot with it. I had to downgrade my motherboard BIOS disk support from AHCI to IDE for proper disk detection.
  2. Setup Linux 2.6.33 for TRIM support in libata sata.
  3. Added the file system discard option in /etc/fstab.
  4. Setup the deadline block I/O scheduler for the SSD in my boot gentoo local script (CFQ is near useless, I read).

I hope motherboard manufacturers will use this method to flash the firmware.


CREATED 20091019

we do not live in a free world

I tried today to buy a SSD drive from OCZ (the SLC agility EX of 60 GB)... amazon.com sent me to hell. Why? I checked the announced price on a french online shop... well twice the price, and not even available because it was sold out. If you have found cheaper and available in the USA, you are not free to buy it from Europe even if your are willing to pay the taxes (will still be ways cheaper).


CREATED 20090726

Change Of Plan For World Domination

Well... I'm going to drop the nvidia reverse engineering plan... indeed I would not stand to have my work used by a BSD driver without a real and GPL Linux driver. Then, as soon as I finished the AMD documentation, I start a new GPL driver for those GPUs... AGPLv3, for Linux only and I'll try to make one module per GPU flavors with a source based framework.


CREATED 20090302

SVG (Scalable Vector Graphics)

Okay, I got the things right in order to author SVG diagrams cleanly. The idea is to use the user unit everywhere then to scale to a real size thought the top level width/height properties. The size of the canvas is defined with the viewBox attribute on the root svg xml element. The other little trick is about the borders. Indeed most of the time, with diagrams reaching the border of the viewBox, since the border is drawn on both sides of an edge, the outer part of the border gets out of the viewBox. The trick is the resize the viewBox accordingly and translate the lot of half the size of the border thickness. It's neat because using real units does not feel right. For instance, basic paths are defined using the user unit, even if real units seems to be supported by many SVG renderer.


CREATED 20090214

bittorrent filtering?

Got some torrents for an online game... they are stuck. Plus, it happened for other torrents as well. My download rate average is significantly dropping. Something is wrong.


CREATED 20090210

RSVP (Resource reSerVation Protocol) or diffserv (differentiated services) for ISPs (Internet Service Provider) terminal equipement?

Had a bit of discussion on a IPv6 (Internet Protocol version 6) related mailing list. And it seems that we will have to choose between RSVP and diffserv for the QoS (Quality Of Serice) IP protocol for ISPs clients. The trick is that RSVP is a fine grained QoS definition protocol that can only handle internet borders regarding its scalabily. Diffserv is coarsed grained, but it scales well better... and it seems quite more "internet spirit". Namely an ISP client should be regarded as a ISP itself, like two networks connecting. Really it seems just a matter of setting the IP traffic class field to the proper diffserv normalized value... need to recompile my Linux network stack in order to become diffserv aware. Hope my futur FTTH (Fiber To The Home) connection will be diffserv aware and my ISP's IPv6 will be native on its backbone.


last edit 20090210
CREATED 20090206

little seletion of pieces of sofware for everyday happy hacking

C language and most of the libs are LGPL

Once you know how to use them... well...


CREATED 20090206

IPv6 and traffic classes

My SSH sessions are barely usable when my internet connection is loaded... Stop loading my connection? Nope, that's lame and won't help when my neighbors will load the upstream link with their FTTH (Fiber To The Home) connection, or with many people surfing on video web sites on my currently connected wireless bus. One clean answer: need traffic class support. We have the proper fields in the IPv6 header, now we need a good IP based protocol with Link Level collaboration in order to publish to the ISP (Internet Service Provider) equipement traffic classes of IP streams. This is not filtering, it's IP(low) level priority management and bandwidth limitation on client initiative. If you think it's deep packet inspection with limitations based on ISP initiatives, think twice my Big Brother.


CREATED 20090206

reading SVG (Scalable Vector Graphics) specifications

In order to write my nvidia programming manual, I decided to use SVG for illustration. Well... those specifications do not deal with graphics :)! It would be more appropriate to use "Full Fledge Graphical Document" instead.


CREATED 20090202

Tokyo Cabinet

In War Against Bloat, there is the SQL engine threat. Indeed, many coders do not realize the software cost of an SQL engine, it's just massive. Let's take for instance mysql... have a look at the code: C++ brainfuckage!!! EEEERK! An insane amount of brain fucked C++ code lines!!! And believe me or not, but if you want to create a storage plugins in order to plug your disk data format on their SQL engine, you have to write code using C++, there is even the use of a C++ template in the plugin interface! Yes you read well... and that sucks big. Some of you will shout that I should use postgreSQL. Well it's C code, still very complex and it's a BSD license, then if I can avoid it I will. Moreover, there is no storage plugin interface that I would have needed... so that's a dead end. Finally, I'm back to serialize my memory structures on disk since nowadays computers have much more memory (yes, 32GB DIMMS are on their way thanks to 50nm process). Well, it does not work with big indexes and huge amount of data. And I found out about tokyocabinet, a DBM style database which is damn sexy. And that does the trick: once computer memory is not enough-->tokyocabinet, there is even transaction support! Of course, it's lean beautifull C code under a LGPL license... And the final blow, there is a network access library and a text indexer optimized for tokyocabinet. BSD integrists and bloat lovers can just go to hell.


CREATED 20090202

ISPs (Internet Service Providers) Grand jump: FTTH (Fiber To The Home)

FTTH will be a crash test for many ISPs. Indeed, the available upstream bandwidth will be so much higher compared to xDSL that ISP backbones will suffer a lot. In France, FTTH deployment is taking ages, to such an extreme extend that in some communities suspicion is on the rise. Rumors are even amplified with all the filtering (under the cover of digital freedom killing systems) which becomes really ridiculous. Rumors tend to that: The backbones of many ISPs won't handle the load with FTTH mass deployement, they are forging a set of a good excuses to cut down the bandwidth. And of course, it will target p2p. People reading this blog should start to download legal content (we have plenty) or content that probably won't find their way in your country, and share it in order to maximize the use of their bandwidth (the idea is to delete files on a regular basis in order to maximize bandwidth use). A good ISP would increase properly the size of its backbone in order to handle the load, not forging good excuses to justify the freedom killing means to cut down the bandwidth. Well, an other interpretation of this issue would be: western countries are heading straight towards a digital totalitarian system for the sake of a few. That can only end badly.


CREATED 20090129

xpdf and the 72 DPI (Dots Per Inch) limitation

Damn! I got some PDF files and wanted to visualize them in real size on my screen. Well... better know than the zoom factor of 100% is hardcoded to 72 DPI. In theory, an xwindow screen can be an assembly of several displays, with different DPI. That would mean that a properly coded xwindow application must be able to draw parts of itself based on the target display. I haven't check but I hope xwindow is able to do so. I think it deals with the xdamage extension, but I may be quite wrong.


CREATED 20090127

inkscape and the 90 DPI (Dots Per Inch) limitation

To get used to write SVG file, I finally decided myself to start with inkscape. Well... I really do not like the code since it's C++ with garbage collection, but it's GPL without the constraint to surrender your copyrights to a company, then a way lesser evil. From there, I wanted to draw SVG using mm units and be able to see the real size on my screen.... Ouch! Inkscape has an hardcoded DPI of 90 for screen display. Namely it does not care about the screen advertised screen DPI... BAD! After talking to one of the inkscape hackers in the XMPP/jabber conference, the problem is well known and not addressed because they are much more important problems to solve first. Of course I got the now classical "provide patch!"... well, no and no again, and I would have done it, but C++/garbage collection for such a program when you look at gnumeric and abiword code? Well, inkscape is very nice, but the code... adding the requirement of a C++ compiler PLUS weakly surrendering memory management to a garbage collector, sorry that's way above my tolerance threshold... and I don't have the time to recode it using C (here I'm being a smart ass).


last edit 20090127
CREATED 20090118

ahem...

Well, tought to maintain properly a blog. I have so many other things to do at higher priority. Well, on the GPU reverse engineering project, since I do consider the BSD license as a threat, the best thing to do is to write a kind of programming manual in order to be able to write very quickly a GPL driver for Linux. Then the focus will be writing code in order to reverse engineer specific hardware functions, and certainly not in order to achieve a functional driver (since GPL only code will be tossed away from the project).


last edit 20080819
CREATED 20080219

WiMAX (Worldwide interoperability for Microwave Access), IPv6 (Internet Protocol version 6), mobility, QoS (Quality of Service), XMPP (eXtensible Messaging and Presence Protocol) and jingle

IPv6-->no more NAT (Network Address Translation). Means any terminal can reasonably be public on the net. IPv6 mobility means your mobile phone can become a mere terminal on the net. The main issues with mobility are expensive signaling, home network management, latency on highly loaded WiMAX network during hand over. Thanks to jingle, which seems to be a quite simple protocol compared to SIP (Session Initiation Protocol), XMPP will be able to negotiate a RTP stream for video/voice call. They are client program issues:in order to have a decent latency for voice and to make the emergency calls going through, priority traffic and QoS will have to be set up. IPv6 label? ethernet VLAN (Virtual Local Area Network)? Well, the client will have to deal with those settings. Question, how those settings will be known to the client program... hum probably new IPv6 advertisements messages are needed.

Why QoS since priority traffic should be enough? Well, in order to avoid smart people to make their P2P (Peer to Peer) traffic high priority on the net, let be this priority traffic just big enough to let a compressed voice stream go through... namely... almost nothing... Careful on the QoS algo in order to avoid burst abuse by clever people (heard that QoS over WiMAX had troubles).

LTE (Long Term Evolution) is the next telecom mobile network after the 3G network. Indeed, it will probably be competition for WiMAX. But, like WiMAX, it will be full IP (Internet Protocol). That makes the use of ethernet VLAN quite inappropriate for QoS and priority traffic, except if LTE is ethernet based too. Indeed, client terminals will have to be able to auto-configure themselves for voice (low latency, low bandwidth), and if link layers are to be different, IP will be the lowest common layer, so it would be IPv6 labels. In order to make the difference between voice for emergency traffic and common voice traffic, it would have to be based on the destination IP addresses.

Presuming we have N IP(TCP/UDP) phones on a house lan which is on the net with IPv6. All those IP(TCP/UDP) phones want to make a call: they are doing their signaling to established a RTP flow to carry the voice stream. A flow is usually a Nuplet made of (label,flow id,source address,source port,destination address,destination port). Some documents I read on the net (one from cisco 2005) seems to say that less than the latter Nuplet would be necessary to track (just what is needed to minimize flow id collision). I presume they got from the proper equipment proper IPv6 labels which define high priority - very low bandwidth channels. That equipment would be similar to the announce done for the DNS service, or another anycast reachable equipment. How the IAP terminal equipment will manage flow count and lifetime? The algorithm has to be smart enough to avoid downgrading already high priority flows on high load (meaning sustained max count for those flows). All that means a SLA (Service Level Agreement) between the customer and its IAP and between IAPs themselves for traffic class transitivity.

Ok... all the trafic classes stuff is named intserv. Seems that, the software will have to make a request on the link using RSVP with the RAPI, then the IAP will or won't honnor the QoS requirements of the flow. Well, have to read all the documentation to check out if it solves that big issue.


LAST EDIT 20080601
CREATED 20080127

Byte code virtual machine for webscript-->dangerous!

That means proprietary rich web clients which will destroy the open source philosophy of the web. I expect the mozilla project to send to hell any poisonous gift that would look like a virtual machine.

Well... the mozilla foundation surrendered: the evil abobe (flash) is destroying spidermonkey to put a bytecode based virtual machine coded in C++. Spidermonkey is currently a clean C coded webscript interpreter. And additionnally they are taking over some parts of the mozilla project. Once again, I was right and there is only one cure:minimalistic/simple code with the father of all languages, C, and no more.


LAST EDIT 20080601
CREATED 20080326

new system: idle power requirements

I bought a wattmeter and plugged it on my brand new system for nvidia chips reverse engineering:

  • first power supply: idle 10W
  • second power supply: idle 10W
  • 21 inch CRT monitor: idle 30W!

That makes my system burning 50W when idle. That's insane.

news:

I replaced my CRT monitor with two LCDs (for video mode setting development). Both burn 12W idled. Saved 20W.


CREATED 20080508

reporting bugs on mainstream GNU/Linux distros

Most of the time, when reporting a bug on a mainstream GNU/Linux distro, using the standard search in the web interface for bug reports, you cannot find your bug and then file a new one... wrong... Indeed, 99% of the time that bug was already filed by someone else and dealed with. But the standard search looks for opened bugs, and a dealed bug is rather often closed. So for geeks who update their box often and get many bugs, better use the advanced search feature of the distro web interface for bug reports. That in order to select bugs of any state (closed, opened, etc...) for their search.


LAST EDIT 20080304
CREATED 20080127

freedom software hijacked

Dual licensing is dangerous for open source software. Indeed, more and more critical pieces of software gets dual licensed. Namely, you have the community edition where you have sources and a FFII compliant license, and you have binaries on a proprietary license. And of course, mysterious forces make the binaries faster, more featured and properly supported. Patches may be applied on binaries significantly before it is made available for the GPL community edition, or even worse, kept only for the proprietary version, screwing big time the GPL spirit. Let's get deeper in the nightmare, for instance with lzo. It is clearly stated that the GPL version of the software is significantly downgraded compared to the proprietary version. Moreover those dual licensed pieces of software can be bought since to be compatible with a proprietary license, one company is behind (it may be a collective of companies, usually there is still an head). They can be bought in order to be sabotaged if any big corporation judges it needs it. It happens more and more in the GPL world (lzo and 7-zip compression software, mysql...), but it is the standard in the BSD world... One more reason to avoid as much as possible the latter world and one more to be cautious with time bomb GPL software.

I asked myself what would happen if you contribute to those pieces of software and do not want to surrender your rights in order to allow the mother company to provide a proprietary version. Because, the spirit of the GPL is the code you contribute stays GPL. Actually I prefere not to think about it...

Another way is the bloat. The mother company, a big one, creates a piece of software huge and complex and brut forces it on the market. Only itself will have the inertia to reasonnably maintain it, trapping any developer or user (java and mono). And, of course, the bloat can be GPL and does not need a proprietary version.

The reality is more subtile than those extreme cases, but it's like a recipe: you add a bit of that, more of this but not too much, etc. It's more and more important that the community of freedom software realises that critical and system software should be secured from those dangerous paths. How to do it? Easy: minimal (C/Linux/filesystem/network) and GPL only.

Look at the mozilla fondation (may need an audit in order to locate where is going the money). Binaries are straight from the same source tree, and if you want to give money, you give it to that fondation which dispatches it to development and communication. Well nothing can be perfect: most of their code is C++.


LAST EDIT 20080224
CREATED 20071111

facebook spam

It's very funny. Just a few days after the famous Al Capone company got some shares of the greatly useless facebook, I got a combo invitation from three of my girlfriends. So I'm going to do like as usual, since I'm like THE counselor on anything about computers for my friends: keep away of this site since it was corrupted with dirty money. I don't need to know when facebook will announce the switch on better (LOL!) technologies (that will be based on technologies from that company), since I know that will happen. Whatever, the initiative from Google with its Open Social federation seems way more elegant and better... as usual.

Well, I got some info about facebook from one of my work colleague. Facebook is a kind of web second life (if you know how to code). I will never register to this site since dirty money went through and its future will probably end in hell, then I'll take my colleague word for the following: if you know your way around using webscript you can develop little funny applications. But is the cost of being jailed into one site (which will turn bad) is justified with the ability to code little web apps only? Does the openness of the Google solution, orkut, not worth it because it's not possible to code little web apps since it's a cluster of web sites with different technologies?

There is a new update: orkut web site does contain that horrible page extension which tells us that Google is using its worst enemy technology! I don't know what they are doing at Google but that's very bad news. It's like Google calendar which does not support the calDAV protocol... All that is smelly.


CREATED 20080219

Nouveau

BLOOOOOOOOOOOOOOOOOOOOOOODY HEEEEEEEEEEEEEEEEELL!

I just do not have brain time for this. I want to contribute the Linux GPL code for mode settings. I read all the documentation. I'm able to navigate through the code. I even wrote some lines of code. And yet, nothing working since I do not have fu***** brain time. No brain time between work and social/couple life. Here and french here, you can find articles quite accurate about the graphic stack revolution happening in the free world. Now, if I want to be a little useful, I should write mode setting code for the new DRM model and the NV50 architecture (somewhat different from the NV40). Now, I need a devel computer with some NV50 cards. What to do? Reduce my sleep time to a couple of hours and pray I'm able to code something that works (and not end in an hospital room)?


LAST EDIT 20080214
CREATED 20080208

Perl code

At work, since many of my colleagues do perl, I coded some perl bindings for some of my C libraries. Then, that was the right time to dive a little bit more in this high level language since coding bindings requires a minimal understanding of its internals. There is extensive documentation, not that accurate, but without it, it would take many more centuries to figure out your way around its code... but hey, a bloat stays a bloat.

To proceed, the idea is to focus on memory structures. Indeed, coding proper C is coding memory. So I focused on its very core memory structures and how it is dealt with. It seems that perl was developed using the monkey patching way. Monkey patching means adding a new functionality with minimized code modification... even if that means crapping the code.

Perl started with the scalar, a structure able to hold several number types and/or a string; then perl wanted to support arrays and hashes... but arghh, we have already all the plumbing for our scalar value management (have a look at sv.h in perl source tree)... be lazy and add a new scalar type to support those new structures... but hey! The scalar value structure now can hold perl scalar/array/hash which have, from a perl point of view, nothing in common. Then careful, in its code perl scalar structures deals with all perl types including arrays and hashes (PVAV and PVHV types for the C scalar value)... messy.

The C scalar value became the head of a perl variable (yeayeeh, monkey acrobatics!), which you must link to a body (usually their structure name starts with an 'x'). This body will be a perl scalar or an array or a hash and so on... the C body for the C scalar value to hold perl arrays is in av.h and for hashes in hv.h.

Okay, you got the feel of it? Well, this structure can hold several encodings for the same data (ex: a number and its string encoding). To deal with this you have many flags namely (private, public flags etc...), well... you are supposed to only rely on the state described by those flags (hold you breath...). But here comes packages support... then let's add a new body-->piece of code. You said object oriented? Let's force some magic (yes, it is really called like that) into our C scalar structure... it should fit with our package monkey acrobatics. Strange, the state chart of this structure is now a giant hairy monster.

Not the least, if you look at a hash key, well we have a 32 bits for the hash itself, signed 32 bits for the length of the key, and a then the key itself... ok, if you have an utf-8 key, you just need to set the proper flag in the flags field... err... where is the flag field? None... I just want 8 bits... I don't want to scramble things up, I don't want to screw the nice memory alignment of the key with a 8 bits value, let's monkey patch my byte of flags at the end of the key value!

Ok that's just a taste of perl internals... you can guess that I simplified a lot. The damage is orders of magnitude higher. To scramble things even more, in order to deal with such a mess there are hundreds of macros! They mix easily the C scalar value as sv with the real perl scalar value sv... So perl code is not anymore C code, it is a whole new language defined with CPP. To try to solve that issue, I tried myself on the output of CPP... My God! You can see all the decades of tricks to make it work on many C compilers.

Okay, let's conclude: from my point of view, perl is raw brain fuckage, but somehow you can understand quite easily how it works, but hacking on it to make something actually work without breaking anything... would lead to insanity. Perl5 code is dead. The idea is to really look at perl6 with the virtual machine parrot. Hopefully we will have a cleaner way to make XS modules.


CREATED 20080129

Coding lesson...

Today, I got a fight with gnash, the flash GNU project... Oh My God! Total C++ object-orientish master brainfuckage! Hardly working on my systems (no remote movies etc...). I went back to my place disgusted where phoronix brought back to my memories the swfdec free flash project from freedesktop... very few dependencies, clean and small C code with a LGPL license, and working with all the trendy stuff I tested it on!

Bloody Hell! I don't know if I have to hate gnash coders or to thank them: indeed, it is a scholar example of 2 similar projects with the same goal, one master brainfuckage and a lovely one. Perfect for illustration.


CREATED 20080127

Recent XORG X11 server gets its xkb configuration from HAL

Indeed, you can wipe out those input sections from your xorg.conf and write a policy file for the HAL. My /etc/hal/fdi/policy/x11-input.fdi is:



  
    
      xorg
      oss_nodeadkeys
      fr
    
  
]]>

The bad news is that the hotplug code is delayed till xorg server 1.5 or 1.6. But... Hey! At the end, full hotplug configuration through HAL.


LAST EDIT 20080127
CREATED 20071116

Words starting with the letter C...

... as in Clearsilver and Citadel. These are a web template system and a groupware server entirely written using C and not vendor locked mid-level languages or high-level bloats. From now, they will be my choice each time I have to build a system which requires such functionality... Since they are written with C they cay interoperate with almost any high level languages on earth (perl, python, ruby, lua...). I have never used them, hopefully a first time will arrive soon enough to make my teeth on them. Well, well, well, I'm not that stubborn: I will be pleased to test a new language, if it's able to produce a decent kernel :)... I may be stuck with C for a long time and those who are thinking C++, think first of how much control you have on the memory layout of objects or shared object ABI...

Great, I finally used Clearsilver. Easy code to get into, template syntax is fairly enough for what I had to do. My weaponry for web templating is now complete.


LAST EDIT 20080127
CREATED 20071214

Sabotage of the web video/audio standard

There was shown strong lobbying to disable the adoption of ogg/theora/vorbis as the web video/audio standard for the W3C. Now, the sabotage is crystal clear. How to behave? What to do against such sabotage? How to fight?

I had a little email exchange with one of the guys in charge of html5 draft. Well, he's perfectly aware that it is quite a non-sens to update html nowadays without minimal media codecs and containers specification. Since, html5 is an effort from IT corporations, and since now it's in the hands of the W3C we may have a chance. One thing you can be sure of:the web now needs video/audio, the codecs that will be standardized properly by the W3C will storm the web.


CREATED 20071205

Are the music and film industries the new SCO?

How would they be financed by The Borg collective? Ads in movies?

The only way to keep their current business model would be to sabotage massively Internet Freedom. That's a dangerous idea since digital freedom is more and more important in people life every new day. I cannot believe they are that ignorant and refusing that hard to face reality: it's a revolution on a bigger scale than the printing revolution one. Some businesses are going to die or significantly change etc... But they show a quite stubborn attitude which favors all the time proprietary close and coercive software of some vendors...


CREATED 20071202

Intel and NVIDIA SLI soap

The very serious web sites related to main stream hardware are worried to know if the Intel X38 will support NVIDIA SLI. Wrong, and wrong again. Till you have a motherboard with 2 PCIe 16x slots properly positioned (to accommodate the huge graphical adapters) and a chipset able to configure 8x PCIe lanes for each of those slots, you are fine with NVIDIA SLI. But NVIDIA does not want Intel chipsets to support SLI since they do manufacture their own motherboard chipsets (nForce series). Then, knowing we live in a beautiful world, NVIDIA is disabling SLI support in its driver for Intel chipsets... err... for non certified chipsets. Then, a properly patched driver will enabled SLI on Intel chipset... till NVIDIA adds hardware to forbid this in its next revision of SLI OR they get along with Intel.


LAST EDIT 20071127
CREATED 20071122

dbus, hal and ivman

On recent GNU/Linux system, the apcid (Advanced Power and Configuration Interface Daemon) is obsolete because kernel acpi specific event file nodes are gone. Namely, functionality provided by acpi is now abstracted by standard Linux interfaces. Namely, on my laptop, the power button is handled by the acpi now abstracted by the Linux input interface. The user level piece of software able to deal with Linux input interface is HAL. HAL has also a generic way to power off the computer through a method of one of its exposed objects on the system dbus (Desktop Bus). The HAL service is identified uniquely on the system dbus with org.freedesktop.Hal bus name. The HAL object describing the computer can be reached with /org/freedesktop/Hal/devices/computer path on the service bus name, and to power off the computer you must call the Shutdown method on the org.freedesktop.Hal.Device.SystemPowerManagement interface implemented by the latter object reachable with the latter path. To sum up: you call the Shutdown method on the org.freedesktop.Hal.Device.SystemPowerManagement interface of the /org/freedesktop/Hal/devices/computer object on the org.freedesktop.Hal service.

But how to handle the power button press event? That's the role of ivman with the collaboration of the HAL. Indeed, ivman must be configured to listen to the HAL power button press event on the system dbus and, when detected, call the previous Shutdown method. Some of you will ask about policy. How to tell a user has the right to perform, let's say a reboot, on a shared workstation? Well, in theory, that will be the task of the policy kit. But, configuring a full policy is not somebody will do everyday and can become bloated and brain damaged quickly (see the Al Capone company OS for reference), that's why we should put a warning on this and not redo the mistakes of others. HAL emits a condition when the power button is pressed, this is basically a dbus signal. Ivman will catch it and perform the call: configure /etc/ivman/IvmConfigConditions.xml with:


  
]]>

Then you will be fine, namely you will have a power button that will work again. Of course, you can also make your system hibernate when the sleep button is pressed, or the lid is closed. You can handle thermal conditions and batteries condition the same way etc etc... You will find hal interested scripts in /usr/lib/hal/scripts.

Be careful, ivman is usually pre-configured to perform automounting of hotplug drives. Then, if you use a modern desktop, it may interfere with its automount system. You have to disable automount features from ivman configuration files if you have such desktop or another automount system.


CREATED 20071125

Memory matters

If you don't understand this document... try harder or drop coding please...


CREATED 20071120

mesa and gcc

Arg: Indeed, the framework that is going to be chosen for opengl shader compilation in mesa will be something on a BSD license used in macapple opengl stack. I suspect not a license issue since the best choice would have been gcc because it is the most advanced compiler framework. What would be required is a C interface to the compiler tool-chain only, not a full blown plug-in architecture like some are scared of (fear of proprietary or third party open source bloated plug-ins). The pseudo C language for opengl shaders would need to be compiled and linked using the opengl libs, putting the results in a buffer that would be DMAed (Direct Memory Access) to the GPU (Graphics Processing Engine). I hope the gcc guys will address that issue before the other framework gets too glued to mesa. I got a quick look, seems mesa is using a clean interface, gcc would be able to get on the boat, if it's able to do better shader optimizations than the other.


LAST EDIT 20071117
CREATED 20071114

java and siblings

This is open source vendor locking. You need the previous closed source SDK (Software Development Kit), plus some additional closed source binaries in order to compile the open source edition of the SDK. The code is C++ and about 80 MB... 80MB of compressed code with a bucket load of redundant code from already existing beautiful light C libraries. The code is, from my point of view, totally brain fucked. How the hell do you want to maintain this? You must have the resources of a huge company or a community of the scale of the Linux one (well, in this case the code is mostly clean C code). This is a scholar example that code written with an object-oriented language can be screwed by orders of magnitude easier than code written with an procedural language. At least there is JNI (Java Native Interface) which allows,in theory, to call C entry points in a shared object from a JVM (Java Virtual Machine) and to call java code from a C program.

Java is everywhere now because it lowers the level of entry for developers. But modern dynamic high level languages, which for most are lesser bloat and more advanced than java itself, lower the level of entry even more. Conclusion, from this perspective, java and its siblings are quite obsolete. I would add that java and its siblings are statically compiled... which makes them even more obsolete compared to modern high level languages. Java has the advantage to be quite more mature than its siblings (guess what I'm mainly referring to...). Good to learn programming, but must be dropped as soon as real stuff kicks in.


LAST EDIT 20071117
CREATED 20071108

groff/man/utf8 mess

I'm french and I have utf-8 encoded man pages (manual pages) since a long time ago and I have never been able to read them properly since that time (understand "never during my entire life"...). So, finally and irritated, I decided to get some information about the issue. Well, well, well, before that you should know that the man processor, groff, is a full blown document processor (think XML (eXtended Markup Language),XSLT (eXtensible Stylesheet Language Transformation) and docbook). Yes, man uses groff to create properly formatted man pages for the output you chose. That can be a terminal, a postscript viewer, x11 and others. As far as I'm concerned, man pages are read on a terminal (an utf-8 terminal of course) and in web browsers. Here goes the sad thing that will be no more very soon (that's theory): groff, in its current stable version, does not support different input encodings for man pages. Yes, you read well and we are year 2007. The good news is that extensions providing support for input variable encoding is in CVS (arg... hope they will migrate to GIT one day...).

... sadly groff is way overkill to generate proper terminal output or web output. We should have a nice simple frozen dialect for man pages, and minimal and light specialized processors for terminal and web outputs. Let's say I will code one when I have time (cough!), irony again! They are 2 man pages dialects:BSD and standard. Most of the man pages on a GNU/Linux system uses the standard dialect. It is documented there $LANG=C man 7 man.


CREATED 20071116

GPL vs BSD

Linus said "I give code, you give code". A lot of coders values the fairness of the GPL license... and I'm one of them. If coders favored the BSD license, that would have helped a lot to make one the *BSD kernels the mainstream community kernel. But Linux is the One, and I'm glad and hope it will last long.


CREATED 20071116

Basic arithmetic for a GPL coder

She must master the whole software stack: from the kernel to her code and the compiler tool-chain. She is responsible of all components she used to produce her systems. The proprietary way where many components are out of control because closed and coders caring really only for their code is obsolete and weak. I know it's hard, I'm unable to do it, but that's the goal, the target for any serious professional coder around the glob, and that should be taught at uni and school. Big chunks are:

  • Kernel components like the vm,schedulers.
  • The optimization part of gcc.
  • The FPU(Floating Point unit).
  • Cryptography and hash.
  • All the maths which come with previous points.

Then, any additional software component in the software stack must be considered as a huge cost. Highly complex bloats are deadly threats beyond the fact they lead to vendor lock-in most of the time (open source or not). How many lines of C code did you add for that high level language? What is the complexity factor of this code? Because 1000 lines of highly complex C code does not have the same cost that 1000 lines of simple C code. Do not be seduced by the easiness of the dark side of the force, payback always comes one day, if it's not for you it will be for a fellow member of our community. And, careful, I'm far from being alone thinking like that.


CREATED 20071116

Data structures and C

Always start with the minimal self-describing data structures (closure?)... then complexify them only if it is required: remember the UNIX philosophy (or near 40 years of programming experience), optimize only if required. Computers love sequential because memory prefetch is then more efficient. When coding using C, you code memory. There is a suite of amazing articles from Ulrich Drepper on Linux Weekly News, you must read them (here): writing efficient programs is going to be harder on near future computer architectures.

The basic structure would be an array with a terminating element. Realloc is your friend and try to (re)allocate big chunks of mem if you can weight the order of their size. To avoid reducing all the time the array when erasing an element, if you can put an "empty" placeholder, do it. Then, you will reduce the array later, namely you will remove all "empty" placeholders in a garbage collecting phase. For binary strings where there is no terminating element, track its size with a variable. If you need to serialize on disk tree like structures, use indexes to build your tree which would be arrays of integers with a terminating value and an empty placeholder value.

Do not forget that the kernel is a wonderful VM (Virtual Machine). It cleans everything at process destruction. Then short lived processes do not need ultra clean memory management: for instance CGI (Common Gateway Interface) programs. Do not free global data at process exit, except if you want a clean valgrind output.


CREATED 20071115

concurrent writes

I went to the 2600 meeting and heard that The Borg OS is not able to deal with concurrent writes on one file. Astonishing. But what's even more astonishing is that there are still people talking about it. The only interesting subject about this proprietary toxic OS is how quickly it can be phased out of existence.


CREATED 20071106

Fonera

One of the most famous people from my Parisian LUG (Linux User Group) got himself a Fonera box during the latest Ubuntu install party at La cité des sciences et de l'industrie, and made some noise (irony inside) on the LUG mailing list about configuration issues. Basically, with this router box, you share your Internet bandwidth with all other fonera owners. There are two physically separated wifi networks. One for your personal use, one for the sharing(God! I'm sooooo nice) network. Beyond the fact that non fonera owners have to pay to access the world-wide Fonera wireless network, you stay perfectly authenticated on those network: War Against Terrorism is Watching YouTM... For those who really want to share properly their bandwidth, and avoid configuration issues: buy a wifi router with quality of service support (namely any good linux router), and leave it open without encryption and avoid web site with flash abuse.