Fragile binary interface problem

The fragile binary interface problem or FBI is a shortcoming of certain object-oriented programming language compilers, in which internal changes to an underlying class library can cause descendant libraries or programs to cease working. It is an example of software brittleness.

This problem is more often called the fragile base class problem or FBC; however, that term has a wider sense.

Cause

The problem occurs due to a "shortcut" used with compilers for many common object-oriented (OO) languages, a design feature that was kept when OO languages were evolving from earlier non-OO structured programming languages such as C and Pascal.

In these languages there were no objects in the modern sense, but there was a similar construct known as a record (or "struct" in C) that held a variety of related information in one piece of memory. The parts within a particular record were accessed by keeping track of the starting location of the record, and knowing the offset from that starting point to the part in question. For instance a "person" record might have a first name, last name and middle initial, to access the initial the programmer writes thisPerson.middleInitial which the compiler turns into something like a = location(thisPerson) + offset(middleInitial). Modern CPUs typically include instructions for this common sort of access.

When object-oriented language compilers were first being developed, much of the existing compiler technology was used, and objects were built on top of the record concept. In these languages the objects were referred to by their starting point, and their public data, known as "fields", were accessed through the known offset. In effect the only change was to add another field to the record, which is set to point to an immutable virtual method table for each class, such that the record describes both its data and methods (functions). When compiled, the offsets are used to access both the data and the code (via the virtual method table).

Symptoms

This leads to a problem in larger programs when they are constructed from libraries. If the author of the library changes the size or layout of the public fields within the object, the offsets are now invalid and the program will no longer work. This is the FBI problem.

Although changes in implementation may be expected to cause problems, the insidious thing about FBI is that nothing really changed, only the layout of the object that is hidden in a compiled library. One might expect that if one changes doSomething to doSomethingElse that it might cause a problem, but in this case one can cause problems without changing doSomething, it can be caused as easily as moving lines of source code around for clarity. Worse, the programmer has little or no control over the resulting layout generated by the compiler, making this problem almost completely hidden from view.

In complex object-oriented programs or libraries the highest-level classes may be inheriting from tens of classes. Each of those base classes could be inherited by hundreds of other classes as well. These base classes are fragile because a small change to one of them could cause problems for any class that inherits from it, either directly or from another class that does. This can cause the library to collapse like a house of cards as many classes are damaged by one change to a base class. The problem may not be noticed when the modifications are being written if the inheritance tree is complex. Indeed, the developer modifying the base class is generally unaware of which classes, developed by others, are using it.

Solutions

Languages

One solution to the fragile binary interface problem is to write a language that knows the problem exists, and does not let it happen in the first place. Most custom-written OO languages, as opposed to those evolved from earlier languages, construct all of their offset tables at load time. Changes to the layout of the library will be "noticed" at that point. Other OO languages, like Self, construct everything at runtime by copying and modifying the objects found in the libraries, and therefore do not really have a base class that can be fragile. Some languages, like Java, have extensive documentation on what changes are safe to make without causing FBI problems.

Another solution is to write out an intermediate file listing the offsets and other information from the compile stage, known as meta-data. The linker then uses this information to correct itself when the library is loaded into an application. Platforms such as .NET do this.

However, the market has selected programming languages such as C++ that are indeed "position dependent" and therefore exhibit FBI. In these cases there are still a number of solutions to the problem. One puts the burden on the library author by having them insert a number of "placeholder" objects in case they need to add additional functionality in the future (this can be seen in the structs used in the DirectX library). This solution works well until you run out of these dummies -- and you do not want to add too many because it takes up memory.

Objective-C 2.0 provides non-fragile instance variables by having an extra level of indirection for instance variable access.

Another partial solution is to use the Bridge pattern, sometimes known as "Pimpl" ("Pointer to implementation"). The Qt framework is an example of such an implementation. Each class defines only one data member, which is a pointer to the structure holding the implementation data. The size of the pointer itself is unlikely to change (for a given platform), so changing the implementation data does not affect the size of the public structure. However, this does not avoid other breaking changes such as introducing virtual methods to a class that has none, or changing the inheritance graph.

Linkers

Another solution requires a smarter linker. In the original version of Objective-C, the library format allowed for multiple versions of one library and included some functionality for selecting the proper library when called. However this was not always needed because the offsets were only needed for fields, since methods offsets were collected at runtime and could not cause FBI. Since methods tend to change more often than fields, ObjC had few FBI problems in the first place, and those it did could be corrected with the versioning system. Objective-C 2.0 added a "modern runtime" which solved the FBI problem for fields as well. Additionally, the TOM language uses runtime collected offsets for everything, making FBI impossible.

Using static instead of dynamic libraries where possible is another solution, as the library then cannot be modified without also recompiling the application and updating the offsets it uses. However static libraries have serious problems of their own, such as a larger binary and the inability to use newer versions of the library "automatically" as they are introduced.

Architecture

In these languages the problem is lessened by enforcing single inheritance (as this reduces the complexity of the inheritance tree), and by the use of interfaces instead of base classes with virtual functions, as interfaces themselves do not contain code, only a guarantee that each method signature the interface declares will be supported by every object that implements the interface.

Distribution method

The whole problem collapses if the source code of the libraries is available. Then a simple recompilation will do the trick.

External links

BeOS's paper on the problem and their solution