fifo.sh

Recently I decided to bring some life to the Grace homepage. I always expected it to spawn some controversy so I’ve not been surprised by seeing a vocal group of people dismissing its ideas out of hand. One of the most colorful reviews is one that could only make me chuckle: “it’s like Java and PHP gang-raped a Makefile”. I’m not likely to make too much out of reactions like that, these are questions about taste where you just can’t please everyone (and shouldn’t try).

Another reason why I don’t worry about art critics in this context is that, even if Grace were a library with only a single user, it would still help me get my stuff done in a way that I enjoy. It has already proven its value and given me an excellent return on investment using it in a lot of roles. It scratched my itch and that is fine.

I do find some of the negativism on sites like reddit interesting as a phenomenon. I’ve thought of this as a factor of the functional programming cartel that seems to be hanging out at such places. And I realized that this way of looking at it is just as dismissive and childish, so it made me wonder how it can be that there seems to be this great wall between that crowd and the typical ISP/Unix nerd demographic I normally interact with.

When looking at software, I reckon there are two approaches. Some people, when they think about code, see a world of math. For me, what I mostly see is flying bits, an interconnected lego-world of action-reaction patterns that makes intuitive sense. I think both approaches are valid, but obviously they lead to a completely different view on software development.

Naturally I may be suffering from confirmation bias on this subject, but I get the impression that those of us that are more into this direct approach to programming are the ones producing most of the real, living software out there. I can definitely see areas where a more distanced and abstract approach like functional programming can make a difference, but a lot of software development is really about moving bits from A to B, more about rolling up your sleeves and building it than mulling about algorithms and monads; lego, not math.

Bookmark It

I’m a strong believer in systems that are easily discoverable, where the structure of the files that make up the system communicate something about the application’s internal structure to the outside world. I tried to do much of the same thing with OpenPanel, but at the end of the day there’s still a lot to go around, so perhaps now’s a good time to write up a bit about the thinking and architecture behind OpenPanel and its components.

When we started the design, we wanted to focus on the following demands:

1. OpenPanel should be modular, and modules should not be restricted to any specific language
2. There should be elementary protection against security defects in module code
3. There should be a possibility of multiple user interfaces (at the very least both a GUI and a command line tool)
4. The system should anticipate the possibility of remote control and clustered integration
5. Configuration provided by OpenPanel should follow the standard configuration practices for the programs involved — no funky business relying on access to a MySQL database
6. It should be possible to dedicate an installation to a single purpose (like mail) without requiring unrelated services (Apache, MySQL) through dependencies

What we came up with, is a design that could best be explained as a domain-specific object database that reflects changes into the configuration of outside programs. The opencore daemon implements this abstraction. At the bottom it has an sqlite database that stores object and class data. Modules are implemented as directories inside /var/opencore/modules that define object classes for the database and the program that executes the necessary changes to outside configuration related to objects of those classes. A JSON-based RPC interface accepts requests through HTTP. This backend port is also used to serve static files related to the web-based GUI.

Obviously, accepting remote RPC requests is something you don’t want to combine with root privileges. Also, since we wanted to keep a low number of privileges exposed directly to module code, running the back-end programs/scripts for module actions was definitely not something we felt should be done as the root user. This is where the authd daemon comes in. It accept requests on a unix domain socket that is only accessible to the opencore daemon. Whenever opencore runs a module program, it will fork to a new process, open a socket to authd, tell authd which module will be executed and drop the privileges needed to open any new connections to authd. The unprivileged module can talk to authd on fd 3, but will be restricted to those privileged actions defined in its module.xml file.

Documentation for the Module API and the RPC Interface is on our site, but some of it may still be a bit rough around the edges.

Bookmark It

People requested a screencast of our command line interface in action as well, so here it is:

We’ll be adding a dedicated screenshots section to our website soon.

Bookmark It

The premise sounds easy enough: There’s no arguing with free. In the server control panel market we found a niche dominated by commercial players with a stranglehold on price and a bunch of products that annoy system administrators to the bone. So we set out to create something appealing — software with disruptive potential. With a product in our hands that ticks all the checkboxes in terms of nerd appeal, you would say that this would be as easy a sell as free blowjobs at a LAN party. Unfortunately, things are not quite that simple.

We went through the hoops submitting the news of our release to a dozen of the usual suspects, got a couple of hits and the first testers have arrived. There’s even a couple of people around doing more involved things like trying to port packages to other distros. It’s not bad and we appreciate all the people helping us out by playing with the beta, but a stampede it is not. Building a community takes time.

In terms of strategy, this may be the real reason for the ‘release early, release often’ mantra. Especially when your project is in a niche area, expect community building to take up maybe just as much time as actual development. We wanted to avoid getting distracted by actual users too early in the game, so our gut feeling at the time was ‘early, schmearly’. We had a couple of other reasons, some of them even sounded real good — like how there are too many open source projects in the field that need so much polishing that the market has been mostly ignoring them. But the fact of the matter is, all the hours of community building we’ve been avoiding up to the beta release will have to be put in double now. We’ll need to go from our current mode of reclusive introversion to full-blown extravert. Tough job for a bunch of nerds.

It all started as an enormous itch that needed serious scratching. We’ve been dealing, in our various capacities at hosting providers, with many incarnations of commercial server control panels like Ensim, Plesk and cPanel. Not only were these, in our view, haphazardly constructed kludges of magic that actively worked against the operating systems they were installed on; we kept coming back to the fact that people were actually paying for such software. Through the nose, no less. This felt wrong and paved the way for the OpenPanel project.

We spent over a year creating something with the capacity to change the landscape, by combining the proper amount of openness and flexibility with a state of the art user interface and a license that should make trumpets sound from the heavens. People we’ve shown it to have been raving. But, now that we’re at the point to hit the submit button for all this on sites like freshmeat, all those little doubts start setting in. It’s scary as hell.

I’m convinced we’ve taken the right step by not fully adopting the ‘release early’ mantra — it has given us the chance to quickly reshape the design where needed (Heck, we’re on the second incarnation of the GUI). But this is the downside. If you release early, expectations are lower and you never get this neckhair-raising fear of throwing your work into the world. Alas.

We’ve opened up the 0.9 (beta) branch of OpenPanel as of today. There’s no longer a requirement to email us to get access, just follow the download instructions from the main site. Messages have also gone out to the usual places. The future is now.

Here’s a preview of the upcoming Beta release of our life’s work. Now that we’re over that “oh no, we’re all going to die and we’ll never finish it” point, things are starting to look well. Or perhaps the other way around:

We’ll pick up the last bugs we found monday and then seed a release candidate to our alpha team. If no show stoppers emerge, that will be followed up shortly by a full public release of the OpenPanel beta.

Bookmark It

A major source of anger and frustration in C++ style OO is multiple inheritance. It’s a source of anger and frustration and one most people recognize as a path best avoided. The Objective-C idea of a protocol is a real life-saver in many occasions where you would need to deal with multiple inheritance otherwise. The concept of a protocol is to have a second way of classifying objects, one that completely sidesteps the hierarchical class model and instead just classifies objects by their common functions.

One area where the concept of protocols is quite convenient is iteration. There is usually a wide area of possible classes that could, theoretically, enumerate a list of contained items. Although C++ lacks protocols proper, template iterators are a fine way to access any class that implements a de facto iterator protocol. The only thing missing is an explicit declaration of the implemented protocol.

The Grace iterator<collectionclass,nodeclass> template is a minimized version of the visitor<> template. It requires one function (visitchild) to be defined inside your class that returns a pointer to a child node. Here is what it would look like for a purely synthetic class:


class syntheticlist
{
public:
   syntheticlist (void) {}
   ~syntheticlist (void) {}
   string *visitchild (int atpos)
   {
       if ((atpos1)) return NULL;
       if (atpos==0) str = “Hello”;
       else str = “world”;
       return &str;
   }
protected:
   string str;
};

int myApp::main (void)
{
   syntheticlist L;
   foreach (n,L) fout.writeln (n);
   return 0;
}

The foreach macro goes through the following steps:

1. It creates an iterator<syntheticlist, string *> pointing to L
2. It sets up a for loop that starts at visitchild(0) and stops when it returns NULL
3. It creates a temporary reference variable n linked to the current node.

The Grace iterator protocol looks a bit ascetic with only visitchild and the lack of the traditional first() and next() pattern. Part of this is due to the visitor protocol (which iterator is a part of) being about more than just of iteration. If your goal, however, is to make your class iterable with foreach, you can safely assume an argument value 0 to mean ‘first’ and any value bigger than 0 to mean ‘next’ if that makes more sense in your context. Most collection classes used within Grace use growable arrays, which means they don’t have to keep any state information in order to be iterable.

Next to string and array handling, memory management has always been one of those reasons users of other languages pointed at C++ programmers and laughed. The problem stems from dealing with temporary objects and keeping track of their lifetime. Although remembering to delete objects after you’re done with them sounds easy on paper, in the real world it’s tedious and you’re always just one forgetful early return statement away from a memory leak.

A lot of modern languages offer a friendlier form of memory management for this reason. They keep track of all references to a particular object and regularly take care of objects that are no longer referenced. This garbage collection process has a number of drawbacks that are usually seen as justified, considering the programming problems it solves. And it does solve them, but not all problems it solves need to be problems and, once you take those non-problems off the list, the smell of over-engineering becomes apparent.

What I see as the most interesting memory management problem that garbage collection solves can be illustrated by the following snippet:

Storpel *createStorpel (int modelNumber)
{
   Storpel *res = new Storpel;
   res->setModel (modelNumber);
   return res;
}

int main (void)
{
   Storpel *myStorpel = createStorpel (42);
   myStorpel->grind (11);
   delete myStorpel;
   return 0;
}

The problem with this pattern is obvious: The receiving function, by calling createStorpel() has taken responsibility for managing the Storpel object. This can get hairy in almost zero time.

In Grace, I prevented this anti-pattern from emerging by sticking to a number of principles:

• Use of pointers and new for objects is minimalized (pointers are awkward when you want to use overloaded operators anyway, so there’s exterior motivation).
• Data container classes are designed to be mere vessels for a raw data block that can be taken away from one object and ‘given’ to another without copying.
• When confronted with a pointer during assignment and initialization, data container classes will ‘take over’ the other object’s data and immediately delete it.
• Functions that want to return objects actually return pointers to temporary objects. The returnclass (type) name retain macro creates such a temporary object plus a temporary reference to make it easier to access the object.

Words are words, code is code. This is what it effectively looks like:

string *getGreeting (void)
{
   returnclass (string) res retain;
   res = "Hello, world.";
   return &res;
}

int main (void)
{
   string s = getGreeting();
   fout.writeln (s);
   return 0;
}

Behind the curtains, the returnclass macro creates a temporary string object using a custom allocator. This string object creates a persistent refblock structure to hold the data. Then, inside main(), we assign it to s:

The receiving object takes a reference to the underlying refblock:

And then the original object is immediately destroyed:

When main() goes out of scope, the ’s’ string object will be automatically cleaned up and, with it, the refblock. No need for an explicit delete.

As for the other problems that garbage collection solves, most of them have to do with interactions between collections of more complex objects. Mostly, if you find yourself in need of tracking such complex relationships, you may need to look at a way of simplifying your design. If that is not possible, then keeping track of memory references takes no more diligence than what the structure already demands.