Felix Programming Language

I've just committed a LLVM code generator for flxc.

#!build/debug/bin/flxc -I build/debug/lib --import nugram.flxh --import flx.flxh
>>> type int = "int";
... BOUND SYM:     type int<3> = "int";
>>> fun add : int*int -> int = "%add";
... BOUND SYM:     fun add<4>: int<3>^2 -> int<3> = "%add";
>>> fun foo (a:int, b:int, c:int) = { val d = a + b; return d + 1; }
... BOUND SYM:     fun foo<5>(val a<8>: int<3>,val b<9>: int<3>,val c<10>: int<3>): int<3>{
...   d<7> := (add<4> (a<8>, b<9>));
...   return (add<4> (d<7>, 1));}
>>> val x = 1;
... BOUND SYM:     val x<11>: int<3>;
... BOUND EXE:     x<11> := 1;
>>> val y = 2;
... BOUND SYM:     val y<12>: int<3>;
... BOUND EXE:     y<12> := 2;
>>> val z = foo (x, y, 3);
... BOUND SYM:     val z<13>: int<3>;
... BOUND EXE:     z<13> := (foo<5> (x<11>, y<12>, 3));
>>> ^D

And here's what it generates:

define void @__init__() {
entry:
  %z = alloca i32                                 ; <i32*> [#uses=1]
  %y = alloca i32                                 ; <i32*> [#uses=1]
  %x = alloca i32                                 ; <i32*> [#uses=1]
  store i32 1, i32* %x
  store i32 2, i32* %y
  %foo = call i32 @foo(i32 1, i32 2, i32 3)       ; <i32> [#uses=1]
  store i32 %foo, i32* %z
  ret void
}

define i32 @foo(i32 %a, i32 %b, i32 %c) {
entry:
  %d = alloca i32                                 ; <i32*> [#uses=1]
  %add = add i32 %a, %b                           ; <i32> [#uses=2]
  store i32 %add, i32* %d
  %add1 = add i32 %add, 1                         ; <i32> [#uses=1]
  ret i32 %add1
}

Of course it doesn't actually execute the commands yet, but the output's so pretty!

bohan on reddit pointed out that my text.m4_substitute doesn't actually have anything to do with m4. So I replaced it with two autoconf inspired functions: text.autoconfig_config_file and text.autoconfig_config_header. Consider this source:

const char* foo = @foo@;
#undef HAVE_FOO
#undef HAVE_BAR
#undef HAVE_BAZ

If you use this function:

text.autoconf_config_file(ctx, 'foo.py', 'foo.h.in', {
    'foo': '"FOO"',
})

which replicates the functionality of AC_CONFIG_FILES, you'll get:

const char* foo = "FOO";
#undef HAVE_FOO
#undef HAVE_BAR
#undef HAVE_BAZ

And if you process it with this:

text.autoconf_config_header(ctx, 'baz.h', 'baz.h.in', {
    'foo': '"FOO"',
    'HAVE_FOO': 1,
    'HAVE_BAR': 0,
    'HAVE_BAZ': '"BAZ"',
})

which replicates AC_CONFIG_HEADERS, you'll get:

const char* foo = "FOO";
#define HAVE_FOO 1
/* #undef HAVE_BAR */
#define HAVE_BAZ "BAZ"

Well that wasn't that terrible. I've removed all the global state from fbuild. Now a context object is passed around that contains this information. Now you must write your build system like:

import fbuild.db
import fbuild.builders.c

@fbuild.db.caches
def foo(ctx, value):
    ctx.logger.log('got: %s' % value)
    return value

def build(ctx);
    foo(ctx, 1)

    static = fbuild.builders.c.guess_static(ctx)
    static.build_exe('foo', ['a.c', 'b.c'])

As you can see, you now must pass the context around as the first argument to any cached function or object.

I've been participating in a lengthy build system discussion on reddit, which has inspired me to start sketching out my ideas for FBuild:

New scheduler: I'm not too happy with the current scheduler, as I had to introduce a timeout on checking if there are done tasks to work around a flaw with python's threadsafe queue structure. It's possible for a KeyboardInterrupt to happen in between incrementing/decrementing the semaphore, and this was leading to a deadlock that needs a "kill -9" to exit. There has to be a better way to do this. I'm experimenting with multiprocessing to see if that'll be faster. In order to do that though we're going to have to implement our own pickling/unpickling of MethodType, and possibly other things too.

A new database: FBuild's a bit faster with Python 3.1's new io module, but it may be even faster if we could use a SQLite database.

Get rid of the global state: For ease of implementation, FBuild has a couple global state objects: the scheduler, the database, the logger, and the commandline options. We could fold all of these into some context object and pass it around. This would mean you'd have to write code like this though:

def build(ctx):
    static = fbuild.builders.guess_static(ctx)

Simplify the builders more: webon on reddit pointed out that the ar builder is a bit more complicated than it needs to be. Part of the reason for this is that we're writing the option list twice. First at the builder level, second at the file level. Perhaps we could simplify this by doing something like:

class Ar(fbuild.builders.Builder):
    options = {
        flags: list,
        libs: list,
        libpaths: list,
        ...
    }

    def __init__(self, exe='ar', flags=['-rc'], **kwargs):
        ...
        super().__init__(flags=flags, **kwargs)

    def __call__(self, dst, srcs, **kwargs):
        options = self.process_options(**kwargs)
        cmd = [self.exe]
        cmd.extend('-l' + lib for lib in options.libs)
        ...

Where the superclass would somehow process the option list and automatically merge the options without us needing to copy the code everywhere all the time.

Speed up FBuild: bohan on the fbuild mailing list pointed me to a benchmark he wrote to test out his build system, wonderbuild. FBuild is beating SCons and Autotools in many tests, but I think we could to a bunch better. For instance, is there any way we could get the number of system calls down?

Finish support for avr-gcc and haskell

Optionally turn of md5ing of files: It'd be pretty easy to add a flag to fbuild.path.Path to turn off using md5 for file digesting. This should help folks like bohan who are willing to sacrifice doing rebuilds of files in order to have a faster build system.

edit: I forgot to include automatic threading. We could use futures transparently thread computation without needing to explicitly use scheduler.map. Unfortunately this can get a little complicated unless we could come up with a good way to use the futures without littering scheduler.future(f, *args, **kwargs) and scheduler.force everywhere.

edit 2: Some more projects:

targets: The idea is to have simple make-like targets, possibly something like:

@fbuild.target
def configure():
    ...

def build():
    conf = configure()
    ...

@fbuild.target
def test():
    builder = build()
    ...

installation: No ideas for that yet!