Now, some people enjoy Lua for scripting, and others have used Python. I don't particularly like Lua's semantics, and I think Python is a bit clumsy to embed. But GNU Guile turned out to be perfect for my use. For those not in the know, GNU Guile is a Scheme implementation, and has been around for ages. It supports:
- Almost everything from the Scheme R5RS standard
- Almost all the SRFI recommendations
- Multithreading and multiple environments/modules
- Unicode
- Extremely simple embedding
The garbage collector in Guile is based on libgc which can be a bit confusing at first because of the stack magic, but code-wise it's extremely easy to use. Unlike Python's reference juggling, you hardly have to worry about such things when extending or embedding Guile. Let's look at a simple example which creates a function in C which can be called from Scheme, in addition to a REPL we can use to test code:
#include <libguile.h>
SCM our_scheme_func(void)
{
printf("inside our scheme function!\n");
return SCM_UNSPECIFIED;
}
void boostrap_main(int argc, char** argv)
{
/* register our function with scheme */
scm_c_define_gsubr("our-scheme-func", 0, 0, 0, our_scheme_func);
scm_shell (argc, argv);
}
int main(int argc, char** argv)
{
scm_boot_guile (argc, argv, boostrap_main, 0);
return 0;
}
Lets go through this in detail. scm_boot_guile() is one of many ways to initialize a new scheme environment, as well as initializing the garbage collector on the current thread. scm_boot_guile() never returns here because of the call to scm_shell which starts a new Guile interactive shell, also known as a read-eval-print-loop, or REPL for short. scm_c_define_gsubr() is the way to create new Scheme functions, which is probably the most useful aspect of this code. The second to fourth parameters are the number of required and optional arguments, as well as "rest" arguments. Rest arguments for scheme are kind of like varargs in C. The last parameter is simply a pointer to the function. So our function returns an unspecified value, and takes no parameters at all (the parameter counts are all 0). A function taking no parameters is known as a 'thunk' in Scheme. You call the function from Scheme like this:
(our-scheme-func)
If our function had more parameters, it would look like this:
(our-scheme-func 'foo '(1 2 3) 4.5)
..which would be a function taking three parameters, a symbol, a list and a decimal number. Scheme supports several datatypes, but the actual code (known as S-expressions) are always lists. So what is this SCM type that is the return type of our function? It's the generic C representation of all Scheme types. SCM::type is a value you can test against enums to get the underlying type, but libguile fortunately has some handy macros for testing each type. You can make your own Scheme types, but I will not explain that in this post.
But running a Scheme interpreter on a single thread isn't very useful, especially if it blocks. Perhaps we even want to run multiple Scheme environments simultaneously. Wouldn't it be great to have a game where each AI or NPC is controlled by a Scheme script alone? That was my main intention behind using Guile, so I made it work with multiple threads. I did run into some obstacles which I'll explain first. Big thanks to Mark Weaver on #guile @ FreeNode for his patience! The first problem I had was that the documentation clearly states that libguile is compatible with posix threads. At least for guile version 2.0.5 which is available in Ubuntu 12.04 LTS, this is not the case. Creating new Scheme environments inside a thread made with pthread_create() crashes as soon as the garbage collector is initialized. The workaround is to first create a scheme environment in the main thread, and then create new threads with scm_spawn_thread(). The second bug I encountered while testing was that scm_c_eval_string() did lazy evaluation, which caused its Scheme symbol to not show up in the thread Scheme environments. Calling it once in the main thread did the trick. But if you want to load Scheme files, it is better to just call scm_c_primitive_load() instead. Anyways, enough beating around the bush; let's make some code:
#include <libguile.h>
typedef struct
{
int val;
const char* filename;
} threaddata_t;
threaddata_t threadData = {99, "file.ss"};
SCM test_func(void)
{
printf("(test-func) : File %s, Value %d\n", threadData.filename, threadData.val);
return SCM_UNSPECIFIED;
}
static SCM scheme_environment(void* userdata)
{
scm_c_primitive_load(threadData.filename);
return SCM_UNSPECIFIED;
}
void* bootstrap_main(void* v)
{
SCM th;
(void)v;
scm_c_define_gsubr("test-func", 0, 0, 0, test_func);
th = scm_spawn_thread(scheme_environment, NULL, NULL, NULL);
scm_join_thread(th);
return NULL;
}
int main(int argc, char* argv[])
{
(void)scm_with_guile(bootstrap_main, NULL);
return 0;
}
First we initialize a Scheme environment in our main thread with scm_with_guile(). The extra indirection is kind of weird, but it is the most portable way to initialize Guile. The actual function executed on the main thread calls scm_c_define_gsubr() which we already know registers a C function with Scheme. Then scm_spawn_thread() is called, which works just like pthread_create(), only it also takes optional parameters to an exception handler. Finally the main thread waits for the thread to finish by polling on scm_join_thread().
The actual work done by the thread is to call scm_c_primitive_load(), which loads a Scheme source file. If the toplevel contains an actual function call, it is also executed. Any C function we register with scm_c_define_gsubr() is available for Scheme to use, and it does not belong to any specific module (nothing to import or require). If we created more threads, all such extended functionality would be visible in all the threads.
But we have a last issue to resolve. scm_c_define_gsubr() doesn't take a void* userdata parameter, so we are unable to pass on data from C into our Scheme functions. The only parameters we can specify are of type SCM, which comes from the Scheme environment itself. There are three ways around this problem:
- We could call scm_current_thread() and use it as a lookup
- We could encapsulate all our state in one or several Scheme types and make them visible at toplevel
- Use environment-specific state, called 'Fluids'
Fluids are state objects whose values are dependent on the current module/environment. It basically works as thread local storage. First you create a single fluid object with scm_make_fluid(). Then each thread can set it individually with scm_fluid_set_x() and fetch the value with scm_fluid_ref(). Putting everything together, we get:
#include <libguile.h>
#define NUM_THREADS 4
/* Our local data thread storage */
SCM fluid_thread_id;
typedef struct
{
int val;
const char* filename;
} threaddata_t;
/* Multiple threads this time */
threaddata_t threadData[NUM_THREADS] = {
{99, "file1.ss"},
{100, "file2.ss"},
{101, "file3.ss"},
{102, "file4.ss"}
};
/* Get the SCM value from the fluid, then convert the SCM to an actual C int.
The value we stored was the iteration value from creating the threads (0 to 3)*/
static int get_thread_id(void)
{
int r = scm_to_int(scm_fluid_ref(fluid_thread_id));
return r;
}
SCM test_func(void)
{
int num;
/* By getting the iteration value we used when creating the threads,
we can look up per-thread data. Our function is reentrant. */
num = get_thread_id();
printf("(test-func) : File %s, Thread %d, value %d\n", threadData[num].filename, num, threadData[num].val);
return SCM_UNSPECIFIED;
}
static SCM scheme_environment(void* userdata)
{
SCM sint;
/* void* to int to SCM, see main for the passed value */
sint = scm_from_int32(*((int*)userdata));
/* Set the local thread storage to the thread number 'sint'.
Thread 0 with be 0, thread 1 will be 1, and so on */
scm_fluid_set_x(fluid_thread_id, sint);
/* Load a Scheme file for this Guile instance */
scm_c_primitive_load(threadData[get_thread_id()].filename);
return SCM_UNSPECIFIED;
}
void* bootstrap_main(void* v)
{
int i;
SCM th[NUM_THREADS];
(void)v;
fluid_thread_id = scm_make_fluid();
/* The Scheme function test-func is available in all the threads. */
scm_c_define_gsubr("test-func", 0, 0, 0, test_func);
for(i=0; i<NUM_THREADS; ++i)
th[i] = scm_spawn_thread(scheme_environment, &i, NULL, NULL);
for(i=0; i<NUM_THREADS; ++i)
scm_join_thread(th[i]);
return NULL;
}
int main(int argc, char* argv[])
{
(void)scm_with_guile(bootstrap_main, NULL);
return 0;
}
That's all there is to it! Now you can write four Scheme files with file extension .ss and they will all run concurrently. Any state changes to the application should of course be synchronized properly with a mutex or similar.
No comments:
Post a Comment