Wednesday 13 November 2013

How to add docstrings for the C procedures when embedding Guile with libguile

I'm currently embedding scheme Guile into a game/demo engine. I've written an article on this blog before on how to embed Guile and add new subroutines from C/C++.
While doing that, what baffled me was that the docstrings I added when using Guile's snarfing didn't show up in Geiser's C-c C-d C-d "document symbol under cusor" mode. I asked around on #guile on Freenode, and they told me it was a complex beast involving texinfo and other black magic. So I set out to investigate (I need documentation when coding in Scheme damnit!). But first, what the hell is 'snarfing' ?

Snarfing


Snarfing is an extra preprocessor step guile has, to make it easier to write Scheme subroutines from the C interface. Without snarfing, you have to first define your function, and then call scm_g_define_subr somewhere else. Naturally, it follows that these two steps can get out of synch. scm_g_define_subr defines the number of arguments the subroutine takes and whatnot. Snarfing uses the SCM_DEFINE macro defined in snarf.h. It's included by libguile.h, so that's still the only header you need:



SCM_DEFINE(Scheme_CreateTexture, "texture-create", 0, 0, 1,
		   (SCM rest),
		   "Create a texture without any contents.\n"
		   "Optional keywords/parameters:\n"
		   "#:target      ('texture-1d | 'texture-2d | 'texture-3d)\n" 
		   "#:num-channels 1|2|3|4\n"
		   "#:format      ('fixedpoint | 'float | 'depth)\n"
		   "#:width       width\n"
		   "#:height      height\n"
		   "#:depth       depth\n")
#define FUNC_NAME s_Scheme_CreateTexture
{

	SCM_VALIDATE_SYMBOL(SCM_ARGn, scm_target);
	SCM_VALIDATE_SYMBOL(SCM_ARGn, scm_format);
	SCM_VALIDATE_NUMBER(SCM_ARGn, scm_numchannels);
	SCM_VALIDATE_NUMBER(SCM_ARGn, scm_width);
	SCM_VALIDATE_NUMBER(SCM_ARGn, scm_height);
	SCM_VALIDATE_NUMBER(SCM_ARGn, scm_depth);

	/* more code */

}
#undef FUNC_NAME


The arguments are: the symbol of the C function itself, then the scheme symbol, the number of mandatory, optional and vararg arguments, then the parameter list, and finally the docstring. Basically the same as scm_g_define_subr. Then we define the macro FUNC_NAME. And where did s_Scheme_CreateTexture come from? SCM_DEFINE defines < functionname > for you as the scheme symbol string. We must define FUNC_NAME for docstring snarfing to work, as we will see later. In addition, defining FUNC_NAME makes the macros SCM_VALIDATE_xxx, SCM_ASSERT and SCM_ASSSERT_TYPE work like they should, so you can bail out on type mismatches and stuff. Very handy.
So this showed how to define the actual function. Where did the call to scm_g_define_subr go? It's not gone. It's rather generated for you:



void initSchemeTextureBindings()
{
	createSchemeTextureType();
	#ifndef SCM_MAGIC_SNARFER
	#include "scheme_bind_texture.x"
	#endif
}


The ".x" file is generated by a tool named guile-snarf. It should be installed when you install guile. So guile-snarf runs som preprocessor magic on your .c or .cpp file and spits out an .x file which contains all your define_g_subr calls and other info, based on the information you added in the SCM_DEFINE macro. Now let's see how we invoke guile-snarf: guile-snarf -o scheme_bind_texture.x scheme_bind_texture.cpp -I/extra/include/paths Guile-snarf runs gcc -E, so it needs to know about your include paths in order to not bail with an error. It at least needs to know about where it can find libguile.h, which has all the SCM_DEFINE macro magic. If you want it to ignore all the other headers, wrap them inside a SCM_MAGIC_SNARFER #ifndef. SCM_MAGIC_SNARFER is defined while guile-snarf runs. In the example above, scheme_bind_texture.x is not included while snarfing is done, because it doesn't exist yet. That's all what normal snarfing does. Now when main calls initSchemeTextureBindings(), it takes care of the setup for all the guile functions inside scheme_bind_texture.cpp.


Docstring-snarfing


Nothing new has to be done with the source. If you use SCM_DEFINE as above and #define and #undef FUNC_NAME above, the next steps should work to get you the docstrings. You will need the guile source. I base this article on stock guile 2.0.9. Let's assume the source code is in a directory called guile-2.0.9.

First, build guile and libguile as usual. Install it if you like, but you don't have to. Next, get the scripts and programs guile_filter_doc_snarfage, guile-func-name-check guile-snarf-docs from the directory 
guile-2.0.9/libguile. You also need the script guild from guile-2.0.9/meta. Then you follow these steps to first output a .doc file, which in turn is converted to a .texi texinfo file, and finally a texinfo file in the text format:

./guile-snarf-docs -o foo.doc foo.cpp -- -I/extra/include/paths
cat foo.doc | guild snarf-check-and-output-texi > foo.texi
makeinfo --force --plaintext -o foo-procedures.txt foo.texi


Guile-snarf-docs generates dot .doc files similar to how guile-snarf works; by running the preprocessor with gcc -E. The "--" separates snarf-doc parameters from the parameters sent to the preprocessor. Just like guile-snarf you need to pass on the extra include paths, if any. 

The second step guild snarf-check-and-output-texi is an undocumented argument guild takes. It converts dot .doc files into the texinfo format .texi and spits it out to stdout. Note that if you got several .doc files from several .c or .cpp files, you can cat them together here.

The third and final step just runs makeinfo on the .texi file and turns it into a final .txt file we can give to guile.


How to use the doc .txt file with guile

Guile has a module named ice-9 documentation. It works like this:

(use-modules (ice-9 documentation))
(set! documentation-files (cons "/path/to/docs.txt" documentation-files))

I haven't tested to see whether the string can take a relative path, but I think it does.

That's it. Now the documentation should pop up with Geiser when you press C-c C-d C-d :-)

Subscribe to RSS feed