Introduction

In this short article I share the results of using LuaTeX’s token library as one way to intermix TeX and MetaPost code. I used LuaTeX 1.0.1 which I compiled from the source code in the experimental branch of the LuaTeX SVN repository but, I believe, it should also work with LuaTeX 1.0 (may also work with some earlier versions too). In addition, I used luamplib version 2016/03/31 v2.11.3 (I downloaded my copy from github). Note that I do not have a “standard” TeX installation—I prefer to build and maintain my own custom setup (very small and compact).

The LuaTeX token library

I will not try to explain the underlying technical details but simply point you to read this article by Hans Hagen and the relevant section in the LuaTeX Reference Manual (section 9.6: The token library). Here, I’ll just provide an example of using the token library—it is not a sophisticated or “clever” example but one simply designed to demonstrate the idea.

Objective

The goal is to have TeX macros that contain a mixture of TeX and MetaPost code and find a way to expand those macros into a string of MetaPost code which can be passed to LuaTeX’s in-built MetaPost interpreter: mplib.  Suppose, just by way of a simple example, that we defined the following simple TeX macros:

\def\mpbf#1{beginfig(#1);\space}
\def\fc{fullcircle\space}
\def\pp#1#2{pickup pencircle xscaled #1mm yscaled #2mm;\space}
\def\mpef{endfig; end}
\def\draw#1{draw \fc scaled #1;\space}

and we’d like to use them to write TeX code that can build MetaPost graphics. Note that the definition of \draw also contains the command \fc

The scanner

The following code uses LuaTeX’s token library function scan_string() to generate a string from a series of incoming TeX macros—by expanding them. It stores the resulting string into a toks register so that we can later use and obtain the result.

\newtoks\mpcode
\def\scanit{%
\directlua{
local p = token.scan_string()
tex.toks["mpcode"]=p
}}

We could use this macro like this:

 \scanit{...text and TeX macro in braces...}

but instead we’ll add just a little more functionality. Suppose we further define another TeX macro

\def\codetest{\mpbf{1} \pp{0.3}{0.75} \draw{12}\mpef }

which contains a sequence of commands that, once expanded, will generate MetaPost program to produce our graphic. To expand our TeX macro (\codetest) we can do the following:

\scanit{\codetest} and the output is \the\mpcode

Here, the braces “{” and “}” are needed (I think…) in order to make token.scan_string() work correctly (I maybe wrong about that so please run your own tests). Anyway, the \codetest macro is expanded and (due to \scanit) the resulting MetaPost code is stored in the toks register called mpcode. We can see what the toks register contains simply by typesetting the result: doing \the\mpcode—note you may get strange typesetting results, or an error, if your MetaPost contains characters with catcode oddities. You can use \directlua{print(tex.toks["mpcode"])} to dump the content of the mpcode toks register to the console (rather than typesetting it). What I see in my tests is that the mpcode toks register contains the following fully expanded MetaPost code:

beginfig(1); pickup pencircle xscaled 0.3mm yscaled 0.75mm; draw fullcircle scaled 12; endfig; end

And this is now ready for passing to MetaPost (via mplib). But how? Well, one option is to use the package luamplib. If you are using LaTeX (i.e., luaLaTeX format) then you can use the following environment provided by luamplib:

\begin{mplibcode}
...your MetaPost code goes here
\end{mplibcode}

However, our MetaPost code is currently contained in the mpcode toks register. So here’s my (rather ugly hack) that uses \expandafter to get the text out of mpcode and sandwiched between \begin{mplibcode} and \end{mplibcode}. I am sure that real TeX programmers can program something far more elegant!

\def\sa{\begin{mplibcode}}
\def\mcodex{\expandafter\sa\the\mpcode\relax\end{mplibcode}}

Consequently, we  just need to issue the command \mcodex to draw our MetaPost graphic. I hope this is an interesting suggestion/solution that others might find useful.