A minimal LuaTeX setup on Windows (Part 2)

In the first part of this post we very briefly looked at what happens when you grab the raw luatex.exe binary and run it: in short, not a lot!

Of environment variables and search paths

In today’s era of graphical user interfaces: Windows, Mac OS X, Gnome, KDE and so forth, the experience of software installation, especially on Windows and Mac, is typically a point-and-click affair. Not a command line in sight. For those who are taking their first steps into the TeX world, it can, depending on your setup, be a very new experience. During the course of these tutorials, which focus on a do-it-yourself LuaTeX install, the DOS prompt and command line will become your closest allies. When writing this sort of tutorial it is always difficult to balance the desire to be thorough without boring more experienced readers with detail they find too trivial. I’ll do my best to get it right but I’ll assume very little apart from feeling comfortable with a command-line world.

Telling LuaTeX about its environment

One of the first things you need to realise is that LuaTeX needs to know something about the “computer environment” in which it is running; you need to provide it with some fundamental information so that it is able to locate the resources it needs to do its job of typesetting. This help or guidance for LuaTeX is provided through a mechanism called environment variables. In essence, environment variables are used to store values which a program can access when it is running. Windows predefines a number of standard “system environment variables”, such as PATH, but you are free to add new ones according to the needs or requirements of the different programs you use. For LuaTeX, there are a number of very important environment variables you need to set before it can find anything, such as fonts, .tex files, graphics, encoding vectors etc, on your computer.

Web2C and Kpathsea: a primer
If you want to skip past my explanations and plunge straight into the detail then head on over to http://www.tug.org/texinfohtml/kpathsea.html. A great resource which provides interesting historical background by Karl Berry.

LuaTeX environment variables

Firstly, I should say that I’m using the term “LuaTeX environment variables” which may imply that any environment variables I mention are specific to LuaTeX: generally, they are not, so please forgive this slight simplification in the interest of clarity. Occasionally there may be an environment variable which is specfic to LuaTeX and where that is the case I’ll try to point it out. In the next installment I’ll zoom in on texmf.cnf, your passport to the joys of a minimal LuaTeX installation.

LuaTeX-related mailing lists and web sites

Just a quick post to list a few resources for LuaTeX.

Mailing lists
  1. dev-luatex: the mailing list for LuaTeX developers at http://www.ntg.nl/mailman/listinfo/dev-luatex. This list is frequented by the team who are building LuaTeX and is ideal if you want to stay at the “bleeding edge”. Great for notifications of code updates.
  2. lualatex-dev: the mailing list for LuaLaTeX development at http://tug.org/mailman/listinfo/lualatex-dev.
  3. luatex: general discussion of LuaTeX and probably most suited for user-type questions http://tug.org/mailman/listinfo/luatex.

These lists are a fantastic resource but remember that your questions are being answered by people who are willing to share their considerable expertise, and their time, for free. A little courtesy and respect goes a long way towards getting the help you need.

LuaTeX Wiki

Code snippets and more at http://wiki.luatex.org.

Other sites

Great site from Patrick Gundlach with lots of code fragments. In German only http://www.luatex.de/.

node.js: sometimes you see a game-changer but can’t say why!

Like many others I too was “wowed” by the vision offered by Google Wave and the potential for changing how we communicate via the web. Within the business of STM publishing Google Wave caused a huge stir, purely because it seemed to offer a “new paradigm” for collaboration, especially for global authorship and writing of scientific journal articles and the potential for redefining the “scholarly authorship workflow”. Yet, it was not to be. Like thousands of others, I eventually received my Wave invite and eagerly logged in to participate with the brave new “Wave world”. Along with like-minded colleagues, I created Waves and spent time playing with collaborative authorship but, sadly, the experience did not meet our hopes and visions for a new world of collaborative content-creation. The “Wave experience” failed, in some part, not because it was a bad idea but because user behaviour had not sufficiently evolved to define the norms for a Wave experience. In addition, there was simply no way to take your wave-created content and export it to mundane applications like Word or TeX for use in standard workflows. In my opinion, that was a catostrophic failure of vision. Yet, the demise of Google Wave most definitely should not be seen as an experiment which failed, for most certainly it did not. The huge “buzz of expectation” must, surely, have been fed from, or be predicated by, some innate recognition that we want “some way” to work together in ways that the current set of web technologies don’t yet quite deliver. Wave created a hope and vision for how we might work together without borders or time constraints, with the ability to create tools and plugins that allowed us, in theory, to create and interact with content-types previously held hostage within the sandboxed domain of our desktop computers. Yet, the process of creating robots or plugins to create a “distributed, shared, experience with everyday tools” was still far too deeply rooted in complex development frameworks. In hindsight, it would have been far better to talk to software companies to create “Wave ready” toolsets to bootstrap the user experience and foster engagement, especially for authors of complex scientific content. Among the first experiments in the “STM publishing world” were tools to support and share authorship of LaTeX code: none of which were really satisfactory.

node.js

I must predicate this with a disclaimer: I am intrigued, fascinated and in awe of node.js but readily admit that I’m not sure precisely why. To me, it seems to be in the same “solution domain” as Google Wave: comet servers. If you have not heard of node.js, take a few minutes to peruse the following content. For STM publishing, herein lies some really cool opportunities… just Google for “node.js” and ask yourself: non-blocking IO, server-side JavaScript, massive connectivity, now there’s an interesting paradigm for STM content applications?

Brilliant resources for learning Arabic

Just a short post to suggest a site which has some absolutely superb resources for learning Arabic. About a year ago I purchased Basic Arabic Grammar – Part A and have nothing but the greatest praise for the quality of the materials. Having worked my way through all 250 or so exercises, watched all the videos, I urge you to buy it and support this initiative. I have no relationship with arabic-studio.com other than being a totally satisfied customer. Absolutely, utterly outstanding. If you want to see my reviews of Arabic language resources on amazon.co.uk, then click here.

A minimal LuaTeX setup on Windows (Part 1)

The LuaTeX executable (luatex.exe) can be installed as part of mainstream TeX distributions such as TeX Live or, for Windows users, MiKTeX. However, with just a little bit of work you can create your own minimal LuaTeX setup under Windows, which is the route I chose to follow. TeX Live and MiKTeX are truly amazing pieces of work and provide extremely comprehensive TeX installations, but they are rather large. In addition, through the process of “rolling your own setup” you learn a lot of very useful things about the way that TeX looks for files and resources on your computer. I do have to admit that, initially, it was quite frustrating to “get the picture” but it soon made sense. I hope to share some of the lessons I learned, save you some time but also to provide the basic groundwork through which you can further explore the amazing LuaTeX engine.

To obtain the raw luatex.exe you can either compile the source code or download the latest beta via the LuaTeX web site. My personal preference is to compile LuaTeX from the latest source code but that requires you to install some additional software, namely MinGW and MSYS. I’m not going to cover MinGW and MSYS here because that deserves a separate post.

Getting the LuaTeX source code: a primer
Again, I’m restricting my discussions to Windows because that’s what I know. The LuaTeX source code is made publicy available from the GForge server at Supelec and can be obtained using an SVN client such as TortoiseSVN. The beauty of this process is that you can keep your local copy of the LuaTeX code fully synchronised with the master repository which is maintained by the LuaTeX developers. Every time the master codebase is modified you simply use TortoiseSVN to download the updates. Marvellous stuff!

The mysterious and magical texmf.cnf file

So, you downloaded luatex.exe, start a DOS prompt and type luatex to be presented with…

Um, OK. I’ll press enter to see what happens…

OK, I have a LaTeX file c:\test.tex

\documentclass[11pt,twoside]{article}
\begin{document}
\pagestyle{empty}
Hello Lua\TeX
\end{document}

I’ll run that, typing test.tex and I see… nothing, luatex.exe exits back to the DOS prompt. Clearly, something is not working!

What went wrong?: a primer
OK, we’re jumping forward and it is way, way too early to explain in detail here but the error is caused by the fact that we’ve not told luatex.exe anything about the world in which it is running. In ultra-simplistic terms, luatex.exe is completely unaware of its environment and needs to have some additional information, which is a combination of the mysterious and magical texmf.cnf file, together with something called ".fmt" files.


kpathsea: Running mktexfmt luatex.fmt
luatex.exe: fatal: kpathsea: CreateProcess() failed for `mktexfmt luatex.fmt' (Error 2)

Over the course of a number of tutorials I will do my best to explain the truly wacky world of texmf.cnf and the magic of .fmt files, which are compiled versions of macro packages such as plain TeX, LaTeX and so forth. Stay tuned…

LaTeX page layout parameters (Part 2)

In Part 1 of this tutorial we looked at the relationship between LaTeX’s page layout parameters and the conventional “DTP” layout model used by designers to parameterise page layout (refer to this diagram for a refresher). So, the next step is how do we use the formulae in Part 1 to do something useful? In this tutorial I will show one way to do this using a Lua script and LuaTeX.

LuaTeX: a primer
In future tutorials I will explain how to get started with a minimal LuaTeX setup but for present purposes I will have to assume that you have a working LuaTeX installation, perhaps via TeX Live, MiKTeX or something similar. In essence, LuaTeX is derived from, and extends, pdfTeX through the integration of Lua as an embedded scripting language and a whole host of additional functionality. It is the integration of the Lua scripting language which, in my opinion, makes LuaTeX so powerful and interesting. To run Lua code you use a LuaTeX command called \directlua{...} where ... is your Lua code. Through the \directlua{...} command you can execute your own Lua code and access the “LuaTeX API” which provides a huge range of libraries and functions that give you access to, and control of, many aspects of TeX’s internals through Lua code. This is an extremely simplistic overview, so should you want more information, grab a copy of the LuaTeX Reference Manual which is constantly updated as the LuaTeX program evolves. Note that the LuaTeX Reference Manual is a highly technical document which presents the APIs and libraries, it is not a user guide and is written by programmers for programmers.

Where are we heading?

The goal is to create a simple setup so that you can define custom pages and margins through LaTeX code such as this:

\documentclass[11pt,twoside]{article}
\input setpage
\setpage{300}{485}{250}{255}{5}{5}{5}{5}
\begin{document}
....
\end{document}

Where \input setpage inputs a file (called setpage.tex) which contains the macro to define a command called \setpage. This takes a number of arguments to define your custom page and margins. Of course, this should be wrapped up into a proper LaTeX package but that is left as an exercise for the reader.

So, what do we need to do?

  1. Create a Lua script that will perform calculations of the formulae presented in Part 1.
  2. Use \directlua{...} to run the Lua script in (1) via LuaTeX.
  3. Combine (1) and (2) into a very simple command (\setpage) you can use in your LaTeX document to define custom pages.

Create a Lua script

The starting point is, of course, that we want to have any size of paper with a document (book, business card etc) centred on the paper area. Because LuaTeX is based on pdfTeX it uses the parameters \pdfpagewidth and \pdfpageheight to set the width and height of the paper (i.e., the PDF page size). See the pdfTeX user manual for details. So, using \pdfpagewidth and \pdfpageheight we can set the size of the paper we want to use. Next, we need to know the page size of the document we want to produce. With the PDF paper size and document page size we can calculate the values of ΔX and ΔY to centre the document on our paper. The following SVG graphic (displayed via an iframe) shows the scenario we are aiming for.

So, our starting point is a set of page layout variables from the “DTP design world” which we will use to calculate the appropriate LaTeX page parameters. Our “DTP design world” parameters are:

  • PaperWidth = the value of \pdfpagewidth
  • PaperHeight = this is the value of \pdfpageheight
  • BookPageWidth = your document’s page width
  • BookPageHeight = your document’s page height
  • BookOuterMargin = BOM (refer to this diagram for a refresher)
  • BookInnerMargin = BIM (refer to this diagram for a refresher)
  • BookTopMargin = BTM (refer to this diagram for a refresher)
  • BookBottomMargin = BBM (refer to this diagram for a refresher)

Using the “DTP design world” values, we need to calculate the following LaTeX page layout parameters:\textwidth, \topmargin, \oddsidemargin, \evensidemargin and \textheight.

The following code shows one simple Lua script that will do the job. It defines a Lua function calcvals(arg) which has a single argument called arg . In Lua-speak arg is a table so that when we call the function calcvals({...}), the data in braces {...} is passed in as the value of arg and will contain a number of values which are accessed using the standard Lua method for accessing table values, such as arg.PaperWidth and arg.BookPageWidth. The code also sets some LaTeX parameters to 0 and defines “OneInch” as 25.4 – note that we are working with units in mm, just because it is convenient; hence 1 inch is 25.4 mm.

The most important point to note is the line

local marg = assert(io.open("path_to_your_tex_setup/margins.tex","w"))

We are creating a file called margins.tex which will define all the LaTeX parameters to achieve our preferred page layout. It will subsequently be input into our LaTeX document via TeX code such as \input margins.tex, hence you will need to output margins.tex into a location where the LuaTeX engine can find it.

Use \directlua{...} to run the Lua script

OK, at this point we have a Lua script which takes, as input, our “DTP design world” values and uses them to calculate, and write out, the LaTeX page layout values into a file called margins.tex. The next question is how do we get LuaTeX to run this Lua script? Answer, of course, \directlua{...}! Let us assume that the Lua code above is saved into a file called, say, code.lua.

loadfile(): a primer
At this point I need to briefly explain a Lua function called loadfile(). Quoting from the official documentation “… loadfile also loads a Lua chunk from a file, but it does not run the chunk. Instead, it only compiles the chunk and returns the compiled chunk as a function.” What this means is that calling loadfile(code.lua) does not actually execute the code in code.lua but compiles it and returns a function that you need to run in order to actually execute code.lua and so gain access to the calcvals(arg) function we defined in our Lua script. This may all sound a bit weird if you have not programmed in Lua but like most descriptions, it sounds more difficult that it actually is in practice.

Ignoring the LuaTeX side of things for the moment, concentrating just on Lua, to gain access to the function calcvals(arg) defined and stored in code.lua we have to

  1. run loadfile("path_to_your_tex_setup/code.lua")
  2. strictly speaking, make sure the code.lua loaded without error (check error return from loadfile(...))
  3. if successful, run the function returned from the call to loadfile("path_to_your_tex_setup/code.lua")

pagecalcs, loaderror = loadfile("path_to_your_tex_setup/code.lua")
-- you should check loaderror.
-- if pagecalcs == nil there was an error.
-- the error text will be in loaderror.
-- if pagecalcs is not nil, then code.lua compiled OK.
-- and will be in pagecalcs as function we can call as pagecalcs().

Assuming loadfile("path_to_your_tex_setup/code.lua") is successful, we can execute the returned function pagecalcs() which will enable us to use the calcvals(arg) function we defined and stored in code.lua.

Back to \directlua{...}

After the loadfile(...) detour we can now look at wrapping this up into something we can use with LuaTeX via \directlua{...}

As discussed, \directlua{...} is the command provided by LuaTeX which is, in effect, our “interface” or “gateway” to running Lua code from within a TeX document. The minimal LuaTeX code to run code.lua would be


\directlua {%
pagecalcs, loaderror= loadfile("path_to_your_tex_setup/code.lua")
pagecalcs()
}

And finally… \setpage

With no further ado, here is the LuaTeX code for setpage.tex. You’ll notice that \setpage takes 8 TeX parameters:

\def\setpage#1#2#3#4#5#6#7#8

which will be explained below.

The 8 parameters for \setpage#1#2#3#4#5#6#7#8 are passed straight into \directlua{...} so that our Lua function calcvals(arg) can use those 8 TeX parameters to do all the calculations, which result in the margins.tex file containing the LaTeX page parameters. This is where TeX meets Lua! The key thing to observe is the call to calcvals({...}). We mentioned earlier that when we defined calcvals(arg) in our Lua file code.lua that arg was a Lua table, now we can see this in action. The code in bold is the actual value of arg:


calcvals({
PaperWidth=#1,
PaperHeight=#2,
BookPageWidth=#3,
BookPageHeight=#4,
BookOuterMargin=#5,
BookInnerMargin=#6,
BookTopMargin=#7,
BookBottomMargin=#8
}
)

In addition, the definition of calcvals(arg) in our Lua file code.lua made calcvals(arg) return a number of values. Note that a nice feature of the Lua language is that it will return multiple values from function calls.

deltax, deltay, textwidth,topmargin,oddsidemargin,evensidemargin,textheight =
calcvals({PaperWidth=#1, PaperHeight=#2, BookPageWidth=#3, BookPageHeight=#4, BookOuterMargin=#5,
BookInnerMargin=#6, BookTopMargin=#7, BookBottomMargin=#8})

Why might we want to return those values? Well, if you look at code.lua carefully, you will see that it creates another table called pagevars and stores various page layout parameters in that table. The reason for doing this is to make the pagevars table available to other Lua scripts in our LuaTeX document. What we create is a Lua table which saves useful page parameter values we can use for other things, such as printers marks because we now have easy access to the necessary data.

pagevars={}
pagevars["paperwidth"]=#1
pagevars["paperheight"]=#2
pagevars["bookpagewidth"]=#3
pagevars["bookpageheight"]=#4
pagevars["bookoutermargin"]=#5
pagevars["bookinnermargin"]=#6
pagevars["booktopmargin"]=#7
pagevars["bookbottommargin"]=#8
pagevars["deltay"]=deltay
pagevars["deltax"]=deltax
pagevars["textwidth"]=textwidth
pagevars["topmargin"]=topmargin
pagevars["oddsidemargin"]=oddsidemargin
pagevars["evensidemargin"]=evensidemargin
pagevars["textheight"]=textheight}

Conclusion and example

To use all of this you will need to make sure that the TeX and Lua code is placed in locations where your LuaTeX executable can find them. In a future tutorial I will explain how to set up a minimal LuaTeX system on Windows (sorry, I don’t work on Linux or Mac). Here is a mimimal file demonstrating the use of \setpage

\documentclass[11pt,twoside]{article}
\input setpage
\setpage{300}{485}{250}{255}{5}{5}{5}{5}
\begin{document}
Hello Lua\TeX\
\end{document}

Looking back at our code for calcvals(arg)


calcvals({
PaperWidth=#1,
PaperHeight=#2,
BookPageWidth=#3,
BookPageHeight=#4,
BookOuterMargin=#5,
BookInnerMargin=#6,
BookTopMargin=#7,
BookBottomMargin=#8
}
)

we can see that \setpage is called with the following values (note: all assumed to be in mm!)

\setpage{PaperWidth}{PaperHeight}{BookPageWidth}{BookPageHeight}{BookOuterMargin}{BookInnerMargin}{BookTopMargin}{BookBottomMargin}

So, this is where we hoped to get to: using “DTP world” parameters to achieve the equivalent layout in LaTeX.

Example


\documentclass[11pt,twoside]{article}
\input setpage
\setpage{500}{500}{300}{300}{10}{10}{10}{10}
\begin{document}
\pagestyle{empty}
Hello Lua\TeX
\end{document}

Here is the PDF document output by LuaTeX but note that the printers marks code is not included in this example, that’s for another day!