Introduction

In this example we will use the “post_linebreak_filter” to iterate over all the lines in a typeset paragraph and use the data provided by LuaTeX to insert pdf_literal nodes which draw a box to show the size of the glue between words. It is based on a nice piece of code on the LuaTeX wiki by (I believe) Patrick Gundlach.

Note: The code does not recurse into nested lists; so if you have further boxes within the lines of your paragraph you should note that this code is not designed to handle them. An exercise for the reader ;-).

If you extend the code to deal with nested boxes, other glue types, such as \baselineskip etc, and account for glyph widths, heights and depths, what this leads to is the ability to calculate precise (x,y) positions within many types of node lists.

Code and explanations

Here, we are placing a paragraph into a \vbox{There are many variations of ….} and hooking into the post_linebreak_filter to access the node list after TeX has broken the paragraph into lines. Each line in the paragraph is accessible from LuaTeX as an hlist which is traversed using the LuaTeX API call node.traverse_id(…). We then scan through the nodes contained in each hlist, looking for glue. As mentioned, the code does not recurse so we are just choosing the individual paragraph lines and not exploring anything below that level.

Each hlist contains its glue setting parameters which let us calculate by how much the individual glue items are stretched or shrunk within each line of the paragraph. From those values you can calculate the size of the set glue and draw an appropriately sized box with a pdf_literal, whose zero size does not affect the positioning of the text. The pdf_literal is inserted into the hlist node tree just before each glue node.

For further reading on TeX’s glue calculations see page 77 of The TeXBook.

Resulting PDF

As usual, through the Google Docs viewer or download here.

Introduction

In this article I’ll show a technique for producing crop marks using the MetaPost library (MPlib) which is built into LuaTeX. There is a lot of ground to cover so I’ll try to focus on/summarise the most important/useful areas to prevent this article becoming far too long! It has already taken a few days to write it and prepare the graphics… I’m not going to attempt a tutorial on the MetaPost language because there are many excellent articles already written by people who are best qualified to produce that material. A great starting point is the TeX Users Group page on MetaPost.

SVG-enabled browser needed: This post uses inline SVG and SVG via in iframe, which may not work in all browsers.

Citing various sources: a few thanks are in order

Much of the material in this post is derived from existing work by members of the LuaTeX community, so I’d like to acknowledge those sources. Firstly, a huge thank you to Hans Hagen for creating the Lua code which converts the MPlib data to raw PDF code. It is a really excellent and hugely useful piece of code which is part of the ConTeXt distribution. In addition, I learnt a lot by reading the source code of luamplib by Hans Hagen, Taco Hoekwater, Elie Roux and Manuel Pégourié-Gonnard. I also discovered some code by Dohyun Kim which makes some very helpful additions to luamplib through which you can support the traditional btex … etex construct for including TeX within MetaPost code processed by MPlib (more of that in another post).

Pieces of the process

The key elements of the technique I’ll describe are:

1. Embedding MetaPost code (to draw a crop mark) within your LuaTeX document.
2. Converting the MetaPost output to PDF code (using the code from Hans Hagen’s ConTeXt).
3. Generating pdf_literal nodes representing the crop marks (modification to luamplib).
4. Placing the crop marks in the appropriate locations through an output routine.

What are crop marks?

Crop marks, also referred to as “printers marks”, “cut marks” or “trim marks”, are small graphics placed at the corners of a page to indicate the physical size of a final printed document pages. They are used during commercial printing activities, such as page imposition, colour separation, folding and trimming. The physical appearance of crop marks will vary depending on the application used to generate the pages but, of course, with LuaTeX and other TeX engines you are free to create your own designs. Advanced readers will be aware that for multi-colour separations (spot and 4 colour CMYK) the crop marks must appear on all plates but I’ll not cover that topic here. It would be fairly easy to do through injecting appropriate PDF code to set the colour space for the crop marks or using the Printer’s Mark Annotations feature of PDF.

The following graphic (produced with Inkscape) shows the general idea.

The following inline SVG graphic (produced directly by exporting from MPlib) shows the design of the crop mark we will be producing via MetaPost code.

The MetaPost code

I’ll readily admit that I’m an amateur when it comes to programming with MetaPost so due apologies to any experts reading the code :-). The idea here is that the MetaPost code is inline in the LuaTeX document and I’m using the process_input_buffer callback (see this post) to store the MetaPost code into a buffer which will processed via MPlib. The \startbuffer and \stopbuffer commands were described in an earlier article.

Very brief introduction to MPlib

As mentioned, MPlib is the library version on the MetaPost interpreter built into LuaTeX. Instead of using MetaPost as a standalone executable (e.g., mpost.exe) you access it through an API provided by the LuaTeX engine. The value of the integration of MetaPost with LuaTeX is well demonstrated by the truly stunning results achieved by Hans Hagen and the ConTeXt distribution. The ambitions of this article are rather more modest.

In outline the steps are as follows.

1. You need to create a “finder” function which MPlib will call to locate any files it needs.
2. Provide the finder function as one of the arguments to the API call mplib.new() which
   is responsible for creating an instance of the interpreter.
3. Load the “format file” containing the macro package you want to use (e.g., plain.mp).
4. Use the interpreter instance returned by mplib.new() to execure your code.
5. Process the figure objects generated and returned by MPlib (if your code worked without error).

Here is some sample code for a finder function and creating an instance of the MetaPost interpreter.

MetaPost to PDF or SVG

One nice feature of MPlib is that it will automatically generate an SVG representation of the graphic produced from the MetaPost code, and that is how the inline SVG crop mark (above) was produced. The MPlib library will also PostScript code and an “object representation” of the graphic. The “object representation” can be parsed to convert the graphic into other data formats and this is how the PDF data is generated by the ConTeXt Lua code contained in luamplib. It runs over the collection of objects and converts them to the equivalent representation in PDF data/structures. I really admire that code and it’s great to have it available.

Re-using code and ideas in luamplib

Within the luamplib distribution is the core Lua code from ConTeXt which does the “heavy lifting” to convert MPlib data structures to PDF code. For the purposes of the work described in this article I re-used that Lua code and placed it into Lua file (mpnodes.lua) which can be downloaded here. To use it you’ll need to load it as a Lua module:

\directlua{require(“mpnodes.lua”)}

Using mpnodes.lua

The mpnodes.lua module contains functions which create a new instance of the MetaPost interpreter (via MPlib), execute MetaPost code and use the functions from ConTeXt to generate the PDF data. During the process of generating PDF data, the Lua code makes a number of calls to defined TeX macros (originally in luamplib.sty) which, in effect, pass the PDF data to the (Lua)TeX engine. The TeX macros of interest here are:

• \def\mplibstarttoPDF#1#2#3#4{….}
• \def\mplibtoPDF#1{…}
• \def\mplibstoptoPDF{…}

As part of this implementation those macros were redefined to work with LuaTeX nodes. Examples will be provided later in the article.

Note: the mpnodes.lua module also contains other functions which I won’t describe: including work based on the code from Dohyun Kim which implements the btex … etex functionality of the standalone MetaPost interpreter. The MPlib version of MetaPost does not directly support the btex … etex construct; other methods have to be employed to include TeX code within MetaPost graphics processed by MPlib. Again, those methods are based on the pioneering work of the ConTeXt distribution. I’ll write about this in a future post.

Creating nodes to store crop marks

If you look at the figure above, which shows 4 crop marks on a page, you can see that you only need to create 1 crop mark graphic and then rotate it in increments of 90 degrees as you place it at the 4 corners. In the approach, described below, the PDF data generated from the MetaPost graphic is stored as LuaTeX pdf_literal nodes which are then drawn at each corner via the \output routine described below.

Enough description, here’s the code

Here’s the Plain-TeX-based code with inline comments.

Sample PDF

Here’s an example PDF produced by the code above.

Have you spotted the flaw?

Readers who have a background in print production may have spotted a problem with the positioning of the crop marks and the text area defined by \box255: there is no white space between the text area and the crop marks, they are too close to the live text material. Can this be fixed? Yes, very easily. What you need to do is insert some vertical kerns to move the crop marks vertically, and offset the crop marks horizontally by adjusting the width of the horizontal boxes containing the crop marks. These actions will create offsets between the crops and the text area. Of course, these will have to be factored into any calculations for your page design but that should not be difficult. What this will mean is that the page size defined by the crop marks would then be:

• width of page = \hsize + 2*(horizontal offset)
• height of page = \vsize + 2*(vertical kern offset)

We’ll look at this below.

The output routine

A crucial part of the process is to assemble everything onto the final PDF page with the crop marks placed as required. In the following I’ll discuss a basic \output routine, that ignores many complications, such as inserts, but which could form the basis for your own experiments. It could, for example, be used as a starting point for with simpler documents such as business cards. I’ll also give an example of fixing the “crop marks offset” problem mentioned above.

So where do we start?

The \output routine is responsible for assembling the page components to achieve your desired page design and the way that the following output routine works is by wrapping \box255 in a series of \vboxes and \hboxes with flexible glues to centre \box255 on the page. The overall structure is an outer \vbox to the same size as the height of the PDF page followed by another \vbox to the value of \vsize. Very flexible (stretchy) vertical glue is used to fill the “space” above and below the inner \vbox, i.e., the space that needs to be filled due to the different heights of the two \vboxes (\pdfpageheight versus \vsize). This glue is responsible for vertical centring the inner \vbox.

Inside the second \vbox (the one to \vsize) we place a series of horizontal boxes (to a width of \pdfpagewidth) to contain the crop marks above and below \box255, plus the actual typeset content of \box255 itself. We use horizontal glue to centre everything… horizontally.

Placing the crop marks

Two of the three “\hboxes to \pdfpagewidth” contain yet another \hbox (of width \hsize) and the purpose of those is to place the crop marks above and below \box255.

\hbox to\pdfpagewidth{\hfill\hbox to \hsize{\copy2000\hfill\copy2001}\hfill}%
\hbox to \pdfpagewidth{\hfill\box255 \hfill}%
\hbox to \pdfpagewidth{\hfill\hbox to \hsize{\copy2003\hfill\copy2002}\hfill}%

Let’s take a look at the first one:

\hbox to\pdfpagewidth{\hfill\hbox to \hsize{\copy2000\hfill\copy2001}\hfill}%

The inner \hbox to \hsize{\copy2000\hfill\copy2001} contains an infinitely stretchable \hfill glue which streches to fill the \hbox. The key point is that the pdf_literal nodes from which the boxes 2000 and 2001 have zero width and so the \hfill glue pushes them to the far left and right of the containing \hbox.

If you look back at the Lua code which created the pdf_literal nodes:

pdftopleft = node.new(“whatsit”,”pdf_literal”)
pdftopleft.mode=0
pdftopleft.data=buffy

you’ll see that the “mode” is set to 0. This defines the origin for drawing them as the point on the page where they appear, which is just above and below \box255 thanks to the actions of the glue.

And finally, offsetting the crop marks

• Vertically: one solution is simply to absorb some of the space which is occupied by the strechable glues, thus preventing the crop marks being pushed up against \box255.
• Horizonally: adjust the size of the \hboxes containing the crop marks (make them wider).

Here’s one very quick example where we add 20pt offset vertically and increase the \hboxes containing the crop marks to 1.25 x \hsize. A proper solution would of course paramaterise everything.

And the resulting PDF:

Download mpnode.lua

You can download it here.

Introduction

As mentioned in a previous post, LuaTeX provides a facility called callbacks in which you write a Lua function (your callback) and register it with LuaTeX through the callback.register() API function.

callback.register() takes two parameters:

• A predefined callback name: as defined by LuaTeX. This defines the action or purpose of your of function and is the “hook” into LuaTeX telling it where/why/when your function should be called.
• Your actual callback function: either a named or anonymous Lua function.

In this post I’ll give an example/framework for one simple way to use the process_input_buffer callback. This callback allows you to hook into LuaTeX’s input buffer before LuaTeX actually starts looking at it. With this callback you can, for example, completely re-process the input and return the processed results back to LuaTeX, or filter out entire sections of the input–perhaps to store it elsewhere for later use or processing.

Code example and explanation

We will register a callback function which strips out the input sandwiched between two macros called \startbuffer{..} and \stopbuffer. Again, this is a simple example and there are undoubtedly many much more sophisticated ways to do this.

The basic idea here is that the macro

switches on intercepting the input, and a dummy macro (\stopbuffer{}) is “intercepted” to switch off processing the input.

There’s a very important point here in that after we make a call to \startbuffer{…} all our input bypasses LuaTeX processing from that point on and is passed to the function we have registered: “dobuffer”. To detect the end of the buffer, and switch off the callback, dobuffer must scan the input for the string “stopbuffer” and “unregister” the callback so that LuaTeX reverts back to its standard processing of the input. For sure, this approach could fail if the string stopbuffer occurs elsewhere within the input stream. I did say it was a simple example ;-).

In the example you can see a string of characters ” ^ & # _ $$ % \” whose \catcode settings could, without their \catcode being reset temporarily, cause problems if input and processed directly by LuaTeX.

\startbuffer{figure1}
Let’s have some characters with various catcodes:
^
&
#
_
$$
%
\
\stopbuffer

However, in the example these characters are being intercepted long before LuaTeX starts to process them and so the idea of catcodes, at this point, has no meaning. Of course, they can still trip you up later in your processing.

Explanation

After the call to \startbuffer{…}, every line in the input is passed to the function dobuffer() which in our example does very little except print it out to the screen. Because the dobuffer() function returns the empty string “” the entire input between \startbuffer and \stopbuffer is removed from LuaTeX’s input: it never reaches the typesetting engine. You could use this as a simple way to implement long comment sections in your document.

You could implement the dobuffer function to perform all sorts of tasks, such as save a copy of the filtered input and store it into, say, a table perhaps indexed by the value passed into \startbuffer{…..}. The options and possibilities are endless!

Introduction

As I explore more of LuaTeX I’m constantly amazed by its versatility. Over the last few days I have been reading about \output routines which are, deservedly, deemed some of the more complex parts of TeX. Caveat to readers: I do not profess to understand very much about \output routines and am not going to cover the genuinely complicated areas such as inserts (footnotes, graphics etc), main vertical lists, page breaking, recent contributions, etc. The following is based on typesetting pure text with no bells or whistles such as headers, footers, page numbers etc.

My objective was to experiment with Plain TeX to understand more about the relationship between the size of the PDF page and just how you control where the typeset results are physically placed within the PDF page area. With the clear cavets above in mind, the way I think about this is that TeX is busily building the typeset output into a box (number 255) with width \hsize and height \vsize and when it decides there is enough to dump out a page it will “call” the \output routine. It is through the \output routine that you control the physical location of TeX’s page (box 255) within your PDF page area. This is undoubtedly a pretty simple-minded view but it works for me (until I know better…).

Setting the page parameters via Lua code

To start with, here is a small LuaTeX routine to set up the PDF page size together with \hsize and \vsize, again for use with Plain TeX. Note you can also set \hoffset, \voffset and \topskip through the LuaTeX API.

\setpageparams#1#2#3#4{…} takes 4 values (assumed in mm), converts them to TeX’s special points, and assigns the values to the appropriate dimension parameter.

Centering \box255 within our PDF page area

The following code performs three main tasks:

1. It creates an \output routine which places \box255 horizontally and vertically centred within the PDF page area.
2. It sets up a Lua table (coords) which contains the corner coordinates of the centred \box255 relative to the PDF page origin, which I’m assuming is the standard lower-left corner of the PDF page.
3. Using LuaTeX nodes (pdf_literal) it creates a shaded box that sits behind \box255 and is shipped out on every page using the \output routine.

Example results

Again I prefer to use Google Docs PDF viewer. Download here if you can’t see the resulting PDF. I hope this example is useful and, as always, I’d appreciate any notice of any technical errors.