Building and using CTIE, CWEAVE and CTANGLE on Windows

Posted on by

Introduction

Before continuing, I should say that this post is a sort of aide-mémoire for myself but I hope it is useful to others as well. Anyone who has looked into building TeX from the WEB source code soon finds that the process is somewhat “less than straightforward”. Life can get a bit more complicated if, like me, you prefer to use Microsoft’s Visual Studio rather than MSYS/MinGW – which gives you a cut down “Linux-like” build environment. I use MSYS/MinGW for building LuaTeX and it works really well, but I confess to being seduced by the nice IDE of Microsoft’s Visual Studio. Having used Visual Studio to build a couple of C-based TeX distributions (Y&Y TeX, now open sourced), together with CXTeX, part of MetaTeX (by Taco Hoekwater), I have decided to “bite the bullet” and create a Visual Studio build for LuaTeX. I’m sure this will take quite some time but, you know, sometimes you get one of those itches you just have to scratch! And I’ve been meaning to attempt this for a long time, purely as an exercise.

CWEB

A lot of LuaTeX source code (apart from the libraries it uses) is written in CWEB, a dialect of literate programming by Silvio Levy and Donald Knuth. The original WEB sources of TeX use Pascal as the programming language, but for CWEB it is C – which, thankfully, saves you the painful process of converting Pascal to C via Web2C. Anyone who has looked at the C code generated by Web2C will, I’m sure, be dismayed because it’s almost impossible to read. This is not so surprising given that it’s machine generated. The joy of LuaTeX’s C code, derived from CWEB, it that it is much, much more readable than the C code derived WEB –> Web2C –> C. Not suprising, of course, because CWEB C code is not generated mechanically.

So, how do you process CWEB code?

Enter CWEAVE and CTANGLE. What are these, you may well ask. Their task is to process a file written using CWEB (typically, with an extension “.w“) and output the C source code (using CTANGLE) or the program’s documentation in TeX (using CWEAVE). From http://sunburn.stanford.edu/~knuth/cweb.html:

  • CTANGLE: converts a source file foo.w to a compilable program file foo.c;
  • CWEAVE: converts a source file foo.w to a prettily-printable and cross-indexed document file foo.tex.

The LuaTeX build process (with MSYS/MinGW) also generates the executable ctangle.exe, so my first thought was “Great, I’ll just use that to generate C source from the CWEB *.w files in the LuaTeX distribution”. Running ctangle (from the LuaTeX build) using the command line (under the Windows cmd shell, or the MSYS BASH shell):

ctangle --help

you get the following output:

$ ctangle --help
Usage: ctangle [OPTIONS] WEBFILE[.w] [{CHANGEFILE[.ch]|-} [OUTFILE[.c]]]
Tangle WEBFILE with CHANGEFILE into a C/C++ program.
Default CHANGEFILE is /dev/null;
C output goes to the basename of WEBFILE extended with `.c'
unless otherwise specified by OUTFILE; in this case, '-' specifies
a null CHANGEFILE.

But first, change files and CTIE

From the output of ctangle --help you can see that its command line includes reference to CHANGEFILE.ch. So, what is that? Suppose that you have some program foo.w written in CWEB and you want to make some platform-specific modifications to foo.w. Rather than amending foo.w itself (e.g., to keep it platform-independent) you “merge” foo.w with a change file which, for example, may contain Windows-specific code. You would put your Windows CWEB code into, say, win32.ch and merge this code with foo.w. So how do you do this merge? There are two main ways:

  1. you can combine foo.w and win32.ch using CWEAVE or CTANGLE, or
  2. you can use another program called CTIE

What is CTIE?

The idea behind CTIE is that it lets you merge a master CWEB file with multiple change files, whereas CWEAVE or CTANGLE support only 1 change file on their command line. The source code of CTIE is also part of the LuaTeX distribution, in the directory ..\source\texk\web2c\ctiedir\. In there you’ll find a file called ctie.c which compiles easily using Visual Studio to give you ctie.exe. If you want to read more about CTIE I have processed the documentation which you can download as a PDF.

Default CHANGEFILE is /dev/null

On reading the Usage information output by ctangle --help you should note that the Usage instructions state: Default CHANGEFILE is /dev/null. The explanation of /dev/null on http://en.wikipedia.org/wiki/Dev-null state that:

“In Unix-like operating systems, /dev/null or the null device is a special file that discards all data written to it.”

The Usage instructions are a little cryptic but what it is saying is that if you want to run CWEAVE or CTANGLE without using a change file, you would run it like this:

ctangle foo.w -

where the hyphen (-) in effect says “don’t use a changefile”. Let’s take an example CWEB file from the LuaTeX distribution, align.w, and try to generate the C source code using the version of ctangle built during the LuaTeX compilation process using MSYS/MinGW. Here we don’t want to apply a change file so we’ll use the hyphen option in place of a change file (the C output file will default to align.c):

ctangle align.w -

The resulting output is:

This is CTANGLE, Version 3.64 (TeX Live 2011)
! Cannot open change file NUL. (l. 0)

(That was a fatal error, my friend.)

That’s a bit annoying, but the fix is very simple and there are a couple of ways to do it.

How to fix this?

To build CTANGLE you needs two files: ctangle.c and common.c, both of which are located in the source directory of LuaTeX (..\source\texk\web2c\cwebdir\). The “offending” code which causes the fatal error is located in common.c (and, of course, common.w).

In common.c (or common.w) you’ll find the line:

if (found_change<=0) strcpy(change_file_name,"/dev/null");

and that needs changing for Windows. Fortunately, in the LuaTeX distribution (..\source\texk\web2c\cwebdir\) there is a change file, comm-w32.ch, taking care of this (written by Fabrice Popineau, in February 2002). In comm-w32.ch you'll find the above line replaced with:

if (found_change<=0) strcpy(change_file_name,"NUL");

Of course, the proper way to fix this is to apply a change file (such as comm-w32.ch) to the CWEB source of common.w and re-generate common.c with the above fix. You can fix common.c in at least two ways:

  1. manually edit common.c to replace "/dev/null" with "NUL" in the line above, or
  2. use the LuaTeX-build created version of ctangle but with the comm-w32.ch change file – it was the absence of a change file that we are trying to fix.

Note: If you are experimenting with these CWEB tools I strongly suggest you make a copy of all your *.w files into a working directory in case you make an error and accidentally overwrite any files.

Copy ctangle.exe, common.w and comm-w32.ch to a working directory away from your main source code, CD into that directory (make it the current directory), and run the following command line (it works under DOS and the MSYS BASH shell). the "./" simply tells ctangle to look in the current directory.

$ ctangle ./common.w ./comm-w32.ch ./mycommon.c

If successful, the output should be:

ctangle ./common.w ./comm-w32.ch ./mycommon.c
This is CTANGLE, Version 3.64 (TeX Live 2011)
*1*5*7*27*56*67*77*81
Writing the output file (./mycommon.c):.....500.....1000
Done.
(No errors were found.)

giving you a new version of common.c (which I called mycommon.c) with the fix applied by comm-w32.ch. If you look at the last lines of the mycommon.c file you just generated you should see something like this:

#line 78 "./comm-w32.ch"
if(found_change<=0)strcpy(change_file_name,"NUL"); #line 1283 "./common.w"

You can see that the line 78 of comm-w32.ch has been applied. Now, with the fixed file (mycommon.c) you can proceed to build CTANGLE using Visual Studio to generate an executable that accepts the "NULL" change file. We'll see that in a moment.

Let's use a different approach: using CTIE to merge common.w and comm-w32.ch into, say, mycommon.w. From mycommon.w we'll use our newly compiled CTANGLE to output mycommon.c. The the following CTIE command line does the trick:

ctie -m mycommon.w common.w comm-w32.ch

The -m option is documented here. If successful, you should see something like this:

This is CTIE, Version 1.1
Copyright 2002,2003 Julian Gilbey. All rights reserved. There is no warranty.
Run with the --version option for other important information.
(common.w)
(comm-w32.ch)
....500....1000....
(No errors were found.)

However, you can of course simply edit the file common.c directly to make the change. Once you fix common.c, both CWEAVE and CTANGLE compile nicely with Visual Studio and work perfectly when you use the "-" option to indicate no change file. With a working CTANGLE you can generate the C source of newcommon.w like this:

G:\CWEB\cwebtools\Debug>ctangle newcommon.w -
This is CTANGLE (Version 3.64)
*1*5*7*27*56*67*77*81
Writing the output file (newcommon.c):.....500.....1000
Done.
(No errors were found.)

With CTANGLE in place you can now run it on the CWEB *.w sources in LuaTeX to generate the C code. Clearly, for Visual Studio one way to proceed is to incorporate CWEB sources into your project and have a "Custom Build Step" for .w files, processing them with CTANGLE.

Happy TeXing, or should I say ctieing, ctangling and cweaving!