DOCMaker

From April 1995 to April 1996, ATPM was available exclusively in DOCMaker format. DOCMaker was a nifty application that provided a primitive, SimpleText-style editing environment. DOCMaker didn’t support spell-checking, tabs, or wrapping text around graphics, but its signature feature was that instead of saving its documents in SimpleText format, it could save them as stand-alone documents. These “documents” were actually applications: your document, combined with the software needed to view it. So, you could double-click an issue of ATPM, and it would display itself in a nice interface, complete with a search feature and commands for navigating between different articles. Most people received the ATPM DOCMaker files via e-mail or downloaded them from AOL or eWorld.

HTML

In May 1996, we first made ATPM available on the Web in HTML format. At first, Evan Trent and Nancy Ross converted the DOCMaker issues to HTML by hand. Starting with ATPM 3.09, I began using Myrmidon to do this (mostly) automatically by printing to HTML. Most people, however, preferred to download the DOCMaker files from our FTP site. They were compressed with StuffIt for faster downloading and, once downloaded, provided a faster and more Mac-like reading experience. Web browsers weren’t always as nice as they are today.

PDF

A few years later, Adobe Acrobat was widely available, and ATPM readers were clamoring for PDF-formatted issues. There were a variety of reasons for this. DOCMaker documents didn’t print well. Graphics would shift around because the layout was done using spaces, and sometimes graphics or even lines of text would be split across page boundaries. The main body font that we used, Geneva, looked great on-screen, but wasn’t optimal for printing. DOCMaker couldn’t embed fonts, so text using fonts that weren’t built into the Mac OS had to be converted into graphics. (This also let us “de-bump” the edges of the text, since font smoothing wasn’t yet built into the Mac OS.) Anyway, these fonts were rasterized at screen resolution, so they looked blocky when printed.

In July 1999, we introduced the first PDF version of ATPM. Aside from printing better (because of better pagebreaks, vector graphics, and embedded fonts), PDF offered a number of benefits. Since it was a cross-platform format, ATPM readers could download and view it using Windows machines at work. By this time, URLs were more common in ATPM articles. DOCMaker, via ICeTEe, could make URLs in the text clickable if you held down the Command key. We wanted to include more URLs, but sprinkling too many of them throughout the text made the paragraphs unreadable. PDF had the ability to make any text clickable, just like the Web, and the actual URL could be hidden at the bottom of the page (so it would be visible when printed).

Finally, switching to PDF made it much easier to produce ATPM. We could now use Adobe FrameMaker, a real word-processing and page-layout program. Besides providing a nice editing environment, FrameMaker made it possible to set up the clickable links and bookmarks (for article navigation) ahead of time. Most other software, even today, requires a separate, manual pass using Adobe Acrobat to add these features. Aside from being more work (you have to draw boxes to set the clickable area for each link), this wasn’t feasible for us because it would have to be done at the last minute, after converting the issue to PDF. Finally, in addition to exporting to PDF, FrameMaker had a customizable HTML export feature, which actually generated pretty good HTML, needing just a little post-processing via a Perl script.

eDOC and Offline Webzine

The PDF-formatted ATPM got a great reception, but many ATPM readers continued to prefer the DOCMaker format because it was easier on the eyes (on-screen) and because Acrobat Reader was slow and unwieldy. With ATPM now being produced in FrameMaker, it wasn’t possible (given our staff and time constraints) to continue producing a DOCMaker version. However, FrameMaker had some powerful features that allowed us to solve this problem with the help of a wonderful utility called eDOC. Like Myrmidon, eDOC is a printer driver that intercepts drawing commands to create a file instead of a printed document. However, instead of creating HTML, it creates a stand-alone document that’s similar to DOCMaker, but that supports arbitrary layout (like PDF). Thus, with the help of FrameMaker’s stylesheets and its conditional text feature, we were able to produce PDFs as well as stand-alone eDOC documents from the same source files. The eDOC version of ATPM was introduced in August 1999, along with the Offline Webzine edition that continues to this day.

Screen and Print PDFs

In May 2000, we had to stop producing the popular eDOC edition. The eDOC driver had stopped working with FrameMaker, and despite a clean installation and help from eDOC’s author, we were not able to get it working again.

Instead, we upgraded from FrameMaker to FrameMaker+SGML. FrameMaker works like a word processor, with both character- and paragraph-level stylesheets. The more powerful FrameMaker+SGML is designed for creating structured documents. Instead of having two styles (character and paragraph), each run of text is part of a tree of elements (think HTML tags). This richer markup enables the creation of context-sensitive layout rules, and this made it possible to create screen- and print-optimized layouts, which we debuted in August 2000. The screen version had larger fonts and a smaller page size, while the print version had a two-column US Letter layout with smaller fonts and footnotes showing where each URL would lead.

Goodbye FrameMaker

This combination of four formats, online and offline HTML editions and two PDF formats generated with FrameMaker+SGML, has served us well. Each has a loyal following, and FrameMaker+SGML’s power, combined with various scripts developed over the years, made this a comfortable combination of formats to produce. However, the Mac world has changed in the nearly five years since we started using FrameMaker+SGML. FrameMaker (version 7 incorporates the SGML features and is simply called FrameMaker) is out of favor at Adobe, which is now focussing its development efforts on InDesign. FrameMaker, which started out as a Unix application, was behind the times interfacewise when we started using it. The latest version is little better, and Adobe has halted development of the Mac version of FrameMaker. It will never be Carbonized for Mac OS X.

FrameMaker continues to run very well in the Classic environment. In fact, it’s the only Classic application that I use regularly, and it runs much faster now on my G5 with Mac OS X 10.4 than it ever did running directly in Mac OS 9. All that said, it’s time for a change. FrameMaker has bugs and limitations that add several hours to the production of each issue of ATPM. It no longer makes sense to live with them now that it’s clear that they’ll never be addressed. Also, the upcoming Macs with Intel processors will not be able to run Classic, which means that in a few years Apple will no longer make any Macs that run FrameMaker. (It seems that Apple’s own technical writers are in the process of switching from FrameMaker to a system using XML and XEP.)

New Tools

There are surprisingly few options on Mac OS X for producing rich PDFs (with bookmarks and clickable links) and HTML documents from the same source. One might first consider Microsoft Word, or Apple’s new Pages, but the Mac version of Word can’t create rich PDFs, and neither can Pages, despite Mac OS X’s overall excellent support for PDF. Plus, both have poor HTML export abilities. Adobe InDesign can, but it’s more for page-layout than word-processing, and it lacks some of FrameMaker’s powerful auto-formatting features. Every time I contemplated switching to InDesign, I vowed to continue using FrameMaker, even if that meant running it on an older Mac if it stopped working with the latest Apple hardware and software.

Fortunately, Mac OS X’s Unix base provides new options in the form of open-source software that can be programmed to do what we want. The core of our new publishing system is Docutils, a text processing system designed for use with the Python programming language. I now format ATPM articles in reStructuredText format, a plaintext markup language that’s similar to Setext and Markdown.

Docutils includes a parser, which reads reStructuredText files into an internal representation. Then, a writer translates the internal representation into an output format such as HTML. The beauty of the Docutils system is that both sides of this process are extensible. reStructuredText doesn’t have support for the blue ATPM article headers, but I was able to teach it about them by extending the parser. Similarly, Docutils doesn’t have writers that generate ATPM-style HTML or PDFs, but it lets you plug in writers of your own. As a programmer, it was simple for me to make these and other extensions, and Docutils’ good design meant that I didn’t have to re-invent the wheel to do so.

Actually, developing a writer for the PDF format would be quite a bit of work; PDF is much more complicated than HTML. Instead, I created a writer that, with the help of PyObjC, generates output in LaTeX format. LaTeX is a somewhat arcane publishing system built on top of Donald Knuth’s TeX language, and TeX incorporates a lot of typesetting knowledge, such as how much space to put between each word so that the edges of the columns are flush. In fact, it’s significantly better at this than FrameMaker, or pretty much any other program. So, the writer produces LaTeX (slightly different LaTeX for the screen and print versions), which I then feed into the open-source pdfLaTeX program to produce the PDFs.

My ATPM holy grail is to be able to edit and assemble the articles for an ATPM issue, press a button, and have all four formats automatically generated. With this new publishing system, we are finally close to this goal. Since it’s programmed using lower-level tools rather than off-the-shelf applications, it’s possible to get the new system to do exactly what we want (subject to the limitations of my LaTeX skills), and easier to automate it. In 1996, I never would have expected that ATPM would someday be published in this way, but Mac OS X opened the doors for this new system, even as it hastened the end of the old FrameMaker-based one.

Finally, it occurs to me that nearly all of ATPM is now produced using BBEdit, the flagship product of our current sponsor, Bare Bones Software. BBEdit provides spell-checking and various other tools for editing ATPM. Its function pop-up lets me navigate the articles by section, and its text formatting tools make it easy to convert our submissions into reStructuredText format. But BBEdit is also a programmer’s editor, and so it was essential in writing the Python, Perl, and Make code that comprises the new publishing system, in debugging the HTML and LaTeX files that said code generated, in writing the content management system that combines the HTML files into the ATPM Web site, and in helping to store all the text and code files safely using the Subversion version control system. I suppose every Mac user has one primary application, be it a mail program, an outliner, Photoshop, or Microsoft Word. For me, all roads lead to BBEdit.