Vim wnext, and Macros for Multiple Files

I have finished the pages for the home section. Yipee!, praise God! Almost done.
But while I did these pages I found a few things I needed to change on all the other cards. The first two were warnings from the validator:

1. My XHTML namespace declaration was invalid because I hadn't stated which language I was using. I needed to delete the whole line and insert a declaration that included xml:lang to denote the primary language of the page.
   
2. Most of the tooltips had a space at the end of the title's value, just before the closing quote. I needed to delete them all.
   
3. I needed to add a copyright notice on every page.
   

This meant at least two changes on every one of 988 files.... but thanks to Vim the whole thing took under an hour.

First I made two files and put them in the home directory:

lang.txt was

<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">

And copy.txt:

<p>© Copyright 2010 Bruce Thompson</p>

Then I moved into a section (eg. Diseases with 202 pages) and in the console:

gvim *.html

This started Vim editing all 202 html files in the directory and displayed the first file. I then recorded the following macro "@A":

/<html xml^Mddk:read ../lang.txt^MG?div^Mk:read ../copy.txt^M:%s/ ">/">/ge^M

/<html xml^Mdd Find the present namespace declaration (^M is Enter - didn't type it, just pressed Enter) and delete the line
k:read ../lang.txt^M - Move up one line and read the lang.txt file
G?div^M - Move to the end of the file and search backwards for the last closing div
k:read ../copy.txt^M - Move up a line (to where the copyright needed to be) and read in the copy.txt file.
:%s/ ">/">/ge^M - Substitute the attribute ending with the space for one without a space. The "g" flag means that it will substitute all the occurrences on the line but the "e" flag is essential in a multifile macro because this doesn't report an error when there is no occurence in a particular file (and therefore stop the macro repeating).

After I checked the file and validated it. I made a new macro "@B":

:wnext^M/<html xml^Mddk:read ../lang.txt^MG?div^Mk:read ../copy.txt^M:%s/ ">/">/ge^M

:wnext - Wrote the file and moved to the next file in the series then repeated what I had done in previous macro, "@A".

When this file validated correctly, I issued the following command

:200@b

This repeated the "@b" macro on the remaining 200 files in the directory.

I repeated this for each of the remaining three sections and had it all done in under an hour. So it is now finished... I've only got to find a place to host it.

Vim Pagemaking Macro

Today is a high day. I finished making content pages (all the Hydrotherapy stuff) at 2.30 this afternoon. I wondered if it would ever end.
The numbers breakdown is:
Diseases: 202
Effects: 54
Problems: 437
Techniques: 295
Making a grand total of 988 pages.

The Vim pagemaker macro I used was:

/#card^Mf<"ly2f>:sp PageIndex.html^MG$o^[p:wq^Mj^"ty$j"fy$:sp ^Rf^M:read top.h^MG$"tp:read head.h^MkJxG$:read cont.h^MG$:read breadcrumb.h^Mj/<\/p>^Mh"tpG$"tpa</h1>^[^Wwjma/#end^Mkd'a^WWpG$:read rightbox.h^MG$:read badge.h^MG$:read bottom1.h^MG$:read col1.h^MG$:read bottom2.h^MG$:read col2.h^MG$:read bottom3.h^MG$:read col3.h^M/^Rf^M^faa class="here"^[G$:read footer.h^MG:w^M

To tease it out a bit we need to remember that the first three lines of the card "Visceral Congestion" were:

#card @<a href="VisceralCongestion.html">Visceral Congestion</a>
Visceral Congestion
VisceralCongestion.html

And we need to know that:
"l = register L etc
^M = Enter
^[ = Escape
^W = Ctrl-W (for window navigation with a split screen)
^R = Ctrl-R (reads in the contents of following register)

The macro would start with the cursor in the previous card so:

/#card^Mf<"ly2f>:sp PageIndex.html^MG$o^[p:wq^M

will find the first line of the card, yank the link, <a href="VisceralCongestion.html">Visceral Congestion</a>, to Register L (Link) and append this link to the end of the Page Index file.

j^"ty$j"fy$:sp ^Rf^M

Then moving down to the second line (Visceral Congestion) then yanks it to Register T (Title). Then it moves down to the third line (VisceralCongestion.html), appends it to Register F (Filename) and splits to a window editing the file, VisceralCongestion.html.

:read top.h^MG$"tp:read head.h^MkJxG$:read cont.h^MG$

This starts the Visceral Congestion.html with the DTD and other HTML down to the <title> tag by "reading" in the contents of a file I had previously made called "head.h". It then puts the contents of Register T (Visceral Congestion) and moves to the end of the file (with G$) before reading in the contents of the file "cont.h" which takes us down to the top of the content of the file.

:read breadcrumb.h^Mj/<\/p>^Mh"tpG$"tpa</h1>^[

This reads in the breadcrumb navigation, which I made at the beginning of each section, and places the title at the end then completes the heading of page by placing the title in h1 tags.

^Wwjma/#end^Mkd'a^WWpG$

Using window navigation we return to the lower window containing the contents of the card, places Mark A (ma) on the first line of the file (just below the filename and searches for the end of the card. It then deletes the card's contents, navigates up a window puts the contents then moves to the end of the file.

:read rightbox.h^MG$:read badge.h^MG$:read bottom1.h^MG$:read col1.h^MG$:read bottom2.h^MG$:read col2.h^MG$:read bottom3.h^MG$:read col3.h^M

A long list of file with various HTML are read in at the end of the file, the last ones containing the site map at the bottom of the page. The "col" files had the contents of the map.

/^Rf^M^faa class="here"^[G$:read footer.h^MG:w^M

Once the contents of the site map were read in, the page's title needed highlighting with the class "here" and the footer file finished off the page.

I then opened the page in Firefox, corrected any obvious problems then validated it with Total Validator (6.3.0) as desktop tool.

Total Validator was great with British English spellchecking and accessibility validation as well as the expected HTML validation.

The whole process for one page without major problems took only a minute.

But now it's finished. Just the main index page and the messy stuff up the top of the site to write now.

Tooltips with Vim Regular Expressions

Off the top of your head, can you define "anodyne" or "faradic"?

Probably not, but these and many more obscure words, occur throughout Traditional Hydrotherapy. To overcome the lack of knowledge of these terms I had made up a glossary. I now wanted to make this available easily to the reader of Traditional Hydrotherapy. The answer was in Tooltips.

To illustrate a tooltip just hover your mouse over anodyne or faradic. That box that springs up is a tooltip and exactly what I needed... a simple way to get the definition to the reader.

The code for the "anodyne" tooltip looks like this:

<span title="a treatment to relieve pain">anodyne</span>

Problems presented themselves immediately, and, as always, Vim was up to the task.

This is the text from the glossary file (made under DOS with PCWrite):

#card @faradic
electric current used mainly for stimulating innervated muscle, consists of pulsed DC current
#end

Problem: I had to turn the definition into a single line.
Answer: Use "J", which appends the following line with a space in between.

Second Problem: These words are often part of larger words and I didn't want to chop words up.
Answer: Use the Vim regular expressions:
Word Beginning, "\<", and Word End, "\>".

This way I would be able to make tooltips around whole words only.

Thirdly: how do I substitute the text for both upper and lowercase occurrences, "Faradic" and "faradic"?

The answer is two macros that begin on the "#card" in the glossary file, glos.txt and split to a single file which contained all four html files (appropriately called "big.html" as it ended up at over 3Mb:

@G - which deals with lowercase version

then @M which does the tooltips for the upper-case and also deals with words in the titles of pages.

and @N a final macro that corrects some of the changes @M made and goes to the next card in the glossary.

Let's have a look at the macros:
@G

/@^Ml"tyEj^"dy$:sp big.html^M:%s+\<^Rt\>+<span title="^Rd">^Rt</span>+gc^M

/@^M -search for "@"
l"tyE - move right one (onto the first letter of the title) and yank to End of the word into register "t"
j^"dy$ - move down one line and to the beginning of the word then yank to end of line into register "d"
:sp big.html^M - split screen and open into the file containing all the html
:%s+\<^Rt\>+<span title="^Rd">^Rt</span>+gc^M
:%s+\<^Rt\>+^Rt+gc^M - the business substitution - using Word Beginning, "\<", and Word End, "\>", this is to prevent words as part of URLs being substituted. The registers are called using the "-R (register letter)" form. The substitution is for all occurrences on a line and "c"hecks. so I can be sure it is going OK.


@M

^Wpk/@^Ml~h"lyE^Wp:%s+^Rl.html+LL^Rl.html+ge^M:%s+\<^Rl\>+<span title="^Rd">^Rl</span>+gc^M

^Wp - return to the previous window (glos.txt) with -W p
k/@^M - move up a line and search for "@"
l~h - move one letter to the right and change the case of the letter (to capitalise)
"lyE^Wp - yank to end of the word into register "l" and return to the previous window (big.html)
:%s+^Rl.html+LL^Rl.html+ge^M - a substitution to add "LL" to the beginning of the word if it is a filename and thus preventing it being changed.
:%s+\<^Rl\>+<span title="^Rd">^Rl</span>+gc^M - much the same as the substitution in @G except using register "l" instead of "t"

@N

:%s+LL^Rl+^Rl+ge^M:wq^Mj/#card^M

:%s+LL^Rl+^Rl+ge^M - a substitution to remove the, now unneeded "LL" notice the "e" tag which means it doesn't stop on an error (usually where the LL doesn't exist because there were no pages to be made with that filename)
:wq^M - write and quit the big.html file
j/#card^M - now in glos.txt, move down a line and search for the next occurrence of "#card" (for the next word)


Using these files I managed to add tooltips to about 200 words in my spare time last week.

Now to make all those files (and validate them).

CSS Compliant Facebook Badge

I made a Fan Page for Traditional Hydrotherapy. There is nothing on it yet, I just needed to get a URL so I could finish my page layout. I decided to place a Facebook badge for this page in the right column of the website's pages if there was nothing else to go there.

When I created a badge the Facebook way. I ended up with this HTML to put on each page:

<!-- Facebook Fan Badge START -->
<div style="width: 100%;">
<div style="background: #3B5998;padding: 5px;">
<img src="http://www.facebook.com/images/fb_logo_small.png" alt="Facebook"/>
<img src="http://badge.facebook.com/badge/111459758873296.100000637781892.2048613001.png" alt="" width="0" height="0"/>
</div>

<div style="background: #EDEFF4;display: block;border-right: 1px solid #D8DFEA;border-bottom: 1px solid #D8DFEA;border-left: 1px solid #D8DFEA;margin: 0px;padding: 0px 0px 5px 0px;">

<div style="background: #EDEFF4;display: block;padding: 5px;">

<table cellspacing="0" cellpadding="0" border="0">
<tr>
<td valign="top"><img src="http://www.facebook.com/images/icons/fbpage.gif" alt=""/></td>
<td valign="top">
<p style="color: #808080;font-family: verdana;font-size: 11px;margin: 0px 0px 0px 0px;padding: 0px 8px 0px 8px;"><a href="http://www.facebook.com/people/Bruce-Thompson/100000637781892" title="Bruce Thompson" target="_TOP" style="color: #3B5998;font-family: verdana;font-size: 11px;font-weight: normal;margin: 0px;padding: 0px 0px 0px 0px;text-decoration: none;">Bruce Thompson</a> is a fan of</p>
</td>
</tr>
</table>
</div>

<div style="background: #FFFFFF;clear: both;display: block;margin: 0px;overflow: hidden;padding: 5px;">
<table cellspacing="0" cellpadding="0" border="0">
<tr>
<td valign="middle">
<a href="http://www.facebook.com/pages/Traditional-Hydrotherapy/111459758873296" title="Traditional Hydrotherapy" target="_TOP" style="border: 0px;color: #3B5998;font-family: verdana;font-size: 12px;font-weight: bold;margin: 0px;padding: 0px;text-decoration: none;">
<img src="http://www.facebook.com/profile/pic.php?oid=AAAAAwAgACAAAAAPnFSoNc8Trl6ORItfDTJhYKYveiENe4kqugsRcDaZtA5_f_aRwFLP_WeddB9Cb9kck-0FC-b94fEd8ed_T_OwQCrjqZ_8mijIMwVS7kGeT7K_tbEyj2S2RzD0BrqH-a2o&size=square" style="border: 0px;margin: 0px;padding: 0px;" alt="Traditional Hydrotherapy"/>
</a></td>
<td valign="middle" style="padding: 0px 8px 0px 8px;">
<a href="http://www.facebook.com/pages/Traditional-Hydrotherapy/111459758873296" title="Traditional Hydrotherapy" target="_TOP" style="border: 0px;color: #3B5998;font-family: verdana;font-size: 12px;font-weight: bold;margin: 0px;padding: 0px;text-decoration: none;">Traditional Hydrotherapy
</a></td>
</tr>
</table>
</div>
</div>
</div>
<!-- Facebook Fan Badge END -->

Which produces a badge that looks like this:

I thought the HTML was far too messy so decided to hack it and make an equivalent using CSS.

My CSS looks like this:

/*Facebook fan page badge */
#fbbadgetop{
width: 100%;
background: #3b5998;
padding: 5px;
bottom:40;
}
#fbbadgemid{
background: #edeff4;
display: block;
border-right: 1px solid #d8dfea;
border-bottom: 1px solid #d8dfea;
border-left: 1px solid #d8dfea;
margin: 0px;
padding: 0px 0px 5px 0px;
}
#fbbadgecont{
background: #ffffff;
clear: both;
display: block;
margin: 0px;
overflow: hidden;
padding: 5px;
}
#fbbadgecont a{
border: 0px;
color: #3b5998;
font-family: verdana;
font-size: 12px;
font-weight: bold;
margin: 0px;
padding: 0px;
text-decoration: none;
}
#fbbadgecont p{
padding-top: 5px;
}
.align-left{ float:left; margin: 0 15px 0 0; }

The HTML looks like this:

<div class="rightbox">
<div class="product_wrapper">
<p>Become a fan at...</p>
<div id="fbbadgetop">
<img src="../images/fb_logo_small.png">
</div>
<div id="fbbadgemid">
<div id="fbbadgecont">
<a href="http://www.facebook.com/pages/Traditional-Hydrotherapy/111459758873296" title="Traditional Hydrotherapy" target="_TOP"><img src="../images/fbtartan.jpeg" class="align-left"></a>
<p><a href="http://www.facebook.com/pages/Traditional-Hydrotherapy/111459758873296" title="Traditional Hydrotherapy" target="_TOP">Traditional Hydrotherapy</a></p>
</div>
</div>
</div>
</div>

And the new badge is:

I'm quite pleased with this but I'm sure that there are plenty of people who can make it even lighter. Or maybe I should just turn it into a png image...

Which would be lighter and faster? Please let me know.

And this whole exercise begs the question...WHY?

Well the simple answer is that I don't want the validator going off with every page I make. (and I love messing around with stuff).

HTML Finished!

Yep, I finished the most draining part so far yesterday morning.

Now I have just four files
Diseases.html   586,302 kb
Effects.html    138,709
Problems.html   753,148
Techniques.html 519,271
Total.........1,997,430

That's nearly 2 Mb... a whole lot of typing!

It was great to have this over. Pretty boring and repetitive, so will probably be riddled with mistakes.

Thanks Steve and other LOGIN folks for the suggestion of HTML Tidy to fix these mistakes. It sounds great but isn't in the Vector repositories. Maybe I'll package it up as I will have to do most of it to install anyway It looks like a fairly straight-forward thing, not too many dependencies.

Next steps in Traditional Hydrotherapy:
1. Getting a valid sample page up.
2. Installing (packaging?) HTML tidy
3. Working out how to quickly create web pages - no doubt it will be those Vim macros again.
4. Seeing how to integrate Vim and HTML tidy (I've got a Vim addon - never used one for Vim).
5. That' enough for now.

Vim Keyboard Macros for HTML Lists

Traditional Hydrotherapy is mainly lists. To complete marking up these lists I am using a few macros:

@r to replace the "/li" at the start of a list with "ul" on the first line of the list and thus start an unordered list. (From a previous post you will know that the dash (-) bulleted lists now have "</li><li>" instead of the dash.)

:s+/li+ul+^Mj

This is a simple substitution and I used "+" to delimit it as the usual "/" is used in the "from" string. It means:
:s+/li+ul+ - on this line substitute "ul" for "/li"
^Mj - <Enter> and move the cursor down a line

@e To end the unordered list, on the line after the last list item:

O</li></ul>^[j

O</li></ul> - open a line above the cursor, change to Insert mode and insert "</li></ul>"
^[j - <Escape> to Normal mode and move the cursor down a line, to where it was.

@b - for a one-line unordered list

@r@e

Simply a combination of the two previously shown macros.

The two macros, "@r" and "@e", were all I needed to finish marking up lists that had the list-items marked. But there are still lists without any markup, many of them the big Section indexes. For these unmarked lists I used "@e" from above and:

@u start an unordered list, called on the first item of the list:

I<ul><li>^[j

I<ul><li> - Open insert mode at the beginning of the line and insert the markup that starts an unordered list and the first list item
^[j - Escape back to normal mode and move down a line

@l To put "</li><li>" at the beginning of lines:

I</li><li>^[j

I can repeat this for all the remaining list item lines by typing, for instance "10@l" for a list with ten more list items.

@o - unordered list with a single list item. (start on line)

@u@e

This simply calls two previously mentioned macros, one starts the list the other ends it.

@d - for a two-list item unordered list (which I seem to have lots of):

@u@l@e

In the same vein as "@o" and "@b", above, this macro calls three previous macros

@c to markup comma separated lists. I'm dealing with a lot of these in Techniques.

f,2xi^M</li><li>^[

This finds the next comma, deletes it and the following space, start Insert mode and <Enter> to move the remainder of the line down one line and mark it up as a list item, then escape back to Normal mode

I have a question out of this, maybe you can answer. Is there any way of simply pushing the remainder of a line down onto a new line the way that <Enter> does in Insert mode?

And finally today, before I get back to work.

@m - To delete a number and fullstop (or dash and space), and put in regular list-item markup:

^2x@l

This moves the cursor to the first character of the line, deletes 2 characters and calls @l, the list-item macro.

HTML Markup with Vim Macros

So after the last post I had four files:

1. Diseases.txt
2. Effects.txt
3. Problems.txt
4. Techniques.txt

They had all their links marked up in html and all the bullet points converted to html list items. Now I had the daunting task of marking up the rest of the html.

To make markup easier I turned on Vim's syntax highlighting by simply changing the filenames to .html like this:

1. Diseases.html
2. Effects.html
3. Problems.html
4. Techniques.html

My present job is going through each of these files and manually marking up the html. Once again Vim macros have come to the fore. As I found previously the edits were quite slow on the bigger files, so split them into smaller ones. At present I am making good progress having completed Diseases, Effects and Problems in the last month and must be about halfway through Techniques.

So here is how I'm using the macros for general html tags:

@h - To make subheadings "H3"

I<h3>^[A</h3>^[j

I<h3>^[ - move to beginning of line, change to Insert mode, insert "<h3>" to begin the heading, then <Escape> to Normal mode
A</h3>^[j - start Insert mode after the last character on the line, Insert "</h3>" to end the heading, <Escape> to Normal mode and move the cursor down a line (to begin the next macro)

@p for a one-line paragraph

I<p>^[A</p>^[j

This is the same as "@h" above, except the paragraph markup is inserted. To ensure that paragraphs are one-line, I use "J", which appends the next line to the end of the present line with a space between.

@t - inserts treatment heading - often missing in Diseases

O<h3>Treatment:</h3>^[j

This opens a line above the cursor into Insert mode and insert the treatment heading then <Esc> back to Normal mode and move the cursor down a line.

I use other substitutions to make headings as well, like:

:%s+^Prevention:+<h3>Prevention<\/h3>+

and

:%s+Indications:+<h3>Indications:-</h3>+

And to get rid of redundancy in the links in the Diseases.html file:

:%s+../Diseases/++g