Jack Sleight's Blog

Firefox 3's Buggy Text Rendering

Tue, 24 Jun 2008 11:00:00 +0000

With the recent release of Firefox 3 I’ve found that the way certain elements are rendered has changed, and in some situations not necessarily for the better. This is obviously due to the new version of the Cairo rendering engine, and no doubt, these changes are all supposed to be improvements (and mostly are). However, I’ve run into one problem with text rendering in some specific situations. Take a look at this:

That’s Arial set at 1.35em (from a global font size of 10px) on Windows with ClearType enabled. I know 13.5px isn’t a round font size, but that’s not to blame here, really that’s just 14px. Set it to 1.3em, you get 13px text, set it to 1.4em and it looks the same but with slightly larger leading.

The new rendering engine is supposed to optimise the text for legibility; I’d argue that that is not legible. Thankfully, Firefox 3 now supports the text-rendering CSS property, which we can use to set the rendering mode to speed optimised, which actually gives much more legible text (in this situation, in my opinion). This gives us something much closer to the Firefox 2 rendering (if not identical).

text-rendering: optimizeSpeed; /* The default is optimizeLegibility */

Which gives us:

As you can see, much better. I’m not discounting the improvements made in the new rendering engine, but I do feel that it could still do with some work to iron out these rare, but pretty annoying issues. That can’t be how the text is supposed render, right?

JS_Extractor 0.1.1

Sat, 08 Mar 2008 12:00:00 +0000

Just a quick note to let you know I’ve just released a new version of JS_Extractor. This release fixes a bug related to extracting attribute values from elements in a hierarchy.

Download from the Code page

Ready For Review

Sat, 23 Feb 2008 17:00:00 +0000

I’m pleased to announce that the two Zend Framework proposals I’ve been working on, Zend_Color and Zend_Db_Table_Plugin (along with Simon Mundy) are now complete and ready for community and team review. Even if you don’t use the Zend Framework, the Zend_Color component can be used very easily standalone, and is intended to be a replacement for my previous Colour Tools class, so check it out.

Full details can be found on the individual proposal pages, preview code downloads for Zend_Color can be found on the code page.

JS_Extractor! And the death of Table Extractor

Sun, 10 Feb 2008 12:00:00 +0000

So, it’s been a long time since I wrote (or even looked at) Table Extractor, and almost as soon as I wrote it I knew there were a lot of problems. For a start:

It only worked with tables
It didn’t really do that properly, or at least reliably
It was a horrible mess of hacky code designed to workaround hacky HTML

It was written for PHP 4, pah! Seriously, no one can still be using that, can they?

Despite all of these problems, it was surprisingly popular, and I still regularly get emails asking how to use it, suggesting new features or reporting bugs, which although I appreciate the time people have taken, I just couldn’t do anything about.

Anyway, that’s in the past, and today I’m releasing the first (beta) version of JS_Extractor, a brand new, completely reworked in every conceivable way, class library, designed for extracting data from HTML documents. And when I say data, I mean any data, not just tables.

Before I get into the examples I want to explain the new approach I’ve taken, and the various aspects of the new extractor. If you don’t care about that and just want to get your hands dirty then head down to the examples, but don’t complain to me if you don’t get it.

DOM Extension and XPath

JS_Extractor is actually an extension of the PHP DOM extension, and you can therefore use all of the DOM methods with any JS_Extractor or JS_Extractor_Element object. If you don’t know or have never used the DOM extension I seriously suggest you take a quick look over the documentation, just so you’re aware of what’s possible.

The second important aspect of JS_Extractor is that it uses XPath, a lot, and adds one very useful method to the vanilla DOM classes, query(). This method is really nothing more than a wrapper for a new DOMXPath object, but makes it much more convenient to run XPath queries on an element. For example, rather than this:

$xpath = new DOMXPath($doc);
$nodes = $xpath->query($expression, $element);

You can do this:

$nodes = $element->query($expression);

Which also allows you to easily chain queries together. (Yes, I stole this idea from SimpleXML.)

I’m not going to cover the details of XPath here, so if you’re not familiar with it already take a look a the syntax guide from W3Schools (good for beginners) or the full XPath spec. You should find the examples below fairly self explanatory though.

The DOM extensions ability to parse (even dirty) HTML, XPath support and new query() method are the heart of JS_Extractor, and give you a lot of power even without the specific “extractor” methods. For example, you could get every link on the page and then echo the href attribute values as simply as:

$extractor = new JS_Extractor(file_get_contents('sample.html'));
foreach ($extractor->query("//a") as $link) {
	echo $link->getAttribute('href');
}

Easy! Of course, the additional extractor methods make this even easier, and make other, more complicated problems easy as well.

Utility Methods

There are also methods available for “tidying up” the data before you extract it. There’s actually only one of these right now, and that’s splitCells(), which applies to table, thead, tfoot, and tbody elements. This method will scan through the table cells and split any with a colspan or rowspan attribute, duplicating the content for each. This is essential for retrieving tabular data in a simple two dimensional structure.

`extract()`

This does all the magic. Actually, it’s really nothing more than a convenience, you can do everything this does with the standard DOM methods, but why make things more complicated than they need to be? This is primarily aimed at extracting the text within elements, in a hierarchical structure you define, or a specific attribute from a number of elements. This covers the most common uses, and anything more complicated can be achieved with the DOM and query() methods.

Examples

All of these examples are run on this sample data.

Before you start you’ll need to add the library path to your PHP include_path and then include the Extractor.php class file, by doing something like this:

set_include_path(get_include_path() . PATH_SEPARATOR . './library/');
require_once 'JS/Extractor.php';

Right, first we need to create the extractor object. The constructor requires a string of HTML, so use file_get_contents(), or another function to retrieve the contents of a local or remote file if you need to:

$extractor = new JS_Extractor($html); // or
$extractor = new JS_Extractor(file_get_contents('sample.html')); // or
$extractor = new JS_Extractor(file_get_contents('http://example.com/'));

Next I’m going to retrieve the body element of the document, this is necessary due to the way the extension of the DOM classes work. The utility and extract() methods are not available on the JS_Extractor object, only JS_Extractor_Element objects.

$body = $extractor->query("body")->item(0);

As you can see, the query() method returns a DOMNodeList rather than a single element, therefore if you only need one element you have to call the item() method.

Tables

Now we grab the first table in the body, like this:

$table = $body->query("//table")->item(0);

As far as selecting the right table goes, you’re only limited to what you can do with XPath, which is pretty much anything. I’ll cover some more examples on selecting elements later on.

Now before we start extracting data from this table we need to clean it up, because this table has colspans and rowspans, which need splitting and duplicating in order to create a simple two dimensional structure, this is as easy as:

$table->splitCells();

Right, let’s extract all the cell data from the rows in the tbody, grouped by row:

$data = $table->extract(array("tbody/tr", "td"));

As you can see, the first argument of the extract() method is an array of XPath expressions. These define the hierarchical structure of the array that is returned. What you’re saying here is: get all the tr elements from the tbody element, and then get the text from all the td elements within those. This will return something like:

array
  0 => 
    array
      0 => string 'A' (length=1)
      1 => string 'A' (length=1)
      2 => string 'A' (length=1)
      3 => string 'A' (length=1)
  1 => 
    array
      0 => string 'B' (length=1)
      1 => string 'A' (length=1)
      2 => string 'A' (length=1)
      3 => string 'B' (length=1)
  2 => 
    array
      0 => string 'C' (length=1)
      1 => string 'C' (length=1)
      2 => string 'C' (length=1)
      3 => string 'C' (length=1)

Great! Now how about all trs rather than just the ones in the tbody? You could do:

$data = $table->extract(array(".//tr", "td"));

But you’ll run into a problem, because the thead contains th elements rather than tds, so instead we need to do:

$data = $table->extract(array(".//tr", "th|td"));

But now we have another problem. We have all the rows, but no idea which ones came from the thead, tbody and tfoot, which is kinda important. The first thing we need to do is separate the tr expression into individual parts for the thead, tbody and tfoot, like so:

$data = $table->extract(array(
	array("thead/tr", "tbody/tr", "tfoot/tr"),
	"th|td",
));

Then we need to name these parts so that when the array comes back the rows are grouped into the right section:

$data = $table->extract(array(
	array('head' => "thead/tr", 'foot' => "tfoot/tr", 'body' => "tbody/tr"),
	"th|td",
));

This will give you something like:

array
  'head' => 
    array
      0 => 
        array
          0 => string 'H' (length=1)
          1 => string 'H' (length=1)
          2 => string 'H' (length=1)
          3 => string 'H' (length=1)
  'foot' => 
    array
      0 => 
        array
          0 => string 'F' (length=1)
          1 => string 'F' (length=1)
          2 => string 'F' (length=1)
          3 => string 'F' (length=1)
  'body' => 
    array
      0 => 
        array
          0 => string 'A' (length=1)
          1 => string 'A' (length=1)
          2 => string 'A' (length=1)
          3 => string 'A' (length=1)
      1 => 
        array
          0 => string 'B' (length=1)
          1 => string 'A' (length=1)
          2 => string 'A' (length=1)
          3 => string 'B' (length=1)
      2 => 
        array
          0 => string 'C' (length=1)
          1 => string 'C' (length=1)
          2 => string 'C' (length=1)
          3 => string 'C' (length=1)

And that’s it! Of course this is in no way limited to tables, and below are some further examples of extracting data from other elements:

Lists (`ul`, `ol`)

Here we get the ul element with the id “list”, and then extract the text from each li:

$list = $body->query("//ul[@id='list']")->item(0);
$data = $list->extract("li");

Custom Markup

You can even extract data from custom markup based on divs and spans (or any other element type):

$data = $body->extract(array(
	"div[@class='article']",
	array('title' => "h2", 'date' => "span[@class='date']", 'body' => "p"),
));

What you’re doing here is getting all the div elements with a class of “article”, and then extracting the title, date and body text from the relevant elements.

Attribute Data

The attribute extraction method returns the value of a specified attribute, rather than the elements text content. Although not demonstrated here, the hierarchical structure feature works in exactly the same way with the attribute extraction. Here we’re going to get all the href values of all a elements in the body:

$urls = $body->extract(".//a", JS_Extractor::EXTRACT_ATTRIBUTE, 'href');

Or alternatively, all href values of any element in the body that has an href attribute:

$urls = $body->extract(".//*[@href]", JS_Extractor::EXTRACT_ATTRIBUTE, 'href');

That covers everything I wanted to demonstrate today. I plan to post further, more specific examples in the future, so if you have any requests let me know.

Download

Please bear in mind that this is a beta version, and therefore the API may change in future releases.

Download from the Code page

I will also be posting the full API documentation in the near future.

What about Table Extractor?

Well, this post marks the death of Table Extractor, anyone using it should start using JS_Extractor. If you’re on PHP 4 then upgrade, seriously, there’s no good reason not to, and many reasons to do so. If you don’t have the DOM extension enabled then enable it, I mean come on, it comes with PHP and is enabled by default anyway.

My HTML 5 Favourites

Thu, 24 Jan 2008 12:00:00 +0000

With the W3C publishing the latest HTML 5 working draft and supporting differences from HTML 4 document, I thought I’d list some of my favourite additions and changes:

New Elements

figure to associate a caption with some content, such as an image.
audio and video can be used for multimedia content.
canvas can be used for rendering dynamic graphics such as graphs.
datagrid for interactive tabular data.

The input element now has some new types: datetime, datetime-local, date, month, week, time, number, range, email and url.

New Attributes

The a and area elements have a new attribute called ping that specifies a list of URIs which have to be pinged when the hyperlink is followed.
The new autofocus attribute can be specified to focus a form control during page load.
The new form attribute allows for controls to be associated with multiple forms.
The new required attribute on form controls indicates that the user has to fill in a value in order to submit the form.

The input element has several new attributes to specify constraints: autocomplete, min, max, pattern and step.

These are just the specific bits I’m most interested in, take a look at the full HTML 5 differences from HTML 4 document for the full list of new features/changes, in more detail.

Of course, none of this matters for now, as it’s going to be a long time before we get to use any HTML 5 goodness in real applications and sites, but it’s good to see things moving forward.

X-UA-Compatible in the Future?

Thu, 24 Jan 2008 11:00:00 +0000

After all the talk about the IE8 “super standards mode” (something that should really be called “the standard, normal, default mode”) opt-in, I’ve put down my thoughts in the comments on Eric Myer’s latest blog post. The gist being that if we have to have it at all, it should be a one off, short term solution specifically for the jump from IE7 to IE8.

IE Floated Columns Totaling 100% Bug

Sun, 20 Jan 2008 15:00:00 +0000

I feel I’ve been blogging a lot about IE bugs so far, which I guess I have; but then, I wouldn’t have to if there weren’t so many! Anyway, this particular bug has no doubt been worked around in many of the column based layout templates available on the web, it may have even been worked around in this exact way, but I didn’t find it. And when specifically Googling for it nothing of any use came up, so hopefully this should help some people out.

If you’ve ever tried to create a liquid column layout by floating a set of divs next to each other, with a total width of 100%, then you’ve no doubt run into this problem, where IE calculates the total width as slightly more than 100%, and ends up wrapping the last column below the others. ~~I assume it’s some kind of rounding error.~~ It’s due to the way IE rounds sub-pixel widths.

See Problem (if it looks right, try adjusting the browser window width)

It’s a pain, but I’ve found a way to fix it using negative margins (again); I’m staring to think negative margins are IE’s best friend. The fix is simple, just apply a negative right margin to the last column, like so:

#right	{ margin-right: -2px; }

The actual number of pixels you should set it to varies depending on the number of columns you have, but I find a reliable rule to go by is equal to or greater than half the total number of columns. So if you have three columns, set it to two, if you have ten, set it to five, and so on.

This accommodates for the additional width and stops the last column wrapping. It also has no absolutely no effect visually (the last column remains the same width, and stays in the same position). ~~I tried this with 1 pixel but that wasn’t quite enough to fix it completely. I’ve had no wrapping whatsoever with 2 pixels.~~

See Solution

This problem exists in both IE6 and IE7, this solution works in both.

Update: John Resig has posted an excellent article describing what causes this and the problems browsers have in calculating the correct widths. I’ve also learnt that this can’t technically be considered an IE “bug”, as the behaviour in this situation isn’t actually standardized. Of course, that doesn’t mean IE’s behaviour is right.

IE7 Dotted Borders: Close, But Not Close Enough

Wed, 16 Jan 2008 21:00:00 +0000

As if we needed any more IE bugs, I’ve just run into another. Although dotted borders were supposedly fixed in IE7, it turns out that they still render as dashed if applied to a fieldset with a legend (which is of course required). There’s no workaround as far as I know.

Example

Three IE Form CSS Problems & Solutions

Tue, 15 Jan 2008 11:00:00 +0000

I was recently writing some CSS for a form, and ran into a few IE problems/bugs which I’ve either not found documented elsewhere, or at least not found my solutions mentioned, so I figured it would be beneficial to post them here.

Checkbox/Radio Margins

I’m not entirely sure if this should be considered a bug, or if it’s just simply the way IE likes to render these elements, but as you may well know, both of these elements appear to have a three or four pixel margin around them, but it can’t be removed with margin: 0;. There are a few suggestions around for fixing this, but I didn’t find any using negative margins, which as it happens, work perfectly, like so:

input.checkbox, 
input.radio	{ margin: -4px -3px -3px -4px; }

See Problem, See Solution

This problem exists in both IE6 and IE7, and this solution works in both.

Considerations

Remeber that reducing the margins removes the clickable white-space around the element, potentially reducing usability.

Legend Left Margin/Padding

Again, bug or feature, who knows? But by default you get a seven pixel gap on the left hand side of legend element text (in addition to the fieldset left padding), which can’t be removed by setting the margin or padding to zero. However, negative margins to the rescue again and we can fix that:

legend	{ margin-left: -7px; }

See Problem, See Solution

This problem exists in both IE6 and IE7, and this solution works in both.

Fieldset `border-top: 0;` and Legend `display: none;`

This is a fairly specific problem that most people probably never run into, but I had a couple of fieldsets which I wanted to hide the legends for and ran into a problem. When you have a fieldset with borders, but no top border, and the legend set to display none you still get a top border. Make the legend visible and no problem, the border goes away. The only way I found to fix this was by setting the border colour to the same as the fieldsets background colour, which is in no way ideal.

See Problem, See Solution

This problem exists in both IE6 and IE7, and this solution works in both.

Archive & Tags Added

Mon, 14 Jan 2008 17:00:00 +0000

So, it took me less than a day to realise I needed a better archiving/category system for the blog. So I’ve added one. You can now view posts by tag (via the blog home page) or through the archive via the date links under each post title (or by just guessing the URL if you really want).

Jack Sleight's Blog

Firefox 3's Buggy Text Rendering

JS_Extractor 0.1.1

Ready For Review

JS_Extractor! And the death of Table Extractor

DOM Extension and XPath

Utility Methods

extract()

Examples

Tables

Lists (ul, ol)

Custom Markup

Attribute Data

Download

What about Table Extractor?

My HTML 5 Favourites

New Elements

New Attributes

X-UA-Compatible in the Future?

IE Floated Columns Totaling 100% Bug

IE7 Dotted Borders: Close, But Not Close Enough

Three IE Form CSS Problems & Solutions

Checkbox/Radio Margins

Considerations

Legend Left Margin/Padding

Fieldset border-top: 0; and Legend display: none;

Archive & Tags Added

`extract()`

Lists (`ul`, `ol`)

Fieldset `border-top: 0;` and Legend `display: none;`