Chapter Three LLC

The Amazing Scene at Drupalcon Szeged

Matt Cheney

Drupalcon Szeged 2008 was the annual European gathering of some of the best Drupal minds of our generation with plenty of excitement and style thrown into the mix. For the event I flew across the pond to represent for Chapter Three and chase down some amazing vegan food throughout Central Europe with Drupal genius Neil Drumm. It was a wonderful time all around and worth the trek from San Francisco through Prague and Vienna to the town of Szeged in southern Hungary.

The conference was kicked off with Dries’ regular State of Drupal presentation where he dropped that there are over 20,000 Drupal 6 installations currently online and laid out his vision for Drupal 7 which included naming Angie “webchick” Byron to be co-maintainer of Drupal 7. Angie is a big fan of automatic testing and improved usability so wait for that to break out big in Drupal 7.

There were a couple great sessions by Sam Boyer about using Panels and using the Panels API where he laid out some of the existing power and the future direction of this very important module (including a Drupal 6 release in the coming weeks). I attended all the talks on Drupal security (a favorite topic of mine), including some particularly neat automatic security auditing approaches demonstrated by Barry Jaspan. The talk on running a Drupal consultancy by Tiffany Farriss (Palantir), Eric Gundersen (Development Seed), and Robert Scales (Raincity Studios) was fantastic and I strongly encourage anyone in this business to watch it a couple times.

Despite the serious time difference, but thanks to the hotel and conference internet I was able to keep up with the Chapter Three business and got to spend some quality time creating a Drupal 5 and Drupal 6 release for Drupal for Firebug, a module and Firefox extension to help with Drupal debugging and development. Rolled a couple of patches, got up to speed with the latest in Drupal 7, and made some additional improvements to Drupal for Firebug during the Code Sprint on the final day.

I met some amazing people at the conference (including a number of European developers I had only spoken with online) and left feeling really pumped and excited about everything Drupal. The future here is quite bright and I look forward to seeing everyone at Drupalcon North America next March.

Drupal Elegance: Not Just For Code Anymore

Matt Cheney

We all love the elegance the Drupal. The code style is clean, the architecture beautiful, and it’s all pretty well thought out. Plus the logo is pretty amazing.

To find a fitting tribute to our favorite content management system, I spoke with a friend from college, the very elegant Lilly Russo, who makes amazing glass mosaics through her business in Oakland, California.

Drupal for Firebug: Firebug talking Drupal

Matt Cheney


Firebug is a hell of a thing and at Chapter Three we use it regularly - as many others do - to analyze and debug our web development work. It can do things like inspect the HTML, manipulate the CSS, execute and debug Javascript, and monitor site performance and download times. It is so great that we wished it spoke Drupal and knew our name - kind of like the cute girl at the dance. Things get a little better when people know your name.

It is in that spirit that that Drupal for Firebug was developed as a way to extend the capabilities of Firebug by mixing in a little Drupal. Drupal for Firebug is both a FireFox Extension and a Drupal Module. To celebrate Drupalcon Szeged there are now releases for both Drupal 5 and Drupal 6 available. Get the extension and download the module, check our the test and demonstration site, and watch the drupal magic happen right in your Firebug window:

With a copy of the module and the Firefox extension installed, a developer can see debug messages, examine the SQL query log (with Devel module), look at how a Drupal page load is handling a site’s users, views, nodes, and forms, and even execute PHP. The goal of the module and extension is to create a framework for centralizing Drupal development and debugging work using power of Firebug. The major features include:

  • More centralization of web development debugging information by moving that information from Drupal page view into separate Firebug window. This helps to provide a clean separation of development debug information from the rest of the site.
  • Clean and external debugging message text and dumpable PHPs arrays and objects which can be displayed in the Firebug debugging window using the handy firep() function where they can be viewed without disrupting the rest of the page flow. There is also a handy “debug message only” search feature which is helpful in locating specific pieces of information in the debugging output.
  • Integration with the wonderful Devel module is available to provide a Firebug accessible SQL query log. Since the Firebug debugging window can be resized, hidden, or broken out as a separate window, it provides additional options (beyond appending the log to the bottom of the screen) for viewing the SQL query log.
  • Complete information about each user and node and view and form that is accessed during a page load is available. This can be invaluable in determining what Drupal items are being referenced and what $op is being performed on them. There is awareness of the original and modified state of the view and form items and even smart color highlighting to tell you when additions, changes, or deletions have happened to the item during the hook_form_alter or hook_views callbacks.

This is only the initial release of the module and there are plenty of tweaks and improvements to make down the road. The real secret sauce will be figuring out what are the most useful pieces of information to display in the Firebug window and making some important XUL interface improvements to the Firefox Extension to increase usability around the viewing and sorting of debugging data. Think HTML browser style functionality for the User, Form, Node, and View arrays and objects. But for now, at least it speaks our language.

Rolling Securely: 5 Tips to Protect Your Personal Data

Matt Cheney

We live in a very insecure digital world where it is very possible for those unsavory elements of our society to spy, sniff, inject, and otherwise interfere with the work we do on our computers.

In real terms the odds of anything particular bad happening to any of us while we build Drupal websites and sell awesome fixies is low, as a matter of best practices and general sanity here are a few things you can do to keep your local workstation or laptop a little more secure.


1.) Make sure to use a secure/encrypted session to the company email.

There are generally two ways you can access your webmail account (see below for examples from our company) and while they look and work the same way there are critical differences. If you use the normal “HTTP” connection, the emails you are sending and receiving can be snooped/hijacked by anyone on your same wifi or cable modem network. You must use “HTTPS” to have a secure/encrypted session to the email provider making it more difficult to snoop on your traffic.

HTTP - http://mail.google.com/a/chapterthreellc.com (bad - dont use this)
HTTPS - https://mail.google.com/a/chapterthreellc.com (good - use this)

2.) Be aware that IRC channels are often *not* secure (same goes for private messages through IRC)

The IRC network as a whole is not encrypted and conversations and passwords that flow through it can be easily snooped and recorded. This is true not only for people on your local network, but also for server operators and people with some smart masking/spoofing skills. IRC is often full of very shady people. In general, IRC should not be used to communicate sensitive information or server passwords.

3.) Securely communicate with your colleagues and clients using AIM *with* the OTR

The best way to discuss sensitive topics or share server/login credentials is through AIM (AOL Instant Messenger) using a plugin called OTR (Off the Record). If both users have this plugin, their communication is encrypted and cannot be read by anyone else. Without this plugin, anyone can read your messages and all data goes through the AOL main network to be relayed out….

Main OTR Page: http://www.cypherpunks.ca/otr/
Adium - http://www.adiumx.com/ (included by default)
iChat - http://chris.milbert.com/AIM_Encryption/#MacOTR
Pidgeon/GAIM - http://www.cypherpunks.ca/otr/README-pidgin-otr-3.2.0.txt and http://www.cypherpunks.ca/otr/binaries/windows/pidgin-otr-3.2.0-1.exe
Trillian - http://trillianotr.kittyfox.net/downloads.php

4.) Encrypt Your Harddrive

There are lots of pieces of sensitive data on your harddrive - from server passwords to saved emails and logged IRC/AIM chats - that should be protected. The best way to protect this data, of course, is to keep your laptop under your control, but if the laptop is stolen or misplaced bad things can happen. A very simple solution is to encrypt your user directory so others cannot read that data (even if they steal your computer).

OSX: http://www.youtube.com/watch?v=VQ9HQwGC_64
Windows: http://www.youtube.com/watch?v=coAMTC7uFxA or http://www.youtube.com/watch?v=-8YTaBVB7f0

5.) Consider Locking Your User Session When You Leave

Along the same lines as encrypting your harddrive, its helpful to also “lock” your computer session when you are not using it. This is less important in the office or a trusted work space, but if you have sketchy friends or are in a public place locking your computer will prevent others from stealing your data or pretending to be you.

OSX: http://technology.cca.edu/support/knowledgebase/index.php?article=24#mac
Windows: http://technology.cca.edu/support/knowledgebase/index.php?article=24#Win...

HOWTO: Upgrade an SVN Managed Drupal Installation (without CVS)

Matt Cheney

At Chapter Three we use the Subversion Version Control System to manage our client and internal Drupal projects. When we kick off a new project we roll out the latest version of Drupal, stick it in the SVN repository, and start developing.

As a matter of best practices it is usually a good idea to check Drupal directly out of the Drupal CVS repository which makes upgrading as simple as "cvs update -dP -r DRUPAL-5--X". However, we have a number of sites that were not checked out in this way and this makes updating Drupal sort of a pain. Our SVN management system wants us to update each file individually (so the changes can be versioned), but individually updating each of Drupal core's 300+ files is a little bit tedious.

However, with a little command line wizardry we can quickly download a copy of Drupal and generate a command to copy the new version of each Drupal core file on top of our installation in a way that can be committed up to SVN. To do this we need to wget a copy of Drupal and then use the all powerful find command to detect and copy the relevant files.

# Download a Copy of Drupal, Extract it, and Enter its Directory
wget http://ftp.drupal.org/files/projects/drupal-5.9.tar.gz
gzip -d drupal-5.9.tar.gz
tar xf drupal-5.9.tar
cd drupal-5.9

# Use Find to Detect Each Local File in Drupal Core and Execute a
# Command to Copy the File to the Live Version of the Website
find * -type f -exec echo cp {} /path/to/www/{} \;   # this just echos the  commands
find * -type f -exec cp {} /path/to/www/{} \;  # this actually runs the commands

To make sure the copy worked we can use the same find command do an individual diff on each file. This will assure us that we successfully updated our website with the version of code we downloaded from Drupal. This command can also be used prior to updating to compare different versions of Drupal to see what core changes are going to be (or have been) made.

find * -type f -exec diff {} /path/to/www/{} \;

HOWTO: Use TinyMCE in a Panel Pane Popup

Matt Cheney

The magic of Panels 2 is just the sort of thing you might expect from a wizard. Users can create pages with flexible and customizable layouts, they can populate those pages through an extensible block system, and they can configure and drag-and-drop those blocks around the page. Check out the demo page.

Sadly, the panels system does not allow its textareas to be used as TinyMCE WYSIWYG areas because of the way javascript and javascript events are handled. The basic problem is that TinyMCE runs on page load and is not set up to be activated on the Panel pane textareas that are created after the fact. This is a problem that hopefully the Drupal 6 version of Panels can resolve, but for Drupal 5.x the following hook_form_alter solution is possible.

First you need to set up a hook_form_alter callback as part of a new custom module or as part of an existing module you are modifying.

function my_module_form_alter($form_id, &$form) {
  switch($form_id) {

Second you need to render a hidden TinyMCE field on each panels editing page. This is not a particular clean way to proceed, but it will ensure that all of the appropriate TinyMCE .js is loaded on each page.

    case 'panels_edit_display':
      $form['tinymce_hidden'] = array(
        '#type' => 'fieldset',
        '#attributes' => array('style' => 'display: none'),
      );
      $form['tinymce_hidden']['tinymce_prerender'] = array(
        '#type' => 'textarea',
      );
      break;

Third you need to assign a special submit handler javascript call to convert all TinyMCE content back into its normal textarea content so it can be appropriately saved when the Panel pane is submitted.

     case 'panels_content_config_form':
       $form['next']['#attributes'] = array('onclick' => 'tinyMCE.triggerSave(true,true);');

Finally each of the textareas that are being rendered on the panel pane page need to have a special enable/disable TinyMCE link assigned to them. This allows users to turn on the TinyMCE functionality if they want. The code belows assumes your default TinyMCE state is off. I am sure there is a better and more generalized way to add these links, but this is a first step.

       // Load a disable or enable link below each textarea
       global $user;
       $enable  = t('enable rich-text');
       $disable = t('disable rich-text');
       $user = user_load(array('uid' => $user->uid));
       $profile = tinymce_user_get_profile($user);
       $status = tinymce_user_get_status($user, $profile);
       $link_text = $status == 'true' ? $disable : $enable;
       foreach($form['configuration'] as $index_raw => $value) {
         $index = str_replace('_','-', $index_raw);
         if ($value['#type'] == 'textarea') {
           $form['configuration'][$index_raw]['#description'] .= "<div><a href=
\"javascript:mceToggle('edit-configuration-$index', 'wysiwyg4-configuration-$index');\" class=\"wysiwyg-editor\" title=\"edit-configuration-\"" . $index . "\" id=\"wysiwyg4-configuration-$index\">$link_text</a></div>";
         } else {
           if (is_array($value)) {
             foreach($form['configuration'][$index_raw] as $index2_raw => $value2) {
               $index2 = str_replace('_', '-', $index2_raw);
               if ($value2['#type'] == 'textarea') {
                 $form['configuration'][$index_raw][$index2_raw]['#description'] .= "<div><a href=\"javascript:mceToggle('edit-configuration-$index-$index2', 'wysiwyg4-configuration-$index-$index2'); \"  class=\"wysiwyg-editor\" title=\"edit-configuration-\"" . $index . '-' . $index2 . "\" id=\"wysiwyg4-configuration-$index-$index2\">$link_text</a></div>";
               }
             }
           }
         }
       }
      break;

And you should be good to go.

  }
}

The Day You Dress Yourself: The New Chapter Three Website

Matt Cheney

There comes a time in the life of every young child when begin to dress yourself. No longer are you content with the dorky collared shirt your mother laid out for you each morning or the matching shoes you and your siblings wore because it was just easier that way. Instead, you yearn for a little class, a little flash, and for the older kids some san francisco style.

In this spirit, for our first year and a half of doing business, Chapter Three has rolled with a functional and utilitarian website design created by, well, Josh’s mother. A great graphic designer in her own right, Josh’s mother helped us with an initial design and it helped kick off our successful business.

However, with no disrespect to Jeremy Bentham or Josh’s mother, we recently put our collective heads together with the inspired graphic artist Monica Katzenell and are launching a new website. The new design is a little sleeker and gives a lot more information throughout the site about who we are as a company, the projects we have done, and the way we have participated in the Drupal community.


To make it all happen, on a technical level we make moderately liberal use of the Drupal block system and created several of our pages with the soon to be dominate Drupal content layout system called Panels module. Much of the image resizing is done through Imagecache and there is some neat integration with the Bio Module to generate our About Page listing and store biographical about everyone who works for the company.

Now that we are all dressed up, anyone want to dance?

Drupalcon Boston Recap: Bringing Guests to the Drupal External Data Party

Matt Cheney

This year’s Drupalcon was the biggest Drupal Party to date with over 800 attendees and many different sponsors (including Chapter Three of course). Drupal rocked Boston all week and gave everyone in the community a chance to catch up and keep abreast of some of the cool new stuff going on in the Drupal World. Plus, there was some scrumpcious and tasty vegetarian food on the scene.

In my Drupalcon rockstar moment, I took the stage and gave a session with Drupal Genius Neil Drumm and Digital Newspaper Extraordinaire Ken Rickard on Using External Data Sources with Drupal (slides here). The general idea is that Drupal is marvelous for content management, but becomes even better when it integrates with data from external sources and Drupal allows a lot of different ways to make that happen.

Ken kicked off our session by laying the groundwork for why using external data is important and transitioned into talking about some of the basic tools (drupal_execute, drupal_http_request, database switching) and techniques (lazy instantiation, hook_search) needed to make it all happen. He has a lot of good experience around using external data sources from his work in the newspaper world and provided a detailed orientation to the 150 odd people who attended our session.

Neil talked about his work with a wonderful organization called Map Light that records and analyzes information about campaign contributions and congressional votes. For that project Neil had to import *millions* of pieces of information and developed a couple modules to assist him in his work. Job Queue is his module that helps to import external data in batches (instead of all at once) and Import Manager allows administrators to monitor and manage the overall import process.

For my part, I concluded the session by discussing a few specific case examples of how to integrate external data into a Drupal website. I started with an example, which was dear to my heart as a librarian and admirer of philosophy, was to use the Swish-E Module (which I maintain on Drupal.org) to full text index a folder of philosophical essays in .PDF format and make them available via the Drupal search. The second example (see below) was probably the coolest and involved some data mining magic to use the Data Miner API to take a Drupal user’s MySpace name and automatically download their picture from MySpace for display on their user profile. This general technique I was calling “Data Enrichment” and could be used to enhance the data a Drupal site has around either users or specific pieces of node content. Careful consideration here needs to be made to respect both the terms of service and privacy of your users. The final example I presented had to do with some client work Chapter Three did involving the migration of 20,000+ pieces of raw HTML content through a cool audit system set up as several Views and managed by the Workflow System. This system seemed to have a lot of practical value for people and I got a lot of questions about it after the session.

All in all, Drupalcon was awesome and we are looking forward to Drupalcon Europe later this year and Drupalcon San Francisco in 2009.

example of using external data

HOWTO: Fully Theme and Customize the Drupal User Registration Form

Matt Cheney

Just a little to the left please. Flip it around. Put that on top of this. Call it by a different name. It is the little changes, that seem trivial and small, that often end up being real headaches to make and support our clients in making. Do we really want to try to build capacity with clients by teaching them to adjust #weight in hook_form_alter?

The Drupal Theming System is pretty powerful and, when done right, can offer a good avenue for our clients and their staff to edit, modify, and change their own website content. Its a lot easier to modify HTML files than Drupal module files.

A good example of where this kind of process is needed is on the user registration page. There are a lot of little bits of language and ordering to change and add, but to do so in Drupal module code can get a little hairy. Observe our technique to abstract the user/register form into a flat template file (while maintaing most of the other good Drupal goodness).

Step One: Create a theme override in your module code for the user/register form that executes a _phptemplate_callback to use a separate template file.

<?php
function theme_user_register($form) {
 
$vars = array();
 
$output _phptemplate_callback('user_registration_form', $vars);
 
$output .= drupal_render($form);
  return
$output;
}
?>

Step Two: Expand the theme override function made in step one to remove the titles and descriptions Drupal provides for the form elements. We do this in the theme function (instead of in a hook_form_alter) to preserve the original field titles so they can be used as part of any error messages coming out of form validation.

<?php
 
foreach($form as $key => $value) { // loop through top level
   
if (is_array($form[$key])) {
     
$form[$key]['#title'] = '';
     
$form[$key]['#description'] = '';
      foreach(
$form[$key] as $key2 => $value2) { // loop through second level
       
if (is_array($form[$key][$key2])) {
         
$form[$key][$key2]['#title'] = '';
         
$form[$key][$key2]['#description'] = '';
        }
      }
    }
  }
?>

Step Three: Create "rendered" versions of each of the form elements and set them as variables that can be passed to the template file.

Note: This can also be done with a generic foreach loop (similar to the one in step two) that renders each form element automatically.

<?php
 
// Set up the Vars Array
 
$vars = array();

 
// Render Specific Fields You Want on Your Registration Form
  // note - the specific location of the element in the form array varies
 
$vars['name_element'] = drupal_render($form['account']['name']);
 
$vars['mail_element'] = drupal_render($form['account']['mail']);
 
// continue for each field you want...

  // Don't Forget the Submit Button 
 
$vars['submit_button'] = drupal_render($form['submit']);

?>

Step Four: Create a template file in your site's theme directory to build the user/register form with the customized variables we defined in step three.

Note: This file needs to be the same name as specified in the _phptemplate_callback (example: user_registration_form.tpl.php).

<div class="user-register-element">
  <label>Enter a screen name:</label>
  <div class="user-register-element-input">
    <?php print $name_element; ?>
  </div>
  <div class="user-register-element-description">
    Screen names can be up to 13 characters in length.
  </div>
</div>

<div class="user-register-element">
  <label>Enter an Email:</label>
  <div class="user-register-element-input">
    <?php print $mail_element; ?>
  </div>
  <div class="user-register-element-description">
    Emails must be valid.
  </div>
</div>

<?php // continue on for each rendered form element ?>

The drupal magic here is that the user registration form is now uniquely customizable by anyone who can edit the theme template. This allows for customized "prompts" for each profile field element, without changing the site-wide field name in admin/user/profile, and it allows for customization of the username and email titles and descriptions.

This technique will need to be modified to support external modules that modify the user/register form like LoginToboggan. It also needs to take into account things like "required" fieldstates.

Drupal Administration: Keeping it Simple

Matt Cheney

Much like the parent who gives their child the keys to the family car, there comes a time in our client engagements where we set up administrative accounts for our clients to begin managing their own content. At Chapter Three we call this Capacity Building and try to make the process as smooth and intuitive as possible. The problem is the default Drupal administration interface makes this difficult.

In general, the Drupal administration navigation menu (see right) is simply too complex to be really intuitive. There are all sorts of options (read noise) available that have little relevance to the particular administration needs of our clients. Some of this menu can be distilled down through permissioning, but with general permission categories in Drupal like “access administration pages” its not possible to remove all the options. Plus, many of the most relevant options are buried several layers deep in the navigation. Its unlikely our clients will regularly find them on their own.

In the past we have build a static “administration page” where we listed the major administration pages and provided links to each, but with our latest project (that had a number of different editors and content managers who all needed to update the site) we wanted to improve our business practices and give our clients a better way to manage their website. The solution we came up with was to provide a constantly present top menu that would only appear for site administrators and would allow them easy access to their specific administration options in a nice drop down menu. We populated the menu through the Drupal menu system with the 20 or so administration pages that were needed to manage the site. This menu display was accomplished by using the SimpleMenu module and can be seen below.

To manage the individual pieces of content on the site we built one, general purpose “Content Management View” using the wonderful Views module. The idea here is to provide a consistent UI to manage each type of content and to allow site administrators to filter, search, and sort that content for their editorial and review purposes. Setting this interface up as a View allows us a lot of flexibility later to add additional administrative options and support different content types.

The end goal is to make it as easy as possible for our clients to easily work with the Drupal CMS and to manage their specific content. This kind of system is something we intend to refine and deploy in future sites.

Syndicate content