The Website RFP & The Impossible-to-Estimate GI Joe Line Item
The following article was originally published here at drud.com.
My bold claims:
Every feature within a website RFP can be delivered with a low, medium, and high level of effort (LOE) solution.
While the difference between low and high can be measured in single or double digit percentage (e.g., 1-99%), the difference can sometimes span several orders of magnitude (10X to 10,000X).
One or two of these massive discrepancies can kill the budget and, as a result, the entire project.
We can learn a lot by reviewing GI Joe, the line item scenario.
Why Do I Believe This?
I’ve spent over a decade in the web development space working on the project budgets ranging from the very small (circa $1,000) to the somewhat large (circa $1,000,000 account). I’ve also worked across all stages of the project, ranging from pre-contract as a sales engineer to post-delivery for ongoing iteration and improvement. During this time I’ve experienced (much to my joy) spectacular successes. I have also experienced (much to my dismay) complete and utter failures where the delivery of a 30K project was 4-5X over budget.
Oops.
When these things happen (and they will happen), there is a lot of focus and blame on the estimation process. Unfortunately, I’ve observed that most people and teams spend time in the wrong places. Too much time is spent trying to gain an additional 10-20% of precision when the real issue is they have completely missed an important detail that moves the level of effort up or down a level of magnitude. This is particularly true in web development. Vesa Palmu, the former CEO of a 200+ member web development agency, once stated that “With Drupal features are cheap and details expensive.” This quote has struck a chord with me the second I heard it because it mirrors a truism that I’ve seen again and again: that a single feature can span between 5 minutes and 1,000 hours. You may be skeptical, and that’s fine. I’ll illustrate this with analogies and prove it with some real-world examples from the field.
Author’s Note: This is ONE expanded aspect of my talk: Estimates, Expectations, and Evolution During a Project's Journey from RFP to Release, which I will be presenting on at Drupalcon Nashville. The recorded presentation and slide deck will be linked here.
Let’s begin!
It Started Off So Innocently…
The first pass through an RFP reads so innocently. While I don’t discount the value of missions, visions, goals, and dreams, the real work starts once you get to the subheading marked “Requirements,” which is often represented by a bulleted list. If you’re lucky, you’ll get a description that is at least a couple of sentences long for additional detail and context.
GI Joe, The Line Item Hero
However, imagine coming across a line-item request that containing two simple words: “GI Joe.” Now, of course, this is a website RFP, but let’s further imagine the client wanted an actual, physical action figure as part of their deliverable. At first, you might scoff, but this client seems serious about this request because they did include a graphic and stated that it would go along with the launch of their online 80s nostalgia store.
With no additional information, what would you even guess as a starting point? 5 minutes? 5 hours? Let’s see just how extreme we can make the overall LOE.
99.9% of us might start with the expectation and assumption that the client merely wants an off-the-shelf GI Joe. However, anyone who has built a website has realized that assumptions are not always correct.
The 5-Minute Solution
At first, you might think, “This is easy! I’ll go to Amazon.com, find the first action figure that looks GI Joe-ish, and Amazon Prime two-day ship that stupid thing to their office.” The longest part is filling out the shipping address for the client’s office.
The 1-Hour Solution
The client receives the package but then says “Welllllll… I was looking for a specific outfit, one that matches our company colors.” You think, no problem! Your resourcefulness kicks into high gear, you find a second GI Joe of the same height but with an outfit that kinda/sort of matches the style on the RFP letterhead. You then ship both action figures to your office, where you then swap clothes, repackage, and ship to the client.
The 10-Hour Solution
You then receive a message from the client. “Welllllll… this solution is certainly better, but it needs to match our branding guidelines exactly. See the attached PDF with exact Pantone colors.” Again, no sweat… maybe. You print out the PDF hoping that you did a proper job of color syncing before you high tail it to the nearby home depot. There you do your damndest to try, and color match the pain for the shirt and pants. An hour later you are Google searching while trying to Craigslist or Task Rabbit a designer that can apply the paint in a way that doesn’t look like a four-year-old did it. By the end of the day, the GI Joe with somewhat dried paint found it back into a package and sent over to the client.
The 100-Hour Solution
The client calls, squeezing with delight! “It’s almost perfect!” Almost? He then goes on to tell you that he forgot one critical bit, that the action figure has to be two inches taller to exactly match the other six that will be featured at the retail stores grand opening the same day the website is launched. Your heart sinks. So do you get out a blowtorch and melt the plastic? Or do you attempt to get a Stretch Armstrong doll and get it the right size before trying to freeze the body in place?
Then inspiration hits. 3D printing! Of course, you’re back on Google, but this time searching for a contractor with Autocad skills. You’re also searching for a more talented artist that can paint and match the skin tone. Additionally, you hope/pray this same person can sew (from raw materials) the shirt and pants to match the size.
At this point, this one feature is almost a third of the 300-hour budget, but you’ll try and skimp and save on some of the development tasks.
The 1,000-Hour Solution
The client hears your story of how you went above and beyond for them. They are impressed and eager to see the new deliverable. TADA! It looks very, very impressive… but something is not quite right. Yes, you fixed the height issue, but the 3D printer (using a low-end model to keep costs down) has left the surface with a somewhat grainy texture. And the skin color is not only off, but it’s literally flaking off because it didn’t bind to the plastic.
The client tries not to hurt your feelings, but they firmly insist that they need
The body proportions, look, and feel of the 5-minute solution.
The clothing style of the 10-hour solution.
The colors of the 100-hour solution.
The height of the 1,000-hour solution.
And while you’re at it (NEW IDEA), can you change the gender? And the owner is heavy into metaphysics, so can you also insert a rose quartz heart into the center even though nobody else can see it (but maybe they can sense it)?
The 10,000 to 100,000 Solution
If the project hasn’t gone to a legal dispute by this point, it’s a bloody miracle. Still, the engineering mind will still try and solve problems that are far beyond what any sane budget would allow. Maybe you go directly to the manufacturing plant and try to create a new, one-off mold. Maybe this gets you back down to 100 or 1,000 hours (depending on how fast they can retool their machines), or maybe it opens yet another can of worms.
The Aftermath
Ultimately, a $10 GI Joe with small yet “must have” changes can cost $10,000 to $100,000 to produce. And in the end, would anyone at the grand opening even notice or care about the small differences in size, color, and outfit?
It’s not the client's fault (usually). It’s not always easy or obvious for them to know when a “simple” change request will do this because the outward changes appear too trivial and they mask all the behind-the-scenes heroics that (sometimes) need to occur.
It’s also not the manufacturer's fault that they didn’t (or can’t, or won’t) make every conceivable version of a GI Joe action figure. In their business, it’s a game of margins, and they know that a generic, mass-produced version will meet 90% and this will maximize ROI between the company (profit) and the people that collect them (happiness).
There are many parallels to draw in the Web CMS space. Solutions like WordPress, Drupal, Magento, etc. provide a ton of value out of the box, but seemingly reasonable requests can push the project to require a significant amount of customization that can kill a budget and therefore the project itself. Let’s explore.
eCommerce Functionality
In Drupal feature are cheap, and details are expensive. - Vemu
This example hits close to home because it includes several actual project outcomes woven together as a singular example. It is also a cautionary tale because clients (usually) try to both squeeze in features while squeezing down the budget. When this happens, we can begin to rationalize how the project is still possible if more features are moved into the “low” end of the estimate. This rationalization is a trap because clients do not always fully realize how and where a change will move them up or down an order of magnitude.
One example that I used for years was a Drupal 7 site with Drupal Commerce. The requirement included a store that had coupons and sales tax. Both features by themselves were trivial and could be installed within 1-2 hours each. The challenge occurred when combining these modules. They were both attempting to apply to the original price instead of applying the tax after the discount. Nobody had solved this exact problem in the contributed module space, so we had to write the custom code. One week and 30+ billable hours later, we had a solution.
Could we have predicted that 10X overage? Yes and no. One of the things I love about Web CMS Frameworks is that they represent 100,000s of hours of open source contributions that we can all download and use for free. That’s amazing! And out of the box, they can deliver a ton of functionality. They are also highly configurable to meet a wide range of use cases, although that is just the beginning of where time/effort starts to ramp up. Eventually, you hit a point where you can’t do something directly from the admin interface and need to start writing custom code. Compared to configuration changes, custom code requires a higher degree of skill and significantly more time. Even then, there is pluggable custom code and built-from-scratch custom code. The latter is where the LOE ramps up even more.
While the following example is using Drupal and Magento, it probably applies equally to WordPress, Joomla, TYPO3, etc. Let’s begin…
eCommerce: The Quick Snapshot
A client wanting eCommerce could span these use cases.
~5 Minutes: Paypal Buttons
~1 Hour: Out of the box Shopify
~10 Hours: Custom Drupal Commerce Kickstart
~100 Hours: Custom Drupal Commerce
~1,000 Hours: Full-blown Drupal Commerce/Magento Experience w/ERP Integrations
~10,000-100,000 AND BEYOND!
Let’s break this down in more detail.
~5 Minutes: Paypal Buttons
The client has 1-2 products with an expectation of selling only a few a month. Simply creating and dropping a Buy Now button and pasting the snippet in a WYSIWYG Editor is enough to get the job done.
~1 Hour: Out of the box Shopify
Here the client may have a dozen products representing a few sales a month and is still looking for something that’s a step up from PayPal Buttons but quick and turnkey.
~10 Hours: Custom Commerce Kickstart
Here the client loves what the Commerce Kickstart distribution provides out the box, but needs a bit of time configuring the right modules to meet the needs of the client. The client is willing to use the default theme across the site.
~100 Hours: Custom Drupal Commerce
This falls into the bucket of a “standard” Drupal Commerce site in the sense that the customer has some reasonable requests that will take additional time and care. Multiple features need to be enabled and tested together to ensure compatibility. A custom site theme needs to be created to match the organization branding guidelines. Some existing content and orders must be migrated over. There may be some off-the-shelf API integrations, such as the ability to sign up for a newsletter or push a lead into a CRM.
~1000 Hours: Full-blown Commerce Experience w/ERP Integrations
This is where things escalate. To be fair, when you’re at this level, there should be an incredible amount of detail about the specific needs of the project to warn you that you are likely to meet/exceed this threshold. However, this is not always the case. I still remember the punch-in-the-gut feeling I had when a customer’s “inventory management” feature was a full-blown ERP integration with real-time inventory management, customer-specific pricing, warehouse allocation management, pick lists, etc. We later learned that over a million dollars was spent over several years and it took a full-time developer to maintain and operate it.
~10,000+ AND BEYOND!
Piggybacking off the last example, there was one situation where a client was trying to effectively replace their ERP with Drupal without an API integration. Even standing up a ad-hoc API would have cost an obscene amount of money and the retooling costs across 100 end-users made this project dead on arrival. Ultimately, Drupal was not the right approach here, and trying to do otherwise would have cost north of a million dollars. The project ended early before we ultimately spent the time and budget it would have taken.
How to Blow a Website Budget
Earlier I claimed that a single 10X feature could be enough to derail the entire project. Here’s a simple calculation to demonstrate this. Suppose you had a site that had 10 key features each requiring about 10 hours of time for a total of 100 hours. If one of those features goes 10X, you’ve effectively doubled your budget because 9*10 + 100 = 190, which is approximately 2X the original budget of 100.
Can’t happen? In each of the smell tests below, I have at least one project example where a single feature caused a 50% to 100% increase in the overall budget. Thankfully, some of those were not losses to the company because we had to have some difficult but productive conversations with our clients. Still, we weren’t always so lucky, and we sometimes had to eat the loss.
Smell Tests for Website Customizations
When you see these words or phrases in an RFP, you are potentially pushing deep into customization territory.
“API”: While some services connect with WordPress and Drupal out of the box (Google Analytics is a popular plugin that can just drop in), many are non-standard, and ANY deviation from standard behavior can be problematic.
Integrations: Anytime you need to connect with additional systems, particularly those that are using completely different technology stacks altogether with no known or publicly documented API, you are likely going to spend several hours/days/weeks just getting a proof of concept going.
Extensive Requirements: If a client goes into a page of detail for a single feature, chances are there are strict requirements as well as data not yet surfaced. I encourage to try and simplify if the client is willing and able to make said simplifications but beware if they come back and expanded.
Migrations: Sites involving only 5-10 pages of content are trivial to migrate. Sites that have 1,000s of records across multiple content types with multiple fields and quickly absorb a ton of time setting up the proper translations. Additionally, you may have dirty or incomplete data that needs to be filtered and processes to be compatible with the new data model.
Products: In the simplest case, all products have a title, SKU, image, description, and price. Ultimately, products can become way more complicated than this and have dozens if not 100+ metadata fields per type.
Mobile Apps: While CMSes like Drupal 8 exposes REST APIs to provide a means to communicate with other applications and devices, the second you add a non-trivial mobile app on top of a web app you are already adding a decent amount of complexity.
Features with Known Complexity: Most modern CMSes can provide internationalization and Localization, but edge cases can make things tricky (e.g., swapping out image assets to provide language-specific versions).
The Key: Ongoing Evolution of Estimates and Expectations
Given the impossibility of perfection, how does one move forward with any project knowing that you could hit a landmine at any time? Yes, developing a spidey sense on how to detect and avoid these will serve you well, but you will never catch all of them. The key is to set expectations in the beginning that you will iterate through the estimates at each stage of the project, further refining them with the additional information you’ve gleaned. Ideally, you’ve at least sized each feature to the right level of magnitude, and you can hone in the last 10-30% before your 20% through the project.
For more details on how to do this, be sure to check out my Drupalcon Nashville talk (or the recording if you can’t make it).
Photo by Helloquence on Unsplash