Tag Archives: MVC

Ten ways to build an unmaintainable web application

Old-school hackers had a long tradition of ensuring job security by building applications so unmaintainable that only the original authors could work on them. But in these days of web applications, unmaintainability has fallen by the wayside. Instead, design fads like CRUD, REST, MVC, DRY, and KISS, have eliminated the average programmer’s job security.

Here are ten quick tips for achieving maximum unmaintainability in your web application. Following them will ensure that, in thirty years, a web programmer like you will be as valuable as a fifty-eight year old COBOL programmer contracting at $200/hr for a Fortune 500 company that still hasn’t migrated off of PL/1. You too will be able to live on a dairy farm in Pennsylvania, grow a beard down to your navel, and work in your underwear. And you’ll never have to learn anything new, work with anyone else, or start another new project.

  1. Mix it up. Put some JavaScript into external files, but be sure to intersperse JavaScript into your HTML, some of it in <script> tags. Cram multiple JavaScript statements into onclick and other event attributes — the longer, the better. Do the same with CSS; put some into external files, some in <style> tags, and also put some critical CSS into complex style attributes. And remember to put most of your <script> and <style> tags in the middle of the page content, instead of in the <head>, so that they will be difficult to find.
  2. Make everything dynamic. Generate JavaScript and CSS in your HTML templates. Think of it as another type of eval. Generate HTML server-side using templates and browser-side using JavaScript. What’s harder than working around a obscure IE layout bug with weird markup tweaks? Making sure both your server templates and your JavaScript HTML generation work around the same bug with the same HTML black magic.
  3. Abstraction, Shmabstraction. Pass lots of data from the server to the browser, store it in hidden form fields in the page, and then pass it back, unchanged, when submitting the form. That way, when the back-end data model changes, you get to rewrite part of the interface too. Allow data-model or server implementation details to creep into the interface implementation. Is the database sharded? Is the cache dirty? Does this row use a composite key? No need to have the server abstract these details, just pass that information to the JavaScript and let it sort everything out. That way, a sysadmin or a DBA can break the UI just as easily as a web designer can.
  4. Keep your data unstructured. Make sure all communication between the browser and the server is just a flat list of key/value parameters. Some of your parameters will be data to store, others will be modes or flags that affect the behavior of the service you’re hitting, and still others will be modifiers to display messages or affect the behavior of the UI. Keeping your data unstructured ensures these different types of parameters will collide. Often.
  5. Commit to a platform. Don’t waste your time checking to see if your pages work in all browsers (at least not until you’re totally done). Better yet, develop only in a single browser and don’t even bother to find out whether the features you’re relying on even exist in other browsers. Nothing is more fragile than an application that’s tightly tied to a single platform.
  6. Trust the browser. Rely soley on JavaScript input checking for some data — don’t check input on the server-side. Store sensitive data in hidden form fields. Put authorization checks in the JavaScript rather than on the server. Parameters like authorized=1 just scream out for URL hacking, and storing them in hidden form fields is only slightly harder to hack.
  7. Trust the server. Rely soley on the server to check, store, and generate only valid data in some places. That way, a DBA can change a single column constraint or data-type, and parts of the UI start to fail.
  8. Don’t use DOCTYPEs. That way you’ll never be sure what rendering mode different browsers are going to use to render your content.
  9. Ignore the cascade. Don’t bother to understand what the C in CSS stands for.  Just keep overriding styles until a page element looks the way you want. That way, your styles will be fragile and will break unexpectedly when an intern changes something a reasonable person would expect to be unrelated.
  10. Don’t use classes or ids. Instead, always write JavaScript and CSS that finds nodes based on tag name, name, alt or title attributes, or by their position in the DOM. That way when anything in the page changes, the hierarchy, the attributes, or when the site is translated into another language, things break. If you do end up using class or id, be sure to make a separate class for every node in your document and assign the same id to several different nodes.

If, however, you want to write flexible code that can react to and evolve with the ever-changing needs of its users, even after you have left the project in the hands of a clever but inexperienced hacker, you should probably avoid these techniques, and read up on some of those lame new design fads instead.

Special thanks to all the programmers whose code has illuminated these techniques over the years. My job may not be as secure as yours, but at least my code, and my conscience, are clear.

The next big thing, part 3: Taking the relational out of relational databases

Part of an ongoing series.

The relational database is an extremely powerful tool. But sometimes data isn’t very relational, and sometimes transactional, relational, integrity is not as important as it is for, say, a bank. This is one reason why so many sites can get away with mySQL backed by myISAM tables — they’re fine if you’re read-heavy and data integrity is not mission-critical.

Some new projects have sprung up which provide key-value stores or simpler kinds of databases without all the overhead and inflexibility of a relational database.

On the other hand, sometimes data is way more interrelated than a traditional relational database is prepared to handle. Sometimes different kinds of items (i.e. rows) in a database can be related to many other kinds of items in that database, and sometimes end users can create not just new items or new relationships, but new kinds of relationships between items. This type of database is called a graph database, and there are also projects pushing the boundaries of relational in this completely opposite direction.

Pretty much everywhere I interviewed back in February 2008 was either building their own graph database, working on an existing one, or repurposing a relational database (or, in one case, a search backend), to kinda, sorta behave like one. The w3c, not one to be left behind when there’s a specification to be written, is even working on a SQL-inspired query language intended to search them1.

Most applications have some combination of totally un-relational data that can go in a key-value store, some strictly relational data that belongs in a SQL database, and some flexible, highly relational data that belongs in a graph database.

What will happen when these alternative databases start giving traditional relational databases a run for their money? Well, sharding, caching, and normalization all start to sound a lot more complex when the data is in a few different kinds of databases — but then again, maybe optimization won’t be as necessary if a single SQL database isn’t doing all the heavy lifting. Object-relational mappers (and the web frameworks that use them) might need to talk to, and abstract away from, different kinds of databases2.

And the different types of data won’t always be easily separated along table boundaries. Maybe these different types of databases will talk to each other, or maybe they will mature into über-databases that understand lots of different types of data relationships.

But the monolithic, strictly relational, master SQL database is eventually going to go the way of Cobol3.

  1. Of course, if it’s anything like other technologies designed by the w3c, it’s a steaming pile. [↩]
  2. Some can already handle talking to multiple SQL databases, and of course there’s two-phase commit. [↩]
  3. Or Kobol. [↩]

The next big thing, part 2: Taking the web out of web applications

Part of an ongoing series.

A web application is just a stateless1 application that responds to various requests by performing actions and providing resources. There’s no fundamental reason an application must only communicate over HTTP. Web applications are going to start adding alternative methods of interaction, and I think the first common one will be email.

Perhaps an example will best illustrate this:

Like many web forums, posts to Mosuki‘s discussion forums get mailed out in email. But, unlike any other web forums I know of, they also behave like mailing lists. All the emails have a reply-to header with an email address that identifies the message, the recipient, and the action to be taken if that email address is used. In this case, the contents of a reply email are posted to the forum exactly as if a reply had been posted via the website.

In other words, the action “post a message” can be accessed via a web page and a browser or via a reply-to header and your mail client.

There are other examples of this separation between input/output channels and the application logic. The most obvious is Twitter, which of course can be interacted with via HTTP or SMS2. And the Son of Sam project intends to let you “use modern concepts like handlers, requests, responses, state machines” to interact with email.

Confirm a Facebook friend request, RSVP to an Evite, revert a Wikipedia edit, or reassign a bug report, just by replying to an email or sending an SMS.

There are a number of technical issues inherent a system like this.  An application’s framework has to handle multiple input channels, and massage email bodies, HTTP requests, and other input into a least common denominator “request.” Authenticating a user via email, an intrinsically forgeable medium, and protecting against spam, are non-trivial challenges. And a suite of templates suddenly gets a lot more complex when it has to provide views for multiple types of interfaces3 .

This blurring of the line between email, HTTP, SMS, and other communications is not new, strictly speaking. But I think it will become commonplace and even expected. Rather than writing a modern (MVC, stateless, REST-ful, &c.) web application, people will be writing modern (MVC, stateless, REST-ful, blah, blah, blah) applications that have web interfaces, email interfaces, and whatever other interfaces they need.

Stay tuned for the next installment of The next big thing: Taking the relational out of relational databases.

  1. More or less stateless, that is, authentication tokens like cookies notwithstanding. [↩]
  2. As well as more standalone apps than you can shake a stick at. [↩]
  3. Generating text and HTML responses for email that look good and work well in the top 75% of desktop and web email clients is a lot harder than testing a site’s HTML in Firefox, IE, Safari and Opera. [↩]

Another browser-side Model-View-Controller analogy

Coding Horror presents another way to think about the browser side of web apps as underlyingly MVC in Understanding Model-View-Controller. It’s interesting, but I still prefer my analogy for primarily AJAX web apps; when the data comes in primarily through XMLHTTPRequest, it doesn’t make much sense to think of anything but the JavaScript that handles XMLHTTPRequest responses as the model.

Ruby’s not ready: comments, corrections, and clarifications

Some good discussion on this one. It’s nice to see Ruby people saying things like this (5th message from the top, from Song Ma):

Interesting. But what I am thinking about is not the attitude of the author, but the points he was trying to make. The deep review and discussion will benefit the language insights.

Or this one (from Trans, on the same forum):

Why is everyone getting so worked up? It’s a critique. Biased it may be, but that in itself does not make it worthless. In fact, it can be very constructive b/c it uncovers “attack points” with the language. With each point we can ask ourselves objectively is this a misconception or a fair point? In either case we have an opportunity, to address misconceptions in our Ruby evangelizing blogs and to work to improve Ruby where a point has merit.

Bias can work both ways. But I think the Ruby community can rise above it, and Ruby will be all the better for it.

And from Peter Cooper at Ruby Inside:

As it is, I think he’s missing the point a lot of the time (he tends to think Python’s better because he likes its conventions more than Ruby’s – not a compelling argument), but it’s an interesting read none the less. Anything that keeps our minds open to the fact that Ruby != perfection is worth a look.

And a comment on the same post:

Let’s take his best points and incorporate them into future versions of Ruby.

Sounds like a plan.

I saw a few counterarguments like this:

Everything he’s saying is well known.

Just because a problem is well known inside a community doesn’t make it any less of a problem.

Everybody who mentioned documentation, even those who disagreed strongly with the rest of my post, agreed that Ruby’s documentation is seriously lacking. In fact, a lot of the mistakes in my original post are due to me not being able to easily find an explanation of something on the various Ruby doc sites. Which leads me to…

Continue reading →

Ruby’s not ready

Introduction

A few weeks ago, I learned Ruby and Ruby on Rails to compare them head-to-head against Python and Pylons, in preparation for a new project. When I began, I knew nothing about Ruby or Ruby on Rails. I have tried to be as objective as possible: before beginning this project, I wrote in email on March 5th:

I promise we’ll be as objective as humanly possible; if Ruby and Ruby on Rails truly is better, we’ll happily use RoR and never look back. I want to know that I’m using the absolute best tool for the job.

Since then, I have reimplemented one complex nine-hundred line Python library, PottyMouth, in Ruby. Another team member has also reimplemented parts the Pylons web application Spydentify in Ruby on Rails.

The best tool for the job is Python & Pylons. While Rails and Pylons are similar, shortcomings in Ruby compared to Python make Python & Pylons the clear choice. I make three basic arguments against using Ruby:

  • The language and its implementation are incomplete and immature. Immature implementations breed performance issues. A project loses time when it must implement missing or incomplete functionality.
  • The language is inconsistent and needlessly complex. Inconsistency and complexity confuses people and confusion breeds bugs.
  • The documentation is incomplete. Incomplete documentation breeds bugs as you might misuse a feature. And a project slows down while you read the language or library source code, or ask the community for help with undocumented features.

I believe Ruby would fare poorly against other languages, not just Python, on these angles as well.

Why have Ruby and Ruby on Rails gained so much traction, despite these issues? Aside from the Rails hype, it’s because they are not insurmountable issues. It is possible to build a large application in Ruby; many people have. But any programmer building a large application in Ruby will have to deal with the issues listed here at some point. These are all issues that do not appear right away. A project doesn’t face them until a website reaches maturity, develops lots of features, fields traffic from lots of users, or until a project hires programmers who aren’t Ruby experts or experienced enough to anticipate these issues.

My point is simply that Python (and other languages), allow you to handle most of these issues more elegantly, or avoid them completely.

Subscribe here if you’d like to be notified of any follow-up posts (for an article this long, I’m sure there will be a few), or if you’d like to read my critiques, positive and negative, of other things technological.

Contents

  1. Unicode and encodings
  2. Regular Expressions
  3. Documentation
  4. Migration to Ruby 1.9/2.0
  5. Performance
  6. Scoping
    1. One nice thing about Ruby’s scoping
  7. There’s more than one way to do it
    1. String conversion
    2. print, p and puts
    3. Ranges and slices
    4. require and load
    5. Raising exceptions, throwing strings
    6. do and then are extraneous
    7. length and size, update and merge
  8. Object model
  9. Faking keyword arguments
  10. Libraries
    1. SAP support
    2. DateTime support
  11. Debugging
  12. Rails & Pylons
  13. Cool things about Ruby
  14. Conclusion
  15. Further reading

Unicode and encodings

If you’re already familiar with Ruby’s problems with Unicode, feel free to skip this section. Ruby did not have any support for Unicode character strings when it was originally released in 1996. This is only slightly silly for a language that was invented after Unicode 1.0 was released in 1992. It is inexcusably shortsighted that Ruby has not added Unicode objects over the last twelve years.

A third-party Ruby library for conversion ties into the Unix iconv program, allowing conversion between two different encodings. However, converted strings are still sequences of bytes. This means that using most of the string methods (slice, reverse, size, index, downcase, upcase, strip) and indexing into the string with [] notation do not work in non-ASCII encoded strings. You can get the desired results out of these methods by first accessing the .chars attribute of non-ASCII strings. This is less desirable because the programmer must remember to use .chars whenever he or she is working with non-ASCII strings.

A better solution would be to support first-class Unicode objects, as strings of Unicode characters, natively in the language.

There is a third-party Unicode support library that replaces Ruby’s String class and adds Unicode support, but it is acknowledged to be hackish, potentially dangerous, and makes Ruby somewhat slower.

Unicode support may or may not be forthcoming in Ruby 2.0. There are certainly members of the community advocating it.

This means, among other things, that there is no built-in support in Rails’ HTML generation1 , for converting Unicode characters to HTML entities. This page details how to hack around this problem; but this is something that should be automatic and built-in, not hacked around.

Python’s built-in Unicode and encodings support, which is a first-class, native Unicode object and a full suite of built-in encodings, was introduced in Python 1.6 in 2000. It has evolved into an extremely reliable, secure and versatile Unicode implementation. It is also extremely simple to use.

Python supports all of the encodings that Ruby supports via iconv, and a number that it doesn’t, including Quoted-Printable, the encoding used for the vast majority of email messages, and MBCS, the encoding used by Windows FAT32 and NTFS file-systems.

Because Python’s built-in Unicode support is so robust, the vast majority of Python libraries all convert to Unicode when accepting input, and convert to the proper encoding when producing output. Multi-language support and correct encoding handling is usually a non-issue when building a Python application. For example, non-ASCII input to, and output from, a Pylons web application Just Works™.

Regular Expressions

For a language that borrows so heavily from Perl, the regular expression support in Ruby is pretty disappointing. Regular expressions might not seem like a very important part of a language, but it’s an interesting litmus test because Python, Perl, and JavaScript all support essentially the same regular expression syntax. Ruby’s regular expressions, however, were so broken that I switched to Ruby 1.9 to finish porting PottyMouth.

Ruby’s Regexp::MULTILINE flag doesn’t behave the way multiline does in other languages. In other languages, the multiline flag is off by default, and when enabled, it considers . to include newlines and ^ and $ to match right after, and right before, every newline:

In Perl:

if ( "foonbarnbaz" =~ /^bar/m ) { print "yesn"; } else { print "non";}
yes
if ( "foonbarnbaz" =~ /^bar/ ) { print "yesn"; } else { print "non"; }
no

In Python:

>>> import re
# This matches
>>> re.search('^baz', "foonbarnbaz", re.MULTILINE)
<_sre.SRE_Match object at 0xb7c4bf38>

# This does not match
>>> re.search('^baz', "foonbarnbaz")

However, in Ruby, the Regexp::MULTILINE flag appears to only affect the interpretation of ., not ^ and $, making it more like Python’s re.DOTALL or Perl’s /s switch.

in Ruby: