The M in MVC: Why Models are Misunderstood and Unappreciated

I've been writing a book about the Zend Framework recently - I'm sure some of you have noticed . A while back I finished a draft of two chapters called "The Architecture of Zend Framework Applications" and "Understanding The Zend Framework". The first chapter is an exploration of the Model-View-Controller (MVC) architectural pattern and why it has become the defacto standard for adoption in web applications. The second explores how the MVC architecture relates to the Zend Framework's components, their design and interactions.

By the time I'd finished both chapters I realised I had spent a lot of space explaining the Model, most of it discussing how the Zend Framework does not actually give you one. In fact, no web application framework offers a complete Model (for reasons I'll explain later). However nearly all frameworks manage to avoid making that perfectly obvious. Instead they continually link the concept of a Model to the related, but not identical, concept of data access. This is quite misleading.

This side of frameworks never gets a lot of attention. Yet it is a massive contributor to a whole class of problems in applications which attempt to utilise MVC by adopting web application frameworks. Further, I've always found myself better entertained by banging my head against brick walls than by trying to get the idea of a Model across to other developers. I'm not saying all developers are stupid or dumb, or that they don't get the concept in general, but all developers (PHP or not) don't quite link Models to the area of practice which justifies them - Object Oriented Programming principles.

In this entry I explore Models in terms of how developers relate them to Controllers and Views in applications and note a few strategies you can employ when using proper Models.

Models Misunderstood
The Model can be explained a few ways. In fact, you can write entire books about just the Model and many people have! The Model generally has two roles broadly defined:

1. The Model is responsible for maintaining state between HTTP requests.

Basically any data whether on a database, in a file, stored in the Session, or cached inside APC must be preserved between requests and thus forms part of the state of the application at the time the last request was completed. So always remember that the Model is not just the database. Even the data you get from web services can be expressed as a Model! Yes, even Atom feeds! Frameworks which rattle off introductions to the Model, almost never explain this upfront which only exacerbates misunderstandings.

Here's an example, while it's called Zend_Feed_Reader that new component which I'm developing is actually a Model. It reads feeds, manipulates them, interprets data, adds restrictions and rules, and generally creates a usable representation of the underlying data. Without it, you have Zend_Feed (the current top dog in feed reading) which requires lots of additional work to develop it into a viable Model. In other words, where Zend_Feed_Reader is a partial Model, Zend_Feed is pure data access.

2. The Model incorporates all the rules, restraints and behaviour governing and utilising this information.

For example, if you wrote business logic for an Order Model in an inventory management application, company internal controls could dictate that purchase orders be subject to a single cash purchase limit of €500. Purchases over €500 would need to be considered illegal actions by your Order Model (unless perhaps authorised by someone with elevated authority). The Model should be capable of enforcing this.

This makes incredible sense if you examine the meaning of the word "model". We have climate models in climatology, for example, which define the data, run processes, assume behaviours and calculate probable outcomes from these. The M in MVC is called a Model for a reason. The Model doesn't just represent mere data, it represents the entire system for which that data is useful. Of course any system can be complex enough to require multiple interacting Models but you get the idea.

If you read these two points, you may realise something startling. With the exception of the UI, most of any application can be expressed within Models. It's the Model that centers on all the data and underlying behaviour of that data, and sometimes even how to display that data. It's the Model which can understand, interpret and represent that data and provide it with meaningful uses.

In Programming, Fat Models Are Preferable To Size Zero Models
Jamis Buck (the author of Capistrano now employed by 37signals) once wrote about a concept called "Skinny Controller, Fat Model" (our own Chris Hartjes wrote a blog entry on the same topic). I've always liked the simplicity of this concept since it illustrates a key facet of MVC. In this concept, it is considered best practice to maintain application logic (like our business logic from above) within the Model as much as possible and not in either of the View or Controller.

The View should only be concerned with generating and presenting a UI so users can communicate intent to the Model. Controllers are the orchestrators who translate UI inputs into actions on the Model and pass back output from whatever View has been made aware of the Model(s) its presenting. Controllers must define application behaviour only in the sense that they map user input onto calls in Models, but beyond that role it should be clear all other application logic is contained within the Model. Controllers are lowly creatures with minimal code who just set the stage and let things work in an organised fashion.

PHP developers, by and large, don't fully understand what a Model is. Many see the Model as a fancy term for database access and others equate it with various design patterns for database access like Active Record, Data Mapper and Table Data Gateway. This is the misconception frameworks often promote, unintentionally I'm sure, with abandon. By not fully comprehending what a Model is, why it's such a great idea, and how one should be developed and deployed, developers unintentionally kick themselves into a dark path which leads into what can only be described as poor practice.

Here's a thought exercise to get you thinking. Imagine you just built the finest web application imaginable using the Zend Framework. Your client is incredibly impressed and their praise (and fees) are most welcome. Unfortunately they forgot to mention that their new CTO is insisting all new applications utilise Symfony and they'll offer you a nice shiny cheque if you'll convert your application over for them. The question is - how easy will that be? Think about it for a moment...

If you have your application logic bound up in the Model - then celebrate. Symfony, like many (but not all) frameworks, accepts Models regardless of what they are written on top of. You can port your Model, its unit tests, and its supporting classes with little to no modification into Symfony. If you stuck it all into Controllers, start feeling worried. Do you think Symfony can use Zend Framework controllers? Do you think those functional tests using the Zend Framework PHPUnit plugin will magically keep work? Ta-da. That's why Controllers are not a substitute for Models. They have minimal reuse.

Unappreciated, Underestimated, Unloved: Depressed Models
Since developers often demote Models to mere database access as supplied by default in 99.9% of frameworks, it's no surprise they are unimpressed by the theoretical ideals attached to them. By focusing and obsessing about data access, developers completely miss one important factor: Models are classes which are not coupled to the underlying framework. They have no expensive setup - you simply instantiate and use them.

Maybe we can't blame the average developer though. Web applications do have a certain behavioural pattern which makes that crooked path more attractive - lots of them are just big data readers. If there is little data manipulation, only data reading, then any Model will look similar to plain old reliable data access. It's unfortunate in a way, but don't let the simplicity of reading data fool you. Not all applications are going to be read only - some will no doubt have something to do with that data besides displaying it raw and unchanged from the database

Models are the underdog in PHP. Views have captured everyone's imagination since the advent of Smarty and it's brethren. Controllers make almost as much sense since they read from databases, and pass any retrieved data into templates (a common interpretation of VC). Yep - Controllers are the logical evolution of the mentally ingrained Page Controller approach every PHP developer and their dog has used since PHP3. At least - that's the obvious conclusion for most. We'll bust the "Controller = Page Controller" myth later.

Models though. Models don't have the same ideological attraction or similarity to old habits which is why people pass them over and use "data access" as the underlying meaning. You know, like references in PHP pointing to the same value in memory? The language has changed but the same old ideas are still in play behind the scenes, strumming their discord in our neural nets.

But wait...developers do write functioning applications! So if they are not using Models brimming with application logic, what the hell are they using?!

Fat Stupid Ugly Controllers: SUC It Up
Since Models are so lame to developers, they invented a new concept. Fat Stupid Ugly Controllers (FSUC). I put the SUC in FSUC on purpose. It seemed funny at 10pm after a few drinks . And less offensive than FUC. These were invented since Models were strange, alien, and possible terrorist entities nobody really trusted with anything other than data access methods.

The average FSUC reads data from databases (using a data abstraction layer developers like to pretend is a Model), or manipulates, validates and writes it, and passes other data to the View for rendering. It's incredibly popular. I'd say most people using frameworks create them as naturally as they ever did Page Controllers. They are popular because developers have figured out that they can treat Controllers almost like Page Controllers which requires little change from their historical practice using single PHP files for every application "page".

Notice something odd? The FSUC is performing all sorts of actions on data. Why? Because without a Model, all the application logic must go into the Controller, which makes the Controller - a type of Mutated Model! I use "Mutated" with reason. FSUCs are big, unwieldy, ugly looking and definitely fat. The quasi programming term that applies is "bloated". They perform roles they were never intended for. They are a complete reversal of everything you are taught to do in applying Object Oriented Programming principles. They are also pointless! Developers, for some mysterious reason, seem to prefer FSUCs to Models despite the fact those Controllers really are just Mutated Models.

Remember our thought exercise? If you put everything into Controllers, you will never be able to efficiently migrate an application to another framework. This type of tight coupling is the kind of thing Kent Beck fans are supposed to identify as a "code smell" in Refactoring. But it's not the only smelly artifact of FSUCs. FSUCs are big, with massive methods, multiple roles (one per method usually), massively duplicated code, lacking refactored external extracted classes and...they are a nightmare to test. With many frameworks, you can't even properly apply Test-Driven Design (TDD) on Controllers without writing custom plugins and being forced to manage request objects, sessions, cookies, and Front Controller resets. Even then you have to test the View it creates because Controllers don't have output independent of Views!

So keep asking those questions! How do you test a Controller? How do you refactor a Controller? How do you limit the role of a Controller? What is the performance of a Controller? Can I instantiate a Controller outside the application? Can I chain multiple Controllers in a single process and still remain sane? Why am I not just using simpler independent classes and calling them Models? Am I considering these questions at all?

Models Are Inevitable (Like Taxes And Death)
FSUCs make the classic mistake. By assuming a Model is a silly idea and plain old data access works best, developers unwittingly stick application logic into Controllers. Yes - congratulations. You just created a Model, a butt ugly Mutated Model you insist is a Controller. But this pretend Controller isn't easily portable, testable, maintainable, or refactorable (assuming refactorable is a real word...might be!). The nature of Controllers is such that they are tightly coupled to the underlying framework. You can only make a Controller execute by initialising the entire framework's MVC stack (potentially dozens of other class dependencies!).

Either way you try to go - you end up writing the application's logic somewhere. Putting it in a Model, away from the Controller, simply creates lots of classes which are independent of the framework you're using. Now you can test those buggers all day using PHPUnit without ever seeing a Controller or View in sight or having to torture yourself silly trying to make the entire framework stack reset after every test, and by seeing them as real classes with specific roles you can also put yourself in the right mindset to apply proper refactoring and create real maintainable code without duplicating code across multiple classes.

Models are inevitable. One can call that FSUC a Controller, but it's really a Controller+Model - an incredibly inefficient Model replacement. Someone just shuffled the source code around while laughing at all the fools talking nonsense about the importance of good independent domain models. Well, laugh back. They're the ones who have to test and maintain their mess.

This is really where most web application frameworks let their users down. They are surrounded by a lot of marketing garbage which subtly suggests that they offer a full Model. Have you ever seen a framework say anything different in an obvious way? Afterall, they are MVC frameworks. If they are forced to admit the M is something developers will have to create themselves, it might look bad. So they bury the truth in the documentation scattered throughout their data access feature, if they mention it at all.

In reality, they only offer data access classes - the real Model is something that's application specific and must be developed independently by the developer after discussion with clients (you can pick your buzzword practice to achieve that - I'm into eXtreme Programming (XP) personally). It needs to be tested, validated, updated and you face the same probability of failure/success regardless of what framework you use. A bad application in Rails will still make a bad application on Code Igniter.

Controllers Should Not Guard Data
The other facet of this lack of Model consideration is that when developers are convinced that Models should be used minimally, it reinforces a reliance on Controllers to take on a new role as Data Gatekeepers (a large reason why they mutate into FSUCs). Want some more opinionated fuel to fire up everyone disagreeing with me right now?

A while back when I was writing the proposal (Zend_View Enhanced) that would eventually be adopted into the Zend Framework to create an object oriented approach to generating complex Views, I started moaning about Controllers being used as the sole method of getting data from Models into Views. My own thoughts were that Views could skip the middleman and instead use View Helpers to read data from Models more directly. This would support an architecture where in many read-only page requests, your Controller Action would be...empty. Devoid of all code. Nada.

In my mind, the best Controller is a Controller which doesn't need to exist .

I was immediately treated to a mass of resistance. Few people seemed to understand that an empty Controller, with all Model interaction extracted into simple reusable View Helpers, would reduce code duplication in Controllers (lots of code duplication!) and avoid the necessity of chaining Controller Actions. Instead there was lots of muted puzzled expressions. Most people simply assumed that MVC worked as follows: requests go to Controllers, Controllers get data from Model, Controller gives Model data to View, Controller renders View. In other words: Controller, Controller, Controller, Controller. At one point I remarked that the entire community was obsessed with Controllers . It's still tough going just to get anyone to give Views data objects, let alone let them read directly from Models...

Note: Not to sound like a complete pratt - some people did get the idea .

None of the objections really caught on that this was an old idea. Java coined the term View Helper years ago as a design pattern in J2EE showing how View Helpers could assist Views in accessing Models (only on a read only basis since any writing should go through a Controller!) without the intermediary Controller. In MVC, there is every indication that Views should know about the Models they are presenting. Not just whatever arrays we decide to spoon feed them from Controllers.

So go further! How many Views only need one Model? A lot of my Views use several Models, with highly repetitive Model calls. A View Helper is a single class - but adding these repetitive calls to Controllers, means you need to duplicate calls into multiple methods!

One crazy solution created to outwit the View Helper, is often to reimagine Controller Actions as reusable commands. Whenever a View needs several Models, you just call several Controllers in sequence. This concept is something I call Controller Chaining - its premise is to create supporting code to make reusing Controllers possible. You can translate that as: reusing every class needed to execute a single Controller Action. Remember - you can't use any Controller without initialising the entire framework . Though there are (there always are!) some exceptions.

Models are Classes, Controllers are Processes
I'm running thin on quickie ideas, but I mentioned Controller Chaining and it bears further examination. Chaining is used either to access multiple Models, or to merge the results of multiple Views, or both. Usually both - Controllers nearly always access Models and pass data into Views when you are not using View Helpers to shortcut the process.

Imagine you create three Controllers, each of which generates a View. To build some new web page, you need to merge all three Views into a single template page (or Layout). This is accomplished by telling Zend_Layout (or some other alternative solution for aggregating Views into common layouts/segments) to call all three Controllers sequentially. Now consider what is happening - three Controllers means you are making three dispatches into the MVC stack. Depending on the application this can pose a significant performance cost. Symfony, to illustrate how real that cost can be, uses "Components" as a specialised Controller type just to offset this performance hit but the Zend Framework has no equivalent. Rails has a similar idea which is rife with complaints about performance hits. The prevailing wisdom across frameworks is that using multiple full Controllers is horribly inefficient and a last resort when no other reuse strategy is available.

I'll say it again - Controller Chaining is a code smell. It's inefficient, clunky, and usually unnecessary.

The alternative is obviously to just use View Partials (template snippets capable of being merged into a single parent View) which can directly interact with Models using View Helpers. Skip the Controllers entirely - afterall they have no application logic unless translating user input into relevant Model calls (unless FSUCs ).

This highlights that Models are just a collection of loosely coupled classes. You can instantiate and call them from anywhere - other Models, Controllers and even Views! A Controller, on the other hand, is a tightly integrated cog in the machinery of a whole process. You can't reuse a Controller without also dragging in the entire process of setting up request objects, dispatching, applying action helpers, initiating the View, and handling returning response objects. It's expensive and clumsy in comparison.

Signing Off
There is a fast paced urgency to this article you'll have noticed. I'm a self confessed nut about applying the principles of elegant Object Oriented Programming to MVC frameworks. It's why I went all out with Zend_View Enhanced and saw it adopted, debated, and adapted with great success, into the Zend Framework. A lot of that is owing to Matthew Weier O'Phinney and Ralph Schindler who joined that parade. If we lose sight of simple OOP practice and principles, and get lost in trying to fight MVC into submission, the plot ss quickly lost. The MVC is a great architectural pattern - but at the end of the day our MVC interpretations and ingrained beliefs are all subservient to OOP principles. If we forget that, we do Bad Things™.

Hopefully this torrent of ideas about the Model in Model-View-Controller is somewhat enlightening and makes you think. Thinking should be your goal - question everything, and rebel when you feel something going wrong. If we run around on blind faith than we deserve everything coming to us .

Referencias
Food for thought: utilizing models in MVC
“What is a model” and “Is Zend_Db_Table a model” seem to be asked once in a while on #zftalk. Frameworks say they have a full model available, thus they are MVC frameworks. ORM libraries say they generate models. It seems the A...
Weblog: CodeUtopia
Tracked: Dec 06, 12:15

Das M im MVC-Entwurfsmuster
Ich habe vor einiger Zeit einmal über die nicht vorhandene Komponente Zend_Model geschrieben und dort den Stand der Dinge aus meiner Sicht zusammen gefasst. Das Kapitel zum Thema Models in meinem Zend Framework Buch ist bereits fast 20 Seiten lang und...
Weblog: Ralfs Zend Framework und PHP Blog
Tracked: Dec 09, 21:14 Sphere: Related Content

How to Design Programs

An Introduction to Programming and Computing by Matthias Felleisen, Robert Bruce Findler, Matthew Flatt, Shiram Krishnamurthi
(September 2003 Version, The MIT Press)

This is the first update since the third printing by the MIT Press. This release corrects mistakes and typographical errors in the first three printings.

The Book the complete text
Solutions solutions to all exercises, for teachers only !??!
Problem Sets additional problem sets not found in the book
Companion hints on how to use DrScheme
Teachpacks if you encounter bugs in your Teachpacks
Known Mistakes known typos and mistakes
DrScheme programming environment
TeachScheme! our educational outreach effort
Mail how to reach us Sphere: Related Content

The Schematics Scheme Cookbook

The Schematics Scheme Cookbook is a collaborative effort to produce documentation and recipes for using Scheme for common tasks. See the Book Introduction for more information on the Cookbook's goals, and the important ContributorAgreement statement.

Cookbook Starting Points
• Table of Contents (with recipes)
• Getting started with PltScheme
• The FAQ answers common questions about the Cookbook
• Common Scheme idioms
• Getting started with macros
• Using strings
• Using files

Editing Pages
•Register as an author
•Information on TWiki the software running this site.
•AuthorChapter -- A guide to contributions

Other Popular Pages:
The following pages are autogenerated:
•RecipeIndex - Recipes by chapter (similar to TOC).
•ParentTopics - List of Chapters & Sections
•AllRecipes - List of all recipes
•LibraryIndex - List of external software libraries

Administrative Pages
•AdminTopics - Summary of all administrative topics
•CookbookBrainStorm - ideas for the structure of the cookbook.
•ToDo - items to be done (a very incomplete list)
•RecipeStubs - Recipes that need to be written or completed
•HelpNeeded - Pages with markup problems
•OriginalCookbook - links to a converted version of the original cookbook
•RecentSiteChanges (last changed 09 Apr 2007 - 00:15)
•SectionIndex - This page has errors but somebody may want to fix it.

Notes:
•You are currently in the Cookbook web. The color code for this web is this background, so you know where you are.
•If you are not familiar with the TWiki collaboration platform, please visit WelcomeGuest first.

Schematics Cookbook Web Site Tools
• (More options in WebSearch)
• WebChanges: Display recent changes to the Cookbook web
• WebIndex: List all Cookbook topics in alphabetical order. See also the faster WebTopicList
• WebNotify: Subscribe to an e-mail alert sent when something changes in the Cookbook web
• WebStatistics: View access statistics of the Cookbook web
• WebPreferences: Preferences of the Cookbook web (TWikiPreferences has site-wide preferences)

The PLT Scheme is now Racket, a new programming language. The above page is for compatiblity and historical reference only. See the Racket site for up-to-date information. Sphere: Related Content

The Four Elements of Simple Design

I realize that I've never really written this down before, and I say it so frequently in my work as a trainer and mentor, that I think it bears repeating.

I have reduced everything I've ever learned about effective object-oriented design to the four elements of simple design that I first learned from Kent Beck's work. Maybe you can, too.
I define simple design this way. A design is simple to the extent that it:
* Passes its tests
* Minimizes duplication
* Maximizes clarity
* Has fewer elements
Note that I put these properties in priority order. I'm willing to copy-and-paste to get a test passing, but once the test passes, I can usually remove the duplication quickly. I'm willing to extract code into a method and call it foo() in order to get rid of duplicate code, although that name foo() rarely survives more than 15 minutes. Finally, I will gladly introduce interfaces, classes, methods and variables to clarify the intent of a piece of code, although generally speaking once I make things more clear, I can find things to cut.

Some people put “minimize duplication” and “maximize clarity” in a tie for second place. I don’t. My experience has led me to conclude that removing duplication helps more than fixing bad names does. Moreover, removing duplication tends to allow a suitable structure to emerge, whereas bad names highlight an inappropriate distribution of responsibilities. I use this observation as a key element of my demonstration, “Architecture Without Trying”.
Now I should point out that, as a test-driven development (and now also a behavior-driven development) practitioner, I write tests as I draw breath, so I don't really need to emphasize that part.
* Passes its tests
* Minimizes duplication
* Maximizes clarity
* Has fewer elements
I should also point out that I’ve yet to see a codebase with low duplication and high clarity that, nonetheless, had considerably more design elements than it needed, so I don't really need to emphasize that part, either.
* Passes its tests
* Minimizes duplication
* Maximizes clarity
* Has fewer elements
That leaves me with two key elements of simple design: remove duplication and fix bad names. When I remove duplication, I tend to see an appropriate structure emerge, and when I fix bad names, I tend to see responsibilities slide into appropriate parts of the design.
I claim these to be axioms of modular design, with a “parallel postulate” of whether you use objects or not. If you use objects, you get object-oriented design, and if you don’t, you get structured design. (I don’t know how functional design fits into this yet, because I haven’t done enough of it.)
I claim that developing strong skills of detecting duplication, removing duplication, identifying naming problems, and fixing naming problems equates to learning everything ever written about object-oriented design.
Put more simply, if you master removing duplication and fixing bad names, then I claim you master object-oriented design.
Now I wouldn't bother burning your old OOD/OOP books, but I will tell you that if you have an interested buyer, then feel free to sell them. Sphere: Related Content

30 Programming eBooks

Since this post got quite popular I decided to incorporate some of the excellent suggestions posted in the comments, so this list now has more than 30 books in it. [UPDATED: 2010-10-10]

Learning a new programming language always is fun and there are many great books legally available for free online. Here’s a selection of 30 of them:

Lisp/Scheme:
How to Desing Programs
Let Over Lambda
On Lisp
Practical Common Lisp
Programming in Emacs Lisp
Programming Languages. Application and Interpretation (suggested by Alex Ott)
Structure and Interpretation of Computer Programs
Teach Yourself Scheme in Fixnum Days
Visual LISP Developer’s Bible (suggested by skatterbrainz)

Ruby:
Data Structures and Algorithms with Object-Oriented Design Patterns in Ruby
Learn to Program
MacRuby: The Definitive Guide
Mr. Neighborly’s Humble Little Ruby Book (suggested by @tundal45)
Programming Ruby
Read Ruby 1.9
Ruby Best Practices
Ruby on Rails Tutorial Book (suggested by @tundal45)

Javascript:Building iPhone Apps with HTML, CSS, and JavaScript
Eloquent Javascript
jQuery Fundamentals
Mastering Node

Haskell:
Learn You a Haskell for Great Good
Real World Haskell

Erlang:
Concurrent Programming in Erlang
Learn You Some Erlang for Great Good

Python:
Dive into Python
How to Think Like a Computer Scientist – Learning with Python

Smalltalk:
Dynamic Web Development with Seaside
Pharo by Example (based on the next book in this list, suggested by Anonymous)
Squeak by Example

Misc:
Algorithms
The Art of Assembly Language
Beginning Perl
Building Accessible Websites (suggested by Joe Clark)
The C Book
Compiler Construction
Dive Into HTML 5 (suggested by @til)
Higher-Order Perl
The Implementation of Functional Programming Languages (suggested by “Def”)
An Introduction to R
Learn Prolog Now!
Objective-C 2.0 Essentials
Programming Scala

Of course there are many more free programming eBooks, but this list consists of the ones I read or want(ed) to read. This is far from comprehensive and languages that are completely missing are mostly left out on purpose (e.g. PHP, C++, Java). I’m sure somebody else made a list for them somewhere. Sphere: Related Content