How SproutCore loads views

I’ve been asked this question several times in the last few days so I thought I might help to put an explanation about how SproutCore handles binding HTML to JavaScript.  If you’ve ever tried to build a JavaScript app before, you probably know that the biggest cost when you load the browser is hooking up all of the JavaScript and event bindings to the HTML in you web page.  There are actually four things that are real performance hogs in a browser:

 

  1. Creating DOM.  If you want to build new HTML to replace existing HTML it will be 10-20x slower than simply having the HTML baked into the page on load in the first place. 
  2. Searching the DOM.  In general browsers are not super fast at finding elements in the DOM except for those named by IDs.  Even for those browsers with XPath support, the cost is nontrivial.
  3. Adding JS properties to DOM elements.  Almost all libraries out there today work by patching the HTMLElement prototype to add nifty utilities like $().  The only problem is that this trick does not work in IE, so instead you have to monkey patch every single DOM element when you touch it.  In IE6, that’s a 10msec penalty for every element.
  4. Binding to events.  Every time you add an event listener in the web browser, its another penalty.

 

Oddly enough most JS frameworks use techniques that hit all four of these bottlenecks, mostly because they want to “attach” to your HTML on page load.  If you’re adding just a few wiz-bang Ajaxy/Web2.0 effect this is not a problem, but once you start building entire applications with dozens or hundreds of views, these little costs add up.

The SproutCore Solution

The first versions of SproutCore actually faced these same problems as well.  In fact, I spent the better part of 6 months working through all of them.  Now SproutCore takes a decidedly non-traditional approach to avoid these issues.  Here is what happens when you build a SproutCore app:

 

  1. Static HTML is pre-generated.  Whenever possible, SproutCore views expect the HTML they will be managing to be pregenerated in the main document HTML.  The only time SproutCore views create new content is usually for collection views such as lists, grids, and tables.  By using this approach, SproutCore can use the most efficient code path in the browser (parsing on page load) to build out any DOM that will be required by the app.
  2. HTML is consumed on demand.  Most HTML and JS views are not needed right when a page loads.  It might be a hidden dialog or popup for example, that will not be used until later in the app.  Rather than finding all of these DOM elements right away, most HTML is stored in a single div called “resources”, which is pulled out of the main document as soon as it loads.  The SproutCore views and the DOM they use are consumed only when they are actually needed by your application.  
  3. Views manage DOM; DOM is not to be modified.  SproutCore views wrap DOM objects and reproduce most of the common DOM API you might use.  The idea is that you can generally work with SC.View objects and let them take care of the DOM elements for you.  The view’s in turn, take great pains to avoid actually adding properties to the DOM elements, avoiding the heavy performance penalty that comes along with it.
  4. Events are managed at the window level.  Instead of listening for an onclick() handler on 50 different elements on your page, SproutCore apps list to exactly 6 events on the window object instead.  SproutCore code then takes care of routing those events to the proper SC.View instance to handle the event.  Views themselves do not need to register for events to receive them; they just implement the appropriately named responder methods.  This means you can respond to as many events as you want on your page and you won’t pay any performance penalty on page load.

 

Overall SproutCore views definitely favor performance over convenience.  To make views convenient as well, SproutCore has the view builder system which uses to Ruby code to generate both the HTML and JavaScript for your views so you don’t have to.  This generally makes views both easy to use and performant.

 

New About Page/FAQ

Check out the updated about page.  I know there are some typos I need to fix in the morning, but in the meantime please leave feedback if you think the questions are not answered well enough and/or there are other basic introductory questions you think we should answer here.

 

New Feature: Style Properties

One of the coolest things about the SproutCore property system is the unknownProperty() handler.  This method is called on your object whenever someone tries to get or set a property that is undefined.  Normally, this method does not do much, but you can use it to implement all sorts of exotic behaviors with just a tiny bit of code if you like.  This is how SC.Controller objects work, for examples.

The latest bit of magic made possible by unknownProperty is the style properties just added to SC.View in trunk.  Now you can directly get or set most CSS properties on your view just by adding “style” to the property name.  For example, if you want to set the background color, you could do:

myView.set(’styleBackgroundColor’, ‘red’);

Likewise, if you want to get the left setting, you could do:

myView.get(’styleLeft’);

Certain properties, notably those ending in “Left”, “Right”, “Top”, “Bottom”, “Width” and “Height” will be automatically converted to integers (we assume pixel measurements for now).  Most other values are passed through directly.

These properties can help make your code easier to read today and they will become very useful for the new animation system in the future.

Remarkably, all of these properties were added with about 10-lines of code thanks to unknownProperty(). If you want to see how some of this magic works for yourself, checkout the code in SC.View.

 

Important: Action Required to Make Your Apps work in IE

UPDATE: I have released a new SproutCore gem (0.9.3) that includes these updated templates.  Update to make sure new apps you create include this fix.

I am beginning to re-enable IE support for SproutCore now.  Unfortunately I found a bug in the templates for clients and foundations that has now probably made its way into all of your code.  Please do the following in your projects to make sure this is not a problem for you:

  1. For each client or framework, open the core.js file.
  2. Look at the last line just before the });  
  3. If it has a trailing comma, delete it.  The last item in a hash should never have a comma trailing it.

This has been fixed in the templates so new clients and frameworks you create will not have this problem.  The SproutCore frameworks, sample apps and built in apps also have this fix.

By the way, the test runner and docs apps now work in IE7.  All unit tests for SproutCore pass except for the view-level ones.

Everything Old Is New Again: Why Client-Server is the Future of the Web

I’ve been talking to people a lot recently about the future of the web as I’ve worked on SproutCore.  I think most people agree on at least two things:

  1. Future web apps will be expected to be more “app-like”.  That is fast, fluid, and feature-rich. (the 3Fs of the next web)  Because of Ajax people are less and less willing to wait for their browser to refresh just to change a setting or edit some data.   
  2. No one likes plugins.  This future must somehow be built primarily on “web native” technologies - HTML, JavaScript, and CSS with Flash lending an assist where necessary.

As web apps creep towards more desktop-like behavior, they are quickly approaching that uncanny valley, where some switch will go off in users minds and they will suddenly expect web apps to have all the attributes of a desktop app.  Reasonable or not, its up to developers to figure out how to make this happen.

Here’s the fundamental problem:  web apps tend to be slow because they often live mostly in the server.  You take an action, wait on the server to respond, take another action, wait on the server to respond.  etc.  This is just a form of the old “terminal” approach used by mainframes.  Like the mainframes, the performance of an app like this is necessarily limited;  you simply will not be able to make this lag go away.

A lot of people say that Ajax is the solution to this.  In some ways it is but for most apps I’ve seen, Ajax is actually just a way to optimize this terminal behavior a bit.  Instead of refreshing the entire page for each click, you only update a part of it.  (By the way, did you know that mainframe terminals had their own version of Ajax?  It was called the IBM 3270.  The IBM 3270’s primary benefit was-you guessed it-partial screen updates.  It revolutionized mainframe computing…until the PC came along.)

So clearly we’ve seen this pattern before.  Users want a richer, faster more fluid experience than our old terminal-like technology can provide.  Tools like Ajax extend the life of our old model somewhat but they are too limited to deliver the kinds of experiences people will ultimately want.  So how do we solve this?

I propose that we look to the same solution IBM and friends came up with: client-server.

Client-server computing simply means that instead of building one app on the server and projecting it to the browser, you build two complete apps - one on the client and the other on the server - that both share your business logic and can operate on their own, sharing data when needed.  This allows the client to do most things with your data without consulting the server, hence delivering on the 3Fs.  In this case, the client will be written in Javascript and running in your web browser, but that does not change the model.

Thanks to more bandwidth, smarter browsers and faster PCs, the client-server model on the web is finally possible again, and without the major drawback that plagued itin the past.  Before you had to download and install clients onto every machine which was difficult to do and expensive to maintain.  Now, a 200K compressed “web app” of HTML, JS, and CSS can download and start running in just a few seconds.  And for the most part we don’t have to worry about cross-platform issues like before.

But not everything is peaches and cream.  For starters, writing an app in Javascript on the web is akin to writing C on the desktop: it is just one level above the “bare metal”.  When you start building full client apps in the browser, even simple things like displaying a list of items efficiently, can quickly consume endless hours of mind-numbing work to get it right.

The productivity issue can be solved pretty easily though.  You just need a framework that will do a lot of these basic tasks for you.  A good framework that leverages the natural strengths of Javascript and gets you 80% of the feature you need but stays out of your way for the other 20% you need to build can actually make writing these apps easy and a lot of fun.  

These frameworks are coming (SproutCore is one of these, of course), so I’m not to worried about that.  The even bigger challenge for client-server will not be a technical one, but a people one.  Many people in the web world do not have much experience with the client-server model.  Those that do consider this new-old pattern to be counter to the nature of the web.  Adopting this new approach requires some really mind-bending work that many people naturally resist.

In the end though, the fact is that the growing demands from users for fast, fuild, and feature-rich (3Fs) is going to demand that people begin to adopt this model.  Once the frameworks mature that make it easy to build these apps, then the pressure will really be on.  Expect to hear a lot of war stories as technical teams work through this philosophical shift and then expect some amazing work from those that get it early on.

The trends of history are clear.  Client-server and terminal-style technologies rotate in dominance every 15 years or so and client-server’s time has come.  The smart money is on client-server — coming soon to a browser near you.

 

New Feature: Inline Text Editing and SC.Editable

Two new features to announce and explain to day, both of them are related.

Edit Here…

Many times you want to display content in a list view and then click to edit it.  Now you can do this quite easily with the inline text editor.  To enable this feature on any label view just set isEditable: YES (or add :editable => true to the view helper).  

To enable this feature on a collection view, just set contentValueIsEditable: YES (or add :content_value_editable => true to the collection view view helper.)  For the collection view to work you must be using either SC.ListItemView or SC.LabelView for your example view type.  If you are using one of the built in classes such as SC.ListView or SC.SourceListView then this is already done for you.

To see an example of inline editing in action, just visit the sample controls demo and try clicking twice on list item in the sidebar for either the Collections or Collections2 tabs.  You can also select an item in one of these views and hit return to activate the editor.

Edit There…

Of course there is a lot more to this feature than just enabling inline editing in a few places.  You can also reuse the inline editor in your own views as well.  To activate inline editing, you just need to do two things:

 

  1. Implement the SC.InlineEditorDelegate protocol.  You can apply this mixin if you want but mostly you should override the four methods described here.
  2. When you want to begin editing, call the SC.InlineTextFieldView.beginEditing() method with any options you want to set.  You should at least provide a frame, which will tell the text field view where to size and position itself, and an exampleElement.  The exampleElement is DOM element that the text field view will use to clone its text styles.  This makes the inline editor automatically appear to similar to the element that it is supposed to be editing.

 

For an example of how to use the inline editor, checkout the source for SC.LabelView and SC.SourceListView.  It is really easy and generally takes only a few minutes to get going.

Edit Everywhere!

Paired with the new inline editor control is another new mixin that is going to be used throughout the framework for version 1.0 called SC.Editable.  This protocol defines a few standard methods that will can be used by other parts of your application to begin and end text editing of a control.  The collection view for example, looks for this protocol when it tries to begin editing the content value of an item view.  This is how you can trigger inline editing of a control in the collection view in a standard way.

This protocol is implemented by SC.LabelView, SC.SourceListItemView, SC.InlineTextFieldView, SC.TextFieldView, and SC.TextareaFieldView and you can implement it for your own views as well.  If you do implement it, then all of the built in cues that SproutCore recognizes to begin text editing will automatically work for your views as well.

As usual, for more info on these items, checkout the sample code using svn co http://sproutcore.googlecode.com/svn/trunk/samples, consult the reference documentation and ask on the Google Group if you have any questions.  Cheers!

 

SproutCore Tutorial

Once again I have stayed up WAY to late working on things that interest me.  But my pain is your gain and now I am happy to announce the first ever Hello World tutorial for SproutCore!

Follow the Tutorial »

I am sure there are horrible mistakes in this thing that I will regret in the morning but while I slumber you can try it out and send me caustic emails about the terrible mistakes I have made.  Or more likely, you will learn a thing or two about SproutCore.  

Seriously though, please give it a look over…follow the direction and let me know what you think!

New Feature: Timers

Speaking of Cocoa, I’ve added a new feature to SproutCore today very much inspired by Cocoa called Timers.

Timers are a better way to execute deferred code than setTimeout() and setInterval(), especially when you need to execute several different methods and you want them to stay in sync, such as during animation.  

The problem with timeouts is that they are both inefficient and non-deterministic.  That is, even if you set a timeout to execute in 100msec, you don’t know when the timeout will actually run.  It may be 100msec, it may be 150msec, it may be 1500msec.  Anything could happen.  To make matters worse, browsers get really slow when they start executing lots of timeouts, meaning the time ranges when your timeouts execute will start to vary even more.

Although we can’t completely overcome the non-deterministic nature of timeouts, Timers at least make it better.  Instead of setting one timeout per Timer, SproutCore sets only one timeout on an object called SC.runLoop.  This object manages all of the timers you have registered in the system.  When its timeout triggers, it will check your timers and fire any of them whose time has come.  Since this is all done in the context of a single timeout handler, timers that are set to expire at the same time will tend to fire together just like you might expect.

In addition to providing a more consistent firing experience, Timers also schedule their time intervals based on when the run loop currently began running.  This means that if you are in an event handler and you create three timers to fire in 100msec, they will all fire at the same time.

Using Timers is extremely simple.  You can easily create them yourself (see the documentation here), or you can use the invokeLater() method defined on both objects an functions.  Here is how you could invoke a method on an object after 100msec:

myObject.invokeLater('updateAnimation', 100) ;

If you just want to invoke a method the next runloop but as soon as possible, you can also omit the time:

myObject.invokeLater('updateAnimation');

Or sometimes you may have a function you’ve defined that you want to execute which is not attached to the object.  You can do that too:

function() {
  // handle animation
}.invokeLater() ;

If you want the function to execute in the context of an object you can do that also:

function() {
  // handle animation
}.invokeLater(myView, 100);

In the past if you wanted your timeout function to execute in the context of a specific object, you had to use bind() which created a nasty closure to do its work.  Thanks to the way timers are implemented, no closures are created during this process unless you pass a bunch of custom parameters (which is not the common case).

If you do want to pass custom parameters though, you can do so:

myObject.invokeLater('updateAnimationTo', 100, 'newState');

That’s it.  Timers are really easy to use and much more performant than setting your own timeouts.
 

Emulating Cocoa in JavaScript

The #1 thing I learned at the JS Meetup in San Francisco last week is that everyone seems to want Cocoa on the web.  I know of at least five frameworks now that try to essentially bring Cocoa to the web.  (Only SproutCore is open source so far afaik).

Cocoa is a great framework.  It makes writing applications on the desktop not only easy but fun.  There are a lot of good reasons to copy Cocoa for the web and a few really bad ones.  Here’s why I think it’s good:

 

  1. Cocoa is hackable.  Cocoa’s greatest strength is that it does everything the right out of the box 80% of the time, but because of how its late-binding object system works, it’s easy to hack it to do pretty much anything that you want.  Other frameworks, especially those with static bindings like Java and C++ are not nearly as easy to hack which is why they instead have to try to do everything, leading me to my next point…
  2. Cocoa is small.  Compared to the Swing APIs or to MFC, Cocoa is really a very small framework.  It has a handful of controls that can be configured to do most of what you want.  Since it is so hackable, Cocoa can get away with this.  Obviously on the web small is important for performance.
  3. Cocoa is clean.  Possibly the best reason to copy Cocoa is that for a 20 year old framework, the Cocoa team has done an outstanding job at keeping their API’s clean and consistent.  Using the same design patterns everywhere in a framework makes a huge difference to its usability.  Once you learn how to do things one way, you can assume they will work that way everywhere.

 

All of these things are good reasons to copy Cocoa for the web.  In many ways it already has built into its design some of the very things that are important to a web-based application.  However, not all in sunshine and roses with Cocoa, especially when it comes to bringing the framework to the web.  For example:

 

  1. Cocoa is built on Objective-C.  And fundamentally, it is built on C, a very static, strong typed language that leaves a lot to the developer to handle, including memory management.  A significant chunk of Cocoa’s functionality (i.e. most of Foundation) goes into overcoming this limitation with C.  Of course, JavaScript is a highly dynamic language with late binding and memory management built right in.  So copying a lot of this to the web is simply wasted space and processor cycles.
  2. Cocoa has cruft.  Like I said, for a 20-year old framework, Cocoa is pretty clean, but it does have some cruft built up.  For example, powerful new features such as bindings and automated garbage collection were just introduced in the last five years.  Bindings in particular make a number of older design patterns still supported by the framework unnecessary.  No doubt, if the team were starting from scratch today, building Cocoa with these new features from the beginning would yield and much smaller more concise framework.

 

So Cocoa is a great framework.  In many ways it has the kinds of things you would want to build apps on the web, which is a great reason to use it as inspiration.  That said, there are a couple of really terrible reasons why you might try to clone Cocoa as well, which unfortunately is what most of the frameworks I’ve seen so far seem to do.  Here’s what I hear from developers a lot of the time:

 

  • I don’t like JavaScript.  It’s true, JavaScript has a bad rep.  But like it or not, JavaScript is the third most widely distributed language on the planet.  It’s built into every web browser and it forms a fundamental part of the fabric of the web.  Not only that, but once you get to know it, JavaScript is actually one of the most dynamic, powerful languages to ever achieve mass adoption.  If you have ever wished you could just build things in smalltalk, objective-c, lisp, ruby, or python in the web browser, then you might be surprised to find most of the stuff you love about these languages is actually in JavaScript.
  • I really love Cocoa.  This comes frequently from people who have been building Cocoa apps for a long time and want to reuse the knowledge they’ve built up for their leap to the web.  Unfortunately, Cocoa was designed to overcome specific limitations with running on desktop machines and writing code in C that simply do not exist in the web world.  The reality is, not everything you learned from Cocoa is relevant for the web.  

Building a Cocoa-like framework for either of these reasons inevitably leads to a framework that tries to clone Cocoa method for method in spite of the fact that it is not being used in the same way.  This yields a framework that works against the grain of the web browser, not with it.  A framework that will do a lot of extra work unnecessarily just to maintain some abstraction, that will in the end break down at some point anyway due to the principle of leaky abstractions.  Ironically, these complications usually end up making Cocoa clones more difficult to use and less fun to code in; exactly the opposite of the intended effect.

Cocoa is a great framework.  It has a lot of lessons to teach us about building applications on the web, but the appropriate response to Cocoa is to be inspired by it, not to copy it.  SproutCore, for example, borrows many design patterns from Cocoa including MVC, Bindings, Delegates, Timers, Run Loops, and more.  But it doesn’t try to clone Cocoa method for method.  For example, SproutCore uses the built in Array class (with some enhancements), instead of trying to have an NSArray and NSMutableArray (which would both have to be mutable anyway).

The idea is to capture the essence of Cocoa (easy and fun to use), while still embracing the native tools of the web (i.e. JavaScipt), to yield a framework that feels at home on the web.  And that is the right way to bring Cocoa to the web.

 

 

Improved Documentation

Did you know that SproutCore has support for JSDoc-like documentation built right in?  If you use JSDoc compatible comments in your code, you can just visit http://localhost:4020/YOURAPPURL/-docs and click “Rebuild”.  In a few seconds you’ll have nice reference docs for your own code.

Of course, SproutCore is heavily commented in this way, though not everything has been converted over yet, it is pretty complete.  I’ve just updated the docs viewer to make it play nice with the latest version of the framework. It’s also reskinned so it is more pleasant to look a.

Checkout the latest SproutCore reference docs.

Oh and one more thing: the SproutCode Doc browser comes with a built-in symbol search.  Just type a method, property or class name you are interested in in the search field at the top and the source list to the left will filter.  I don’t have a nice Mac-like “X” icon showing yet to clear the search field so for now just delete the text to remove the filter.  

Just like the JSDoc tool itself, this nice doc viewer and symbol search are available for use in your app as well.