Building ExtJS app with fabric

How to make a custom build flow of ExtJS applications with fabric

6 minute read

The problem

ExtJS is a well known javascript framework for creating rich and powerful web applications. Basically it allows you to implement the whole user experience in javascript on client side in a way very similar to desktop widget toolkits. The ExtJS comes both as GPL and commercial and has several tools used to work with it.

First of all there is a stand alone development IDE with interface designer called Architect. And there are several command line SDK tools which are free and basically the foundation of the IDE. The designer is a separate product which you need to pay for if you want to use it for javascript development. I haven’t bought it mostly because I prefer using my Textmate to do everything but there is a catch (Yea right…) So the thing is that you need to minimize and obfuscate your javascript to reduce the impact on a page load time and this is where SDK tools come in action. There is a tool called JSBuilder that is used to combine each separate js file into one big file and then minimize it. If you’re using designer to write your js code then its all fine because it has a project file (.jsb, JSBuilder right), which is a JSON file with references to all of your application code files. JSBuilder reads that file and does the magic. The problem for me was that I don’t have designer and I didn’t want to maintain jsb file manually. It’s format is pretty clear but still to much explicit to my taste and the project I’ve been working on has a lot of js in it. Really a lot, so manual update on each and every touch to a project tree was not possible.

ExtJS project files

There is another way of getting jsb files, since SDK tools can create it for you by loading your main app.js into fake browser and querying Ext.Loader for the list of loaded files and the order there were loaded. It wasn’t an option for my case since loading my app on itself without server side providing a lot of apis and statics at the load time wasn’t possible. So what? Well, I had to roll my own jsb creator script which would simply ready my project tree and produce jsb file which JSBuilder would use later on. Also here in Anturis we’re using fabric to build, package and deploy our service so my thing had to be implemented as a fabric task. So here it is.

The the assumed idea is that source tree itself is a definition of the application structure, where each folder and subfolder is a module. Another assumption comes from the way how JSBuilder works. Since it combines all of the js files into single one the order of class definitions in that file matters a lot. I believe that designer must be maintaining the order in jsb file itself by paying attention to require argument of every class. Or maybe it doesn’t, don’t know for sure since I don’t have it anyway. So back to build-your-own-jsb thing… each module is compiled into a separate file before being merged into final file. In order to make sure that requirements for each class are available I had to specify the order of modules in some way. Now since ExtJS gives you a very rich components model and suggests using MVC to implement your application logic is already was split into models, controllers, stores, utils and whatever else common stuff you made. And now all you need is to make sure that classes in the particular module do not require each other (you may need to make several common sub-modules in each module together with top-level common) and specify a the order by cascading the dependencies from the most common to the most specific. Ok, thats the theory now here is what I’ve got:

The solution

First we need a file to specify the order of modules and submodules, something like this:

{
    "modules": [
        "util",
        "model",
        "store",

        "view/common",
        "view/form/common",
        "view/form",
        "view/something",
        "view/somethingelse",
        "view/foo/common",
        "view/foo",

        ...etc...

        "controller"
    ],
    "scripts": [
        "app/login.js",
        "app/somescript.js",
        "app/baz.js",

        ...etc..

        "lib/ajax.js",
        "lib/json2.js"
    ]
}

I keep such files in the app folder, one where app.js lives. The scripts section is used to specify stand-alone scripts for yuicompressor (it comes with SDK tools but originally from Yahoo).

Fabric script

Next we need to create fabric script to use it in a build stage for an application in the fabfile. Something like this:

import os
import sys
import json
from fabric.api import *

#local tools
env.jsbuilder_path = '~/tools/jsbuilder/JSBuilder.sh'
env.ycompressor = '~/tools/yuicompressor-2.4.7/build/yuicompressor-2.4.7.jar'

def build_jsb_file(local_js_path, app_name, modules = None):
    if modules == None:
        modules, scripts = get_build_info(local_js_path, app_name)

    print 'building jsb from ' + local_js_path

    jsb_content = {
        'projectName': '%s' % app_name,
        'licenseText': 'Copyright(c) MegaCorpThatRocks',
        'packages': [{
            'name': 'Application',
            'target': '../build/pkgs/%s/app.js' % app_name,
            'id': 'application',
            'files': [{
                'path': '',
                'name': 'app.js'
            }]
        }],
        'builds': [{
            'name': '%s release' % app_name,
            'target': '%s.js' % app_name,
            'options': {
                'debug': False
            },
            'compress': True,
            'packages': [m.replace('/', '-') for m in modules]
        }]
    }

    for module in modules:
        print 'processing module ' + module
        module_path = os.path.join(local_js_path, app_name, module)
        files = os.listdir(module_path)
        module_name = module.replace('/', '-')
        module_info = {
            'name': module_name.capitalize(),
            'target': os.path.join('../build/pkgs', app_name, module_name + '.js'),
            'id': module_name,
            'files': []
        }
        for f in sorted(files):
            if f.startswith('.'): continue
            if os.path.isdir(os.path.join(local_js_path, app_name, module, f)):     continue
            print 'processing file ' + f
            module_info['files'].append({
                'path' : module + '/',
                'name': f
            })
        jsb_content['packages'].append(module_info)

    for build in jsb_content['builds']: build['packages'].append('application')

    jsb_file_path = os.path.abspath(os.path.join(local_js_path, app_name, '%s.jsb'     % app_name))
    print 'jsb file at ' + jsb_file_path
    jsb_file = open(jsb_file_path, 'w')
    jsb_file.write(json.dumps(jsb_content, indent=4))
    jsb_file.close()

def get_build_info(local_js_path, app_name):
    build_file_path = os.path.join(local_js_path, app_name, 'build.json')
    if not os.path.exists(build_file_path):
        print 'Build file not found at %s' % build_file_path
        sys.exit(1)

    with open(build_file_path, 'r') as build_file:
        build_info = json.loads(build_file.read())

    if not 'modules' in build_info:
        raise Exception('No app modules defined')
    if not 'scripts' in build_info:
        raise Exception('No scripts defined')

    scripts = build_info.get('scripts')
    modules = build_info.get('modules')
    if not modules:
        raise Exception('Specified app_modules are empty')

    return modules, scripts

def build_js_all(local_js_path, jsbuilder = env.jsbuilder_path, ycompressor =     env.ycompressor):
    for app_name in ['app', 'admin']: 
        build(local_js_path, app_name, jsbuilder, ycompressor)

def build(local_js_path, app_name, jsbuilder = env.jsbuilder_path, ycompressor =     env.ycompressor):
    modules, scripts = get_build_info(local_js_path, app_name)
    build_jsb_file(local_js_path, app_name, modules)
    build_app(local_js_path, app_name, jsbuilder)
    build_scripts(local_js_path, scripts, ycompressor)

def build_app(local_js_path, app_name, jsbuilder = env.jsbuilder_path):
    deploy_dir = os.path.join(local_js_path, 'release')
    if not os.path.exists(deploy_dir): os.makedirs(deploy_dir)
    jsb_file = os.path.join(local_js_path, app_name, '%s.jsb' % app_name)
    local(jsbuilder + ' -p ' + jsb_file + ' -d ' + deploy_dir)

def build_scripts(local_js_path, scripts = None, ycompressor = env.ycompressor):
    if scripts == None:
        modules, scripts = get_build_info(local_js_path)

    if not os.path.exists(os.path.join(local_js_path, 'release')):
        os.makedirs(os.path.join(local_js_path, 'release'))

    compressor_cmd = 'java -jar ' + ycompressor
    for script in scripts:
        script_dir = os.path.dirname(script)
        if script_dir: 
            script_dir = os.path.join(local_js_path, 'release', script_dir)
            if not os.path.exists(script_dir):
                os.makedirs(script_dir)
        local(compressor_cmd + ' ' + os.path.join(local_js_path, script) + ' -o ' +     os.path.join(local_js_path, 'release', script))

def clear_release(local_js_path):
    local('rm -fr %s' % os.path.join(local_js_path, 'release'))

def clear_sources(local_js_path):
    for path in os.listdir(local_js_path):
        if not path in ['release', 'ext-4.0']:
            local('rm -fr %s' % os.path.join(local_js_path, path))

The idea is to read build.json with modules order and produce jsb, then call JSBuilder to compile raw js into minimized and obfuscated version and then remove original raw javascript. Also it processes stand-alone scripts with yuicompressor.

And finally in the fabfile you can add a task to build your ExtJS frontend:

env.frontend_deployed_location = '~/server'
env.frontend_javascript_path = '/frontend/static/js'

@task
def frontend_build(branch = 'trunk', rev = 'HEAD'):
    build_dir = checkout_sources(branch, rev)
    local_js_path = os.path.join(build_dir, 'server', env.frontend_javascript_path)
    deploy_js_path = os.path.join(env.frontend_deployed_location, env.frontend_    javascript_path)
    build_js_all(local_js_path)
    clear_sources(local_js_path)
    env.tarball = create_tarball('%s-%s' % (branch, rev), build_dir)
    print 'redist tarball at ' + env.tarball

Thanks, that is all for today. Now go back to work.

comments powered by Disqus