Bootstrap launches on GitHub: Free for everyone!

Go and grab it!

Get Bootstrap on GitHub

About GitHub

GitHub is a distributed version control system similar to Mercurial with a free Mac client. GitHub makes it easier to collabortate with others and share projects with the universe. GitHub has grown into an application used by over a million people to store over two million code repositories, making GitHub the largest code host in the world.

Goal for Bootstrap: “To help ease with starting up new projects in Python on Django with jQuery, Twitter’s Bootstrap, Modernizr & Handlebars.js.”

If you have any suggestions how to make this solution better please Leave me feedback.

Cool things you will find inside

  • Plenty of internal documentation to why something is being done. Also utilize the “Ref:” URLs. They are there as a reference for the source of the content. Great for digging deeper into those specifics.
  • split into Development & Production modes based on platform node. The base template contains the platform node wrapped in comment declaration so you can view source and see if you are running in development or production mode.
  • HTTP error 404, 500 & 403 handlers with associate Templates – these are bare atm so feel free to expand and contribute better versions. See Quora for funniest 404 pages.
  • If you add {% include ‘django_debug.html’ %} to your base.html Django Template footer you will see Debugging information in the footer of all your pages. Not recommended in Production mode.
  • Handlebars & Django Template consistencies to help use.
  • Custom Handlerbars Helpers to help with index counting in a grid table and generating Twitter Bootstrap buttons.
  • All JavaScript files are loaded using Modernizr’s yepnope, a asynchronous conditional resource loader.
  • Few custom JavaScript modules using the Modular Pattern. I’m a big fan of the Modular Pattern due to its encapsulation, namespacing and seperation of concern possibilities.
  • What custom Modular JavaScript you will find under /media/js/
    • – highlight the correct primary navigation tab based on page visited.
    • – wrapper for Google Analytics (GA) to automatically tag all links on page by parsing the DOM for href. Also uses “data-ga” attribute (where present) to push custom labels for GA event tracking.
    • – for HTML5 supported browsers the JSON load uses either the browser local or session store to work with the JSON data. Default is always local store. Great for use with JSON/API call data you want to cache locally vs constantly making calls to the API – see /about/ page for this example in use.

Frameworks used

  • Django: Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.
  • jQuery: The Write Less, Do More, JavaScript Library: A fast, concise, library that simplifies how to traverse HTML documents, handle events, perform animations, and add AJAX.
  • Bootstrap, from Twitter: Simple and flexible HTML, CSS, and Javascript for popular user interface components and interactions.
  • Modernizr w/ yepnope.js: Modernizr is an open-source JavaScript library that helps you build the next generation of HTML5 and CSS3-powered websites. yepnope is an asynchronous conditional resource loader that’s super-fast, and allows you to load only the scripts that your users need.
  • Handlebars.js: Minimal Templating on Steroids: Handlebars provides the power necessary to let you build semantic templates effectively with no frustration. Mustache templates are compatible with Handlebars, so you can take a Mustache template, import it into Handlebars, and start taking advantage of the extra Handlebars features.

Thanks to

makinacorpus (Makina Corpus) for custom template tag code to get around handlebars mustache tags which are natively parsed by Django templating engine. Ref:


The following are blog posts I have written in the past which will help you when working with the code in Bootstrap

A bit about the author Ernest Semerda

Bsc. Comp Sci & AGSM MBA. I Lead & Code for Prior software architect at AMP Ltd, later CTO of Couponstar Ltd followed by an acquisition by, a $1B company. I am an Aussie entrepreneur & software engineer working on refreshing & innovative products in Silicon Valley like GTDfaster, Facebook ReadingList app et al.

This is my personal website:

I am responsible for the following properties

  • The Road to Silicon ValleyThis blog. A Blog which provides tools and resources to anyone wanting to come and work in Silicon Valley’s high tech industry. Experience is based on my own personal journey arriving in Silicon Valley on March 2009.
  • GTDfaster iPhone app – GTDfaster rests on David Allen’s principle of Getting Things Done (GTD). That a person needs to move stuff out of the mind by recording them externally into a trusted source. GTDfaster is this trusted GTD source. Also has a seperate posterous blog on GTDfaster.
  • Facebook Readinglist app – A popular free Facebook app to help you share, inspire & create a collection of books you have read, want to read or are reading. Start influencing your friends on Facebook today.

To-do list for Bootstrap

There is always a never ending to-do list. If you want to dd anything to it please feel free to.

  • Separate Handlebars Helpers into Standard & Twitter specifics.
  • Give SEMERDA prototype object a unique handle like jQuery does with $. Maybe use “$S”. This should shorten the calls a bit.
  • Throw in more Modernizr examples for Browser detection versus feature detection. Always aim for browser detection hence why we are using Modernizr here. We do not want to block users but gracefully degrade the experience where required.
  • Provide a set of minified custom JavaScript modules. For now just use [Google Closure Tools]( to do the minification.
  • Add a real API vs calling /json/my_data.json dummy data.
  • Add some funny custom 404 pages. Inspired by Quora for funniest 404 pages.
  • Add Django Debug Toolbar – A configurable set of panels that display various debug information about the current request/response.
  • Add Python Modules examples and associated Unit Tests.
  • Add Django Middleware example and associated Unit Tests.
  • Cleanup/feedback from users – that’s you 🙂

Go and grab Bootstrap and enjoy the ease of building powerful Python based apps. Bootstrap primetime:

~ Ernest

Gentle Introduction to Python

Right, let’s dig into my favorite language. Python. It’s super easy to read & learn, it’s concise and one of the hot languages in Silicon Valley. In fact, Python is also one of the easiest languages to grasp if you want to learn to code on mobile.

The following assumes you understand basic software engineering concepts.

A bit about Python

  • Design philosophy emphasizes on code readability. Important because software engineers spend most of their time trying to understand code. (Ref Coding Horror)
  • Has a nice MVT open-source web framework called Django. Django emphasizes reusability and pluggability of components, rapid development, and the principle of DRY (Don’t Repeat Yourself).
  • It features a fully dynamic type system with late binding (duck typing) and automatic memory management, similar to that of Scheme, Ruby, Perl, and Tcl. More here.
  • Runs on LAMP, where the P = Python. Here’s how to set it up.
  • Currently one of the hottes languages (alongside Ruby/Ruby on Rails) in Silicon Valley especially among startups.

Sample of popular sites build in Python

Google, Dropbox, Reddit, Disqus, FriendFeed (Sold to Facebook to drive their News Feeds), YouTube, Quora (rising star), Douban. Comprehensive list here.


  • Uses whitespace indentation, rather than curly braces or keywords, to show & delimit block structure. I prefer 4 spaces.
  • Everything is an object (first class) and everything has a namespace accessed by dot-notation.
  • Naming convention UpperCamelCase for class names, CAPITALIZED_WITH_UNDERSCORES for constants, and lowercase_separated_by_underscores for other names. See Python style guide and The Zen of Python for guiding principles for Python’s design into 20 aphorisms. Basically write self-documenting code by chosing explicit naming convention.
  • A comment starts with a hash character (#). For longer then a line (and as Doc strings) use triple quotes: ”’ xyz ”’.
  • Variable names have to start with a letter or underscore, and can contain numbers but no spaces or other symbols.
  • File extension is always .py. If you see .pyc this is source code compiled into bytecode for execution by a Python VM (virtual machine).
  • Use command line python shell to test assumptions by getting immediate results.
  • No case/switch statements. Switch is better solved with polymorphism (object that has more than one form) instead. Good example here.
  • Data types
    • Immutable (can’t be updated or changed): strings, tuple, int, float, complex, bool
    • Mutable (can be updated or changed): list, dictionary (dict) & mutable except for it’s keys
  • Editors I use: IDLE (for basic shell work & comes with install), PyCharm (with Django support) and Sublime Text 2 (lightweight TextMate replacement).


Arithmetic Boolean
2 > 3 → False
2 == 3 → False
2 The opposite of == is != (“not equals”):
2 != 3 → True
You can chain together comparison operators:
2 < 3 < 4 → True
Equality works on things besides numbers:
“moose” == “squirrel” → False
True and True → True
True and False → False
True or False → True
not False → True
(2 < 3) and (6 > 2) → True
Under the hood,
True is equal to 1,
and False is equal to 0.
Booleans are a subtype of integers.


== Equal to
!= Not Equal to
is Identical
and Boolean and
or Boolean or
& Bitwise and
| Bitwise or
not Boolean not (not the !)

Built in functions that are always available

len(s) Return the length of an object. Can also be a sequence (string, tuple or list) or a mapping (dictionary).
print(obj) Print object(s) to the stream file.
help(list) See basic help on any object.
dir(list) Return a list of valid attributes for that object.
type(list) Return the type of an object

More built in functions here:


Always starts with a “def” and ends with “:”.

# define a new function with 1 default argument. Can also have no arguments.
def function_purpose(arg1=1):
     ''' This is a doc string '''
     print 'Python code'
     return (arg1, arg1+7,) # returns 2 values as a tuple (note the comma), else None
# call the function, returns a tuple that we assign to 2 variables
item1, item2 = function_purpose(1)

If you want to assign a value to a variable outside the function within a function you must prepend the variable with “global”.

Calling methods on objects

Just like calling functions, but put the name of the object first, with a dot

words = 'some monkeys here'
e = words.count('e')
# returns 4


Are a sequence of characters.

# creation
name = 'Ernest Semerda'

# accessing, returns 's'

# splitting, returns a list ['Ernest', 'Semerda']
the_string.split(' ')

Strings can be subscripted/sliced like the list (see lists in Data Structures below).

# selected range returns 'nest '

# get first two characters returns 'Er'

# get everything except the first two characters returns 'nest Semerda'

Sample of  some string methods. They come with 8-bit & Unicode support.

name.capitalize() # changes to 'ERNEST SEMERDA'
name.find(sub[, start[, end]])
name.split([sep[, maxsplit]]) and new_name.join(list)

More string methods:

Data Typing

Python is strongly typed which won’t allow you to automatically converted from one type to another.

Python also has a strong tradition of duck-typing (dynamic typing) in which an object’s current set of methods and properties determines the valid semantics. Trusting that those methods will be there and raising an exception if they aren’t. Be judicious in checking for ABCs and only do it where it’s absolutely necessary.

An important feature of Python is dynamic name resolution (late binding), which binds method and variable names during program execution.

# fails because (str + int + str) != str
'There are ' + 8 + ' aliens.'
# perfect, str() = type conversion
'There are ' + str(8) + ' aliens.'

To achieve Reflection, a process by which a computer program can observe and modify its own structure and behavior, use the built-in functions. I.e. getattr

Over a “sys” module’s method “path”:

path = getattr(sys, "path")

Over a function1 with sample input:

result = getattr(sys.modules[__name__], "function1")("abc")

And/or use the Reflection Utilities API for deeper execution frame, execution model, class/obj inspection for methods & attributes etc… See:

Data Structures


Set of key:value pairs. Keys in a dictionary must be unique. Values Mutable.

# creation, empty dictionary
peopleDict = {}

# creation, with defaults
aliensDict = {'a':'ET', 'b':'Paul', 'c':42}

# accessing, returns 'ET'

# deleting, 'Paul' is removed from dictionary
del alientsDict['b']

# finding, returns False (note capital F)

# finding, returns ['a', 'c']

# finding, returns [('a', 'ET'), ('c', 42)]

# finding, returns True
'c' in aliensDict


Lists can carry any items ordered by an index. Lists are Mutable.

# creation, empty list
peopleList = []

# creation, with defaults of any type
codesList = [5, 3, 'p', 9, 'e']

# accessing, returns 5

# slicing, returns [3, 'p']

# finding, returns ['p', 9, 'e']

# finding, returns [5, 3]

# returns ['p', 9]

# length, returns 5

# sort, no return value

# add

# return, returns 37

# remove, returns 5

# insert
codesList.insert(2, 'z')

# remove

# delete
del codesList[0]

# concatenation, returns ['z', 9, 'p', 0]
codesList + [0]

# finding, returns True
9 in codesList

Apply set(list) and it becomes a set – an unordered collection with no duplicate elements. Also support mathematical operations like union, intersection, difference, and symmetric difference.


Tuples are similar to lists: they can carry items of any type & useful for ordered pairs and returning several values from a function. Tuples are Immutable.

# creation, empty tuple
emptyTuple = ()

# note the comma! = tuple identifier
singleItemTuple = ('spam',)

# creation, with defaults of any type
codesTuple = 12, 89, 'a'
codestuple = (12, 89, ‘a’)

# accessing, returns 1

More on data structures here:

Control & Flow

For loop

# Collection iterator over dictionary w/ tuple string formatting
people = {"Ernest Semerda":21, "Urszula Semerda":20}
for name, age in people:
    print "%s is %d years young" % (name, age)

To loop over two or more sequences at the same time, the entries can be paired with the zip() function.

More on string formatting operations here:

For loop with if else

# Iterate over a sequence (list) of numbers (1 to 10) with if/else Conditionals. The range function makes lists of integers.
 for x in range(1, 10):
     if x == 8:
         print "Bingo!"
     elif x == 10:
         print "The End"
         print x

While loop

# using request to ask user for input from interactive mode
request = "Gimme cookie please: "
while raw_input(request) != "cookie":
    print "But me want cookie!"

Switch-statements do not exist. In OO they are irrelevant & better solved with polymorphism instead. Examples here.

More control flow tools here:


Chaining into few lines.

[x * x for x in [1, 2, 3, 4, 5]]
# returns [1, 4, 9, 16, 25]

Can get messy & complicated to read.

print [x * x for x in range(50) if (x % 2 ==0)]
def is_palindrome(word):
    word = re.compile(r'[!? ]').sub("", word.lower())
    return True if word == word[::-1] else False


# open, defaults to read-only + note single forward slash
contents = open('data/file.txt')

# accessing, reads entire file into one string

# accessing, reads one line of a file

# accessing, reads entire file into a list of strings, one per line

# accessing, steps through lines in a file
for line in contents:
    print line

More on IO:


All methods (but not functions) are closures – see “self” below. A closure is data attached to code. All variables are public, private variables are established by convention only.

# SuperHero inherits from Person class - also supports multiple inheritance using comma
class SuperHero(Person):
    # constructor
    def __init__(self, name):
        self._name = name

    # method
    def shout(self):
        print "I'm %s!" % self._name

The __name__ below allows Python files to act as either reusable modules, or as standalone programs. Also think Unit Tests benefits!

if __name__ == '__main__':
    # instantiate the class
    batman = SuperHero('Batman')

    # call to method in class, returns "I'm Batman!"

    # returns "I'm Batman!"

More on Classes in Python here:


Modules are Libraries that hold common definitions and statements. They can be combined into an importable module.
More on modules here:

To use a module, use the import statement:

import math

# returns 1.0
math.sin(math.pi / 2)

Some commonly used modules

  • math – trigonometry, the constants e and pi, logarithms, powers, and the like.
  • random – random number generation and probability distribution functions.
  • os – tools for talking to your OS, including filesystem tools in os.path.
  • sys – various system information, as well as the handy sys.exit() for exiting the program.
  • urllib2 – tools for accessing Web resources.

Useful modules:

Error & exception handling

import sys
    f = open('myfile.txt')
    s = f.readline()
    i = int(s.strip())
except IOError as (errno, strerror):
    print "I/O error({0}): {1}".format(errno, strerror)
except ValueError:
    print "Could not convert data to an integer."
    print "Unexpected error:", sys.exc_info()[0]


Fun – easter egg; The antigravity module

Released in Google App Engine on April 7, 2008. The antigravity module ( can be enabled like this:

import antigravity

def main():

if __name__ == '__main__':

Speed – always a common topic

Classic computer programs had two modes of runtime operation = interpreted (as code runs) or static (ahead-of-time) compilation.

Just-In-Time compilation (JIT), also known as dynamic translation is a new hybrid approach. It caches translated code (bytecode into native machine code) to minimize performance degradation. Used in .NET, Java & Python via PyPy.

PyPy is a fast, compliant alternative implementation of the Python language. It has several advantages and distinct features like Speed (Just-in-Time JIT compiler), Memory usage (better then CPython), Compatibility (works with twisted & django frameworks), Sandboxing (run untrusted code), Stackless (providing micro-threads for massive concurrency). Check it out:




Finally, it is important that you have a network of like minded people around you whom you can regularly work on Python with, bounce ideas & question and support (help) each other out.

Happy Learning and if you have any questions please contact me. Always happy to help.

~ Ernest

J.J. Abrams’ Bad Robot releases iPhone special effects app

J.J. Abrams’ company Bad Robot has released an awesome iPhone app called Action Movie FX.

J.J. Abrams is one of my favorite Directors known for awesome movies Super8, Armageddon, Mission Impossible – Ghost Protocol, Cloverfield and kick ass TV series like Fringe, Lost & Alias etc.. He is an awesome American film and television producer, screenwriter, director, actor, and composer. J.J. Abrams makes kick ass movies and now this kick ass iPhone app. Without further ado, here’s what you too can create at home with your iPhone and Bad Robot’s Action Movie FX app.

Starring.. yours truly. Me! Ernest Semerda. Maybe I should get into the entertainment industry 🙂

Tornado hits Hacker Dojo!

Ula sent a cavalry to the Dojo.. to teach Ernest a lesson!

Ula in training!

Ula was naughty & needed to be taught a lesson 😉

Chopper touch down

Download Action Movie FX iPhone app and start making your own flix!

~ Ernest

How to setup Django + mod_wsgi on Apache & Ubuntu

Django is a great Web Framework which complements the Python programming language. Having switched to Python development few months ago I was on the lookout for a MVC like Web Framework that emphasizes Convention over Configuration (CoC), Rapid Development Principle of Don’t Repeat Yourself (DRY), Separation of concerns has Python as the underlying core language. And lo and behold Django proved to be that beast.

What is Django

Django is a MVT (model-view-template) framework build on the Python language. The MVT is identical to MVC (model-view-controller) but where in the common MVC pattern the view here is the template and controller the view. They both perform the same function separating the business logic from the presentation from the data layer (model). However Django does goes one step further and provides a view/template without native core language polluting the HTML.

Django emphasizes Reusability and “Pluggability” of components, Rapid Development, and the Principle of DRY (Don’t Repeat Yourself). Stuff you hear a lot with Ruby on Rails – another great web framework built on a solid language, Ruby.

Ok, enough of my rambling about how great this framework is. If you want to read more about it I recommend the following 2 links:

The LAMP stack

Ok so everyone is talking about a LAMP stack and everyone wants to run their site on a LAMP stack. Why not when you can tap into great open source tools for free. The word LAMP is a bit over emphasised since most sites today run on a custom configured implementation using best of breed open source software.

“LAMP is an acronym for a solution stack of free, open source software, referring to the first letters of Linux (operating system), Apache HTTP Server, MySQL (database software) and PHP (or sometimes Perl or Python), principal components to build a viable general purpose web server” ~ Wikipedia

Few years ago when open source software was limited, LAMP had typically meant Linux, Apache, MySQL and PHP. Today Linux can take many flavours from CentOS, Red Hat or the popular Debian-derived Linux distribution Ubuntu. The web server Apache can be replaced with a super fast & lighter HTTP and reverse proxy server called nginx. MySQL, post Oracle acquisition is often replaced with PostgreSQL or a NoSQL version called MongoDB. And PHP by cleaner & concise languages like Python or Ruby.

In this post I will cover how to setup a LAMP stack where:

  • Linux = Ubuntu 11.10. This can also work with 10.xx version of Ubuntu.
  • Apache = we will leave it as is since it is still a great web server but hook in mod_wsgi – a Python WSGI adapter module for Apache. WSGI is nothing more than an interface specification by which server and application communicate. In this case with Python.
  • MySQL = will stay with MySQL but you can also install MongoDB.
  • PHP = will be replaced with Python and drop in the Django MVT framework.

Ok let’s get started.

How to Setup Django on Ubuntu

The following assumes you have already installed Ubuntu and LAMP. If not head over to my prior posts on how to:

  1. Install Ubuntu on EC2 and
  2. Install LAMP.

Installing pre-req components

1. Standard procedure is to always run update on your Ubuntu installation.

sudo apt-get update

2. Install mod_wsgi on Apache

sudo apt-get install libapache2-mod-wsgi

3. Install Python setup tools with pip. pip installs packages. Python packages. An easy_install replacement.

sudo apt-get install python-setuptools
sudo apt-get install python-pip

4. Install Django MVT framework using pip.

sudo pip install Django

Now that’s done. Easy hey. Now let’s configure your 1st Django site.
Assuming your site will be called “

Configuring your 1st Django site

5. Recall from the LAMP setup that we were storing all web sites in /var/www/ folder. For example purposes, I will use purpleblue_com as my Django project. Notice the use of to create the Django site and the use of underscores vs decimal point. Only numbers, letters and underscores are allowed in the Django project name.

cd /var/www/
sudo startproject purpleblue_com

5.1 Verify new project by typing this in and hitting enter. ls lists the contents of a directory showing all files inc hidden ones, hence the -all option.

ls –all

If you see purpleblue_com listed, then change directory to it.

cd purpleblue_com

and verify you can see 4 .py files – init, manage, settings & urls. Those are Django new project default install files.

6. Now we create a wsgi file for the site so apache knows how to run this site.

sudo mkdir /var/www/purpleblue_com/apache
sudo nano /var/www/purpleblue_com/apache/django.wsgi

… and add this into django.wsgi

import os
import sys

path = '/var/www/purpleblue_com'
if path not in sys.path:
    sys.path.insert(0, '/var/www/purpleblue_com')

os.environ['DJANGO_SETTINGS_MODULE'] = 'purpleblue_com.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

TIP: It’s always a good idea to specify the actual application module.

6. Using webmin (ref our LAMP install) setup new apache site for this project.

6.1 Go to: https://YOUR_DEV_VM_IP:10000/
6.2 Servers > Apache Webserver > Create virtual host
6.3 Configure your new host
Document Root = /var/www/purpleblue_com/
Server Name =
6.4 Click on “Create Now” button. And go back into the settings by clicking on “Virtual Server” button next to your new host.
6.4 Click on “Edit Directives” and paste the following in:

DocumentRoot /var/www/purpleblue_com

<Directory /var/www/purpleblue_com>
    Order allow,deny
    Allow from all

WSGIDaemonProcess purpleblue_com processes=2 threads=15 display-name=%{GROUP}
WSGIProcessGroup purpleblue_com

WSGIScriptAlias / /var/www/purpleblue_com/apache/django.wsgi

The last 3 lines above allow this Django site to run in daemon mode.

TIP: When you make any .py changes in your Django site, rather then restarting/refreshing apache you can touch this wsgi and the changes will be picked up.

sudo touch /var/www/purpleblue_com/apache/django.wsgi

7. Restart apache to take new site & settings into effect.

sudo /etc/init.d/apache2 restart

8. Add the new domain to your primary OS host file pointing to your VM where the Django project purpleblue_com resides.

9. Test your new project by going to the Server Name ( you just setup. You should see a similar default Django installation page:

It worked! Congratulations on your first Django-powered page.

If you get an error or cannot see the default Django installation page make sure you check out apache log file. Common approach is to grab the tail end of the apache error log file to see what just happened.

tail /var/log/apache2/error.log

Bonus – Django CMS

If you have the time also check out Django CMS that is built on top of the Django Web Framework.

Django CMS site:

Happy Django exploring & Python coding!


Silicon Valley Code Camp 2011

Last weekend myself and Omid Mozayani (another Aussie; recent addition to the valley) spent the weekend of Saturday and Sunday, October 8th and 9th, 2011 at Silicon Valley Code Camp 2011. Held at the Foothill College in Los Altos/CA with an attendance of well over 2,000 (registered 3,417) people and 209 sessions organized in a University style setup over these 2 days. Impossible to attend each one but plenty to cherry pick from. It felt like I was back at Uni(versity) running between lectures. I loved it! Here’s what happened.

What is Code Camp

Code Camp is a new type of free community event by and for the developer community where developers learn from fellow developers. It is 2 full days of talking about code with fellow developers. Sessions range from informal “chalk talks” to presentations. Basically there is lots of education, networking and good food.

Pictures from Code Camp @ Foothill College

Lecture – Crockford on ECMAScript5

Lunch – feeding time for thousands of nerds

Ernest Semerda – nerd out and about

Lecture – multi threaded design

More pictures here on my Flickr.

Code Sessions

Choosing what sessions to attend was key since it was impossible to attend everything when you consider that each session took around 45 min. Here is a wrap of the 3 areas I focused on; ECMAScript (JavaScript) with code encapsulation, Writing Clean Code and Web Analytics. This website will highlight key points.

ECMAScript 5 (JavaScript)

Douglas Crockford ran 2 sessions on ECMAScript 5: The New Parts and ECMAScript: What Next?. Crockford is involved in the development of the JavaScript language and popularizing the data format JSON (JavaScript Object Notation), and for developing JSLint. You may recall me talking about JSLint in a previous blog post on JavaScript Patterns.

More on Crockford is here (his personal site) and here: Crockford on JavaScript. Also read about how JavaScript is the world’s most misunderstood language. Recommended reading! You will want to learn more about this great language.

ECMAScript’s Fifth Edition is now available in all of the best browsers. This is what you are getting:

  • Use “Strict mode” in all your JavaScript code.
    • As per my previous post on JavaScript Patterns here.
    • Old JS compiler ignores it so it’s safe to use today in all your JavaScript.
    • A bug in JS used to advantage today for backward compatibility.
    • “Strict mode” will break old bad programs. If you are writing good programs they will just work.
  • No more implied global variables within functions.
    • Yipee! Major fail in JavaScript where undefined variables would chain to the prototype making it very difficult to isolate your code, and to reuse it in new contexts.
    • If you forget to “var” a variable it now becomes undefined so you can address it vs. letting it slide by. Bad coders beware!
    • To append something to the global object add “window.function/var”.
  • “apply” and “call” do not default to the global object.
  • No “with” statement. Read why it was considered harmful to use.
  • Restrictions on “evil” eval. Here’s why it’s evil. *Cough* code injection!
  • No more octal literals. Not needed since punch card days.
  • Forgetting to use the new prefix will now throw an exception, not silently clobber the global object.

Use something like this today to check if your browser is running in Strict mode:

function in_strict_mode() {
    return (function () {
        return !this;
  • Make it a habit to do static analysis using JSLint. If you are using it already your code is probably good and ready for strict mode.
  • Use “Safe JavaScript Subsets” like Caja & ADsafe projects which allow you to safely run 3rd party software on your website.

JavaScript Code Organization and Encapsulation

Shawn Van Ittersum presented ideas for organizing, encapsulating, and simplifying JavaScript code, including: creating JavaScript objects and classes with truly private members, using Douglas Crockford’s module pattern and classical inheritance constructs provided by Dean Edward’s Base.js, Prototype, and MooTools; abstracting JavaScript code from HTML templates, CSS styling, and the DOM; designing with polymorphism (duck typing) for code reuse and simplicity; and using JSLint to identify problems in your code before runtime.

The idea behind the power of Encapsulation is simple:

  • Encapsulation limits widespread interconnection,
  • Gives you Freedom to change internal implementations without affecting external code and
  • is referred to as Loose Coupling – an approach viewed as positive in the field of computer science to remove dependency between systems and allow horizontal scaling.

Ittersum’s lides with code samples are located here.

Writing Clean Code

Theo Jungeblut ran this session and basically the crust of the presentation was that writing clean code makes us more efficient.

This presentation was based on C# and Visual Studio 2010. However the following patterns and practices can be applied to every other programming language too.

Over the lifetime of a product, maintaining the product is actually one – if not the most – expensive area(s) of the overall product costs. Writing clean code can significantly lower these costs. However, writing clean code also makes you more efficient during the initial development time and results in more stable code.

  • Readability
    • Follow coding guidelines.
  • Simplification and Specialization
    • KISS (Keep it simple, Stupid!) – most systems work best if they are kept simple rather than made complex.
    • SoC (Separation of concern) – focus on 1 thing.
    • SRP (Single responsibility principle) – every object should have a single responsibility, and that responsibility should be entirely encapsulated by the class.
    • OCP (Open/closed principle) – software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification aka inheritance & encapsulation.
  • Decoupling
    • LSP (Liskov substitution principle) – guarantee semantic interoperability of types in a hierarchy, object types in particular.
    • DIP (Dependency inversion principle) – high-level modules should not depend on low-level modules. Both should depend on abstractions. And abstractions should not depend upon details. Details should depend upon abstractions.
    • IHP (Information hiding principle) – prevent certain aspects of a class or software component from being accessible to its clients aka encapsulation.
    • Contracts – 2 categories: preconditions and postconditions where a precondition states that certain things must be true before a method can execute, and a postcondition states that certain things must be true after a method has executed.
    • LoD (Law of Demeter or Principle of Least Knowledge) – specific case of loose coupling where units (code) only talk to immediate friends (not strangers) assuming as little as possible about the structure or properties of anything else.
    • IoC (Inversion of control) – reusable generic code controls the execution of problem-specific code. It carries the strong connotation that the reusable code and the problem-specific code are developed independently, which often results in a single integrated application.
    • SOA (Service-oriented architecture) – develop code in business functionalities (discrete pieces of code and/or data structures) that can be reused for different purposes.
  • Avoiding Code Blow
    • DRY (Don’t repeat yourself) – reduce repetition of information.
    • YAGNI (You ain’t gonna need it) – do not add functionality until it is necessary.
    • Quality through testability (all of them!)

There is also a good bunch of tips from The Pragmatic Programmer located here.

And don't forget, Scout rule = leave a better place!

Recommended book

SOLID (Single responsibility, Open-closed, Liskov substitution, Interface segregation and Dependency inversion) is a mnemonic acronym introduced by Robert C. Martin

Web Analytics

Massimo Paolini presented on Google Analytics. The most profound thing he said which delivered a strong message is:

"Web analytics is about Revenue NOT traffic."

Bang! Right there. Too many site owners are mislead into trying to decipher visitor totals vs. focusing on the true picture; and that is how to understand and optimize their site visitors across online marketing efforts. The process is then rinse & repeat.

Finally, Web Analytics is NOT a precise science. You are looking for trends. Differences. To help you make decisions moving forward. This is because some people clean their systems ie.delete cookies etc..

Presentations from Silicon Valley Code Camp

For more presentations from these 2 days follow the link below to presenter uploaded presentations to SlideShare. Includes 2011 and previous years. Enjoy!

More links to Silicon Valley Code Camp

Code Camp main site:
On Facebook:
On PB Wiki:
On Twitter:!/sv_code_camp

So, in conclusion, this was an awesome & insightful 2 days packed with plenty of great applicable content.

~ Ernest

Object-orientated development: a lesson in PHP5

PHP has been given a major stabbing and bashing in the last few years. Mainly due to inconsistency, non native support for UTF8 and the fact that only from v5 has it been “object-orientated capable”. However, PHP just works. Out of the box. It delivers on a promise to produce dynamic web pages.

Below is a best practice guide on using PHP in an Object Orientated (OO) manner.

Why Object-Orientated (OO) Programming

  • Objects can store data internally. Therefore variables don’t need to be passed from function to function like in procedural programming.
  • Better organization, portability and reuse of code. You can place common functionality in a Class inside its own separate file using a common naming convention that can be reused in other applications by using an include. Think DRY (Don’t Repeat Yourself) principle.
  • Maintainability is improved. Code is easier to spot and alter in the 1st place. Another DRY principle.
  • Gets you building sites using MVC (Model-view-controller) framework like CodeIgniter, Zend Framework et al.. which are built around PHP5 OO and enforce good coding standards, practices and patterns.
  • Allows you to experience Convention over Configuration (CoC) principles in your MVC application.

Ok, so hopefully you are now convinced that OO is the way forward.

Object-Orientated (OO) Paradigms

Before I begin, let’s cover some basics of OO. That is, a Class represents an object, with associated methods and variables. Therefore all the functionality we will build will be wrapped in Classes. Think of them as blueprints for an object.

PHP treats objects in the same way as references or handles, meaning that each variable contains an object reference rather than a copy of the entire object. Note, since objects can be passed as arguments to a function they are by default “copied” unless you specify the argument as a reference variable &$variable.

Here’s a run down of some fundamental OO Paradigms you should be considering and even using to write kick ass OO code.

These paradigms are all used in my sample code below.


Inheritance extends a Class (base) to bring additional functionally to your Class (child). This allows you to create a hierarchy of interlinked classes. In PHP5 only 1 Class can be inherited.

For example, if we have a parent Class “Dog” which holds common Dog like behavior like “bark” we can extend that into our child Classes say “Poodle”, “Husky” etc… Therefore Poodle and Husky can both inherit the same method called “bark” which specifies the shared functionality without having to repeat it across other children Classes.

More on inheritance is covered here:

Constructor and destructor methods

These are methods called upon creation or destruction of Class object. Typically if you want to initialize a certain property on the creation of an object or cleanup post use.

More on these methods is covered here:

Member privacy (visibility)

This defines who has access to members and methods.

<strong opinion>I believe visibility is a lot of hog wash in PHP. PHP was designed by Rasmus Lerdorf for “web development to produce dynamic web pages” (ref: wiki). Member privacy is largely important when building components that are used by 3rd parties. Like something you would do in Java/C++ with dll/com objects. There is absolutely no good reason I know of why you’d want to hide certain Class functionality from your own developers. Use Interfaces if you must “guide” them on what they can and cannot use. Python got this right!</strong opinion>

More on member privacy is covered here:


Encapsulation is a reference to well-defined interface to a set of functions in a way which hides their internal workings. The benefit here is it can reduce complexity for another developer by not exposing the whole inner workings of a class. This is achieved using member privacy (discussed above).

More on encapsulation here:


Allows you to specify a Class templates for other developers to showing which public methods can be implemented without exposing how these methods are handled. Only methods can be declared in an Interface, not variables. More than 1 Interface can be extended in a Class.

More on interfaces can be found here:


Overloading allows you to dynamically create or extend public properties or methods. These dynamic entities are processed via magic methods.

More on overloading here:


Factory: The Factory pattern allows for the instantiation of objects at runtime. It is called a Factory Pattern since it is responsible for “manufacturing” an object. A Parameterized Factory receives the name of the class to instantiate as argument. And;
Singleton: The Singleton ensures that there can be only one instance of a Class and provides a global access point to that instance. Often implied in Database Classes, Loggers, Front Controllers or Request and Response objects.

More on Patterns here:

Now let’s look at how all of this would fit into code.

Note that I have tried to fit all these practices into the 1 sample of code below. Read the internal comments I put in to understand what is going on. Any mistakes or questions please use the comments section below to let me know.

The CODE – in PHP

// Interface implementation
interface BaseInterface
	function method0();

// Class definition with keyword "implements" to implement BaseInterface interface above.
class BaseClass implements BaseInterface
	// property declaration
	private $attribute3 = "This is PHP5";

	// method declaration, by default this is public
	function method0() {
		print "This is Method0";

// Always use Upper camel case naming notation for class names
// Inheritance is achieved by using the "extends" keyword with class you want to extend.
class ChildClass extends BaseClass
	// This variable is accessible without needing an instantiation of the class.
	public static $attribute0 = "Yes I'm visible";

	// This variable can only be accessed within this class
	// This is also an encapsulated attribute never exposed outside the class.
	private $attribute1;

	// This variable can be accessed outside this class
	public $attribute2;

	// Note the above visibility of the attributes.
	// There are 3 types and can be accessed - public (everywhere), protected (class & inherited) or private (only by class).

	// A class can only have 1 constructor
	// Suitable for any initialization that the object may need before it is used.
	function __construct($arg1, $arg2, …) {
		// Use operator arrow to access class variable
		// Note the lack of $ in front of attribute1. This is valid and also a common mistake among newbies.
		$this->attribute1 = "ABC";

	// A class can only have 1 destructor
	// Called when the object is explicitly destroyed.
	private function __destructor($arg1, $arg2, …) {
		$this->$attribute1 = "";

	// This method has no "visibility" declared so by default it becomes "public".
	function method1() {
		print "This is Method1";

		// Access to base class method / attribute using parent::
		// This call docent automatically evoke base class contractor / desconstructor unless called via parent::
		print "From BaseClass: " . parent::$attribute3;

	// "final" stops this method from being overridden.
	final public function method2($a) {
		$this->attribute2 = $a;

		// Displays to screen the "encapsulated" attribute
		print $this->attribute1;

// Accessing static attribute without instantiating a class
print ChildClass::$attribute0;	// Output: 'Yes I'm visible'

// Creating new instances of the child class
// Note that since ChildClass has arguments in the constructor function those need to be passed during this object's instantiation.
$foo = new ChildClass(1, 2, ...);
$bar = new BaseClass();
// Like functions, this instantiated object will have its own "class scope" so any method/variable is different to say $bar object.
$soap = new BaseClass();

// Accessing object members
$foo->attribute1;		// Output: 'ABC'
$foo->method1();		// Output: 'This is PHP5'
$bar = method0();		// Output: 'This is Method0'

I think that is enough to digest for now. If you want to read more on Object-Orientated PHP5 I recommend you visit PHP.NET here:

Hope this post has shed more light on OO in PHP and you are ready to crank out some clean and usable OO code.

~ Ernest

ECMAScript: The history of JavaScript

JavaScript is a very interesting language that has caught my eye in the last year. Previously I blogged about the power of the Module Pattern in JavaScript; why it’s powerful and how to make sure your site complies to this great pattern. The language has come a long way and is growing up fast to be the defacto language of choice for all front-end UI/behavioral functionality. Flash, the old heavy dog, is on its way out. JavaScript, part of the HTML5 stack, is looking the most promising.

This post isn’t about how great JavaScript is, it’s about its history. Before one delves into anything one should always understand the basics. So let’s begin with some basics and then its history. The history is rather interesting.

What is JavaScript anyway?

JavaScript is a prototype-based scripting language that is dynamic, weakly typed and has first-class functions.

For the business guy

  • JavaScript is client-side language. Meaning anyone with a browser can view it in its entirely. Nothing is hidden. This is how we have access to any site’s code. Right click on any webpage and select “View Source” – Bingo!
  • JavaScript is downloaded via your browser and then executed within the browser. This is where performance bottlenecks are experience and browser compatibility issues.
  • JavaScript’s real name is actually ECMAScript.

For the tech guy

  • JavaScript is a “prototype-based scripting language” meaning it is object-oriented where classes are not present, and inheritance is performed via a process of cloning existing objects that serve as prototypes.
  • JavaScript is “dynamic”; meaning that it executes at runtime.
  • JavaScript is “weakly typed”; meaning you can cast memory to any type and
  • JavaScript has “first-class functions”t meaning it supports passing functions as arguments to other functions, returning them as the values from other functions, and assigning them to variables or storing them in data structures.

History of JavaScript

There are many stories online but I think this one below (source) explains it nicely.

Here is something else you should know about Javascript that seems like it was almost designed to be confusing as possible:

  • There is a programming language called Javascript.
  • There is a programming language called Java.
  • There is an interpreter (‘thing that makes it go’) for the programming language Javascript built into most web browsers
  • There is an interpreter for the programming language Java that is sort of built in to most web browsers, or was.

Back in the olden days, there was a company called Sun that had invented a programming language they decided to call Java. It was very trendy and exciting. There was a company called Netscape that made the trendiest and excitingest web browser. Sun wanted Netscape to include the magical crap that would make Java work with their browser, because everyone was gonna get Netscape, so everyone would also automatically have the stuff that makes Java programs work. On the other hand, Netscape was saying, “but lets also make our own tiny programming language that runs right in the browser so instead of having to make up new html tags like ‘blink’, web authors can make annoying shit we haven’t even thought of yet!” So they said, “Sun, we’ll ship your ‘Java’, but we want to be allowed to call our other programming language, which has absolutely no relationship to yours and is intended to do completely different things, JavaSCRIPT.” And Sun said, “OK, swell. go nuts. That won’t matter to us. ONCE JAVA TAKES OVER THE WORLD!”

So Netscape said to a guy named Brendan, who worked at Netscape, “Please make us a programming language. Also, you have to call it Javascript. Also, if you can make some of it kind-of sort-of look a bit like Java, that would be even better. Also, you have only 10 days to do this so get cracking!”

Fortunately, it turned out that Brendan was a cool genius and he secretly designed a cool programming language and dressed it up in some vaguely Java-looking disguises, kind of like how children will sometimes wear a huge cloak and stand on top of one another in order to get into R-rated films.

But Brendan’s disguise worked too well! People were like “wtf is with this crap version of Java? IT SUCKS!” Also, even a cool genius like Brendan has some some limits, so he did make a few mistakes when he was making his programming language in only 10 days. Also, because Netscape had basically thrown down the gauntlet and said, “You think the blink tag is annoying? Marquee makes you want to rip your eyeballs out? YOU HAVEN’T SEEN NOTHING!” people did manage to find insanely annoying things to do with Javascript. All the cool people installed special software on their web browser JUST TO MAKE JAVASCRIPT NOT WORK. So it took many years before people started to figure out that Brendan’s language was wrapped in an elaborate disguise and that it was actually cool.

Weirdly, some of the main people who did this were people at Microsoft, who had tried to confuse this whole mess out of existence by coming out with programming languages named stuff like J++ and JScript. Netscape, at the time, was threatening to rip Microsoft apart, like a crab rips up a cuttlefish with his claws. So Microsoft, adopting the strategy of the cuttlefish, made a million confusing “J” programming languages, hoping to escape intact. A prophet by the name of Douglas started saying, “guess what nerds, it turns out Javascript is actually sort of awesome.” He managed to attract a fair number of acolytes, who fiddled with Brendan’s invention and realized that it was rather elegant and could certainly be made to do all sorts of useful, non-annoying things on webpages, if only people would stop blocking it.

And so the era now known as Web 2.0 began. There are a lot of things people associate with Web 2.0, but for people who make the internet, one of the biggest things was seeing sites like Flickr or Oddpost do cool stuff with Javascript and other technologies that had been previously considered lame.

And then, in a sort of poetic irony that makes this story almost seem like it was pre-scripted to Teach us a Lesson, JAVASCRIPT succeeded in doing what JAVA had intended to do. Microsoft, Java, Sun, Netscape, all were brought low by their hubris. But humble Javascript, the throwaway, ‘you get 10 days to make this’, blink-tag-replacing runt of a language was able to sneak onto every computer in the world thanks to its clever disguise. Servers are written in Javascript. Databases are built to talk Javascript. The people who build browsers and operating systems move heaven and earth to make Javascript just a tiny bit faster. Java’s still out there, of course. In various forms. It probably makes sure your account is updated when you pay your water bill. It’s making the underpinnings of your android phone work. It’s figured out a way to play host to a zillion new trendier programming languages. But Javascript won the original prize.

Anyway, I’m just pointing this out because I remember the time when I didn’t know the difference between Java and Javascript and I’d find a tutorial for one or an article about the other and I was like “wtf, how do these go together.” The answer is, “they don’t. Marketing people just tried to name them as confusingly as possible.”

JavaScript today is..

  • Used in every website you visit online.. like 99.9999% of them.
  • At the core of famous server-side Node.js, an evented I/O framework.
  • If you heard of beautiful frameworks like jQuery, YUI, and more recently SproutCore and Cappuccino (Objective-J).. all JavaScript.
  • HTML5? replacement for Flash and other slick interface functionality, yap you guessed it.. JavaScript.
  • It is the most important language today!

What next

Plenty more informative & juicy articles on this front are being lined up. Not to mention I am starting a fun project with few smart software engineers in the valley which aims to give back to the community around this most important language today, JavaScript. Stay tuned to find out more on what’s next!

~ Ernest

JavaScript: made better using the Module Pattern

I’ve just finished reading JavaScript Patterns by Stoyan Stefanov and boy has it opened up my mind and inspired me to write better, faster, cleaner and more modular JavaScript code using the Module Pattern.

Stoyan’s book covers many patterns and anti-patterns (the common old approach to JavaScript coding).

Stoyan provides detailed explanations why and where to use each pattern inc. module pattern. Since there is so much content in this book I decided to write about what I believe will add immediate value to any developer working with JavaScript, how to make JavaScript better using the module pattern.

No hacks around poor JavaScript

JavaScript is unlike back-end server-side code where if the code becomes a bottleneck you just throw more boxes (hardware) at it to speed thinks up. JavaScript downloads & runs within the user’s browser. So if you got lazy or just didn’t know better to follow Convention over Configuration (CoC) or the rapid development principle of Don’t Repeat Yourself (DRY) and you hacked something together.. the user will get poor experience and even limited to a single browser. This is where JavaScript Patterns play an important role and understanding them is the 1st step in becoming a JS Ninja.

JavaScript – what you need to know

Let’s shake some old grounds and cover the what you need to know about JavaScript in 2011.

  • There are no classes in JavaScript. You create a blank object when you need one and then start adding members to it.
  • Sometimes when people say “Singleton” in a JavaScript context, they mean the Module Pattern – covered below and discussed in detail in Stefanov’s Chapter 5. Every time you create an object using the object literal, you’re actually creating a Singleton. And there’s no special syntax involved.
  • When you think about objects in JavaScript, simply think about hash tables of key – value pairs. Similar to what are called associative arrays in other languages.
  • Use literal notation patterns instead of object definitions. Literal is more concise, expressive and less error-prone. See Appendix 1.1 at the end of this post for examples.

Module Pattern – organize your code

Let’s start with what’s most common out there on the internet. You may have seen (or even do) this.

Code sample – anti pattern


var array_string = "[object Array]";
var ops = Object.prototype.toString;
var MAX_NUM = 123;

function inArray (haystack, needle) {
    for (var i = 0, max = haystack.length; i < max; i += 1) {
        if (haystack[i] === needle)
            return i;
    isEnabled = false;
    return −1;

function isArray (a) {
    return === array_string;

var pageTitle = "New title";



The problems here are:

  • All these are global (variable and functions) are shared among all the code in your application living in the global namespace. High chance of naming collisions between your and other developer’s code (think maintenance).
  • isEnabled above has no var declaration so automatically becomes chained to the global namespace outside the function.
  • When something in your code is chained to the top of the prototype it takes longer time to be found then if it’s within a module. I will discuss the JavaScript Prototype in another post and show examples with performance differences.
  • Some best practices are broken – see below what these best practices should be.

Module Pattern – the solution, organize your code

There are many great patterns described in JavaScript Patterns by Stoyan Stefanov but my preference today is to use the Module Pattern. The Module Pattern is widely used because it provides structure and helps organize your code into self-contained de-coupled pieces of code. If you look at the source of jQuery you will find a similar pattern being used.

The Module Pattern is a combination of several patterns described in Stoyan’s book.
They include:

  • Namespaces (Stefanov’s pg.87)
  • Immediate functions (Stefanov’s pg.73)
  • Private and privileged members (Stefanov’s pg.92) and
  • Declaring dependencies (Stefanov’s pg.90)

And here’s how the same sample of code above would look when the Module Pattern was followed.

Module Pattern code sample – the pattern



// note the use of namespacing - similar to jQuery.
MYCOMPANY.Page = (function () {
    'use strict';

    // [ private properties ]
    // note the chaining of var declarations vs a var on each line
    var pageTitle    = "blank",
        isEnabled    = false,
        array_string = "[object Array]",
        ops          = Object.prototype.toString,
        MAX_NUM      = 123; // constants always are in capital case
    // end var

    // [ private methods ]
    // note the use of literal patterns vs constructor functions
    function inArray(haystack, needle) {
        var i   = 0,
            max = 0;
        for (max = haystack.length; i < max; i += 1) {
            if (haystack[i] === needle) {
                return i;
        return -1;
    function isArray(a) {
        return === array_string; // end var

    // [ public methods ]
    return {
        init: function (title, enabled) {
            pageTitle = title;
            isEnabled = enabled;
        isArray: isArray,
        indexOf: inArray


MYCOMPANY.Page.init('Page title', true);

Code is now modularized and less prone to collisions.

JSLint: The JavaScript Code Quality Tool

Once you have written your new modularized code it’s a best practice to run it past JSLint. JSLint is a free JavaScript program that looks for problems & violations of some of the patterns in your JavaScript code. It is a code quality tool. Check it out here:

JavaScript best practices

  • Curly brackets should always be used even if there is 1 expression. Code is then more readable and maintainable.
  • Use 4 space indentation – also JSLint default. This way no matter what editor engineers use it is always displayed in the same manner.
  • Like curly brackets, always use semicolons, even when they are implied by the JavaScript parser. Why slow down the compiler to clean up the mess during runtime.
  • Naming convention for constructors is typically UpperCamelCase, functions lowerCamelCase, variables lowercase_separated_by_underscore and constants UPPER_CASE.
  • Use whitespace as much as possible by separating all operators and their operands with spaces.
  • Use Google’s Closure Compiler to minify your code before production AFTER you have passed it via JSLint (above).


1.1 Constructors and their corresponding and preferred literal patterns.

Built-in constructors (avoid) Literals and primitives (prefer)
var o = new Object(); var o = {};
var a = new Array(); var a = [];
var re = new RegExp( “[a-z]”,”g”); var re = /[a-z]/g;
var s = new String(); var s = “”;
var n = new Number(); var n = 0;
var b = new Boolean(); var b = false;
throw new Error(“uh-oh”); throw { name: “Error”, message: “uh-oh” };… or throw Error(“uh-oh”);

And that is it for now. Have I missed something or gotten something wrong above? If so please let me know and I will correct it. How is your JavaScript looking? Need help to get it fixed? Post below or contact me.

~ Ernest

MongoDB office hours in Mountain View

You may recall my previous post on MongoDB and how powerful it is as an alternative to a relational database. Since then I’ve had a bunch of discussions with other software engineers around this space and even met up with MongoDB core engineer Chris Westin from 10gen at Red Rock to gain further insights into MongoDB.

MongoDB Leaf

New kick ass GUI for MongoDB

Chris introduced me JMongoBrowser written by Antoine Girbal (10gen engineer). It’s written in Java so you can run it on Linux, Windows and Mac OSX. So far this GUI has proven to be a success and fills the holes where MongoHub couldn’t. Out goes MongoHub and in goes JMongoBrowser.

JMongoBrowser - MongoDB GUI admin tool.

Fast crash recovery using Journaling

MongoDB uses memory 1st to write data to vs directly to file/store. This is where huge performance gains are attained. It also has Journaling, a write-ahead for operations to facilitate fast crash recovery in the storage engine. This means the stuff in memory is stored in a log incase your server goes down without affecting MongoDB’s performance.

So what happens if your box goes down? .. a common question amongst new engineers to MongoDB. Does this data also gets lost? The answer lays somewhere in between how you balance your performance needs against the risks you are willing to take with your data.

The journal is synced to disk every 100ms. So the maximum that can be lost is up to 100ms worth of changes. At the cost of additional performance degradation, you can make your application proof against even that.  The j option to getLastCommand will cause the application to block until the journal entries with the last change have been written to disk.  See .  Of course, if you use this, your call to getLastCommand can wait up to 100ms, depending on where in the flush cycle you are. Therefore, this is left up to the user to change this default of 100ms.

Therefore, always load MongoDB with Journaling enabled, like this:

mongod --journal

This also auto cleans up crashes and puts crashed data back into MongoDB.

Don’t be alarmed when you see this

I found this in the /Journal (1GB files):

-rw------- 1 root root 1.0G 2011-06-23 02:26 prealloc.0
-rw------- 1 root root 1.0G 2011-06-23 02:25 prealloc.1
-rw------- 1 root root 1.0G 2011-06-23 02:26 prealloc.2

With Journaling enabled the server always creates those three 1GB files.  It rotates through them, recycling them.  They won’t grow any more.  But they are always that size, regardless of the size of your database. If the server dies unexpectedly, the files remain, and contain the material necessary for the automated recovery that happens when you restart the server.

More here:

Checking server memory usage

As mentioned above, Memory is used by Mongo to speed things up. The more memory you have the better and MongoDB will use your RAM as it sees need for it taking into consideration other server resources.

It’s always a good practice to check memory usage.
Details here:

Your MongoDB configuration file

Here’s a recommend set of switches to have enabled in your mongodb.conf.

sudo nano /etc/mongodb.conf

add or update your settings to these:

  • journal = as discussed above to enable fast recovery from crashes.
  • directoryperdb = creates a new physical directory for each new database. Clean way to seperate your databases.
  • logappend = Whether the log file will be appended or over-written. Always have true else after a reboot your old logs will be overwritten and you may lose important crash specific data.

More detail here:

MySQL to MongoDB

Finally, I found this fantastic chart illustrating the difference in commands between MySQL and MongoDB. Should help the transition for us MySQL folks. Click the image below to download a large PDF version (Size: 213Kb).

There is also mapping chart SQL to Mongo located on the MongoDB website here.

Have more questions? Attend the weekly MongoDB Office Hours in Mountain View Red Rock or post a question/comment below.

~ Ernest

Mercurial: how-to guide to distributed version control

Mercurial is one of the 2 famous free / open source, distributed source control management tools. You may also have heard of Git, another great tool. Both Mercurial & Git are the front runners in distributed version control systems. 2 nice articles compare Mercurial to Git. One called Git vs. Mercurial: Please Relax comparing Git to MacGyver and Mercurial to James Bond. The other article The Differences Between Mercurial and Git. The conclusion was both are great tools so stop worrying and start using the one which you are most comfortable with. I went down the Mercurial route.

Mercurial - Work easier, Work faster

Mercurial is a distributed version control systems (DVCS) which keeps track of software revisions and allows many developers to work on a given project without necessarily being connected to a common network. This is a peer-to-peer approach, as opposed to the client-server approach of centralized systems like Subversion or CVS. It is a more cleaner, faster and refined way of working. More on this is explained on Wiki here.

Let’s get started with Mercurial

As explained above you Mercurial is a DVCS. So you will have a “local” version of Mercurial which may have a number of “repositories” on your development machine. A repository is essentially a package which identifies your application. One for each application. Your code will get “committed” to this local version daily as you work. When you have a working piece of functionality only then you will “push” it to “central”. Central is typically hosted on a remote server (hosted in-house or via a 3rd party like BitBucket) where everyone else in your team can push or clone code to. Typically “central” is hooked up to a continuous integration tool to facilitate nightly builds. More on that in another post. So whatever code you push there make sure it is working!

Local (dev) install or Mercurial

1. Download Mercurial on your development box from here:

2. Run the install file and follow through with the default questions.

3. Your done.

Quickstart – starting your 1st Mercurial Repository


In your terminal / dos window, navigate to a folder where you will either clone a project to or create one and do the following.

Clone a project and push changes

$ hg clone
$ cd hello
$ (edit files)
$ hg add (new files)
$ hg commit -m 'My changes'
$ hg push

Create a project and commit

$ hg init (project-directory)
$ cd (project-directory)
$ (add some files)
$ hg add
$ hg commit -m 'Initial commit'

Let’s look at each command in more detail:

hg init

Run this 1st time in your working directory to mark that folder as your project repository which you want Mercurial to version control. Mercurial will create a hidden folder there called “.hg”. This is where all the detail (changesets and manifests) on your commits is stored.

hg clone http://(my-domain)/hg/(project-repository)

Run this in your working directory where you want to drag code down from central Mercurial (See below on Central).

hg status

To see what is not yet added & committed yet to the repository.

hg add

If you have new files this command adds them to your repository.

hg commit

Commit your changes to “local” Mercurial. Make this a habit of doing it frequently (at least once a day). Mercurial will ask you to put in comments with your commit. Also make this a habit to put in as much detail into your comments as possible to help others understand your changes + if you have a reference to Jira case / other Agile tracker add this in as well.

hg pull -u

To pull changes and update.

hg push http://(my-domain)/hg/(project-repository)

To push your local repository changes to a central repository.

Tools for Mercurial

Netbeans – IDE Integration

If you are using Netbeans for development and have pointed it to a project where you “hg init” was ran, Netbeans will automatically recognize this and integrate Mercurial into your workflow. In Netbeans, do this by right clicking on your project folder and look under “Mercurial” sub-folder. Get it from here:

MacHG – GUI for managing repositories on the Mac

This is the nicest GUI for Mercurial that I could find for the Mac. It allows you in a nice graphical way to manage a collection of files, to add things to the collection, to save a snapshot of the collection, to restore the collection to an earlier state and in general to work with the files. Get it from here:
Alternatively go through a list of other tools here:

“Central” – home of all repositories

The following is only required if you want to setup Mercurial “central”. A single place (mothership) for all developers to push working code to for storage & versioning. This is also the place where you would hook up a continuous integration tool to facilitate nightly builds.

Bitbucket: free Mercurial code hosting

If you do not have enough resources to run Mercurial “central” in-house or want someone else to manage that for you then use one of the hosted solutions on the internet. Bitbucket from Atlassian (kick ass Aussie firm) can provide you with unlimited private code hosting, free. Yes I use it and I love it. Less maintenance & headaches and everything you learnt above can be applied here.

BitBucket by Atlassian - free Mercurial code hosting

Get your free account here:

Home solution: In-house Mercurial code hosting

Ok the following will require Ubuntu terminal access to the Ubuntu box where you want to install Mercurial “central”. All of the code below can be highlighted and pasted to run directly in your terminal window to make this process fast.

The following is based on the steps I followed from Emran Hasan’s Blog with some changes to the configuration to make it work on Ubuntu 10.10 based additional input from Stack Overflow.

1. Install Mercurial

sudo apt-get update
sudo apt-get install mercurial

2. Create a store for Mercurial configuration & repository files

cd /var/
sudo mkdir hg
sudo mkdir hg/repos
sudo chown -R www-data:www-data hg/repos

3. Creating a configuration file to host multiple repositories on this server & use CGI to serve the files through Apache:

cd /var/hg
sudo cp /usr/share/doc/mercurial/examples/hgweb.cgi .
sudo chmod a+x hgweb.cgi

4. Configure Mercurial (also refered to as “hg”) to point to point to our new config file:

sudo nano /var/hg/hgweb.cgi

and update the config like this:

config = "/var/hg/hgweb.config"

5. Create the file /var/hg/hgweb.config and write the location of the repositories:

sudo nano hgweb.config

and add this to the file:

/var/hg/repos = /var/hg/repos

6. Make “central” accessible via HTTP (optional if you don’t want to setup a SSL certificate).

sudo nano /etc/mercurial/hgrc

add/update this line and save file:

allow_push = * push_ssl = false

7. Now update the Apache configuration so that it executes the CGI when requested with a /hg prefix:

cd /etc/apache2/sites-available
sudo nano default

at the end of the file (before < /VirtualHost >), add the following and save it:

ScriptAlias /hg "/var/hg/hgweb.cgi"
< Directory "/var/hg/repos" >
   AuthType Basic
   AuthName "Mercurial Repositories"
   AuthUserFile /var/hg/hgusers
   Require valid-user
< /Directory >

8. Add permissions for users who will be accessing central. First add admin account:

sudo cd /var/hg htpasswd -mc hgusers admin

then normal user accounts:

sudo htpasswd -m hgusers

Each request will prompt you for a password.

9. Create a repository

cd /var/hg/repos
sudo mkdir (my-repository)
cd (my-repository)
sudo hg init

Note: Recall “hg init” from above (local setup).

10. Restart apache

sudo /etc/init.d/apache2 restart

and browse: http://(my-domain)/hg
you will be prompted to login (using one of the users names you added above) and see the whole Mercurial central repository.

URL access: http://(my-domain)/hg/(project-repository)
Cloning from “central” to your local:

cd (my-local-project-working-directory)
hg clone http://(my-domain)/hg/(project-repository)

And… that concludes this short introduction to Mercurial. Hope you found it useful!

Bonus: Download this Mercurial Cheat Sheet: (847kb)

Mercurial Usage - 1 pager

I have it at my desk. It is the best one I could find out of the many I reviewed.

Happy coding!

Useful links

Hosted Mercurial solutions

More Mercurial education

Bitbucket education


~ Ernest