How to setup Django + mod_wsgi on Apache & Ubuntu

Django is a great Web Framework which complements the Python programming language. Having switched to Python development few months ago I was on the lookout for a MVC like Web Framework that emphasizes Convention over Configuration (CoC), Rapid Development Principle of Don’t Repeat Yourself (DRY), Separation of concerns has Python as the underlying core language. And lo and behold Django proved to be that beast.

What is Django

Django is a MVT (model-view-template) framework build on the Python language. The MVT is identical to MVC (model-view-controller) but where in the common MVC pattern the view here is the template and controller the view. They both perform the same function separating the business logic from the presentation from the data layer (model). However Django does goes one step further and provides a view/template without native core language polluting the HTML.

Django emphasizes Reusability and “Pluggability” of components, Rapid Development, and the Principle of DRY (Don’t Repeat Yourself). Stuff you hear a lot with Ruby on Rails – another great web framework built on a solid language, Ruby.

Ok, enough of my rambling about how great this framework is. If you want to read more about it I recommend the following 2 links:

The LAMP stack

Ok so everyone is talking about a LAMP stack and everyone wants to run their site on a LAMP stack. Why not when you can tap into great open source tools for free. The word LAMP is a bit over emphasised since most sites today run on a custom configured implementation using best of breed open source software.

“LAMP is an acronym for a solution stack of free, open source software, referring to the first letters of Linux (operating system), Apache HTTP Server, MySQL (database software) and PHP (or sometimes Perl or Python), principal components to build a viable general purpose web server” ~ Wikipedia

Few years ago when open source software was limited, LAMP had typically meant Linux, Apache, MySQL and PHP. Today Linux can take many flavours from CentOS, Red Hat or the popular Debian-derived Linux distribution Ubuntu. The web server Apache can be replaced with a super fast & lighter HTTP and reverse proxy server called nginx. MySQL, post Oracle acquisition is often replaced with PostgreSQL or a NoSQL version called MongoDB. And PHP by cleaner & concise languages like Python or Ruby.

In this post I will cover how to setup a LAMP stack where:

  • Linux = Ubuntu 11.10. This can also work with 10.xx version of Ubuntu.
  • Apache = we will leave it as is since it is still a great web server but hook in mod_wsgi – a Python WSGI adapter module for Apache. WSGI is nothing more than an interface specification by which server and application communicate. In this case with Python.
  • MySQL = will stay with MySQL but you can also install MongoDB.
  • PHP = will be replaced with Python and drop in the Django MVT framework.

Ok let’s get started.

How to Setup Django on Ubuntu

The following assumes you have already installed Ubuntu and LAMP. If not head over to my prior posts on how to:

  1. Install Ubuntu on EC2 and
  2. Install LAMP.

Installing pre-req components

1. Standard procedure is to always run update on your Ubuntu installation.

sudo apt-get update

2. Install mod_wsgi on Apache

sudo apt-get install libapache2-mod-wsgi

3. Install Python setup tools with pip. pip installs packages. Python packages. An easy_install replacement.

sudo apt-get install python-setuptools
sudo apt-get install python-pip

4. Install Django MVT framework using pip.

sudo pip install Django

Now that’s done. Easy hey. Now let’s configure your 1st Django site.
Assuming your site will be called “purpleblue.com

Configuring your 1st Django site

5. Recall from the LAMP setup that we were storing all web sites in /var/www/ folder. For example purposes, I will use purpleblue_com as my Django project. Notice the use of django-admin.py to create the Django site and the use of underscores vs decimal point. Only numbers, letters and underscores are allowed in the Django project name.

cd /var/www/
sudo django-admin.py startproject purpleblue_com

5.1 Verify new project by typing this in and hitting enter. ls lists the contents of a directory showing all files inc hidden ones, hence the -all option.

ls –all

If you see purpleblue_com listed, then change directory to it.

cd purpleblue_com

and verify you can see 4 .py files – init, manage, settings & urls. Those are Django new project default install files.

6. Now we create a wsgi file for the site so apache knows how to run this site.

sudo mkdir /var/www/purpleblue_com/apache
sudo nano /var/www/purpleblue_com/apache/django.wsgi

… and add this into django.wsgi

import os
import sys

path = '/var/www/purpleblue_com'
if path not in sys.path:
    sys.path.insert(0, '/var/www/purpleblue_com')

os.environ['DJANGO_SETTINGS_MODULE'] = 'purpleblue_com.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

TIP: It’s always a good idea to specify the actual application module.

6. Using webmin (ref our LAMP install) setup new apache site for this project.

6.1 Go to: https://YOUR_DEV_VM_IP:10000/
6.2 Servers > Apache Webserver > Create virtual host
6.3 Configure your new host
Document Root = /var/www/purpleblue_com/
Server Name = purpleblue.com
6.4 Click on “Create Now” button. And go back into the settings by clicking on “Virtual Server” button next to your new host.
6.4 Click on “Edit Directives” and paste the following in:

ServerName purpleblue.com
DocumentRoot /var/www/purpleblue_com

<Directory /var/www/purpleblue_com>
    Order allow,deny
    Allow from all
</Directory>

WSGIDaemonProcess purpleblue_com processes=2 threads=15 display-name=%{GROUP}
WSGIProcessGroup purpleblue_com

WSGIScriptAlias / /var/www/purpleblue_com/apache/django.wsgi

The last 3 lines above allow this Django site to run in daemon mode.

TIP: When you make any .py changes in your Django site, rather then restarting/refreshing apache you can touch this wsgi and the changes will be picked up.

sudo touch /var/www/purpleblue_com/apache/django.wsgi

7. Restart apache to take new site & settings into effect.

sudo /etc/init.d/apache2 restart

8. Add the new domain purpleblue.com to your primary OS host file pointing to your VM where the Django project purpleblue_com resides.

9. Test your new project by going to the Server Name (http://purpleblue.com) you just setup. You should see a similar default Django installation page:

It worked! Congratulations on your first Django-powered page.

If you get an error or cannot see the default Django installation page make sure you check out apache log file. Common approach is to grab the tail end of the apache error log file to see what just happened.

tail /var/log/apache2/error.log

Bonus – Django CMS

If you have the time also check out Django CMS that is built on top of the Django Web Framework.

Django CMS site: https://www.django-cms.org/

Happy Django exploring & Python coding!

References

Silicon Valley Code Camp 2011

Last weekend myself and Omid Mozayani (another Aussie; recent addition to the valley) spent the weekend of Saturday and Sunday, October 8th and 9th, 2011 at Silicon Valley Code Camp 2011. Held at the Foothill College in Los Altos/CA with an attendance of well over 2,000 (registered 3,417) people and 209 sessions organized in a University style setup over these 2 days. Impossible to attend each one but plenty to cherry pick from. It felt like I was back at Uni(versity) running between lectures. I loved it! Here’s what happened.

CodeCamp at FootHill College.

What is Code Camp

Code Camp is a new type of free community event by and for the developer community where developers learn from fellow developers. It is 2 full days of talking about code with fellow developers. Sessions range from informal “chalk talks” to presentations. Basically there is lots of education, networking and good food.

Pictures from Code Camp @ Foothill College

Lecture - Crockford on ECMAScript5
Lecture - Crockford on ECMAScript5

Lunch - feeding time for thousands of nerds
Lunch - feeding time for thousands of nerds

Ernest Semerda - nerd out and about
Ernest Semerda - nerd out and about

Lecture - multi threaded design
Lecture - multi threaded design

More pictures here on my Flickr.

Code Sessions

Choosing what sessions to attend was key since it was impossible to attend everything when you consider that each session took around 45 min. Here is a wrap of the 3 areas I focused on; ECMAScript (JavaScript) with code encapsulation, Writing Clean Code and Web Analytics.

ECMAScript 5 (JavaScript)

Douglas Crockford ran 2 sessions on ECMAScript 5: The New Parts and ECMAScript: What Next?. Crockford is involved in the development of the JavaScript language and popularizing the data format JSON (JavaScript Object Notation), and for developing JSLint. You may recall me talking about JSLint in a previous blog post on JavaScript Patterns.

More on Crockford is here (his personal site) and here: Crockford on JavaScript. Also read about how JavaScript is the world’s most misunderstood language. Recommended reading! You will want to learn more about this great language.

ECMAScript’s Fifth Edition is now available in all of the best browsers. This is what you are getting:

  • Use “Strict mode” in all your JavaScript code.
    • As per my previous post on JavaScript Patterns here.
    • Old JS compiler ignores it so it’s safe to use today in all your JavaScript.
    • A bug in JS used to advantage today for backward compatibility.
    • “Strict mode” will break old bad programs. If you are writing good programs they will just work.
  • No more implied global variables within functions.
    • Yipee! Major fail in JavaScript where undefined variables would chain to the prototype making it very difficult to isolate your code, and to reuse it in new contexts.
    • If you forget to “var” a variable it now becomes undefined so you can address it vs. letting it slide by. Bad coders beware!
    • To append something to the global object add “window.function/var”.
  • “apply” and “call” do not default to the global object.
  • No “with” statement. Read why it was considered harmful to use.
  • Restrictions on “evil” eval. Here’s why it’s evil. *Cough* code injection!
  • No more octal literals. Not needed since punch card days.
  • Forgetting to use the new prefix will now throw an exception, not silently clobber the global object.

Use something like this today to check if your browser is running in Strict mode:

function in_strict_mode() {
    return (function () {
        return !this;
    }
}
  • Make it a habit to do static analysis using JSLint. If you are using it already your code is probably good and ready for strict mode.
  • Use “Safe JavaScript Subsets” like Caja & ADsafe projects which allow you to safely run 3rd party software on your website.

JavaScript Code Organization and Encapsulation

Shawn Van Ittersum presented ideas for organizing, encapsulating, and simplifying JavaScript code, including: creating JavaScript objects and classes with truly private members, using Douglas Crockford’s module pattern and classical inheritance constructs provided by Dean Edward’s Base.js, Prototype, and MooTools; abstracting JavaScript code from HTML templates, CSS styling, and the DOM; designing with polymorphism (duck typing) for code reuse and simplicity; and using JSLint to identify problems in your code before runtime.

The idea behind the power of Encapsulation is simple:

  • Encapsulation limits widespread interconnection,
  • Gives you Freedom to change internal implementations without affecting external code and
  • is referred to as Loose Coupling – an approach viewed as positive in the field of computer science to remove dependency between systems and allow horizontal scaling.

Ittersum’s lides with code samples are located here.

Writing Clean Code

Theo Jungeblut ran this session and basically the crust of the presentation was that writing clean code makes us more efficient.

This presentation was based on C# and Visual Studio 2010. However the following patterns and practices can be applied to every other programming language too.

Over the lifetime of a product, maintaining the product is actually one – if not the most – expensive area(s) of the overall product costs. Writing clean code can significantly lower these costs. However, writing clean code also makes you more efficient during the initial development time and results in more stable code.

  • Readability
    • Follow coding guidelines.
  • Simplification and Specialization
    • KISS (Keep it simple, Stupid!) – most systems work best if they are kept simple rather than made complex.
    • SoC (Separation of concern) – focus on 1 thing.
    • SRP (Single responsibility principle) – every object should have a single responsibility, and that responsibility should be entirely encapsulated by the class.
    • OCP (Open/closed principle) – software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification aka inheritance & encapsulation.
  • Decoupling
    • LSP (Liskov substitution principle) – guarantee semantic interoperability of types in a hierarchy, object types in particular.
    • DIP (Dependency inversion principle) – high-level modules should not depend on low-level modules. Both should depend on abstractions. And abstractions should not depend upon details. Details should depend upon abstractions.
    • IHP (Information hiding principle) – prevent certain aspects of a class or software component from being accessible to its clients aka encapsulation.
    • Contracts – 2 categories: preconditions and postconditions where a precondition states that certain things must be true before a method can execute, and a postcondition states that certain things must be true after a method has executed.
    • LoD (Law of Demeter or Principle of Least Knowledge) – specific case of loose coupling where units (code) only talk to immediate friends (not strangers) assuming as little as possible about the structure or properties of anything else.
    • IoC (Inversion of control) – reusable generic code controls the execution of problem-specific code. It carries the strong connotation that the reusable code and the problem-specific code are developed independently, which often results in a single integrated application.
    • SOA (Service-oriented architecture) – develop code in business functionalities (discrete pieces of code and/or data structures) that can be reused for different purposes.
  • Avoiding Code Blow
    • DRY (Don’t repeat yourself) – reduce repetition of information.
    • YAGNI (You ain’t gonna need it) – do not add functionality until it is necessary.
    • Quality through testability (all of them!)

There is also a good bunch of tips from The Pragmatic Programmer located here.

And don't forget, Scout rule = leave a better place!

Recommended book

SOLID (Single responsibility, Open-closed, Liskov substitution, Interface segregation and Dependency inversion) is a mnemonic acronym introduced by Robert C. Martin

Web Analytics

Massimo Paolini presented on Google Analytics. The most profound thing he said which delivered a strong message is:

"Web analytics is about Revenue NOT traffic."

Bang! Right there. Too many site owners are mislead into trying to decipher visitor totals vs. focusing on the true picture; and that is how to understand and optimize their site visitors across online marketing efforts. The process is then rinse & repeat.

Finally, Web Analytics is NOT a precise science. You are looking for trends. Differences. To help you make decisions moving forward. This is because some people clean their systems ie.delete cookies etc..

Presentations from Silicon Valley Code Camp

For more presentations from these 2 days follow the link below to presenter uploaded presentations to SlideShare. Includes 2011 and previous years. Enjoy!

http://www.slideshare.net/search/slideshow?type=presentations&q=silicon+valley+code+camp+2011&searchfrom=basic

More links to Silicon Valley Code Camp

Code Camp main site: http://www.siliconvalley-codecamp.com/
On Facebook: http://www.facebook.com/groups/6125701762/
On PB Wiki: https://codecamp.pbworks.com/w/page/16049741/FrontPage
On Twitter: http://twitter.com/#!/sv_code_camp

So, in conclusion, this was an awesome & insightful 2 days packed with plenty of great applicable content.

Enjoy!
~ Ernest

Object-orientated development: a lesson in PHP5

PHP has been given a major stabbing and bashing in the last few years. Mainly due to inconsistency, non native support for UTF8 and the fact that only from v5 has it been “object-orientated capable”. However, PHP just works. Out of the box. It delivers on a promise to produce dynamic web pages. PHP Elephant, Object-orientated development: a lesson in PHP5

Below is a best practice guide on using PHP in an Object Orientated (OO) manner.

Why Object-Orientated (OO) Programming

  • Objects can store data internally. Therefore variables don’t need to be passed from function to function like in procedural programming.
  • Better organization, portability and reuse of code. You can place common functionality in a Class inside its own separate file using a common naming convention that can be reused in other applications by using an include. Think DRY (Don’t Repeat Yourself) principle.
  • Maintainability is improved. Code is easier to spot and alter in the 1st place. Another DRY principle.
  • Gets you building sites using MVC (Model-view-controller) framework like CodeIgniter, Zend Framework et al.. which are built around PHP5 OO and enforce good coding standards, practices and patterns.
  • Allows you to experience Convention over Configuration (CoC) principles in your MVC application.

Ok, so hopefully you are now convinced that OO is the way forward.

Object-Orientated (OO) Paradigms

Before I begin, let’s cover some basics of OO. That is, a Class represents an object, with associated methods and variables. Therefore all the functionality we will build will be wrapped in Classes. Think of them as blueprints for an object.

PHP treats objects in the same way as references or handles, meaning that each variable contains an object reference rather than a copy of the entire object. Note, since objects can be passed as arguments to a function they are by default “copied” unless you specify the argument as a reference variable &$variable.

Here’s a run down of some fundamental OO Paradigms you should be considering and even using to write kick ass OO code.

These paradigms are all used in my sample code below.

Inheritance

Inheritance extends a Class (base) to bring additional functionally to your Class (child). This allows you to create a hierarchy of interlinked classes. In PHP5 only 1 Class can be inherited.

For example, if we have a parent Class “Dog” which holds common Dog like behavior like “bark” we can extend that into our child Classes say “Poodle”, “Husky” etc… Therefore Poodle and Husky can both inherit the same method called “bark” which specifies the shared functionality without having to repeat it across other children Classes.

More on inheritance is covered here: http://www.php.net/manual/en/language.oop5.inheritance.php

Constructor and destructor methods

These are methods called upon creation or destruction of Class object. Typically if you want to initialize a certain property on the creation of an object or cleanup post use.

More on these methods is covered here: http://php.net/manual/en/language.oop5.decon.php

Member privacy (visibility)

This defines who has access to members and methods.

<strong opinion>I believe visibility is a lot of hog wash in PHP. PHP was designed by Rasmus Lerdorf for “web development to produce dynamic web pages” (ref: wiki). Member privacy is largely important when building components that are used by 3rd parties. Like something you would do in Java/C++ with dll/com objects. There is absolutely no good reason I know of why you’d want to hide certain Class functionality from your own developers. Use Interfaces if you must “guide” them on what they can and cannot use. Python got this right!</strong opinion>

More on member privacy is covered here: http://www.php.net/manual/en/language.oop5.visibility.php

Encapsulation

Encapsulation is a reference to well-defined interface to a set of functions in a way which hides their internal workings. The benefit here is it can reduce complexity for another developer by not exposing the whole inner workings of a class. This is achieved using member privacy (discussed above).

More on encapsulation here: http://en.wikipedia.org/wiki/Encapsulation_(object-oriented_programming)

Interfaces

Allows you to specify a Class templates for other developers to showing which public methods can be implemented without exposing how these methods are handled. Only methods can be declared in an Interface, not variables. More than 1 Interface can be extended in a Class.

More on interfaces can be found here: http://en.wikipedia.org/wiki/Object-oriented_user_interface

Overloading

Overloading allows you to dynamically create or extend public properties or methods. These dynamic entities are processed via magic methods.

More on overloading here: http://www.php.net/manual/en/language.oop5.overloading.php

Patterns

Factory: The Factory pattern allows for the instantiation of objects at runtime. It is called a Factory Pattern since it is responsible for “manufacturing” an object. A Parameterized Factory receives the name of the class to instantiate as argument. And;
Singleton: The Singleton ensures that there can be only one instance of a Class and provides a global access point to that instance. Often implied in Database Classes, Loggers, Front Controllers or Request and Response objects.

More on Patterns here: http://www.php.net/manual/en/language.oop5.patterns.php

Now let’s look at how all of this would fit into code.

Note that I have tried to fit all these practices into the 1 sample of code below. Read the internal comments I put in to understand what is going on. Any mistakes or questions please use the comments section below to let me know.

The CODE – in PHP

// Interface implementation
interface BaseInterface
{
	function method0();
}

// Class definition with keyword "implements" to implement BaseInterface interface above.
class BaseClass implements BaseInterface
{
	// property declaration
	private $attribute3 = "This is PHP5";

	// method declaration, by default this is public
	function method0() {
		print "This is Method0";
	}
}

// Always use Upper camel case naming notation for class names
// Inheritance is achieved by using the "extends" keyword with class you want to extend.
class ChildClass extends BaseClass
{
	// This variable is accessible without needing an instantiation of the class.
	public static $attribute0 = "Yes I'm visible";

	// This variable can only be accessed within this class
	// This is also an encapsulated attribute never exposed outside the class.
	private $attribute1;

	// This variable can be accessed outside this class
	public $attribute2;

	// Note the above visibility of the attributes.
	// There are 3 types and can be accessed - public (everywhere), protected (class & inherited) or private (only by class).

	// A class can only have 1 constructor
	// Suitable for any initialization that the object may need before it is used.
	function __construct($arg1, $arg2, …) {
		// Use operator arrow to access class variable
		// Note the lack of $ in front of attribute1. This is valid and also a common mistake among newbies.
		$this->attribute1 = "ABC";
	}

	// A class can only have 1 destructor
	// Called when the object is explicitly destroyed.
	private function __destructor($arg1, $arg2, …) {
		$this->$attribute1 = "";
	}

	// This method has no "visibility" declared so by default it becomes "public".
	function method1() {
		print "This is Method1";

		// Access to base class method / attribute using parent::
		// This call docent automatically evoke base class contractor / desconstructor unless called via parent::
		print "From BaseClass: " . parent::$attribute3;
	}

	// "final" stops this method from being overridden.
	final public function method2($a) {
		$this->attribute2 = $a;

		// Displays to screen the "encapsulated" attribute
		print $this->attribute1;
	}
}

// Accessing static attribute without instantiating a class
print ChildClass::$attribute0;	// Output: 'Yes I'm visible'

// Creating new instances of the child class
// Note that since ChildClass has arguments in the constructor function those need to be passed during this object's instantiation.
$foo = new ChildClass(1, 2, ...);
$bar = new BaseClass();
// Like functions, this instantiated object will have its own "class scope" so any method/variable is different to say $bar object.
$soap = new BaseClass();

// Accessing object members
$foo->attribute1;		// Output: 'ABC'
$foo->method1();		// Output: 'This is PHP5'
$bar = method0();		// Output: 'This is Method0'

I think that is enough to digest for now. If you want to read more on Object-Orientated PHP5 I recommend you visit PHP.NET here: http://www.php.net/manual/en/language.oop5.basic.php

Hope this post has shed more light on OO in PHP and you are ready to crank out some clean and usable OO code.

~ Ernest

ECMAScript: The history of JavaScript

JavaScript is a very interesting language that has caught my eye in the last year. Previously I blogged about the power of the Module Pattern in JavaScript; why it’s powerful and how to make sure your site complies to this great pattern. The language has come a long way and is growing up fast to be the defacto language of choice for all front-end UI/behavioral functionality. Flash, the old heavy dog, is on its way out. JavaScript, part of the HTML5 stack, is looking the most promising.

This post isn’t about how great JavaScript is, it’s about its history. Before one delves into anything one should always understand the basics. So let’s begin with some basics and then its history. The history is rather interesting. JavaScript history

What is JavaScript anyway?

JavaScript is a prototype-based scripting language that is dynamic, weakly typed and has first-class functions.

For the business guy

  • JavaScript is client-side language. Meaning anyone with a browser can view it in its entirely. Nothing is hidden. This is how we have access to any site’s code. Right click on any webpage and select “View Source” – Bingo!
  • JavaScript is downloaded via your browser and then executed within the browser. This is where performance bottlenecks are experience and browser compatibility issues.
  • JavaScript’s real name is actually ECMAScript.

For the tech guy

  • JavaScript is a “prototype-based scripting language” meaning it is object-oriented where classes are not present, and inheritance is performed via a process of cloning existing objects that serve as prototypes.
  • JavaScript is “dynamic”; meaning that it executes at runtime.
  • JavaScript is “weakly typed”; meaning you can cast memory to any type and
  • JavaScript has “first-class functions”t meaning it supports passing functions as arguments to other functions, returning them as the values from other functions, and assigning them to variables or storing them in data structures.

History of JavaScript

There are many stories online but I think this one below (source) explains it nicely.

Here is something else you should know about Javascript that seems like it was almost designed to be confusing as possible:

  • There is a programming language called Javascript.
  • There is a programming language called Java.
  • There is an interpreter (‘thing that makes it go’) for the programming language Javascript built into most web browsers
  • There is an interpreter for the programming language Java that is sort of built in to most web browsers, or was.
  • THESE TWO PROGRAMMING LANGUAGES HAVE ALMOST NOTHING TO DO WITH EACH OTHER

Back in the olden days, there was a company called Sun that had invented a programming language they decided to call Java. It was very trendy and exciting. There was a company called Netscape that made the trendiest and excitingest web browser. Sun wanted Netscape to include the magical crap that would make Java work with their browser, because everyone was gonna get Netscape, so everyone would also automatically have the stuff that makes Java programs work. On the other hand, Netscape was saying, “but lets also make our own tiny programming language that runs right in the browser so instead of having to make up new html tags like ‘blink’, web authors can make annoying shit we haven’t even thought of yet!” So they said, “Sun, we’ll ship your ‘Java’, but we want to be allowed to call our other programming language, which has absolutely no relationship to yours and is intended to do completely different things, JavaSCRIPT.” And Sun said, “OK, swell. go nuts. That won’t matter to us. ONCE JAVA TAKES OVER THE WORLD!”

So Netscape said to a guy named Brendan, who worked at Netscape, “Please make us a programming language. Also, you have to call it Javascript. Also, if you can make some of it kind-of sort-of look a bit like Java, that would be even better. Also, you have only 10 days to do this so get cracking!”

Fortunately, it turned out that Brendan was a cool genius and he secretly designed a cool programming language and dressed it up in some vaguely Java-looking disguises, kind of like how children will sometimes wear a huge cloak and stand on top of one another in order to get into R-rated films.

But Brendan’s disguise worked too well! People were like “wtf is with this crap version of Java? IT SUCKS!” Also, even a cool genius like Brendan has some some limits, so he did make a few mistakes when he was making his programming language in only 10 days. Also, because Netscape had basically thrown down the gauntlet and said, “You think the blink tag is annoying? Marquee makes you want to rip your eyeballs out? YOU HAVEN’T SEEN NOTHING!” people did manage to find insanely annoying things to do with Javascript. All the cool people installed special software on their web browser JUST TO MAKE JAVASCRIPT NOT WORK. So it took many years before people started to figure out that Brendan’s language was wrapped in an elaborate disguise and that it was actually cool.

Weirdly, some of the main people who did this were people at Microsoft, who had tried to confuse this whole mess out of existence by coming out with programming languages named stuff like J++ and JScript. Netscape, at the time, was threatening to rip Microsoft apart, like a crab rips up a cuttlefish with his claws. So Microsoft, adopting the strategy of the cuttlefish, made a million confusing “J” programming languages, hoping to escape intact. A prophet by the name of Douglas started saying, “guess what nerds, it turns out Javascript is actually sort of awesome.” He managed to attract a fair number of acolytes, who fiddled with Brendan’s invention and realized that it was rather elegant and could certainly be made to do all sorts of useful, non-annoying things on webpages, if only people would stop blocking it.

And so the era now known as Web 2.0 began. There are a lot of things people associate with Web 2.0, but for people who make the internet, one of the biggest things was seeing sites like Flickr or Oddpost do cool stuff with Javascript and other technologies that had been previously considered lame.

And then, in a sort of poetic irony that makes this story almost seem like it was pre-scripted to Teach us a Lesson, JAVASCRIPT succeeded in doing what JAVA had intended to do. Microsoft, Java, Sun, Netscape, all were brought low by their hubris. But humble Javascript, the throwaway, ‘you get 10 days to make this’, blink-tag-replacing runt of a language was able to sneak onto every computer in the world thanks to its clever disguise. Servers are written in Javascript. Databases are built to talk Javascript. The people who build browsers and operating systems move heaven and earth to make Javascript just a tiny bit faster. Java’s still out there, of course. In various forms. It probably makes sure your account is updated when you pay your water bill. It’s making the underpinnings of your android phone work. It’s figured out a way to play host to a zillion new trendier programming languages. But Javascript won the original prize.

Anyway, I’m just pointing this out because I remember the time when I didn’t know the difference between Java and Javascript and I’d find a tutorial for one or an article about the other and I was like “wtf, how do these go together.” The answer is, “they don’t. Marketing people just tried to name them as confusingly as possible.”

JavaScript today is..

  • Used in every website you visit online.. like 99.9999% of them.
  • At the core of famous server-side Node.js, an evented I/O framework.
  • If you heard of beautiful frameworks like jQuery, YUI, and more recently SproutCore and Cappuccino (Objective-J).. all JavaScript.
  • HTML5? replacement for Flash and other slick interface functionality, yap you guessed it.. JavaScript.
  • It is the most important language today!

What next

Plenty more informative & juicy articles on this front are being lined up. Not to mention I am starting a fun project with few smart software engineers in the valley which aims to give back to the community around this most important language today, JavaScript. Stay tuned to find out more on what’s next!

~ Ernest

JavaScript: made better using the Module Pattern

I’ve just finished reading JavaScript Patterns by Stoyan Stefanov and boy has it opened up my mind and inspired me to write better, faster, cleaner and more modular JavaScript code using the Module Pattern.

Stoyan’s book covers many patterns and anti-patterns (the common old approach to JavaScript coding).

Stoyan provides detailed explanations why and where to use each pattern inc. module pattern. Since there is so much content in this book I decided to write about what I believe will add immediate value to any developer working with JavaScript, how to make JavaScript better using the module pattern.

No hacks around poor JavaScript

JavaScript is unlike back-end server-side code where if the code becomes a bottleneck you just throw more boxes (hardware) at it to speed thinks up. JavaScript downloads & runs within the user’s browser. So if you got lazy or just didn’t know better to follow Convention over Configuration (CoC) or the rapid development principle of Don’t Repeat Yourself (DRY) and you hacked something together.. the user will get poor experience and even limited to a single browser. This is where JavaScript Patterns play an important role and understanding them is the 1st step in becoming a JS Ninja.

JavaScript – what you need to know

Let’s shake some old grounds and cover the what you need to know about JavaScript in 2011.

  • There are no classes in JavaScript. You create a blank object when you need one and then start adding members to it.
  • Sometimes when people say “Singleton” in a JavaScript context, they mean the Module Pattern – covered below and discussed in detail in Stefanov’s Chapter 5. Every time you create an object using the object literal, you’re actually creating a Singleton. And there’s no special syntax involved.
  • When you think about objects in JavaScript, simply think about hash tables of key – value pairs. Similar to what are called associative arrays in other languages.
  • Use literal notation patterns instead of object definitions. Literal is more concise, expressive and less error-prone. See Appendix 1.1 at the end of this post for examples.

Module Pattern – organize your code

Let’s start with what’s most common out there on the internet. You may have seen (or even do) this.

Code sample – anti pattern

Declaring:

var array_string = "[object Array]";
var ops = Object.prototype.toString;
var MAX_NUM = 123;

function inArray (haystack, needle) {
    for (var i = 0, max = haystack.length; i < max; i += 1) {
        if (haystack[i] === needle)
            return i;
    }
    isEnabled = false;
    return −1;
}

function isArray (a) {
    return ops.call(a) === array_string;
};

var pageTitle = "New title";

Calling:

isArray(arr_user);

The problems here are:

  • All these are global (variable and functions) are shared among all the code in your application living in the global namespace. High chance of naming collisions between your and other developer’s code (think maintenance).
  • isEnabled above has no var declaration so automatically becomes chained to the global namespace outside the function.
  • When something in your code is chained to the top of the prototype it takes longer time to be found then if it’s within a module. I will discuss the JavaScript Prototype in another post and show examples with performance differences.
  • Some best practices are broken – see below what these best practices should be.

Module Pattern – the solution, organize your code

There are many great patterns described in JavaScript Patterns by Stoyan Stefanov but my preference today is to use the Module Pattern. The Module Pattern is widely used because it provides structure and helps organize your code into self-contained de-coupled pieces of code. If you look at the source of jQuery you will find a similar pattern being used.

The Module Pattern is a combination of several patterns described in Stoyan’s book.
They include:

  • Namespaces (Stefanov’s pg.87)
  • Immediate functions (Stefanov’s pg.73)
  • Private and privileged members (Stefanov’s pg.92) and
  • Declaring dependencies (Stefanov’s pg.90)

And here’s how the same sample of code above would look when the Module Pattern was followed.

Module Pattern code sample – the pattern

Declaring:

var MYCOMPANY = MYCOMPANY || {};

// note the use of namespacing - similar to jQuery.
MYCOMPANY.Page = (function () {
    'use strict';

    // [ private properties ]
    // note the chaining of var declarations vs a var on each line
    var pageTitle    = "blank",
        isEnabled    = false,
        array_string = "[object Array]",
        ops          = Object.prototype.toString,
        MAX_NUM      = 123; // constants always are in capital case
    // end var

    // [ private methods ]
    // note the use of literal patterns vs constructor functions
    function inArray(haystack, needle) {
        var i   = 0,
            max = 0;
        for (max = haystack.length; i < max; i += 1) {
            if (haystack[i] === needle) {
                return i;
            }
        }
        return -1;
    }
    function isArray(a) {
        return ops.call(a) === array_string; // end var
    }

    // [ public methods ]
    return {
        init: function (title, enabled) {
            pageTitle = title;
            isEnabled = enabled;
        },
        isArray: isArray,
        indexOf: inArray
    };
}());

Calling:

MYCOMPANY.Page.init('Page title', true);
MYCOMPANY.Page.isArray(arr_user);

Code is now modularized and less prone to collisions.

JSLint: The JavaScript Code Quality Tool

Once you have written your new modularized code it’s a best practice to run it past JSLint. JSLint is a free JavaScript program that looks for problems & violations of some of the patterns in your JavaScript code. It is a code quality tool. Check it out here: http://www.jslint.com/

JavaScript best practices

  • Curly brackets should always be used even if there is 1 expression. Code is then more readable and maintainable.
  • Use 4 space indentation – also JSLint default. This way no matter what editor engineers use it is always displayed in the same manner.
  • Like curly brackets, always use semicolons, even when they are implied by the JavaScript parser. Why slow down the compiler to clean up the mess during runtime.
  • Naming convention for constructors is typically UpperCamelCase, functions lowerCamelCase, variables lowercase_separated_by_underscore and constants UPPER_CASE.
  • Use whitespace as much as possible by separating all operators and their operands with spaces.
  • Use Google’s Closure Compiler to minify your code before production AFTER you have passed it via JSLint (above).

Appendix

1.1 Constructors and their corresponding and preferred literal patterns.

Built-in constructors (avoid) Literals and primitives (prefer)
var o = new Object(); var o = {};
var a = new Array(); var a = [];
var re = new RegExp( “[a-z]”,”g”); var re = /[a-z]/g;
var s = new String(); var s = “”;
var n = new Number(); var n = 0;
var b = new Boolean(); var b = false;
throw new Error(“uh-oh”); throw { name: “Error”, message: “uh-oh” };… or throw Error(“uh-oh”);

And that is it for now. Have I missed something or gotten something wrong above? If so please let me know and I will correct it. How is your JavaScript looking? Need help to get it fixed? Post below or contact me.

~ Ernest

MongoDB office hours in Mountain View

You may recall my previous post on MongoDB and how powerful it is as an alternative to a relational database. Since then I’ve had a bunch of discussions with other software engineers around this space and even met up with MongoDB core engineer Chris Westin from 10gen at Red Rock to gain further insights into MongoDB.

MongoDB Leaf
MongoDB Leaf

New kick ass GUI for MongoDB

Chris introduced me JMongoBrowser written by Antoine Girbal (10gen engineer). It’s written in Java so you can run it on Linux, Windows and Mac OSX. So far this GUI has proven to be a success and fills the holes where MongoHub couldn’t. Out goes MongoHub and in goes JMongoBrowser.

JMongoBrowser - MongoDB GUI admin tool.
JMongoBrowser - MongoDB GUI admin tool.

Fast crash recovery using Journaling

MongoDB uses memory 1st to write data to vs directly to file/store. This is where huge performance gains are attained. It also has Journaling, a write-ahead for operations to facilitate fast crash recovery in the storage engine. This means the stuff in memory is stored in a log incase your server goes down without affecting MongoDB’s performance.

So what happens if your box goes down? .. a common question amongst new engineers to MongoDB. Does this data also gets lost? The answer lays somewhere in between how you balance your performance needs against the risks you are willing to take with your data.

The journal is synced to disk every 100ms. So the maximum that can be lost is up to 100ms worth of changes. At the cost of additional performance degradation, you can make your application proof against even that.  The j option to getLastCommand will cause the application to block until the journal entries with the last change have been written to disk.  See http://www.mongodb.org/display/DOCS/getLastError+Command .  Of course, if you use this, your call to getLastCommand can wait up to 100ms, depending on where in the flush cycle you are. Therefore, this is left up to the user to change this default of 100ms.

Therefore, always load MongoDB with Journaling enabled, like this:

mongod --journal

This also auto cleans up crashes and puts crashed data back into MongoDB.

Don’t be alarmed when you see this

I found this in the /Journal (1GB files):

-rw------- 1 root root 1.0G 2011-06-23 02:26 prealloc.0
-rw------- 1 root root 1.0G 2011-06-23 02:25 prealloc.1
-rw------- 1 root root 1.0G 2011-06-23 02:26 prealloc.2

With Journaling enabled the server always creates those three 1GB files.  It rotates through them, recycling them.  They won’t grow any more.  But they are always that size, regardless of the size of your database. If the server dies unexpectedly, the files remain, and contain the material necessary for the automated recovery that happens when you restart the server.

More here:

Checking server memory usage

As mentioned above, Memory is used by Mongo to speed things up. The more memory you have the better and MongoDB will use your RAM as it sees need for it taking into consideration other server resources.

It’s always a good practice to check memory usage.
Details here: http://www.mongodb.org/display/DOCS/Checking+Server+Memory+Usage

Your MongoDB configuration file

Here’s a recommend set of switches to have enabled in your mongodb.conf.

sudo nano /etc/mongodb.conf

add or update your settings to these:

journal=true
directoryperdb=true
logappend=true
  • journal = as discussed above to enable fast recovery from crashes.
  • directoryperdb = creates a new physical directory for each new database. Clean way to seperate your databases.
  • logappend = Whether the log file will be appended or over-written. Always have true else after a reboot your old logs will be overwritten and you may lose important crash specific data.

More detail here: http://www.mongodb.org/display/DOCS/File+Based+Configuration

MySQL to MongoDB

Finally, I found this fantastic chart illustrating the difference in commands between MySQL and MongoDB. Should help the transition for us MySQL folks. Click the image below to download a large PDF version (Size: 213Kb).

SQL-to-MongoDB

There is also mapping chart SQL to Mongo located on the MongoDB website here.

Have more questions? Attend the weekly MongoDB Office Hours in Mountain View Red Rockhttp://www.meetup.com/San-Francisco-MongoDB-User-Group/events/16985746/ or post a question/comment below.

~ Ernest

Mercurial: how-to guide to distributed version control

Mercurial is one of the 2 famous free / open source, distributed source control management tools. You may also have heard of Git, another great tool. Both Mercurial & Git are the front runners in distributed version control systems. 2 nice articles compare Mercurial to Git. One called Git vs. Mercurial: Please Relax comparing Git to MacGyver and Mercurial to James Bond. The other article The Differences Between Mercurial and Git. The conclusion was both are great tools so stop worrying and start using the one which you are most comfortable with. I went down the Mercurial route.

Mercurial - Work easier, Work faster
Mercurial - Work easier, Work faster

Mercurial is a distributed version control systems (DVCS) which keeps track of software revisions and allows many developers to work on a given project without necessarily being connected to a common network. This is a peer-to-peer approach, as opposed to the client-server approach of centralized systems like Subversion or CVS. It is a more cleaner, faster and refined way of working. More on this is explained on Wiki here.

Let’s get started with Mercurial

As explained above you Mercurial is a DVCS. So you will have a “local” version of Mercurial which may have a number of “repositories” on your development machine. A repository is essentially a package which identifies your application. One for each application. Your code will get “committed” to this local version daily as you work. When you have a working piece of functionality only then you will “push” it to “central”. Central is typically hosted on a remote server (hosted in-house or via a 3rd party like BitBucket) where everyone else in your team can push or clone code to. Typically “central” is hooked up to a continuous integration tool to facilitate nightly builds. More on that in another post. So whatever code you push there make sure it is working!

Local (dev) install or Mercurial

1. Download Mercurial on your development box from here: http://mercurial.selenic.com/

2. Run the install file and follow through with the default questions.

3. Your done.

Quickstart – starting your 1st Mercurial Repository

Source: http://mercurial.selenic.com/

In your terminal / dos window, navigate to a folder where you will either clone a project to or create one and do the following.

Clone a project and push changes

$ hg clone http://selenic.com/repo/hello
$ cd hello
$ (edit files)
$ hg add (new files)
$ hg commit -m 'My changes'
$ hg push

Create a project and commit

$ hg init (project-directory)
$ cd (project-directory)
$ (add some files)
$ hg add
$ hg commit -m 'Initial commit'

Let’s look at each command in more detail:

hg init

Run this 1st time in your working directory to mark that folder as your project repository which you want Mercurial to version control. Mercurial will create a hidden folder there called “.hg”. This is where all the detail (changesets and manifests) on your commits is stored.

hg clone http://(my-domain)/hg/(project-repository)

Run this in your working directory where you want to drag code down from central Mercurial (See below on Central).

hg status

To see what is not yet added & committed yet to the repository.

hg add

If you have new files this command adds them to your repository.

hg commit

Commit your changes to “local” Mercurial. Make this a habit of doing it frequently (at least once a day). Mercurial will ask you to put in comments with your commit. Also make this a habit to put in as much detail into your comments as possible to help others understand your changes + if you have a reference to Jira case / other Agile tracker add this in as well.

hg pull -u

To pull changes and update.

hg push http://(my-domain)/hg/(project-repository)

To push your local repository changes to a central repository.

Tools for Mercurial

Netbeans – IDE Integration

If you are using Netbeans for development and have pointed it to a project where you “hg init” was ran, Netbeans will automatically recognize this and integrate Mercurial into your workflow. In Netbeans, do this by right clicking on your project folder and look under “Mercurial” sub-folder. Get it from here: http://netbeans.org/

MacHG – GUI for managing repositories on the Mac

This is the nicest GUI for Mercurial that I could find for the Mac. It allows you in a nice graphical way to manage a collection of files, to add things to the collection, to save a snapshot of the collection, to restore the collection to an earlier state and in general to work with the files. Get it from here: http://jasonfharris.com/machg/
Alternatively go through a list of other tools here: http://mercurial.selenic.com/wiki/OtherTools

“Central” – home of all repositories

The following is only required if you want to setup Mercurial “central”. A single place (mothership) for all developers to push working code to for storage & versioning. This is also the place where you would hook up a continuous integration tool to facilitate nightly builds.

Bitbucket: free Mercurial code hosting

If you do not have enough resources to run Mercurial “central” in-house or want someone else to manage that for you then use one of the hosted solutions on the internet. Bitbucket from Atlassian (kick ass Aussie firm) can provide you with unlimited private code hosting, free. Yes I use it and I love it. Less maintenance & headaches and everything you learnt above can be applied here.

BitBucket by Atlassian - free Mercurial code hosting
BitBucket by Atlassian - free Mercurial code hosting

Get your free account here: http://bitbucket.org/

Home solution: In-house Mercurial code hosting

Ok the following will require Ubuntu terminal access to the Ubuntu box where you want to install Mercurial “central”. All of the code below can be highlighted and pasted to run directly in your terminal window to make this process fast.

The following is based on the steps I followed from Emran Hasan’s Blog with some changes to the configuration to make it work on Ubuntu 10.10 based additional input from Stack Overflow.

1. Install Mercurial

sudo apt-get update
sudo apt-get install mercurial

2. Create a store for Mercurial configuration & repository files

cd /var/
sudo mkdir hg
sudo mkdir hg/repos
sudo chown -R www-data:www-data hg/repos

3. Creating a configuration file to host multiple repositories on this server & use CGI to serve the files through Apache:

cd /var/hg
sudo cp /usr/share/doc/mercurial/examples/hgweb.cgi .
sudo chmod a+x hgweb.cgi

4. Configure Mercurial (also refered to as “hg”) to point to point to our new config file:

sudo nano /var/hg/hgweb.cgi

and update the config like this:

config = "/var/hg/hgweb.config"

5. Create the file /var/hg/hgweb.config and write the location of the repositories:

sudo nano hgweb.config

and add this to the file:

[collections]
/var/hg/repos = /var/hg/repos

6. Make “central” accessible via HTTP (optional if you don’t want to setup a SSL certificate).

sudo nano /etc/mercurial/hgrc

add/update this line and save file:

[web]
allow_push = * push_ssl = false

7. Now update the Apache configuration so that it executes the CGI when requested with a /hg prefix:

cd /etc/apache2/sites-available
sudo nano default

at the end of the file (before < /VirtualHost >), add the following and save it:

ScriptAlias /hg "/var/hg/hgweb.cgi"
< Directory "/var/hg/repos" >
   AuthType Basic
   AuthName "Mercurial Repositories"
   AuthUserFile /var/hg/hgusers
   Require valid-user
< /Directory >

8. Add permissions for users who will be accessing central. First add admin account:

sudo cd /var/hg htpasswd -mc hgusers admin

then normal user accounts:

sudo htpasswd -m hgusers

Each request will prompt you for a password.

9. Create a repository

cd /var/hg/repos
sudo mkdir (my-repository)
cd (my-repository)
sudo hg init

Note: Recall “hg init” from above (local setup).

10. Restart apache

sudo /etc/init.d/apache2 restart

and browse: http://(my-domain)/hg
you will be prompted to login (using one of the users names you added above) and see the whole Mercurial central repository.

URL access: http://(my-domain)/hg/(project-repository)
Cloning from “central” to your local:

cd (my-local-project-working-directory)
hg clone http://(my-domain)/hg/(project-repository)

And… that concludes this short introduction to Mercurial. Hope you found it useful!

Bonus: Download this Mercurial Cheat Sheet:
http://www.theroadtosiliconvalley.com/cdn/Mercurial-Usage-v1.0.pdf (847kb)

Mercurial Usage - 1 pager
Mercurial Usage - 1 pager

I have it at my desk. It is the best one I could find out of the many I reviewed.

Happy coding!

Useful links

Hosted Mercurial solutions

More Mercurial education

Bitbucket education

Enjoy!

~ Ernest

MongoDB {name: “mongo”, type: “db”}

MongoDB {name: "mongo", type: "db"}
MongoDB {name: "mongo", type: "db"}

<strong opinions>

Whoah! That was my 1st and 2nd, and 3rd.. and goes on… and on.. impression of this amazing database. Having experience with SQL Server and MySQL (both relational databases) for a few good years I decided to take this NoSQL database for a real world run. I also think the massive signs for MongoDB conference in San Francisco on the 101 had subliminally stamped a mark on my neurons 😉 I also read some interesting articles here and here comparing MongoDB to other NoSQL solutions.. After all that I was convinced that MongoDB would be the NoSQL database I would invest some serious time into.

As expressed above, I was impressed by MongoDB. I hooked it into a Zend MVC (Model–view–controller) application accessing Shanty-Mongo ORM through custom Model classes which I wrote from ground up. Everything just fit in so snugly.. and when I threw data against MongoDB it created collections (SQL world you’d call this tables) on the fly. Yes on the fly! That was super cool – loosely coupled interfacing – > create a Class Model and let the DB handle the rest. Super cool. Plus, this baby just flies! Everything from how fast it retrieves, stores, updates (even partial updates) and searches your data to how it stores it as Binary JSON both on the file structure and in memory (during open connection) on the server to speed things up. Everything about this database is impressive.

</strong opinions>

Ok enough of my ramblings. I think you get the picture. I am impressed.

If you are impressed and want to give MongoDB a try, read on. Next let’s dig in and explore stuff that is important (and what I learnt) about MongoDB, how to set it up and common commands to keep handy when working in the terminal.

Hello MongoDB

“MongoDB (from “humongous”) is a scalable, high-performance, open source, document-oriented database. MongoDB bridges the gap between key-value stores (which are fast and highly scalable) and traditional RDBMS systems (which provide rich queries and deep functionality).” ~ from MongoDB

Then (RDBMS) and now

  • Tables as you know in SQL are called “collections” in MongoDB.
  • Relational DB has records (record sets), MongoDB calls them “documents”.
  • MongoDB stores all data in JSON objects and serialized to BSON (Binary JSON) for storage. CouchDB (you may also know of) stores in just JSON.
  • In MongoDB, “ObjectId” in a collection is similar to auto-incrementing ID in a Relational database table.
  • Here’s a nice mapping chart between SQL and Mongo: http://www.mongodb.org/display/DOCS/SQL+to+Mongo+Mapping+Chart

More FYI notes:

  • You never create a database or collection. MongoDB does not require that you do so. As soon as you insert something, MongoDB creates the underlying collection and database.
  • If you query a collection that does not exist, MongoDB treats it as an empty collection.
  • Switching to a database with the use command won’t immediately create the database – the database is created lazily the first time data is inserted. This means that if you use a database for the first time it won’t show up in the list provided by `show dbs` until data is inserted.
  • Mongo uses memory mapped files to access data, which results in large numbers being displayed in tools like top for the mongod process. Think performance! You can get a feel for the “inherent” memory footprint of Mongo by starting it fresh, with no connections, with an empty /data/db directory and looking at the resident bytes.

Installing MongoDB on a Debian OS (Ubuntu)

If you’re using Ubuntu Server (I used 10.10), you can also install MongoDB using aptitude. Default Ubuntu sources do not contain MongoDB so you need to add distro location to your /etc/apt/sources.list file. That is easily done. 1st open sources.list in terminal editor (nano) like this:

sudo nano /etc/apt/sources.list

and drop & save this line to the end of the file:

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

Exit nano and add the following 10gen GPG key, or apt will disable the repository (apt uses encryption keys to verify the repository is trusted and disables untrusted ones).

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

Now your ready to install the package with aptitude by executing the following commands:

shell> sudo apt-get update
shell> sudo apt-get install mongodb-10gen

Finally, fork mongo as a Daemon (to auto run on boot).

shell> sudo mongod --fork --logpath /var/log/mongodb.log --logappend

You can now use the command-line client to access the MongoDB database server, as below:

shell> mongo

You may want to hook up MongoDB to be accessible from your PHP application by editing your php.ini and allowing mongo to run as an extension. 1st find & edit your php.ini location like this:

sudo find / -name php.ini
sudo nano /php.ini

Then add & save mongo as an extension inside php.ini under “Dynamic Extensions”:

extension=mongo.so

Save & restart Apache:

sudo /etc/init.d/apache2 restart

Common MongoDB commands

Here’s a short list of the most common commands you will end up using when interfacing with the database. If you want to use a GUI to access MongoDB, I found MongoHub the best GUI administration tool for Mac. There is also a very comprehensive MongoDB documentation located here.

Purpose Shell Command
Login to interface mongo
Show all dbs on record show dbs
Switch to my database use mydb
Show all collections on record show collections
List db version db.version()
Insert data into a new collection db.items.insert({ name:’eggs’, quantity: 10, price: 1.50 })
Display a whole list of documents in a collection db.items.find({})
Display a select list of documents in a collection db.items.find({guid:xyz})
Remove a whole list of documents in a collection db.guid.remove({})
or where n == 1
db.things.remove({n:1});
Drop the collection db.<>.drop()

Cons: when you delete a document you cannot return it’s ObjectId. Would be nice to have this feature. MongoDB folks?

MongoDB start ritual (habit forming):

  1. mongo
  2. show dbs
  3. use mydbname
  4. show collections

This will become a habit so don’t resist.

Don’t forget to read the follow up to this post located here: http://www.theroadtosiliconvalley.com/technology/mongodb-update/
Plenty of new knowledge on tools, crash recovery & best practices.

Give MongoDB a spin

If you are in doubt, give MongoDB a spin and make up your own mind ~ http://www.mongodb.org/
Don’t forget to let me know how your experience goes.. and if you have questions on getting this setup please contact me and I will be more than happy to help you out!

~ Ernest

Recommended links

EC2: how to launch Ubuntu into the cloud

Amazon Elastic Compute Cloud (EC2) delivers scalable, pay-as-you-go compute capacity in the cloud. It is a part of a collection of remote computing services (also called web services) from Amazon that together make up a cloud computing platform. The most central and well-known of these services are Amazon EC2 and Amazon S3. The goal in this post is to get you up and running on an EC2 instance super fast.

Get familiar

I’m going to assume you have an account with AWS and are familiar with:

If you do not know those 3, please spend some time learning about them by following the links in the bullet points above. Else, let’s get rollin.

Step by step guide

1. Setting up security

Before we launch an Instance you need to do some Pre-work since both “Security Group” and “Key Pair Name” cannot be changed once an Instance is mapped to one and started.

  1. Setup a “Security Group” for your new Linux Instance.
    1. From Navigation menu select “Security Groups”.
    2. Click on “Create Security Group” button and fill out the form giving your security group a very descriptive name.
    3. Click “Yes, Create” button, select the new group and in the lower half window/frame press the “Inbound” tab.
    4. Inbound allows you to open ports on this Instance. You can add or remove these after the Instance is created. By default allow these: SSH (22), HTTP (80) and MySQL (3306). For extra security limit (source) SSH & MySQL to only your IP address. If you plan to install Webmin add port 10000 here too.
    5. When done, click on “Apply Rule Changes”.
  2. Create a “Key Pair Name”.
    1. This is super important and will be used for accessing your Instance both via SSH & sFTP.
    2. From Navigation menu select “Key Pairs”.
    3. Click on “Create Key Pair” button.
    4. Give it a descriptive Key Pair Name and click on “Create” button. A private key with extension .pem will download. Save this in a secure location since this is your key to access your Instance.
    5. On your local machine (Linux X or Mac OS X), give this file more secure permissions like this:
      chmod 0700 ./keys/mykey.pem

2. Launching an Instance

  1. From Navigation menu select “Instances”.
  2. Click on “Launch Instance” button.
  3. This launches the Request Instance Wizard where you can select an Amazon Machine Image (AMI). Note that Ubuntu is only available from “Community AMIs”. Click the Community AMIs tab.
    1. Here is a list of available Ubuntu images:
      http://uec-images.ubuntu.com/releases/10.10/release/
    2. Make sure you use an EBS root store – it’s better. For benefits see here:
      http://stackoverflow.com/questions/3630506/benefits-of-ebs-vs-instance-store-and-vice-versa
  4. Step through the 5 stages of the Wizard and click on “Launch” button. This will launch your new Instance. The Wizard is straight forward and you will most likely go with all the defaults.
  5. Your Linux Instance will launch pretty fast. You should now see your Instance listed under “My Instances”.
  6. Click on your Instance. Instance properties window/frame shows up in the bottom half of the console. Note down “Public DNS” and “Private DNS/IP Address”. You will need those to access the box – especially the Public DNS.

3. Building a Ubuntu LAMP Web Server on your new Instance

This step is optional.

But should you want to setup LAMP on this new Instance follow the steps outlined in my previous post here: http://www.theroadtosiliconvalley.com/technology/building-ubuntu-lamp-web-server-vm/

The only additions in light of Amazon EC2 host are:

  • When using SSH/sFTP use the private key with extension .pem you downloaded above.
  • Note that root user in EC2 is “ubuntu” not “root” like in a VM Ubuntu setup.
  • To SSH into your new EC2 Instance do this in terminal where the URL after @ is your Public DNS:
    ssh -i ./keys/mykey.pem ubuntu@region.compute.amazonaws.com
  • Use the Public DNS or setup a static IP address to point to your Instance(s). Amazon calls this Elastic IP Address and this allows you to have multiple Instances all pointing to the 1 IP address for dynamic cloud computing.

Now go and build kick ass products!

There you have it folks. How simple is that. Amazon makes cloud computing look simple and launching new servers (Instances) is a breeze.. in a matter of minutes.

If you found this post useful let me know in comments section below. Super!!

~ Ernest

 

Building an Ubuntu LAMP Web Server

Recently I was setting up my Mac OS X with a kick ass development environment and jotted down all the cool steps I took to build an Ubuntu LAMP web server in a virtual machine environment. Here is this in-depth guide translated from paper to this digital copy. Hope you find this guide valuable and it saves you time when you need to do the same.

LAMP (Linux, Apache, MySQL and PHP)
LAMP (Linux, Apache, MySQL and PHP)

Ubuntu Server

The flavor of Linux I like to use as a Web Server is Ubuntu.

What is Ubuntu

Ubuntu , is a secure, intuitive operating system that powers desktops, servers, netbooks and laptops. It is based on the Debian GNU/Linux distribution. Ubuntu is also named after the Southern African ethical ideology Ubuntu (“humanity towards others”) and is distributed as free and open source software with additional proprietary software available.

Why Ubuntu

  1. Reduce costs – free to use with no licensing fees.
  2. Visualization – it runs beautifully & fast in any VM environment (esp. Mac OS X)
  3. Build-in security – tight security, inbuilt firewall and encryption.
  4. It based on a Debian Distribution. A computer operating system composed of software packages released as free and open source software especially under the GNU General Public License and other free software licenses. Debian distributions are slower to release but this means they are extremely thorough.
  5. A lot of the big boys use Ubuntu. See case studies here: http://www.ubuntu.com/business/case-studies

Step by Step – your 1st web server

This guide assumes you have already installed Ubuntu Server. If not, go here and do it first. I recommend you install Ubuntu Server in a VM. I use VMware Fusion to run my instances when developing and Amazon EC2 for production. This guarantees that whatever I do locally in a VM will be compatible when pushed into production.

Ubuntu Server in a VMware Fusion
Ubuntu Server in a VMware Fusion

Goal:

  • Install LAMP – Linux (already done), Apache (web server), MySQL (mysql) and PHP (code compiler).
  • Install Webmin – a web-based interface for system administration for Unix.
  • Allow WWW for sFTP so you can remotely manage your website using a GUI.
  • Setup access to MySQL using MySQL Workbench.

1. Install LAMP

  • SSH into your box as root on Port 22 (default post install).
  • Update your OS software (just in case you are missing some dependencies):
    sudo apt-get update
  • From the terminal window, install LAMP using this 1 line of code (the caret (^) must be included):
    sudo apt-get install lamp-server^
  • The apt package manager will display what it is installing and ask you a bunch of standard questions. Just say yes to all. You will also be asked for a password for your new MySQL database. Type that in and note this down for future.
  • When this finishes you are done. Easy hey! Port 80 (default web server port) is now enabled and pointing to ‘/var/www’. ‘/var/www’ is where your site(s) should be placed.
  • Hit the Public DNS URL of your server (typically your IP) to verify that it’s up. It should show up a page with “It works!” If you are not sure what your box’s IP is, type this in and hit enter (similar to ipconfig on a Windows box).
    ip route
  • Before moving to the next step, you may want to know information about PHP’s configuration inc. installed extensions. You can grab this by creating a PHP file from your terminal window like this:
     sudo nano /var/www/phpinfo.php

    then adding this into it, save it, and quit nano (the editor your in):

    <?php phpinfo(); ?>

    restart Apache:

    sudo /etc/init.d/apache2 restart

    Hit the IP in your browser again with this new file name appended to the end eg. http://170.10.105.110/phpinfo.php – it should show you what is running.

2. Install Webmin

  • Edit “/etc/apt/sources.list” to add 2 new source:
    sudo nano /etc/apt/sources.list
  • … add these 2 new lines to the end, save and exit:
    deb http://download.webmin.com/download/repository sarge contrib
    deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib
  • Now you can run this in your terminal window to install Webmin.
    sudo apt-get update
    sudo apt-get install webmin
  • Webmin should now be accessible from your browser using the server’s ip address followed by port 10,000 eg. https://170.10.105.110:10000
    Note that you do not have HTTPS cert so your browser will throw a warning since https is (and has to be) the protocol. Ignore it and move forward.
  • If you cannot login with your sudo account you may need to enable root. Follow the steps outlined here: https://help.ubuntu.com/community/WebminWithoutARootAccount
  • Or you can change the password of the root user in your terminal window. Then restart webmin.
    sudo /usr/share/webmin/changepass.pl /etc/webmin/ root foo
    sudo /etc/init.d/webmin restart
    
  • If you need to restart webmin run this:
    sudo /etc/init.d/webmin restart

3. Allow WWW for sFTP

  • You need to make sure the group www-data is added to “/var/www”. Run this in your terminal window:
    sudo chgrp www-data /var/www
  • Make “/var/www” writable for the group.
    sudo chmod 775 /var/www
  • Set the GID for www-data for all sub-folders.
    sudo chmod g+s /var/www
  • Your directory should look like this on an ‘ls -l’ output.
    drwxrwsr-x    root www-data
  • Last, add your user name to the www-data group (secondary group) where USERNAME is the “new” username you will use to sFTP. Note that we follow it by “passwd” to give new account a password.
    sudo useradd -G www-data NEW_USERNAME
    sudo passwd NEW_USER

    OR if the username is “existing” one use the command below. Also don’t forget to add “ubuntu” user if you have set this up on an EC2:

    sudo usermod -a -G www-data EXISTING_USERNAME
  • You should now be able to SFTP to your server using this USERNAME and upload data to “/var/www” with no problems.

4. Access to MySQL using MySQL Workbench

  • MySQL Workbench is a nice free GUI tool by the folks at mysql.com to manage your MySQL database. It can be downloaded from here: http://wb.mysql.com/
  • By default MySQL listens on localhost (127.0.0.1) so if you are going to manage your Ubuntu VM instance from say OS X, MySQL wont allow you entry. Here’s what to do to grant remote management of MySQL.
    1. Go to Webmin and login.
    2. In Webmin, navigate here: Servers > MySQL Database Server > MySQL Server Configuration
    3. Change “MySQL server listening address” to “Any”. By default it is 127.0.0.1. Save this.
    4. Now navigate here: Servers > MySQL Database Server > User Permissions
    5. Click on User “root” on the line where it says 127.0.0.1. And under Hosts change it to “Any”. This set the permissions on your db access.
    6. Save & Restart MySQL and you are done.
  • Remember that this is for “development” purposes only. You would not be allowing “Any” to your DB rather a specific static address and username.

5. Bonus – running multiple web applications on the LAMP instance

To save on time, money and managing multiple boxes, you may want to run multiple websites from this same box. I like to do this using ports as the separator. The following can be done in Webmin:

  1. Upload code to /var/www/mynewsite/
  2. Create a Virtual host for your new web application by navigating to:
    Servers > Apache Webserver > Create virtual host
  3. Fill out the form pointing ‘Document Root’ to the location of your code and assign a ‘Port’ number eg. 81, to this new host. Remember port 80 is your default.
  4. Save and click on ‘Apply Settings’ (link top right of the Webmin interface).
  5. Finally you need to tell Apache to listen to this new port. Navigate here:
    Servers > Apache Webserver > Global configuration > Networking and Addresses
  6. Add port 81 (where your new host is configured on) to ‘Listen on addresses and ports’.
  7. Save, apply changes and restart Apache.
  8. Done. You can now access your website via http://IP_DNS:81

Now go and build kick ass products!

There you have it folks. How simple is that. That’s why I love Ubuntu so much. It’s simple and powerful all under the 1 umbrella. That’s how software should be. All the complexities removed so us engineers can get to work and build kick ass products!

If you found this post useful let me know in comments section below. Super!!

~ Ernest