Skip to content

Getting Started

Jump to bottom Edit New page
rehanift edited this page Oct 5, 2012 · 2 revisions

Chapter 1 - Getting Started with Engine.js

Install the package

First, install the package engine.js with NPM

$ npm install engine.js

Start the Server

Now we need to start intake, cylinder, and exhaust components. These three components make up the "server-side" of an Engine.js application.

Create a new file app.js and place the following code in the new file.

var engine = require("engine.js").engine,
    intake = engine.intake.create(),
    cylinder = engine.cylinder.create(),
    exhaust = engine.exhaust.create();

Note: For the sake of simplicity we're starting the server components from within the application. Normally these components are started in a separate process(es).

Create the Client

Now we need to create a client. In your app.js file:

var client = engine.client.create();

The client will create task objects and send them through the server components (intake, cylinder, exhaust). Results from the task will then be sent back to the client.

Create the Task

A unit of work in an Engine.js application is a task. A task contains all the information it needs to execute.

In your app.js file add the following:

var task = client.createTask();
task.setCode("1+1");

By default, a task's code is executed in an empty Javascript environment. The code only has access to built-in Javascript objects. Later we will see how to add custom objects to execution environment.

Run the Task

A task is run asynchronously. When the task has completed the task will emit an eval event passing:

  • err: null unless there was a problem before a task was executed.
  • response: An instance of a TaskResponse object.

In your app.js file add the following:

task.on('eval', function(err, response){
  console.log("task evaluation:", response.getEvaluation());
});

task.run();

Your app.js file should look like this:

var engine = require("engine.js").engine,
    intake = engine.intake.create(),
    cylinder = engine.cylinder.create(),
    exhaust = engine.exhaust.create();
    
var client = engine.client.create();

var task = client.createTask();
task.setCode("1+1");

task.on("eval", function(err, response){
    console.log("task evaluation:",response.getEvaluation());
});

task.run();

If you run node app.js you should see task evaluation: 2

You should also notice that the process does not immediately exit. This is because the client and server components are still connected. To fix this, close the client and server components within the "eval" event's callback:

task.on("eval", function(err, response){
    console.log("task evaluation:", response.getEvaluation());
    
    client.close();
    intake.close();
    cylinder.close();
    exhaust.close();
});

Note: Normally the server components are started in separate process(es) so only closing the client is usually necessary.

Chapter 2 - Extending the Execution Environment

In the last chapter we showed how to execute code in an empty Javascript environment. In this chapter will see show how to add your own functions and runtime data to the execution environment.

An Engine.js Task should include all the information it needs to execute. This includes three things:

  1. An Execution Context (Context)
  2. Local Variable Bindings (Locals)
  3. User Code (Code)

Context: A task's Context explicitly defines the global object in which a task's Code is run. A task's Code does not have access to anything except the defined global object and built-in Javascript objects.

A Context is a Javascript function value (an anonymous function wrapped in parens) that, when invoked, returns an object-literal. This process is known as "Rendering the Context". The Context takes one argument (locals) which binds when the Context is rendered.

For example:

(function(locals){
    return {
        foo: "bar",
        hello: locals.world,
        qux: function() {
          return "qal";
        }
    };
})

In this example the global object will be populated with a foo object whose value is "bar", a hello object whose value is derived from the locals variable hash (see below), and a qux object which is a function. The task's Code will only have access to these three objects (in addition to the built-in objects like Date, Array, etc.).

A Context is set on a Task object using the setContext method. It expects a String.

Local Variable Bindings: There are times when your task will depend on runtime information. The locals argument of the Context function allows you to pass-in an object literal of key/value pairs which will be injected into your rendered Context.

In the example above, our Context references locals.world. If we set the task's locals property to:

{ world: "Mundo" }

then the locals.world object in the rendered Context would be "Mundo"

Locals are set on a Task object using the setLocals method. It expects an object-literal.

Code: The core of the task is the Code to be executed. Code is just Plain Old Javascript (tm). It is executed in the context of the task's corresponding rendered Context. It is set on the task object as a String.

Code is set on a Task object using the setCode method. It expects a String.

Adding new functions to the execution enviornment

Your app.js file from before should look like this:

var engine = require("engine.js").engine,
    intake = engine.intake.create(),
    cylinder = engine.cylinder.create(),
    exhaust = engine.exhaust.create();
    
var client = engine.client.create();

var task = client.createTask();
task.setCode("1+1");

task.on("eval", function(err, response){
    console.log("task evaluation:",response.getEvaluation());
    
    client.close();
    intake.close();
    cylinder.close();
    exhaust.close();    
});

task.run();

Lets define an increment function that will increment a value by 1. The definition of the function looks like this:

function(value){
    return value + 1;
}

Recall that a Context must be a function that returns an object-literal. An empty Context looks like this:

(function(locals){
    return {
    
    }
})

To add the increment function to our Context we must add an increment key to the returned object-literal. Use the previously mentioned function definition as the increment key's value.

(function(locals){
    return {
        increment: function(value){
            return value + 1;
        }
    }
})

Put the above in a new file called context.js. Like a task's code, a task's context is set on the task object as a string. Keeping the code and context in seperate files allows us to easily read them as strings while keeping their contents managable in a text editor.

To set the Context on the task, read in the contents of the context.js file and set the Context on the task:

var fs = require("fs");
var context = fs.readFileSync(__dirname + "/context.js","utf-8");

task.setContext(context);

Now you may call your new increment function from your Code:

task.setCode("increment(2)");

Your app.js file should look like this now:

var engine = require("engine.js").engine,
    intake = engine.intake.create(),
    cylinder = engine.cylinder.create(),
    exhaust = engine.exhaust.create();
    
var client = engine.client.create();

var task = client.createTask();
task.setCode("increment(2)");

var fs = require("fs");
var context = fs.readFileSync(__dirname + "/context.js","utf-8");
task.setContext(context);

task.on("eval", function(err, response){
    console.log("task evaluation:",response.getEvaluation());
    
    client.close();
    intake.close();
    cylinder.close();
    exhaust.close();    
});

task.run();

Run node app.js and you should see task evaluation: 3

Injecting runtime data

Sometimes the execution of a task will depend on information supplied at runtime. This is what Locals are intended for.

Lets modify our existing increment function. Rather than increment the passed-in value by 1, we will determine the value to increment by at runtime.

Modify your context.js file to look like this:

(function(locals){
    return {
        increment: function(value){
            return locals.increment_by + value;
        }
    }
})

The value of the locals argument is set on the task object with the setLocals method. Unlike the values accepted by setCode and setContext, the setLocals method accepts an object-literal of key/value pairs. Since the increment function in our context references locals.increment_by, our Locals must have an increment_by key.

Set the Locals for your task in app.js:

var engine = require("engine.js").engine,
    intake = engine.intake.create(),
    cylinder = engine.cylinder.create(),
    exhaust = engine.exhaust.create();
    
var client = engine.client.create();

var task = client.createTask();
task.setCode("increment(2)");

var fs = require("fs");
var context = fs.readFileSync(__dirname + "/context.js","utf-8");
task.setContext(context);

task.setLocals({
    increment_by: 3
});

task.on("eval", function(err, response){
    console.log("task evaluation:",response.getEvaluation());
    
    client.close();
    intake.close();
    cylinder.close();
    exhaust.close();    
});

task.run();

Run node app.js and you should see task evaluation: 5

Locals cannot be directly accessed by Code. Properties on the locals argument can only be accessed my functions in the Context.

Chapter - Architecture & Configuration

In the last chapter we saw how to run a Task with Engine.js. In this chapter we're going to look at how Engine.js is architected and how it can be configured.

As you might have guessed, Engine.js uses the metaphor of a combustion engine to name its internal components.

Components

  • Client: The Client is responsible for create new Task objects and sending them the Intake. It is also responsible for receiving TaskResponse objects from the Exhaust
  • Intake: The Intake is responsible for accepting incoming Task messages from many clients and forwarding them to many Cylinders
  • Cylinder: When a Cylinder is created it starts its own Piston (in a separate child process). When a Cylinder receives a Task it starts an execution timer and sends the Task to its Piston for execution. When the Piston returns a result the Cylinder forwards it to the Exhaust
    • If the execution timer finishes before a response from the Piston is received, the Piston process is killed and an error is sent back to the Client (via the Exhaust).
  • Piston: The Piston is responsible for creating a code sandbox from the received Task messages's context and run-time variable bindings. The Piston then runs the user-code against the sandbox and returns the results to its parent Cylinder.
  • Exhaust: The Exhaust receives Task result messages from many Cylinders and forwards then back to the appropriate Client.

All components use ZeroMQ to communicate with each other. This allows us to start the components (Intake, Cylinder, and Exhaust) in individual processes, even on different machines (as long as they are in the same network). In the previous chapter we started all of the components in the same process, but this was only for convenience. The ZeroMQ endpoints are configurable for most components.

                                                                             +------------+      +------------+
                                                                        -----|            |      |            |
 +------+                                                               |    |  Cylinder  |<---->|   Piston   |
 | Task +-+   +------------+                   +------------+           |    |            |      |            |
 +------+ |   |            |                   |            |           |    +------------+      +------------+
    (n)   +---+   Client   +----+     +--->----|   Intake   |-->-+      |          (n)
 +------+ |   |            |    |     |        |            |    |      |    +------------+      +------------+
 | Task +-+   +------------+    |     |        +------------+    |      |    |            |      |            |
 +------+          (n)          |     |              (1)         +-->---+    |  Cylinder  |<---->+   Piston   |
    (n)                         +-----+                                 |    |            |      |            |
                                |     |                          +--<---+    +------------+      +------------+
              +------------+    |     |        +------------+    |      |          (n)
 +------+     |            |    |     |        |            |    |      |    +------------+      +------------+
 | Task +-----+   Client   +----+     +---<----|  Exhaust   +-<--+      |    |            |      |            |
 +------+     |            |                   |            |           |    |  Cylinder  |<---->+   Piston   |
    (n)       +------------+                   +------------+           |    |            |      |            |
                   (n)                               (1)                |    +------------+      +------------+
                                                                        |          (n)
                                                                        |    +------------+      +------------+
                                                                        |    |            |      |            |
                                                                        +----|  Cylinder  |<---->|   Piston   |
                                                                             |            |      |            |
                                                                             +------------+      +------------+
                                                                                   (n)

Chapter - Concurrency

  • Start multiple Cylinders
  • ZeroMQ round-robin balancing

Chapter - Logging

  • Standard Logger
  • Custom Logger

Chapter - Console

  • TaskResponse.getDebug()

Chapter - Security

There are several measures taken to secure Engine.js

  • Child process execution
  • V8 Context membrane
  • Execution timeouts
  • Function.toString() removal

Chapter - Advanced Contexts

  • Asynchronous Functions
  • Using 3rd Party Modules
Add a custom footer
Add a custom sidebar
Clone this wiki locally