Hello! This is Pa11y Sidekick's architecture guide. We're really excited to show you how to get started with contributing.
By the end of this guide, you should feel comfortable with the general architecture of the application. We also maintain sections geared towards people who specialise in different areas of development.
This guide does not cover basic running, testing, or deployment of Pa11y Sidekick. Refer to the README for these.
Pa11y Sidekick is a Node.js application. There are a few skills you'll need before understanding/contributing to different parts of the application. We recommend reading these sections after the basics.
-
Back End:
Beginner–intermediate Node.js knowledge will be required. We make use of Promises via async functions for the core library and models. A working knowledge of Express is helpful but not required. -
Front End:
Either HTML/CSS knowledge and/or client-side JavaScript knowledge will be required. For templating, we use Dust.js. An understanding of these is useful but not a prerequisite. -
Databases:
We use PostgreSQL to persist data, and the Bookshelf and Knex.js libraries to communicate with it.
Once you've read up on your preferred areas, you should also review the section on testing.
OK, are you ready? This part of the guide covers things that are useful for all developers to know. We'll be covering the general application structure and where to find things.
The directory structure of Pa11y Sidekick can be seen below. We'll cover each of these folders in turn.
Sidekick
├── controller
├── data
│ ├── migration
│ └── seed
├── lib
│ └── middleware
├── model
├── public
├── script
├── test
│ ├── integration
│ └── unit
└── view
├── helper
├── layout
├── partial
└── template
This is where application controllers live. These files mostly handle routing through Express. Read the controllers documentation for more information.
This is where database migrations live. Migrations are handled by Knex.js, and files in this folder are timestamped and generated by a script. Read the migrations documentation for more information.
This is where database seed data lives, seeding is handled by Knex.js. Read the seeding documentation for more information.
The lib folder is where the core code can be found – the files in here tie together the whole application. Read the entry-points documentation for more information.
This folder contains Express middleware that is used by multiple routes. Read the middleware documentation for more information.
This is where application models live – the only code that should interface with the database. These models use Bookshelf and Knex.js. Read the models documentation for more information.
The public folder is used to serve static assets that don't require any additional back end logic applied, e.g. images. Read the static files documentation for more information.
This folder contains Node.js scripts that are used as part of the build process. Most of these aren't interfaced with directly, they're used by Make.
This is where integration tests live. These tests build and run the entire application, and verify that each page is working as expected. Read the integration tests documentation for more information.
This folder is where the application's unit tests can be found. These test individual files and make heavy use of mocking. The directory structure in here mimics the main application structure. Read the unit tests documentation for more information.
This folder contains [Dust.js helpers and filters][dust helpers], which are JavaScript files. These are used to extend the templating language. Read the view helpers documentation for more information.
View layout templates live in here. These contain most of the boilerplate HTML required to render a page and are used by the application's view files. Read the layouts documentation for more information.
This is where view partials live. Partials are used to reduce repetition across the views, and when the same elements are needed on multiple pages. Read the partials documentation for more information.
This is where most of the application's view layer lives, this is mostly Dust.js files. Read the view templates documentation for more information.
Hiya. If you're all caught up on the application structure then we can start looking into back end specific stuff in more detail! We'll be learning about the application core, how to create Express routes/controllers/middleware, and how to connect the different parts of the application. This section doesn't cover databases, that has a section of its own.
The best place to start is with index.js. This is the main entry-point for the application. You'll notice that this file is very light-weight, that's because its only job is to take config values from the environment and pass them into the library code.
The Dashboard class comes from lib/dashboard.js, which is the most important file in the application – it ties everything else together. The file itself is commented, and should have excellent test coverage. If you're interested in making changes to this file, then head on over to it and read through the comments.
Controllers live in the controller folder and register multiple routes with Express. The controllers are split into files based on the part of the application that they relate to.
Each controller must export a function that accepts a single argument. That argument will be a dashboard object, which is created by lib/dashboard.js. When the application starts lib/dashboard.js, controller/api-v1/index.js, and controller/front-end/index.js decide which files are loaded – you'll need to modify one of these to load your new controller in.
Controllers often load and mount other middleware, as well as using models to access the database. You should look into the existing controllers for examples of this.
Middleware can be used to reduce repetition between controllers, middleware files live in the middleware folder. These files don't have a prescribed structure or exports.
For logging, we use winston and morgan. Winston is set up outside of the main application files in index.js, this allows us to inject a mock/silent logger in the integration tests.
Morgan is used to log requests, and this is mounted as an Express middleware. We actually stream morgan's request logs into winston under the verbose level. This allows users to easily ignore them by tweaking their preferred log level.
OK, so you want to know how to work on Pa11y Sidekick's front end? Let's do it! This section will explain how to add templates (with Dust.js), CSS, and client-side JavaScript. We also cover static files.
View templates live in the view folder, and are written in the Dust.js templating language. We make use of blocks and inline partials for template inheritance, and try to split repeated code into partials. Here's an example view:
<!-- Extend the main layout -->
{>"layout/full"/}
<!-- insert the following into the layout title block -->
{< title}
My Page
{/title}
<!-- insert the following into the layout content block -->
{< content}
<h1>Hello World!</h1>
<p>This is data from the controller: {example}</p>
{/content}In order to see your view rendered in-browser, a controller will have to reference it. Take a look at the controller documentation for more information on how to do this.
The template/error.dust view is a special one – it's used to render errors that get caught by the application.
As mentioned above, we use blocks and inline partials for template inheritance. This allows for multiple modifiable blocks in our layouts which each view can add data to.
The main layout is used by most views; it provides boilerplate HTML and loads in the required scripts and styles.
Partials are used to reduce repetition in view files, and common markup patterns should be moved into partials. The best way to get acquainted with how we use partials is to look through the folder.
View helpers are used to extend the templating language, and to ensure that your views themselves don't have as much logic in them. You should avoid writing these unless absolutely necessary – try to get the data into the format you need in the controller first.
These files live in the view/helper folder; they should be JavaScript files that export a function which accepts a Dust.js instance as an argument:
module.exports = dust => {
dust.helpers.currentYear = chunk => {
const year = (new Date()).getFullYear();
return chunk.write(year);
};
};Read the [Dust.js helpers and filters][dust helpers] documentation for more information, or take a look at some of the existing helpers.
All files in the public folder are served by an Express static middleware. This is where you should put images and assets that need no back end processing.
TODO CSS should probably be compiled from Sass, we haven't written this yet. Currently All CSS is in a static file at public/main.css.
TODO client-side JavaScript should probably be compiled via Webpack or Browserify, we haven't written this yet. Currently there is no client-side JavaScript.
Hello! This section will teach you how to manage Pa11y Sidekick's database. We'll be covering models, migrations, and seeding. We use PostgreSQL to persist data with Bookshelf and Knex.js as client libraries. We use shortid to create url-friendly unique identifiers.
Models live in the model folder and are used to access the database using Bookshelf as an ORM. Each model file represents a single table in the database, and consistency between models is encouraged.
Each model must export a function that accepts a single argument. That argument will be a dashboard object, which is created by lib/dashboard.js. The model function must return a Bookshelf model. You need to update lib/dashboard.js to include new models before they will be loaded in.
Models must be added to the dashboard.model object, so that they can be referenced as dashboard.model.Example in the rest of the codebase.
Migrations are created by using a script, you should never create these files manually. Run the following to create a new migration file:
./script/create-migration.js <migration-name>The migration name should be descriptive of the changes you're making. Once you've created the migration file, it's worth copying the contents of an existing one so that the code style is the same. You should read the Knex.js documentation on migrations for more information on the structure of these files.
Migrations can be run or rolled back using the following make targets (which correspond to files in the script folder):
make db-migrate-up
make db-migrate-downSeed files live in data/seed, and are loaded into a database through either a Make target or automatically when integration tests are run. It's useful to copy an existing seed file when adding new test data, also be aware that modifications to existing seed files may cause issues with the tests.
We split seed data into folders depending on their purpose. For example, data/seed/demo is the seed data used to demonstrate Sidekick, while test/integration/seed/* files are used in the integration tests.
For more information on the structure of these files, read the Knex.js documentation on seeding.
To use the demo seed data, you can run:
make db-seedAutomated testing is an important part of working on Pa11y Sidekick, let's dig into how we do it!
Unit tests live in the test/unit folder, where the directory structure mimics the main structure.
Not all of the codebase is unit tested, we find that trying to unit test models and controllers doesn't really give us much benefit over just integration tests. The code that's extremely important to test is anything in the lib folder. We try to maintain ~100% coverage of the core code, and we also write tests for middleware currently.
For testing we use:
Dig into the test/unit folder to get an idea of what our tests look like.
We try to maintain integration tests for all of Pa11y Sidekick's features. This means that we can feel confident that our changes haven't broken existing functionality, and that our changes won't be accidentally broken by a later commit.
The integration tests spin up the application in test mode and run the full stack from database to front end. We use the following tools:
- Mocha as a test framework
- SuperTest as an HTTP client and assertion library
- Sinon and Mockery for mocks and spies
Dig into the test/integration folder to get an idea of what our tests look like.