For people who do not know what the W3C is, it is the main international standards organization for the World Wide Web.
+ +The WebPlatform project is a rapidly growing web development collaborative documentation website.
+ +> I would even call it the Web developer's missing manual for web developers to build quality web applications using the latest techniques. + +As for the sponsors, it is backed by well known companies, that we refer to as +[stewards](http://web.archive.org/web/20130328132045/http://www.webplatform.org/stewards/), +such as [Adobe][wbm-steward-adobe], +[Google](http://web.archive.org/web/20130807101413/http://www.webplatform.org/stewards/google/), +[Mozilla](https://developer.mozilla.org/en-US/docs/Project:MDN_content_on_WebPlatform.org), +and Microsoft to name a few. + +In the last months, many things happened to me.
+ +First, I worked for almost two years in a contract at Ericsson, but it ended. Therefore, I started searching for a new job. I had some offers and I had a hard time to pick one.
+ +However, after all, I have just received a new offer to work for the W3C; something I cannot that I could not turn down.
I am very excited to have this opportunity. I would even say more: it is a honour.
+ +I will explain more about it in another moment.
+ +For now, I would just like to thank TEKsystems' team and to mention the many good things about working there:
+ +TEKsystems is a company based in the USA and their main solution development center is located in Montreal Downtown. Most of the projects are web based business applications serving many customers in the united states.
+ +Among the things I enjoyed most were:
+ +I worked there for the last month and a half, and it is amazingly well organized company. Based on my own experience in different work contexts, I can say that working with them for web development projects was enjoyable.
+ +If it wasn't because of my new opportunity, I would have stayed there for a long while.
\ No newline at end of file diff --git a/content/blog/2015/07/leaving-w3c.md b/content/blog/2015/07/leaving-w3c.md index 9bc54c949b..9097ecdcc0 100644 --- a/content/blog/2015/07/leaving-w3c.md +++ b/content/blog/2015/07/leaving-w3c.md @@ -10,6 +10,7 @@ categories: - tranche-de-vie tags: - career + - webplatform docs coverImage: src: ~/assets/content/blog/2015/07/20150727-Gift-W3C.jpg alt: Renoir’s picture @@ -18,7 +19,7 @@ coverImage: was an autographied book signed by [Sir Tim Berners-Lee][whois-timbl] and a small trophy. Today (2020), my name appears in [the W3C Team Alumni][w3t-alumni] - [w3t-alumni]: https://www.w3.org/People/Alumni#Boulanger + [w3t-alumni]: https://www.w3.org/People/Alumni#renoir-boulanger [whois-timbl]: https://www.w3.org/People/Berners-Lee/FAQ.html preamble: disable: true @@ -56,8 +57,17 @@ I hope our paths will cross again. -[0]: - https://renoirboulanger.com/blog/2013/08/i-am-joining-w3c-to-work-on-the-webplatform-project/ +--- + +## In retrospective + +**Note**: This was written on 2024-09-30 + +After the project stopped receiving funding in 2015, I voluntarily sustained (2015-2017) the infrastructure until the W3C Systems Team took over to permanently archive the site’s contents on W3C’s infrastructure. Fortunately for me the server infrastructure was "*self-healing*", each service would regularly check if its service was OK and knew how to start it back again, so it was pretty hands-off. + +The last billed hours I made for the project was to host a static HTML version [by having converted every pages, commits, deletes of every pages from MediaWiki into files as if they were commited into GitHub](/blog/2015/07/migrating-webplatform-org-mediawiki-into-git-history-and-into-markdown-files/) ([GitHub *webplatform/docs* repository](https://github.com/webplatform/docs)) + +[0]: /blog/2013/08/i-am-joining-w3c-to-work-on-the-webplatform-project [1]: https://webplatform.github.io/blog/2013/08/hi-my-name-s-renoir-ill-be-your-devops-for-the-web-platform/ [2]: https://www.w3.org/People/ @@ -66,4 +76,4 @@ I hope our paths will cross again. 'Facts about the W3C and its Organizational structure' [4]: https://www.webplatform.org/ [5]: http://schepers.cc/ -[6]: https://www.w3.org/People/gallery/ +[6]: https://www.w3.org/People/gallery/#year2013 diff --git a/content/blog/2017/02/things-i-ve-worked-on-while-maintaining-webplatform-org.md b/content/blog/2017/02/things-i-ve-worked-on-while-maintaining-webplatform-org.md new file mode 100644 index 0000000000..82f605ef09 --- /dev/null +++ b/content/blog/2017/02/things-i-ve-worked-on-while-maintaining-webplatform-org.md @@ -0,0 +1,158 @@ +--- +title: Things I’ve worked on in the last two years while maintaining WebPlatform.org +locale: en-CA +created: 2017-02-09 +updated: 2023-11-20 +canonical: >- + https://renoirboulanger.com/blog/2017/02/things-i-ve-worked-on-while-maintaining-webplatform-org/ +status: publish +revising: true +categories: + - tranche-de-vie +tags: + - career + - webplatform docs +excerpt: >- + Before starting working on the WebPlatform Project, I knew that I was jumping in a complex + project that required all my skills all at once. +--- + +I’ve worked at the W3C for two years. My assignment was on the WebPlatform.org project and my responsibility was to keep everything in order.
+ +While I am archiving things and closing my notes of the last two years I thought I’d share with you from where I started, and what I’ve done.
+ +WebPlatform.org had been running on about 20 VMs. Until my most recent work to convert everything into static-site generators, It still was using that many virtual servers. More on that later.
+ +When I arrived into the project, every component of the site was in a good shape. I could build any server by booting a from a blank Ubuntu 10.04 LTS with a name that’ll tell the configuration management what to install on the new "minion". Ryan Lane, the person I came to replace, did a great job!
+ +This was the first time in my career where I could replace any server using a configuration management tool. I’ve used Puppet, Chef on small projects, but WebPlatform.org was much bigger and was using Salt Stack.
+ +It felt great to be assured that almost every piece was replaceable without worrying about individual pieces. But still, to change a password required to dig in folders, edit and cross fingers. It was a cry for improvement.
+ +As for the code WepPlatform was using to serve the community at that time was basically a bunch of open-source projects with a few manual edits here and there. The theme, the configuration, and so on.
+ +++ +WebPlatform’s Achille’s heel was the deployment server.
+
We had backups, but yet, if "deployment" VM would lose its data, I would have also had to guess how each component were put together and rebuild every server. WebPlatform’s Achille’s heel was the deployment server.
+ +Luckily it didn’t happen.
+ +During the two years I’ve been on WebPlatform, we’ve been through a full system software upgrade, we were initially running on Ubuntu 10.04 LTS, and 2 "cloud hops" (i.e. re-install everything on another cloud).
+ +Our "cloud-hops" were from initial HP Cloud in December 2013 to our very own Open-Stack cluster — a 4 blade server borrowed by our friends at DreamHost.
+ +Thanks to my good friend Etienne Lachance who helped a lot installing the various components. The Open-Stack documentation has a lot of rough edges, but we came through it and ran the system without too much issues for a year.
+ +The second "cloud-hop" was between the self-managed Open-Stack cluster into the very beta DreamCompute platform that DreamHost opened up.
+ +All of this to tell that the challenge wasn’t always to keep things up when crisis was happening, but also the work involved in doing maintenance:
+ +A lot of creeping dependencies and possible places to break.
+ +Most of the code had https://*.webplatform.org hardcoded manually. Meaning that I couldn’t install a full copy of the site. This made it hard to rework parts of the site without impacting the live site.
Like I was saying, the work that has been done before me was great! Everything was in place and the community was already writing docs! In fact, before start working on WebPlatform, I knew that I was jumping in a complex project that required all my skills all at once. Rewriting deployment code crucially needed time.
+ +That’s what I did while making sure the site was running smoothly.
+ +Not only the code was assembled quickly, but also the most important server, the "deployment" server, was the only piece that needed work to be also replaceable like the other parts of the system.
+ +The cherry on the sundae is that the configuration management scripts refactor is now public, it allowed me to re-deploy WordPress, MediaWiki, BugGenie, Dabblet, Etherpad, Piwik and others.
+ +With this refactor, I achieved a "sysadmin dream"; I can control the passwords and secret from one file and apply the change to both the service and the appropriate configuration file at the next configuration management system run.
+ +If your happen to manage servers that runs WordPress, MediaWiki, MariaDB, Memcached, ElasticSearch or a set of static HTML files, it shouldn’t be hard to reuse.
+ +If you want to use my work, you can fork webplatform/salt-states and webplatform/salt-pillar and use the same code as me to run our "deployment" server (now called "salt") for your own site.
+ +All you need is an empty VM called "salt", install the two repositories plus one containing secrets, and you should be good to go.
+ +The installation of the "deployment" VM is a bit more complex than two git-clone, you can refer to the salt-master/ folder in webplatform/ops and use the vagrant-workbench/ to have your own local copy using Vagrant and Virtual Box — more on this later.
+ +NOTE You might need the secrets repository, I will eventually publish an empty shell so people won’t need to reverse engineer.
+ +Setup conventions in deployment strategy so I could run RubyOnRails, NodeJS, Python, PHP and static files without much change in how to to deploy them.
+ +That one was about harmonizing how things are deployed so I could handle separation of concerns when exposing on the web. You can see my monologue on the subject.
+ +Create a local workspace so I can work on server deployment scripts on my local machine, build and destroy VMs to ensure all runs smoothly in the cloud.
+ +Most of the time I was maintaining scripts webplatform/salt-states, webplatform/salt-pillar on a VM called deployment.webplatform.org (now called salt.webplatform.org) on production. With the work I did with the salt-states, I could build a complete mirror of the whole site as webplatformstaging.org. But yet, I needed to use servers exposed to the public.
+ +With my work on webplatform/ops, I could run two or three VMs in VirtualBox and run quick tests prior to run them. I wished I had this when I started.
+ +The salt-master/ folder in webplatform/ops are the scripts I wrote to achieve #1 but now aren’t limited to where you run them.
+ +The vagrant-workbench/ folder in webplatform/ops is a VirtualBox and Vagrant script to create a "salt" master from which I could run locally.
+ +The vagrant-minions/ folder is basically one YAML file where I describe nodes I need to bring up and Vagrant does the rest of the job for me.
+ +At the end of a vagrant up elastic0 I would see on my local vagrant-workbench VM’s salt-master a salt minion called "elastic0" ready to be managed locally.
Design and implement a prototype to achieve SSO for web apps without using SAML, Kerberos, or *LDAP.
+ +I created a small javascript file that bootstraps the local web application to sync session state with a "source of truth".
+ +It goes like that;
+ +I had something running in two separate MediaWiki installations and I have recorded a screencast showing it6.
+ +Basically the JavaScript client webplatform/www.webplatform.org/....sso.js requires the local web application (around here in the code) to receive requests from it, communicate through its backend to "source of truth" ("profile.accounts.webplatform.org") and return an HTTP return code (401, 400, 204) to confirm what happened.
+ +I’ve made all this to have our Wiki, Issue tracker, Blog, and Annotation system to prevent users to have different username passwords to sync, but I lacked time to have it all working and the project died. Other priorities came up.
+ +Luckily for me, that work got Mozilla Firefox Accounts team interested to invite me over to spend a week with them and it was great!
+ +I have hopes to eventually publish a PHP module out of what I’ve done so I could prevent this to be wasted.
+ +That was great! I enjoyed collaborating with an external provider and make something useful elsewhere at W3C.
+ +I had the chance to spend time with Doug and work out all the tiny details to create a schema to store data we could crawl from MDN.
+ +I’ve worked on a system to keep a copy into Memcached of the generated HTML. This helped a lot on page render time.
+ +Now that the site is going into static site generator, this is going to go to waste :(
+ +Hopefully with this in place we’ll be able to shut down everything of WebPlatform, except a simple web server serving HTML files.