Application <=> Server Management Protocol (ASMP)

While I was in Belgrade for WordCamp Europe, I took part in a meeting of a smaller group of core contributors to discuss the status and future of the Servehappy project. One topic we discussed was that the ideal end goal, from the end user’s perspective, would be to have an automatic updating mechanism that keeps the entire server automatically up-to-date and secure. While this might be a lofty goal, we wanted to nevertheless look into what would be necessary to make this a reality.

While we can already automatically update WordPress Core, as well as plugins and themes, we cannot do the same for the components of the WordPress stack that are installed at the server level. There’s currently no standardized protocol for server applications to request information about or changes to the underlying infrastructure. While we have APIs to provision such servers through code, we have no protocol or API to allow the applications that run on them to make changes after the provisioning.

I hereby suggest creating a communication protocol (tentatively called ASMP) between servers and applications that run on these servers that allows for bidirectional communication so that these two actors can coordinate.

Let’s look at a few examples

As an example, think about regular users managing a WordPress site. On the WordPress “Server setup” page in the admin backend, the end user would see the current PHP and MySQL versions used by WordPress, as well as potential notifications about available updates.

The available updates are calculated by querying ASMP about the 'php' and 'mysql' components and then checking the returned ranges of supported versions against its own requirements.

When the user wants to run such an update, they click the corresponding button, which will have WordPress send an ASMP request to the server to 'set' the version of the 'php' component to the requested version.

Then WordPress can poll the ASMP state of that request to get updates. It could return states like 'queued', 'in-progress', 'completed' or 'rejected'` to let WordPress know how this is moving forward and adapt the user interface to give feedback to the user.

This would give more control to users, reduce support tickets for hosting providers and indirectly improve version distribution of the server components managed in this way.

As another example, think about developers that manage their project through a package manager like Composer or NPM. When you provision a server with a composer install instruction, you could have Composer automatically make requests to the underlying server based on the "require" statements that define the platform and extension requirements.

Composer could just as well throw an error when the server is not capable of providing the requested environment, and let the user know which components failed to resolve properly.

Guiding principles

Planning the ASMP protocol and providing reference implementations should follow a few guiding principles:

  • Applications can only make requests, not force any changes.
  • There is no guaranteed support, servers are free to only provide partial implementations.
  • Requests are asynchronous, they don’t return responses.
  • State and capabilities can be queried, so that user interface and expectations can be adapted to the current server.

These guiding principles still allow for more exotic implementations, like a request to update a given component doing nothing more than creating a support ticket with required details. Hosting providers are free to choose the subset to implement and the specific way to implement it. The implementations can then grow from there and be further automated in time.

Technical details

The transport to be used could be regular HTTPS, with a server environment variable providing the host to contact for queries and requests. This environment variable would point to localhost when running on stand-alone servers, but could be deviated to centralized server for hosting providers that have more complex orchestration needs. The server environment should also contain an authentication mechanism, like a token, to make sure queries and requests can only be made from the correct server.

We should provide a reference implementation based on a common server distribution. This makes sure that it is easy for hosting providers to add support to their hosting and helps keep the experience consistent.

Also, focusing on building implementations into popular server control panels would accelerate adoption, as all servers managed through a given control panel would automatically inherit the implementation of the ASMP protocol.

What’s next?

I am in the process of introducing this as a proposal to the Cloudfest Hackathon which will be held on March 23-25, 2019 in Rust, Germany. If this gets accepted, it will probably be an immediate opportunity to collaborate with major hosting/control panel providers to build a working prototype. Anyone who is interested in participating in this endeavor and willing to travel to Germany for the occasion can still apply for the hackathon.

1 Comment

  1. Slava Abakumov on November 26, 2018 at 11:01 pm

    While I admire the initiative, I do think that the majority of app (in your example – WordPress) users should not be responsible for taking this decision. There are several reasons for that:
    1) they are paying for a hosting. It’s not their day to day business to decide which PHP/MySQL to run. If PHP version differences are more or less described out there and also all the benefits for WordPress and its plugins, what about MySQL? MySQL 5.7 breaks things for some plugins. Old WordPress (I’ve tested 3.x) itself can’t be installed in a default MySQL 5.7 because of its strict internal rules. There is no way an average user will know that. There is a great possibility that after upgrading to 5.7 she/he won’t even notice the issue until some records won’t be saved into the DB.
    2) thus hosting company, that is getting money, should be proactive and notify its clients about the new PHP/MySQL/Apache/Nginx versions that were released and are available for an upgrade.
    3) hosting companies should insist on upgrading a server with PHP not further than the latest day of a certain version support. That means that starting from a middle of the PHP support period (not just security, but still regular support) – it should write its clients, and provide deadlines, and actually do scans and server upgrades when the time has come. Currently all hosts are not teaching its clients at all. Some of them are proactive on providing an ability to upgrade, like SiteGround. But even they still allow me to downgrade to PHP 5.5 AND create a site on PHP 5.6 and 7.0.

    Hosting companies have power, people and technological knowledge to fix this issue. Most of them just don’t care.

    The proposed solution in this article looks a lot like yet another support channel with a single topic: please upgrade my PHP for domain example.com from 5.2 to 7.2. Further automation that was mentioned can already be achieved using the technology that is already there. Also, control panels already have this versions management, and companies like SiteGroup already created solution to change PHP version right from WordPress admin area (see their SG Optimizer plugin and its PHP Config section).
    IMO the focus should be changed to hosting companies being proactive, and not their clients.

    https://xkcd.com/927/Report

Leave a Comment