Aaron Parecki

I just launched an update to p3k which adds an integrated authorization server. This means that now when I sign in to Micropub apps like Quill, it will redirect me to my own server where I can have more fine-grained control over the access I am granting the application.

My new authorization endpoint displays the scopes the application is requesting, and lets me modify the scope when granting the request. This means if an application I don't yet trust requests the ability to update or delete my posts, I can un-check those boxes but still allow it to create posts.

I also have the ability to have all posts from an application be added to a specific channel, rather than showing up in my main feed. I use the concept of "channels" to create different lists of posts depending on what kind of content the post has. For example, /photos contains all of my photo posts, and /travel contains travel plans, plane flight and train logs, and events that occur outside of Portland. 

Now that I have a built-in authorization endpoint, I'm looking forward to adding more features to it, such as setting the default privacy of posts so I can allow an application to create posts that are only visible to me (or a specific audience) unless I make them public.

Since publishing the latest W3C draft of Micropub on March 1, there have been a few minor updates based on feedback from people who have implemented it.

Simpler update syntax

The update syntax had an extra nesting that was not necessary. The syntax changed from:

replace[properties][content]=Hello+Moon

to 

replace[content]=Hello+Moon

Moving a post

If the update operation caused the post to be moved to a new URL (such as when the URL scheme is based off of the post's date, or has a custom slug that is editable), the Micropub client needs a way to know the new URL of the post.

The Update Response section now includes HTTP 201 as a possible response, along with:

If the update operation caused the URL of the post to change, the server must respond with HTTP 201 and include the new URL in the HTTP Location header

More Information

You can read the latest editor's draft of the spec here: http://micropub.net/draft/ as well as see the change log. The git commit history shows the full detailed changes.

Thanks to Greg for providing feedback on his Micropub implementation.

As always, feedback from implementation experience is appreciated! Feel free to file an issue, or join the #indiewebcamp chat!

I just spent two days finishing migrating the last of my old websites and services off my old server and into new homes. This project has been on my todo list for over two years, and I've been making slow progress since then, mostly by spending a day on it here and there.

I finally finished moving the last things off the server today. The last two things left were my Twitter streaming search script and a really old MediaWiki website. Both were running on a 6-year old operating system, and under deprecated versions of Ruby and PHP. The MediaWiki instance had been EOL since 2012. The OpenSSL library was so old the Heartbleed bug hadn't even been written yet. Other things on that server were a Gitlab install (which I replaced with Gogs in February), my own SSL root authority (which is significantly less useful now that Letsencrypt makes certificates free and easy), an old web app I haven't used since 2009, and many cobbled together scripts.

This was quite a painful process all told, and I would like to avoid getting into this this situation in the future. Here are some things I'm planning to do that will help make the upgrade process easier in the future.

Upgrade regularly

Spend the time it takes to upgrade the whole server (OS as well as application dependencies) every 3-6 months, instead of letting the machine get to be several years old. The whole reason I got into this situation in the first place is the OS was so old incremental upgrades were no longer possible. Updating the OS, PHP/Ruby versions, and applications regularly means smaller incremental changes rather than getting into a situation where your app no longer runs because the scripting language changed so much since the last upgrade.

Separate the web and database servers

I've had a separate database server for a long time. The main benefit this gives me is I'm able to move the source code for a website to a new server without having to also worry about migrating the database at the same time. It's one less moving part in the migration process, and I will continue to do this in the future.

Productize everything

Some of these projects and websites were so old that I didn't actually have them in Git repos yet. Some of them were in Subversion, some were just piles of files on disk. A few of them shared a common set of files on the server, rather than using any sort of package management and proper dependency chain. Of course my more recent projects are much better packaged, but I had forgotten I had not always done things this way. Making each project a self-contained application with its own Git repo makes it easier to move them around to new servers later.

Avoid using servers in the first place

These days, we have many options for running applications without dealing with the underlying operating system. Google AppEngine (on which I currently run indiewebify.me and xray.p3k.io), Amazon Lambda (where I run ca3db), and Heroku to name a few. Once a service is running in infrastructure such as these, you can forget worrying about the underlying operating system updates, and focus on your application logic. The service providers will handle everything under the hood and you can forget that upgrading OpenSSL is a thing that needs to happen.

You can use Telegraph to receive Webmentions when your site is linked to by any website tracked by Superfeedr, even if that site doesn't send Webmentions itself!

First, sign in to Telegraph using your domain.

From the top right menu, click the "New Site" link.

Enter "Mentions" for the name (or anything you want, but that's what I use), and enter your home page URL. Entering your URL here is how Telegraph knows which links in the Superfeedr feed to send webmentions for.

After you create the site, click on the settings icon next to the name.

Near the bottom, there is a Superfeedr Webhook URL. Copy that URL since we'll need it in the next step.

Now we need to sign up with Superfeedr and create a tracking feed. Create an account by visiting the Superfeedr Tracker page. Make sure to choose "Tracker" from the account type dropdown. If you already have a Publisher or Subscriber account, you'll need to make a new Tracker account for this.

Once you've signed up, you'll land on the Superfeedr dashboard. Click "Search and Track" to create a new tracker.

Enter link:aaronparecki.com as the query term, obviously replacing the domain with your own, and set the format to "json". Paste your Telegraph URL from the setup process into the Callback/Webhook URL field. Then click "Subscribe"!

At this point your tracker feed is live, and Superfeedr will begin sending web hooks to Telegraph whenever a new item is found that links to your website!

Unfortunately nothing will happen right away, so you'll have to wait for someone to publish a blog post that links to you. Check back in a little while and you should see some webmentions show up on your Telegraph dashboard!

Here you can see a few of the mentions I've received from my Superfeedr tracker, including one from Stack Overflow which doesn't yet send webmentions on its own!

Thanks for Ryan Barrett for the inspiration for this idea, as well as contributing to the development of Telegraph to make this possible!

Today there was a nice writeup about Webmention in theregister.co.uk!

http://www.theregister.co.uk/2016/02/18/remember_wordpress_pingbacks_the_w3c_wants_us_to_use_them_across_the_whole_web/

The headline and subhead are a little rough, but the article gets better.

I just had a very pleasant experience installing and setting up Gogs.io on my server, so that I can self-host my private repositories.

When evaluating a system that I plan to use for the foreseeable future, there are a number of factors I take into account. I want:

• a relatively simple installation process
• to know what the dependencies are and how to install them
• to have a clear path for how updates will be installed
• to know where all the pieces live so that I can back everything up
• to be able to switch to another solution in the future if the current system no longer suits my needs or if the project changes direction

After installing the Gogs binary, it was very clear to me how everything related, and it turns out it passes all these tests.

Installing

First check that you have the prerequisites installed. It's pretty simple, you need a recent version of MySQL or Postgres, Git, and SSH. Create the database (they even include the SQL to create the database in the scripts folder) and create a user that has full access to the database.

Create a "git" Linux user which will be used for the incoming SSH connections as well as will run the binary.

Download the zipped binary, extract it into a folder, make sure the git user owns the whole folder, and run ./gogs web as the instructions say. It said it was listening on port 3000, so I then went to my nginx config and set up a reverse proxy, so that I can use my existing wildcard SSL certificate.

My nginx config looks like the below:

server {
    listen 443 ssl;
    server_name code.pin13.net;
    ssl_certificate      /web/conf/ssl/wildcard.pin13.net.crt;
    ssl_certificate_key  /web/conf/ssl/server.key;
    location / {
        proxy_pass http://127.0.0.1:3000/;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Protocol $scheme;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect default;
        proxy_set_header Host $http_host;
    }
}

You can also configure an init script so that the system starts the binary on boot. Save this as /etc/init/gogs.conf

description "gogs"
start on runlevel [2345]
stop on runlevel [016]
respawn
exec sudo -u git /web/sites/gogs/gogs web >> /web/sites/gogs/log.txt 2>&1

Now you can stop the binary in your terminal and run sudo service gogs start instead.

Then I visited the site with its friendly URL, https://code.pin13.net and was presented with a configuration screen. This screen was straightforward, it lets you enter the paths to use to store the git repos, and asks what web address you want to publish, in case you've done what I've done and installed it behind a reverse proxy. After hitting "save", it creates a config.ini and then launches you into the app!

Upgrading

Obviously I haven't had to do this yet, but upgrades look pretty straightforward. Essentially you download the new binary, and copy the data and config folders into the new version's folder. I assume the database migrations will happen automatically or I will be presented with a simple walkthrough.

Backing Things Up

There's no point in self-hosting your code unless you also have a backup plan. If you don't have a plan, you might as well be using Bitbucket and let them handle that for you.

There are essentially three things to back up. You'll need to back up your MySQL or Postgres database, but since you had to install it, you probably already have a way to do that. I run a nightly mysqldump command that creates a sql file.

mysqldump --skip-extended-insert -u backup gogs > /web/backups/gogs.sql

You'll also need to back up the configuration, so you might as well back up the whole folder that was extracted from the download.

Lastly you need to back up the actual git repositories. The great part is that Gogs just creates .git folders (which is also how SSH accesses them) so you can just back up the entire git repo directory.

Migrating Away

If for some reason in the future I decide to stop using Gogs, I have a relatively simple path forward. All the git repos are already regular git repos on disk, so worst case I can simply move those to another server and continue pushing to them as normal. 

Migrating the issues and other non-git data is of course not as straightforward. However, the database schema is relatively simple, so I can definitely envision writing an export or conversion script that pulls the data out of the database as a last resort. 

Background

I've always used GitHub for my public repos, and will continue to do so. GitHub has done an amazing job at creating ways for people to collaborate, and I don't want to lose that by leaving GitHub completely. However, I am not willing to pay what would be quite a high monthly cost for my own personal private repos.

Previously, I had installed Gitlab back when it still required gitolite. A year or so later I checked back and there had been some significant changes, enough that doing an in-place upgrade was going to be prohibitively complicated. So I installed another instance next to the existing one, intending to move things over. By the time I thought about moving things to it, there had been another round of fundamental changes so I put the whole thought on hold. Later, I started using Bitbucket for my private repos, which was simple enough.

I've been trying to move everything off this old VPS that I've had which is now way out of date and should really be shut down. Two of the more complicated things on it were the two Gitlab instances. This meant that I kept putting off the whole project. Finally today, I was inspired to actually do something about it, and started looking into Gitlab again.

Gitlab fails almost all of the above tests. The Docker image abstracts things away too much, but installing the components manually is ridiculously cumbersome. The package version assumes it will be the only thing running on the server, so I can't really run another Nginx or Ruby alongside it.

I was pleasantly surprised by the simple installation procedure of Gogs, and was glad that it made me install a database myself, so that it was less of a black box to me. It's also a bonus that the interface is simple and similar to GitHub. I'm looking forward to hosting my repos myself!

I use nginx to serve my HTTPS domains, and Letsencrypt support for nginx is still in beta. I also have a relatively specialized nginx config setup so I don't want any automated script messing with those files. Below are the steps I took to use the letsencrypt tool to generate and automatically renew a certificate for a domain.

Photo by Jānis Puriņš

$ git clone https://github.com/letsencrypt/letsencrypt
$ cd letsencrypt
$ ./letsencrypt-auto --help

This installs the tool as well as a number of dependencies. After it's done, the letsencrypt-auto command is available. I installed this in my home folder.

The tool expects to be able to write the challenge file to disk and that it will be accessible at example.com/.well-known/acme-challenge/*. The first step to getting the process working was setting up an nginx server block for each http domain.

server {
  listen *:80;
  server_name aaronpk.com example.com;

  location /.well-known/acme-challenge {
    default_type "text/plain";
    root /tmp/letsencrypt;
  }

  location / {
    return 301 https://$http_host$request_uri;
  }
}

This configures nginx to serve the /.well-known/acme-challenge/ path from a folder on disk, and will redirect any other requests to the https version of the domain.

Now we can run the script that will request the cert and respond to the challenge.

~/letsencrypt/letsencrypt-auto certonly -a webroot --webroot-path=/tmp/letsencrypt -d aaronpk.com

If you want to generate a certificate that contains multiple domains, you can add more -d example.com parameters to the command. Alternately, you can run the command for each domain separately to generate separate certificates.

This tells the script to request the cert and write the challenge to disk, and prefills the domain name it's requesting. If everything goes well, you should see:

 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/aaronpk.com/fullchain.pem. Your cert will
   expire on 2016-03-06. To obtain a new version of the certificate in
   the future, simply run Let's Encrypt again.

Now we can set up the virtual host block which serves the certificate!

server {
  listen *:443 ssl;
  server_name aaronpk.com;

  ssl                   on;
  ssl_certificate       /etc/letsencrypt/live/aaronpk.com/fullchain.pem;
  ssl_certificate_key   /etc/letsencrypt/live/aaronpk.com/privkey.pem;

  # your normal config stuff goes here
}

(Note that I deliberately left out the other ssl config options from this example, since it's possible to configure them at the top-level nginx config. See indiewebcamp.com/nginx for more details on configuring nginx with https.)

Now that the two configuration blocks are in place, renewal of the certificate can be automated by running the same command with --renew.

~/letsencrypt/letsencrypt-auto --renew certonly -a webroot --webroot-path=/web/sites/aaronpk.com/letsencrypt -d aaronpk.com
sudo /opt/nginx/sbin/nginx -s reload

werd.io/2015/signed-feeds-pubsub-a-simple-verb-protocol-a-publisher-and

werd.io/2015/scene-100-years-into-the-future-an-apocalyptic-wasteland-shouts

werd.io/2015/one-man-surviving-the-apocalypse-in-his-radiation-proof-exoskeleton-simply

werd.io/2015/in-the-distance-app-city-with-its-gleaming-towers-and

werd.io/2015/the-masters-of-app-city-carefully-snip-and-prune-the

werd.io/2015/on-one-side-coders-fighting-for-the-user-hands-bloodied

werd.io/2015/back-in-the-present-i-dont-know-where-im-going

twitter.com/jack/status/20

twitter.com/biz/status/21

twitter.com/noah/status/22

twitter.com/crystal/status/23

twitter.com/tonystubblebine/status/25

twitter.com/Adam/status/26

twitter.com/jack/status/29

twitter.com/biz/status/30

twitter.com/dom/status/31

twitter.com/rabble/status/32

It's a little known fact that QR codes are actually quite tolerant of errors and missing pixels, so much so that you can completely cover up a bunch of them with junk and the code still scans fine. This is the principle behind the site I made, vanity-qrcode.com, which allows people to create QR codes with text overlaid on them.

The site takes whatever text you type, uses a pixel font I adapted, and places the text with 1px white outline in the middle of the QR code. There's usually enough error correction in the code that the text won't prevent the code from being scanned.

There is, however, a much fancier way of embedding text or pictures in QR codes, that doesn't rely on the built-in error correction. There is a great post by Russ Cox that explains this in great detail. The short version is you give it a URL and a high-contrast photo, and it adds a carefully chosen set of digits to the end of the URL where the digits chosen end up representing the photo. You can try it out here.

I tried this out with my avatar, it took a few tries to get a result that looked any good. I had to drastically increase the contrast of my photo for starters.

I was then able to upload the high-contrast version to the website and use the tools to adjust the position and size of my photo relative to the QR code boundary. After a little fiddling, I got this result:

It's close, but I wasn't super happy with the result. I figured I'd open it up in Photoshop and add a few more pixels to help define my mouth and jawbone a little more. The end result is below, which only introduces a few errors but still scans just fine!