Running Sinatra in the Pakyow Environment

Pakyow's new Runtime Environment lets you mount multiple endpoints in a single process. It supports any Rack-compatible endpoint, which made me wonder if it would be possible to run a Sinatra app within the Pakyow Environment. Here's a screencast with my findings:

Btw, if you want to learn more about Pakyow Environment, last week's post would be a good place to start.

bryanp

Pakyow's New Runtime Environment

I've spent the bulk of my time the last few weeks working towards the Pakyow v1.0 release. While not a from-scratch rebuild, I'm revisiting every single aspect of the design, starting with how Pakyow Apps run within a process.

One of the things that previous releases got wrong was managing application state. Nearly all of an application's state was contained in singleton classes, which, like most bad ideas, seemed like a good idea at the time.

It worked great up to the point you wanted to run more than one Pakyow app within a single process, which you simply couldn't do. Nearly every project I've worked on over the last few years has started as a monolith that's eventually broken down into two or more separate apps. Making the transition has been a painful process every single time.

Changing how app state is managed would let us implement different features as independent apps and run them together in a single monolithic process. You get the ease of creating a monolith, while the framework enforces separation between features so they can later be moved into standalone apps or plugins.


Just yesterday I finished working on a new runtime environment for Pakyow. The entire goal was to make it possible to define, mount, and run multiple apps alongside each other inside of a single process. And it works!

Here's what it looks like:

Pakyow.configure do
  mount Pakyow::App, at: "/"
end

Pakyow.setup(:development).run

Here we use the main Pakyow object to configure the environment and mount a single app at the root path. Next we setup and run our environment in development mode.

Mounting is implemented as a lightweight layer on top of Rack::Builder, which already supports mapping requests to multiple endpoints. Building on top of Rack kept the size of our implementation fairly small, weighing in at 719 LOC.


The Pakyow Environment is pretty simple, but it has some advantages. Perhaps the biggest advantage is that concerns of the environment are shared across every app that runs within it. For example, every app runs on the same host and port and inherits a common request logger.

Pakyow also loads a default middleware stack for the environment. Out of the box all apps support things that most apps need, like HEAD requests and JSON request bodies. There's no configuration necessary.

We've effectively separated the concerns of the app itself from the concerns of the environment that runs it. Admittedly, it's a subtle distinction, but the consistency it brings is really nice.

If you're interested in seeing more of the implementation, it's all pushed up to the environment branch on Github (environment is in lib/pakyow).

Something else that I'm excited about is that the public api for the main pakyow gem is fully documented! We are also sitting at 99.75% test coverage, thanks to the shiny new test suite. This part of the framework feels ready for v1.0.

Keep in mind that core, presenter, mailer, realtime, and ui (okay literally all the other main gems) are broken on the environment branch. This change touches everything and there's still quite a bit of work left to make the rest of the framework compatible. That's all in progress :)

Discuss this post on the forums →

bryanp

On Repeat: Halt & Catch Fire Playlist

This Halt & Catch Fire playlist on Spotify is amazing. I didn't even think I liked music from the 80s. Now I'm only sure I like some of it. Anyway, this is on repeat all week (given an 8-hour workday I'll get through it 1.74 times this week; the playlist is over 23 hours long). Enjoy!


Refine Your Ruby Util Objects

For some time now I've followed a pattern of using utilities to avoid extending core Ruby objects. While this approach avoids the perils of monkey patching, the experience has always bothered me; it's not very Ruby-like.

Last week @searls tweeted this gist demonstrating his recent usage of Ruby Refinements. Refinements are not that new, but for some reason the feature never made it into my toolbox.

Until this weekend, that is. While refactoring some old code I realized that refinements would offer a much more Ruby-like experience than my dumb util approach. Here's an example from one of my projects:

module Utils
  module Dup
    UNDUPABLE = [Symbol, Fixnum, NilClass, TrueClass, FalseClass]

    def self.deep_dup(dupable)
      return dupable if UNDUPABLE.include?(value.class)

      if dupable.is_a?(Hash)
        dupable.each_with_object({}) do |(key, value), hash|
          hash[key.deep_dup] = value.deep_dup
        end
      elsif dupable.is_a?(Array)
        dupable.each_with_object([]) do |value, array|
          array << value.deep_dup
        end
      else
        dupable.dup
      end
    end
  end
end

Utils::Dup.deep_dup just returns a deep copy of the object. It works, but that's about all it has going for it. The need to check if the object is a Hash or Array is just awful. And using it is clunky too. I mean really:

Utils::Dup.deep_dup({ "foo" => "bar" })

Refinements solves both of these problems. Consider if you will:

module SuperDuper
  UNDUPABLE = [Symbol, Fixnum, NilClass, TrueClass, FalseClass]

  refine Object do
    def deep_dup
      if UNDUPABLE.include?(self.class)
        self
      else
        dup
      end
    end
  end

  refine Array do
    def deep_dup
      each_with_object([]) do |value, array|
        array << value.deep_dup
      end
    end
  end

  refine Hash do
    def deep_dup
      each_with_object({}) do |(key, value), hash|
        hash[key.deep_dup] = value.deep_dup
      end
    end
  end
end

You'll notice that the method bodies are identical to how we'd write this as a straight up monkey patch, so I call that a win for code readability. Using the refinements version of deep_dup also feels much more Ruby-like:

using SuperDuper
{ "foo" => "bar" }.deep_dup

And there you have it. The benefits of monkey patching, without the monkeys. Now go read more about refinements and put them to use in your projects.

Talk to me about this post: @bryanp

Web Views as Data

HTML provides a semantic structure around data presented on the web. We can build on this to create a simpler presentation layer that avoids the need for complex view logic. Consider this bit of HTML:

<article>
  <h2>
    hello world
  </h2>

  <p>
    :wave:
  </p>
</article>

The semantics of this simple fragment provide us with a basic understanding of the data that will be presented. We can think of the article as representing a unit of content, each unit containing a heading and a body.

If we think of the article as representing a data object, we can let the data itself drive the rendering process. Doing this requires a bit more knowledge though, since we need to know what part of the data each node represents. We can do this with HTML data attributes:

<article>
  <h2 data-prop="title">
    <!-- post title goes here -->
  </h2>

  <p data-prop="body">
    <!-- post body goes here -->
  </p>
</article>

Now we can parse the view template into a structure; something like this:

view = {
  title: node_for_title
  body: node_for_body
}

Rendering is now a matter of transforming the view structure to match the data that we want to present. Once the view and data match, final presentation happens by mapping each value into its respective node:

data = {
  title: 'hello world',
  body: ':wave:'
}

# present it
data.each do |key, value|
  view[key].text = value
end

When we treat the view as data, rendering is replaced with a transformation of one data structure into another. Given a view template and some data, complex rendering can be expressed as simple presentation:

view.present(data)

We only have to say what should be presented and the data drives the rest of the presentation process. Interested in seeing this in practice? Take a look at Pakyow, a web framework built around this approach.

So, you want to Colemak.

I just made the switch from QWERTY to Colemak over a period of roughly 4 weeks. Several people have asked me about it, so I decided to share why I switched, how I did it, and some lessons learned.

Why Colemak

Having spent more 50% of my life touch-typing, I've done a lot of typing (final test before switching was ~105 WPM). As a result, around two years ago my hands started aching. Nothing too painful, but I took note.

So, over the last 24 months I've done various things that have helped a bit. Along the way I remembered reading about more efficient keyboard layouts and how they help to avoid awkward finger movements.

There are lots of alternative layouts out there. Colemak stuck out because of its similarity to QWERTY. There are only 17 key changes (only two of which are between hands), which is a manageable transition with the proper approach.

Some Colemak statistics (source):

  • Your fingers on QWERTY move 2.2x more than on Colemak.
  • QWERTY has 16x more same hand row jumping than Colemak.
  • 35x more words can be typed using only the home row on Colemak.
  • Many shortcuts (including Ctrl+Z/X/C/V) remain the same.

If less movement means less pain, Colemak wins by a long shot.

Transitioning

It took me three attempts to transition. I went cold-turkey for the first two attempts and switched my layout on my computer, phone, etc. Both times I was back on QWERTY after three days. It was just too much all at once. You know that feeling you get when you try to pat your head and rub your belly? It's like that. All day long. I would not recommend doing this.

Tarmak was the solution for me. It's a 5-step program that moves 3-4 keys in each step. 3-4 new keys was easy to keep in my head while still thinking about what I'm typing; 17 is not.

It took me about a week to get through each step. My typing speed took a big hit at the start of each new step. Because of this, I found it helpful to work back up to ~50 WPM before attempting the next step.

Each step followed a similar flow:

  • Drop to ~32 WPM at the start of a new step
  • Back up to ~54 WPM after ~7 days

Installing Tarmak Layouts

If you're on a Mac, use Karabiner. It ships with Tarmak layouts and you'll be taking your first step in seconds.

For all other operating systems, read here.

Lessons Learned

I'll assume that my superb writing has convinced you to make the switch. Here are some things I wrote down as I went through my transition that you might find helpful.

Take Typing Tests, Keep Notes

Take typing tests twice a day (morning / afternoon) and write down the results. Seeing improvement will help you stick to your goals and know when to take the next step.

Moving Keycaps

I chose not to do this for a few reasons. First, it sounded like a lot of work (especially on a Macbook keyboard). Also, the home row keys with bumps on them would be in the wrong place, which is just weird. I have a Code keyboard so I could order new keycaps. But the reason I'm not is because of the next tip.

Typing Multiple Layouts

Yes, it's possible to go between multiple layouts, at least in my experience. Don't move your keycaps and you should be able to look at the keyboard and type QWERTY without too much trouble. Helpful for typing on a coworkers computer (assuming you can't convince them to switch).

VIM Commands

Don't remap your keys to keep them in the same physical place. This is more confusing than learning the new placements. It only takes a day or two to adjust.

Colemak Everywhere?

There is a Colemak keyboard for iOS, but I recommend that you just stick with the default QWERTY keyboard on mobile. It's such a different typing feel that it shouldn't impact your transition.

Mac Keyboard Prefs

After you're through Tarmak, use the OSX keyboard preference pane to add the Colemak keyboard layout. Then delete the existing one. Otherwise your keyboard will change back to QWERTY in the middle of typing and you'll think you're having a stroke.

Macbook Keyboard Covers

It's a waste of money because Tarmak has 5 steps. Also you're touch-typing, right?

Conclusion

Making the switch was a fun process; 10/10 would do it again (eh maybe not).

Is it worth it? I don't know yet. Really the only thing I've noticed is that Colemak has a really nice rhythm to it. In my experience, typing cadence is important for achieving flow, so this was a pleasant surprise. Over time I hope to see an improvement in my hands, but I expect that'll be hard to measure.

Anyway, maybe this will inspire you to try a new layout. Thoughts or input? Tweet or email me: @bryanp | bryan@metabahn.com

Thinking Work

I had a lot of thinking work to do at the beginning of last week, so I took it to the trail. After an hour drive (8 miles of them down a one-lane forest road), I hiked a couple miles in and was rewarded with this view of Coleman Glacier.

This may be the start of a tradition.

The Minitest Dilemma

If you were to start a new open-source Ruby project today, what testing library would you choose: rspec or minitest? Perhaps you have a strong opinion on this topic and the answer is obvious. For me, it presents a dilemma.

As a bit of background, Pakyow started on minitest and later moved to rspec. This has since been determined to be maybe a bad idea, and thus this post. From what I recall, the decision to use rspec came down to two things:

  1. More developers seemed to know and prefer rspec to minitest.
  2. Rspec appeared to have a more active community, better tooling, etc.

Number 1 was important because we wanted code contributions to be as friction-less as possible. Naturally we'd align ourselves with the tools most likely to be used by potential contributors.

Number 2 was really just personal frustration that built up over time while attempting to find and use basic tooling with minitest. It wasn't that the tooling didn't exist, but it was hard(er than rspec) to find and use due to lack of information (this has improved over the last couple of years).

For example, how would you use a code coverage tool with minitest? Search for minitest coverage and you'll find lots of people asking the same question but no definitive answer.


Before moving Pakyow to rspec, I'd only used the library a handful of times -- in some open-source contributions and on a client project that I inherited from another team. For the most part I preferred the simplicity of minitest.

Rspec was great at first. I liked the tooling and the active community. Sure, there was a learning curve, but I could always find an answer to my question. And this turns out to be a great thing given that I still spend a lot my time looking up obscure parts of rspec's DSL.

ಠ_ಠ


So, back to my dilemma. Minitest is simple, focused, and lightweight. Rspec is none of those things. Minitest::Spec along with a handful of extensions provides a near-rspec-like feel with less magic and complexity. I should prefer minitest. And yet I actually use rspec.

What gives? I have a hunch. Essentially, minitest's simplicity means there's less to talk about. Rspec is more visible because it's more complex and requires more discussion. Look for yourself; when searching for ruby testing library, rspec is the third result and minitest is nowhere to be found. There's a lack of visibility.

If there's a problem to be solved here, it's that the obvious parts of minitest need to be better documented. Organize the new documentation in the format of "I want to do xyz thing", with an example.

I want to see coverage for my test suite.

Add the following to your test helper:

require 'simplecov'
SimpleCov.start

Given the simplicity of minitest, I see no reason that someone should need to browse rdoc to find a list of assertions (or expectations), or read the readme of an extension to learn how to use it in their project. Learning how to start using minitest should be just as easy and obvious as actually using it.

I'm thinking about contributing documentation like this at some point. Should I? Tweet or email your thoughts: @bryanp | bryan@metabahn.com

Heading West

This is my last full week in Huntsville. On July 7, my family and I begin a year-long experiment in the great Pacific Northwest. If you know us well, this news will hardly come as a surprise.

Prior to having our two girls, Amanda and I spent several weeks in Seattle to see if we might like to live there one day. That was more than four years ago. We decided against moving at the time. Looking back, the timing wasn't quite right.

Back in May I booked a last-minute flight to Seattle. We had found a few rental houses and I went to take a closer look. I landed around 8pm and made the 45 minute drive down to Olympia to stay with friends (they were traveling Europe at the time but insisted that I stay at their home).

The next morning, loaded up with coffee and danish from a neighborhood coffee shop, I drove 150 miles north through Seattle to a small city about 30 miles from Canada.

Welcome to Bellingham.

Four years prior, Amanda and I rented a car and set off driving the same route. I don't remember exactly why we decided to do this, but I do remember wanting to get out of the city for a bit.

It was well after dark when we stopped the car to fill up with gas. We breathed the fresh air and remarked on how pretty the area must be during the day. This is the same place I found myself in just a few weeks ago, this time looking to relocate.

Bellingham came up a few months ago when researching various areas of the maritime Pacific Northwest. We love Seattle, but things like traffic, cost of living, and distance from nature made us lose interest. The suburbs were ruled out (because suburbs) so we looked out a little further.

For many reasons, Bellingham quickly made it to the top of our list. And on the morning of my 29th birthday, we signed a lease on a house located in a great little neighborhood on the edge of town. Looking back, it's kind of funny how things work out.

I could go on for days about all the factors playing into this decision. Instead, I'll focus on explaining things at a high level. If you want more detail, let's chat over coffee :-)

Personally, we're looking for a fresh perspective. Amanda and I both grew up in Huntsville. The longest we've ever been away are the few weeks we spent in Seattle. Huntsville has its perks, but we've longed to live in a place that's completely different from the one we grew up in. We don't know a single person where we're moving, and that's incredibly exciting.

From the business side, my role at Metabahn won't be changing even a little bit. Instead, I'll be joining our growing ranks of remote employees. The office in Huntsville will continue to exist and we have plans to expand our presence here. We value the relationships we've built and we plan to be just as involved as ever.

While it's true that we aren't moving because of my job, it just so happens that our personal goals align perfectly with the goals of the business. We'll see how things go.

The best way I know to describe this move is that it's an experiment. We don't really know what living in a new city will lead us to. But we couldn't be more excited to find out.

Pakyow Fundamentals 001: Build an App in 9 LOC

For some time now I've wanted to write about the fundamental concepts behind Pakyow. This is the first of a dozen or so posts covering many of the details and considerations that have made their way into the framework's design.

I also recorded a screencast where I ramble through roughly the same content. It's available on the Pakyow YouTube Channel and is a nice supplement for this post.

Here it is, the simplest app you can build in Pakyow:

require 'pakyow-core'

Pakyow::App.define do
  routes do
    default do
      send 'hello'
    end
  end
end.run

1 library, 3 dependencies, 9 total lines of code.

And yes, you can tweet it.

Save to a file (e.g. app.rb) and run it:

ruby app.rb

See Installation for details on getting your environment setup.

The app server should start up at localhost:3000. This app is nothing complex — it simply accepts a request at the default/ path and returns hello in the response body.

$ curl http://localhost:3000/
hello

Despite its simplicity, there are many things we can learn from this example.

Single Responsibility Libraries

You'll notice that we require pakyow-core rather than pakyow. This is because we only want a portion of the functionality that Pakyow provides. Pakyow consists of 8 different libraries, or gems, each introducing a separate concern.

The primary pakyow gem requires all 8 of the gems. Since we know that this app will only deal with routing, we choose the pakyow-core gem. The core Pakyow gem handles defining the app, routing requests to it, and sending responses.

Single responsibility gems let us start small and only include features in our app when we need them. This keeps apps as lightweight as possible while scaling to meet the requirements of more complex apps (more on this in future posts).

Apps That Grow With You

After requiring core, we define our app.

Pakyow::App.define do
  ...
end

Pakyow is designed to allow every part of the app code to be specified right in the define block. Here we can not only define our routes, but configuration, middleware, and more advanced features like mutators.

While this is an easy place to start, it does start to break down as an app grows larger. Pakyow offers ways to break things out as needed.

Pakyow often provides multiple ways to accomplish the same task. You choose between them based on your needs. For a simple app like this, specifying the routes elsewhere creates more friction than just doing it inline. When the app grows, it's easy to refactor these routes into their own files.

This design decision provides a clear starting point while providing ways to organize code in a larger codebase. You can take it as far as you need to.

Routing Requests

Now that we've defined an app, let's take a closer look at our routes.

default do
  send 'hello'
end

In Pakyow, a route is responsible for routing a request to a bit of logic. Here we see a single route, called default. This route will match a request to GET /, or the default path.

Pakyow couples the definition of the route with the work to be performed when the route is matched. You see this pattern in other frameworks, like Sinatra. We chose this approach to keep routing concerns in one place.

Consider if we had written the route like this:

default :SomeController, :some_action

SomeController#some_action would look like this:

class SomeController
  def some_action
    send 'hello'
  end
end

The call to send is a routing concern. Putting it in the controller (away from routing) requires us to keep two bits of information in mind to understand the entire request / response flow. This is bad.

A pattern I like to use when building Pakyow apps is for routes to contain only routing logic and delegate other concerns. Here's an example:

default do
  send SomeService.hello_or_goodbye
end

Assume that the method on our service object makes some decision about whether it should say "hello" or "goodbye" and returns the appropriate message. We can look at this route and understand everything that happens from request all the way to response. All routing concerns are right in front of us.

Sending Data

Our default route simply sends back hello in the response. This is done through Pakyow's send helper, which accepts a String or IO object (e.g. aFile). It sets the content headers and body of the response, then immediately halts execution and returns the response.

Send is one of a handful of helper methods that provide us with tools for dealing with different routing concerns. We'll look at other helpers in the future including error handlers, redirects, and rerouting.

Choosing an App Server

When your app server started up, it probably booted Webrick. This is the default server in Ruby and is part of the standard library. If Pakyow can't find another app server in its environment, it falls back to Webrick.

You can use a different app server by requiring it:

require 'pakyow-core'
require 'puma'

...

Now the app will boot with Puma instead of Webrick.

Pakyow tries to make sane decisions when things are unspecified, but gives you the power to easily influence or change those decisions. This further reduces friction by letting you deal with things when you want to deal with them, rather than requiring many decisions upfront.

Start with conventions, allow configuration in the future.

Conclusion

We'll build on this 9 line app in the next post and discuss configuration, helpers, and middleware. Thanks for reading!

Pakyow v0.11

Pakyow v0.11 was pushed to RubyGems yesterday. This release introduced some new features like Resources and Binding Parts. It also improves the developer experience by removing the libxml dependency (which can be a pain to install) and adding Dotenv for config management.

v1.0 is the next planned major release. We're setup well to push for 1.0 with a ton of work having focused on improving the framework internals.

You can read the full release announcement on the blog. I also screencasted a walkthrough of some of the new features.

I also want to try something new and include a selfie for every release, wherever I happen to be when I publish the announcement post. Pakyow is a project I plan to keep working on for a long time, so it'll be fun to see how things change. This one was taken at Metabahn HQ:

Enjoy!

Pakyow, Portable View Transformations, and the Distributed Web

I've been doing quite a lot of thinking about IPFS and how Pakyow might fit into the world of distributed web apps. Examples that I've seen lately are built as client-side JavaScript apps, with the underlying source distributed among peers with something like IPFS.

This is interesting, but it requires you to 1) write your app in client-side JavaScript and 2) distribute the source code of your app. Neither of these are necessarily deal-breakers, but I don't like the idea of being forced to do this just to build a distributed web app.

Since Pakyow is a server-side framework implemented in Ruby, how does it fit into this? Eh, I still don't know. But, thinking about it has led me to some interesting realizations. Such as the fact that Pakyow divorces business logic from the rendering of a user-interface.

This separation means that we can render state changes without knowing about the underlying decisions that led to the state change. When the state of a Pakyow app changes, it creates a set of declarative view transformations that can be serialized and passed around.

Given an interpreter and the underlying view template, the transformations can be performed anywhere. Rendering is made completely portable. We could also, in theory, replay the entire state of a UI from beginning to end without having access to the state or the original business logic that created the transformations.

I find this quite profound.

What can we do with this? Well, portable rendering allows views that exist on the client-side to auto-update and reflect new state, without moving any code or state to the client. This concept is at the core ofpakyow-ui andring.js. Might it also have implications for the distributed web?

I considered ending this post with a discussion about embedding the historical state of a Pakyow app in a blockchain, but I honestly don't know enough to sound intelligent about it. Instead, go read Tony Arcieri's post on the dangers of a blockchain monoculture.

Interaction is an Enhancement

Aaron Gustafson for A List Apart:

In February 2011, shortly after Gawker Media launched a unified redesign of its various properties (Lifehacker, Gizmodo, Jezebel, etc.), users visiting those sites were greeted by a blank stare. Not a single one displayed any content. What happened? JavaScript happened. Or, more accurately, JavaScriptdidn't happen.

He goes on to explain why, in the web, content should always comes first with interaction as a layer on top. It's interesting that this still has to be pointed out, as this was commonly accepted just a few years ago. As browsers continue to improve, developers talk less about progressive enhancement. But as Aaron points out, browser support isn't the primary issue here.

Reading this article reminds me of a talk I gave last year at All Things Open, where I made the case that we're building at least two kinds of things for the web:

  1. Content-Focused: New York Times
  2. Interaction-Focused: Slack

Each of these warrants a different approach. To readers of the New York Times, content is the priority and nothing should stand between them and their content. What about Slack? Content isn't the priority to those users. Slack is a tool for collaboration, not content delivery. Supporting progressive enhancement feels like overkill.

Going further, why do tools like React and Ember even try to support server-side rendering? At best it's an afterthought that's proven to be brittle. It seems these frameworks are building in backwards-compatibility in an attempt to become the "one tool to rule them all".

Consider this — when a user prioritizes content over interaction, using a client-side JavaScript framework that prioritizes interaction might be a poor choice.

Of Patterns and Power: Web Standards Then & Now

Separating structure from style and behavior was the web standards movement's prime revelation, and each generation of web designers discovers it anew. This separation is what makes our content as backward-compatible as it is forward-compatible (or "future-friendly," if you prefer). It’s the key to re-use. The key to accessibility. The key to the new kinds of CMS systems we're just beginning to dream up. It's what makes our content as accessible to an ancient device as it will be to an unimagined future one.

Zeldman nailed it in this recent piece. The style / behavior separation he's preached his entire career drove my decision to separate structure / logic in Pakyow. Markup, even in a templating system, should be expressed in terms of content. Semantics are just as important during development as they are in production.

Web standards is more relevant than ever.

Thirsty for more? Browse the archive.