RubyConf 2018 – Building Generic Software by Chris Salzberg


(upbeat electronic music) – So thanks for coming, the title of this talk is
Building Generic Software. My name is Chris Salzberg. I got a lot to talk about
so I’m gonna go through the easy parts quickly and the
hard parts of it more slowly. So just some points about me, my handle is @shioyama if
you’ve seen that name around. I live in Tokyo, in Japan, which you can see in the
background of this slide, this is Tokyo. Don’t be fooled, I’m not Japanese. That’s my stupid ice breaker joke. (laughs) I’m the Canadian originally from Montreal. I work at a company called Degica, based in Tokyo. And in the open source
world I’m the author of a number of gems,
the most well known one is called Mobility, which
I’m gonna talk about a bit. And in my free time, I write about stuff like the module builder pattern, which you may have heard of. And I blog at dejimata.com. Okay so the title of this talk
is Building Generic Software, so the first thing we’re gonna do today is I’m gonna introduce this idea of generic software, what is it, why would you care about that. And then based on that idea, we’re gonna look for a problem. And we’re gonna use that problem as the basis for kinda
building some generic software, which is actually gonna be a framework. And then after that we’re gonna see if we can learn some
lessons from that exercise. Okay so generic software, what’s generic software? So I did not invent this term. As far as I know this term
was coined by Jeremy Evans who you may have seen
because he gave a talk actually here yesterday. Jeremy Evans, is the author of a well known ORM called
Sequel similar to ActiveRecord, also another well known gem called Roda, which actually he talked
about here in 2014. And so I’ve been following Jeremy’s work. He’s really got some great gems out there. And so awhile ago, I was
kinda looking for inspiration in this open-source work I’m doing and I found this quote,
which is like buried, in these slides from
2012 where he gave a talk on the development of Sequel. And he said, in this quote,
I’ll just read it to you. He says, “One of the best ways
to write flexible software is to write generic software. Instead of designing a single API that completely handles a specific case, you write multiple APIs that handle smaller more generic parts of that use case and then
handling the entire case is just gluing those parts together. When you approach code like that, designing APIs that
solve generic problems, you can more easily
reuse those APIs later, to solve other problems.” So I thought this was like a great quote and it really kind of resonated with me with the work that I was doing, developing sort of generic components. And so I decided to make this talk, that’s why I’m up here today. But when I started looking into this idea, I kinda found that you can kinda take this kinda general idea of building blocks and go in a very different
direction with it, depending on how you apply it. So this is an a tweet from
September by Gary Bernhardt. You may already know he is a Rubyist and he says, “Programming should be like putting Lego together;
just decompose everything into tiny pieces with
maximally general interfaces!” And that sounds really similar
to that quote by Jeremy Evans and I mean you can take my word for it but Jeremy Evans worked, this works out, this is great software,
but Gary Bernhardt said, “That sounds great. Then you fast forward a decade and simple tasks have
become wildly complicated, sometimes breaking for reasons
that no one understands.” And so there seems to be some kind of weird split here where, depending on how you apply
this idea of Lego blocks, you either get, end up with really great maintainable software or you end up with this crazy
mess that you can’t maintain. Here’s another kinda data point. This is a comment on a Hacker News thread, responding to a blog post around
the same time in September, which was talking about
how our software is bigger and bigger, doing the same things, but like in way more complicated ways. And this commentator says,
“I put some of the blame on an unhealthy insistence
of code reusability and sticking to paradigms. The result is so much code that it is just abstractions piled onto abstractions piled
on to more abstractions, yada yada, nobody wants to just write some goddamn code anymore. They include entire frameworks instead of just writing a couple helper functions. And so I can certainly sympathize with this sentiment, right? I think there’s a lot of reason and also if you look at the Twitter feed, the Twitter thread that
Gary Bernhardt was on, he kinda suggests kinda similar things, you should just write the code yourself, rather than requiring all these libraries, if you don’t really need them. And that’s reasonable but I think it’s sort of
missing something, right? And so if you imagine this tree
is a giant dependency tree, this is like Ruby gems
or something, right? And you know, most of us most of the time, we’re on the leaves of this tree, right? We’re building applications. And our applications are not required as software by anybody else. We require lots of software. So we have these branches
that go down to these trunks, all the way down to the Ruby interpreter. And we’re most of the times, we’re building these apps on these leaves. And so this suggestion that you should just
take these frameworks out is kinda like chopping off the branches from your leaf there. And that’s a valid approach. That’s a totally valid approach but I think we should also consider, can we improve this tree, right? Can we make these branches better, can we make this thing better? And that’s what I wanna do today, I wanna talk about this idea. And to do this I’m gonna use some ideas that date back way back,
like 30 years back basically, to the eighties, to
before even the Internet. And this is the paper,
this is a great paper, if you haven’t seen it, I
recommend having a look. This is from 1988, literally 30 years ago, this is by two guys, Ralph
Johnson and Brian Foote. Brian Foote, you may know if you’ve heard of the big ball of mud, he’s the person who coined that term. The title’s Designing Reusable Classes and this whole idea of
reusability is really fundamental to everything that object-oriented
programming is about. Stuff like inheritance and polymorphism. Those are there to promote
a code we use, right? And also the idea of a framework, which I’m gonna come back, we’re actually build a framework, is sort of introduced in this paper so I’m gonna use these ideas. So what I wanna do today, is I wanna kinda take some
of these ideas of going back, apply this kind of idea
of generic software as kind of our compass to guide us and I wanna show you that
you don’t have to end up with code that is not maintainable that breaks for no reasons
no one understands, right? So how are we gonna do that? So generally when you wanna
build generic software, software that is reusable,
that people are gonna use in many different contexts. You have to start from something specific, at least in my experience, right? You don’t just go down this tree by just starting from
the base of the tree. You kinda start from the leaves
and you work your way down. So that’s what we’re gonna do. We’re gonna start from a specific example and see if we can kind
of come to something that’s more general. And so since I’m giving this talk, I’m going to pick the specific example. I’m going to pick the
topic of translation. That’s just because I happen
to have been introduced, translation happens to
have been the entry point for me into the Ruby community. So I was working on a translation platform and that’s where I started using Ruby. So the idea that we’re going to look at, is this idea of translated attributes. It’s really, basically a simple idea. I’m using the I18n locale here, that’s not really so important but you have some global language, right? And like any attribute that you have, like any accessor attribute type thing, you create a talk, a class
talk, you could in an instance, you set your title to say,
“Building Generic Software.” And then you can get your title back. And so far this is just
a regular attribute but then we change the locale to Japanese and I get my title and it’s nil. And it’s nil because this value, this attribute is not a normal attribute. This is a translated attribute. So depending on which language you’re in, you’re gonna get the
translation for that language and we don’t have a Japanese one so we can add it and we can get it back and if we switch to English, we’ll get back the English translation. That’s basically it. So that’s pretty straightforward and how do you define these things? Well, there are actually a
number of different gems, they come out all the time. I noticed a new one just the other day. There’s a lot of Ruby gems to do this. They all have their own
kind of conventions. Of course you have to include
or extend some module, but basically they usually have
the same kind of interface. You call some class method, it’s usually called translates, you pass some attribute names in, and kinda like accessor, you get some translated attribute and then you can do stuff with that. And the part where it gets
a little more challenging is how do you handle the storage of these translated attributes. And this is where we’re gonna get into this generic idea, moving forward. So there are a number
of different patterns to store these translations and there are a number
of different patterns to kinda add features to that, which I’m also gonna talk about. So what do these storage
patterns look like? So the most basic way to do this, is this idea of translatable columns. This is really kinda
pretty straightforward. So if you imagine we
hae like a comment model and we have some comments table. We have an attribute name, content. If you call the content
method by default with this, your ORM, whether it’s Sequel
or whether ActiveRecord, it’s not gonna know what
you’re talking about because there’s no content column here. What you do is, you create
columns for every language you wanna translate into. So we have En, Fr, Ja for
English, French, Japanese. And then you would have to
dynamically define some method, which would map from a content method to either En, Fr, or Ja, depending on which language
you’re currently in. So that’s one approach. That’s kind of easy, it’s
pretty straightforward, it’s pretty transparent
and that works nicely. Of course you have to migrate many things and every time you wanna do an attribute, new language, new model, you’re migrating, you’ re migrating. There are other downsides. So there are other approaches to do this. Another one that’s common is
called translation tables. In this situation, this allows
you to scale more easily to more languages. If you wanna add translations, what you do is you create a
separate translation table, so you move your translations
off the model table. And now they’re on this
other table and the language, instead of being the suffix
on a model column name, it’s now got its own column
on the translation table. And you have a reference back so comment_id references back to comments and then you have the locale for which language the translation is in. And you have one or more columns for the translated attributes. And then what you would do
is you create an association and when you wanna get the translation, when you wanna get the title, you go through the translations, find the ones that’s in
the language right now and grab the attribute from that. That’s another way. And they’re actually many
other ways to do this. So to just give you kind
of a taste for this. You can also use JSON and Postgres or recent versions of
MySQl support this as well. And then you just put all the
translations on one column. So you have the keys or the
languages or the locales and the values of the translations. So these are some of the storage
patterns that you can have. And these are kind of the base layer. And then on top of this, these different gems implement different, what I’m calling access patterns, which is maybe not the best term for it, but the idea you kinda add
features that go on top, as the next layer on top of here. So the most common one
is called fallbacks. If you’ve worked with
IT9, you may know this, because for static translation,
this is also a feature that’s often provided and
it’s pretty straightforward. It’s pretty much what it sounds like. You fall back from one
language to another language, so if I create a talk, I set the language to English, I set the title to
Building Generic Software. Now if I did not have fallbacks enabled and I set the locale to en-CA, which is the Canadian regional English, this would be nil because I
don’t have that translation but if you have fallbacks enabled, the general convention, and
you can always customize this, is that you would fall back
from the regional language to the base language or
if you didn’t have it, to the default language. So I would get back the English
translation in this case. And just to show you how this works, it’s not very complicated
but we’re gonna use this, moving forward in this talk, what you basically do is you
have some fallback locale, so in this case, it’s the current locale, and then like English and
you just loop through, right? So you try one, you fetch the
value, however you do that. Again, this could be however
you’re doing your storage. You fetch the value, if
it’s present, you return it, if not you go to the next one. That’s sorta how it works. Then there are other
patterns you can support. I’m not gonna talk about this in detail but just to give you an idea, you can support things
like dirty tracking. So dirty tracking, if you know it, is an ActiveModel also an ActiveRecord, also in Sequel. This allows you to kinda change the value and then see the changes. And this will not work by default
with translated attributes because your ORM is not going to know what that attribute is so you have to do some magic to make this work. So if I set it to generic software and then build it into specific software, you would see the change
and in this implementation, you can see that it shows you
the English title has changed. Like this. And then there are other things you can do and this is getting more
complicated, more tricky. I create a talk with English
and Japanese translations and then I want to find the talk which in English has the title,
Building Generic Software. And then this again, is not
gonna work without some magic to make this work. So these are some of the
things you can implement. And now what I wanna look at is sort of the most interesting part. It’s where the different gems
actually combine these things, using some kind of control flow. And they do this in a
generally kind of similar way. So I’m gonna kind of
sketch this a little bit. So I showed you this translates method. This is a class method. This is like accessor for
translated attributes, right? So what do you do in this thing? Well you’re splatting these
attributes or whatever, and then you’re going
through each of them, and you’re calling some method to define the accessor, right? And that’s what this method
would generally look like. It’s not really complicated,
you’re taking the attribute that’s the name, right? So that’s like title. And then you’re calling define_method, you’re defining a title method and then you’re defining title equals, which is like the setter method. And so the key thing is like
these methods in here, right? Then again, depending on
which gem you’re using, or whatever. These maybe injected right
into the body right here but you may call it another method. It doesn’t really matter. I just wanna sketch how this works and concretely, if this is hard to imagine if my classes talk and
my attribute is title, and you just resolve the
meta-programming there, this is what you would
actually get, right? So I have method title and title equals and they resolve to these methods, read_from-storage and write_to_storage. So the question here is what
do these methods do now, right? So this is paraphrased
from a gem called Traco, which you can look up
if you’re interested. Traco is for translatable columns. And so this is, if your storage pattern
was translatable columns and you were also implementing fallbacks, so you can see what happens in here, is that first we’re going to
do these fallbacks, right? That’s what I showed you earlier. So basically the same thing, we’re just looping
through these fallbacks. And then when you need to get the value, I’m calling this method,
column_value here, right? And this column_value is this
column translation pattern that I mentioned earlier, read_attribute, if you don’t know, is
just an ActiveRecord, just gets the value off a column. And then what I’m doing there is I’m taking the attribute name, which is title and I’m
pasting on the locale, which is English and I get title_en, which is the column name and then I’m saying give
me the value for title_en. And then I’m gonna return
that to this loop above and I’m gonna get the translation. So if you look at from a high level, in this kind of diagram
view, you’ve got your talk. This my application code, right? So I have a class talk and I’m calling this
class method translates, Which is in my gem that I’m using. That’s sort of the high-level
interface for the gem. And then what’s happening
there, like I showed you before, so translates is now defining some methods in terms of some kind of internal methods, which I called read_from_storage
but they could be anything and this is sort of the
low-level implementation. And so what these libraries,
what these gems give to you, is kinda this package, right? This combination of the
high-level interface and how they actually do this. So I said we were gonna
present a problem, right? So what in the world is a problem? So this is the problem
that I wanna look at. This is an issue that
was posted to repository of a gem called Globalize,
which you may have heard of. It’s the most well-known gem
for doing this type of thing. It uses a translation table
pattern that I mentioned. And this is one of the authors, Tomash. He posted this about two
and a half years ago. And I also have contributed
to this project. I don’t contribute so much anymore for reasons that’ll become clear. But he posted this idea
and this was around 2016, and as a kind of context for
this, this was around the time that Postgres, Rails community
was adopting Postgres. And Postgres had these nice features like JSON storage, JSONB and so what he kinda said
was like, “Okay you know, Globalize is nice. Globalize, we kinda have these things, we have this part of the code
that defines these accessors like I showed you just now, defines the setter and the getter. And then we have the part of
the code that actually goes and gets these values from
some association on the table,” or whatever and he said,
“Wouldn’t it be nice if we could add like a layer
between these two things so that we could then swap out
that translation table stuff and swap in, say JSON
storage or JSONB storage.” And he said, “Wouldn’t that be nice?” And I actually saw this later, and I’ve been thinking
about similar stuff. So I went out and did this
and we’re gonna sketch that. But the basic question I
wanna first think about is, So how do you make
something pluggable, right? So we’re talking about translations, this is just to make it concrete but the real question we’re
gonna be getting out here is how do you make something that
wasn’t pluggable pluggable? And this is actually a really interesting and actually really kinda
deep question I think. So now we’re kinda gonna
go generic with this. So what do we do? So we go back to this diagram that I have. We have this high-level
application code which is talk, it’s calling some class
methods it translates, it’s called read_from_storage, which is like the internal implementation. We like the translates part because that’s common to
all these gems, right? But we wanna swap, we wanna swap this out. We wanna kind of cut this part out. And we wanna swap in something else. So we wanna cut out a
translatable columns or whatever and then swap in the JSONB,
JSON storage or whatever. But you can’t do that actually, right? You can’t do that because
it’s all hard-coded, like that code I showed you earlier, the read_from_storage is
hard-coded to go through fallbacks and then through to this
column storage pattern so it’s all hard-coded
and coupled in there, so you can’t actually do it that way. But what you can do is, you can instead of going
down, you can go back up. So instead of the
high-level interface saying, “Okay, we’re just gonna do it this way.” You kinda pass the buck back up to the to the high-level application code. And then the talk says, “Okay,
well you do it this way.” And then high-level
interface says, “Okay,” and then it continues on. And this is actually a
really important idea, And this is called inversion of control. I’m curious how many
people know this term? Okay, like less than half but some. So I think it’s sort of,
I was serving my own team when I was asking about this
but a lot of people knew it. I don’t know if people
know It so intimately. So this is actually from
that paper I mentioned. If you haven’t heard that paper, that’s where it was actually coined. Designing Reusable Classes from 1988, so this is 30 years ago, right? And then I’m just gonna read to you this where it is actually originally used. It says, “One important
characteristic of a framework is that the methods defined by the user to tailor the framework
will often be called from within the framework
itself rather than from the user’s application code. The framework often plays
the role of the main program in coordinating and sequencing
application activity. This inversion of control
gives frameworks the power to serve as extensible skeletons. The method supplied by the user tailor the generic algorithms
defined in the framework for a particular application.” And you can see that we’re
using this word, generic. So the generic algorithm
there in this case would be the high-level part where we’re defining the
setter and the getter. That’s common to everything, right? But we wanna tailor that for each different storage strategy, each different access strategy. And so this is great. This term, I think it’s
hard to understand. But it means what it says, where we inverted the control flow. Instead of going straight
down, we went back up. But it’s still a little un-intuitive. Conveniently, there’s
a much more catchy name and it’s convenient that
we’re in Los Angeles because it happens to be
called the Hollywood Principle. And this is great, I wish we had all our
terms that were like this. So the meaning of this is,
don’t call us, we’ll call you. (audience laughs) So now you’ll never forget it, right? This is what it means, you call back to the application
code when you need to and you’ll see how this works. So what we’re gonna do is we’re gonna build a second
version of this API. Every API has a second version so we’re gonna do our second version now. So we take our same ideas as
before, we have a class talk, we call translates with an attribute but now we have like a keyword argument and we’re gonna pass a class. It’s a gonna be a
backend keyword argument, that’s gonna be a class. We’re gonna define the class later. First we’re gonna see how
we handle this argument. So we’re gonna change this
class method that we had before. I had translates and we were
going through the attributes and defining accessors for each. I’m gonna change how
we define the accessor. But first I wanna look
at how we do this thing to find backend here. So we’re passing in this backend, which to think concretely,
could be like a ColumnBackend. And so it’s gonna look like this, we’re gonna define a method. The method’s gonna be
called attribute backend, which will be like title backend and then in the body of this method, we’re gonna take that class. It’s gonna be like,
something like ColumnBackend, which we don’t know yet
but it’s gonna be a class. And we’re gonna create an
instance of this thing. And we’re gonna pass in self, self is going to be an
instance of this model. So an instance of the talk
and the attribute name, which is like title. And then we’re gonna memoize that. So this probably a little bit hard to see so let’s just make this more concrete. Suppose our class is talk again, and the attribute again is title so we would create, without
the meta-programming now, we’d have the def title_backend and then we would say ColumnBackend.new, we pass in self, which is the
talk, an instance of the talk, and the name title and
then we just memoize it on some hash, don’t worry too
much about the memoization. It’s not really that important, just so we don’t have to keep
creating them over and over. And then we’re going to
define these accessors. Now remember that when I defined
these accessors previously in the way that we do
it in these other gems. We were hard-coded to this
read_from_storage or something, which was hard-coded all the way down to whatever implementation we had. But we don’t wanna do that. We want to make this pluggable, so here’s what we can do here. We call title_backend,
that grabs the backend, then we call these methods. I’m going to call them read and write, and we pass in the locale and
in its case of the writer, the setter, the value, so
this is gonna be our protocol. And protocols are really important and I’m gonna come back to this but we now can kinda see what this backend class is gonna
have to be like, right? So this is kinda the idea. And so now we’re gonna
define this backend class. And when I was preparing
the slides for this, I realize this is kinda
like little bit like TDD, so Test Driven Development,
in Test Driven Development, you write the tests and
then you write the code to pass the tests. So we wrote the protocol
for the framework, now we’re gonna write the code
that satisfies the protocol. So we need initialize, it’s
given an instance of the model like the top and the attribute, we just assign those instances variables and now we need define
this read and write, right? So what are they gonna be? Well this is actually
pretty simple, right? So we have the attribute
name, we have the model, we have the language in
the argument to the method, so we can just do, basically what I did
in that second section where I was showing you
the code from translated from Traco, basically
we take the language, we append to the attribute
name, which will be like title, so you’d be like title_en. And then we again, we’d call
read_attribute on the model. So there’s nothing really
that complicated here. And this is sort of our column back and this sort of encapsulates the translatable column strategy, right? And just to show you in a different way, this is how we would
look so if I have a talk and the talk is called
by the method title. The first thing I’d do is say
do I already have a backend, if I don’t have a backend,
I’m gonna create a backend, I’m gonna pass myself in. I’m the talk and the name title and then what’s gonna happen, then the talk’s gonna say, “Hey backend, give me the value for English.” And then the backend
says, “Oh okay, well, hey, can you tell me what’s the
value of the column title_en? And then talk says, “Oh that’s easy, well that’s Building Generic Software.” And then the ColumnBackend
says, “Well that’s their answer, Building Generic Software.” And the talk goes off and gives that back to whoever called title. So this is the basic idea,
it’s not super complicated but the way this is happening
is really important. And the key thing here is that this is the pluggable storage logic that we were after, right? That’s the part we wanted to cut out. And we have actually managed to cut it out because that backend class
which is past top-down, is what is actually defining
that part of the logic here. But there’s something missing. So I wanna show what’s missing just by sketching a little bit what
would happen if you plugged in a different backend. So suppose we plug in here, the kinda translation table approach. The translation table approach
is a little more complicated. That’s the one where I showed you where we had different translation table and you’re joining stuff and I’m not gonna go
into detail about that because that would take more time. But the key point is important here, so the top part doesn’t change, right? The talk still says, “Hey, hey backend.” The title doesn’t know
what the backend is. It just knows that it has a
method read so the talk says, “Hey backend, give me
the value for English.” And the TableBackend is gonna say, “Okay, I need to join this translation
table onto this talk model and so it’s probably, you’re probably gonna implement
that with an association. So it’s gonna say, “Hey give
me your translations,” right? And then talk’s gonna say,
“Okay, here are my translations.” And the TableBackend is
gonna iterate over them, find the one in the language English and then grab the title in
that thing and return it. That’s roughly what would happen, but that’s not actually gonna work because the talk doesn’t
have an association called translations or
called talk translations or anything, right? It doesn’t have that association, because all we did was just
call this method translates. We never did anything else. So there’s something a little bit missing. We need a hook for the backend to define something on the model. So we need to add something
to this method we have. And what we can do is we
can add a little bit here where we take that backend class, this backend here is a class again. That’s like the ColumnBackend, and we call our method setup_model, and this is gonna be an
added thing to our protocol. And now we pass self, self is
now the class, so like talk. And we pass the attributes in. And this is gonna be a
chance for the backend class to do kinda extra logic to
the model if it needs to. For the column one that
I originally showed, this is just a no-op, it’s
not gonna use this chant. But for the TableBackend, it’s going to do some stuff where you can say, “Okay, hey model class, you’re gonna have has_many translations and pass the class name in.” Basically, it’s gonna
create this association so that when it needs to,
when it gets the recall, it can then go back to the talk and it knows the talk will
have these translations on him. And as it turns out, this is enough to support all those different
translation backends. So if you do that, roughly,
your protocol is just read write on a class method called setup_model, your backend can encapsulate
everything that you need to know about all these
different storage strategies. So you can call them, I kinda
sketched a little bit of table and you can do JSON, you can do pretty much anything
you can really imagine. And you’ve isolated the core logic of the setting and getting stuff. And this is sort of an
extensible skeleton. That idea from that paper from 1988 where they’re talking about
this extensible skeleton, that’s sort of what it is. But we’re not quite finished yet. Because we’re missing
those access patterns. You remember I talked about
fallbacks as an example as an access pattern
that goes on top there. Well, what happens to those, well if you wanna put that stuff in there, there’s not really anywhere
to put it right now, so you’re gonna have to kinda put it into the backend itself. So if you have this
backend, we’re gonna have to like rename it,
ColumnWithFallbacksBackend. And now we can stick the fallbacks in, and this is basically very
similar to the code I showed you in the second section
where we were looking at what Traco does where
it couples them together. So you basically kinda added fallbacks and you’re calling down to get the actual column_value and so it’s all coupled together. And this will work, right, this will work. But the problem is that
you end up with a picture of something like this where
every pair of storage pattern and access pattern becomes
its own backend, right? You’re gonna have to support everyone, probably you’re just gonna
do all of them into one thing but that’s not really very elegant. So you’ve isolated the core, you’ve isolated this top-level stuff that’s defining these
setter and getter methods but everything else is all together. So the problem is that you’ve
put together two things, which are basically
different concerns, right? Storage and access are both
being handled by this thing we’ve called backend and
what do you do with concerns? You separate them. So we just separate these
and we say, “Storage, you just do what you’re supposed to do, you just do the storage, you don’t worry about
all these other things.” And we add some new thing,
we’re gonna call plugin. And that’s gonna do this access stuff, this stuff like fallbacks. And how is this going to work? So we’re gonna have every
good API as a third person. So we’re gonna add another version. So class talk translates
title, we have the backend, that’s a required keyword
argument, call him backend. Now we’re gonna pass in an
optional argument plugins and then we pass an array in there. And this is gonna be an array of what? Of modules. So how does this work? This is the code we had before. I just bumped it down
so we have some space. So it translates, we’re
past some attributes, we go through the attributes, we define the accessors, we define the backend. And then we do some backend hooks so the like translate
backend can do some stuff. But we’re gonna add in this plugin logic, so how’s this gonna look? It’s gonna look like this. And what are we doing? We’re not doing that much here. We’re just taking the
backend class and first, we’re gonna subclass it, we’re gonna create an anonymous class, if you haven’t seen this before, it’s just basically like what
you do when you subclass. This is just so that we don’t
pollute the original class. We don’t wanna start including stuff into some arbitrary class. So we create an anonymous subclass and then we iterate through these plugins. Again these are modules, right? So we’re iterating through the modules. And then we’re gonna say, “Hey subclass, include this plugin.” Right? So that’s gonna be like ColumnBackend, include fallbacks plugin. And that’s what that’s gonna do. And then we’re just
gonna use further down, it’s the same but we’re using the subclass instead of the original class. And so, how do we then do this? Well the backend part is
gonna be what we did before. We just put it back to what it was before. It just raised the value from the column, it doesn’t do any fallback stuff. And the fallback stuff we move
into this plugin and module. And the module is actually very simple. It just has the method
read, which passes locale, which goes through like I’ve showed you
a couple times before, you go through all the locales. But the key thing here is that
we’re calling super, right? And super is one of the most
underrated methods in Ruby. It’s really an amazing thing and method composition in
general is really powerful. And so super is gonna go
back up to the backend and get the value from the column. But the key thing is we’re
gonna decouple these things now. So if you look at it kinda like a diagram, from the model, the model’s
gonna get called by title, then they’re gonna call
through the backend, and so we’re gonna get this
read with something like en-CA. And this is gonna go first to
the fallbacks plugin, right? And the fallbacks plugin
doesn’t know anything about how we’re storing stuff. It just knows that it has to get the value for this language, en-CA,
or a fallback for that. So it’s gonna say,
“Okay, hey ColumnBackend, give me the value for en-CA.” And it’s gonna call super, right? And the ColumnBackend
also doesn’t know anything about fallbacks. It just knows how to get
the value for a locale. So it says, “Okay I don’t
have that one. That’s nil.” And the FallbacksPlugin says, “Okay, well, do you have English? Hey, give me English.” And the ColumnBackend says,
“Oh yeah I have that one, that’s Building Generic Software.” And then the FallbacksPlugin can return the Building Generic Software and that goes back to the model, which returns it for
the value of the title. And the key thing is that
the plugin doesn’t need to know anything about storage. And the storage, the
backend, doesn’t know need to know anything about this extra feature that’s on top of that. So we’ve decoupled these things. And so if we put this all together, right? Backends each solve a generic problem. They solve a problem
like translatable columns or translation tables or JSON tables or JSON translations or whatever. And the plugins each also
solve a generic problem. I’ve only showed you one because
I don’t have that much time but you can do the same kind
of thing with other ones. And what’s the core? What’s left? This is the stuff that glues, that links all the stuff together. So now I can tell you
this is actually a gem. I’ve kinda sketched what is
a gem, it’s basically a gem, I’ve been working on for
the last couple years. It’s called Mobility and Mobility is a pluggable
Ruby translation framework. And it supports all these
different types of strategies for storing stuff and for
doing stuff on top of that. And this is sort of what it looks like. It’s got a little bit more
than what I mentioned, but basically the core of what I explained is what this thing does and
it has this backend protocol that I talked about here. And the plugin has,
everything is decoupled. And they’re connected through
these common protocols. Okay, so I want to now
look at some of the things, we can kinda learn from this exercise. So the first thing is this idea
of generic software, right? Generic software doesn’t
generally live in a vacuum. It really lives in some
kind of a frame, right? And I think when we think of
like frames and frameworks, we think of stuff that has
lots of stuff in them, right? So Rails is kind of the classic one. We think of like, Rails,
it’s got all the stuff in it. But the stuff that’s in it and the frame itself
are actually different. And so this comment in
that Hacker News thread that I mentioned earlier,
which says, like, “Include entire frameworks,” right? Kind of implies that the
framework has to be really big. And this is not true. Like frameworks do not have to be big. And in fact, I would argue,
it’s actually the opposite. So what happens when you
require gem-like mobility, not only mobility just because
I know mobility, right? You get the core. And the core doesn’t have
very much in it at all. And then what happens, well
of course you wanna translate, so you’re gonna need to get a backend. So you gonna grab one of the backends. And then you’re probably gonna
want some kinda features. So there are plugins for that. But the key thing here
is that you don’t need to include anything you don’t want. Right? So if you don’t want
fallbacks or whatever, you can just not include that. And that’s because
mobility is a framework. So the key thing is like, a
framework being pluggable, also implies that it’s unpluggable. And that’s something
I think we don’t think about very much, right? So you can unplug these things. And of course in those
gems that I mentioned, if it’s got fallbacks bundled into it, you can’t take it out. You can of course, add
a conditional in there, which would then say,
“Okay, if it’s enabled, then do this, and if not, do that.” But that’s exactly the
kind of thing that leads to code breaking for reasons
that no one understands. So any well-designed framework
should really allow you to unplug everything, is
my philosophy, anyway. And so the question is what happens when you unplug everything? What’s left, right? And so what’s left is kinda this, I know the font size is small here but this is all what we worked
through in the last section, so you’ve seen all this code already. I just put it in one big thing. I put it into a module called translates. I actually made it a gem of this, so if you’re interested, you can pull it down. It’s also under shioyama translates. But it’s just kind of a sketch version of what Mobility’s doing. But the key thing is
this is the core, right? This is what’s left if you
take away all the backend stuff that we did and you just look
at what’s left over there. This is sort of the key
generic software here. This is what’s holding
everything together. And it’s interesting, I
find it interesting to think about, like what are the
Ruby things used in here? What’s used in here, we use
really, really core Ruby stuff. You know, like subclassing,
module inclusion, little bit of meta-programming, that dynamic method definition. Not really very much but those are really powerful
features actually, right? You don’t need a lot of the fancy stuff, all these innumerable methods that we learned about the other day. They’re great and everything but you don’t need them
for this kind of stuff. And then also you think
about what’s not in here. So there’s no ActiveRecord. All these gems that I’m talking about, they’re all ActiveRecord based things but the base here is not actually doesn’t have any ActiveRecord. There’s no ActiveSupport in here. And this is also true of Mobility, right? So Mobility supports
Sequel, as an other option for our ORM support and it can do that because it doesn’t have
ActiveSupport in the core so you don’t need to
include ActiveSupport. What else? It doesn’t have any mention
of persisted storage, right? Which is kinda funny because
this whole talk started from this idea, we’re gonna do all these
different storage backends. But the core actually has no mention of persisted storage of any kind. What else? This is a translation framework, but there’s hardly any reference
to translation in the core. Right? The only references really used, when you’re defining these accessors, you need to pass in the language
so you pull it from I18N but that’s not really
a very important thing. So I find this really
interesting to think about that the core of the framework actually looks kinda different from what you think it
should look like, I think. And so what’s left, right, what’s left when you when take out everything? It’s the protocols, right? So this is something you learn about when you build a framework, at least in my experience, it was, how important it
is to build good protocols, flexible protocols and
stick to those protocols. And I don’t want to just be talking about my work all the time, so I really wanna reference
some really great gems that are out there. One of them I kind of mentioned earlier, by Jeremy Evans is called Roda. Actually he presented it here in 2014. This is a really great gem,
it’s a little bit like Sinatra. And this other one here,
this is called Shrine. This is a file attachment
toolkit, it’s what it’s called. But toolkit is basically
like an alias for framework. So these are basically frameworks, right? But if you open them
up and you go in Roda, you find only one folder. There’s a folder for
plug-ins and that’s all. And that’s because the
protocol in Roda is so flexible that you can build anything you want out of just the combination
of different plugins, right? And Shrine is the same way. Shrine is a file attachment toolkit. It only has two things,
plugins and storage. Mobility isn’t quite that clean yet. I’m aiming for this, but… So if we go back to this
original thing that I was talking about in the introduction about, you can kinda go off in two
different directions, right? I had that Jeremy Evans quote
that was saying this idea for generic software and then,
oh but we end up with code that nobody can understand. So what is separating
these two different paths? Because we’re kinda aiming
towards similar things, but we seem to be going in
totally different directions. And so the way I think about it is if you kinda put it on two axes, right? On the one axis, you have
the kind of the reusability of the software and the
reusability of the software is something that we always
wanna maximize this, right? If we’re building some kind of gem or something that we wanna share. We want people to use it
in lots of different ways but the other thing is
that I think we don’t think about quite as much is this idea of the complexity of the protocol. So if you don’t focus like a hawk on this complexity of the protocol, you’re going to end up with this. Which I kind of think of as the Swiss Army knife model, right? And this is kinda this maximally
general interface thing that Gary Bernhardt was talking about. This is where you basically
make it more general, whatever you’re making more general by adding stuff to it, right? And this works well up to certain point. But then you just can’t scale it anymore. And unfortunately I think
that a lot of our gems, a lot of the stuff we work with, kinda veers toward this type of model. And so what have I done today? I hope that I’ve shown you
that there’s a different way of doing this and I think of it like this. We started from this idea of a backend. And when we started with
this idea of backend, we had this ColumnBackend, it really could only
do one type of storage, it wasn’t generalizable to other types. But then we kinda added a hook in there and that then made it possible
into many different patterns. And then we pulled out these plugins and suddenly it was possible to combine any different kind of thing. And those were small changes. They were increases in the
complexity of the protocol, it’s true so that base thing was getting a little more complicated but only small changes for
like leaps in reusability. And that’s the kind of thing you’re after. So I kinda think about this as an X-Acto blade type model, right? Where X-Acto blades,
you can pull out a blade and you can put another blade in and you can do that with
many different blades because the interface,
which is the holder, handles many different types of blades. So I really think we need
more of this type of software. And I think the skills
required to build this type of software are kinda different from the skills required
to kind of like pump out a lot of code in a hackathon or something. So I think there’s a real
opportunity for people to get involved in building these kind of simple, very simple components. You can spend a lot of time
building one of these things. If it’s gonna be used in all
kinds of different scenarios, that’s worth the time, right? So I really encourage
you, if you’re interested, to have a look at this stuff. If you’re interested, talk to me. I think we need to get
more people involved in building this really
quality generic software. It’ll help our ecosystem a lot. Thanks very much. (audience applause)

2 thoughts on “RubyConf 2018 – Building Generic Software by Chris Salzberg

  1. actually the idea of generic software is older than that, is called bottom up programming. Where you rise your language level of abstraction closer to your domain of interest.

    Is an idea that has been long around Functional Programming, since the golden age of Lisp, that is why FP doesn't follow "readability" because if you are going to make software generic, the names and so on, cannot be descriptive of any particular domain, because they applying it to other domains becomes weird.

    And you also know it as the UNIX philosophy, "write a program that handles one thing and does it right…."

    this guy contradicts himself, if you build generic software you don't want a framework.

Leave a Reply

Your email address will not be published. Required fields are marked *