Oops, I'm Making A Framework

Ben Lovy on August 19, 2019

In a previous post, I discussed a project I'm working on to level up the complexity present in my portfolio. After a few days of deliberation and ... [Read Full]
markdown guide
 

how does this not always happen?

The most diplomatic way I can put it is, not every developer has the desire and ability to refactor into a framework, which is really the only way that frameworks should be designed.

Or, rather, does this always happen?

With a sufficiently motivated and skilled individual, it does.

Does any sufficiently large project end up with its own highly decoupled bits of logic like this?

If built by a skilled team, it does. If built by an unskilled team, it does not.

Does the fact that I'm writing one from scratch means I should be using a "real" framework instead?

Not at all. You are writing a framework for a specific problem domain, and a general-purpose framework may not be exactly what you find you need.

would this sort of thing be a flag for you in a "real" project?

For me, not anymore. In the beginning, I had a lot of anxiety, but after I had a few successful frameworks under my belt I began to trust my instincts as they seemed to generally correct as assessed by my peers - which is the only assessment that matters when building frameworks for other people.

How much of this would you let yourself implement from scratch before exploring the ecosystem?

I'd wait until I was done, then let someone else tell me I had reinvented a wheel. Creating a framework is difficult enough without outside influences telling you that you are wrong when those influences have no insight into the specific problems you are trying to solve.

The extra control is nice,

It should be more than nice, but rather be the best solution for your problem. It's very hard for a general-purpose framework to happen to provide the best solution for your specific problem domain.

but it's time-consuming...

Remember that you are comparing the time to write your purpose-built framework vs. adapting a general-purpose framework to your specific purpose.

...and this problem can be crowd-sourced.

Any problem can be crowded-sourced if you can find a crowd who wants to help you build whatever you want to build. In practice, however, it's not so easy to attract a crowd to work on a purpose-built framework.

what drives you to decide whether or not to actually separate and generalize it for public consumption?

I never have, as they have been commercial frameworks owned by the company I work for - even if I own that company. It takes a very specific business need to indicate that open-sourcing commercial software makes sense. One such use case is to attract development talent using the open-source project as a recruiting tool.

I don't know if the world needs another homebrew canvas UI framework

That's not relevant. You need it.

but if I've gone and written one anyway, maybe it's worthwhile it to throw out there.

If it's a portfolio piece, then you should put it out there. Otherwise, maintaining OSS is a drain of resources you could put towards solving novel problems.

it's also a whole other set of time-consuming tasks unrelated to this project.

You're training your brain to think and act like an software architect. It's time well-spent.


FWIW, I'm impressed that you are even asking these questions. You are going to make a very fine Mechanic some day in the near future.

 

Wow, incredible response - thank you.

Remember that you are comparing the time to write your purpose-built framework vs. adapting a general-purpose framework to your specific purpose.

This is a great point that I hadn't thought of. It's true, I don't think any of the options I'm currently aware of would have been drop-in replacements for what I'm putting together.

maintaining OSS is a drain of resources you could put towards solving novel problems.

It is a portfolio project, but I'm concerned about this anyway. I suppose worst case scenario is that I release and don't have time to maintain it and nobody uses it, which is more or less how many people currently use it...

You're training your brain to think and act like an software architect. It's time well-spent.

True enough. Time spent solving new problems is rarely if ever wasted.

I like your classifications - Mechanic-style work does seem up my alley interest-wise.

 

It is a portfolio project, but I'm concerned about this anyway. I suppose worst case scenario is that I release and don't have time to maintain it and nobody uses it, which is more or less how many people currently use it...

If it's a portfolio piece, then putting it out on your GitHub as OSS is a default place folks expect to find your portfolio pieces.

If nobody uses it or contributes then it is like 99.9999% of GitHub repos and an even larger proportion of portfolio repos.

If people do use or contribute then it is a better portfolio piece.

Also let me know if you do post to GitHub. I've been reading up on Rust lately and writing a PR would be nice practice.

Absolutely! Still quite underbaked for now but I will keep you posted.

 

When do you follow your advice and when do you draw the line and say: we should not have done this ourselves?

I read the part about learning and doing it anyway to answer that question, but buy vs build isn't always that simple. How and when can justify investing money in building a framework?

 

when do you draw the line

An architect's responsibility is to draw this line and live with the consequences of their decision. I was 15 years into my career before I felt confident enough in my technical, communication, and leadership abilities to draw that line. As a bonus, you only know you were right about 3 years after you make your decision.

buy vs build isn't always that simple

It is when you are a solo dev working on a personal project (the context of the original post). When working in a larger team, this isn't a question of "buy vs build" but instead, "Build purpose-built vs adapt general-purpose." Both cost something. Furthermore, I could look at the exact same challenge, but with two differently skilled teams, and would have radically different recommendations. A relatively low-skilled team should never try to build their own framework, and a highly skilled team will create frameworks and not even notice as a matter of course of refactoring.

How and when can justify investing money in building a framework?

It's pretty impossible to justify, and it's also not a good idea for a framework to be your goal. Your goal is to refactor your real-world application into the best possible version of itself. Sometimes, that results in a framework manifesting itself. If a framework does manifest itself, it needs to prove itself in at least three different applications to consider itself a general-purpose framework. Most often, however, a single-purpose framework provides value in removing code duplication and generally streamlining the process of implementing new features.

Amazing! Thank you do much for your reply. I definitely was mixing up ending up with a framework vs setting out to build one.

 

Oh Ben, oh why oh why oh why ;)

So my first question is: is it really a framework? Or is it a library? And how can you tell?

A rule of thumb I picked up from somewhere is that "you call libraries, frameworks call you". Which is pithy and easy to remember, and sounds true so :shrug:.

I've decoupled the rendering from the game logic to the point where it could be separated out into its own external standalone library. It provides a set of traits you can implement for your game's types which fully handle creating the canvas element, mounting your app to it, and handling clicks. All you need to do is plug in the drawable widgets and it organizes your layout for you.

I'm so hazy on Rust right now that I can't even tell whether this is calling my code or I'm calling it!

Does any sufficiently large project end up with its own highly decoupled bits of logic like this?

Only if it's done right! But decoupling the logic doesn't mean that you've built a framework - you can architect your code around, say, MVC, without it being a 'framework'. So much of what you've said is really just the proper architecting of a project - keeping concerns separated.

You can write a nice UI library that describes and solves your problem domain - and I'm a big fan of building a language 'up' to meet the problem; writing a DSL (domain specific language) that I can use to solve my problem in.

I guess I think it's a framework when

  • it solves a ridiculously generic problem (like building a web server + database + etc..)
  • in a DSL that looks like witchcraft
  • and it calls my code
 

Haha, right, right, don't say the "F" word around David :)

Good point - "framework" is likely not the correct word to apply to this solution. It sits around where React sits, but for a canvas - view layer only, but handles the view entirely.

It's a bit of both, though - I call it when I implement these traits (interfaces) for my types, but then it takes over and manages the entry point - it has some setup code that runs before everything else, and then manages the whole lifecycle of the running application, calling my code when needed to re-render. That's what makes it feel like a framework to me. I refactored away my own app's entry point, and now this separate apparatus handles those details.

 

I'm working with HTML, CSS, and JS and feel the same way. I write blocks of code that I like and would be easy to reuse in other sites so I save it as a VS Code snippet or a separate file or just add it to a growing list of reusable code of mine.

I asked myself "should I just jump into a framework/library? But I enjoy having the control of my code. I also have wondered if every developer has this problem or how many of them just jump into a framework/library.

 

I enjoy having the control of my code

That's it for me too - I want to build everything up myself from complete scratch, but I still want to make it easy on myself to use. This seems to be the natural conclusion.

 

Absolutely. I know people talk about using Frameworks/Libraries to speed up production, and perhaps it's just my inexperience, however, I'm almost "afraid" of using them 😂

My rule of thumb is to look at frameworks for the second go at a problem, after I've explored it myself. I can't choose a framework to solve a problem for me if I don't fully understand the shape of the problem I'm solving, but once I do, they really can save time if you pick the right one.

Lots of "if"s, though.

That's a good rule of thumb, I like it! Lol it is a lot of "if"s but I'm sure you, like me, would rather be self-sufficient and really learn along the way.

 

I believe it is the natural tendency for our projects to end up with their own framework. I mean, our whole job is to automate and save time for others. So it is only natural that we see repeated patterns in our code and then also try to do that for ourselves as well. In the small we call it DRY and in the large, I suppose, frameworks.

The real danger in frameworks though is that they are coupled to everything. Later, something new comes up which the framework doesn't handle. Then like any piece of coupled code, changing the framework can cause rippling breakage throughout the code base. It's why on a deployed project we don't simply upgrade to the latest version of external frameworks when they come out -- to reap any new benefit usually requires first fixing breaking changes. So the upgrade has to be evaluated and planned out. And sometimes the choice may be not to upgrade because there is too much work (based on how dependent you are on the framework).

Don't get me wrong. There is definitely a positive value proposition to be had in centrally handling common plumbing and providing extensibility points for adding new features / capabilities. However, I found once I decided to formalize it into an (ostensibly reusable) abstraction, that's when the trouble began. You end up running into case after case where you either: a) make an opinionated choice that limits the consumer's options b) parameterize a choice which makes your framework more complex to use.

Frankly any app frameworks I made even when I tried to abstract them for reuse, I always felt they were lacking in some way. And in any new project I would start over from scratch and try to "do it right this time". Nowadays I do functionally the same thing, but this time without the pretense of making it reusable across projects. Instead, I reuse the relevant capabilities (sections of code or libraries) in different mixes across apps, as opposed to plugging into a grand abstraction. I believe this might be characterized as what Dan North calls the ginger cake pattern.

 

a) make an opinionated choice that limits the consumer's options b) parameterize a choice which make your framework more complex to use.

Interesting - that makes a lot of sense. This is good to keep in mind - I think I'm going to continue with this present tack, but within the context solely of creating this working application first before I start thinking about what this proto-framework might need to provide in a more general setting.

It's like Neil said, "It's very hard for a general-purpose framework to happen to provide the best solution for your specific problem domain."

Thanks for your perspective! Looking forward to watching this talk.

 

Oh absolutely, pursue that spark of interest. I will say that the ones I made were valuable learning experiences. Often in the "what not to do" kinda way, but still valuable. :)

I enjoy your thoughtful questions.

 

I have a Clojure(script) project that's kind of heading towards a framework. Current API is custom, websocket based, and considering whether to switch to GraphQL separating the two more. But it's overkill if I just use it myself. I didn't want to use an additional framework to clojurescript, which worked, but it might have been less work if I did use one.

 

few months back I also tried to develop a MVC framework which written in PHP . Although this framework never upto the mark but I got to know more about the internal working of MVC based web frameworks . Increasing projects complexity is always help , definetly recommend to other to try this kind of projects .

you can find its code here github.com/thecodestuff/leggo

Best of luck for your

 

What's the point of the framework? I think the point is to have agreed-upon way of building things. Whether the framework is 3rd party or not is not very relevant, unless you're thinking about hiring new people. It's gonna take less time for new people to get familiar with how to build your thing if you're using a 3rd party framework, but really only if the framework is darn popular. If the framework is not popular, there's no reason to use it because new people will have to get familiar with it as well.

It seems like what you're doing is completely right, separating code into smaller independent modules. If an independent framework / engine comes out of it, there's nothing wrong with it. You may choose to publish it or to keep it private, and that's totally up to you - if you're not willing to market or support it, might as well keep it private.

 

I'm sure this always happens, let's arrange the madness, oh shoot it's another framework.

 

Making a framework is good to learn more thing and how it works :) You should keep going !

 
code of conduct - report abuse