I’m on a two-week vacation from Lightrun and I have this urge to build something new. I also have a great product idea: DDT. In the past I built many projects, both commercial and open source. I never documented the complete process. This is something I’d like to change. So in this blog I'll go over that process from concept development to product prototype. Notice that this process is identical for proprietary software too, it's a remarkably similar approach.
You, the reader, have a crucial part in this: You’re my “daily”. I don't have project and product managers who can keep me in check, so I need you...
Procrastination is the biggest point of failure in any project. It’s where projects rise or fall. In normal company settings, we have daily meetings to combat that. You know you’ll have to stand in front of the entire team tomorrow to talk about what you did. So you’ve got to “do something” so you’ll be covered in the daily meeting.
With a single person open source project, you’re all alone. There’s no product team to report to, no product roadmap and you can’t get fired. Procrastination becomes a major temptation. That’s where blogging can help by leveraging the power of the open source community. I hope you will keep me “honest”, I need you to read and follow this so stopping would be embarrassing. But I also need you to ask questions and help keep me focused on the product strategy. It's easy to get carried away and try to create an overly complex product. If it looks like I'm straying from MVP, please call me out on that.
I plan to write 10 blog posts until we have a working first version of the project. I’ll skip working weekends because my family would murder me if I do that... I hope I'll be able to keep the pace and document this process well. I also hope it will be entertaining.
The Process
I already went through the first major part, which is the product idea. I think a lot has been written about idea generation so I won't bother writing about that. As I said, the idea I'm working on is DDT (or DDTJ to be exact), I'll get to that soon enough.
This is the plan for the next 10 days. I don’t know if I’ll be able to stick to it or exceed it, but that’s my general direction. It isn't "really" product management, but in the early stage a hacker mentality is more useful than an organized process:
- Initial Developer Guide and Basic Design
- Scaffold the project and implement CI
- Connect to server with initial server unit tests
- Implement the first version of CLI
- Implement mocking abstraction logic
- Create tests for mocking well known libraries *Performance and Integration tests
This is a flexible guideline and not as a product roadmap. As I move forward, I’m leaving room for mistakes, omissions and delays.
Target Market
There’s one thing that’s missing from this list, which I already did. You need to validate the concept of the project you’re working on, a "product to market fit". There’s a famous “quote” of Henry Ford:
“If I had asked people what they wanted, they would have said faster horses.”
But the thing is, he didn’t say that!
It’s also a bullshit concept. People wanted cars and asked for them. Ford built what people asked for: faster, cheaper cars. So did every innovator. A Successful product or successful open source project starts with a need by real people.
When I explain this to people I often get the response that this is "closed source" thinking related to proprietary software. That's just wrong, open source software needs a proper product development process just like any proprietary software tool. We want people to use our tools... But we want them to spend time with our tools and time is money. We need to offer a sublime product concept regardless of our source license!
Developer Guide is First
I sorted the list in mostly chronological order. I’m a big believer in very “light” design. I really can't stand these huge documents that end up as a legacy of all our mistakes.
You can’t debug design. There are some cases where it's very warranted, but they are usually the exception, not the rule.
I usually start by creating a simple developer guide for the final physical products. This has the following advantages:
- It forces us to think first about the finished product. How it will look and feel
- We maintain it since it's the guide, it won't go too stale. A "living" document is important
- It lets us focus our product strategy on specific goals. E.g. this block in the guide relates to that module in the system
- It explains the product to other people. Testing product to market fit is important and having a clear guide is crucial
I finished the first draft of the developer guide for DDTJ today. You can check it out here.
Scaffolding & CI
I'm a believer in creating mocks for all the big pieces first. Deciding on the big set pieces and flushing them out together. The logic behind this is to see the first full stack process running as soon as possible to find any conceptual problems we might have. It also helps the development teams move faster when we have more than one developer. We can find our respective sandboxes.
In our specific architecture, we have three tiers and a common library among them. We can always refactor after the MVP so we shouldn't get too hung up on decisions. A major part of the scaffolding is the choice of technologies. Mostly, this isn't a big deal. However, we need to limit our scope and be wary of RDD (Resume Driven Design) which is a silent but deadly project killer.
Having a CI build in place with some code quality verification is just good common sense. Especially with security, static analysis, etc. This is important even when there's just one person working on the project...
So What's DDT or DDTJ?
I'll discuss the other points as we move forward, but let's talk a bit about the project. It would be great if you can follow it here.
DDT stands for Development Driven Testing. DDTJ is the implementation of the DDT idea.
The idea is simple. When we have a bug at pretty much any company, there's a requirement to add a test that fails for the bug.This is often harder to do than the fix itself. DDT is about fixing the bug and running your server. Then generating the unit test for the case that failed.
Unit tests are normally easy to write but the mocks aren't trivial. That's where DDT will try to shine.
There are many other uses for the basic technology, e.g. we can detect when code that isn't covered by tests is reached and generate unit tests for that code automatically. But that's not part of the MVP.
The Product Management Roadmap
I said there's no roadmap, but I planned the MVP, which is a bit of a roadmap. There's a minimum we need to "prove" DDT is useful.
- A CLI that generates the tests
- Support for Java with Spring Boot applications
I've set these goals since they will provide something useful for a large enough community and I know Java/Spring Boot well enough. I want the architecture to be generic since the concept is translatable to most languages and frameworks. So if the MVP is successful, DDT will add support to additional platforms/languages.
Technical Challenges
To be clear, I'm not sure if DDT is technically workable. So I'll try to prove it with a product prototype as soon as possible. I think that even if the product development fails, there's still a lot to learn, so this will still be a valuable experience.
I need to be prepared though, so I've given a lot of thought to the challenges that lie ahead and organized them in this ordered list:
It isn't possible - Essentially I would need to monitor every method in a running application. Initially, I thought I would use the debugger API to walk through the app. But I'm not sure that would scale. I'm considering bytecode manipulation, but that has its own problems. The main issue is one of scale. The debugger approach will work for a small application but might fail for larger apps.
Performance - it might be impractical because it has such a significant impact on performance, making the application unusable. It might consume too much RAM in real-world applications.
Generating mocks might be difficult - the generation phase would be pretty difficult since we need to understand the classes involved. We need to generate mocking code that compiles for classes we've never "seen".
Supporting other languages/platforms might be challenging. They don't all have the same capabilities.
I'll address these concerns in my following posts as I explain my architectural choices.
Tomorrow
Tomorrow I plan to talk about why I made some of my architectural choices and how you should choose the right tools for building your MVP.
I will also plan to talk about the scaffolding process and how I got started with the project.
If you want to get updates on what I've been up to. Please follow me on twitter.
Top comments (0)