Disclaimer: The software in question that used this solition was only used by two people - one of which was me. Only one of those two had write access - also me. This solution never was a business solution and was never used in production.
Quite some time ago, I was writing some PHP software for the web that would need to use a database. I wanted to design the software as portable as possible so that it wouldn't be too much hassle to migrate to another provider. MySQL was too big for what I wanted and while SQLite is powerful itself, I figured it would better suit my needs.
Sometimes when writing software, you run into a problem. Usually, there are multiple ways to solve it. My problem was, that I could not use SQLite in my PHP application, since my hosting provider did not allow me to install it and wouldn’t install it themselves.
Solution 1 would be to switch to another hosting provider which supports SQLite and run the program there. Solution 2 would be to search for another database system which does not require any installation at all. And solution 3 was what I chose to do: Write my own database solution.
While this has some very clear disadvantages like costing time, it has two main advantages. Firstly, you can control how the system behaves since you write it from scratch. Secondly, you gain a lot of experience by writing something you have never written before.
And that's how NWDatabase, a database system that comes with a PHP library and creates files with an XML syntax, was born. A software created while learning about XML, classes and much more. Surely not the best idea for production but not all that terrible for learning.
P.S.: This post was slightly altered after the first 24h, reacting on some comments.
Top comments (22)
A bunch of XML files is definitely NOT a database.
Wikipedia: "A database is an organized collection of data."
So I think, I'm not too far away from a database.
Yes you are. A bunch of custom Xml files is no better than a bunch of Excel files. Do yourself a favor and pick a database.
It seems you didn't quite understand the purpose of me making NWDatabase. I'll "pick a database" for any productive business solution of course. But please allow me to learn new things.
An excel sheet could, based on the definition I posted above, also be a database. Not a very sofisticated one - true - but still a database.
I think what they were trying to say was that although it is wonderful to go on a journey of discovery, the effort required to create your own XML database (and they do exist), is immense.
I also understand your perspective is that "Hey, I did this cool thing, it simplified thinking about structure for me".
I'd personally say that what you've done might be more complex than grabbing a thing and simply using it, but as long as nobodies life depends on it, it's a cool experiment and a necessary part of a journey, not a waste.
Next steps I'd suggest is to look into file-system limits, particularly locking and resource sharing, concurrency and document-oriented systems, and max open files. Also if you feel up to it, look into types of injections, handling of problematic artefacts, partial reads, typing and consistency of records etc.
It's mostly not even that DB systems do a good job of communicating where the fences are, solve all these problems. Just that it allows you to go home rather than spend all your efforts in one place. Where you land on spend all your time, vs build lots of things is just the difference between a systems programmer (few things in depth), and an application developer (higher-volume, less detail-oriented compared with crafting a kernel or FS driver).
Thanks for your reply and for pointing me the right direction! There's definitely a lot more interesting stuff to learn for me!
A bunch of excel files are actually a bunch of XML files. >:) Ian doing the same for an Excel addin. I do not have the right for a DB and vsto is super simple. C# plus some attributes are perfect. Though, multi access should be prevented, it's real pain...
You PHP guys are really one of a kind. ;-)
Back in the days, when I was working in a PHP shop, people would do such things all the time. Often they end up with rather incomplete idiosyncratic solutions.
Later I got into JavaScript and now have the different extreme of people pulling in libraries for everything (left pad, lol) ending up with 1MB own code and 4GB libraries which get you in dependency hell on the updates every few months.
Guess, becoming a senior developer is figuring out what to pull and what to write yourself, hehe.
OR you could use PDO and stay with any DB is available. MySQL isn't an overkill, and a custom DB is never a good choice. Yes, it's a good way to learn interesting things, but it's a very poor choice for a business product.
I agree. However this was not a business product, so I think for my very personal use, this solution is okay.
Cool that you created this to advance your learning. A lot of developers that are new to a language will build their own framework, or something else. I like that you tackled an equally necessary tool, a database. I’m sure you learned a lot!
I think, however, you should add a disclaimer to your Github repository cautioning users against using this in production. Even for the most basic application, MySQL is really NOT too big. There are practically millions (or more) tiny applications relying on a tiny MySQL database. It’s perfectly reasonable.
I would also urge you to use your experiences to quickly bring yourself up to speed with MySQL, PDO, and maybe even an ORM like Doctrine. Knowing these tools well will give you a good foundation in technologies that are actually in use around the world by many different developers.
As I said before, be proud of what you did, but remember that “rolling your own” in many cases is not actually a good idea for production applications. You want tools that have been vetted by the community, unit tested, and have, in large part, many serious bugs worked out already. Most business objectives represented in code distill down to the same set of problems most every time. Mastering CRUD, and REST, and some other standards will increase your confidence and ability to tackle anything.
Finally, I’ve probably made a lot of assumptions here about what you’re trying to achieve in your own career. I hope I didn’t offend or seem dismissive about your work. I encourage you to keep playing around and learning. That’s what makes us grow as developers. Best of luck!
Thanks for taking the time to reply to my post and thanks for the encouraging words!
To clarify, I am familiar with MySQL, MS SQL, SQLite, NoDB and of course PDO.
You made a good point, adding a disclaimer. I should also add a disclaimer that this solution was never used in production in the classical sense.
More important than the cost of time, is whether the end result is worth its salt. Given how much is involved in writing something as complex, nuanced and edge-case-filled as a database, the odds would be stacked high against any lone developer.
The end result is - for me personally - totally worth the effort, since it taught me a lot about classes (something fairly new to me at that time), XML and things to think about when writing such a complex program.
Yes, if the end goal is to gain experience and understanding, then it's definitely worth it. I've done similar things, by building framework pocs, for e.g. dependency injection, logging and module loading: what you end up with is typically isn't production ready to any degree, but the exercise itself does make you a better software developer :).
Thanks for the kind words!
I too made a framework for logging! It's called NWLog and also can't be considered production ready, I guess.
There's a difference between a database and a database management system (DBMS). What you built is just a database. MySQL is a DBMS.
As long as you used a good design pattern with your classes, I say kudos!
Thank you! I used the word "system" because I didn't know how to call it since the database itself was just the.xml file for me. The database system was everything else.
Is this not just a flat file system? Or at least very close to it?
Edit: Maybe I’m misunderstanding what you built.
Maybe I'm using the wrong vocabulary, but Wikipedia says that "A database is an organized collection of data." And thats what my solution does.
to be honest, I don't remember. Much too long I guess :D