DEV Community

loading...
Cover image for I write my own web desktop OS for 3 years and this is what it looks like now

I write my own web desktop OS for 3 years and this is what it looks like now

tobychui profile image Toby Chui ・9 min read

This is a story about a poor student who just want a storage solution for his musics, video and photos using a Raspberry Pi.

2021-04-04_21-35-31

GitHub logo tobychui / arozos

General purposed Web Desktop Operating Platform / OS for Raspberry Pis, Now written in Go!

Image

IMPORTANT NOTES

The current arozos is still under intense development. System structure might change at any time. Please only develop on the current existing ArOZ Gateway Interface (AGI) JavaScript Interface or standard HTML webapps with ao_module.js endpoints.

Features

User Interface

  • Web Desktop Interface (Better than Synology DSM)
  • Ubuntu remix Windows style startup menu and task bars
  • Clean and easy to use File Manager (Support drag drop, upload etc)
  • Simplistic System Setting Menu
  • No-bull-shit module naming scheme

Networking

  • FTP Server
  • WebDAV Server
  • UPnP Port Forwarding
  • Samba (Supported via 3rd party sub-services)
  • WiFi Management (Support wpa_supplicant for Rpi or nmcli for Armbian)

File / Disk Management

  • Mount / Format Disk Utilities (support NTFS, EXT4 and more!)
  • Virtual File System Architecture
  • File Sharing (Similar to Google Drive)
  • Basic File Operations with Real-time Progress (Copy / Cut / Paste / New File or Folder etc)

Others

  • Require as little as 512MB system memory and…

How it looks like when it started

Since 2016, I have been working on this project mainly for storing music and video files so that I can play them during my extra time between lectures in university. This is what it looks like back then.
ArOZ Online Preview

The old version use PHP5 and later upgrade to PHP7 with Apache server as back-end. It didn't have a database connection although MySQL is installed on the same host that host the system. The system is glue together with a VB.net binary that scan and buffer the song list in background and the php script just render the list as a list of stream-able music files.

The source code can still be found on Github (But please don't use it)

Migration from PHP to Go

After using this system for a year or so, it started to get slow. One of the reason is that I am putting more and more large files into the system, and php itself cannot handle as many request as I needed to make a web desktop system native like. This is most noticeable when I am trying to copy a large file from one location to another location on disk. Hence, I must make a switch to another language. After some research, I decided to go with Go. The system, original named ArOZ Online System, was also renamed to ArozOS for simplicity.

How ArozOS looks like after 3 years

After 3 years of development, it basically recreate the native desktop experience on the web. Let starts with the desktop!
2021-04-04_21-07-47

Desktop

The desktop basically recreate a mix of Ubuntu 20.04 as well as Windows. You can see a Ubuntu GNOME like calendar with notification bar.

2021-04-04_21-07-52

You can also find a quick menu on the top right hand corner for powering off / restart your host (which can be disabled with startup parameter) and a shortcut for full screen and system settings
2021-04-04_21-07-58

Just like Windows, it is really easy to create new items on your desktop as well as changing your desktop wallpaper.
2021-04-04_21-08-04

2021-04-04_21-08-19

The start menu is kind of remixed style of both world too.
2021-04-04_21-57-11

And of course, my favorite rename UX ever: 2021-04-04_22-56-57
(No pop-up box is just amazing)

File Manager

After some period of time, I am starting to upload other stuffs to my system. The best way to provide a generic interface for all types of files is of course, a File Manager. So I build one into my system!

The File Manager on top of my web desktop system contains two viewing mode. The list mode and the grid mode.

2021-04-04_22-00-40

2021-04-04_22-00-27

In Grid Mode, the File Manager will try to render everything to generate a thumbnail for easier file identification. It works on many different file types from video, music or even 3D model files.
2021-04-04_22-58-36

The File Manager originally didn't have a way to sort files. As the number of files uploaded into the system are growing larger day by day, this was soon added to provide different sorting modes to make finding a file much easier.
2021-04-04_22-01-29

And afterward, search mode (with the magical regex support of course!) was added to make finding files in a large folder much easier.

2021-04-04_22-01-35

2021-04-04_22-07-12

Btw, it also got a dark theme mode for finding your movie at night.
2021-04-04_22-07-48

File Operations

The File Manager provide many basic file functions. You can find all of them in the context menu by right clicking any files or folders.
2021-04-04_22-08-59

Just like a native OS, you can choose the open a file with a given WebApp with a default app selector.
2021-04-04_22-09-09

To make it much easier to use, I also implement drag & drop style of file operation support. So you can easily drag a file from one File Manager window to another and the system will start copying / moving the file for you.

2021-04-04_22-09-45

As Golang provide async way to handle request using go func, it makes it much easier to handle large file operations without freezing the whole system. The progress is powered by websocket and the backend will constantly return the file operation progress using the websocket connection.

File Sharing

Once a while, I would want to share (large) files to my friend. In that case, I can just use my own Web Desktop File Manager's share function!
2021-04-04_22-18-40
(This is just an example. Do not give your friend links starting with domain named localhost)

Then my friend can download the file directly from my web desktop cloud system. Very convenience right?
2021-04-04_22-20-32

Back to basic: Music and Video

This system was originally created for Music and Video streaming. So after 3 years, these function still exists and even get better then before. With the latest PWA technology, the Music module now become something like this
2021-04-04_22-23-16

This interface can also be installed onto an Android phone as WebApp and even the player can be shown in native Android notification (which is call mediaSession btw).

2021-04-02_22-31-12

The video module also get updated and powered by latest HTML technology, it can playback video with streaming mode, making movie streaming possible anywhere with a computer and browser.
2021-04-04_22-27-44

System Settings

Basics

To manage such a complex system, a complex system setting menu is needed. We try to make it easier for anyone to manage the system so we go with the category approach that categorize system functions into different types.
For the most basic one, we can see CPU usage as well as host hardware information from the setting interface.
2021-04-04_22-29-58

2021-04-04_22-30-38

WebApps and Subservices.

Of course, as a web desktop platform, I also make an interface for those who need to manage their apps. Here is a list of apps I have written so far.
2021-04-04_22-30-51

It doesn't make sense to ask the user (aka me) to change the code every time I want to modify any web apps right?
So I decided to split the webapps into two parts. WebApps are those only require basic permissions and do not interact with the host OS. They will be run inside a sandbox created using ECMA5 VM called Otto. The other type is called Subservice, in which it will need extra permission to interact with the OS and require higher level of access to the backend file system.

You can easily install and uninstall webapps using git link or zip files uploaded to the web desktop system.
2021-04-04_22-31-31

Subservices can only be toggled on / off. If you want to make changes to subservice (which might effect system stability), you have to do it with the terminal instead. Here, showing 3 subservice I use most of the time.
2021-04-04_22-31-16

Of course just like Windows, you can choose what application to open what file extensions using this menu.
2021-04-04_22-30-59

Network and Connections

The web desktop can be accessed via FTP or WebDAV. So to make it easier for me to enable them only if I needed to, I make an interface for them in the System Setting module.
2021-04-04_22-37-53

2021-04-04_22-37-58

Mounting WebDAV folder on Windows without TLS but with user isolation is quite tricky. I got a few hack to make it to works. But this makes the connection process a bit complex and difficult.

For devices with WiFi, you can even change the device WiFi connection via the system setting interface.
2021-04-04_22-40-54

If you enabled uPnP in the startup parameter, you can also setup custom rules in port forwarding. So you can expose more ports for other web services running on the same host.
2021-04-04_22-41-00

User System

First of all, a basic user setting page for changing user icons and password
2021-04-04_22-40-09

Sometime my friend with no disk space will "borrow" some of my storage places to put his file temporally. In order to isolate users in the system, I created a structure that allows each users belongs to a permission group and then by assigning permission to the group, I can limit what a user can access. Then, I build an UI for setting up those complex group systems.
2021-04-04_22-40-14

I also make an UI for enabling the public register function, just in case my friend don't want to let me know his password for creating a new account.
2021-04-04_22-40-20

Storage Management

Finally, to manage so many files, we will need disk manager. The System Setting have build in tools to visualize disk usage, help you find space in disk by locating large files and even DISK SMART thanks to a few PR by a user that actually use the system as his NAS OS.

2021-04-04_22-52-05

2021-04-04_22-52-11

2021-04-04_22-52-19

To assign storage devices to permission group, a storage pool architecture was designed to fit this purpose. I doubt this system is the most basic way to implement this feature, but this can leave some space for future system expansion if I want to connect other cloud drive like Google Drive or OneDrive to the web desktop system.

2021-04-04_22-52-24

It also got an advance function that allow you to format a partition just like what you do on Windows Disk Manager
2021-04-04_22-54-38

Others

There are so many other WebApps that I want to share but due to time limit, I can only show you the most basic one. Maybe later, when I got the time, I will share with you more WebApps I wrote for this system. Here are some screenshot of them2021-04-04_22-58-50

2021-04-04_22-58-12

Know More

Feel free to leave a star if you are interested in this project :)

GitHub logo tobychui / arozos

General purposed Web Desktop Operating Platform / OS for Raspberry Pis, Now written in Go!

Image

IMPORTANT NOTES

The current arozos is still under intense development. System structure might change at any time. Please only develop on the current existing ArOZ Gateway Interface (AGI) JavaScript Interface or standard HTML webapps with ao_module.js endpoints.

Features

User Interface

  • Web Desktop Interface (Better than Synology DSM)
  • Ubuntu remix Windows style startup menu and task bars
  • Clean and easy to use File Manager (Support drag drop, upload etc)
  • Simplistic System Setting Menu
  • No-bull-shit module naming scheme

Networking

  • FTP Server
  • WebDAV Server
  • UPnP Port Forwarding
  • Samba (Supported via 3rd party sub-services)
  • WiFi Management (Support wpa_supplicant for Rpi or nmcli for Armbian)

File / Disk Management

  • Mount / Format Disk Utilities (support NTFS, EXT4 and more!)
  • Virtual File System Architecture
  • File Sharing (Similar to Google Drive)
  • Basic File Operations with Real-time Progress (Copy / Cut / Paste / New File or Folder etc)

Others

  • Require as little as 512MB system memory and…

Discussion (53)

pic
Editor guide
Collapse
faraazahmad profile image
Syed Faraaz Ahmad • Edited

I had a similar idea when I was in 6th or 7th grade of making an "OS in the browser" but back then I was a noob little kid with no skills, the best I got was a lockscreen and an app drawer like that of Firefox OS (remember that?). Seeing this fully realised is very satisfying to me. Great job!

Collapse
tobychui profile image
Toby Chui Author • Edited

Glad that you like the ArozOS project :)

I still remember Firefox OS as a mobile OS instead of a full desktop interface, and it is also one of the design concepts I studied before I started working on this system. This is the reason that I implemented the web desktop for mobile in ArozOS as well.

I guess now you should have the required skills to make your own "OS in the browser". Maybe you can consider give it a go again after all these years?

Collapse
faraazahmad profile image
Syed Faraaz Ahmad

Haha I wish! Now that I have enough skills to figure out how to build it, I don't get enough free time 😅. I tried building an actual OS from scratch using an awesome tutorial by Phillip Opperman but it was too much work. Maybe if I get super excited about this again I might give it a go. Until then I might just find some hobbyist OSes and contribute to them.

We shall see!

Collapse
manoharreddyporeddy profile image
Manohar Reddy Poreddy

OMG!
I can't even think of doing an OS (done by many many people) all by myself 😊
I am into other things. If you like, find me on LinkedIn via my profile.
OS is just beautiful & you are really brave to attempt & do such a thing.
Best of luck.

Collapse
kodekrash profile image
James Linden • Edited

This is neat! I started a similar project at least 3 times but never finished one. Kept getting distracted by the hardware aspect because I'm bad with UI and wimped out. My use case was an RPi with 8" touchscreen as a frontend to an Intel i7 NUC... in my truck. Music, GPS/nav, ODB2, internal and exterior cams, wifi hotspot, video calls, lighting, plus integration with my various ham gear. Hopefully, I'll get back to that project one day!

Collapse
tobychui profile image
Toby Chui Author

Your system sounds more powerful than mine! Looking forward to your sharing after it is finished :)

Collapse
shmuelhizmi profile image
shmuelhizmi

Wow, I have been working on a web based desktop environment for almost a year and Im really surprised this is the first time I see your project.

This is my project by the way

Collapse
tobychui profile image
Toby Chui Author

Glad that you finally notice this project XD
I have been following a few web desktop projects (including your project of course) on Github & dev.to for quite some time. I am amazed by your idea to move Linux GUI application to web interface. I am looking forward to see your implementation as well as new updates on your designs. :)

Collapse
shmuelhizmi profile image
shmuelhizmi

Thanks, that means a lot. 😄

Collapse
aurelmegn profile image
Aurel

it would be interesting to list the tool you used. Or do you write all by yourself?

Collapse
tobychui profile image
Toby Chui Author

Yes, I write it all by myself through out my 3 years journey of developing this system. You can find the source of all the WebApps in my repo.

Collapse
aurelmegn profile image
Aurel

It is awesome by the way

Collapse
mikeschinkel profile image
Mike Schinkel

Hi Toby,

This is pretty amazing.

I do have a question about this though. I looked for a license but could not find one. I see here on dev.to that you refer to yourself as an open-source developer so I assume you mean for ArozOS to be open source? If yes, will you add a license to your GitHub repo so that anyone who seeks to contribute might know what they are contributing to?

This link explains how to assign a license:

docs.github.com/en/github/creating...

BTW, I also created an issue on your repo asking the same:

github.com/tobychui/arozos/issues/57

Collapse
tobychui profile image
Toby Chui Author • Edited

Hi! First of all, as an open source developer, sometime I do make things that is close source :P
For the ArozOS project, as you can see, all source code are available on Github for anyone who want to read it (and hence the "source is opened" ). There is a section in README that explains the usage requirement / license of this system. Feel free to contact me via email or telegram if you have any inquiry of using this system for any other use cases other than those listed in README.

Collapse
mikeschinkel profile image
Mike Schinkel

Hi Toby,

Thanks for the quick reply.

Hmm. I assumed you knew the definition of open-source already — especially being a student in CS — maybe your university does not cover it?

Please forgive me in advance if you do already know this, but your reply makes me think maybe you do not?

So there is well known definition of "open source" and you can find it here on the Open Source Initiatives website. Unfortunately while having source be readable is necessary, it is not sufficient to qualify a project as "open-source":

opensource.org/osd

In a nutshell being "open source" refers to the type of legal license you provide for your source code that allows users to freely modify a work, use said work in new ways, integrate the work into a larger project or derive a new work based on the original.

Here is the list of licenses that are currently recognized as open-source:

opensource.org/licenses/alphabetical

Given your stated usage terms ArozOS would sadly not be considered open-source by the OSI's definition, which AFAIK is pretty universally accepted.

Why do I care? I have a project called Launch I am developing designed to provide a super easy way to run Docker containers without having to even know anything about Docker:

github.com/gearboxworks/launch

When I saw AroxOS I thought Launch would be a great tool to help you let other people easily run your solution without having to figure out how.

Of course our motivation would be that everyone who used Launch to run ArozOS would of course end up with Launch installed on their computer.

However, for software where the license in unclear there is too much legal liability for me to wrap your software and distribute it via our Launch repository. That is why I wanted to know if you planned to license it as open source.

If you do decide later to license it with one of the established open-source licenses please do reach out to me via email at mike at newclarity.net because I think there could be some real synergy between our two projects, assuming you are looking to make it really easy for people to try it and/or use it (which I assume you do since you wrote this article.)

Whatever the case, you've created some pretty amazing software. Kudos!

-Mike

P.S. We have not started a public push for Launch adoption, but plan to soon. Also, our Launch source code is not currently on our repo but will will release it and license with an OSI-approved license once we reach 1.0 and really start promoting it.

Thread Thread
tobychui profile image
Toby Chui Author

Hi Mike, thanks for your detail explanation. This is just a simplified reply that I post on dev.to. Please proceed to your GitHub issue to see the full reply regarding the license issue. We are looking forward to cooperate with you if you are interested in using this project for your business and we can make a great deal out of it :)

Thread Thread
mikeschinkel profile image
Mike Schinkel

Thanks!

Collapse
lukaszcierocki profile image
Collapse
tobychui profile image
Toby Chui Author

Thank you!

Collapse
ianwijma profile image
Ian Wijma

This has been something I've been thinking about. Really interesting on how you implemted it. Any plans for future features or changes. Just curious.

Collapse
tobychui profile image
Toby Chui Author

I am thinking of supporting IoT devices (as a control hub) as well as cluster features for redundant file storage across different hosts. But feel free to share your thought by creating an Github issue. I would love to hear your ideas as well :)

Collapse
dystopiandev profile image
Redhart

I did something once. It's small but could add to ideas you'll come up with to standardize IoT support in ArOZ: github.com/dystopiandev/remoffice-...

I wired an Arduino UNO to a multi-channel relay board (as a sample IoT control gateway), then I linked the UNO to a RPi3B over I2C, which I interacted with from a Node.js backend over a Vue-based interface.

ArOZ is a wonderful concept with gorgeous aesthetics!

Thread Thread
tobychui profile image
Toby Chui Author

Thanks so much for you sharing!

I have checked your Github repo and seems you have done something similar of what I am working on. In ArOZ, it use something similar to your design but instead of I2C + Arduino, it use WiFi with ESP8266. I have uploaded some alpha testing code on Github over here and named it Home Dynamic v2
github.com/tobychui/Home-Dynamic

Maybe I will write a new post sharing my design concepts on this IoT system later.

Collapse
richzendy profile image
Edwind Richzendy

Well, is really amazing all your work there, will be great if someday a NAS hardware maker use your OS on his devices and pay you for support and next releases.

Collapse
tobychui profile image
Toby Chui Author

Thanks! I think I will switch to full time dev if that really happens :)

Collapse
crazycoder009 profile image
RRN

Bro! I checked out your repo! Totally loved the Whole OS. I must say I am impressed and you got me there.

Collapse
tobychui profile image
Toby Chui Author

Thanks! Glad that you like this OS. Hope this project can bring you some insight for you next projects :)

Collapse
crazycoder009 profile image
RRN

Just a Small suggestion, If there was a easier to remember name for the OS it would have been great. I literally forget the name everytime, maybe it will take some time for me to register.. ha ha but anyway thank you for the gr8 work!

Collapse
crazycoder009 profile image
RRN

Definitely!

Collapse
bawa_geek profile image
Lakhveer Bawa

awesome,

Collapse
tobychui profile image
Collapse
davidag9 profile image
Agyakwalf

I am a little lost here , As I am still green horn at systems programming ....I wan to ask if it is an OS written in go that sits on a web server or a Webapp that's interfaces with a desktop or laptop ....or probably I am asking the wrong question ...someone help

Collapse
tobychui profile image
Toby Chui Author

This is an Online System that reproduce a native desktop environment on the web.

Collapse
ahmedash95 profile image
Ahmed Ashraf

Impressive

Collapse
mtzagkarakis profile image
Manos Tzagkarakis

I could not imagine the amount of man hours you put in it. Good effort

Collapse
tobychui profile image
Toby Chui Author

Thanks! I can't count how many man hours I put into this project but I do put a lot of my free time improving this system so as to fit my own needs XD

Collapse
theowlsden profile image
Shaquil Maria

This is awesome! I wanted to create a GUI for file management on raspberry pi once but never got to do it. I'll be keeping an eye out for this project.👊🏿

Collapse
tobychui profile image
Toby Chui Author

Glad you like this project :)
For Raspberry Pi, there are quite a lot of different options for file management like the golang file browser project. You can take a look at them if you only need a file browser instead of full desktop experience.

Collapse
pssolanki111 profile image
P S Solanki

This is brilliant work, Toby! Keep up the good work <3

Collapse
arwen profile image
田博文

This is so cool!

Collapse
nickdivona profile image
Nick DiVona

Incredible work. Quite the undertaking, but seeing the end result of your work is pretty inspiring.

Collapse
tobychui profile image
Toby Chui Author

Thanks! Hope this project can inspire you some new insights on your own projects too :)

Collapse
bytemazicodes profile image
byteMaziCodes

This is a Brilliant work. Awesome!

Collapse
frankity profile image
Douglas Brunal

it looks cleaner than Chrome Os <3

Collapse
sarahob profile image
Sarah 🦄

This is so cool! Really awesome work 👏🏻👏🏻

Collapse
tobychui profile image
Collapse
tlacalv profile image
Tlacaelel Leon Villaseñor

It's amazing and overwhelming, how did you approach such a large project, I can't even think on how to start something like this.

Collapse
tobychui profile image
Toby Chui Author

It starts small (just a few hundred lines of PHP code), then I add things that I want into it during my free time between lectures (and also use it to automate some of my daily works). After years of tiny improvements it grow this large, which is kind of unexpected :D

Collapse
rageobi profile image
Abinanthan Raghunathan

Kudos man! Great work.

Collapse
kievandres profile image
Kiev Andres

Awesome! 🔥🔥

Collapse
jmau111 profile image
Julien Maury

impressive work

Collapse
tobychui profile image
Collapse
joshbnntt profile image
Joshua Bennett

Love it!