My concern is to automatize the backup of my configuration files (aka dotfiles) and to generate a snapshot of all installed apps, whether it be through my distro's package manager or through third-party package manager like npm or pip. This is the first step before being able to sync my apps automatically on a newly installed computer (still to be done).
Backuping dotfiles
What I did is create a git repo in my $HOME dir as all my dotfiles are located here. But as I don't want to push all the other files (Pictures, Downloads, Desktop...), I put a .gitignore file with just a wildcard (*) in it. This way, when I want to add a file to the repo, I have to force with: git add -f newfile.
cd ~
git init
echo "*" > .gitignore
git add -f .gitignore
git add -f .bashrc
git add -f .zshrc
git add -f .config/fish
git add -f .tmux.conf
git add -f .SpaceVim.d/init.toml
git commit -m "First commit"
git remote add origin git@github.com:username/mydotfiles
git push -u origin master
⚠️ But be aware that it could be dangerous for your sensible data. For example, don't push your ~/.ssh config!
Thus, you have a quick way to retrieve your configuration in case of a computer fresh install or in case of a crash.
The next step is to do the same with your apps.
Generate a snapshot of your apps
Regarding the apps, it's not so easy. It's not unusual to have several package managers to install new apps. Personally, I use 7 of them: npm, cargo, ghc-pkg, composer, gem, pip, and pacman, the manager of my Manjaro linux distribution. Each of them has its own syntax to install, delete or update packages.
But each of them also has its own syntax to output a list of installed packages.
So basically, I just use the following bash script to generate the list of all installed packages, across all the package managers:
#!/bin/bash
echo "Generating the lists of explicitly installed packages in ~/.backup"
pacman -Qe > ~/.backup/pacman_packages || echo "pacman failed"
gem list > ~/.backup/gem_packages || echo "gem failed"
npm list -g --depth=0 > ~/.backup/npm_packages || echo "npm failed"
pip list > ~/.backup/pip_packages || echo "pip failed"
cargo --list | tail -n +2 | tr -d " " > ~/.backup/cargo_packages || echo "cargo failed"
ghc-pkg list > ~/.backup/ghc-pkg_packages || echo "ghc-pkg failed"
composer global show | cut -d ' ' -f1 > ~/.backup/composer_packages || echo "composer failed"
git add -f .backup
exit 0
Note the git add command at the end of the script. I do this to automatically add the ~/.backup folder to the git index so that it gets pushed to the remote. And as I put this script in the pre-commit hook, my packages list gets synced with git each time I commit:
biros on master [!]
➜ git commit -m "Update conf"
Generating the lists of explicitly installed packages in ~/.backup
[master 27f73e4] Update conf
1 file changed, 1 insertion(+)
⏭️ Next step: be able to re-install all the apps automatically from the backup.
💡 Hint: you can sync your Atom plugins with the sync-settings plugin.
📚 Other resource on the same topic:
Top comments (17)
What I do to back up my dot files is I create a symbolic link in dropbox.
For example:
With this setup I don't have to do a git commit & push each time I change one of my dot files :)
Smart idea and so simple that I didn't even think of it!
That's a nice approach, and I thought about it when I tried to sync my dotfiles with git, but it sounded weird to me to have my home folder as an entire repo, so I decided do move all my config files into a
dotfilesfolder, init the repo there, and create symlinks to those into my home folder withstow(if you've never heard about it, you should give it a go).I wrote a backup tool to automate both back up and reinstallation of dotfiles, packages and config files: github.com/alichtman/shallow-backup
Hi, it sounds great. But it seems to be limited to a list of pre-configured apps, right ?
It backs up lists of packages installed from most popular package managers, backs up whatever dotfiles you want (since the config file is user editable), and some configs (this is the only place it's limited right now.) I'm working on adding arbitrary path backing up, which would make it possible for you to back up whatever you want. I just finished up git integration, and the next big step for this project is expanding the scope of what can be backed up.
One cool thing is that it's open source, so if it's not backing up something that you think it should be, you can just write the code and open a PR!
Idea with backup list of instaled apps is awesome. I doesnt think about it firstly and then after format I have missing many apps :c
You should check my instalators on my github (github.com/westscz/.dotfiles/tree/...). I created instalators for apt, flatpak, pip, snap, git (still under development), maybe you can learn something from them :)
I will have a look on it ! Thx
Creating a git repository in a home is not a good practice, for a lot of reasons (especially as it basically works recursively). Your first commit is a good illustration: you add all your files, then remove those you do not want to consider. And you will have to keep in mind and perpetuate this scheme potentially each time a new critical file will be added.
Why do not consider an appropriate and standard tool like vcsh (github.com/RichiH/vcsh)? You find it in standard Linux distributions, it even allows multiple repos (if needed) in home and do not change the common workflow: you only add the files you need in repos.
In fact, it's the opposite of what you describe: it's a whitelist. First you ignore ALL, then you add the files you want to save.
Maybe it's not the best way but it's quite simple and it does the job ☺️
Thanks for your suggestion, I'll have a look on it !
Yep, you're right, I misspoke :-) I described the technical steps instead of the conceptual ones (as you initially add your files to a blacklist). Nevertheless, the facts are still there with your workflow: if you insert in your home a new file, independent of your project, you will have to add it to the .gitignore. However, as it is independent it shouldn't involve side effect on your project.
vcsh has been developed for this kind of purposes and helps you to keep a clean workflow. And if you have several repos in your home (for my own, I have an emacs repo, a zsh repo, etc.), you can couple it with mr (multiple repositories) in order to manage them more efficiently.
No, if I insert a new file in my home, it will be automatically ignored by my git config.
Oups, it's echo "*" and not echo * :-)
I do something similar, but I use symlinks and an installer (sort of, it’s a ruby rake task).
You can find it here: github.com/andreapavoni/dotfiles
Do you also automatically update your "backup" on GitHub with a cron for example?
Not yet 😊
Otherwise I could use a hook in my package managers to automatically push to git when installing new stuff.