Katie

Posted on Feb 12, 2019

How should my team use Git for this specific project?

#discuss #git

TL/DR: We don't need everything Git does, and transitioning to Git is going to add more steps to our job. How can we ease the pain?

I'm involved with a small team that has been using SFTP to put code and configuration files onto a side-project server.

A newer member of the team said, "Hey! Why don't we use Git to track the contents of the CUSTOM_CODE folder on this server, and keep a copy of that repository with a reliable cloud provider? We'd have good backup, change logs, and we'd have a better idea of who changed what, why, when."
The response was basically, "Okay, sounds good. Set it up and teach us how when you're ready."

The other day, he announced that he's all ready to go with a 3rd-party cloud repository that we should treat as the "source of truth," that he has set up with continuous integration so that if we change something in our 3rd-party repository, it will overwrite the contents of the CUSTOM_CODE folder on our server.

Therefore, rather than SFTPing into the server and uploading files to it, we'll want to use standard Git procedures to throw our code into the 3rd-party repository (and then, apparently, just wait for the CI system to pass our changes over to the server).

Now, the files in CUSTOM_CODE aren't really part "software running on" the server in the way that, say, the code responsible for keeping dev.to online is "running on" a web server.

They're mostly-unrelated-to-each-other scripts, scheduled through a task scheduler, to clean up data, move it around, etc.
Typically, a file in CUSTOM_CODE:
- is its own self-contained world of business logic,
- is only edited once every 3 months - 10 years,
- when it is edited, takes about 5 minutes - 2 hours to edit,
- often takes more like 1 minute, because someone without server access made a 1-line edit to a copy of the script they'd kept all along on their hard drive and now just wants a "gatekeeper" to update the server with the latest version of the file containing the script.

In other words, a lot of the "distributed development" and "merging" problems that Git is so great at solving aren't problems we currently face, or expect to face in the near future.

(There's a different team where I work that develops traditional web apps ... I'm sure their workflow is quite different. I feel like I've seen them share responsibilities editing a tightly-coupled set of files for months on end, rather than "for 5 minutes - 2 hours.")

I asked my colleague who set up the 3rd-party remote and the CI, "Okay, so instead of just logging into the server and dropping a file into its filesystem, we'll do the following?"

Make sure we have a fresh copy of the "official" codebase on our computer in a repository that's a "local" for the 3rd-party "remote"
Edit the file on our local computer (either according to our own preferences or as specified by someone asking us to "gatekeep" their script updates)
Use various "Git" commands to get our "new & improved" copy of the file's contents over to the 3rd-party "remote" along with our commit notes, doing "merges" to fix any issues that came up from our own failure to plan in "step 1" (or, extremely unlikely given the low frequency of file updates, a colleague working simultaneously to us)
Wait for the CI to put the file on the server
Be patient and resist the temptation to just SFTP into the server and drop the file there

He basically confirmed "yes," and added, "The CI will overwrite anything the 3rd-party remote doesn't know about that you SFTPed onto the server, if that helps w/ #5."

That said, he said he's relatively new to Git himself, so he didn't have a lot of opinions about best practices.

For example, I said, "Do you think we might want to share git fetch --all + git reset --hard origin/master with the team as a "before beginning to edit code" best practice?
He answered, "Why not just merge if there's a problem?"
I replied, "I dunno. Because I've never merged before and it sounded harder, given the way we typically work with files. It kind of feels like the way I occasionally use my computer to edit my blog files on GitHub. I don't really care what's on my computer; I just care what's on GitHub. But I have no idea what merging is like. It just sounded like work."

It seems neither of us have enough experience with Git to have any strong opinions -- just things we have / haven't yet tried.

So ... Dev community ...

Any "war stories" & "best practices" from collaboratively editing filestores like ours through Git?

It feels like there is a lot of potential for frustration compared to the old "SFTP, drag, drop, exit" system.

Tips to help old dogs learn new tricks, so that the whole team will enjoy the transition and feel productive, would be much appreciated!

Top comments (14)

Nicole Archambault • Feb 13 '19

Thanks for tagging me in this Katie! I don't post on here nearly enough. :)

So, interesting story on my end... my first dev job was a teeny tiny Cold Fusion shop on Cape Cod. It was two guys, a dog, and myself.

The first day on the job, I was met with a disastrous server with many, many files cluttering it up. If you didn't know what file was what, and weren't paying attention, things would get messy fast. The dog barked at me once for touching a file and saving it without changes, which updated the modified date and supposedly making things difficult for someone. I couldn't help but think to myself that I wasn't the one who initiated that process.

There was constant "who dis?" discussions in our open-plan office space, which took away a lot of time and added confusion to the experience. I suggested Git, but they did not know ANYTHING about version control. I mean, they thought it was a great idea, but also were totally unwilling to switch to anything new during such a busy time.

Now, that's a pretty extreme example of SFTP gone awry, but the lessons I took from it were three-fold:

Your code needs to be safely stored in multiple locations, BUT
You need to know who's doing what with your code and when, AND
Figuring that out can't take too much time out of your normal flow

I'd ask myself (and your coworker) if the trade-off is sound. On one hand, if it ain't broke don't fix it totally applies. But I wasn't sure if y'all were more "let's try something new" compared to "this is broken". So that's going to be totally relative and based on your prior experience with SFTP.

If it really comes down to multiple steps being an issue, you can set up some Git workflows that might help create your own customized "best practices" for your team.

I'm excited to read others' advice! This is a really interesting issue that affects smaller teams in unique ways.

Katie • Feb 13 '19 • Edited

I think people are definitely excited about the promise of:

More granularity about who changed what when
... backups? I haven't actually been on the team long enough to know if it's already backed up through other means (e.g. some sort of simple nightly clone), and if this would just be a different sort of backup

Everyone seems to be pretty open to it, so to me, what I guess I'm really hoping to get a better sense of is "Git workflows and concepts that might ease our lives."

I think "foolproofing" is a good part of a smooth transition. The easier it is to avoid getting into pickles that require people who don't know Git to spend an afternoon researching Git, the better.

Sort of like when I suggested to my colleague, "Hey, what if we taught everyone to hard-reset before they muck around, so it feels more like a slightly-convoluted SFTP upload than a whole new concept?"

Only I don't actually have experience to know if that is a good idea.

So looking for ... the actual good ideas. :-)

Shaun McWhinnie • Feb 13 '19

For the your query about the merge issues, you could look at a workflow like Git Flow to ease the problem of having messy merges.

Each update lives on its own feature branch (feature/my-update etc.) which is merged into develop when completed - if you have people working on files simultaneously the most you need to do usually is make sure you git pull and git merge develop into your feature before PRing. Features are cheap and offer a lot of the benefits you've mentioned above about granularity.

Im not sure if it's 100% relevant to your use case, and I'm not a git jedi by any stretch of the imagination, but if you have questions about this, I'd be happy to try and answer.

Katie • Feb 13 '19

Thanks; this is exactly the kind of comment I was hoping to hear! Will read this doc and practice a bit ... maybe on my next updates to my blog.

Mikael Klages • Feb 13 '19

Consider this: if a merge conflict occurs with git, two people have changed the same file. If you just replaced the file in the normal system, then you lost the previous changes.

If you only do a small change, and you forgot to pull before changing the file, then it's still not too late to pull. Just pull, commit, then push, and no merging is needed unless two people have changed the same file.

And in the scenario that you comitted once before pulling, just soft reset to master, pull, and do the commit again.

Katie • Feb 13 '19

Thanks -- I haven't done the things you described, but your comment is very clear, so I know what to Google and practice next so that I can help be a point-person for my team!

Jesse Phillips • Feb 13 '19

If you make changes and then pull you will be fine, rest is not needed and makes it complicated.

You can look into the rebase workflow for some more advanced tools.

Katie • Feb 13 '19

Next up on my "to practice" list: editing my blog directly through the web interface, editing the same file on my computer on a different line w/o first fetching a fresh copy, and learning to pull & merge. Thanks for the idea!

Jesse Phillips • Feb 14 '19

This will help, and it doesn't require doing changes (your idea is probably still valuable though)
learngitbranching.js.org/

dallgoot • Feb 14 '19

i'm giving you some opinions and french training ;)
alors d'abord GIT est le meilleur moyen de tracer tout changement.
MAIS il faut avoir un workflow clair et suivi par tout le monde.
Le Merge est l’étape finale : avant ça chaque Dev doit apprendre à gérer son GIT local de manière propre.
Ca veut dire que chaque nouveau développement doit avoir les phases suivantes :

git pull pour récupérer la dernière version (de Dev ou de production selon la politique choisie)
git branch MyBranch : chaque développement est une branche spécifique avec une convention de nommage ie. "Feature-654" "Fix-786" : un nom qui permet de lier la demande business avec les modifications.
git stash va devenir ton ami : permet de mettre "en pause" des changements, changer de branche sans perdre son travail, switcher entre projets sur un même code
git push : simple : on envoie MAIS utile seulement si il ya une validation par quelqu'un d'autre sinon ça revient au même que pousser en SFTP

En résumé : c'est toute une autre façon de gérer son code. Ça permet de revenir à une version précise , d'identifier les auteurs, si les demandes sont passées en production (est-ce que le Fix-454654 est merged en production ?) et globalement ça permet aussi de prendre du recul sur l'architecture du code : 2 fichiers modifiés pour un même besoin = possibilité de créer un module qui concentre la fonctionnalité.

Concernant ces scripts spécifiques le 1er gain ça va être de pouvoir avoir des Dev différents qui travaillent - en même temps - sur un script et de pouvoir intégrer chacune des modifications sans devoir éditer manuellement.
J'espère que ça te sera utile.

Katie • Feb 14 '19

C'est parfait; merci ! :-D

Jesse Phillips • Feb 13 '19

I believe utilizing git is the right choice, and will try to argue the system was broken (but may fail not being in the system)

Git's distributed nature provides an architecture with benefits not related to work distribution.

It sounds like these files drive an important function, I would expect issues could come from incorrect edits. Git will provide the simplicity of allowing any member to roll back without knowledge of the edit causing this issue.

On the matter of merging, you'll have issues when there are two edits and those edits are in the same area. If you choose a hard reset then you prevent the tool from helping you keep all changes happening.

If there are patterns to the changes then history provides a list of example edits.

My claim that the old system is broken comes from the idea that mistakes get made and versions history gives insight to those mistakes for the better.