Git for 7D2D mod authors

[khzmusik]

Hello, all.

 

I was recently asked for advice about setting up and using Git. It is a subject that I think every mod developer should know about, so I decided to make this post.

 

What is Git and why should you use it?

 

Git is a version control system (VCS). A VCS includes a centralized repository ("repo") for source code, with tools and commands to organize it such that multiple developers can work on that source code at the same time. The centralized repo is called the "remote" repo, since it's usually hosted on a server somewhere. For most mod developers, the remote repo will be hosted on either GitHub or GitLab (though there are other choices, like Bitbucket, and large companies often host their own).

 

It's not meant for the same kind of filesharing that Dropbox or Google Drive offer. But that is a good thing. Once you learn how to use Git (and it's really not that difficult), it will make your life a whole lot better.

 

Version control systems have been around for decades. But unlike the older systems (like CVS or SVN), Git is distributed. This means when you copy the repo from remote to your local computer, you get a copy of the entire repository - including all its files, branches, and history. This is very useful. If the centralized repo goes down, you can still do your work. You can also work without the Internet, like on a plane or on vacation or something. And if you made a mistake with the repo itself (like committing something you shouldn't have), you can fix it locally, and only push to the remote repo when you're ready.

 

Repos can be either public or private. In most cases, public is better. Users can download from them, you have more options for free Git hosts, and it's generally better if other people can see your source code. But you can use private repos if you really want; most Git hosts provide at least one private repo for free.

 

Like most VCSs, Git is build around a model of branches and commits. A branch represents a set of code changes that are separate from the main line of development. A commit is a snapshot of the code at a particular moment in time. Commits are associated with one particular branch; the branch holds "indexes" that point to its commits. In Git, HEAD is the index of the latest commit on a branch.

 

All Git repositories have a default branch. Nowadays, this is called "main". It used to be called "master", but most Git services stopped using that term years ago. In older version control systems, this was often called "trunk", so you will sometimes see that phrase used, but it's not common in Git.

 

This is the branch that users will see when they navigate to the repo in the web browser, and it's the branch that will be checked out when the repo is cloned (unless specified otherwise). For most software teams, this is also the "production ready" branch - all work-in-progress code changes are done on other branches. (I'll talk about that later.)

 

How do I set up and use a Git repo?

 

The first thing you need to do is install Git client software on your computer. Git clients with a graphical user interface are easier to use, and there are many free options. I use, and recommend, GitKraken (it integrates directly with GitLab and GitHub repositories if they're public). But, if you're using GitHub, then it has its own Git client called GitHub Desktop, which is also pretty good (it's the first one I ever used). There are others; many are good but not free, like SmartGit. You can choose whichever one you want, but those are the ones I have used and can recommend.

 

Also, many code editors (like Visual Studio Code) have plug-ins that integrate with Git. These plug-ins allow you to issue Git commands directly inside the code editor, without needing an additional tool. Personally, I prefer the graphical Git clients, but many people like the plug-ins because they don't require you to switch to another program.

 

The clients and plug-ins are actually interfaces for Git commands issued on the command line, which means you'll need to install Git itself. Linux distros usually come with Git already, but for Windows you'll need to install it separately. The clients or plug-ins usually walk you through this process.

 

Next, sign up for a free account at one of the Git services. I use both GitHub and GitLab (the latter also in my day job), and though I prefer GitLab, they're both very good. I have also used Bitbucket (again, in my day job), but its main appeal is that it integrates with other Atlassian tools like Jira (an issue-tracking system). I doubt that most 7D2D mod authors will be using those tools.

 

Because it seems to be used by slightly more 7D2D mod authors, I'll use GitHub as the example Git host. But most Git hosts work basically the same.

 

Once you're signed in, create a repository. If you want this repository to be tracked in the Mod Launcher (which I recommend), then you'll need a separate repository for each major 7D2D alpha version. So, I recommend including that version as part of the name for your repo ("A20" or "Alpha20" or whatever).

 

I'm pretty sure SphereII still has to add Git repos to the Mod Launcher by hand, so once your code is ready, you should send him a message and ask him about it.

 

GitHub will add a couple files to the repo by default (like a README and a .gitignore file). Leave those in, they're useful. It will also create the default branch ("main").

 

Next, clone the repository. This will copy it to your local computer. Depending upon the Git client, this will be something like "Clone Repo", and it's usually in the "File" menu. You'll probably have to log in to the remote repo host (GitHub) and allow the Git client software to access your repo. This is usually painless, the client just opens a browser window which you can close afterwards.

 

Clone the new repo to an empty folder. You don't want to clone it to a folder where you already have source code. (I tried that once and it just causes unnecessary issues.) If you do have existing source code, then once you create the folder with the repo, copy your existing code into it.

 

When you're adding files for the first time, you should also update the README in the repo's root directory. In both GitHub and GitLab, these are written in Markdown. But if you don't want to learn that, you can just use normal text and it should still be readable. You might also want a LICENSE.txt file, especially if it's a standard license where the legal text is already written (like an MIT or CC license). But I usually put a link to the license in the README.

 

Hosting multiple mods/modlets in one repository

 

If you have (or plan to create) multiple mods or modlets, then I recommend hosting them all in the same repository.

 

If you want to do this, then you would make a subdirectory in the repo for each mod or modlet (the same way it is structured in the Mods folder). The Mod Launcher is smart enough to look in subdirectories of the repo for different mods.

 

There is a catch. GitHub provides a download option which creates a .zip file of the repo - click the "Code" button, choose "Local", and it's at the bottom. You can right-click on that link and share it. But that is only available for the entire repo - not for subdirectories.

 

So if you want users to be able to download mods individually from GitHub, they'd have to use something like https://download-directory.github.io/ . An alternative is to manually create .zip files yourself of each of the sub-folers, host the .zip files in the repo, and provide users with links to those files. But this is a pain, and it's easy to forget to update those .zip files when you make code changes.

 

None of this applies to GitLab. In GitLab, users can download individual subdirectories, hence individual modlets. They will be "wrapped" in an additional folder in the .zip file, but you can warn users about that. It's one of the reasons I prefer GitLab. If you decide to use that, then navigate to the sub-folder, go to the download button (next to the "Clone" button), and there will be a "Download this directory" option. Right click on the ".zip" link and share that.

 

On the other hand, if you're only going to use Git for a single mod or modlet (like a mod pack or an overhaul), then I have seen plenty of mod authors devote one repo to it. In this case, the repo's root directory would be the root directory of the mod pack or overhaul. This also bypasses the issue with .zip files in GitHub.

 

Committing and Pushing Changes

 

Any changes to the source files should show up in your Git client. If you're happy with everything, then you should commit those files. This means you take the code changes and make them an "official" part of the repo.

 

Git allows you to pick and choose which files you want to commit. To commit changes, you first need to stage those changes. This tells Git "yes, I want those changes to be included in the repo." You can also unstage changes if you change your mind, but only prior to making the commit. The conceptual "place" where changed files are marked to be committed, is usually called the staging environment.

 

Whenever you commit anything, you should create a commit message. This can consist of a short summary and a longer description. If you don't need to provide details, you can omit the description, but you should always have a summary. The first time I commit code for a new mod or modlet, I use "Initial commit" as the summary.

 

One thing to note - you can only commit to the branch you currently have checked out. If you haven't switched branches, you'll still be on the main branch. That's fine for the initial commit if you're adding existing code.

 

Once you commit, your repo is updated - but it's only updated in your local copy of the repo. In order for it to be available on GitHub, you need to push those changes to the remote repo. In most graphical Git clients, there is a dedicated button for pushing.

 

Most Git clients allow you to amend a commit - take some changes and include them in the previous commit. That works fine if the commit only exists on your local machine, but if it's already pushed to the remote repo, then you would have to actually replace the remote repo's branch with your own. This is called a force push and it's generally not recommended.

 

If you totally screwed up a commit, and want to start over from a previous commit, then you can also reset your local branch to a previous commit. Resetting comes in three flavors - "hard" (discard all changes), "mixed" (keep the working copy and reset the branch's index), or "soft" (keep all working changes). Personally, I've only used "hard," because anything else can be done by amending the last commit or by making a new commit. And, if you already pushed your changes, then resetting also means you have to force push. So I use resetting only as a last resort.

 

If you have committed and pushed your code, it should now be in the remote repo, and visible in the GitHub website. If that's all you want, then it's ready to go.

 

So what do you do if you want to make changes?

 

Branching and Merging

 

Changing code is where branches come in. It's also where issues come in, which many people use to track work.

 

Issues are basically "tickets" that say "This work needs to be done." They are often associated with bugs, but most people also use them for new features, updates, whatever. When you know you have to do something, you create a new issue. The issue has a title as a short summary, and usually a longer description of the work. The Git host will give the ticket a number when it's created.

 

The nice thing about them is that both GitHub and GitLab allow you to create branches directly from issues. In GitHub, there is a link for this that says "Create a branch", on the right, in the "Development" section. The branch that is created will automatically be named after the ticket number and title. When you make a commit, you can put the ticket number in the commit summary (like "#3: Update entityclasses.xml") and the commits will be tracked on the issue page. Also, when that branch is merged to main, the ticket will be closed.

 

If you're using GitLab, then by default, it will not just create a branch, it will also create a merge request. (See below for info about merge requests.) I don't find this useful, so I always choose the option to only create the branch. (That's also what we do where I work.)

 

You can also create branches without tickets, if you want. In GitHub it's in the same drop-down where you select a branch, on the top left of the page when viewing the source code. I just never do this because tickets are so useful.

 

These branches are usually called feature branches to distinguish them from the main branch.

 

Once you've created the feature branch, you need to check out that branch on your local machine. In most Git clients, this is as simple as double-clicking on the branch name. But before you can do that, you need to fetch from the remote repo, so the local repo's index is updated, and it "knows" the branch exists. Some Git clients have a dedicated button for fetching, others use the same button you would use to pull code changes from the remote repo.

 

Pulling, as opposed to fetching, gets the latest code changes for the branch that you have checked out. It is useful if there are code changes in an existing branch that aren't in your local repo (for example, changes in the main branch after a feature is completed).

 

Once you are on the feature branch, do whatever code changes you need to do. You can commit periodically, and push to the remote branch, as you think appropriate. When you think the work is done, then do a final commit and push.

 

To get the code from the feature branch into main, you should issue a pull request. In GitLab, these are called merge requests, and are basically the same thing. (GitLab also allows you to mark a merge request as "Draft," signifying that it isn't ready yet, probably because their default workflow is to create the merge request as soon as you start work.)

 

These were designed for teams where one developer would make a change, and before that change makes it into production, all the other developers can take a look at the code. They could either approve it, or add comments that need to be addressed. In rare situations, they could reject the code changes altogether by closing the pull request without merging. GitLab and GitHub both have features related to this; for example, both allow only certain people (usually "maintainers") to merge to main, and they can name specific people that must approve the pull request.

 

You're probably working by yourself, so you might think you don't need a pull request. But I find them very useful. It's one final set of eyes on the code, so that I don't accidentally commit something I didn't mean to, or leave in a TODO comment that should be removed, or whatever.

 

Once approved, the branch can be merged by clicking a button on the pull request.

 

You can also merge branches manually. In fact, you can even do away with feature branches altogether, and do all your work on the main branch. But I do not recommend this. You will end up making a mistake, and without feature branches and pull requests, that mistake will make it to "production" without getting caught.

 

More complicated teams or software release schedules might require a more complicated Git branching model. For example, we use Gitflow at work. But that's almost certainly overkill for what most mod authors are doing.

 

Further Reading

 

These readings are online, and all are free.

 

• Pro Git by Scott Chacon and Ben Straub
• The beginner’s guide to Git & GitHub by Thanoshan MV at Free Code Camp
• An Intro to Git and GitHub for Beginners at HubSpot
• git - the simple guide by Roger Dudler

Edited March 26, 2023 by khzmusik
Added link to Markdown cheat sheet (see edit history)

[work22]

thank you!  this is very helpful

[doughphunghus]

Very nice write up!

 

For anyone who thinks this may be a bit intimidating: it kinda is when you start. Just make a test "project" with some text files and a few directories/folders and mess around, change the files, commit them, push them, etc. see how it works.  it may seem like "too many steps, why cant i just copy to Dropbox or copy my entire folder and rename it "v2" for the next version?" and those methods do work, just git /hub/lab usage is basically "because this is the next step up in development, especially from a mass copy". Once you het used to a few basic git commands or a workflow, you will likely never want to copy a project to a new folder to version it again.
 

the links above are great, and theres a lot of "git documentation" on the web thats.... a lot harder to read, especially for a beginner.

 

Also: If you're a "tools minimalist" that last link, "the simple guide" which is basically "how to use git using the command line" is essentially how i learned to use git, and for "a single developer doing personal projects" is all you need to get started even without using github/lab, just using local "on your pc" repositories. A big benefit of using github/lab for personal projects (other than sharing your project) is you have a nice easy "cloud backup" of it, accessible from anywhere.
 

Another possible resource? if you do not want to use gitlab/github at first and want to play around with a "personal github/lab like environment" on your PC, try Gitea (the one you install, not their "try gitea!" Online sandbox). I use it for personal/test/junk projects that i don't want to share on github/lab yet. Gitea is free. Its just not as feature rich as the others.

Edited March 26, 2023 by doughphunghus (see edit history)

[khzmusik]

2 hours ago, doughphunghus said:

Another possible resource? if you do not want to use gitlab/github at first and want to play around with a "personal github/lab like environment" on your PC, try Gitea (the one you install, not their "try gitea!" Online sandbox). I use it for personal/test/junk projects that i don't want to share on github/lab yet. Gitea is free. Its just not as feature rich as the others.

 

Thanks for that link. I had never heard of Gitea before.

 

It's a shame that it's limited to being a self-hosted repo, and isn't a hosted repo service like GitLab or GitHub. (The "Try Gitea" site is kind of like this, but it's designed as a test/demo site so nothing is permanent.) So it's basically for people who want to roll their own GitHub.

 

There is a hosted Gitea service called Gitea Hosting, but it's not free.

 

Still, it looks like it has mostly the same stuff as GitHub and GitLab, especially the stuff mod authors would use. So it does seem like it is a good way to get your feet wet with learning stuff - provided you don't mind the fact that you can't share anything with the wider world.

[haama]

Next step: githooks or build tools? I'd wager some xml validation would prevent a few fist through monitor problems.