1 Foreword
The following tutorial have been successfully reproduced on macOS 12 and 13 (Monterey and Ventura), and Pop! OS 22.04 (Ubuntu-based).
Please see the requirements below (Section 1.1) and the useful links (Section 5) for external resources or tutorials.
When using text in CAPITAL_LETTERS inside a code block/line, it usually means that you need to replace those with your own variables. For example, for a code like mkdir YOUR_FOLDER_NAME, if you want your folder to be called kart_example, you will need to write mkdir kart_example.
During the Getting Started (Section 2) and Getting Started with Kart (Section 3) sections, the examples will provide mostly terminal commands, since it will be impossible to provide instructions for all the existing graphical file managers (e.g. Finder, Windows Explorer, Dolphin, Nautilus, etc.). You can use a graphical file manager, of course, the textual documentation should be clear enough for that workflow too. Kart commands (identified by the kart prefix) should of course be run in the terminal.
In the Working with Kart (Section 3.5) section, we will instead use the graphical plugin for QGIS but, when possible, corresponding terminal commands will be given. If you are unsure about some commands, the kart command reference is an excellent documentation, otherwise man pages are available for individual commands using kart COMMAND --help, e.g. kart status --help.
While Kart can also be used to version raster data, this tutorial will only cover the vector dataset usage.
We use Github here as our git forge and we will usually mention it throughout the text. However, it is completely possible to reproduce this tutorial on other forges such as Gitlab or Codeberg. When possible we will mention guides for those forges as well.
While this tutorial is focused on QGIS, editing with ArcGIS/GDAL, etc. is supported by kart, although you will not have access to the graphical plugin.
We noticed some difficulties in getting kart to work properly on older versions of macOS, likely because of outdated versions of git, xcode, xcode command line tools, etc. For a smoother experience we recommend using AT LEAST macOS 12 (Monterey) and XCode 13.
On Linux, if you are running the Flatpak version of QGIS, the kart plugin (see below) might have problems finding the kart binary. Switching to a different version of QGIS (e.g. following these instructions) seems to solve the issue, otherwise you might need to see if the issue can be solved using some apps such as Flatseal.
While this tutorial is presented here in a stable state, it is by no means finalized, we will keep adding and correcting things based on our continuing experience with kart and the evolution of kart itself.
1.1 Requirements
To avoid an endless guide, the instructions and tutorial below assumes the following:
You are familiar with git and you have git installed on your machine (there are many guides available online, for example de1v.to, or Atlassian, or this other one). If you don’t have git installed, please follow the instruction on the Git website.
You have at least one authenticated profile for Github (see https://docs.github.com/en/get-started/getting-started-with-git) or for your git forge of choice (e.g. ).
You are are familiar or at least not afraid of using a terminal application (e.g. Terminal.app, Powershell, Konsole, Gnome Terminal, Kitty, Wezterm, Alacritty).
You are familiar with QGIS and its interface, and you have QGIS 3.16 or higher installed on your machine (otherwise please install it from the QGIS website).
2 Getting Started
2.1 Install Kart
Go to https://kartproject.org/ and in the dropdown menu under “Download for your OS” select the appropriate package for your OS. Then, open the file and follow the instruction to install the software.
Alternatively, platform specific instructions follows below. These instructions are taken from the kart official documentation, please always double check the source website before running terminal commands:
Windows
Download the .msi installer from the release page.
macOS
Download the .pkg installer from the release page.
Or use Homebrew to install: brew install koordinates/kart/kart --cask
Linux
For Debian/Ubuntu-based distributions, download the .deb package from the release page and install via dpkg -i kart_*.deb
For RPM-based distributions, download the .rpm package from the release page and install via rpm -i kart-*.rpm
Testing install
Test your install by running kart --version in your terminal. If you want, you can follow the Quick Start guide in kart documentation and running kart status at one point.
3 Getting Started with Kart
3.1 Set up
If you have never used git you should run the following commands (replacing the quoted text with your actual information):
kart config --global user.email "you@example.com"kart config --global user.name "Your Name"To understand what the above do, you can take a look at the Getting Started with Git guide from the Github Docs, specifically the parts about Setting up your username and your commit email address.
If you have already used git, chances are you have the above already set-up, and you can continue on.
Create an empty folder where your kart projects will live and move into it:
mkdir kart-workflow && cd kart-workflowAs with git, there are different ways in which to get started with kart. Here we will take a look at the most likely (in our opinion) cases:
We have already existing data and we want to start versioning from scratch.
We have versioned data already existing on a remote repository and we want to import those data on our machine.
3.2 A word about HTTPS and SSH
HTTPS connection to github and usually other forges requires additional steps to avoid having git prompting you with a username and password request every time you interact with the server. This is usually solved by setting-up a Personal Access Token or by caching your credentials in Git. Note that failing to cache or save the credentials on your machine in some ways may result in an error when pushing to a remote repository from the QGIS interface.
For a more seamsless experience (in our opinion), you can follow this guide to setup an SSH key pair for Github, but guides are also available for Gitlab and Codeberg.
3.2.1 Generating SSH keys (Optional)
The guides above for creating and adding an SSH key should be exhaustive enough, but we can repeat some steps here. These steps below are general but have been reproduced on MacOS only.
If you have no previous experience, follow the guide above, these steps are not meant to replace it.
Generate a new ssh key
ssh-keygen -t ed25519 -C "your_email@example.com"(the mail should be the same used for your github account) and follow the on-screen instructions for inputting a passphrase.Ensure the ssh-agent is running
eval "$(ssh-agent -s)"If you are on macOS 10.12.2 or later, manually edit the ~/.ssh/config file as explained in the same guide.
Add the ssh key to the ssh-agent
ssh-add ~/.ssh/id_ed25519(if you are on macOS, you can store the ssh passphrase in your keychain by runningssh-add --apple-use-keychain ~/.ssh/id_ed25519)Add the key to your github account following the respective guide.
3.3 Example 1: use an empty Kart repository
3.3.1 Download data
Download the sample data. You can download this file anywhere on your machine.
You can get the data from the same folder where this tutorial lives, by going to this link.
Alternatively, you can use the following CLI tools by pasting the commands in your terminal (Aria2 is likely not installed in your system).
Wget
wget https://github.com/UnitoAssyrianGovernance/kart4arch/raw/main/static/kart-workflow-example.gpkgAria2
aria2c https://github.com/UnitoAssyrianGovernance/kart4arch/raw/main/static/kart-workflow-example.gpkgCurl
curl -O https://github.com/UnitoAssyrianGovernance/kart4arch/raw/main/static/kart-workflow-example.gpkg3.3.2 Import data in Kart
Create an empty folder on your machine and move inside that folder (you can use “kart-tutorial” or replace it with your own folder name). Be sure to copy the absolute path of where you saved your geopackage before (the .gpkg extension MUST be included).
kart init kart-tutorial --import GPKG:path/to/your/geopackage.gpkgFor example, if you created the kart-workflow folder in the above steps, and you are on MacOS, your command will look like (run inside the kart-workflow folder):
kart init kart-tutorial --import GPKG:kart-workflow-example.gpkgThe terminal should output something like this (cut for convenience, your output will be longer)
Initialized empty Git repository in path/to/kart-workflow/kart-tutorial/.kart/
Starting git-fast-import...
Importing 4 features from kart-workflow-example.gpkg:archaeo_locationquality to archaeo_locationquality/ ...
Added 4 Features to index in 0.0s
Overall rate: 1390 features/s)
Closed in 0s
Importing 106 features from kart-workflow-example.gpkg:archaeo_periodisation to archaeo_periodisation/ ...
Added 106 Features to index in 0.0s
Overall rate: 21077 features/s)
Closed in 0s
[...]
Creating GPKG working copy at kart-tutorial.gpkg ...
Writing features for dataset 1 of 12: archaeo_status
archaeo_status: 100%|█████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 11679.23F/s]
[...]3.4 Example 2: Clone an existing Kart repository
Do not use standard git commands (git clone) or Github CLI (gh repo clone) or any other similar commands or CLI tools to clone the repository, as this will not work as intended. Use only the kart clone command.
3.4.1 Clone the repository
We provide a versioned github repo ready to clone.
3.4.1.1 Cloning with HTTPS
kart clone https://github.com/UnitoAssyrianGovernance/kart-tutorial.gitYou can use your own folder name instead of the default “kart-tutorial” by adding a name at the end of the command, e.g. kart clone https://github.com/UnitoAssyrianGovernance/kart-tutorial.git kartproject
3.4.1.2 Cloning with SSH
kart clone git@github.com:UnitoAssyrianGovernance/kart-tutorial.git3.5 Working with Kart
Run kart status , if you see something like
On branch main
Nothing to commit, working copy cleanyou are good to go, the repository has been correctly initialized or cloned. If not, please repeat the above steps carefully.
3.5.1 Install the Kart QGIS Plugin
Kart comes with a useful plugin for QGIS. The source code for this plugin is available on Github. To install the latest version of the plugin, use the QGIS Plugin Manager and search for the Kart plugin. Alternatively, you can get the latest version from the release page, then open the QGIS Plugin manager and install the downloaded zip file.
As long as you have kart installed, the plugin can also take care of creating a new repo, cloning an existing one (i.e. the steps we did above), etc. The plugin has an excellent documentation with substantial graphical examples, so we link to the official docs instead of rewriting our own.
3.5.2 Set up the plugin
Not much setup is needed, the plugin should identify the kart binary automatically. If not, go to Plugins > Kart > Settings menu to open the settings dialog. In the Kart executable folder field, enter the path to your Kart executable. If the Kart folder is not configured, the Kart plugin will use the default install locations, as follows:
Windows:
%PROGRAMFILES%\Kart\OSX:
/Applications/Kart.app/Contents/MacOS/Linux:
/opt/kart/
We also recommend toggling off the option “Commit automatically after closing editing”. Depending on your editing workflow, you might end up with hundreds of commits adding one point each without realizing. Of course, this means that you will have to remember to commit your edits manually.
When changes have been done, close the plugin dialog box with “Ok”.
3.5.3 Connect to a local repository
If a side panel has not already appeared, click on Plugins > Kart > Repositories to open the kart panel. The panel will look empty, but we will populate it with the repository we created before.
In order to do so, right click on “Repositories”. You will see a list of options: the last two (Create new repository and Clone repository) are graphical replacement to the steps we did before (Section 3.3 and Section 3.4).
Click on Add existing repository… and navigate to the kart-workflow folder we created before, then click on “Open”.
You will see the repository path now listed inside the panel under the name path/kart-workflow/kart-tutorial [main]. The [main] bit indicates the branch on which we are working (just as the git branches, more on this on the kart documentation and below in Section 3.6.1).
3.5.4 Load data inside the project
What you connected to is what kart calls working copy, i.e. a file living on your computer that you can interact with through GIS. Kart uses different working copy types, in our case the Geopackage working copy.
To add the layers to our QGIS project, expand the repository tree by clicking on the arrow button on the left in the kart panel. Right-click on the archaeo_sites layer and click on Add to QGIS Project. A point layer should appear on the map and the CRS should change to EPSG:32636. Add also all the other layers (you should have 10 tabular layer and another 1 geometry layer. Add all of them except the layer_styles layer. These table layer are background data useful for populating records in the main layer (archaeo_sites) through QGIS value relation widgets.
3.5.5 Add new geometry to existing layers
You can now proceed to add new data as you would do normally in QGIS. Toggle the editing for the archaeo_sites layer and click on the Add Point Feature button. Click anywhere and add a new point. Fill the attribute table however you like. The tab structure you will see in the new feature prompt is thank to QGIS attribute form feature, and because the style of the layer is saved inside the geopackage, and versioned in kart as well (this was the layer_styles layer present in the repository listing in the kart panel.
Save the edits by either closing the editing tool or clicking on the Save Layer Edits button.
3.5.6 Edit an existing geometry
Let’s edit an existing point now, for example by changing its position and modifying some attributes. Use the identify features tool to quickly access the attribute table of a point of your choice, and right click on the identify panel to bring up a contextual menu. From this menu select Edit Feature Form. Edit the attribute table, e.g. changing the site type or the periodization, and saving your edits.
Now let’s also change the location of one of the points. It can either be the same one or another, but for an easier visualization later, it might be best to edit points near the one we added before. Use the move feature tool to do it and move a point a bit further from its actual location, saving the edits after.
3.5.7 Inspect working copy changes
One of the interesting aspects of Kart is the possibility of inspecting changes before commits. The QGIS plugin in particular offers a way of graphically visualize changes, both on a 2D map or in a table layout. To do so, return to the kart repository panel and right-click on the repository name, then click on the Show working copy changes… button (you can also right-click on a single layer to inspect changes only for that one).
The visual diff viewer panel will open. On the left we can see (in a tree-like structure) the layer that was edited (archaeo_sites, there would be more if we had edited more than one layer), the type of change (Added, Modified)m and the primary key of the added/modified entry. On the right, there are two types of diffs available, the Attributes table, and the Geometries tab. For the Added feature, the attribute table will be all green (as these are all new entries), and the geometry tab will not be much useful. However, if we switch to the Modified features, we can see that kart highlights the rows that have been modified in a convenient table with Old/New value pairs.
If we switch to the Geometries tab instead, and select the point that we have moved from the list on the left, then we will see how Kart highlights with transparency the previous location of the geometry/nodes and in a darker color the new location.
On the top of the Geometries tab, there are also two dropdown dialogs, one (Additional Layers) related to the layer we want to show in the Diff Viewer in addition to the modified geometries (options are: Project Layer, OSM Basemap, No additional layers), and another (Diff type) related to how we want to visualize the Diff (options are: Transparency, Swipe, Per-vertex diff). We found both transparency and per-vertex diff to be particularly useful in case of polygon geometries.
Note that the kart CLI also offers a way to inspect changes to the working copy, through the kart status and kart diff commands. For example, for the edits that we just made the output would be something similar:
3.5.8 Commit working copy changes
Now that we inspected the changes made during editing, we can commit them, just like git. In QGIS, go back to the kart repositories panel and right-click on our repository, then select Commit working copy changes… (you can also discard them if you noticed some errors). A dialog will pop-up prompting you to enter a commit message. Write a descriptive commit message and then click on “Ok”. A green message at the top should inform you that the changes have been committed successfully (Changes correctly commited).
You can achieve the same by using the CLI through the command kart commit (relevant documentation).
kart commit -m 'YOUR COMMIT MESSAGE'
[main be39e89] your commit message
archaeo_sites:
feature:
1 inserts
2 updates
Date: Wed Apr 10 16:40:04 2024 +0200Add a couple more points or modify existing ones, and make at least other two-three commits, by repeating the same steps above. You can also edit different layers, such has the table layers, or the survey_areas layers.
3.5.9 Inspect the commit tree
Now that we have more than just a couple of commits, we can use the visual log to inspect commits to our main branch. To access the log, right-click on our repository in the kart panel and click on Show log…
The log panel will appear (if you have worked with VSCode or any other editor providing a git-tree functionality, this will look very familiar). For now it will look rather simple as we are directly committing on a single branch and we have few commits. The corresponding command for the CLI is kart log (note: if you are using pagers for git, these will not work with kart of course)
3.5.10 Inspect changes introduced in a specific commit (and other options)
In the QGIS plugin, we can inspect, using the Visual Diff tool, changes introduced by each commit we made. We can bring up a contextual menu by right-clicking on a commit, and a number of options will show. We will not inspect all the options visible in the image below, but they should be self-explanatory, especially if you are familiar with git.
3.6 Collaborating with Kart
While kart is perfectly usable on a local machine only, the goal here is to collaborate with other people. In our workflow, every person can make edits to the layers that need to be worked on. Git is, in general, a very good collaboration tool, and thus kart potentially offers similar advantages. We will explore a very simple worklow: we will simulate one colleague creating a secondary branch from main, adds the edits needed, and then merge back into main, where another colleague have already added geometries to the same layers. Since we are using a geopackage, we will have to deal with conflicts, so we will explore that too.
Here we are using the main branch as an example to avoid too much redundancy, but this is usually not a good practice, as usually is best to have everyone working on secondary branches. As suggested in this comment, it is recommended that one user will do the merge, push the changes, and any other user will use the merged branch as starting point for any new edits in order to avoid further conflicts. This means that after the merge into main is completed and the updated main has been pushed to remote, new edits should be done on new branches created from the updated main branch (after updating the local main branch with kart pull).
3.6.1 Branching
Branching is a common way to implement features and changes isolated from the main working tree and from other people working on the same dataset. The main branch is the default branch upon repository creation.
In order to create a new branch, you can use the graphical plugin by right-clicking on the repository name and then click on Switch branch.. On the the new dialog click on Create New. In the new dialog pop-up select the name of the new branch (e.g. dev) and then click on OK.
You can reproduce the exact same behavior with less clicks using the following command in your terminal when in the kart repo: kart checkout -b dev.
Your worktree will switch to the new branch automatically. If you need to switch back to main again, from the plugin panel use the same Switch branch.. option and select main from the dropdown menu. From the CLI you can use kart switch main or kart checkout main.
If you use the CLI, the changes should be immediately visible in the plugin panel in QGIS, if that’s not the case, you can refresh using F5 or by right-clicking on the repo and click Refresh.
You can also see the two branches (at the same point in the kart tree) using the Show log.. button from the kart repo panel.
For more general information about branches: see the Kart QGIS Plugin docs and the Kart docs and command reference.
3.6.1.1 Add features on the new branch
Let’s now add 3 new points to the archaeo_sites layer while on the dev branch. The procedure is the same as the one highlighted in the previous sections (Section 3.5.5). Let’s also commit this changes and inspect the worktree to see that the dev branch is now ahead of main by 1 commit.
3.6.1.2 Add features to the main branch
To simulate a collaborative situation, switch back to the main branch (using the plugin interface of kart switch) and add 2 points to the same archaeo_sites layer and commit our changes.
3.6.2 Merging
We want our changes on the dev branch to be incorporated on the main branch. To do this we can use the graphical plugin to start a merge. While on the main branch, right-click on the repository and select Merge into current branch.. In the new panel you can provide a merge message and select other options, but for now let’s just make sure to select dev from the dropdown corresponding to Branch.
You can do the same in the terminal by using kart merge dev, make sure you are on the main branch.
If there are no conflicts, kart will merge using the fast-forward option(thus we will not find a “merge” commit in your commit history) and our branch main will be updated with the edits from the other branch. However, this is not the case now.
If you did like we did above, you will likely see a message telling you that there were conflicts during the merge, and that these conflicts need to be resolved before continuing with the merge.
If you use the command line, you will receive a similar message:
kart merge dev
Merging branch "dev" into main
Conflicts found:
archaeo_sites:
archaeo_sites:feature: 2 conflicts
Repository is now in "merging" state.
View conflicts with `kart conflicts` and resolve them with `kart resolve`.
Once no conflicts remain, complete this merge with `kart merge --continue`.
Or use `kart merge --abort` to return to the previous state.3.6.3 Dealing with conflicts
When conflicts are encountered, the repository enters into a merging state. In this state no changes can be made to the dataset until the conflicts are resolved. Three options are available to us:
- Solve the conflicts
- Continue merge (this will not work until conflicts are solved)
- Abort the merge (return to the previous state before the merge command)
We want to solve our conflicts. When working with geopackages, the most likely conflict comes from conflicting primary keys (in the fid field).
3.6.3.1 Viewing conflicts
To solve our conflicts we need first to see what is actually conflicting. With the QGIS plugin, right click on the repository and select Resolve conflicts…
Currently we are unable to access the graphical tool due to a possible bug in kart, thus we will continue with the CLI only. We will update this tutorial with the relevant info once we get past this issue. If you don’t want to use the CLI, you can try following the kart plugin docs about solving conflicts and then continue on to the next section.
To see the conflicts on the CLI, use kart conflicts to generate a list of conflicting features. While not immediately noticeable, you will see that each feature has a :ours or :theirs after its name, indicating at which branch they pertain, ours meaning the branch you are currently on, and theirs meaning the branch you are mergin from. In our case, ours is main and theirs is yourbranchname (or how your feature branch is called). This view does not really tell you which fields are conflicting, but if you look at the fid you will see that they are duplicated.
3.6.3.2 Solving conflicts
We need now to resolve the conflicts. In git you will be thrown in the text editor, but kart lets us choose the way in which to resolve it. However, the way indicated on the official documentation (kart revolve --with=(ancestors|ours|theirs|delete)) does not include, at the time of writing (2023-12-06) the most suitable option, i.e. automatically renumbering the features. This is instead only documented on a pull request, so we report it here too (note that however the option is visible in the man page using kart resolve --help).
Since Kart version 0.12.3, there is a kart revolve --renumber=(ours|theirs) command that is intended to automatically resolve primary key conflicts during merge operations. The --renumber flag takes one of the two options: ours or theirs, with ours meaning the branch we are currently on, and theirs meaning the branch we are merging from.
Let’s use kart resolve --renumber=theirs or ours by typing the command in our terminal.
We suggest using theirs as to preserve the order of features originally present in the main branch, but feel free to use ours if you prefer (see below).
If there is only a partial overlapping of primary keys, kart will merge those keys that can be merged, while the other features will wait for the conflict resolution to be renumbered. This means that if you have some keys in the yourbranchname branch that do not overlap with primary keys in your main branch, and you use theirs as an option for renumbering, these primary keys will be added first after the keys already present in main (as mentioned here). This will break the sequential order of features from the secondary branch, however, it is more of a visual issue than a real one, unless you are relying on the order of those primary keys for other operations.
After running the command, you should see something like this:
kart resolve --renumber=theirs
Resolved 2 conflicts. 0 conflicts to go.
Use `kart merge --continue` to complete the mergeAs adivides by the CLI, let’s run kart merge --continue to finish the merge (this can be done also in the graphical plugin).
3.6.3.2.1 A note about terminal text editors
By default, you will be prompted to insert a commit message inside a terminal text editor. Which text editor this will be is dependent on your machine, OS, and configuration. If you have set the $VISUAL or $EDITOR variables in your shell, then you will know what will be opened. If not, you will either be put inside nano/emacs or vi/vim.
