1 version=pmwiki-2.3.20 ordered=1 urlencoded=1
2 agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
11 text=! Summary%0a%0aThis document describes a way to configure hosting git/got repos for users. The goal is to provide shared hosting of repos, from multiple users, without needing to grant shared group privileges or even accounts on the public server. This maximises the isolation, for security reasons, between users and the public facing server. Users will store their repos in their home directory (~/public/gitgot/) and push/pull changes to it locally or via ssh as normal (details below).%0a%0aTwo scripts will be used to accomplish this; gitgot-staging.pl will run on the '''User Host''', and gitgot-deploy.pl will run on the '''Public Host'''. They will need to run as root via a cron job at regular intervals (5 - 15 min). The gitgot-staging.pl script will rsync repos from each user's home directory to the staging area on the '''User Host'''. It'll then rsync the '''User Host''' staging area to the '''Public Host's''' staging area. The gitgot-deploy.pl script will rsync the files from the staging area to the web server's chroot for deployment.%0a%0aThe staging area is needed for security reasons. The script needs access to rsync the files between hosts without a password which would be a security concern if it can directly modify other areas on the '''Public Host'''. It's on both the the User and '''Public Host''' to allow the same scripts to be used even for user accounts hosted on the '''Public Host'''. The scripts will need an account (''gitgot'') on each machine and have an ssh key allowing the '''User Host''' to connect to the '''Public Host''' without a password. The staging area on both hosts will be owned by the ''gitgot'' user. The repos will be owned by the web server user once deployed.%0a%0aThe repo will be staged in /var/gitgot/user/repo directory structure. %0a%0a! 1. Setup the Hosts%0a%0aThis configuration assumes two hosts, but can also be run on a single or multiple hosts. The host where the user accounts and home directories are will be known as the '''User Host''', and the host where the web server runs will be known as the '''Public Host''' in this configuration. You can have multiple '''User Hosts''' that all use the same '''Public Host''', and the '''Public Host''' can also have user accounts on it. Advanced configurations could also have multiple public hosts with a load balancing configuration, but that is beyond the scope of this document.%0a%0a!! 1.1 User Host%0a%0aThe configuration of the '''User Host''' consists of creating the ~/public/gitgot directory where users will put the repos they wish to publish. The gitgot-staging.pl script can be configured to create this folder for you if it is missing for existing users or you can use other means to setup the directory. %0a%0a!!! 1.1.1 Modify /etc/skel%0aFor new accounts, setting up the ~/public/gitgot directory can be automated by adding it to /etc/skel directory like this:%0a%0a[@%0adoas mkdir -p /etc/skel/public/gitgot%0adoas chown root.wheel /etc/skel/public/gitgot%0adoas chmod 755 /etc/skel/public/gitgot%0a@]%0a%0a!!! 1.1.2 Create the gitgot user%0aThe ''gitgot-staging.pl'' script will move repos from the users directory to the staging area and change ownership to the gitgot user. It skips empty directories. Both the '''user Host''' and '''Public Host''' will need a gitgot user created like this:%0a%0a[@%0adoas useradd -m gitgot%0adoas su - gitgot%0a@]%0a%0a!!! 1.1.3 Generate an SSH Key%0aThe '''User Host''' will need an ssh key so it can send the changes to the '''Public Host'''.%0a[@%0assh-keygen -t ed25519%0aexit%0a@]%0a%0aUse the default path for the key file and leave the password empty. The output will look something like this:%0a%0a[@%0aGenerating public/private ed25519 key pair.%0aEnter file in which to save the key (/home/gitgot/.ssh/id_ed25519): %0aEnter passphrase (empty for no passphrase): %0aEnter same passphrase again: %0aYour identification has been saved in /home/gitgot/.ssh/id_ed25519%0aYour public key has been saved in /home/gitgot/.ssh/id_ed25519.pub%0aThe key fingerprint is:%0aSHA256:lyrfN6pDSRuVI112nfZiMh02aOoHxvgq5Y5jyY2QKpI gitgot@planetofnix.com%0aThe key's randomart image is:%0a+--[ED25519 256]--+%0a| . oo....|%0a| . =.o.+o.|%0a| = + o.o.|%0a| + =.o + .|%0a| . .SBo. + . |%0a| o =oo . |%0a| . . o.B.. . |%0a|E . B+=. o |%0a|.. ..++ooo . |%0a+----[SHA256]-----+%0a@]%0a%0aYou'll need to do the same on the '''Public Host''' and copy the public key over. Run the following, but change %3cadminUser> and %3cpublicHost> as needed. Doas is broken and wont let you cat >> so a few extra steps to do this as a copy/paste...%0a%0a%25note%25'''NOTE:''' The %3cadminUser> has to exist on the '''Public Host''' and needs to have write permission on /home/gitgot/.ssh/authorized_keys for this to work. You can use other methods to get the gitgot public key onto the '''Public Host'''. %0a%0a!!! 1.1.4 Create gitgot User on Public Host%0a[@%0assh_as='%3cadminUser>@%3cpublicHost>'%0assh $ssh_as doas useradd -m gitgot%0adoas cat /home/gitgot/.ssh/id_ed25519.pub | ssh $ssh_as 'cat >>~/doasbroken.tmp'%0a@]%0a%0a!!! 1.1.5 Copy User Host Public Key to Public Host%0a[@%0a# Log in to the Public Host%0assh $ssh_as%0a# Copy paste these separately after logging in%0adoas su %0acat doasbroken.tmp >>/home/gitgot/.ssh/authorized_keys%0arm doasbroken.tmp%0aexit%0aexit%0a@]%0a%0a!!! 1.1.6 The gitgot-staging.pl script%0a%0aWe'll put the script for copying the files in /home/gitgot/bin/gitgot-staging.pl This script is also available via a [[https://got.ircnow.org/cgi-bin/gotweb/gotweb?path=izzyb-gitgot.git&action=summary | git repo]].%0a%0a[@%0amkdir /home/gitgot/bin%0avim /home/gitgot/bin/gitgot-staging.pl%0a@]%0a%0a%25code%25[@%0a#!/usr/bin/perl%0ause strict;%0ause warnings;%0amy $config={%0a remoteHost=>'got.ircnow.org',%0a remoteUser=>'gitgot',%0a remoteStage=>'/var/gitgot',%0a repoDir=>'public/gitgot',%0a createRepoDir=>1,%0a stageDir=>'/var/gitgot',%0a gitgotUser=>'gitgot',%0a key_path=>'/home/gitgot/.ssh/id_ed25519',%0a rsync_cmd=>'openrsync',%0a};%0a# verify we have a local staging area%0aif (not -d $config->{stageDir}) {%0a my (undef,undef,$uid,$gid) = getpwnam($config->{gitgotUser})%0a or die "User " . $config->{gitgotUser} . " doesn't exist";%0a mkdir $config->{stageDir}, 0750;%0a chown $uid,$gid,$config->{stageDir};%0a}%0aopendir (my $DH, '/home/') || die "Can't opendir /home : $!";%0awhile (my $user = readdir($DH)) {%0a if (-d "/home/$user/" . $config->{repoDir}) {%0a # Skip if no repos%0a opendir (my $RDH, "/home/$user/" . $config->{repoDir})%0a or die "Can't opendir /home/$user/ : $!";%0a my @repos = readdir($RDH);%0a next unless (scalar @repos > 2);%0a%0a # Make sure user has a staging directory%0a if(not -d $config->{stageDir} . "/$user") {%0a mkdir ($config->{stageDir} . "/$user");%0a }%0a # rsync files to local staging area%0a my $src=qq(/home/$user/$config->{repoDir}/*);%0a my $dst=qq($config->{stageDir}/$user/);%0a my $rv=system(qq{%0a openrsync --rsync-path=openrsync -a --delete $src $dst%0a });%0a warn "rsync failed $user\n$!\n" if ($rv != 0);%0a } elsif ($config->{createRepoDir}) { # create missing repo dirs%0a # skip dirs without users.%0a my (undef,undef,$uid,$gid) = getpwnam($user) or next;%0a my $repoPath="/home/$user";%0a for my $dir (split('/', $config->{repoDir})) {%0a $repoPath.="/$dir";%0a unless (-e $repoPath) {%0a mkdir $repoPath, 0755;%0a chown $uid,$gid,$repoPath;%0a }%0a }%0a }%0a}%0a# rsync to remote staging area if we have a remote host%0aif (defined $config->{remoteHost}) {%0a # Only sync if we have repos to deploy%0a opendir (my $SDH, $config->{stageDir})%0a or die "Can't opendir ". $config->{stageDir} . " : $!";%0a my @repos = readdir($SDH);%0a%0a if(scalar @repos > 2) {%0a my $src=$config->{stageDir} . "/*";%0a my $dst=$config->{gitgotUser}. "@" . $config->{remoteHost}%0a . ":" . $config->{remoteStage} . "/";%0a%0a my $rv=system(qq{%0a openrsync --rsync-path=openrsync -ae %0a "ssh -i $config->{key_path}" --delete $src $dst%0a });%0a }%0a}%0a@]%0a%0a!! 1.2 Public Host%0a%0aThe '''Public Host''' is where the web server runs. It will run the ''gitgot-deploy.pl'' script from a cron job as root to copy the files from the staging area to the proper location for the web server. %0a%0a'''NOTE:''' This part of the documentation is a bit in flux because the current server doesn't have a way to publish repos in user folders. This is a project for another day, hopefully not too far into the future. As a work around, the users name is added as a prefix to the repo name to avoid one user's repo clobbering another users repo with the same name. Hopefully this will be fixed soon.%0a%0a!!! 1.2.1 The gitgot-deploy.pl Script%0a%0a%25code%25[@%0a#!/usr/bin/perl%0ause strict;%0ause warnings;%0amy $config={%0a stageDir=>'/var/gitgot',%0a gitgotUser=>'gitgot',%0a gotwebDir=>'/var/www/got/public',%0a};%0a# Sync changes from stageDir to the webdir%0aopendir (my $DH, $config->{stageDir})%0a or die "Can't opendir " . $config->{stageDir} . " : $!";%0awhile (my $user = readdir($DH)) {%0a next if ($user =~ /^\./); # skip . files %0a if (-d $config->{stageDir} . "/$user") {%0a # Need the name of each repo so we can add username ass prefix%0a opendir (my $uDH, $config->{stageDir} . "/$user")%0a or die "Can't opendir ". $config->{stageDir} . "/$user/ : $!";%0a while (my $repo = readdir($uDH)) {%0a next if ($repo =~ /^\./); # skip . files%0a # need destination folder name to change to prefix%0a # the username to it to prevent one user from clobbering%0a # another users repos. Can't do a rename of the folder%0a # with rsync so have to create the destination folder then%0a # rsync the contents of it.%0a my @stat=stat($config->{stageDir} . "/$user/$repo");%0a my $prefixName="$user-$repo";%0a my $dst=qq($config->{gotwebDir}/$prefixName/);%0a mkdir $dst,$stat[2]; #stat[2] is mode of original dir%0a opendir (my $rDH, $config->{stageDir} . "/$user/$repo")%0a or die "Can't opendir ". $config->{stageDir} . "/$user/$repo : $!";%0a while (my $git = readdir($rDH)) {%0a next if ($git eq '.' or $git eq '..'); # skip . and ..%0a my $src=qq($config->{stageDir}/$user/$repo/$git);%0a my $rv=system(qq{%0a openrsync --rsync-path=openrsync -a --delete $src $dst%0a });%0a warn "rsync failed $prefixName\n$!\n" if ($rv != 0);%0a }%0a }%0a }%0a}%0a@]%0a%0a%0a! 2.0 Using the Repo%0a%0a%25note%25'''NOTE: ''' For most of these commands, ''git'' and ''got'' can be used interchangeably. ''Got'' is a clone of ''git'' after all.%0a%0aGit repos exist in two forms; working or bare. A bare repo contains no working tree and is generally used for published versions of the repo. The bare version is really just the contents of the .git directory in your working tree. It is usually the repo name with a .git extension by convention. So a repo named my myProgram would be published in the directory ~/public/gitgot/myProgram.git.%0a%0aYour working repo is where you make your changes. It has a hidden directory of .git which contains the bare repo contents. If you are using your shell account on the User Host machine for your working repo, you are a local user. If you are using a remote computer for your working repo then you'll use the remote User commands to access the public repo. The examples below will assume you are using a working directory of ~/myProjects/ where you'll put the working versions of your repos. %0a%0a!! 2.1 Creating a New Repo%0a%0aA new repo is created using the git init command. Its easiest if you create the bare repo first since cloning sets up the remote details for you. %0a%0a!!! 2.1.1 Local User%0a%0a%25commands%25[@%0acd ~/myProjects/%0agit init --bare ~/public/gitgot/myProgram.git%0agit clone ~/public/gitgot/myProgram.git%0agit push %0a@]%0a%0a!!! 2.1.2 Remote User%0a%25commands%25[@%0acd ~/myProjects/%0assh username@example.com git init --bare ~/public/gitgot/myProgram.git%0agit clone ssh://username@example.com/~/public/gitgot/myProgram.git%0agit push%0a@]%0a%0a!! 2.2 Pushing an Existing Repo%0a%0aIf you have an existing working repo that you want to publish you first need to create an empty bare repo then add a remote url to your working repo to push to like this:%0a%0a!!! 2.2.1 Local User%0a%25commands%25[@%0acd ~/myProjects/myProgram%0agit init --bare ~/public/gitgot/myProgram.git%0agit remote add origin ~/public/gitgot/myProgram.git%0agit push --set-upstream origin master%0a# After the first push you only need this%0agit push%0a@]%0a%0a!!! 2.2.2 Remote User%0a%25commands%25[@%0acd ~/myProjects/myProgram%0assh username@example.com git init --bare ~/public/gitgot/myProgram.git%0agit remote add origin ssh://username@example.com/~/public/gitgot/myProgram.git%0agit push --set-upstream origin master%0a# After the first push you only need this%0agit push%0a@]%0a%0a!! 2.3 Clone a Bare Repo%0a%0aAnother way to create a bare repo of an existing repo is to clone it as a --bare repo like this:%0a%0a!!! 2.3.1 Local User%0a%25commands%25[@%0acd ~/public/gitgot/%0agit clone --bare ~/myProjects/myProgram%0acd ~/myProjects/myProgram%0agit remote add origin ~/public/gitgot/myProgram.git%0agit push --set-upstream origin master%0a# After the first push you only need this%0agit push%0a@]%0a%0a!!! 2.3.2 Remote User%0aIts harder to do a clone of your working repo from remote since you probably don't have an easy way to access it from remote. We'll assume you have the ability to ssh%0a%0a%25commands%25[@%0acd ~/public/gitgot/%0agit clone --bare ssh://username@example.com/~/myProjects/myProgram%0a# On your local machine%0acd ~/myProjects/myProgram%0agit remote add origin ssh://username@UserHost.com/~/public/gitgot/myProgram.git%0agit push --set-upstream origin master%0a# After the first push you only need this%0agit push%0a@]%0a%0a%0a
13 author:1685248008=izzyb
14 diff:1685248008:1685246262:=238c238%0a%3c A new repo is created using the git init command. Its easiest if you create the bare repo first since cloning sets up the remote details for you. %0a---%0a> A new repo is created using the git init command. Its easiest if you create the bare repo first since cloning setups up the remote details for you. %0a
15 host:1685248008=75.158.138.249
16 author:1685246262=izzyb
17 diff:1685246262:1685246098:=70c70%0a%3c %25note%25'''NOTE:''' The %3cadminUser> has to exist on the '''Public Host''' and needs to have write permission on /home/gitgot/.ssh/authorized_keys for this to work. You can use other methods to get the gitgot public key onto the '''Public Host'''. %0a---%0a> %25note%25[@'''NOTE:''' The %3cadminUser> has to exist on the '''Public Host''' and needs to have write permission on /home/gitgot/.ssh/authorized_keys for this to work. You can use other methods to get the gitgot public key onto the '''Public Host'''. %0a
18 host:1685246262=75.158.138.249
19 author:1685246098=izzyb
20 csum:1685246098=cleanup
21 diff:1685246098:1684809973:=5,21c5,18%0a%3c Two scripts will be used to accomplish this; gitgot-staging.pl will run on the '''User Host''', and gitgot-deploy.pl will run on the '''Public Host'''. They will need to run as root via a cron job at regular intervals (5 - 15 min). The gitgot-staging.pl script will rsync repos from each user's home directory to the staging area on the '''User Host'''. It'll then rsync the '''User Host''' staging area to the '''Public Host's''' staging area. The gitgot-deploy.pl script will rsync the files from the staging area to the web server's chroot for deployment.%0a%3c %0a%3c The staging area is needed for security reasons. The script needs access to rsync the files between hosts without a password which would be a security concern if it can directly modify other areas on the '''Public Host'''. It's on both the the User and '''Public Host''' to allow the same scripts to be used even for user accounts hosted on the '''Public Host'''. The scripts will need an account (''gitgot'') on each machine and have an ssh key allowing the '''User Host''' to connect to the '''Public Host''' without a password. The staging area on both hosts will be owned by the ''gitgot'' user. The repos will be owned by the web server user once deployed.%0a%3c %0a%3c The repo will be staged in /var/gitgot/user/repo directory structure. %0a%3c %0a%3c ! 1. Setup the Hosts%0a%3c %0a%3c This configuration assumes two hosts, but can also be run on a single or multiple hosts. The host where the user accounts and home directories are will be known as the '''User Host''', and the host where the web server runs will be known as the '''Public Host''' in this configuration. You can have multiple '''User Hosts''' that all use the same '''Public Host''', and the '''Public Host''' can also have user accounts on it. Advanced configurations could also have multiple public hosts with a load balancing configuration, but that is beyond the scope of this document.%0a%3c %0a%3c !! 1.1 User Host%0a%3c %0a%3c The configuration of the '''User Host''' consists of creating the ~/public/gitgot directory where users will put the repos they wish to publish. The gitgot-staging.pl script can be configured to create this folder for you if it is missing for existing users or you can use other means to setup the directory. %0a%3c %0a%3c !!! 1.1.1 Modify /etc/skel%0a%3c For new accounts, setting up the ~/public/gitgot directory can be automated by adding it to /etc/skel directory like this:%0a%3c %0a---%0a> Two scripts will be used to accomplish this; gitgot-staging.pl will run on the User Host, and gitgot-deploy.pl will run on the Public Host. They will need to run as root via a cron job at regular intervals (5 - 15 min). The gitgot-staging.pl script will rsync repos from each user's home directory to the staging area on the User Host. It'll then rsync the User Host staging area to the Public Host's staging area. The gitgot-deploy.pl script will rsync the files from the staging area to the web server's chroot for deployment.%0a> %0a> The staging area is needed for security reasons. The script needs access to rsync the files between hosts without a password which would be a security concern if it can directly modify other areas on the Public Host. It's on both the the User and Public host to allow the same scripts to be used even for user accounts hosted on the Public Host. The scripts will need an account (''gitgot'') on each machine and have an ssh key allowing the User Host to connect to the Public Host without a password. The staging area on both hosts will be owned by the ''gitgot'' user. The repos will be owned by the web server user once deployed.%0a> %0a> The repo will be staged in /var/gitgot/user/repo directory structure.%0a> %0a> ! Setup the Hosts%0a> %0a> This configuration assumes two hosts, but can also be run on a single or multiple hosts. The host where the user accounts and home directories are will be known as the User Host, and the host where the web server runs will be known as the Public Host in this configuration. You can have multiple user hosts that all use the same public host, and the public host can also have user accounts on it. Advanced configurations could also have multiple public hosts with a load balancing configuration, but that is beyond the scope of this document.%0a> %0a> !! User Host%0a> %0a> The configuration of the user host consists of creating the ~/public/gitgot directory where users will put the repos they wish to publish. The gitgot-staging.pl script can be configured to create this folder for you if it is missing for existing users or you can use other means to setup the directory. For new accounts, this can be done automatically by adding it to /etc/skel directory like this:%0a> %0a28,30c25,26%0a%3c !!! 1.1.2 Create the gitgot user%0a%3c The ''gitgot-staging.pl'' script will move repos from the users directory to the staging area and change ownership to the gitgot user. It skips empty directories. Both the '''user Host''' and '''Public Host''' will need a gitgot user created like this:%0a%3c %0a---%0a> The ''gitgot-staging.pl'' script will move repos from the users directory to the staging area and change ownership to the gitgot user. It skips empty directories. Both the user Host and Public Host will need a gitgot user created like this:%0a> %0a33a30,31%0a> ssh-keygen -t ed25519%0a> exit%0a36,37c34,35%0a%3c !!! 1.1.3 Generate an SSH Key%0a%3c The '''User Host''' will need an ssh key so it can send the changes to the '''Public Host'''.%0a---%0a> Use the default path for the key file and leave the password empty. The output will look something like this:%0a> %0a39,45d36%0a%3c ssh-keygen -t ed25519%0a%3c exit%0a%3c @]%0a%3c %0a%3c Use the default path for the key file and leave the password empty. The output will look something like this:%0a%3c %0a%3c [@%0a68,72c59,60%0a%3c You'll need to do the same on the '''Public Host''' and copy the public key over. Run the following, but change %3cadminUser> and %3cpublicHost> as needed. Doas is broken and wont let you cat >> so a few extra steps to do this as a copy/paste...%0a%3c %0a%3c %25note%25[@'''NOTE:''' The %3cadminUser> has to exist on the '''Public Host''' and needs to have write permission on /home/gitgot/.ssh/authorized_keys for this to work. You can use other methods to get the gitgot public key onto the '''Public Host'''. %0a%3c %0a%3c !!! 1.1.4 Create gitgot User on Public Host%0a---%0a> You'll need to do the same on the Public Host and copy the public key over. Run the following, but change %3cadminUser> and %3cremotehost> as needed. Doas is broken and wont let you cat >> so a few extra steps to do this as a copy/paste...%0a> %0a74c62%0a%3c ssh_as='%3cadminUser>@%3cpublicHost>'%0a---%0a> ssh_as='%3cadminUser>@%3cremotehost>'%0a77,80d64%0a%3c @]%0a%3c %0a%3c !!! 1.1.5 Copy User Host Public Key to Public Host%0a%3c [@%0a85c69%0a%3c cat doasbroken.tmp >>/home/gitgot/.ssh/authorized_keys%0a---%0a> cat doasbroken.tmp >>/home/gitgot/authorized_keys%0a91,94c75,78%0a%3c !!! 1.1.6 The gitgot-staging.pl script%0a%3c %0a%3c We'll put the script for copying the files in /home/gitgot/bin/gitgot-staging.pl This script is also available via a [[https://got.ircnow.org/cgi-bin/gotweb/gotweb?path=izzyb-gitgot.git&action=summary | git repo]].%0a%3c %0a---%0a> !!! gitgot-staging.pl%0a> %0a> We'll put the script for copying the files in /home/gitgot/bin/gitgot-staging.pl This script is also available via a git repo here, but you can copy and paste it from here also.%0a> %0a175,178c159,162%0a%3c !! 1.2 Public Host%0a%3c %0a%3c The '''Public Host''' is where the web server runs. It will run the ''gitgot-deploy.pl'' script from a cron job as root to copy the files from the staging area to the proper location for the web server. %0a%3c %0a---%0a> !! Public Host%0a> %0a> The Public Host is where the web server runs. It will run the ''gitgot-deploy.pl'' script from a cron job as root to copy the files from the staging area to the proper location for the web server. %0a> %0a181,182c165,166%0a%3c !!! 1.2.1 The gitgot-deploy.pl Script%0a%3c %0a---%0a> !!!gitgot-deploy.pl%0a> %0a228,229c212,213%0a%3c ! 2.0 Using the Repo%0a%3c %0a---%0a> ! Using the Repo%0a> %0a236,237c220,221%0a%3c !! 2.1 Creating a New Repo%0a%3c %0a---%0a> !! Creating a New Repo%0a> %0a240,241c224,225%0a%3c !!! 2.1.1 Local User%0a%3c %0a---%0a> !!! Local User%0a> %0a249c233%0a%3c !!! 2.1.2 Remote User%0a---%0a> !!! Remote User%0a257,258c241,242%0a%3c !! 2.2 Pushing an Existing Repo%0a%3c %0a---%0a> !! Pushing an Existing Repo%0a> %0a261c245%0a%3c !!! 2.2.1 Local User%0a---%0a> !!! Local User%0a271c255%0a%3c !!! 2.2.2 Remote User%0a---%0a> !!! Remote User%0a281,282c265,266%0a%3c !! 2.3 Clone a Bare Repo%0a%3c %0a---%0a> !! Clone a Bare Repo%0a> %0a285c269%0a%3c !!! 2.3.1 Local User%0a---%0a> !!! Local User%0a296c280%0a%3c !!! 2.3.2 Remote User%0a---%0a> !!! Remote User%0a
22 host:1685246098=75.158.138.249
23 author:1684809973=izzyb
24 csum:1684809973=Added code for gitgot-deploy.pl
25 diff:1684809973:1684738374:=171,208c171%0a%3c my $config={%0a%3c stageDir=>'/var/gitgot',%0a%3c gitgotUser=>'gitgot',%0a%3c gotwebDir=>'/var/www/got/public',%0a%3c };%0a%3c # Sync changes from stageDir to the webdir%0a%3c opendir (my $DH, $config->{stageDir})%0a%3c or die "Can't opendir " . $config->{stageDir} . " : $!";%0a%3c while (my $user = readdir($DH)) {%0a%3c next if ($user =~ /^\./); # skip . files %0a%3c if (-d $config->{stageDir} . "/$user") {%0a%3c # Need the name of each repo so we can add username ass prefix%0a%3c opendir (my $uDH, $config->{stageDir} . "/$user")%0a%3c or die "Can't opendir ". $config->{stageDir} . "/$user/ : $!";%0a%3c while (my $repo = readdir($uDH)) {%0a%3c next if ($repo =~ /^\./); # skip . files%0a%3c # need destination folder name to change to prefix%0a%3c # the username to it to prevent one user from clobbering%0a%3c # another users repos. Can't do a rename of the folder%0a%3c # with rsync so have to create the destination folder then%0a%3c # rsync the contents of it.%0a%3c my @stat=stat($config->{stageDir} . "/$user/$repo");%0a%3c my $prefixName="$user-$repo";%0a%3c my $dst=qq($config->{gotwebDir}/$prefixName/);%0a%3c mkdir $dst,$stat[2]; #stat[2] is mode of original dir%0a%3c opendir (my $rDH, $config->{stageDir} . "/$user/$repo")%0a%3c or die "Can't opendir ". $config->{stageDir} . "/$user/$repo : $!";%0a%3c while (my $git = readdir($rDH)) {%0a%3c next if ($git eq '.' or $git eq '..'); # skip . and ..%0a%3c my $src=qq($config->{stageDir}/$user/$repo/$git);%0a%3c my $rv=system(qq{%0a%3c openrsync --rsync-path=openrsync -a --delete $src $dst%0a%3c });%0a%3c warn "rsync failed $prefixName\n$!\n" if ($rv != 0);%0a%3c }%0a%3c }%0a%3c }%0a%3c }%0a---%0a> %0a
26 host:1684809973=75.158.138.249
27 author:1684738374=izzyb
28 diff:1684738374:1684730920:=163,166d162%0a%3c '''NOTE:''' This part of the documentation is a bit in flux because the current server doesn't have a way to publish repos in user folders. This is a project for another day, hopefully not too far into the future. As a work around, the users name is added as a prefix to the repo name to avoid one user's repo clobbering another users repo with the same name. Hopefully this will be fixed soon.%0a%3c %0a%3c !!!gitgot-deploy.pl%0a%3c %0a168,170d163%0a%3c #!/usr/bin/perl%0a%3c use strict;%0a%3c use warnings;%0a
29 host:1684738374=75.158.138.249
30 author:1684730920=izzyb
31 diff:1684730920:1684730548:=174,175c174,175%0a%3c Your working repo is where you make your changes. It has a hidden directory of .git which contains the bare repo contents. If you are using your shell account on the User Host machine for your working repo, you are a local user. If you are using a remote computer for your working repo then you'll use the remote User commands to access the public repo. The examples below will assume you are using a working directory of ~/myProjects/ where you'll put the working versions of your repos. %0a%3c %0a---%0a> Your working repo is where you make your changes. It contains the bare repo contents in the .git hidden directory in each repo. You can use the shell account on the User Host machine for your working repo or a remote computer and push updates via ssh to the User Host machine for publishing. The examples below will assume you are using a working directory of ~/myProjects/ where you'll put the working versions of your repos. %0a> %0a249,250d248%0a%3c %0a%3c %0a
32 host:1684730920=75.158.138.249
33 author:1684730548=izzyb
34 diff:1684730548:1684730201:=170,171c170,171%0a%3c %25note%25'''NOTE: ''' For most of these commands, ''git'' and ''got'' can be used interchangeably. ''Got'' is a clone of ''git'' after all.%0a%3c %0a---%0a> %25note%25''NOTE: ''' For most of these commands, git and got can be used interchangeably. Got is a clone of git after all.%0a> %0a174c174%0a%3c Your working repo is where you make your changes. It contains the bare repo contents in the .git hidden directory in each repo. You can use the shell account on the User Host machine for your working repo or a remote computer and push updates via ssh to the User Host machine for publishing. The examples below will assume you are using a working directory of ~/myProjects/ where you'll put the working versions of your repos. %0a---%0a> Your working repo is where you make your changes. It contains the bare repo contents in the .git hidden directory in each repo. Your can use the shell account on the User Host machine for your working repo or a remote computer and push updates via ssh to the User Host machine for publishing. The examples below will assume you are using a working directory of ~/myProjects/ where you'll put the working versions of your repos. %0a
35 host:1684730548=75.158.138.249
36 author:1684730201=izzyb
37 diff:1684730201:1684710754:=167a168,171%0a> %0a> %0a> %0a> %0a169,181c173,177%0a%3c %0a%3c %25note%25''NOTE: ''' For most of these commands, git and got can be used interchangeably. Got is a clone of git after all.%0a%3c %0a%3c Git repos exist in two forms; working or bare. A bare repo contains no working tree and is generally used for published versions of the repo. The bare version is really just the contents of the .git directory in your working tree. It is usually the repo name with a .git extension by convention. So a repo named my myProgram would be published in the directory ~/public/gitgot/myProgram.git.%0a%3c %0a%3c Your working repo is where you make your changes. It contains the bare repo contents in the .git hidden directory in each repo. Your can use the shell account on the User Host machine for your working repo or a remote computer and push updates via ssh to the User Host machine for publishing. The examples below will assume you are using a working directory of ~/myProjects/ where you'll put the working versions of your repos. %0a%3c %0a%3c !! Creating a New Repo%0a%3c %0a%3c A new repo is created using the git init command. Its easiest if you create the bare repo first since cloning setups up the remote details for you. %0a%3c %0a%3c !!! Local User%0a%3c %0a---%0a> For most of these commands, git and got can be used interchangeably. Got is a clone of git after all. %0a> %0a> !! Creating a bare repo for publishing%0a> Git repos can exist in two forms; working or bare. You usually use git with a working tree for making changes in. You don't need or want the working tree in a version intended for sharing. Instead you create a bare version of it. The bare version is really just the contents of the .git directory in your working tree. You can create a bare version of any git repo for publishing like this:%0a> %0a183,186c179,180%0a%3c cd ~/myProjects/%0a%3c git init --bare ~/public/gitgot/myProgram.git%0a%3c git clone ~/public/gitgot/myProgram.git%0a%3c git push %0a---%0a> cd ~/public/gitgot/%0a> git clone --bare /path/to/repo%0a189c183,184%0a%3c !!! Remote User%0a---%0a> /path/to/repo can also be a remote url supported by git/got. Generally it will be a clone of a working directory you created with git init, but it can be a fresh init of a bare repo created like this:%0a> %0a191,194c186,187%0a%3c cd ~/myProjects/%0a%3c ssh username@example.com git init --bare ~/public/gitgot/myProgram.git%0a%3c git clone ssh://username@example.com/~/public/gitgot/myProgram.git%0a%3c git push%0a---%0a> cd ~/public/gitgot/%0a> git init --bare myRepo.git%0a197,201c190,192%0a%3c !! Pushing an Existing Repo%0a%3c %0a%3c If you have an existing working repo that you want to publish you first need to create an empty bare repo then add a remote url to your working repo to push to like this:%0a%3c %0a%3c !!! Local User%0a---%0a> !! Configuring your working Repo%0a> If you don't have a working tree yet, you can clone the bare repo you've already created above. Assuming you use ~/git/ as your working directory, you'd clone your bare repo like this:%0a> %0a203,208c194,195%0a%3c cd ~/myProjects/myProgram%0a%3c git init --bare ~/public/gitgot/myProgram.git%0a%3c git remote add origin ~/public/gitgot/myProgram.git%0a%3c git push --set-upstream origin master%0a%3c # After the first push you only need this%0a%3c git push%0a---%0a> cd ~/git/%0a> git clone ~/public/gitgot/myRepo.git%0a211c198,199%0a%3c !!! Remote User%0a---%0a> If you already had a working repo, and want to push changes to it to your public bare repo, you'll do something like this:%0a> %0a213,215c201,202%0a%3c cd ~/myProjects/myProgram%0a%3c ssh username@example.com git init --bare ~/public/gitgot/myProgram.git%0a%3c git remote add origin ssh://username@example.com/~/public/gitgot/myProgram.git%0a---%0a> cd ~/git/myRepo%0a> git remote add origin ~/public/gitgot/myRepo.git%0a217,218d203%0a%3c # After the first push you only need this%0a%3c git push%0a221,225c206,208%0a%3c !! Clone a Bare Repo%0a%3c %0a%3c Another way to create a bare repo of an existing repo is to clone it as a --bare repo like this:%0a%3c %0a%3c !!! Local User%0a---%0a> !! Configuring a repo on a remote machine%0a> If you have a local copy of your repo and wish to push it to your Public repo, you can do this:%0a> %0a227,230c210%0a%3c cd ~/public/gitgot/%0a%3c git clone --bare ~/myProjects/myProgram%0a%3c cd ~/myProjects/myProgram%0a%3c git remote add origin ~/public/gitgot/myProgram.git%0a---%0a> git remote add origin ssh://username@example.com:public/gitgot/myRepo.git%0a232,233d211%0a%3c # After the first push you only need this%0a%3c git push%0a236,238c214,216%0a%3c !!! Remote User%0a%3c Its harder to do a clone of your working repo from remote since you probably don't have an easy way to access it from remote. We'll assume you have the ability to ssh%0a%3c %0a---%0a> !! Publish a New Bare Repo from remote%0a> If you have a repo on your local machine that isn't already in your Public/gitgot/ on your User Host, you will need to create a bare repo on the User Host then push it like this:%0a> %0a240,244c218,219%0a%3c cd ~/public/gitgot/%0a%3c git clone --bare ssh://username@example.com/~/myProjects/myProgram%0a%3c # On your local machine%0a%3c cd ~/myProjects/myProgram%0a%3c git remote add origin ssh://username@UserHost.com/~/public/gitgot/myProgram.git%0a---%0a> ssh username@example.com git init --bare public/gitgot/myRepo.git%0a> git remote add origin ssh://username@example.com:public/gitgot/myRepo.git%0a246,247d220%0a%3c # After the first push you only need this%0a%3c git push%0a248a222,223%0a> %0a> %0a
38 host:1684730201=75.158.138.249
39 author:1684710754=izzyb
40 csum:1684710754=Almost complete, Need to create gitgot-deploy.pl script still
41 diff:1684710754:1684710438:=
42 host:1684710754=75.158.138.249
43 author:1684710438=izzyb
44 diff:1684710438:1684710404:=
45 host:1684710438=75.158.138.249
46 author:1684710404=izzyb
47 diff:1684710404:1684707260:=25,26c25,26%0a%3c The ''gitgot-staging.pl'' script will move repos from the users directory to the staging area and change ownership to the gitgot user. It skips empty directories. Both the user Host and Public Host will need a gitgot user created like this:%0a%3c %0a---%0a> The ''gitgot-staging.pl'' script will move them from the users directory to the staging area and change ownership to the gitgot user. Both the user Host and Public Host will need a gitgot user created like this:%0a> %0a161,164c161,174%0a%3c The Public Host is where the web server runs. It will run the ''gitgot-deploy.pl'' script from a cron job as root to copy the files from the staging area to the proper location for the web server. %0a%3c %0a%3c %25code%25[@%0a%3c %0a---%0a> The Public Host is where the web server runs. It can be the same machine as the User Host. The gitgot-staging.pl script will copy the repos from each users home dirs to /var/gitgot/username/. A second script called gitgot-deploy.pl will run from a cron job as root to copy the files from the staging area to the proper location for the web server. %0a> %0a> %0a> %0a> %0a> %0a> %0a> %0a> %0a> ! Using the Repo%0a> %0a> [@%0a> %25commands%25[@%0a> ssh username@got.ircnow.org git init --bare /var/git/reponame%0a167,180c177,179%0a%3c %0a%3c %0a%3c %0a%3c %0a%3c %0a%3c ! Using the Repo%0a%3c For most of these commands, git and got can be used interchangeably. Got is a clone of git after all. %0a%3c %0a%3c !! Creating a bare repo for publishing%0a%3c Git repos can exist in two forms; working or bare. You usually use git with a working tree for making changes in. You don't need or want the working tree in a version intended for sharing. Instead you create a bare version of it. The bare version is really just the contents of the .git directory in your working tree. You can create a bare version of any git repo for publishing like this:%0a%3c %0a%3c %25commands%25[@%0a%3c cd ~/public/gitgot/%0a%3c git clone --bare /path/to/repo%0a---%0a> [@%0a> git remote add origin ssh://got.ircnow.org/var/git/reponame%0a> git push --set-upstream origin master%0a182,222d180%0a%3c %0a%3c /path/to/repo can also be a remote url supported by git/got. Generally it will be a clone of a working directory you created with git init, but it can be a fresh init of a bare repo created like this:%0a%3c %0a%3c %25commands%25[@%0a%3c cd ~/public/gitgot/%0a%3c git init --bare myRepo.git%0a%3c @]%0a%3c %0a%3c !! Configuring your working Repo%0a%3c If you don't have a working tree yet, you can clone the bare repo you've already created above. Assuming you use ~/git/ as your working directory, you'd clone your bare repo like this:%0a%3c %0a%3c %25commands%25[@%0a%3c cd ~/git/%0a%3c git clone ~/public/gitgot/myRepo.git%0a%3c @]%0a%3c %0a%3c If you already had a working repo, and want to push changes to it to your public bare repo, you'll do something like this:%0a%3c %0a%3c %25commands%25[@%0a%3c cd ~/git/myRepo%0a%3c git remote add origin ~/public/gitgot/myRepo.git%0a%3c git push --set-upstream origin master%0a%3c @]%0a%3c %0a%3c !! Configuring a repo on a remote machine%0a%3c If you have a local copy of your repo and wish to push it to your Public repo, you can do this:%0a%3c %0a%3c %25commands%25[@%0a%3c git remote add origin ssh://username@example.com:public/gitgot/myRepo.git%0a%3c git push --set-upstream origin master%0a%3c @]%0a%3c %0a%3c !! Publish a New Bare Repo from remote%0a%3c If you have a repo on your local machine that isn't already in your Public/gitgot/ on your User Host, you will need to create a bare repo on the User Host then push it like this:%0a%3c %0a%3c %25commands%25[@%0a%3c ssh username@example.com git init --bare public/gitgot/myRepo.git%0a%3c git remote add origin ssh://username@example.com:public/gitgot/myRepo.git%0a%3c git push --set-upstream origin master%0a%3c @]%0a%3c %0a
48 host:1684710404=75.158.138.249
49 author:1684707260=izzyb
50 diff:1684707260:1684699029:=3,10c3,8%0a%3c This document describes a way to configure hosting git/got repos for users. The goal is to provide shared hosting of repos, from multiple users, without needing to grant shared group privileges or even accounts on the public server. This maximises the isolation, for security reasons, between users and the public facing server. Users will store their repos in their home directory (~/public/gitgot/) and push/pull changes to it locally or via ssh as normal (details below).%0a%3c %0a%3c Two scripts will be used to accomplish this; gitgot-staging.pl will run on the User Host, and gitgot-deploy.pl will run on the Public Host. They will need to run as root via a cron job at regular intervals (5 - 15 min). The gitgot-staging.pl script will rsync repos from each user's home directory to the staging area on the User Host. It'll then rsync the User Host staging area to the Public Host's staging area. The gitgot-deploy.pl script will rsync the files from the staging area to the web server's chroot for deployment.%0a%3c %0a%3c The staging area is needed for security reasons. The script needs access to rsync the files between hosts without a password which would be a security concern if it can directly modify other areas on the Public Host. It's on both the the User and Public host to allow the same scripts to be used even for user accounts hosted on the Public Host. The scripts will need an account (''gitgot'') on each machine and have an ssh key allowing the User Host to connect to the Public Host without a password. The staging area on both hosts will be owned by the ''gitgot'' user. The repos will be owned by the web server user once deployed.%0a%3c %0a%3c The repo will be staged in /var/gitgot/user/repo directory structure.%0a%3c %0a---%0a> This document describes a way to configure hosting of users git/got repos. The goal is to provide shared hosting of repos without needing to grant shared group privileges or even accounts on the hosting server. This maximises the isolation between users and the public facing server. Users will store their repos in their home directory (~/public/gitgot/) which can be on the public host, or on a different server all together. They will push/pull changes to it locally or via ssh as normal (details below).%0a> %0a> Two scripts will be used to accomplish this; gitgit-staging.pl will run on the User Host, and gitgot-deploy.pl will run on the Public Host. They will need to run as root via a cron job at regular intervals (15 min). The gitgot-staging.pl script will rsync repos from each user's home directory to the staging area on the User Host. It'll then rsync the staging area to the Public Host's staging area. The scripts will need an account (''gitgot'') on each machine and have an ssh public key allowing the User Host's to connect to the Public Host without a password. The staging area on both hosts will be owned by the ''gitgot'' user. %0a> %0a> The repo will be staged in /var/gitgot/user/repo directory structure. The staging area is needed for security reasons. The script needs access to rsync the files without a password without having access to directly modify the web servers chroot. The gitgot-deploy.pl script will rsync the files from the staging area to the web server's chroot for deployment.%0a> %0a17,18c15,16%0a%3c The configuration of the user host consists of creating the ~/public/gitgot directory where users will put the repos they wish to publish. The gitgot-staging.pl script can be configured to create this folder for you if it is missing for existing users or you can use other means to setup the directory. For new accounts, this can be done automatically by adding it to /etc/skel directory like this:%0a%3c %0a---%0a> The configuration of the user host consists of creating the ~/public/gitgot directory where users will put the repos they wish to publish. The gitgot-staging.pl script will create this folder for you if it is missing for existing users. For new accounts, this can be done automatically by adding it to /etc/skel directory like this:%0a> %0a59,60c57,58%0a%3c You'll need to do the same on the Public Host and copy the public key over. Run the following, but change %3cadminUser> and %3cremotehost> as needed. Doas is broken and wont let you cat >> so a few extra steps to do this as a copy/paste...%0a%3c %0a---%0a> You'll need to do the same on the remote system and copy the key over. Run the following, but change %3cadminUser> and %3cremotehost> as needed. Doas is broken and wont let you cat >> so a few extra steps to do this as a copy/paste...%0a> %0a64,65c62%0a%3c doas cat /home/gitgot/.ssh/id_ed25519.pub | ssh $ssh_as 'cat >>~/doasbroken.tmp'%0a%3c # Log in to the Public Host%0a---%0a> doas cat /home/gitgot/.ssh/id_ed25519 | ssh $ssh_as 'cat >>~/doasbroken.tmp'%0a67c64%0a%3c # Copy paste these separately after logging in%0a---%0a> # you'll have to copy and paste these desperately after logging in%0a72d68%0a%3c exit%0a85,97c81,112%0a%3c #!/usr/bin/perl%0a%3c use strict;%0a%3c use warnings;%0a%3c my $config={%0a%3c remoteHost=>'got.ircnow.org',%0a%3c remoteUser=>'gitgot',%0a%3c remoteStage=>'/var/gitgot',%0a%3c repoDir=>'public/gitgot',%0a%3c createRepoDir=>1,%0a%3c stageDir=>'/var/gitgot',%0a%3c gitgotUser=>'gitgot',%0a%3c key_path=>'/home/gitgot/.ssh/id_ed25519',%0a%3c rsync_cmd=>'openrsync',%0a---%0a> #!/usr/bin/perl %0a> # %0a> # This script copies files from the users ~/public/gitgot directory %0a> # and transfers it to the server's /var/gitgot/%3cusername>/ directory. %0a> # The server will run a cronjob as root to move the files from there %0a> # to the /var/www/htdocs/got/public/ directory for sharing online. %0a> # %0a> # Ownership on the files will change as configured. %0a> # It is assumed the owner of the files wont have an account on the %0a> # destination server. The user will push changes to the repo in their %0a> # home directory as normal via an ssh:// git repo url. %0a> # %0a> use strict; %0a> use warnings; %0a> %0a> use Net::OpenSSH; %0a> #Using file find is probably too heavy for the need here %0a> #use File::Find; %0a> %0a> # Doing it with system calls instead %0a> #use File::Rsync; %0a> #$rsync=File::Rsync->new(); %0a> %0a> my $config={ %0a> remoteHost=>'got.ircnow.org', %0a> remoteUser=>'gitgot', %0a> remoteStage=>'/var/gitgot', %0a> repoDir=>'public/gitgot',%0a> stageDir=>'/var/gitgot',%0a> gitgotUser=>'gitgot',%0a> key_path=>'/home/gitgot/.ssh/id_ed25519',%0a> rsync_cmd=>'openrsync',%0a98a114%0a> %0a100,104c116,117%0a%3c if (not -d $config->{stageDir}) {%0a%3c my (undef,undef,$uid,$gid) = getpwnam($config->{gitgotUser})%0a%3c or die "User " . $config->{gitgotUser} . " doesn't exist";%0a%3c mkdir $config->{stageDir}, 0750;%0a%3c chown $uid,$gid,$config->{stageDir};%0a---%0a> if (not -d "/var/gitgot") { %0a> mkdir "/var/gitgot";%0a105a119%0a> %0a108,117c122,133%0a%3c if (-d "/home/$user/" . $config->{repoDir}) {%0a%3c # Skip if no repos%0a%3c opendir (my $RDH, "/home/$user/" . $config->{repoDir})%0a%3c or die "Can't opendir /home/$user/ : $!";%0a%3c my @repos = readdir($RDH);%0a%3c next unless (scalar @repos > 2);%0a%3c %0a%3c # Make sure user has a staging directory%0a%3c if(not -d $config->{stageDir} . "/$user") {%0a%3c mkdir ($config->{stageDir} . "/$user");%0a---%0a> if (-d "/home/$user/" . $config->{repoDir}) {%0a> # Make sure user has a staging directory%0a> if(not -d $config->{stageDir} . "/$user") {%0a> mkdir ($config->{stageDir} . "/$user");%0a> }%0a> # rsync files to local staging area%0a> my $src=qq(/home/$user/$config->{repoDir}/*);%0a> my $dst=qq($config->{stageDir}/$user/); %0a> my $rv=system(qq{%0a> openrsync --rsync-path=openrsync -a --delete $src $dst%0a> });%0a> warn "rsync failed $user\n$!\n" if ($rv != 0);%0a119,137d134%0a%3c # rsync files to local staging area%0a%3c my $src=qq(/home/$user/$config->{repoDir}/*);%0a%3c my $dst=qq($config->{stageDir}/$user/);%0a%3c my $rv=system(qq{%0a%3c openrsync --rsync-path=openrsync -a --delete $src $dst%0a%3c });%0a%3c warn "rsync failed $user\n$!\n" if ($rv != 0);%0a%3c } elsif ($config->{createRepoDir}) { # create missing repo dirs%0a%3c # skip dirs without users.%0a%3c my (undef,undef,$uid,$gid) = getpwnam($user) or next;%0a%3c my $repoPath="/home/$user";%0a%3c for my $dir (split('/', $config->{repoDir})) {%0a%3c $repoPath.="/$dir";%0a%3c unless (-e $repoPath) {%0a%3c mkdir $repoPath, 0755;%0a%3c chown $uid,$gid,$repoPath;%0a%3c }%0a%3c }%0a%3c }%0a139c136%0a%3c # rsync to remote staging area if we have a remote host%0a---%0a> # rsync to remote staging area if we have a remote host %0a141,155c138,151%0a%3c # Only sync if we have repos to deploy%0a%3c opendir (my $SDH, $config->{stageDir})%0a%3c or die "Can't opendir ". $config->{stageDir} . " : $!";%0a%3c my @repos = readdir($SDH);%0a%3c %0a%3c if(scalar @repos > 2) {%0a%3c my $src=$config->{stageDir} . "/*";%0a%3c my $dst=$config->{gitgotUser}. "@" . $config->{remoteHost}%0a%3c . ":" . $config->{remoteStage} . "/";%0a%3c %0a%3c my $rv=system(qq{%0a%3c openrsync --rsync-path=openrsync -ae %0a%3c "ssh -i $config->{key_path}" --delete $src $dst%0a%3c });%0a%3c }%0a---%0a> my $src=$config->{stageDir} . "/*"; %0a> my $dst=$config->{gitgotUser}. "@" . $config->{remoteHost} . ":" . $config->{remoteStage} . "/";%0a> %0a> my $rv=system(qq{ %0a> openrsync --rsync-path=openrsync -ae "ssh -i $config->{key_path}" --delete $src $dst%0a> }); %0a> #IB Net::OpenSSH requires rsync; doesn't work with openrsync%0a> #use Net::OpenSSH to rsync to remote server using a key%0a> #my $ssh = Net::OpenSSH->new( $config->{remoteHost}, %0a> # user => $config->{remoteUser}, %0a> # key_path => $config->{key_path},%0a> # rsync_cmd => $config->{rsync_cmd}, %0a> #);%0a> #$ssh->rsync_put({safe_links => 1}, $config->{stageDir}, $config->{remoteStage});%0a
51 host:1684707260=75.158.138.249
52 author:1684699029=izzyb
53 diff:1684699029:1684697463:=3,7c3,9%0a%3c This document describes a way to configure hosting of users git/got repos. The goal is to provide shared hosting of repos without needing to grant shared group privileges or even accounts on the hosting server. This maximises the isolation between users and the public facing server. Users will store their repos in their home directory (~/public/gitgot/) which can be on the public host, or on a different server all together. They will push/pull changes to it locally or via ssh as normal (details below).%0a%3c %0a%3c Two scripts will be used to accomplish this; gitgit-staging.pl will run on the User Host, and gitgot-deploy.pl will run on the Public Host. They will need to run as root via a cron job at regular intervals (15 min). The gitgot-staging.pl script will rsync repos from each user's home directory to the staging area on the User Host. It'll then rsync the staging area to the Public Host's staging area. The scripts will need an account (''gitgot'') on each machine and have an ssh public key allowing the User Host's to connect to the Public Host without a password. The staging area on both hosts will be owned by the ''gitgot'' user. %0a%3c %0a%3c The repo will be staged in /var/gitgot/user/repo directory structure. The staging area is needed for security reasons. The script needs access to rsync the files without a password without having access to directly modify the web servers chroot. The gitgot-deploy.pl script will rsync the files from the staging area to the web server's chroot for deployment.%0a---%0a> This document describes a way to configure a remote repo for hosting git/got repos. The goal is to provide shared hosting of repos without needing to grant shared group privileges or even accounts on the hosting server. This maximises the isolation between users and the public facing servers. Users will store their repos in their home directory (~/public/gitgot/) which can be on the host, or on a different server all together. They will push/pull changes to it locally or via ssh as normal (details below).%0a> %0a> A cronjob on the machine hosting the users home directory will rsync the users repo to a staging area in /var/gitgot/'''username'''/ and their ownership changed as specified in the configuration. The script will need an account with permission to access the destination server on the public facing server. The script uses rsync to copy the file and an ssh key for authentication (details below).%0a> %0a> A second script running on the web host will take the files from the staging area, change their ownership, and place them where needed to share them via gotweb or to allow public cloning. The use of the staging area is to avoid needing to give the remote script root permission on the public server. Both scripts will need to run as root on their respective machines, have doas capability, or use the openbsd pledge/unveil mechanism. The scripts below assume the scripts will be running as root via cron.%0a> %0a> The repo will be staged in /var/gitgot/user/repo directory structure.%0a
54 host:1684699029=75.158.138.249
55 author:1684697463=izzyb
56 diff:1684697463:1684695581:=13,14d12%0a%3c This configuration assumes two hosts, but can also be run on a single or multiple hosts. The host where the user accounts and home directories are will be known as the User Host, and the host where the web server runs will be known as the Public Host in this configuration. You can have multiple user hosts that all use the same public host, and the public host can also have user accounts on it. Advanced configurations could also have multiple public hosts with a load balancing configuration, but that is beyond the scope of this document.%0a%3c %0a17,18c15,16%0a%3c The configuration of the user host consists of creating the ~/public/gitgot directory where users will put the repos they wish to publish. The gitgot-staging.pl script will create this folder for you if it is missing for existing users. For new accounts, this can be done automatically by adding it to /etc/skel directory like this:%0a%3c %0a---%0a> The user host is where the user's home directories are. It can be the same machine as the public host; its also possible to have both local and remote user accounts in this configuration. We'll discuss them as being separate with notes as needed for both operations. You'll want to add a ~/public/gitgot folder to the home directory of existing users. This can be done automatically for new users by adding it to /etc/skel directory like this:%0a> %0a25c23%0a%3c The ''gitgot-staging.pl'' script will move them from the users directory to the staging area and change ownership to the gitgot user. Both the user Host and Public Host will need a gitgot user created like this:%0a---%0a> ~/public/gitgot is where users will put the repos they wish to publish. The ''gitgot-staging.pl'' script will move them from the users directory to the staging area and change ownership to the gitgot user. Both the user Host and Public Host will need a gitgot user created like this:%0a
57 host:1684697463=75.158.138.249
58 author:1684695581=izzyb
59 diff:1684695581:1684694817:=23,24c23,24%0a%3c ~/public/gitgot is where users will put the repos they wish to publish. The ''gitgot-staging.pl'' script will move them from the users directory to the staging area and change ownership to the gitgot user. Both the user Host and Public Host will need a gitgot user created like this:%0a%3c %0a---%0a> This is where users will put the repos they wish to publish. The script will move them from the users public/gitgot/ directory to the staging area and change ownership to the gitgot user. Both the user Host and Public Host will need a gitgot user created like this:%0a> %0a71,74c71,72%0a%3c !!! gitgot-staging.pl%0a%3c %0a%3c We'll put the script for copying the files in /home/gitgot/bin/gitgot-staging.pl This script is also available via a git repo here, but you can copy and paste it from here also.%0a%3c %0a---%0a> We'll put the script for copying the files in /home/gitgot/bin/gitgot-staging.pl %0a> %0a79a78,82%0a> %0a> !!! gitgot-staging.pl%0a> %0a> This script is also available via a git repo here.%0a> %0a154,163d156%0a%3c %0a%3c !! Public Host%0a%3c %0a%3c The Public Host is where the web server runs. It can be the same machine as the User Host. The gitgot-staging.pl script will copy the repos from each users home dirs to /var/gitgot/username/. A second script called gitgot-deploy.pl will run from a cron job as root to copy the files from the staging area to the proper location for the web server. %0a%3c %0a%3c %0a%3c %0a%3c %0a%3c %0a%3c %0a
60 host:1684695581=75.158.138.249
61 author:1684694817=izzyb
62 diff:1684694817:1684651231:=59c59%0a%3c [@%0a---%0a> @]%0a78,82d77%0a%3c %0a%3c !!! gitgot-staging.pl%0a%3c %0a%3c This script is also available via a git repo here.%0a%3c %0a159c154%0a%3c ! Using the Repo%0a---%0a> !! Using the Repo%0a
63 host:1684694817=75.158.138.249
64 author:1684651231=izzyb
65 diff:1684651231:1684645017:=3,8c3,6%0a%3c This document describes a way to configure a remote repo for hosting git/got repos. The goal is to provide shared hosting of repos without needing to grant shared group privileges or even accounts on the hosting server. This maximises the isolation between users and the public facing servers. Users will store their repos in their home directory (~/public/gitgot/) which can be on the host, or on a different server all together. They will push/pull changes to it locally or via ssh as normal (details below).%0a%3c %0a%3c A cronjob on the machine hosting the users home directory will rsync the users repo to a staging area in /var/gitgot/'''username'''/ and their ownership changed as specified in the configuration. The script will need an account with permission to access the destination server on the public facing server. The script uses rsync to copy the file and an ssh key for authentication (details below).%0a%3c %0a%3c A second script running on the web host will take the files from the staging area, change their ownership, and place them where needed to share them via gotweb or to allow public cloning. The use of the staging area is to avoid needing to give the remote script root permission on the public server. Both scripts will need to run as root on their respective machines, have doas capability, or use the openbsd pledge/unveil mechanism. The scripts below assume the scripts will be running as root via cron.%0a%3c %0a---%0a> This document describes a way to configure a remote repo for hosting via git/got. The goal is to provide shared hosting of repos without needing to grant shared group privileges or even accounts on the hosting server. Users will store their repos in ~/public/gitgot/ which can be on the host, or on a different server all together. They will push/pull changes to it locally or via ssh as normal. A cronjob on the machine hosting the users home directory will rsync the users repo to a staging area in /var/gitgot/'''username'''/ and their ownership changed as configured. The script will need an account with permission to do this on the web host.%0a> %0a> A second script running on the web host will take the files from the staging area, change their ownership, and place them where needed to share them via gotweb or git clone. The use of the staging area is to allow for a remote script to place the files using ssh keys without a password and without permissions to modify anything except the git files on the public host. Both scripts will need to run as root on their respective machines, have doas capability, or use the openbsd pledge/unveil mechanism.%0a> %0a15,16c13,14%0a%3c The user host is where the user's home directories are. It can be the same machine as the public host; its also possible to have both local and remote user accounts in this configuration. We'll discuss them as being separate with notes as needed for both operations. You'll want to add a ~/public/gitgot folder to the home directory of existing users. This can be done automatically for new users by adding it to /etc/skel directory like this:%0a%3c %0a---%0a> The user host can also be the web host, but we'll discuss them as being separate with notes as needed. You'll want to add a ~/public/gitgot folder to the users home directory if it already exists. This can be done automatically for new users by adding it to /etc/skel directory like this:%0a> %0a23,24c21,22%0a%3c This is where users will put the repos they wish to publish. The script will move them from the users public/gitgot/ directory to the staging area and change ownership to the gitgot user. Both the user Host and Public Host will need a gitgot user created like this:%0a%3c %0a---%0a> The script that does the moving of the files should have an account with access to the files in the staging area. It will need permissions to read the users home directory as well as write to the staging area. For now, we'll assume it'll be a script running via roots crontab, but ideally it should be run as a restricted user. It will need to have an ssh key to access the remote web host so we'll generate that also:%0a> %0a29d26%0a%3c exit%0a57,75c54,58%0a%3c You'll need to do the same on the remote system and copy the key over. Run the following, but change %3cadminUser> and %3cremotehost> as needed. Doas is broken and wont let you cat >> so a few extra steps to do this as a copy/paste...%0a%3c %0a%3c @]%0a%3c ssh_as='%3cadminUser>@%3cremotehost>'%0a%3c ssh $ssh_as doas useradd -m gitgot%0a%3c doas cat /home/gitgot/.ssh/id_ed25519 | ssh $ssh_as 'cat >>~/doasbroken.tmp'%0a%3c ssh $ssh_as%0a%3c # you'll have to copy and paste these desperately after logging in%0a%3c doas su %0a%3c cat doasbroken.tmp >>/home/gitgot/authorized_keys%0a%3c rm doasbroken.tmp%0a%3c exit%0a%3c @]%0a%3c %0a%3c We'll put the script for copying the files in /home/gitgot/bin/gitgot-staging.pl %0a%3c %0a%3c [@%0a%3c mkdir /home/gitgot/bin%0a%3c vim /home/gitgot/bin/gitgot-staging.pl%0a---%0a> We'll put the script for copying the files in ~/bin/gitgot-staging.pl These commands assume you are still logged in as the gitgot user.%0a> %0a> [@%0a> mkdir bin%0a> vim bin/gitgot-staging.pl%0a
66 host:1684651231=75.158.138.249
67 author:1684645017=izzyb
68 diff:1684645017:1684619988:=3,6c3,6%0a%3c This document describes a way to configure a remote repo for hosting via git/got. The goal is to provide shared hosting of repos without needing to grant shared group privileges or even accounts on the hosting server. Users will store their repos in ~/public/gitgot/ which can be on the host, or on a different server all together. They will push/pull changes to it locally or via ssh as normal. A cronjob on the machine hosting the users home directory will rsync the users repo to a staging area in /var/gitgot/'''username'''/ and their ownership changed as configured. The script will need an account with permission to do this on the web host.%0a%3c %0a%3c A second script running on the web host will take the files from the staging area, change their ownership, and place them where needed to share them via gotweb or git clone. The use of the staging area is to allow for a remote script to place the files using ssh keys without a password and without permissions to modify anything except the git files on the public host. Both scripts will need to run as root on their respective machines, have doas capability, or use the openbsd pledge/unveil mechanism.%0a%3c %0a---%0a> This document describes a way to configure a remote repo for hosting via git/got. The goal is to provide shared hosting of repos without needing to grant shared group privileges or even accounts on the hosting server. Users will store their repos in ~/public/gitgot/ which can be on the host, or on a different server all together. They will push/pull changes to it locally or via ssh as normal. A cronjob on the machine hosting the users home directory will rsync the users repo to a staging area in /var/gitgot/'''username'''/ and their ownership and permissions changed as configured. The script will need an account with permission to do this on the web host.%0a> %0a> A second script running on the web host will take the files from the staging area, change their ownership and permissions as needed to place them where needed to share them via gotweb or git clone. The use of the staging area is to allow for a remote script to place the files using ssh keys without a password and without permissions to modify anything except the git files on the public host. Both scripts will need to run as root on their respective machines, have doas capability, or use the openbsd pledge/unveil mechanism.%0a> %0a61,134c61,62%0a%3c %25code%25[@%0a%3c #!/usr/bin/perl %0a%3c # %0a%3c # This script copies files from the users ~/public/gitgot directory %0a%3c # and transfers it to the server's /var/gitgot/%3cusername>/ directory. %0a%3c # The server will run a cronjob as root to move the files from there %0a%3c # to the /var/www/htdocs/got/public/ directory for sharing online. %0a%3c # %0a%3c # Ownership on the files will change as configured. %0a%3c # It is assumed the owner of the files wont have an account on the %0a%3c # destination server. The user will push changes to the repo in their %0a%3c # home directory as normal via an ssh:// git repo url. %0a%3c # %0a%3c use strict; %0a%3c use warnings; %0a%3c %0a%3c use Net::OpenSSH; %0a%3c #Using file find is probably too heavy for the need here %0a%3c #use File::Find; %0a%3c %0a%3c # Doing it with system calls instead %0a%3c #use File::Rsync; %0a%3c #$rsync=File::Rsync->new(); %0a%3c %0a%3c my $config={ %0a%3c remoteHost=>'got.ircnow.org', %0a%3c remoteUser=>'gitgot', %0a%3c remoteStage=>'/var/gitgot', %0a%3c repoDir=>'public/gitgot',%0a%3c stageDir=>'/var/gitgot',%0a%3c gitgotUser=>'gitgot',%0a%3c key_path=>'/home/gitgot/.ssh/id_ed25519',%0a%3c rsync_cmd=>'openrsync',%0a%3c };%0a%3c %0a%3c # verify we have a local staging area%0a%3c if (not -d "/var/gitgot") { %0a%3c mkdir "/var/gitgot";%0a%3c }%0a%3c %0a%3c opendir (my $DH, '/home/') || die "Can't opendir /home : $!";%0a%3c while (my $user = readdir($DH)) {%0a%3c if (-d "/home/$user/" . $config->{repoDir}) {%0a%3c # Make sure user has a staging directory%0a%3c if(not -d $config->{stageDir} . "/$user") {%0a%3c mkdir ($config->{stageDir} . "/$user");%0a%3c }%0a%3c # rsync files to local staging area%0a%3c my $src=qq(/home/$user/$config->{repoDir}/*);%0a%3c my $dst=qq($config->{stageDir}/$user/); %0a%3c my $rv=system(qq{%0a%3c openrsync --rsync-path=openrsync -a --delete $src $dst%0a%3c });%0a%3c warn "rsync failed $user\n$!\n" if ($rv != 0);%0a%3c }%0a%3c }%0a%3c # rsync to remote staging area if we have a remote host %0a%3c if (defined $config->{remoteHost}) {%0a%3c my $src=$config->{stageDir} . "/*"; %0a%3c my $dst=$config->{gitgotUser}. "@" . $config->{remoteHost} . ":" . $config->{remoteStage} . "/";%0a%3c %0a%3c my $rv=system(qq{ %0a%3c openrsync --rsync-path=openrsync -ae "ssh -i $config->{key_path}" --delete $src $dst%0a%3c }); %0a%3c #IB Net::OpenSSH requires rsync; doesn't work with openrsync%0a%3c #use Net::OpenSSH to rsync to remote server using a key%0a%3c #my $ssh = Net::OpenSSH->new( $config->{remoteHost}, %0a%3c # user => $config->{remoteUser}, %0a%3c # key_path => $config->{key_path},%0a%3c # rsync_cmd => $config->{rsync_cmd}, %0a%3c #);%0a%3c #$ssh->rsync_put({safe_links => 1}, $config->{stageDir}, $config->{remoteStage});%0a%3c }%0a%3c @]%0a---%0a> %0a> %0a
69 host:1684645017=75.158.138.249
70 author:1684619988=izzyb
71 diff:1684619988:1684548351:=3,14c3,12%0a%3c This document describes a way to configure a remote repo for hosting via git/got. The goal is to provide shared hosting of repos without needing to grant shared group privileges or even accounts on the hosting server. Users will store their repos in ~/public/gitgot/ which can be on the host, or on a different server all together. They will push/pull changes to it locally or via ssh as normal. A cronjob on the machine hosting the users home directory will rsync the users repo to a staging area in /var/gitgot/'''username'''/ and their ownership and permissions changed as configured. The script will need an account with permission to do this on the web host.%0a%3c %0a%3c A second script running on the web host will take the files from the staging area, change their ownership and permissions as needed to place them where needed to share them via gotweb or git clone. The use of the staging area is to allow for a remote script to place the files using ssh keys without a password and without permissions to modify anything except the git files on the public host. Both scripts will need to run as root on their respective machines, have doas capability, or use the openbsd pledge/unveil mechanism.%0a%3c %0a%3c The repo will be staged in /var/gitgot/user/repo directory structure.%0a%3c %0a%3c ! Setup the Hosts%0a%3c %0a%3c !! User Host%0a%3c %0a%3c The user host can also be the web host, but we'll discuss them as being separate with notes as needed. You'll want to add a ~/public/gitgot folder to the users home directory if it already exists. This can be done automatically for new users by adding it to /etc/skel directory like this:%0a%3c %0a---%0a> This document describes a way to configure a remote repo for hosting via git/got. The goal is to provide shared hosting of repos without needing to grant shared group privileges to a common folder. Users will store files in their home directory with standard access controls. They will push/pull changes to it locally or via ssh. The repo will be synced to a staging area in /var and their permissions changed.%0a> %0a> It assumes the user has a shell account on the host machine.%0a> %0a> The repo will be staged in /var/git/user/repo directory structure with symlinks back to their home directory.%0a> %0a> %25commands%25[@%0a> ssh username@got.ircnow.org git init --bare /var/git/reponame%0a> @]%0a> %0a16,18c14,15%0a%3c doas mkdir -p /etc/skel/public/gitgot%0a%3c doas chown root.wheel /etc/skel/public/gitgot%0a%3c doas chmod 755 /etc/skel/public/gitgot%0a---%0a> git remote add origin ssh://got.ircnow.org/var/git/reponame%0a> git push --set-upstream origin master%0a20,76d16%0a%3c %0a%3c The script that does the moving of the files should have an account with access to the files in the staging area. It will need permissions to read the users home directory as well as write to the staging area. For now, we'll assume it'll be a script running via roots crontab, but ideally it should be run as a restricted user. It will need to have an ssh key to access the remote web host so we'll generate that also:%0a%3c %0a%3c [@%0a%3c doas useradd -m gitgot%0a%3c doas su - gitgot%0a%3c ssh-keygen -t ed25519%0a%3c @]%0a%3c %0a%3c Use the default path for the key file and leave the password empty. The output will look something like this:%0a%3c %0a%3c [@%0a%3c Generating public/private ed25519 key pair.%0a%3c Enter file in which to save the key (/home/gitgot/.ssh/id_ed25519): %0a%3c Enter passphrase (empty for no passphrase): %0a%3c Enter same passphrase again: %0a%3c Your identification has been saved in /home/gitgot/.ssh/id_ed25519%0a%3c Your public key has been saved in /home/gitgot/.ssh/id_ed25519.pub%0a%3c The key fingerprint is:%0a%3c SHA256:lyrfN6pDSRuVI112nfZiMh02aOoHxvgq5Y5jyY2QKpI gitgot@planetofnix.com%0a%3c The key's randomart image is:%0a%3c +--[ED25519 256]--+%0a%3c | . oo....|%0a%3c | . =.o.+o.|%0a%3c | = + o.o.|%0a%3c | + =.o + .|%0a%3c | . .SBo. + . |%0a%3c | o =oo . |%0a%3c | . . o.B.. . |%0a%3c |E . B+=. o |%0a%3c |.. ..++ooo . |%0a%3c +----[SHA256]-----+%0a%3c @]%0a%3c %0a%3c We'll put the script for copying the files in ~/bin/gitgot-staging.pl These commands assume you are still logged in as the gitgot user.%0a%3c %0a%3c [@%0a%3c mkdir bin%0a%3c vim bin/gitgot-staging.pl%0a%3c @]%0a%3c %0a%3c %0a%3c %0a%3c %0a%3c %0a%3c !! Using the Repo%0a%3c %0a%3c [@%0a%3c %25commands%25[@%0a%3c ssh username@got.ircnow.org git init --bare /var/git/reponame%0a%3c @]%0a%3c %0a%3c [@%0a%3c git remote add origin ssh://got.ircnow.org/var/git/reponame%0a%3c git push --set-upstream origin master%0a%3c @]%0a%3c %0a
72 host:1684619988=75.158.138.249
73 author:1684548351=izzyb
74 diff:1684548351:1684548319:=1,3c1%0a%3c ! Summary%0a%3c %0a%3c This document describes a way to configure a remote repo for hosting via git/got. The goal is to provide shared hosting of repos without needing to grant shared group privileges to a common folder. Users will store files in their home directory with standard access controls. They will push/pull changes to it locally or via ssh. The repo will be synced to a staging area in /var and their permissions changed.%0a---%0a> ! Summary: This document describes a way to configure a remote repo for hosting via git/got. The goal is to provide shared hosting of repos without needing to grant shared group privileges to a common folder. Users will store files in their home directory with standard access controls. They will push/pull changes to it locally or via ssh. The repo will be synced to a staging area in /var and their permissions changed.%0a
75 host:1684548351=75.158.138.249
76 author:1684548319=izzyb
77 diff:1684548319:1684548319:=1,14d0%0a%3c ! Summary: This document describes a way to configure a remote repo for hosting via git/got. The goal is to provide shared hosting of repos without needing to grant shared group privileges to a common folder. Users will store files in their home directory with standard access controls. They will push/pull changes to it locally or via ssh. The repo will be synced to a staging area in /var and their permissions changed.%0a%3c %0a%3c It assumes the user has a shell account on the host machine.%0a%3c %0a%3c The repo will be staged in /var/git/user/repo directory structure with symlinks back to their home directory.%0a%3c %0a%3c %25commands%25[@%0a%3c ssh username@got.ircnow.org git init --bare /var/git/reponame%0a%3c @]%0a%3c %0a%3c [@%0a%3c git remote add origin ssh://got.ircnow.org/var/git/reponame%0a%3c git push --set-upstream origin master%0a%3c @]%0a
78 host:1684548319=75.158.138.249
