Setup a KVM for Uploads Packages
This post is aiming to explain how to create a KVM and configure it for uploads packages to the Shacache system. Shacache is a cache system that caches packages like Python packages in PyPI(Shacache Network Cache Concept), variety source codes(from GitHub,etc). When running a software release on a machine, slapos will try to fetch packages or source code from shacache instead of the original site. If slapos didn't find the corresponding package from the Shacache, it will try to get it from the original site. On a non-configured machine, slapos will just fetch the following packages. On a configured machine, slapos will upload the missing packages with predefined keys. This post will explain how to add keys to configure a machine. But how to get such keys is another topic. The user should contact Julien.
This post assumes the reader has some basic background, so it omitted some details regarding how to install KVM, Debian, etc.
1. Create a KVM
Create a service on the panel.rapid.sapce. Select KVM with version 1.0.232. (https://handbook.rapid.space/user/rapidspace-HowTo.Request.A.KVM) The disk space needs 100GB. Set the RAM size to 16384. Click "Show Parameter XML", then copy the following XML to the editor:
<?xml version="1.0" encoding="utf-8"?>
2. Install Debian
Normal procedure. Just make sure you installed the SSH component.
2.1 Enable IPv6
On the instance page which you requested the KVM, you can see there has a description about how to enable IPv6 on your Debian system. See below:
PERMANENT SOLUTION: in your VM, add the lines below in /etc/network/interfaces and then run: "ifup ens4"
iface ens4 inet6 static
After this operation, you can use ssh to login into the machine:
3. Install slapos on the Debian
Run the following command to install Slapos:
wget https://deploy.erp5.net/vifib; bash vifib
The script will do some processing, after that. It will ask for some information, provide it as follow:
SlapOS Master API
SlapOS Master Web UI
For the first two options, just use the default one. For the token, you need to go to the panel.rapid.space, in the "Server" tab, request a token.
And then set up a name for this computer, eg: Shacache-testnode
Note: The most important part is, don't forget to set ens4 network as the IPv6 provider. The default one is "lo". We need to change it to "ens4".
4 Get the certificate files
Get YOUR_NAME.crt YOUR_NAME.key ca.crt, these three files were given by the shacache manager. You should contact him.
5 Place the cerfiticate files
Put the given key and certificate in /etc/opt/slapos/shacache/YOUR_NAME.key and /etc/opt/slapos/shacache/YOUR_NAME.cert
6 Modify the slapos configuration file
In your slapos configuration file (/etc/opt/slapos/slapos.cfg), add/modify the following section:
# Define options for binary cache, used to download already compiled software.
download-binary-cache-url = http://www.shacache.org/shacache
download-binary-url = http://www.shacache.org/shadir
# More options for 'networkcache-download' command
download-cache-url = http://www.shacache.org/shacache
download-dir-url = http://www.shacache.org/shadir
# Upload Configuration
signature-private-key-file = /etc/opt/slapos/shacache/signature.key
signature-certificate-file = /etc/opt/slapos/shacache/signature.crt
upload-binary-dir-url = https://www.shacache.org/shadir
upload-binary-cache-url = https://www.shacache.org/shacache
upload-cache-url = https://www.shacache.org/shacache
upload-dir-url = https://www.shacache.org/shadir
# Options for HTTPS URLsYOUR_NAME
shacache-ca-file = /etc/opt/slapos/shacache/ca.crt
shacache-cert-file = /etc/opt/slapos/shacache/
shacache-key-file = /etc/opt/slapos/shacache/
shadir-ca-file = /etc/opt/slapos/shacache/ca.crt
shadir-cert-file = /etc/opt/slapos/shacache/
shadir-key-file = /etc/opt/slapos/shacache/
That's is add the path of YOUR_NAME.crt and YOUR_NAME.key to the shacache-cert-file and shacache-key-file.
7 Generate SSL key
Create a directory: mkdir /etc/opt/slapos/shacache, run this command: "/opt/slapos/bin/generate-signature-key /etc/opt/slapos/slapos.cfg".
If you encountered an error like "ConfigParser.NoOptionError: No option 'signature-certificate-file' in section: 'networkcache'". Which means in step 6, your slapos configuration file (/etc/opt/slapos/slapos.cfg) is still using the field name 'signature_certificate_file'. All you need to do is modify the configuration file, use the dash to replace the underscore, then rerun the command.
This command will generate a file signature.crt, this is YOUR certificate. Put the content of signature.crt to the /etc/opt/slapos/slapos.cfg, in the signature-certificate-list field. This means you are trusted, and the file that you want to upload to the Shacache will be considered trusted.
Change the permission of signature.key
chmod 644 /etc/opt/slapos/shacache/signature.key
You can refer to the step in "How to upload to cache" section of this article.
Then build and install SR as a normal machine. One thing worth to mentioning: when providing SR, it has to come from the branch that has this signature.cert content.
5 Further Configuration for TestNode.
If we had set up a testnode on this KVM, and we want to upload the missing packages during the testnode running SR. In addition to step 4, we need to do something more.
1. We need to commit the content of signature.cert, which is generated in step 4, to stack/slapos.cfg. And when setting up a testnode, we need to use this testnode to use the slapos branch which contains the modified stack/slapos.cfg.
2. If you are set up with more than one machine. No need to generate the certificate for each machine, always use ONE certification generated based on your key on the first machine. This means on different test nodes you had set up, always use the same slapos branch.
3. If you want to download the packages that you uploaded by yourself, you have to add your certificate to two files:
- signature-certificate-list in /etc/opt/slapos/slapos.cfg -> download binary from cache
- signature-certificate-list in slapos/stack/slapos.cfg -> downloda source from cache