Get an ID and the token to reach the EOSC cloud.
This is done accessing the portal using ID Federation eduTEAMS. So your university/observatory should be part of eduTEAM/eduGAIN.
Open https://aai.egi.eu/token/ in a (private) browser window and click `Authorise`
Click 'EduTeams', search for your institution ID provider and authenticate using your federation ID.
- Copy from web interface : `Access token` that is necessary for the following steps (copy and paste it somewhere locally).
- Copy and paste the curl command on a shell, and gather the access_token thus generated.
export ACCESS_TOKEN=************* curl*****************************
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 757 100 757 0 0 2938 0 --:--:-- --:--:-- --:--:-- 2934
{
"sub": "***",
"voperson_id": "***",
"email_verified": true,
"name": "***",
...
}
Interact with Openstack to create a virtual machine using Python library
Create the Python virtual environment
Using fedcloud to help interacting with the federation. If you face any trouble restart from scratch using rm of ~/.my_venvs
mkdir ~/.my_venvs python3 -m venv ~/.my_venvs/fedcloud source ~/.my_venvs/fedcloud/bin/activate
Install fedcloud and configure
pip install fedcloudclient sudo mkdir -p /etc/grid-security/certificates CA_BUNDLE=https://dist.eugridpma.info/distribution/igtf/current/accredited/igtf-preinstalled-bundle-classic.tar.gz curl -s $CA_BUNDLE | sudo tar -xvz -C /etc/grid-security/certificates cat /etc/grid-security/certificates/*.pem >> $(python -m requests.certs)
Now make a VM at IN2P3-IRES
fedcloud endpoint list --site IN2P3-IRES fedcloud endpoint projects --site IN2P3-IRES --oidc-access-token=$ACCESS_TOKEN
gather the id and store it in the variable PROJECT_ID
export PROJECT_ID=*******
fedcloud endpoint env --site IN2P3-IRES --oidc-access-token=$ACCESS_TOKEN --project-id $PROJECT_ID | tee env-VESPA-IN2P3-IRES.sh
export OS_PROJECT_ID="***";
export OS_AUTH_URL="https://identity.cloud.muni.cz/v3";
export OS_AUTH_TYPE="v3oidcaccesstoken";
export OS_IDENTITY_PROVIDER="egi.eu";
export OS_PROTOCOL="openid";
export OS_ACCESS_TOKEN="*****"
Setup the returned environment variables for IN2P3-IRES endpoint
source env-VESPA-IN2P3-IRES.sh
Interact with APIs
openstack network list
+--------------------------------------+---------------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+---------------+--------------------------------------+
| 0967d6df-ef2c-40ef-af4b-c8e3bc9b2bfd | egi-vespa-net | 2cde45d4-fa10-4b60-8071-db7b03bd90af |
| 8fa5eb12-ee72-4e77-a45b-8fde11efc75f | ext-net | 6e8ffc7a-4536-4c21-a8e5-00227ee8619a |
| a306e9f5-41a4-421c-a953-d24e3f18b183 | mynetwork | |
+--------------------------------------+---------------+--------------------------------------+
Create an openstack publick key and security access
openstack keypair create --public-key ~/.ssh/id_rsa.pub my-key1
Get a fresh token
fedcloud token list-vos --oidc-access-token $OS_ACCESS_TOKEN
Check available OpenStack endpoints
fedcloud endpoint projects --site IN2P3-IRES --oidc-access-token=$OS_ACCESS_TOKEN
Lists the available os :
openstack image list
+--------------------------------------+------------------------------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+------------------------------------------------------+--------+
| aea85771-8c8e-4082-a547-88c28526cb07 | Image for EGI CentOS 7 [CentOS/7/VirtualBox] | active |
| e54f282d-fc62-4785-b95f-2d475e2b3110 | Image for EGI Ubuntu 18.04 [Ubuntu/18.04/VirtualBox] | active |
| 6169c608-e90b-462a-89d7-edb324d6ebf2 | almalinux-8-x86_64 | active |
| f5566e8a-2e11-4eb1-9828-fa1ca4926c35 | almalinux-8-x86_64-2022-04-18 | active |
| 8d9ce611-49d0-49b3-8b3f-f0c4d46f4854 | almalinux-8-x86_64-2022-11-28 | active |
| 99a1d4bb-42ab-48bb-bdd3-a0e98faa3c08 | almalinux-8-x86_64-2023-02-01 | active |
Lists the different machine sizes available:
openstack flavor list
+--------------------------------------+----------------------+-------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+--------------------------------------+----------------------+-------+------+-----------+-------+-----------+
| 4c153ce3-a163-4668-baa7-2cbcb57e2dd8 | standard.medium | 4096 | 80 | 0 | 2 | True |
| 57bf9ed0-cd71-4c7c-b886-2a5263d52678 | standard.small | 2048 | 80 | 0 | 1 | True |
| 6c329868-8337-4084-9971-6f58f8208221 | standard.tiny | 1024 | 80 | 0 | 1 | True |
| 75038da9-d297-4f80-a144-b7ffbd55f162 | standard.memory | 32768 | 80 | 0 | 2 | True |
| bbca574d-b0ea-4738-be73-a0ae0ac67a6d | standard.large | 8192 | 80 | 0 | 4 | True |
| e7a3872a-cc28-48cc-9647-985acf5dce1d | standard.2core-16ram | 16384 | 80 | 0 | 2 | True |
+--------------------------------------+----------------------+-------+------+-----------+-------+-----------+
Check if security group exists :
openstack security group show vespagroup
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2021-02-16T09:52:44Z |
| description | opening port 80 and 8080 for vespa |
| id | 198661a5-d18f-4190-96ed-9ebb212b0aa9 |
| name | vespagroup |
| project_id | 89140661b6be4ef281ab7a67d4c83e0c |
| revision_number | 4 |
| rules | created_at='2021-02-16T09:53:14Z', direction='ingress', ethertype='IPv4', id='27ff1163-c2ee-4269-aa5c-75252eef9056', port_range_max='80', port_range_min='80', protocol='tcp', remote_ip_prefix='0.0.0.0/0', updated_at='2021-02-16T09:53:14Z' |
| | created_at='2021-02-16T09:52:44Z', direction='egress', ethertype='IPv4', id='ba0f2912-cc64-488f-a292-a10594aaf2c4', updated_at='2021-02-16T09:52:44Z' |
| | created_at='2021-02-16T09:53:25Z', direction='ingress', ethertype='IPv4', id='c38c5ff1-a1e2-4b8e-b07e-51c318ebc948', port_range_max='8080', port_range_min='8080', protocol='tcp', remote_ip_prefix='0.0.0.0/0', updated_at='2021-02-16T09:53:25Z' |
| | created_at='2021-02-16T09:52:44Z', direction='egress', ethertype='IPv6', id='ca96c0b7-891e-4a01-85dc-e048967e05c2', updated_at='2021-02-16T09:52:44Z' |
| | created_at='2021-02-16T09:54:55Z', direction='ingress', ethertype='IPv4', id='d3213236-b5f6-4d9b-885b-7390e227e8b3', port_range_max='22', port_range_min='22', protocol='tcp', remote_ip_prefix='0.0.0.0/0', updated_at='2021-02-16T09:54:55Z' |
| stateful | None |
| tags | [] |
| updated_at | 2021-02-16T09:54:55Z |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
IF VESPAGROUP DOESN'T EXIST ONLY :
Build security group and open the necessary port with the following commands:
It could answer that security group exists.
openstack security group create vespagroup --description "opening port 80 and 8080 for vespa" openstack security group rule create vespagroup --protocol tcp --dst-port 80:80 --remote-ip 0.0.0.0/0 openstack security group rule create vespagroup --protocol tcp --dst-port 8080:8080 --remote-ip 0.0.0.0/0 openstack security group rule create vespagroup --protocol tcp --dst-port 22:22 --remote-ip 0.0.0.0/0w
If you have done these steps but security group already exists, you will have to delete and restart because two security group with the same name will generate a conflict.
We choose ubuntu image 18.04 and 2cpu 4G ram - Take the ubuntu image id and the flavour name found with openstack image/flavor list
export IMAGE_ID=<<image ID>> export FLAVOR=<<flavor Name>>
example:
export IMAGE_ID=1d3d16c0-24b0-4960-80fe-64b9f4e1b4f1
export FLAVOR=standard.medium
Test if network exist
openstack network list
If not, create it
openstack network create mynetwork
We take testpls as VM name
openstack server create --flavor $FLAVOR --image $IMAGE_ID \ --nic net-id=egi-vespa-net --security-group vespagroup \ --key-name my-key1 testpls
openstack server add network testpls
openstack floating ip list
openstack server add floating ip testpls *******
To access again (find the ip address of your created machine)
Re- generate an access token
Open https://aai.egi.eu/token/ in a (private) browser window and click `Authorise`
Click 'EduTeams', search for your institution ID provider and authenticate using your federation ID.
- Copy and paste the curl command on a shell, and gather the access_token thus generated.
export ACCESS_TOKEN=************* source ~/.my_venvs/fedcloud/bin/activate fedcloud endpoint projects --site IN2P3-IRES --oidc-access-token=$ACCESS_TOKEN export PROJECT_ID=******* fedcloud endpoint env --site IN2P3-IRES --oidc-access-token=$ACCESS_TOKEN --project-id $PROJECT_ID | tee env-VESPA-IN2P3-IRES.sh
Setup environment variables for IN2P3-IRES endpoint
source env-VESPA-IN2P3-IRES.sh
openstack server list
+--------------------------------------+------------+--------+---------------------------------------------+-------------------------------------+-----------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------------+--------+---------------------------------------------+-------------------------------------+-----------+
| 7078c354-5451-4f27-8325-a5dd25bf523a | vespatuto1 | ACTIVE | egi-vespa-net=134.158.151.238, 172.16.12.95 | Image for Debian 11 [Debian/11/KVM] | m1.medium |
+--------------------------------------+------------+--------+---------------------------------------------+-------------------------------------+-----------+
gather the displayed public ip of the machine
connect
ssh <<os name>>@<<ip address>>
ssh debian@134.158.151.238
you can add your collaborator's ssh keys in ~/.ssh/authorized_keys to enable them to connect