Create Azure Resource on Covalent

Create Azure Batch for Covalent

1. Install Terraform

sudo dnf install -y dnf-plugins-core
sudo dnf config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
sudo dnf -y install terraform

2. Install Azure CLI

Refer to this link: Install the Azure CLI on Linux | Microsoft Learn

# Take Centos8 as an example
sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc
sudo dnf install -y https://packages.microsoft.com/config/rhel/9/packages-microsoft-prod.rpm
sudo dnf install azure-cli

3. Login into Azure CLI

az login #　Follow the prompts to perform actions　
image-20231027173800955

4. Download terraform scripts for Azure Batch

Note: We just provide an fixed version scripts, it may not work properly since azure upgraded. If you encounter problems, please refer to this to prepare scripts.

Download terraform azurebatch_terraform_scripts.tar to HOME folder:

Extract the files from azurebatch_terraform_scripts.tar

tar -xvf azurebatch_terraform_scripts.tar

5. Configuration

cd $HOME/azurebatch_terraform_scripts

vi terraform.tfvars
prefix          = "my-prefix"
subscription_id = "my-subscription-id"
tenant_id       = "my-tenant-id"
vm_name         = "Standard_A1_v2"
owners          = ["my-user-id"]


vi versions.tf
required_version = "~> 1.11.2"

Note:

6. Create Azure Batch Resource

Initial terraform environment

[root@head azurebatch_terraform_scripts]# terraform init
Initializing the backend...
Initializing provider plugins...
- Reusing previous version of hashicorp/local from the dependency lock file
- Reusing previous version of hashicorp/template from the dependency lock file
- Reusing previous version of hashicorp/azurerm from the dependency lock file
- Reusing previous version of hashicorp/azuread from the dependency lock file
- Using previously-installed hashicorp/local v2.5.2
- Using previously-installed hashicorp/template v2.2.0
- Using previously-installed hashicorp/azurerm v3.117.1
- Using previously-installed hashicorp/azuread v3.3.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

generate the execution plan

[root@head azurebatch_terraform_scripts]# terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  + create
 <= read (data resources)

Terraform will perform the following actions:

.....
.....

Plan: 16 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + acr_login_server           = (known after apply)
  + covalent_azurebatch_object = (known after apply)
  + plugin_client_secret       = (sensitive value)
  + plugin_client_username     = (known after apply)
  + user_identity_resource_id  = (known after apply)

.....

create the resources

[root@head azurebatch_terraform_scripts]# terraform apply -auto-approve

.....
.....

Apply complete! Resources: 16 added, 0 changed, 0 destroyed.

Outputs:

acr_login_server = "my-acr-login-server"
covalent_azurebatch_object = <<EOT
    executor = ct.executor.AzureBatchExecutor(
        tenant_id="my-tenant-id",
        client_id="my-client-id",
        client_secret=plugin_client_secret,
        batch_account_url="my-batch-accoynt-url",
        storage_account_name="my-storage-account-name",
        pool_id="my-pool-id",
    )

EOT
plugin_client_secret = <sensitive>
plugin_client_username = "fca14a6a-f942-4ccd-a7db-740a70950834"
user_identity_resource_id = "878c2402-43af-43be-8d6e-e066d6363066"

Get the client_secret

[root@head azurebatch_terraform_scripts]# terraform output -raw plugin_client_secret
OZB8Q~YeIC5uroIuG6znMcyniTutKaKXyn46dcee

7.Add covalent executor for Azure Batch in LiCO web portal

image-20231020153104212
image-20231031164927553

In the Azure Batch Executor form, filled with data which Terraform apply outputs, then Covalent template can use Azure Batch Executor.

8. Delete Azure Batch resource

If you don’t need to use Azure Batch resource, you can use following command to delete resources.

[root@head azurebatch_terraform_scripts]# terraform destroy -auto-approve
azuread_application.batch: Refreshing state... [id=/applications/91b9081b-1cb0-4ddf-8c02-f42743824908]
azuread_service_principal.batch: Refreshing state... [id=/servicePrincipals/fe165ce7-67e8-4d56-b227-e4f336c8ba4f]
azuread_service_principal_password.covalent_plugin: Refreshing state... [id=fe165ce7-67e8-4d56-b227-e4f336c8ba4f/password/0037d532-e152-4ed6-86b1-6a0af9dc7bd8]
azurerm_role_definition.covalent_batch: Refreshing state... [id=/subscriptions/66139a44-e4f5-4135-aa86-de2152485836/providers/Microsoft.Authorization/roleDefinitions/2bbb78b7-63bf-1872-47a1-59a63a55d080|/subscriptions/66139a44-e4f5-4135-aa86-de2152485836]
azurerm_role_assignment.covalent_plugin_storage: Refreshing state...
......
......

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  - destroy

Terraform will perform the following actions:

.....
.....


azurerm_role_definition.covalent_batch: Destruction complete after 3m27s

Destroy complete! Resources: 16 destroyed.