Skip to content
Snippets Groups Projects
Unverified Commit d5480fae authored by Alex's avatar Alex Committed by GitHub
Browse files

Merge pull request #2 from Networks-it-uc3m/release-2.0

bug fixes
parents 0a035c1a 92dd6d7f
No related branches found
No related tags found
1 merge request!2repo: added new directory where utils scripts will be
...@@ -4,7 +4,7 @@ This guide details the necessary steps to install the L2S-M Kubernetes operator ...@@ -4,7 +4,7 @@ This guide details the necessary steps to install the L2S-M Kubernetes operator
# Prerequisites # Prerequisites
1. Clone the L2S-M repository in your host. This guide will assume that all commands are executed in the directory where L2S-M was downloaded. 1. Clone the L2S-M repository in your host. This guide will assume that all commands are executed insideo the L2S-M directory.
2. As a prerequisite to start with the installation of L2S-M, it is necessary to set up an IP tunnel overlay among the nodes of your k8s cluster (see [how L2S works](https://github.com/Networks-it-uc3m/L2S-M/tree/main/K8s). To do so, **the installation needs 10 vEth pairs in order to support the attachment of pods to virtual networks.** 2. As a prerequisite to start with the installation of L2S-M, it is necessary to set up an IP tunnel overlay among the nodes of your k8s cluster (see [how L2S works](https://github.com/Networks-it-uc3m/L2S-M/tree/main/K8s). To do so, **the installation needs 10 vEth pairs in order to support the attachment of pods to virtual networks.**
...@@ -13,7 +13,7 @@ This guide details the necessary steps to install the L2S-M Kubernetes operator ...@@ -13,7 +13,7 @@ This guide details the necessary steps to install the L2S-M Kubernetes operator
You can create all the vEth interfaces with the provided script using the following command: You can create all the vEth interfaces with the provided script using the following command:
```bash ```bash
sudo sh ./L2S-M/K8s/provision/veth.sh sudo sh ./K8s/provision/veth.sh
``` ```
**IMPORTANT** In order to keep the configuration after the host has been rebooted, a cron job should be written in order to use this script to create and configure the virtual interfaces. To enable its use, open (or create) a new crontab in the host: **IMPORTANT** In order to keep the configuration after the host has been rebooted, a cron job should be written in order to use this script to create and configure the virtual interfaces. To enable its use, open (or create) a new crontab in the host:
...@@ -42,7 +42,7 @@ kubectl taint nodes --all node-role.kubernetes.io/control-plane- node-role.kuber ...@@ -42,7 +42,7 @@ kubectl taint nodes --all node-role.kubernetes.io/control-plane- node-role.kuber
1. Create the virtual interface definitions using the following command: 1. Create the virtual interface definitions using the following command:
```bash ```bash
kubectl create -f ./L2S-M/K8s/interfaces_definitions kubectl create -f ./K8s/interfaces_definitions
``` ```
**NOTE:** If you are using interfaces whose definitions are not present in the virtual interfaces definitions in the folder, you must create the corresponding virtual definition in the same fashion as the VXLANs. For example, if you want to use a VPN interface called "tun0", first write the descriptor "tun0.yaml": **NOTE:** If you are using interfaces whose definitions are not present in the virtual interfaces definitions in the folder, you must create the corresponding virtual definition in the same fashion as the VXLANs. For example, if you want to use a VPN interface called "tun0", first write the descriptor "tun0.yaml":
...@@ -65,12 +65,12 @@ kubectl create -f tun0.yaml ...@@ -65,12 +65,12 @@ kubectl create -f tun0.yaml
``` ```
2. Create the Kubernetes account Service Account and apply their configuration by applying the following command: 2. Create the Kubernetes account Service Account and apply their configuration by applying the following command:
```bash ```bash
kubectl create -f ./L2S-M/operator/deploy/config/ kubectl create -f ./operator/deploy/config/
``` ```
3. Create the Kubernetes Persistent Volume by using the following kubectl command: 3. Create the Kubernetes Persistent Volume by using the following kubectl command:
```bash ```bash
kubectl create -f ./L2S-M/operator/deploy/mysql/ kubectl create -f ./operator/deploy/mysql/
``` ```
4. Before deploying the L2S-M operator, it is neccessary to label your master node as the "master" of the cluster. To do so, get the names of your Kubernetes nodes, select the master and apply the "master" label with the following command: 4. Before deploying the L2S-M operator, it is neccessary to label your master node as the "master" of the cluster. To do so, get the names of your Kubernetes nodes, select the master and apply the "master" label with the following command:
...@@ -82,13 +82,13 @@ kubectl label nodes [your-master-node] dedicated=master ...@@ -82,13 +82,13 @@ kubectl label nodes [your-master-node] dedicated=master
5. Deploy the L2S-M Controller by using the following command: 5. Deploy the L2S-M Controller by using the following command:
```bash ```bash
kubectl create -f ./L2S-M/operator/deploy/controller/ kubectl create -f ./operator/deploy/controller/
``` ```
You can check that the deployment was successful if the pod enters the "running" state using the *kubectl get pods* command. You can check that the deployment was successful if the pod enters the "running" state using the *kubectl get pods* command.
6. After the previous preparation, (make sure the controller is running) you can deploy the operator in your cluster using the YAML deployment file: 6. After the previous preparation, (make sure the controller is running) you can deploy the operator in your cluster using the YAML deployment file:
```bash ```bash
kubectl create -f ./L2S-M/operator/deploy/deployOperator.yaml kubectl create -f ./operator/deploy/deployOperator.yaml
``` ```
Once these two pods are in running state, you can finally deploy the virtual switches Once these two pods are in running state, you can finally deploy the virtual switches
...@@ -97,7 +97,7 @@ Once these two pods are in running state, you can finally deploy the virtual swi ...@@ -97,7 +97,7 @@ Once these two pods are in running state, you can finally deploy the virtual swi
**First deploying the virtual OVS Daemonset:** **First deploying the virtual OVS Daemonset:**
```bash ```bash
kubectl create -f ./L2S-M/operator/daemonset kubectl create -f ./operator/daemonset
``` ```
And check there is a pod running in each node, with ```kubectl get pods -o wide``` And check there is a pod running in each node, with ```kubectl get pods -o wide```
...@@ -106,7 +106,7 @@ And check there is a pod running in each node, with ```kubectl get pods -o wide` ...@@ -106,7 +106,7 @@ And check there is a pod running in each node, with ```kubectl get pods -o wide`
In order to connect the switches between themselves, an additional configuarion must be done. A configuration file specifying which nodes we want to connect and which IP addresses their switches have will be made, and then a script will be run in each l2sm switch, using this configuration file. In order to connect the switches between themselves, an additional configuarion must be done. A configuration file specifying which nodes we want to connect and which IP addresses their switches have will be made, and then a script will be run in each l2sm switch, using this configuration file.
a. Create a file anywhere or use the reference in ./L2S-M/operator/src/switch/sampleFile.json. In this installation, this file will be used as a reference. a. Create a file anywhere or use the reference in ./operator/src/switch/sampleFile.json. In this installation, this file will be used as a reference.
b. In this file, you will specify, using the template shown in the reference file, the name of the nodes in the cluster and the IP addresses of **the switches** running on them. For example: b. In this file, you will specify, using the template shown in the reference file, the name of the nodes in the cluster and the IP addresses of **the switches** running on them. For example:
```bash ```bash
$ kubectl get pods -o wide $ kubectl get pods -o wide
...@@ -118,7 +118,7 @@ In order to connect the switches between themselves, an additional configuarion ...@@ -118,7 +118,7 @@ In order to connect the switches between themselves, an additional configuarion
``` ```
In this example we have two nodes: l2sm1 and l2sm2, with two switches, with IP addresses 10.1.14.58 and 10.1.72.111. In this example we have two nodes: l2sm1 and l2sm2, with two switches, with IP addresses 10.1.14.58 and 10.1.72.111.
We want to connect them directly, so we modify the reference file, ./L2S-M/operator/src/switch/sampleFile.json: We want to connect them directly, so we modify the reference file, ./operator/src/switch/sampleFile.json:
```json ```json
[ [
{ {
...@@ -139,7 +139,7 @@ Note: Any number of nodes can be configured, as long as the entry is in this fil ...@@ -139,7 +139,7 @@ Note: Any number of nodes can be configured, as long as the entry is in this fil
Once this file is created, we inject it to each node using the kubectl cp command: Once this file is created, we inject it to each node using the kubectl cp command:
```bash ```bash
kubectl cp ./L2S-M/operator/src/switch/sampleFile.json <pod-name>:/etc/l2sm/switchConfig.json kubectl cp ./operator/src/switch/sampleFile.json <pod-name>:/etc/l2sm/switchConfig.json
``` ```
And then executing the script in the pod: And then executing the script in the pod:
```bash ```bash
......
...@@ -100,12 +100,12 @@ def build_db(body, logger, annotations, **kwargs): ...@@ -100,12 +100,12 @@ def build_db(body, logger, annotations, **kwargs):
db.close() db.close()
logger.info(f"Node {body['spec']['nodeName']} has been registered in the operator") logger.info(f"Node {body['spec']['nodeName']} has been registered in the operator")
@kopf.on.update('pods.v1', labels={'l2sm-component': 'l2sm-switch'}) @kopf.on.field('pods.v1', labels={'l2sm-component': 'l2sm-switch'}, field='status.podIP')
def update_db(body, logger, annotations, **kwargs): def update_db(body, logger, annotations, **kwargs):
if 'status' in body and 'podIP' in body['status']: if 'status' in body and 'podIP' in body['status']:
db = pymysql.connect(host=databaseIP,user="l2sm",password="l2sm;",db="L2SM") db = pymysql.connect(host=databaseIP,user="l2sm",password="l2sm;",db="L2SM")
cur = db.cursor() cur = db.cursor()
updateQuery = "UPDATE switches SET ip = '%s' WHERE node = '%s'" % (body['status']['podIP'], body['spec']['nodeName']) updateQuery = "UPDATE switches SET ip = '%s', OpenFlowId = NULL WHERE node = '%s'" % (body['status']['podIP'], body['spec']['nodeName'])
cur.execute(updateQuery) cur.execute(updateQuery)
db.commit() db.commit()
db.close() db.close()
...@@ -270,17 +270,28 @@ def delete_vn(spec, name, logger, **kwargs): ...@@ -270,17 +270,28 @@ def delete_vn(spec, name, logger, **kwargs):
cur = db.cursor() cur = db.cursor()
sql = "DELETE FROM networks WHERE network = '%s'" % (name) sql = "DELETE FROM networks WHERE network = '%s'" % (name)
cur.execute(sql) cur.execute(sql)
db.commit()
response = session.delete(baseControllerUrl + '/l2sm/networks/' + name)
if response.status_code == 204:
# Successful request
logger.info(f"Network has been deleted")
db.commit()
else:
# Handle errors
logger.info(f"Error: {response.status_code}")
db.close() db.close()
logger.info(f"Network has been deleted")
#DELETE DATABASE ENTRIES WHEN A NEW L2SM POD IS DELETED (A NEW NODE GETS OUT OF THE CLUSTER) #DELETE DATABASE ENTRIES WHEN A NEW L2SM SWITCH IS DELETED (A NEW NODE GETS OUT OF THE CLUSTER)
@kopf.on.delete('pods.v1', labels={'l2sm-component': 'l2sm-switch'}) @kopf.on.delete('pods.v1', labels={'l2sm-component': 'l2sm-switch'})
def remove_node(body, logger, annotations, **kwargs): def remove_node(body, logger, annotations, **kwargs):
db = pymysql.connect(host=databaseIP,user="l2sm",password="l2sm;",db="L2SM") db = pymysql.connect(host=databaseIP,user="l2sm",password="l2sm;",db="L2SM")
cur = db.cursor() cur = db.cursor()
sql = "DELETE FROM interfaces WHERE node = '%s'" % (body['spec']['nodeName']) sql = "DELETE FROM interfaces WHERE node = '%s'" % (body['spec']['nodeName'])
switchSql = "DELETE FROM switches WHERE node = '%s'" % (body['spec']['nodeName'])
cur.execute(sql) cur.execute(sql)
cur.execute(switchSql)
db.commit() db.commit()
db.close() db.close()
logger.info(f"Node {body['spec']['nodeName']} has been deleted from the cluster") logger.info(f"Node {body['spec']['nodeName']} has been deleted from the cluster")
......
...@@ -53,7 +53,7 @@ func main() { ...@@ -53,7 +53,7 @@ func main() {
err = createVxlans(configDir, nodeName) err = createVxlans(configDir, nodeName)
if err != nil { if err != nil {
fmt.Println("Vxlans not created. Error:", err) fmt.Println("Vxlans not created: ", err)
return return
} }
} }
...@@ -122,7 +122,7 @@ func createVxlans(configDir, nodeName string) error { ...@@ -122,7 +122,7 @@ func createVxlans(configDir, nodeName string) error {
/// Read file and save in memory the JSON info /// Read file and save in memory the JSON info
data, err := ioutil.ReadFile(configDir) data, err := ioutil.ReadFile(configDir)
if err != nil { if err != nil {
fmt.Println("Error reading input file:", err) fmt.Println("No input file was found.", err)
return err return err
} }
...@@ -134,33 +134,22 @@ func createVxlans(configDir, nodeName string) error { ...@@ -134,33 +134,22 @@ func createVxlans(configDir, nodeName string) error {
// Search for the corresponding node in the configuration, according to the first passed parameter. // Search for the corresponding node in the configuration, according to the first passed parameter.
// Once the node is found, create a bridge for every neighbour node defined. // Once the node is found, create a bridge for every neighbour node defined.
// The bridge is created with the nodeIp, neighborNodeIP and VNI. The VNI is generated according to the position of the node in the Json file. The first node will have the number 5 // The bridge is created with the nodeIp and neighborNodeIP and VNI. The VNI is generated in the l2sm-controller thats why its set to 'flow'.
// as a reference, the second a 6, and so on. And if a bridge between node 1 and node 2 is generated, it will have a vni of 5006, the two references with two 0s in between.
// Another example would be node 3 of the cluster and node 9. Node 3 will have the reference 7 (5+3-1), and the Node 9 the reference 13(5 + 9 -1), resulting in the VNI 70013.
// There's up to 2 ^ 24 possible vnis that are reduced to (2 ^24)/100 because of this measure (2 decimal digits are lost). So in total, a number of 167.772 virtual networks can be created.
nodeVniRef := 5
for _, node := range nodes { for _, node := range nodes {
if node.Name == nodeName { if node.Name == nodeName {
nodeIP := strings.TrimSpace(node.NodeIP) nodeIP := strings.TrimSpace(node.NodeIP)
for _, neighbor := range node.NeighborNodes { for _, neighbor := range node.NeighborNodes {
neighborVniRef := 5 vxlanNumber := 1
for _, n := range nodes { for _, n := range nodes {
if n.Name == neighbor { if n.Name == neighbor {
//var vni string
//if nodeVniRef < neighborVniRef {
// vni = fmt.Sprintf("%d00%d", nodeVniRef, neighborVniRef)
//} else {
// vni = fmt.Sprintf("%d00%d", neighborVniRef, nodeVniRef)
//}
neighborIP := strings.TrimSpace(n.NodeIP) neighborIP := strings.TrimSpace(n.NodeIP)
commandArgs := []string{ commandArgs := []string{
"add-port", "add-port",
"brtun", "brtun",
fmt.Sprintf("vxlan%d", neighborVniRef), fmt.Sprintf("vxlan%d", vxlanNumber),
"--", "--",
"set", "interface", "set", "interface",
fmt.Sprintf("vxlan%d", neighborVniRef), fmt.Sprintf("vxlan%d", vxlanNumber),
"type=vxlan", "type=vxlan",
fmt.Sprintf("options:key=flow"), fmt.Sprintf("options:key=flow"),
fmt.Sprintf("options:remote_ip=%s", neighborIP), fmt.Sprintf("options:remote_ip=%s", neighborIP),
...@@ -174,12 +163,11 @@ func createVxlans(configDir, nodeName string) error { ...@@ -174,12 +163,11 @@ func createVxlans(configDir, nodeName string) error {
fmt.Println(fmt.Sprintf("Created vxlan between node %s and node %s.", node.Name, neighbor)) fmt.Println(fmt.Sprintf("Created vxlan between node %s and node %s.", node.Name, neighbor))
} }
} }
neighborVniRef++ vxlanNumber++
} }
} }
} }
nodeVniRef++
} }
return nil return nil
} }
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment