diff --git a/README.md b/README.md index 3ca71667565ae80193302d3498a3380b15a30c29..5023b98a23b67129dfecb398492dcac30c719b77 100644 --- a/README.md +++ b/README.md @@ -9,11 +9,11 @@ L2S-M provides its intended functionalities using a programmable data-plane base The main K8s interface of pods remains intact (provided by a CNI plugin), retaining the compatibility with all the standard K8s elements (e.g., services, connectivity through the main interface, etc.). Moreover, the solution has the potential to be used for inter-cluster communications to support scenarios where network functions are spread through multiple distributed infrastructures (this is still a work in progress). -The figure outlines the design of L2S-M. See [how L2S-M works](https://github.com/Networks-it-uc3m/L2S-M/tree/main/K8s) to read further details on the L2S-M solution. +The figure outlines the design of L2S-M. See [how L2S-M works](https://github.com/Networks-it-uc3m/L2S-M/tree/main/additional-info) to read further details on the L2S-M solution. -If you want to learn how to install L2S-M in your cluster, see the [installation guide](https://github.com/Networks-it-uc3m/L2S-M/tree/main/operator) of this repository to start its installation. +If you want to learn how to install L2S-M in your cluster, see the [installation guide](https://github.com/Networks-it-uc3m/L2S-M/tree/main/deployments) of this repository to start its installation. -Did you already install the operator and you cannot wait to start building your own virtual networks in your K8s cluster? Check out our [ping-pong](https://github.com/Networks-it-uc3m/L2S-M/tree/main/descriptors) example! +Did you already install the operator and you cannot wait to start building your own virtual networks in your K8s cluster? Check out our [ping-pong](https://github.com/Networks-it-uc3m/L2S-M/tree/main/examples/ping-pong) example! If you want more information about the original idea of L2S-M and its initial design, you can check our latest publication in the [IEEE Network journal](https://ieeexplore.ieee.org/document/9740640): @@ -34,7 +34,7 @@ The solution can work jointly with L2S-M or be used standalone through the [Mult The solution enables the creation and deletion of virtual link-layer networks to connect application workloads running in different virtualization domains. This way, it supports inter-domain link-layer communications among remote workloads. ### Additional information about L2S-M -In the [following section](https://github.com/Networks-it-uc3m/L2S-M/tree/main/additional%20info) of the repository, you can find a series of documents and slides that provide additional information about L2S-M, including presentations where our solution has been showcased to the public in various events. +In the [following section](https://github.com/Networks-it-uc3m/L2S-M/tree/main/additional-info) of the repository, you can find a series of documents and slides that provide additional information about L2S-M, including presentations where our solution has been showcased to the public in various events. L2S-M has been presented in the following events: diff --git a/additional info/README.md b/additional info/README.md deleted file mode 100644 index 39f0cb5a05d19c367d587a3b3a2781b6a51ac1a7..0000000000000000000000000000000000000000 --- a/additional info/README.md +++ /dev/null @@ -1 +0,0 @@ -# Additional info for L2S-M diff --git a/additional info/OSM#13 Plenary Meeting.pdf b/additional-info/OSM#13 Plenary Meeting.pdf similarity index 100% rename from additional info/OSM#13 Plenary Meeting.pdf rename to additional-info/OSM#13 Plenary Meeting.pdf diff --git a/additional-info/README.md b/additional-info/README.md new file mode 100644 index 0000000000000000000000000000000000000000..5ae597e18362e41ec9ae011e079031545c42b977 --- /dev/null +++ b/additional-info/README.md @@ -0,0 +1,19 @@ +# How does L2S-M work? + +L2S-M takes a different approach to K8s networking in comparison with other solutions available, which mostly implement CNI plugins to be used as the main connectivity basis for a cluster. L2S-M is deployed as a complementary solution to these CNI Plugins, since it allows the creation and management of virtual networks in a K8s cluster in order to provide workloads with one (or several) interface(s) to communicate with other workloads attached to the same network(s) at the link-layer. The main CNI Plugin interface in these pods remains intact, allowing the standard K8s functionalities to still be available for the pods (services, communications using the main interface, etc.). + +The following figure outlines a high-level overview of L2S-M, with an illustrative example of a K8s cluster with L2S-M installed and running. L2S-M builds a programmable data plane using SDN switches over a K8s infrastructure. These switches can be either virtual (deployed by the L2S-M operator) or physical (such as those that can be found in a classic datacentre infrastructure). + + + +**NOTE**: The current version of L2S-M utilizes an infrastructure of virtual switches based on [Open Virtual Switch (OVS)](http://www.openvswitch.org). The integration of physical switches is currently ongoing. + +In L2S-M, a k8s node deploys a virtual SDN switch or is connected to a physical SDN switch. Virtual switches are interconnected through point-to-point links. These links are established using IP tunnels (based on VXLAN technologies). This way, SDN switches build an overlay network that interconnects all the K8s nodes. L2S-M uses an SDN controller to install forwarding rules on the virtual/physical switches. This way, data traffic among workloads is appropriately distributed through isolated virtual networks (i.e., the SDN controller instructs the switches which ports should be used to forward and/or block incoming/outgoing traffic). + +Specifically for K8s clusters, the element in charge of managing the creation, deletion and management of virtual networks is the L2S-M operator. This operator treats virtual networks as Multus CRDs, using the K8s events to detect the instances where a pod wants to attach/detach from a virtual network. In the former case, the operator will select one of the available interfaces in the SDN switch, and associate it with the virtual network that wants to be used. This interface will appear in the pod as a secondary interface that can be used to communicate with other pods attached to the network, which will be seen as if they were deployed in the same Local Area Network (LAN). The CNI interface remains intact. + +To provide isolation among virtual networks, the operator interacts with the SDN controller component to communicate which ports are associated with each virtual network, updating its status every time a pod is deployed/deleted. Using this information, the SDN controller injects the corresponding rules in the switches, forwarding and/or blocking traffic according to the virtual networks being used at each moment. + +**NOTE**: The current version of L2S-M utilizes a simple-switch implementation based on the [RYU](https://ryu.readthedocs.io/en/latest/) SDN controller. An SDN application to specifically support virtual network isolation is currently under implementation. + +More information on how to deploy virtualise workloads attached to virtual networks can be seen in the [ping-pong](https://github.com/Networks-it-uc3m/L2S-M/tree/main/examples/ping-pong) example. diff --git a/deployments/README.md b/deployments/README.md index 0e410bc6864c66d7d6a7943abbbfd4fc8d75866d..acb000c735a42113f8a5d52dec75671df1dace68 100644 --- a/deployments/README.md +++ b/deployments/README.md @@ -6,33 +6,11 @@ This guide details the necessary steps to install the L2S-M Kubernetes operator 1. Clone the L2S-M repository in your host. This guide will assume that all commands are executed within the L2S-M directory. -2. As a prerequisite to start with the installation of L2S-M, it is necessary to set up an IP tunnel overlay among the nodes of your k8s cluster (see [how L2S works](https://github.com/Networks-it-uc3m/L2S-M/tree/main/K8s). To do so, **the installation needs 10 vEth pairs in order to support the attachment of pods to virtual networks.** +2. Install the Multus CNI Plugin in your K8s cluster. For more information on how to install Multus in your cluster, check their [official GitHub repository](https://github.com/k8snetworkplumbingwg/multus-cni). - This repository contains a script to generate all the necessary interfaces with their respective names. (this is the **recommended option**). +3. The host-device CNI plugin must be able to be used in your cluster. If it is not present in your K8s distribution, you can find how to install it in your K8s cluster in their [official GitHub repository](https://github.com/containernetworking/plugins). - You can create all the vEth interfaces with the provided script using the following command: - - ```bash - sudo sh ./K8s/provision/veth.sh - ``` - - **IMPORTANT** In order to keep the configuration after the host has been rebooted, a cron job should be written in order to use this script to create and configure the virtual interfaces. To enable its use, open (or create) a new crontab in the host: - - ```bash - sudo crontab -e - ``` - - Once opened, append the following line at the end of the file: - - ```bash - @reboot sh [directory-where-L2S-M-is-located]/L2S-M/K8s/provision/veth.sh - ``` - -3. Install the Multus CNI Plugin in your K8s cluster. For more information on how to install Multus in your cluster, check their [official GitHub repository](https://github.com/k8snetworkplumbingwg/multus-cni). - -4. The host-device CNI plugin must be able to be used in your cluster. If it is not present in your K8s distribution, you can find how to install it in your K8s cluster in their [official GitHub repository](https://github.com/containernetworking/plugins). - -5. Your K8s Controller node must be able to deploy K8s pods for the operator to work. Remove its master and control-plane taints using the following command: +4. Your K8s Control-Plane node must be able to deploy K8s pods for the operator to work. Remove its master and control-plane taints using the following command: ```bash kubectl taint nodes --all node-role.kubernetes.io/control-plane- node-role.kubernetes.io/master- ``` @@ -42,35 +20,17 @@ kubectl taint nodes --all node-role.kubernetes.io/control-plane- node-role.kuber 1. Create the virtual interface definitions using the following command: ```bash -kubectl create -f ./K8s/interfaces_definitions +kubectl create -f ./deployments/custom_installation/interfaces_definitions ``` -**NOTE:** If you are using interfaces whose definitions are not present in the virtual interfaces definitions in the folder, you must create the corresponding virtual definition in the same fashion as the VXLANs. For example, if you want to use a VPN interface called "tun0", first write the descriptor "tun0.yaml": - - ```yaml -apiVersion: "k8s.cni.cncf.io/v1" -kind: NetworkAttachmentDefinition -metadata: -name: tun0 -spec: -config: '{ -"cniVersion": "0.3.0", -"type": "host-device", -"device": "tun0" -}' -``` -Afterwards, apply the new interface definitions using kubectl: - ```bash -kubectl create -f tun0.yaml -``` 2. Create the Kubernetes account Service Account and apply their configuration by applying the following command: ```bash -kubectl create -f ./operator/deploy/config/ +kubectl create -f ./deployments/custom_installation/config/ ``` 3. Create the Kubernetes Persistent Volume by using the following kubectl command: ```bash -kubectl create -f ./operator/deploy/mysql/ +kubectl create -f ./deployments/custom_installation/mysql/ ``` 4. Before deploying the L2S-M operator, it is neccessary to label your master node as the "master" of the cluster. To do so, get the names of your Kubernetes nodes, select the master and apply the "master" label with the following command: @@ -82,13 +42,13 @@ kubectl label nodes [your-master-node] dedicated=master 5. Deploy the L2S-M Controller by using the following command: ```bash -kubectl create -f ./operator/deploy/controller/ +kubectl create -f ./deployments/custom_installation/deployController.yanl ``` You can check that the deployment was successful if the pod enters the "running" state using the *kubectl get pods* command. 6. After the previous preparation, (make sure the controller is running) you can deploy the operator in your cluster using the YAML deployment file: ```bash -kubectl create -f ./operator/deploy/deployOperator.yaml +kubectl create -f ./deployments/custom_installation/deployOperator.yaml ``` Once these two pods are in running state, you can finally deploy the virtual switches @@ -97,7 +57,7 @@ Once these two pods are in running state, you can finally deploy the virtual swi **First deploying the virtual OVS Daemonset:** ```bash -kubectl create -f ./operator/daemonset +kubectl create -f ./deployments/custom_installation/deploySwitch.yaml ``` And check there is a pod running in each node, with ```kubectl get pods -o wide``` @@ -106,7 +66,7 @@ And check there is a pod running in each node, with ```kubectl get pods -o wide` In order to connect the switches between themselves, an additional configuarion must be done. A configuration file specifying which nodes we want to connect and which IP addresses their switches have will be made, and then a script will be run in each l2sm switch, using this configuration file. - a. Create a file anywhere or use the reference in ./operator/src/switch/sampleFile.json. In this installation, this file will be used as a reference. + a. Create a file anywhere or use the reference in ./configs/sampleFile.json. In this installation, this file will be used as a reference. b. In this file, you will specify, using the template shown in the reference file, the name of the nodes in the cluster and the IP addresses of **the switches** running on them. For example: ```bash $ kubectl get pods -o wide @@ -119,7 +79,7 @@ In order to connect the switches between themselves, an additional configuarion ``` In this example we have two nodes: l2sm1 and l2sm2, with two switches, with IP addresses 10.1.14.58 and 10.1.72.111. - We want to connect them directly, so we modify the reference file, ./operator/src/switch/sampleFile.json: + We want to connect them directly, so we modify the reference file, ./configs/sampleFile.json: ```json [ { @@ -158,7 +118,7 @@ Note: Any number of nodes can be configured, as long as the entry is in this fil Once this file is created, we inject it to each node using the kubectl cp command: ```bash -kubectl cp ./operator/src/switch/sampleFile.json <pod-name>:/etc/l2sm/switchConfig.json +kubectl cp ./configs/sampleFile.json <pod-name>:/etc/l2sm/switchConfig.json ``` And then executing the script in the switch-pod: ```bash @@ -181,4 +141,4 @@ Created vxlan between node l2sm2 and node l2sm1. ``` -You are all set! If you want to learn how to create virtual networks and use them in your applications, [check the following section of the repository](https://github.com/Networks-it-uc3m/L2S-M/tree/main/descriptors) +You are all set! If you want to learn how to create virtual networks and use them in your applications, [check the following section of the repository](https://github.com/Networks-it-uc3m/L2S-M/tree/main/examples/ping-pong) diff --git a/examples/README.md b/examples/README.md index d16006a39c8d79181ff789a62c59e5b826de0eca..7ba37940cbbfab9796bfb8a4fd8090e108fff174 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,23 +1,23 @@ # L2S-M Ping Pong example This section of L2S-M documentation provides an example that you can use in order to learn how to create virtual networks and attach pods to them. To do so, we are going to deploy a simple ping-pong application, where we will deploy two pods attached to a virtual network and test their connectivity. -All the necessary descriptors can be found in the *'./descriptors'* directory of this repository. +All the necessary descriptors can be found in the *'./examples/ping-pong/'* directory of this repository. This guide will assume that all commands are executed within the L2S-M directory. ### Creating our first virtual network -First of all, let's see the details of an L2S-M virtual network. This is the descriptor corresponding to the virtual network that will be used in this example: my-first-network +First of all, let's see the details of an L2S-M virtual network. This is the descriptor corresponding to the virtual network that will be used in this example: ping-network ```yaml apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: - name: my-first-network + name: ping-network spec: config: '{ "cniVersion": "0.3.0", - "type": "host-device", + "type": "dummy", "device": "l2sm-vNet" }' ``` @@ -28,7 +28,7 @@ As you can see, L2S-M virtual networks are a [NetworkAttachmentDefinition](https To create the virtual network in your cluster, use the appropriate *kubectl* command as if you were building any other K8s resource: ```bash -kubectl create -f ./descriptors/networks/myfirstnetwork.yaml +kubectl create -f ./examples/ping-pong/network.yaml ``` Et voilá! You have successfully created your first virtual network in your K8s cluster. @@ -37,22 +37,23 @@ Et voilá! You have successfully created your first virtual network in your K8s After creating our first virtual network, it is time to attach some pods to it. To do so, it is as simple as adding an annotation to your deployment/pod file, just like you would do when attaching into a multus NetworkAttachmentDefinition. -For example, to add one deployment to my-first-network, enter the following annotation in your descriptor in its metadata: +For example, to add one deployment to ping-network, enter the following annotation in your descriptor in its metadata: ```yaml annotations: - k8s.v1.cni.cncf.io/networks: my-first-network + k8s.v1.cni.cncf.io/networks: ping-network ``` If you want to add your own Multus annotations, you are free to do so! L2S-M will not interfere with the standard Multus behavior, so feel free to add your additional annotations if you need them. -To assist you with the deployment of your first application with L2S-M, you can use the deployments available in this repository. To deploy both "ping-pong" pods (which are simple Ubuntu alpine containers), use the following command: +To assist you with the deployment of your first application with L2S-M, you can use the pod definitions available in this repository. To deploy both "ping-pong" pods (which are simple Ubuntu alpine containers), use the following commands: ```bash -kubectl create -f ./descriptors/deployments/ +kubectl create -f ./examples/ping-pong/ping.yaml +kubectl create -f ./examples/ping-pong/pong.yaml ``` -After a bit of time, check that both deployments were successfully instantiated in your cluster. +After a bit of time, check that both pods were successfully instantiated in your cluster. ### Testing the connectivity @@ -109,7 +110,7 @@ ping 192.168.12.1 -I eth0 If you are tired of experimenting with the app, you can proceed to delete both deployments from the cluster: ```bash -kubectl delete deploy/ping-l2sm -kubectl delete deploy/pong-l2sm +kubectl delete ping +kubectl delete pong ``` diff --git a/examples/ping-pong/ping.yaml b/examples/ping-pong/ping.yaml index c03b0d2696a585e0d58682d3f402776af63d223b..dc86712d2c5b12c9d86b733c8720307565f0dbf3 100644 --- a/examples/ping-pong/ping.yaml +++ b/examples/ping-pong/ping.yaml @@ -3,9 +3,9 @@ kind: Pod metadata: name: ping labels: - app: test2 + app: ping-pong annotations: - k8s.v1.cni.cncf.io/networks: first-network + k8s.v1.cni.cncf.io/networks: ping-network spec: containers: - name: router @@ -15,4 +15,4 @@ spec: capabilities: add: ["NET_ADMIN"] # Use this parameter if you want to place the pod in a specific node - nodeName: l2sm1 + #nodeName: masterk8s diff --git a/examples/ping-pong/pong.yaml b/examples/ping-pong/pong.yaml index 7a62409855e2d67b1a02f627f327bde17386a63f..e9fa4abe7ca6f279f04c7bda03b4728149f73dff 100644 --- a/examples/ping-pong/pong.yaml +++ b/examples/ping-pong/pong.yaml @@ -3,9 +3,9 @@ kind: Pod metadata: name: pong labels: - app: test2 + app: ping-pong annotations: - k8s.v1.cni.cncf.io/networks: first-network + k8s.v1.cni.cncf.io/networks: ping-network spec: containers: - name: router @@ -15,4 +15,4 @@ spec: capabilities: add: ["NET_ADMIN"] # Use this parameter if you want to place the pod in a specific node - nodeName: l2sm2 + #nodeName: workerk8s