Skip to content
Snippets Groups Projects
Commit b1818220 authored by Alex de Cock Buning's avatar Alex de Cock Buning
Browse files

refactor code: removed unused directories and files

parent e4b9c4fc
No related branches found
No related tags found
1 merge request!2repo: added new directory where utils scripts will be
Showing
with 10 additions and 19602 deletions
configs/networkTopology.json
.vscode
src/kubernetes-api/dist/install.yaml
src/kubernetes-api/bin
\ No newline at end of file
tls.b64
bin/
\ No newline at end of file
......@@ -120,10 +120,10 @@ docker-buildx: ## Build and push docker image for the manager for cross-platform
.PHONY: build-installer
build-installer: manifests generate kustomize ## Generate a consolidated YAML with CRDs and deployment.
mkdir -p dist
echo "" > dist/install.yaml
echo "---" >> dist/install.yaml # Add a document separator before appending
echo "" > deployments/install.yaml
echo "---" >> deployments/install.yaml # Add a document separator before appending
cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG}
$(KUSTOMIZE) build config/default >> dist/install.yaml
$(KUSTOMIZE) build config/default >> deployments/install.yaml
##@ Deployment
......
# L2S-M Development Guide
Welcome to the L2S-M development guide. This README provides detailed instructions for setting up and developing L2S-M, which consists of four main components: `l2sm-controller`, `l2sm-operator`, `l2sm-switch`, and `l2sm-ned`. Follow the steps below to set up your development environment and deploy each component.
## Table of Contents
- [Repository Structure](#repository-structure)
- [Prerequisites](#prerequisites)
- [Component Development and Deployment](#component-development-and-deployment)
- [L2SM-Controller](#l2sm-controller)
- [L2SM-Operator](#l2sm-operator)
- [L2SM-Switch](#l2sm-switch)
## Repository Structure
Below is a brief overview of the repository's structure and the purpose of major directories:
```bash
L2S-M
├── .vscode
│ └── launch.json
├── LICENSE
├── README.md
├── deployment
| └── custom-installation
├── build
... [shortened for brevity] ...
└── src
├── controller
├── operator
└── switch
```
In the L2S-M/src directory you will find the source code of each component, which is used to build the images in L2S-M/build.
In L2S-M/build/README.md there is a guide of how to use the Dockerfiles and build the docker images referencing the code in L2S-M/src. A script has been made to ease this task.
In L2S-M/deployment/custom-installation there is a guide on how to install each component in kubernetes, that will enable the developing of specific parts of L2S-M
## Prerequisites
Before you begin, ensure you have met the following requirements:
- A kubernetes cluster you have complete access to.
- Multus installed in the cluster.
- For each component you're gonna develop, you may need specific tools and software.
- [L2S-M custom installation](../deployments/custom-installation/). Install L2S-M up to the component you want to modify/debug/develop, and come back here to check how to proceed with the installation.
## Component Development and Deployment
### L2SM-Controller
1. **Custom installation**: The source code for `l2sm-controller` is hosted in a separate repository. Refer to it to see how this component works and how to change it and deploy it manually.
2. **Configuration**: Specify the IP address the `l2sm-controller` is working on in the `deployOperator.yaml` and `deploySwitch.yaml` files, in the [custom-installation](../deployments/custom-installation/) directory.
3. **Custom Installation**: Follow the custom installation instructions exactly as described in the [custom-installation](../deployments/custom-installation/) directory.
### L2SM-Operator
>**Note:** you need python3 and the requirements specified in the [L2S-M/src/operator/requirements.txt](../src/operator/requirements.txt) to run it.
1. **Database Setup**: Run the MySQL development database using `mysql-development.yaml`.
2. **Configuration**: Update `launch.json` with the `l2sm-controller` service IP Address and the database IP Address. This file has been made to help launching the application locally.
3. **Debugging**: In Visual Studio Code, run the debug Kopf application. It will launch the app in a terminal, but it doesn't allow actual debugging tools such as custom breakpoints, as it's not a feature in kopf applications.
### L2SM-Switch
1. **Deployment**: Deploy `l2sm-switch` normally, ensuring to comment out `initContainers` in the YAML file. Remove the initial configuration script by using as input spec.container.args: ["sleep infinity"]
2. **Debugging**: For debugging, remove the initial configuration script by and use `exec -it` on the pods to achieve the desired configuration. Since it doesn’t run any background process, no specific image is needed, the current one implements custom commands that enable the current configuration, you can check in the script [L2S-M/src/switch/setup_switch.sh](../src/switch/setup_switch.sh) how the configuration is made.
# L2S-M KAPI
The "l2s-m kapi" component is essentially a set of Custom Resource Definitions (CRDs) accompanied by a controller and a manager. It's designed to manage the overlays and virtual networks that L2S-M uses between pods within a K8s cluster. These virtual networks facilitate isolated link-layer connectivity among pods, enhancing security and network efficiency.
## Description
# L2S-M Development
This component is essentially a set of Custom Resource Definitions (CRDs) accompanied by a controller and a manager. It's designed to manage the overlays and virtual networks that L2S-M uses between pods within a K8s cluster.
Link-Layer Secure connectivity for Microservice platforms (L2S-M) is a K8s networking solution that complements the CNI plugin approach of K8s to create and manage virtual networks in K8s clusters. These virtual networks allow workloads (pods) to have isolated link-layer connectivity with other pods in a K8s cluster, regardless of the k8s node where they are actually deployed. L2S-M enables the creation/deletion of virtual networks on-demand, as well as attaching/detaching pods to that networks. [More info can be found in the original repo.](https://github.com/Networks-it-uc3m/L2S-M)
## Getting Started
### Prerequisites
......@@ -85,29 +84,5 @@ its dependencies.
Users can just run kubectl apply -f <URL for YAML BUNDLE> to install the project, i.e.:
```sh
kubectl apply -f https://raw.githubusercontent.com/<org>/l2network/<tag or branch>/dist/install.yaml
```
## Contributing
// TODO(user): Add detailed information on how you would like others to contribute to this project
**NOTE:** Run `make help` for more information on all potential `make` targets
More information can be found via the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html)
## License
Copyright 2024.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
kubectl apply -f https://raw.githubusercontent.com/<org>/l2network/<tag or branch>/deployments/install.yaml
```
\ No newline at end of file
{
"Nodes": [
{"name": "l2sm1", "nodeIP": "10.1.14.29"},{"name": "l2sm2", "nodeIP": "10.1.72.109"}
],
"Links": [
{"endpointA": "l2sm1", "endpointB": "l2sm2"}
]
}
[
{
"name": "l2sm1",
"nodeIP": "10.1.14.34",
"neighborNodes": ["l2sm2"]
},
{
"name": "l2sm2",
"nodeIP": "10.1.72.84",
"neighborNodes": ["l2sm1"]
}
]
{
"Nodes": [
{"name": "l2sm1", "nodeIP": "10.1.14.29"},{"name": "l2sm2", "nodeIP": "10.1.72.109"}
],
"Links": [
{"endpointA": "l2sm1", "endpointB": "l2sm2"}
]
}
This diff is collapsed.
#!/bin/bash
# Path to the configuration file
configFilePath="./configs/switchConfig.json"
# Use kubectl to get pods and grep for those starting with 'l2sm-switch'
# Then, use awk to extract the pod name and node name by matching an IP pattern and taking the next column as the node
mapfile -t podInfo < <(kubectl get pods -o wide | grep 'l2sm-switch' | awk '{for (i=1; i<=NF; i++) { if ($i ~ /^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$/) { print $1 ":" $(i+1); break; } }}')
# Declare an associative array to store pod names and their respective nodes
declare -A pods
# Fill the associative array with pod names as keys and node names as values
for entry in "${podInfo[@]}"
do
podName="${entry%%:*}"
nodeName="${entry##*:}"
pods["$podName"]=$nodeName
done
# Loop through the array
for pod in "${!pods[@]}"
do
nodeName=${pods[$pod]}
# Copy the configuration file to the pod
echo "Copying config file to $pod..."
kubectl cp "$configFilePath" "$pod:/etc/l2sm/switchConfig.json"
# Execute the script inside the pod
echo "Executing configuration script on $pod..."
kubectl exec -it "$pod" -- /bin/bash -c "l2sm-vxlans --node_name=$nodeName /etc/l2sm/switchConfig.json"
done
echo "Configuration deployment completed."
#!/bin/bash
# Filename for the JSON config
jsonConfigPath="./configs/switchConfig.json"
# Use kubectl to get pods, grep for those starting with 'l2sm-switch', and parse with awk
mapfile -t podInfo < <(kubectl get pods -o wide | grep 'l2sm-switch' | awk '{for (i=1; i<=NF; i++) { if ($i ~ /^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$/) { print $(i+1) ":" $i; break; } }}')
# Associative array to store node names and their respective IPs
declare -A nodeIPs
# Fill the associative array with node names as keys and IPs as values
for entry in "${podInfo[@]}"
do
nodeName="${entry%%:*}"
nodeIP="${entry##*:}"
nodeIPs["$nodeName"]=$nodeIP
done
# Prepare Nodes array
nodesArray=()
for nodeName in "${!nodeIPs[@]}"
do
nodesArray+=("{\"name\": \"$nodeName\", \"nodeIP\": \"${nodeIPs[$nodeName]}\"}")
done
# Prepare Links array
linksArray=()
declare -A linkPairs
for srcNode in "${!nodeIPs[@]}"
do
for dstNode in "${!nodeIPs[@]}"
do
if [[ "$srcNode" != "$dstNode" && -z ${linkPairs["$srcNode:$dstNode"]} && -z ${linkPairs["$dstNode:$srcNode"]} ]]; then
linksArray+=("{\"endpointA\": \"$srcNode\", \"endpointB\": \"$dstNode\"}")
linkPairs["$srcNode:$dstNode"]=1
fi
done
done
# Generate JSON output
echo "{
\"Nodes\": [
$(IFS=,; echo "${nodesArray[*]}")
],
\"Links\": [
$(IFS=,; echo "${linksArray[*]}")
]
}" > $jsonConfigPath
echo "Network topology configuration has been generated at $jsonConfigPath."
#!/bin/bash
# Define the directory containing the YAML files
DIRECTORY="./deployments/custom-installation"
# Define the output file
OUTPUT_FILE="./deployments/l2sm-deployment.yaml"
# Start with an empty output file
> "$OUTPUT_FILE"
# Find all YAML files within the directory and all subdirectories,
# concatenate their contents into the output file and add '---' after each file's content
find "$DIRECTORY" -type f -name '*.yaml' | sort | while read file; do
cat "$file" >> "$OUTPUT_FILE"
echo "---" >> "$OUTPUT_FILE"
done
echo "All YAML files, including those within subdirectories, have been concatenated into $OUTPUT_FILE with delimiters."
File deleted
1. Modify the api/v1/ned.proto
2. Generate protofiles:
```bash
protoc -I=api/v1 --go_out=paths=source_relative:./pkg/nedpb --go-grpc_out=paths=source_relative:./pkg/nedpb api/v1/ned.proto
```
\ No newline at end of file
package v1
type NedSettings struct {
ConfigDir string
ControllerIP string
NodeName string
NedName string
}
syntax = "proto3";
package nedpb;
option go_package = "github.com/Networks-it-uc3m/l2sm-switch/pkg/nedpb";
service NedService {
// Creates a VxLAN with the specified IP address.
rpc CreateVxlan(CreateVxlanRequest) returns (CreateVxlanResponse);
// Attaches the specified interface to the bridge.
rpc AttachInterface(AttachInterfaceRequest) returns (AttachInterfaceResponse);
// Returns this neds node name
rpc GetNodeName(GetNodeNameRequest) returns (GetNodeNameResponse);
}
message CreateVxlanRequest {
// The IP address to attach to the VxLAN.
string ip_address = 1;
}
message CreateVxlanResponse {
// Indicates if the VxLAN was created successfully.
bool success = 1;
// Optional message providing additional information.
string message = 2;
}
message AttachInterfaceRequest {
// The name of the interface to attach to the bridge.
string interface_name = 1;
}
message AttachInterfaceResponse {
// The OpenFlow ID of the attached interface.
int64 interface_num = 1;
}
message GetNodeNameRequest {
}
message GetNodeNameResponse {
string node_name = 1;
}
\ No newline at end of file
package v1
type Node struct {
Name string `json:"name"`
NodeIP string `json:"nodeIP"`
NeighborNodes []string `json:"neighborNodes,omitempty"`
}
type Link struct {
EndpointNodeA string `json:"endpointA"`
EndpointNodeB string `json:"endpointB"`
}
type Topology struct {
Nodes []Node `json:"Nodes"`
Links []Link `json:"Links"`
}
package main
import (
"errors"
"flag"
"fmt"
"github.com/Networks-it-uc3m/l2sm-switch/pkg/ovs"
)
// Script that takes two required arguments:
// the first one is the name in the cluster of the node where the script is running
// the second one is the path to the configuration file, in reference to the code.
func main() {
portName, err := takeArguments()
bridge := ovs.FromName("brtun")
if err != nil {
fmt.Println("Error with the arguments. Error:", err)
return
}
err = bridge.AddPort(portName)
if err != nil {
fmt.Println("Port not added: ", err)
return
}
}
func takeArguments() (string, error) {
portName := flag.String("port_name", "", "port you want to add. Required.")
flag.Parse()
if *portName == "" {
return "", errors.New("port name is not defined")
}
return *portName, nil
}
package main
import (
"errors"
"flag"
"fmt"
"os/exec"
"regexp"
"github.com/Networks-it-uc3m/l2sm-switch/pkg/ovs"
)
// Script that takes two required arguments:
// the first one is the name in the cluster of the node where the script is running
// the second one is the path to the configuration file, in reference to the code.
func main() {
vethNumber, controllerIP, switchName, err := takeArguments()
if err != nil {
fmt.Println("Error with the arguments. Error:", err)
return
}
fmt.Println("Initializing switch, connected to controller: ", controllerIP)
bridge, err := initializeSwitch(switchName, controllerIP)
if err != nil {
fmt.Println("Could not initialize switch. Error:", err)
return
}
fmt.Println("Switch initialized and connected to the controller.")
// Set all virtual interfaces up, and connect them to the tunnel bridge:
for i := 1; i <= vethNumber; i++ {
veth := fmt.Sprintf("net%d", i)
if err := bridge.AddPort(veth); err != nil {
fmt.Println("Error:", err)
}
}
fmt.Printf("Switch initialized, current state: ", bridge)
}
func takeArguments() (int, string, string, error) {
vethNumber := flag.Int("n_veths", 0, "number of pod interfaces that are going to be attached to the switch")
controllerIP := flag.String("controller_ip", "", "ip where the SDN controller is listening using the OpenFlow13 protocol. Required")
switchName := flag.String("switch_name", "", "name of the switch that will be used to set a custom datapath id. If not set, a randome datapath will be assigned")
flag.Parse()
switch {
case *controllerIP == "":
return 0, "", "", errors.New("controller IP is not defined")
}
return *vethNumber, *controllerIP, *switchName, nil
}
func initializeSwitch(switchName, controllerIP string) (ovs.Bridge, error) {
re := regexp.MustCompile(`\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b`)
if !re.MatchString(controllerIP) {
out, _ := exec.Command("host", controllerIP).Output()
controllerIP = re.FindString(string(out))
}
controller := fmt.Sprintf("tcp:%s:6633", controllerIP)
datapathId := ovs.GenerateDatapathID(switchName)
bridge, err := ovs.NewBridge(ovs.Bridge{Name: "brtun", Controller: controller, Protocol: "OpenFlow13", DatapathId: datapathId})
return bridge, err
}
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"net"
"os"
"github.com/Networks-it-uc3m/l2sm-switch/pkg/ovs"
)
type Node struct {
Name string `json:"name"`
NodeIP string `json:"nodeIP"`
NeighborNodes []string `json:"neighborNodes,omitempty"`
}
type Link struct {
EndpointNodeA string `json:"endpointA"`
EndpointNodeB string `json:"endpointB"`
}
type Topology struct {
Nodes []Node `json:"Nodes"`
Links []Link `json:"Links"`
}
// Script that takes two required arguments:
// the first one is the name in the cluster of the node where the script is running
// the second one is the path to the configuration file, in reference to the code.
func main() {
//configDir, _, fileType, err := takeArguments()
configDir, nodeName, fileType, err := takeArguments()
bridge := ovs.FromName("brtun")
if err != nil {
fmt.Println("Error with the arguments. Error:", err)
return
}
switch fileType {
case "topology":
var topology Topology
err = readFile(configDir, &topology)
if err != nil {
fmt.Println("Error with the provided file. Error:", err)
return
}
fmt.Println(topology)
err = createTopology(bridge, topology, nodeName)
case "neighbors":
var node Node
err := readFile(configDir, &node)
if err != nil {
fmt.Println("Error with the provided file. Error:", err)
return
}
err = connectToNeighbors(bridge, node)
if err != nil {
fmt.Println("Could not connect neighbors: ", err)
return
}
}
if err != nil {
fmt.Println("Vxlans not created: ", err)
return
}
}
func takeArguments() (string, string, string, error) {
configDir := os.Args[len(os.Args)-1]
nodeName := flag.String("node_name", "", "name of the node the script is executed in. Required.")
fileType := flag.String("file_type", "topology", "type of filed passed as an argument. Can either be topology or neighbors. Default: topology.")
flag.Parse()
switch {
case *nodeName == "":
return "", "", "", errors.New("node name is not defined")
case *fileType != "topology" && *fileType != "neighbors":
return "", "", "", errors.New("file type not supported. Available types: 'topology' and 'neighbors'")
case configDir == "":
return "", "", "", errors.New("config directory is not defined")
}
return configDir, *nodeName, *fileType, nil
}
/*
*
Example:
{
"Nodes": [
{
"name": "l2sm1",
"nodeIP": "10.1.14.53"
},
{
"name": "l2sm2",
"nodeIP": "10.1.14.90"
}
],
"Links": [
{
"endpointA": "l2sm1",
"endpointB": "l2sm2"
}
]
}
*/
func createTopology(bridge ovs.Bridge, topology Topology, nodeName string) error {
nodeMap := make(map[string]string)
for _, node := range topology.Nodes {
var nodeIP string
if net.ParseIP(nodeIP) != nil {
nodeIP = node.NodeIP
} else {
ips, err := net.LookupHost(node.NodeIP)
if err != nil || len(ips) == 0 {
fmt.Printf("Failed to resolve %s\n", node.NodeIP)
continue
}
nodeIP = ips[0]
}
nodeMap[node.Name] = nodeIP
}
localIp := nodeMap[nodeName]
for vxlanNumber, link := range topology.Links {
vxlanId := fmt.Sprintf("vxlan%d", vxlanNumber)
var remoteIp string
switch nodeName {
case link.EndpointNodeA:
remoteIp = nodeMap[link.EndpointNodeB]
case link.EndpointNodeB:
remoteIp = nodeMap[link.EndpointNodeA]
default:
continue
}
err := bridge.CreateVxlan(ovs.Vxlan{VxlanId: vxlanId, LocalIp: localIp, RemoteIp: remoteIp, UdpPort: "7000"})
if err != nil {
return fmt.Errorf("could not create vxlan between node %s and node %s. Error:%s", link.EndpointNodeA, link.EndpointNodeB, err)
} else {
fmt.Printf("Created vxlan between node %s and node %s.\n", link.EndpointNodeA, link.EndpointNodeB)
}
}
return nil
}
func readFile(configDir string, dataStruct interface{}) error {
/// Read file and save in memory the JSON info
data, err := os.ReadFile(configDir)
if err != nil {
fmt.Println("No input file was found.", err)
return err
}
err = json.Unmarshal(data, &dataStruct)
if err != nil {
return err
}
return nil
}
/*
*
Example:
{
"Name": "l2sm1",
"nodeIP": "10.1.14.53",
"neighborNodes":["10.4.2.3","10.4.2.5"]
}
*/
func connectToNeighbors(bridge ovs.Bridge, node Node) error {
for vxlanNumber, neighborIp := range node.NeighborNodes {
vxlanId := fmt.Sprintf("vxlan%d", vxlanNumber)
err := bridge.CreateVxlan(ovs.Vxlan{VxlanId: vxlanId, LocalIp: node.NodeIP, RemoteIp: neighborIp, UdpPort: "7000"})
if err != nil {
return fmt.Errorf("could not create vxlan with neighbor %s", neighborIp)
} else {
fmt.Printf("Created vxlan with neighbor %s", neighborIp)
}
}
return nil
}
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment