# AI4EU Experiments Onboarding Tutorial: Entity Recognition using gRPC
This tutorial provides a basic Python programmer’s introduction to working with gRPC.
By walking through this example you’ll learn how to:
* Define a service in a .proto file.
* Generate server and client code using the protocol buffer compiler.
* Use the Python gRPC API to write a simple client and server for your service.

It assumes that you have read the Overview(https://grpc.io/docs/guides/#overview) and are familiar with protocol buffers.(https://developers.google.com/protocol-buffers/docs/overview)
You can find out more in the proto3 language guide and Python generated code guide.
(https://developers.google.com/protocol-buffers/docs/reference/python-generated)

# What is gRPC?
With gRPC you can define your service once in a .proto file and implement clients and
servers in any of gRPC’s supported languages, which in turn can be run in
environments ranging from servers inside Google to your own tablet - all the
complexity of communication between different languages and environments is
handled for you by gRPC. You also get all the advantages of working with protocol
buffers, including efficient serialization, a simple IDL, and easy interface updating.
This example is a Deep Learning NER example that lets clients extract a set of entities of types Person, Location, Organization or Miscellaneous
from a given input text.

# Steps
1. Write the service to be served.
2. Make a proto file to define the messages and services.
3. Use the proto file to generate gRPC classes for Python.
4. Create the server.
5. Create the client.
6. Include a license
7. Prepare the docker file, run the docker and the run the client in a new tab.

## Step 1: Write the Service:
In our case, the service is extracting entities from text. Below is code snippet.

```python
from flair.models import SequenceTagger
from flair.data import Sentence

# predicts entities in a given sentence and returns a new sentence with highlighted entities
def predict_entity_from_text(sentence) -> str:
    sentence_tagged = Sentence(sentence)
    tagger.predict(sentence_tagged)
    new_sent = ""
    idx = 0
    for entity in sentence_tagged.get_spans('ner'):
        new_sent += sentence[idx:entity.start_pos]
        replacement = "|" + entity.text + "| (" + entity.tag + ", " + str(round(entity.score, 2)) + ")"
        new_sent += replacement
        idx = entity.end_pos

    new_sent += sentence[idx:]
    return new_sent

# load the pre-trained NER model 
tagger = SequenceTagger.load('./best-model.pt')
```
This model has only one input argument (a sentence or a group of sentences, whose entities we want to extract).

## Step 2: Make the Proto File:
```proto
//Define the used version of proto
syntax = "proto3";

#Define a text to hold the input/output text
message Text {
    string text = 1;      // Text to be processed
}

//Define the service
service Predict {
    rpc extract_entities_from_text(Text) returns (Text);
}
```
Here, we did not give values to the text, the number indicates the order of serializing
the features of the Text object.

## Step 3: Generate gRPC classes for Python:

Open the terminal, change the directory to be in the same folder that the proto file is
in.
To generate the gRPC classes we have to install the needed libraries first:

* Install gRPC :
```cmd
python3 -m pip install grpcio
```

* To install gRPC tools, run:
```commandline
python3 -m pip install grpcio-tools googleapis-common-protos
```

* Now, run the following command:
```commandline
python3 -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. model.proto
```
This command used model.proto file to generate the needed stubs to create the
client/server.
The files generated will be as follows:

model_pb2.py — contains message classes

* model_pb2.Features for the input features
* model_pb2.Prediction for the prediction price

model_pb2_grpc.py — contains server and client classes

* model_pb2_grpc.PredictServicer will be used by the server
* model_pb2_grpc.PredictStub the client will use it

## Step 4: Creating the Server:

The server will import the generated files and the functions that will handle the
entity prediction and training of new NER models. Then we will define a class that will take a request from the client and
uses the predict_entity_from_text() or start_training() functions to return a response. 
The response for entity prediction is a new text with highlighted entities, while for model training it's the train/dev-loss history and dev-score history.
After that, we will use add_FlairModelServicer_to_server function from (model_pb2_grpc.py)
file that was generated before to add the class FlairModelServicer to the server.
Once you have implemented all the methods, the next step is to start up a gRPC
server so that clients can actually use your service.
The gRPC server is expected to run on port 8061
The optional HTTP-Server for a Web-UI for human interaction is expected to run on
port 8062.

## Step 5: Creating the Client:

Please note that the below client is just to execute and test a gRPC server implemented for a single model. 
To execute a pipeline consisting of more than 1 node, kindly follow https://github.com/ai4eu/generic-serial-orchestrator

In the client file we will do the following:
* Open a gRPC channel
* Create a stub
* Create a request message
* Use the stub to call the service

Below is the code snippet for client:

```python
import logging
from timeit import default_timer as timer
import grpc
import model_pb2
import model_pb2_grpc


def run_predictor(data):
    logging.basicConfig()
    print("Calling FlairModel_Stub..")
    start_ch = timer()
    with grpc.insecure_channel('localhost:8061') as channel:
        stub = model_pb2_grpc.FlairModelStub(channel)
        ui_request = model_pb2.Text(text=data)
        print("-------------- GetEntities --------------")
        response = stub.extract_entities_from_text(ui_request)

    return response.text


if __name__ == '__main__':
    logging.basicConfig()
    text = "George Washington ging nach Washington."
    run_predictor(text)
```

## Step 6: Include a license File
We need to include a license file before building a docker image. 

## Step 7: Prepare the Docker file
```dockerfile
FROM ubuntu:18.04

MAINTAINER Andel  "andel.gugu@iais.fraunhofer.de"

RUN apt-get update -y
RUN apt-get install -y python3-pip python3-dev
RUN pip3 install --upgrade pip

COPY requirements.txt ./requirements.txt
RUN pip3 install -r requirements.txt

COPY static ./static
COPY templates ./templates
COPY license-1.0.0.json model.proto model_pb2.py model_pb2_grpc.py results.txt best-model.pt ./
COPY app.py ner_model_server.py predict.py ./


WORKDIR /

EXPOSE 8061 8062

ENTRYPOINT [ "python3","ner_model_server.py" ]
```
In the docker file, we copy the license file along with other files required to the
container.
Whenever any layer is re-built all the layers that follow it in the Dockerfile need to be
rebuilt too. It's important to keep this fact in mind while creating Dockerfiles.
The dockerfile here separates out the gRPC specific requirements in a separate file
called requirements.txt. The reason for doing this is to separate the application
dependency from the gRPC dependency. gRPC dependency in requirements.txt will be
built as a separate layer when the Docker image is built. This avoids rebuild of this
layer every time a change is made in the application. Below is the contents of gRPC
requirement.txt.

```requirements.txt
# GRPC Python setup and other requirements
torch==1.7.1
transformers==3.5.1
protobuf==3.16.0
flair==0.7
packaging~=21.3
grpcio==1.38.0
grpcio-tools==1.38.0
googleapis-common-protos==1.53.0
Bootstrap-Flask==1.5.2
Flask==1.1.2
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.14.3
google==3.0.0
WTForms==3.0.0
Jinja2==2.11.3
markupsafe==2.0.1
itsdangerous==2.0.1
werkzeug==2.0.3
```

Build the docker image

```commandline
docker build -t entity-recognizer:v1 .
```

Run the docker image

```commandline
docker run -p 8061:8061 --rm -t entity-recognizer:v1 /bin/bash
```
The -p option maps the port on the container to the host.
The Docker run internally executes ner_model_server.py.
Open one more terminal and run the client which now can access the docker server

```commandline
python3 ner_model_client.py
```

# Other modules for Entity Recognition model above
All the above 7 steps described for the NER model remains the same for the NER Training Config, Tensorboard and SharedFolder provider modules.
The source code for the training config module reads the input fields and passes the record to the model module
sequentially. The values from the input fields will be used to configure the training parameters for the NER model to be trained.

For the orchestrator to regard an edge as completed, the model should set
the below grpc status codes.
```
grpc.StatusCode.NOT_FOUND or

grpc.StatusCode.OUT_OF_RANGE
```
grpc.StatusCode.NOT_FOUND is used in the below gRPC server code for the 
data broker.