Commit e9f3a5fe authored by Martin Lowe's avatar Martin Lowe 🇨🇦
Browse files

Merge branch 'dev' into 'master'

Update to member org to use web/print logos

See merge request !442
parents 39f70cf4 dec703d1
Pipeline #1526 passed with stage
in 0 seconds
......@@ -79,4 +79,6 @@ docker.secret.properties
.env
#Test resources
src/test/resources/schemas
\ No newline at end of file
src/test/resources/schemas
/.apt_generated/
/.apt_generated_tests/
compile-java: validate-spec;
mvn compile package
compile-java-quick: validate-spec;
mvn compile package -DskipTests
mvn compile package -Dmaven.test.skip=true
compile-react: install-react;
yarn --cwd src/main/www build
compile: clean compile-react compile-java;
......
......@@ -79,6 +79,8 @@ quarkus.datasource.jdbc.url = jdbc:mariadb://<host><:port?>/<databaseName>
Once this is set, set the `quarkus.datasource.username` and `quarkus.datasource.password` fields to the user with access to the given database in the `secret.properties` file.
The above will need to be repeated for the database used to access the EclipseDB data. This data is a separately managed datasource that is connected to to read and update organization information. The base definitions of these tables as required for the connection of this API are defined under `./src/main/resources/sql/` in the files `rem_ddl.sql` and `eclipsedb_ddl.sql`.
The other half of secret configuration is setting up the OIDC credentials for connecting to a keycloak server. This server will require a realm to be set up for access. Using the name `rem_realm` is easiest as it requires no changes to the configuration to work.
The `quarkus.oidc.auth-server-url` property in the `secret.properties` file will need to be updated. The value set should be the public realm address for your server and realm. The rest of the endpoints will be taken care of by the wellknown endpoint available in Keycloak, and don't need to be configured. For the dockerized service, this should be set to your local IP address (note, not your public address). This can be retrieved from your IP configuration application and added in the format displayed in the `sample.secret.properties` file.
......@@ -87,10 +89,11 @@ Inside that realm, create a client and update the `quarkus.oidc.client-id` prope
With these properties updated, the server should be able to start and authenticate properly. If the 3 users mentioned within the OIDC configuration section were added, the data should be accessible in a way that is comparable to how it would be in production.
As a side note, regeneration of the database on start along with the insertion of data into the database can be disabled for development environments by setting the following fields within `src/main/resources/application.properties`:
To enable connections to the FoundationDB API, a second client will need to be created in the same realm as was created for the FoundationDB API service. This second client will be set up similarly to the first, but have service accounts enabled. Once enabled, roles will need to be set within the service account giving all org related roles, as well as sys read access. This should properly restrict service access to the API.
As a side note, the insertion of data into the database can be enabled/disabled for development environments by setting the following fields within `src/main/resources/application.properties`:
1. Setting `%dev.eclipse.dataloader.enabled` to false. This property is what enables the Data bootstrap to load in mock data.
2. Removing the `%dev.quarkus.hibernate-orm.database.generation` property or commenting it out. This is what resets the database to empty on start.
1. Setting `%dev.eclipse.dataloader.enabled` to true/false. This property is what enables the Data bootstrap to load in mock data.
[^ Top](#react-eclipsefdn-members)
### Running
......@@ -197,6 +200,8 @@ Clients tab allows you to manage list of allowed applications.
To create a client, click on `Clients` in the left menu. You can set the client_id to `rem_app` and the `Root URL` to `http://localhost:3000`. Make sure that the `Client Protocol` is set to `openid-connect` and the `Access Type` is set to `confidential`.
An additional client will be required for the FoundationDB API access. Information on setting up this client should be defined under the FoundationDB API README file. Once the client is acquired, it will need to have its client ID and secret set in the secret.properties file. They will be respectively set under the properties `quarkus.oidc-client.client-id` and `quarkus.oidc-client.credentials.secret`. The URL of the client within the FoundationDB API realm will need to also be set within the secret.properties under the `quarkus.oidc-client.auth-server-url` property.
[^ Top](#react-eclipsefdn-members)
## Contributing
......
quarkus.datasource.jdbc.url = jdbc:mariadb://<server + host to mariadb instance>/<staging db name>
quarkus.datasource.username = sample
quarkus.datasource.password = sample
quarkus.oauth2.introspection-url=http://accounts.eclipse.org/oauth2/introspect
quarkus.oauth2.client-id=sample
quarkus.oauth2.client-secret=sample
quarkus.http.port=8095
\ No newline at end of file
......@@ -32,6 +32,10 @@ server {
etag off;
}
location /organization/images {
alias /usr/share/nginx/imagestore;
}
location / {
root /usr/share/nginx/html;
index index.html index.htm;
......
eclipse.internal.docker-host=host.docker.internal
eclipse.app.base-url=https://www.rem.docker/
security.token.salt=somesaltvalue
# JDBC connection values
quarkus.datasource.username=sample
quarkus.datasource.password=sample
quarkus.datasource.jdbc.url=jdbc:mariadb://${eclipse.internal.docker-host}/rem_quarkus_api
quarkus.datasource."eclipsedb".username = sample
quarkus.datasource."eclipsedb".password = sample
quarkus.datasource."eclipsedb".jdbc.url=jdbc:mariadb://${eclipse.internal.docker-host}/eclipse
quarkus.oidc.auth-server-url=http://host.docker.internal:8080/auth/realms/rem_realm
quarkus.datasource.jdbc.url = jdbc:mariadb://host.docker.internal/rem_quarkus_api
quarkus.oidc.client-id=sample
# OIDC values
# standard client
quarkus.oidc.auth-server-url=http://${eclipse.internal.docker-host}:8080/auth/realms/rem_realm
quarkus.oidc.credentials.client-secret.value=sample
security.token.salt=somesaltvalue
eclipse.app.base-url=https://www.rem.docker/
quarkus.oidc.client-id=sample
# service account
quarkus.oidc-client.auth-server-url=http://${eclipse.internal.docker-host}:8080/auth/realms/fdbapi
quarkus.oidc-client.credentials.secret=sample
quarkus.oidc-client.client-id=sample
## Used to send mail through the EclipseFdn smtp connection
quarkus.mailer.password=YOURGENERATEDAPPLICATIONPASSWORD
......
......@@ -10,6 +10,8 @@ services:
environment:
- VIRTUAL_HOST=nginx.rem.docker
- VIRTUAL_PORT=8080
volumes:
- ./volumes/imagestore:/usr/share/nginx/imagestore
nodejs:
build: ./src/main/www
image: eclipsefdn/membership-www:latest
......@@ -26,7 +28,7 @@ services:
build:
context: .
dockerfile: ./src/main/docker/Dockerfile.jvm
image: eclipsefdn/membership-rest-api:latest
#image: eclipsefdn/membership-rest-api:latest
ports:
- 8090
environment:
......@@ -35,6 +37,7 @@ services:
- VIRTUAL_PORT=8090
volumes:
- ./config/secret.properties:/var/run/secrets/secret.properties
- ./volumes/imagestore:/app/organization/imagestore
deploy:
restart_policy:
condition: on-failure
......@@ -42,6 +45,19 @@ services:
depends_on:
- mariadb
- keycloak
- foundationdb
foundationdb:
image: eclipsefdn/foundationdb-api:staging-d8e9371-9
ports:
- '8095:8095'
environment:
- CONFIG_SECRET_PATH=/var/run/secrets/secret.properties
volumes:
- ./config/foundationdb:/var/run/secrets
deploy:
restart_policy:
condition: on-failure
max_attempts: 5
mariadb:
image: mariadb:latest
command: --max_allowed_packet=100000000
......
......@@ -6,18 +6,21 @@
<artifactId>react-container</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
<eclipse-api-version>0.2-SNAPSHOT</eclipse-api-version>
<eclipse-api-version>0.4-SNAPSHOT</eclipse-api-version>
<surefire-plugin.version>2.22.1</surefire-plugin.version>
<compiler-plugin.version>3.8.1</compiler-plugin.version>
<maven.compiler.target>11</maven.compiler.target>
<maven.compiler.source>11</maven.compiler.source>
<quarkus.platform.artifact-id>quarkus-universe-bom</quarkus.platform.artifact-id>
<quarkus.platform.group-id>io.quarkus</quarkus.platform.group-id>
<quarkus.platform.version>1.13.7.Final</quarkus.platform.version>
<quarkus.platform.version>2.1.4.Final</quarkus.platform.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>${project.build.sourceEncoding}</project.reporting.outputEncoding>
<maven.compiler.parameters>true</maven.compiler.parameters>
<openhtml.version>1.0.9</openhtml.version>
<auto-value.version>1.8.2</auto-value.version>
<hibernate.version>5.5.6.Final</hibernate.version>
<org.mapstruct.version>1.4.1.Final</org.mapstruct.version>
</properties>
<repositories>
<repository>
......@@ -64,6 +67,10 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-oidc</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-oidc-client-filter</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-rest-client</artifactId>
......@@ -74,7 +81,11 @@
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-resteasy-jsonb</artifactId>
<artifactId>quarkus-resteasy-multipart</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-resteasy-jackson</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
......@@ -100,28 +111,59 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-hibernate-validator</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-text</artifactId>
<version>1.9</version>
</dependency>
<dependency>
<!-- ALWAYS required, usually included transitively. -->
<groupId>com.openhtmltopdf</groupId>
<artifactId>openhtmltopdf-core</artifactId>
<version>${openhtml.version}</version>
</dependency>
<dependency>
<!-- Required for PDF output. -->
<groupId>com.openhtmltopdf</groupId>
<artifactId>openhtmltopdf-pdfbox</artifactId>
<version>${openhtml.version}</version>
</dependency>
<dependency>
<!-- Required for image output only. -->
<groupId>com.openhtmltopdf</groupId>
<artifactId>openhtmltopdf-java2d</artifactId>
<version>${openhtml.version}</version>
</dependency>
<!-- Required for PDF output. -->
<dependency>
<groupId>com.openhtmltopdf</groupId>
<artifactId>openhtmltopdf-core</artifactId>
<version>${openhtml.version}</version>
</dependency>
<dependency>
<groupId>com.openhtmltopdf</groupId>
<artifactId>openhtmltopdf-pdfbox</artifactId>
<version>${openhtml.version}</version>
</dependency>
<dependency>
<groupId>com.openhtmltopdf</groupId>
<artifactId>openhtmltopdf-java2d</artifactId>
<version>${openhtml.version}</version>
</dependency>
<!-- Annotation preprocessors - reduce all of the boiler plate -->
<dependency>
<groupId>com.google.auto.value</groupId>
<artifactId>auto-value</artifactId>
<version>${auto-value.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.auto.value</groupId>
<artifactId>auto-value-annotations</artifactId>
<version>${auto-value.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>org.mapstruct</groupId>
<artifactId>mapstruct</artifactId>
<version>${org.mapstruct.version}</version>
</dependency>
<dependency>
<groupId>org.mapstruct</groupId>
<artifactId>mapstruct-processor</artifactId>
<version>${org.mapstruct.version}</version>
<scope>provided</scope>
</dependency>
<!-- Testing dependencies only -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-junit5</artifactId>
......@@ -137,6 +179,11 @@
<artifactId>quarkus-test-security</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-test-security-oidc</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-test-oidc-server</artifactId>
......@@ -152,6 +199,28 @@
<artifactId>quarkus-junit5-mockito</artifactId>
<scope>test</scope>
</dependency>
<!-- Following H2/devservices deps are made to circumvent need for docker -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-devservices-h2</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-jdbc-h2</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>test</scope>
</dependency>
<!-- Flyway specific dependencies, used to setup tables in test -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-flyway</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
......@@ -174,13 +243,25 @@
<artifactId>maven-compiler-plugin</artifactId>
<version>${compiler-plugin.version}</version>
<configuration>
<parameters>${maven.compiler.parameters}</parameters>
<annotationProcessorPaths>
<path>
<groupId>com.google.auto.value</groupId>
<artifactId>auto-value</artifactId>
<version>${auto-value.version}</version>
</path>
<path>
<groupId>org.mapstruct</groupId>
<artifactId>mapstruct-processor</artifactId>
<version>${org.mapstruct.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire-plugin.version}</version>
<configuration>
<skipTests>false</skipTests>
<systemPropertyVariables>
<java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
<maven.home>${maven.home}</maven.home>
......
This diff is collapsed.
package org.eclipsefoundation.api;
import java.io.InputStream;
import java.net.URI;
import java.util.LinkedList;
import java.util.List;
import java.util.function.IntFunction;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import javax.inject.Inject;
import javax.inject.Singleton;
import javax.ws.rs.core.Response;
import org.jboss.resteasy.spi.LinkHeader;
import com.fasterxml.jackson.databind.ObjectMapper;
/**
* Serves as a middleware for the FoundationDB api. This generalizes retrieving multiple pages of data using the Link
* header rather than manually iterating over the data or using less robust checks (such as no data in response).
*
* @author Martin Lowe
*
*/
@Singleton
public class APIMiddleware {
private static final Pattern PAGE_QUERY_STRING_PARAM_MATCHER = Pattern.compile(".*[\\?&]?page=(\\d+).*");
@Inject
ObjectMapper objectMapper;
/**
* Returns the full list of data for the given API call, using a function that accepts the page number to iterate
* over the pages of data. The Link header is scraped off of the first request and is used to determine how many
* calls need to be made to get the full data set.
*
* @param <T> the type of data that is retrieved
* @param supplier function that accepts a page number and makes an API call for the given page.
* @param type class for the entity type returned for the response.
* @return the full list of results for the given endpoint.
*/
public <T> List<T> getAll(IntFunction<Response> supplier, Class<T> type) {
// get initial response
int count = 1;
Response r = supplier.apply(count);
// get the Link response header values
int lastPage = getLastPageIndex(r);
List<T> out = new LinkedList<>();
readInValue(out, r, type);
// iterate through all pages before returning results
while (++count <= lastPage) {
r = supplier.apply(count);
readInValue(out, r, type);
}
return out;
}
@SuppressWarnings({ "unchecked" })
private <T> void readInValue(List<T> out, Response r, Class<T> type) {
Object o = r.getEntity();
try {
if (o instanceof InputStream) {
out.addAll(objectMapper.readerForListOf(type).readValue((InputStream) o));
} else if (type.isAssignableFrom(o.getClass())) {
out.add((T) o);
} else if (o instanceof List) {
out.addAll((List<T>) o);
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private int getLastPageIndex(Response r) {
List<Object> links = r.getHeaders().get("Link");
// convert it to retrieve the index of the last page
if (links != null && !links.isEmpty()) {
LinkHeader h = LinkHeader.valueOf(links.get(0).toString());
// check what the last page link has as its 'page' param val
URI lastLink = h.getLinkByRelationship("last").getUri();
Matcher m = PAGE_QUERY_STRING_PARAM_MATCHER.matcher(lastLink.getQuery());
if (m.matches()) {
return Integer.parseInt(m.group(1));
}
}
return 0;
}
}
package org.eclipsefoundation.api;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;
import org.eclipsefoundation.api.model.CBISponsorships;
@Path("cbi/sponsorships")
@Produces(MediaType.APPLICATION_JSON)
@RegisterRestClient(configKey = "ef-api")
public interface CBISponsorshipAPI {
@GET
CBISponsorships getSponsorships();
}
package org.eclipsefoundation.api;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import javax.inject.Singleton;
import org.eclipsefoundation.core.namespace.UrlParameterNamespace;
@Singleton
public class FoundationDBParameterNames implements UrlParameterNamespace {
public static final UrlParameter USER_NAME = new UrlParameter("username");
public static final UrlParameter RELATION = new UrlParameter("relation");
public static final UrlParameter ORGANIZATION_ID = new UrlParameter("organizationID");
public static final UrlParameter TYPE = new UrlParameter("type");
private static final List<UrlParameter> params = Collections.unmodifiableList(Arrays.asList(USER_NAME, RELATION, ORGANIZATION_ID, TYPE));
@Override
public List<UrlParameter> getParameters() {
return new ArrayList<>(params);
}
}
package org.eclipsefoundation.api;
import java.util.List;
import javax.annotation.Nullable;
import javax.annotation.security.RolesAllowed;
import javax.ws.rs.BeanParam;
import javax.ws.rs.DELETE;
import javax.ws.rs.GET;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;
import org.eclipsefoundation.api.model.OrganizationContactData;
import org.eclipsefoundation.api.model.OrganizationData;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder;
import com.google.auto.value.AutoValue;
import io.quarkus.oidc.client.filter.OidcClientFilter;
import io.quarkus.security.Authenticated;
@OidcClientFilter
@Authenticated
@Path("organizations")
@Produces(MediaType.APPLICATION_JSON)
@RegisterRestClient(configKey = "fdndb-api")
public interface OrganizationAPI {
@GET
@RolesAllowed("fdb_read_organization")
Response getOrganizations(@QueryParam("page") Integer page);
@GET
@Path("{id}")
@RolesAllowed("fdb_read_organization")
OrganizationData getOrganization(@PathParam("id") String id);
@GET
@Path("{id}/memberships")
@RolesAllowed("fdb_read_organization")
Response getOrganizationMembership(@PathParam("id") String id, @QueryParam("page") Integer page);
@GET
@Path("contacts")
@RolesAllowed("fdb_read_organization_employment")
Response getOrganizationContacts(@QueryParam("page") Integer page, @BeanParam OrganizationRequestParams params);
@GET
@Path("{id}/contacts")
@RolesAllowed("fdb_read_organization_employment")
Response getOrganizationContactsWithSearch(@PathParam("id") String id, @QueryParam("page") Integer page,
@BeanParam OrganizationRequestParams params);
@PUT
@Path("{id}/contacts")
@RolesAllowed("fdb_write_organization_employment")
OrganizationContactData updateOrganizationContacts(@PathParam("id") String id, OrganizationContactData contact);
@GET
@Path("{id}/contacts/{personID}")
@RolesAllowed("fdb_read_organization_employment")
Response getOrganizationContact(@PathParam("id") String id, @PathParam("personID") String personID,
@QueryParam("page") Integer page, @BeanParam OrganizationRequestParams params);
@DELETE
@Path("{id}/contacts/{personID}/{relation}")
@RolesAllowed("fdb_delete_organization_employment")
Response removeOrganizationContacts(@PathParam("id") String id, @PathParam("personID") String personID,
@PathParam("relation") String relation);
@GET
@Path("{id}/documents")
@RolesAllowed("fdb_read_organization_documents")
Response getOrganizationDocuments(@PathParam("id") String id, @QueryParam("page") Integer page,
@BeanParam OrganizationRequestParams params);
@GET
@Path("{id}/employment_histories")
@RolesAllowed("fdb_read_organization_employment")
Response getOrganizationEmploymentHistory(@PathParam("id") String id, @QueryParam("page") Integer page);
@AutoValue
@JsonDeserialize(builder = AutoValue_OrganizationAPI_OrganizationRequestParams.Builder.class)
public abstract static class OrganizationRequestParams {
@Nullable
@QueryParam("id")
public abstract String getOrganizationID();
@Nullable
@QueryParam("personID")
public abstract String getPersonID();
@Nullable
@QueryParam("email")
public abstract String getEmail();
@Nullable
@QueryParam("fName")
public abstract String getFirstName();
@Nullable
@QueryParam("lName")
public abstract String getLastName();
@Nullable
@QueryParam("relation")
public abstract String getRelation();
@Nullable
@QueryParam("is_not_expired")
public abstract Boolean getIsNotExpired();
@Nullable
@QueryParam("ids")
public abstract List<String> getIds();
@Nullable
@QueryParam("documentIDs")
public abstract List<String> getDocumentIds();