eclipsefdn-api-common issueshttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues2024-03-26T13:08:45Zhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/55Migrate the OAuth request filter to efservices2024-03-26T13:08:45ZMartin Lowemartin.lowe@eclipse-foundation.orgMigrate the OAuth request filter to efservicesAs we've used it in multiple projects to preload the authenticated used and provide a simple way of applying Authenticated without a security provider, we should move the OAuth filter to the common lib. In this version, we will need to a...As we've used it in multiple projects to preload the authenticated used and provide a simple way of applying Authenticated without a security provider, we should move the OAuth filter to the common lib. In this version, we will need to allow for configured scopes, paths to enforce login, and a flag to overall enable the functionality. We should add a config with the following fields:
|Property|Type|Default|Purpose|
|-|-|-|-|
|eclipse.services.auth.request-filter.enabled|Boolean|False|Allow us to turn on and off the filter at a base level, independent of the auth service|
|eclipse.services.auth.request-filter.scopes|Optional List of strings||Set scopes to be used when fetching a user for authentication.|
|eclipse.services.auth.request-filter.paths|Optional List of strings||The paths in which a signed-in user should be enforced. Can be regexes to allow for flexible paths.|https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/6Improve HealthCheck of backend service for OKD integration2024-03-25T14:41:26ZMartin Lowemartin.lowe@eclipse-foundation.orgImprove HealthCheck of backend service for OKD integrationMartin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/2Update SQL generators to use sanitization of strings used in query2024-03-20T18:01:36ZMartin Lowemartin.lowe@eclipse-foundation.orgUpdate SQL generators to use sanitization of strings used in queryWe currently use prepared statements to sanitize our fields and block injection based attacks. There is a few fields where, with enough effort, the checks could potentially be circumvented. An example of this is the field name in the ord...We currently use prepared statements to sanitize our fields and block injection based attacks. There is a few fields where, with enough effort, the checks could potentially be circumvented. An example of this is the field name in the order by clause of the HQLGenerator class. Usage of this class is still rare, so the risk is low but we should fix this hole sooner rather than later.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/54Update Project model to handle future industry_collaborations2024-03-12T15:47:53ZMartin Lowemartin.lowe@eclipse-foundation.orgUpdate Project model to handle future industry_collaborationsAs part of the D9 migration, we will be renaming the working_groups property under the `Project` model to be industry_collaborations. This represents a breaking change that is happening eventually. Since we don't want to create an issue ...As part of the D9 migration, we will be renaming the working_groups property under the `Project` model to be industry_collaborations. This represents a breaking change that is happening eventually. Since we don't want to create an issue where in a few months we will have to change how we access and filter the models, we should make the change now when it's only used in a couple of projects.
I'm thinking what we can do is the following:
- Add JSON mapping fields for both industry_collaborations and working_groups to the Project model. The new mapping will be the same as the current working_group property.
- Include documentation that these fields should not be used for data access, as they are deprecated and will be removed eventually.
- Add 3rd memoized field that looks at both the industry_collaborations and working_groups fields, and combines them into a single output list. This 3rd field is the one downstream services should read from and access to do filters.
The idea is that once the switch over is done, we can remove the deprecated fields and change the memoized field to be just the natural field. This adds future compatibility with the new field and reduces the migration activities we will have with it.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/102Project collaborations without names will break Project preloading2024-03-11T15:57:57ZMartin Lowemartin.lowe@eclipse-foundation.orgProject collaborations without names will break Project preloadingDiscovered through the Project Adopters API, if a project has a collaboration set without a name, it will cause the project loading to break and may cause cascading failures through the API.Discovered through the Project Adopters API, if a project has a collaboration set without a name, it will cause the project loading to break and may cause cascading failures through the API.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/101Catch Hibernate exception for empty results in framework2024-03-05T15:54:19ZMartin Lowemartin.lowe@eclipse-foundation.orgCatch Hibernate exception for empty results in frameworkAt some point in the Quarkus 3.x upgrade, the platform switched from returning empty lists to throwing runtime exceptions when there are no results for SQL queries. We already handle checking as needed, so to go back to previous state an...At some point in the Quarkus 3.x upgrade, the platform switched from returning empty lists to throwing runtime exceptions when there are no results for SQL queries. We already handle checking as needed, so to go back to previous state and handle this a little more cleanly, a patch will need to be made to return an empty list in these cases.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/99Add eager token refresh to token service2024-02-28T16:15:46ZZachary SabourinAdd eager token refresh to token serviceWhile monitoring the Profile-API, multiple unauthenticated requests were made to accounts at the same time using an expired token. This caused the required `mail` field to be missing, and caused the requests to fail.
We should make the ...While monitoring the Profile-API, multiple unauthenticated requests were made to accounts at the same time using an expired token. This caused the required `mail` field to be missing, and caused the requests to fail.
We should make the token refresh happen shortly before the token is set to expire to avoid this ever happening.Zachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/27Look into how we can register LoadingCaches more easily2024-02-21T19:32:08ZMartin Lowemartin.lowe@eclipse-foundation.orgLook into how we can register LoadingCaches more easilyCurrently, we implement loading caches manually in classes that require them making them harder to manage. We should look into implementing some sort of harness that would allow us to register a runnable as a loading cache source and sta...Currently, we implement loading caches manually in classes that require them making them harder to manage. We should look into implementing some sort of harness that would allow us to register a runnable as a loading cache source and standardize how we manage them as well. This would give us better control over caching in services and use new configuration models to better enable highly configurable and low-effort caching solutions.
Some things to consider for the service:
- allowing for different refresh rates to be configured
- Optionally allowing scheduled calls to force a cache to refresh if it hasn't been called (keep it fresh)
- Allow for cache usage to be configurable according to the current profile (dev vs prod)
- Create mechanism to use bookkeeping/stats which are available from the Caffeine cacheMartin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/79Create POC for switching common lib to use reactive instead of blocking2024-02-21T14:01:21ZMartin Lowemartin.lowe@eclipse-foundation.orgCreate POC for switching common lib to use reactive instead of blockingCurrently, we use blocking/synchronous calls when executing code in our stacks. There have been large changes in Quarkus where reactive technologies have been upgraded and promoted over traditional blocking calls. We should start the pro...Currently, we use blocking/synchronous calls when executing code in our stacks. There have been large changes in Quarkus where reactive technologies have been upgraded and promoted over traditional blocking calls. We should start the process of migrating our calls to use non blocking calls to improve performance and use the recommended stacks over the legacy standard.
The main targets of these changes should be anything with caching, REST client calls, or persistence queries. Those changes should represent the large majority of code where we are currently blocking when we could be using reactive calls.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/80Switch to Quarkus 3.x from 2.x2024-02-15T20:52:20ZMartin Lowemartin.lowe@eclipse-foundation.orgSwitch to Quarkus 3.x from 2.xThe new Quarkus library has had time to stabilize, so we should look at migrating to it if possible. We will need to evaluate what libraries will be impacted and if there are any deprecations. The base libraries also use the new `jakarta...The new Quarkus library has had time to stabilize, so we should look at migrating to it if possible. We will need to evaluate what libraries will be impacted and if there are any deprecations. The base libraries also use the new `jakarta` namespace, so there will be some updates to namespaces for any package that uses the new version.
We'll want to increment a major version as well, since it represents a major shift in the APIs and downstream library support.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/96Update mail_history field in EfUser model2024-02-02T15:13:04ZZachary SabourinUpdate mail_history field in EfUser modelAn error was made and the `mail_history` field is not in the proper format. It needs to be updatedAn error was made and the `mail_history` field is not in the proper format. It needs to be updatedZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/73Add ability to view cached data via the cache resource endpoints2024-01-25T18:32:14ZZachary SabourinAdd ability to view cached data via the cache resource endpointsTo increase the utility of our cache resource, we should add the ability to also view data for our regular and loading caches. A few endpoints will need to be added for this:
- `/caches/keys/{key}/data`
- `/caches/loading/keys/{key}/data`To increase the utility of our cache resource, we should add the ability to also view data for our regular and loading caches. A few endpoints will need to be added for this:
- `/caches/keys/{key}/data`
- `/caches/loading/keys/{key}/data`Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/95Update template header logo to 150px width2024-01-08T21:16:03ZZachary SabourinUpdate template header logo to 150px widthZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/93Set maximum default cache sizes to avoid cache OOMing the service2024-01-02T15:55:09ZMartin Lowemartin.lowe@eclipse-foundation.orgSet maximum default cache sizes to avoid cache OOMing the serviceWhile testing profile, Zach and I noticed that while initial cache capacity is set, we never set a maximum. This could cause issues where the cache completely fills the containers, or grows to an unmanageable size. To combat this, we sho...While testing profile, Zach and I noticed that while initial cache capacity is set, we never set a maximum. This could cause issues where the cache completely fills the containers, or grows to an unmanageable size. To combat this, we should set a default limit of 10k entries, which should be enough for most of our use cases. Any other cases, we can just override those build properties in the downstream applications easily.Zachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/47Create response filter that adds cache-control headers2023-12-08T16:36:00ZZachary SabourinCreate response filter that adds cache-control headersIn a few instances, we've encountered an issue with NGINX caching some results for a few minutes. In most cases this is desired and expected. However, when it is not desired, we need to be able to disable it. We currently have an easy so...In a few instances, we've encountered an issue with NGINX caching some results for a few minutes. In most cases this is desired and expected. However, when it is not desired, we need to be able to disable it. We currently have an easy solution that leverages undertow, as seen here: https://gitlab.eclipse.org/eclipsefdn/it/api/eclipse-openvsx-api/-/issues/18
For our use case, Undertow reads the `undertow-handlers.conf` file and adds the following cache breaking headers to all outbound GET responses:
- `cache-control: no-cache, no-store, must-revalidate`
- `pragma: no-cache`
- `expires: 0`
We should think about how we want to configure a filter that adds these same headers on GET requests. We could have it `enabled=false` by default. All we would need to do is enable it on a project by project basis
/cc @maloweZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/91Update caching service to allow proper distinction between error states2023-12-05T15:29:32ZZachary SabourinUpdate caching service to allow proper distinction between error states### Problem
The current implementation of our caching layer has a few holes that need to be addressed.
If an error occurs while caching data such as LDAP responses, HTTP responses, or large data processes, the `CachingService`'s current...### Problem
The current implementation of our caching layer has a few holes that need to be addressed.
If an error occurs while caching data such as LDAP responses, HTTP responses, or large data processes, the `CachingService`'s current behavior is to log the stack trace, suppress the error, and return an empty `Optional`. This solution works well for some cases, but is insufficient when it comes to passing a meaningful error to the client.
**EXAMPLE:** The RESTEasy client throws a `ResteasyWebApplicationException` for both 404 and 500s. This causes an error while creating a cache key, but the cache only returns an empty `Optional`. In the case of fetching a user from Accounts or fetching from Foundationdb-api, this makes it impossible to determine the difference between data not existing and the external service being down without looking at the logs. Seeing as we generally assume an empty `Optional` means the data doesn't exist, the final return to the client is always a 404.
In the case of the sync script, this would lead to user permissions being removed if LDAP or Accounts were experiencing some service disruptions.
### Proposed Solution
Create a cache wrapper object with some relevant fields:
- An `Optional` containing the cached data
- An `Optional` containing the error if there is one
- We can ad additional fields that we deem relevant
Instead of returning and `Optional`, the caching service would return this object. Also in the case of an error, we could allow for a refresh if the error is not a 404.
/cc @maloweZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/86Switch runtime dockerfile to use new Java 17 base image2023-11-29T21:17:02ZMartin Lowemartin.lowe@eclipse-foundation.orgSwitch runtime dockerfile to use new Java 17 base imagehttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/78Update HTML template to use Astro theme2023-11-29T19:11:16ZMartin Lowemartin.lowe@eclipse-foundation.orgUpdate HTML template to use Astro themeMartin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/92Properly map Runtime Exceptions to 500 returns2023-11-27T14:51:25ZZachary SabourinProperly map Runtime Exceptions to 500 returnsOur [RuntimeMapper](https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/blob/main/core/src/main/java/org/eclipsefoundation/core/resource/mapper/RuntimeMapper.java?ref_type=heads) currently maps RuntimeExceptions to `400 ...Our [RuntimeMapper](https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/blob/main/core/src/main/java/org/eclipsefoundation/core/resource/mapper/RuntimeMapper.java?ref_type=heads) currently maps RuntimeExceptions to `400 Bad Request`. They should be mapped to `500 Internal Server Error`.
A call to foundationdb-api earlier today received a 400 after a DB error occured. It was not immediately clear from the error what the problem was. A 500 should make this easier to understandZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/88Update WGService to include additional features and filtering2023-11-09T14:27:33ZZachary SabourinUpdate WGService to include additional features and filteringZachary SabourinZachary Sabourin