Skip to content
Snippets Groups Projects
Commit faa74446 authored by Martin Lowe's avatar Martin Lowe :flag_ca:
Browse files

Add new spam report and some additional helpers

parent 5fcc1d59
No related branches found
No related tags found
1 merge request!2Add new spam report and some additional helpers
Expand all Hide whitespace changes
Inline Side-by-side
Showing
with 4128 additions and 53 deletions
.gitignore
+ 2
− 3
View file @ faa74446
......@@ -48,7 +48,6 @@ release.properties
.project
.classpath
.settings/
bin/
# IntelliJ
.idea
......@@ -71,11 +70,11 @@ nb-configuration.xml
*.rej
#environment variables
.env
/.env
#Test resources
/.apt_generated/
/.apt_generated_tests/
report.csv
*.csv
README.md
+ 21
− 0
View file @ faa74446
# standalone-reports-scripts
## Usage
With a local running LDAP instance, the following commands can be used to run the various reports provided by this repository.
### Spam report
This report will read all of the disposable email domains from a [popular utility project in Github](https://github.com/unkn0w/disposable-email-domain-list) and iterate over each, looking in the LDAP instance for mail addresses matching the given domains. These entries will then be pulled and formatted into a report item to be printed to the current working directory at the end of the run. Additionally, there is some effort made to try and extract the groups where possible to give better indicators whether the user is legitimate.
To use this repo, connections to 2 services are required. The first is a connection to the `eclipse` database to look up user session logs, which are used to indicate last login for a user. This will be configured in `.env` file to simplify the process and to remove the need to enter the values into a file that might accidentally get pushed to the repo. A sample environment file is provided under `/config/.env`, and should be copied to the root of the project and updated to reflect the current environment.
The second is a tunnel to the LDAP instance that should be scanned for spam entries. Currently, these are configured through the command line call that runs the report below, where all of the CLI flags in the `quarkus.args` argument should be updated to reflect the tunneled/local environment.
Running the below command will start a dev server that will run the report locally and when done will notify the console. During the run, there are progress indicators that will be provided to the console to help gauge the current progress of the report.
```
source .env && mvn compile quarkus:dev -Dquarkus.args="spam-report --basedn=dc=example,dc=org --host=localhost --port=389 --binddn=cn=admin,dc=example,dc=org --password=<password here>"
```
### Foundation report
TBD
\ No newline at end of file
config/.env 0 → 100644
+ 3
− 0
View file @ faa74446
export QUARKUS_DATASOURCE_USERNAME=sample
export QUARKUS_DATASOURCE_PASSWORD=sample
export QUARKUS_DATASOURCE_JDBC_URL=jdbc:mariadb://localhost:3306/eclipse
\ No newline at end of file
pom.xml
+ 50
− 3
View file @ faa74446
......@@ -2,11 +2,11 @@
<modelVersion>4.0.0</modelVersion>
<groupId>standalone-java-reports</groupId>
<artifactId>standalone-java-reports</artifactId>
<version>0.0.1-SNAPSHOT</version>
<version>0.1.0-SNAPSHOT</version>
<properties>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<surefire-plugin.version>2.22.0</surefire-plugin.version>
<quarkus.version>2.7.2.Final</quarkus.version>
<surefire-plugin.version>3.0.0-M5</surefire-plugin.version>
<quarkus.version>2.16.7.Final</quarkus.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.test.skip>true</maven.test.skip>
<maven.compiler.source>11</maven.compiler.source>
......@@ -14,6 +14,12 @@
<maven.compiler.parameters>true</maven.compiler.parameters>
<compiler-plugin.version>3.8.1</compiler-plugin.version>
<auto-value.version>1.8.2</auto-value.version>
<sonar.sources>src/main</sonar.sources>
<sonar.tests></sonar.tests>
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.coverage.jacoco.xmlReportPaths>${project.basedir}/target/jacoco-report/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>
<sonar.junit.reportPath>${project.build.directory}/surefire-reports</sonar.junit.reportPath>
</properties>
<dependencyManagement>
<dependencies>
......@@ -27,6 +33,7 @@
</dependencies>
</dependencyManagement>
<dependencies>
<!-- Quarkus bindings for CLI -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-arc</artifactId>
......@@ -35,6 +42,18 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-picocli</artifactId>
</dependency>
<!-- Persistence bindings -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-hibernate-orm-panache</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-jdbc-mariadb</artifactId>
</dependency>
<!-- Serialization -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-jackson</artifactId>
......@@ -43,6 +62,18 @@
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-csv</artifactId>
</dependency>
<!-- Proxied automated REST clients -->
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-rest-client</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-resteasy-jackson</artifactId>
</dependency>
<!-- Modelling -->
<dependency>
<groupId>com.google.auto.value</groupId>
<artifactId>auto-value</artifactId>
......@@ -54,11 +85,27 @@
<artifactId>auto-value-annotations</artifactId>
<version>${auto-value.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
</dependency>
<!-- LDAP binding -->
<dependency>
<groupId>com.unboundid</groupId>
<artifactId>unboundid-ldapsdk</artifactId>
<version>6.0.4</version>
</dependency>
<!-- Misc. commons comparisons -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-jacoco</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
......
scripts/ldap-loader.rb 0 → 100755
+ 3331
− 0
View file @ faa74446
This diff is collapsed.
ldap-report.sh → scripts/ldap-report.sh
+ 0
− 0
View file @ faa74446
File moved
src/main/java/org/eclipsefoundation/reports/api/DisposableDomainGithubProjectRaw.java 0 → 100644
+ 34
− 0
View file @ faa74446
/**
* Copyright (c) 2023 Eclipse Foundation
*
* This program and the accompanying materials are made
* available under the terms of the Eclipse Public License 2.0
* which is available at https://www.eclipse.org/legal/epl-2.0/
*
* Author: Martin Lowe <martin.lowe@eclipse-foundation.org>
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.eclipsefoundation.reports.api;
import javax.ws.rs.GET;
import javax.ws.rs.HeaderParam;
import javax.ws.rs.Produces;
import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;
/**
* API binding to fetch the raw disposable domain list from Github. Authorization is added but not currently used in
* case this gets pushed to a shared environment where the unauthenticated rate limit may have been consumed in the
* future.
*
* @author Martin Lowe
*
*/
@RegisterRestClient
@Produces("application/vnd.github.VERSION.raw")
public interface DisposableDomainGithubProjectRaw {
@GET
String getRawDomainList(@HeaderParam("Authorization") String authorization);
}
src/main/java/org/eclipsefoundation/reports/LDAPReport.java → src/main/java/org/eclipsefoundation/reports/cli/FoundationLDAPReport.java
+ 62
− 47
View file @ faa74446
package org.eclipsefoundation.reports;
package org.eclipsefoundation.reports.cli;
import java.io.IOException;
import java.io.StringWriter;
......@@ -8,8 +8,12 @@ import java.util.Arrays;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import org.eclipsefoundation.reports.config.LdapCLIOptions;
import org.eclipsefoundation.reports.helper.LDAPHelper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
......@@ -22,84 +26,70 @@ import com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import com.google.auto.value.AutoValue;
import com.unboundid.ldap.sdk.LDAPConnection;
import com.unboundid.ldap.sdk.LDAPException;
import com.unboundid.ldap.sdk.SearchRequest;
import com.unboundid.ldap.sdk.SearchResult;
import com.unboundid.ldap.sdk.SearchResultEntry;
import com.unboundid.ldap.sdk.SearchScope;
import picocli.CommandLine;
/**
* Generates an LDAP report
* Generates an LDAP report using locally-stored foundation DB query results to build a mapping of LDAP users to
* Foundation DB users, marking users that are missing from either side or that have mismatched emails/user names.
*
* @author Martin Lowe
*
*/
@CommandLine.Command(name = "ldap-report")
public class LDAPReport implements Runnable {
public static final Logger LOGGER = LoggerFactory.getLogger(LDAPReport.class);
public class FoundationLDAPReport implements Runnable {
public static final Logger LOGGER = LoggerFactory.getLogger(FoundationLDAPReport.class);
// setting for CSV base
@CommandLine.Option(names = { "-b", "--base" }, description = "Path to CSV base of report", required = true)
String path;
// settings for LDAP connection
@CommandLine.Option(names = { "-h", "--host" }, description = "Host for the LDAP", defaultValue = "localhost")
String host;
@CommandLine.Option(names = { "-p", "--port" }, description = "Port for the LDAP connection", defaultValue = "389")
Integer port;
@CommandLine.Option(names = { "-d",
"--basedn" }, description = "Base DN for LDAP query", defaultValue = "dc=eclipse,dc=org")
String baseDN;
@CommandLine.Option(names = { "-B", "--binddn" }, description = "Bind DN for LDAP query")
String bindDN;
@CommandLine.Option(names = { "-P",
"--password" }, description = "Password to authenticate for LDAP query", interactive = true)
String password;
final CsvMapper CSV_MAPPER = (CsvMapper) new CsvMapper().registerModule(new JavaTimeModule())
@CommandLine.Mixin
LdapCLIOptions ldapConfig;
final CsvMapper csvMapper = (CsvMapper) new CsvMapper()
.registerModule(new JavaTimeModule())
.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
@Override
public void run() {
// attempt to open ldap connection
try (LDAPConnection conn = getConnection();
StringWriter sw = new StringWriter()) {
try (StringWriter sw = new StringWriter(); LDAPHelper ldap = new LDAPHelper(ldapConfig)) {
// read in the base csv
List<FoundationUserEntry> foundationUsers = readInBase();
// use the LDAP connection and foundation users to generate the report
List<ReportEntry> report = generateReport(conn, foundationUsers);
List<ReportEntry> report = generateReport(foundationUsers, ldap);
// output the report to the string writer
CSV_MAPPER.writer(CSV_MAPPER.schemaFor(ReportEntry.class).withHeader()).writeValues(sw).writeAll(report);
csvMapper.writer(csvMapper.schemaFor(ReportEntry.class).withHeader()).writeValues(sw).writeAll(report);
LOGGER.info("OUTPUT:\n{}", sw);
} catch (LDAPException e) {
LOGGER.error("Error while connecting to LDAP host", e);
} catch (IOException e) {
LOGGER.error("Error while reading the base CSV file", e);
} catch (Exception e) {
LOGGER.error("Error while handling LDAP connection", e);
}
}
private LDAPConnection getConnection() throws LDAPException {
return password != null && password != "" ? new LDAPConnection(host, port, bindDN, password)
: new LDAPConnection(host, port);
}
private List<ReportEntry> generateReport(LDAPConnection conn, List<FoundationUserEntry> foundationUsers)
throws LDAPException {
private List<ReportEntry> generateReport(List<FoundationUserEntry> foundationUsers, LDAPHelper ldap) throws LDAPException {
List<ReportEntry> report = new ArrayList<>();
for (FoundationUserEntry user : foundationUsers) {
// query ldap for the current user
SearchResult result = conn.search(new SearchRequest(baseDN, SearchScope.SUB, String
.format("(|(uid=%s)(mail=%s))", user.getPersonId(), user.getEmail())));
List<SearchResultEntry> out = result.getSearchEntries();
List<SearchResultEntry> out = ldap
.search(Map.of("uid", user.getPersonId(), "mail", user.getEmail()), Optional.of(SearchScope.SUB),
Optional.empty());
// build the report item
ReportEntry.Builder b = ReportEntry.builder();
b.setPersonId(user.getPersonId());
b.setFoundationEmail(user.getEmail());
b.setMemberLevel(user.getMemberLevel());
b.setOrganizationId(user.getOrganizationId());
b.setRelations(user.getRelations());
if (out.isEmpty()) { // if there is no match for current user in LDAP
b.setCode(StatusCode.MISSING);
b.setLdapAccounts(Collections.emptyList());
......@@ -137,7 +127,8 @@ public class LDAPReport implements Runnable {
*/
private List<FoundationUserEntry> readInBase() {
List<FoundationUserEntry> a = new ArrayList<>();
try (JsonParser parser = CSV_MAPPER.reader(CSV_MAPPER.schemaFor(FoundationUserEntry.class).withHeader())
try (JsonParser parser = csvMapper
.reader(csvMapper.schemaFor(FoundationUserEntry.class).withHeader())
.createParser(Path.of(path).toFile())) {
Iterator<FoundationUserEntry> entries = parser.readValuesAs(FoundationUserEntry.class);
entries.forEachRemaining(a::add);
......@@ -166,10 +157,9 @@ public class LDAPReport implements Runnable {
* @author Martin Lowe
*
*/
public static enum StatusCode {
public enum StatusCode {
OK, MISSING, AMBIGUOUS_ENTRY, MISMATCH_UID, MISMATCH_EMAIL;
}
/**
......@@ -184,15 +174,21 @@ public class LDAPReport implements Runnable {
*
*/
@AutoValue
@JsonPropertyOrder({ "personId", "email" })
@JsonDeserialize(builder = AutoValue_LDAPReport_FoundationUserEntry.Builder.class)
public static abstract class FoundationUserEntry {
@JsonPropertyOrder({ "personId", "email", "organizationId", "memberLevel", "relations" })
@JsonDeserialize(builder = AutoValue_FoundationLDAPReport_FoundationUserEntry.Builder.class)
public abstract static class FoundationUserEntry {
public abstract String getPersonId();
public abstract String getEmail();
public abstract Integer getOrganizationId();
public abstract String getMemberLevel();
public abstract List<String> getRelations();
public static Builder builder() {
return new AutoValue_LDAPReport_FoundationUserEntry.Builder();
return new AutoValue_FoundationLDAPReport_FoundationUserEntry.Builder();
}
@AutoValue.Builder
......@@ -202,6 +198,12 @@ public class LDAPReport implements Runnable {
public abstract Builder setEmail(String email);
public abstract Builder setOrganizationId(Integer organizationId);
public abstract Builder setMemberLevel(String memberLevel);
public abstract Builder setRelations(List<String> relations);
public abstract FoundationUserEntry build();
}
}
......@@ -213,19 +215,26 @@ public class LDAPReport implements Runnable {
*
*/
@AutoValue
@JsonPropertyOrder({ "personId", "foundationEmail", "ldapAccounts", "code" })
@JsonDeserialize(builder = AutoValue_LDAPReport_ReportEntry.Builder.class)
public static abstract class ReportEntry {
@JsonPropertyOrder({ "personId", "foundationEmail", "ldapAccounts", "organizationId", "memberLevel", "relations",
"code" })
@JsonDeserialize(builder = AutoValue_FoundationLDAPReport_ReportEntry.Builder.class)
public abstract static class ReportEntry {
public abstract String getPersonId();
public abstract String getFoundationEmail();
public abstract List<String> getLdapAccounts();
public abstract Integer getOrganizationId();
public abstract String getMemberLevel();
public abstract List<String> getRelations();
public abstract StatusCode getCode();
public static Builder builder() {
return new AutoValue_LDAPReport_ReportEntry.Builder();
return new AutoValue_FoundationLDAPReport_ReportEntry.Builder();
}
@AutoValue.Builder
......@@ -237,6 +246,12 @@ public class LDAPReport implements Runnable {
public abstract Builder setLdapAccounts(List<String> ldapAccounts);
public abstract Builder setOrganizationId(Integer organizationId);
public abstract Builder setMemberLevel(String memberLevel);
public abstract Builder setRelations(List<String> relations);
public abstract Builder setCode(StatusCode code);
public abstract ReportEntry build();
......
src/main/java/org/eclipsefoundation/reports/cli/ReportTop.java 0 → 100644
+ 28
− 0
View file @ faa74446
/**
* Copyright (c) 2023 Eclipse Foundation
*
* This program and the accompanying materials are made
* available under the terms of the Eclipse Public License 2.0
* which is available at https://www.eclipse.org/legal/epl-2.0/
*
* Author: Martin Lowe <martin.lowe@eclipse-foundation.org>
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.eclipsefoundation.reports.cli;
import io.quarkus.picocli.runtime.annotations.TopCommand;
import picocli.CommandLine;
/**
* Common top command for the multiple LDAP reports generated by this package. Required so that the multiple different
* reports can all be easily called.
*
* @author Martin Lowe
*
*/
@TopCommand
@CommandLine.Command(mixinStandardHelpOptions = true, subcommands = { SpamLDAPReport.class,
FoundationLDAPReport.class })
public class ReportTop {
}
src/main/java/org/eclipsefoundation/reports/cli/SpamLDAPReport.java 0 → 100644
+ 246
− 0
View file @ faa74446
/**
* Copyright (c) 2023 Eclipse Foundation
*
* This program and the accompanying materials are made
* available under the terms of the Eclipse Public License 2.0
* which is available at https://www.eclipse.org/legal/epl-2.0/
*
* Author: Martin Lowe <martin.lowe@eclipse-foundation.org>
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.eclipsefoundation.reports.cli;
import java.io.FileWriter;
import java.io.IOException;
import java.io.StringWriter;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import java.util.stream.Stream;
import javax.annotation.Nullable;
import javax.enterprise.context.control.ActivateRequestContext;
import javax.inject.Inject;
import org.apache.commons.lang3.StringUtils;
import org.eclipse.microprofile.rest.client.inject.RestClient;
import org.eclipsefoundation.reports.api.DisposableDomainGithubProjectRaw;
import org.eclipsefoundation.reports.config.LdapCLIOptions;
import org.eclipsefoundation.reports.dtos.EvtLog;
import org.eclipsefoundation.reports.helper.LDAPHelper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import com.google.auto.value.AutoValue;
import com.unboundid.ldap.sdk.LDAPException;
import com.unboundid.ldap.sdk.SearchResultEntry;
import picocli.CommandLine;
/**
* CLI command to generate spam report that looks for entries that have a disposable email domain associated with the
* account. These accounts will have their uids, mail, groups, and last login time listed to help provide context to
* whether these users are spam or just questionable.
*
* @author Martin Lowe
*
*/
@CommandLine.Command(name = "spam-report")
public class SpamLDAPReport implements Runnable {
public static final Logger LOGGER = LoggerFactory.getLogger(SpamLDAPReport.class);
private static final DateTimeFormatter CREATE_TIMESTAMP_FORMATTER = DateTimeFormatter
.ofPattern("yyyyMMddHHmmss'Z'");
// add common options to the command
@CommandLine.Mixin
LdapCLIOptions ldapConfig;
@RestClient
DisposableDomainGithubProjectRaw gh;
@Inject
ObjectMapper om;
// generate the non-standard CSV mapper object
final CsvMapper csvMapper = (CsvMapper) new CsvMapper()
.registerModule(new JavaTimeModule())
.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
@ActivateRequestContext
@Override
public void run() {
// get the disposable domains from the GH project
final List<String> value = getDisposableDomains();
LOGGER.info("Found {} disposable domains to process", value.size());
// for each domain, run an LDAP query to fetch users that match the domain and parse into output record format
List<ReportEntry> results = Collections.emptyList();
try (LDAPHelper ldap = new LDAPHelper(ldapConfig)) {
// create an index stream to enable better logging, and retrieve all LDAP entries for the given domains
List<SearchResultEntry> entries = IntStream
.range(0, 200)
.mapToObj(idx -> fetchAccountsForDomain(idx, value, ldap))
.flatMap(i -> i)
.collect(Collectors.toList());
// start conversion process, using logging to demarcate process in results
LOGGER.info("Beginning conversion of {} entries to final report output", entries.size());
int fivePercentCount = entries.size() / 20;
results = IntStream.range(0, entries.size()).mapToObj(e -> {
// give an indicator every 5% roughly
if (e % fivePercentCount == 0) {
LOGGER
.info("{}% of report post-processing complete",
(int) Math.ceil(((float) e) / entries.size() * 100));
}
// convert LDAP entry to the report entry and return it
return convertLdapRecord(entries.get(e));
}).collect(Collectors.toList());
} catch (Exception e) {
LOGGER.error("Unable to close LDAP connection", e);
}
LOGGER.info("Found {} potentially disposable accounts", results.size());
// build a string writer, and write the results of the report to the file (too big for console)
try (StringWriter sw = new StringWriter(); FileWriter fw = new FileWriter("sample.csv")) {
csvMapper.writer(csvMapper.schemaFor(ReportEntry.class).withHeader()).writeValues(fw).writeAll(results);
fw.flush();
} catch (IOException e) {
LOGGER.error("Error while preparing final report data", e);
}
LOGGER.info("Done! The server may now be safely closed");
}
/**
* Using the common LDAP helper, retrieve the disposable domain and do a search for any account that uses that
* domain.
*
* @param idx the current index of the domain, used for lookup and logging
* @param value the list of known disposable domains
* @param ldap the common LDAP helper object for lookups
* @return the stream of values for accounts using the given domain.
*/
private Stream<SearchResultEntry> fetchAccountsForDomain(int idx, List<String> value, LDAPHelper ldap) {
// progress indicator
if (idx % 100 == 0) {
LOGGER.info("Processed {} of {} domains", idx, value.size());
}
// get the actual domain value
String d = value.get(idx);
try {
// do an LDAP query with the spam domain, retrieving additional properties used in generating the report
return ldap
.search(Map.of("mail", "*" + d), Optional.empty(),
Optional.of(Arrays.asList("uid", "mail", "createTimestamp")))
.stream();
} catch (LDAPException e) {
LOGGER.error("Error while retrieving disposable accounts for domain {}", d);
}
return Stream.empty();
}
/**
* Using a basic fetch call, retrieves raw stringified domain list from a Github project that maintains a list of
* disposable email domains. This list is then serialized into a list of domains to be used in downstream LDAP
* queries.
*
* @return list of known disposable email domains
*/
private List<String> getDisposableDomains() {
String rawDisposableDomains = gh.getRawDomainList(null);
try {
return om.readerForListOf(String.class).readValue(rawDisposableDomains);
} catch (JsonProcessingException e) {
// cannot continue in this state, throw and end processing
throw new IllegalStateException(e);
}
}
/**
* Does conversion from raw LDAP entry to a report entry, using a DB binding to check for login events for the user
* for last login time, and splitting the DN to get the groups the user is a part of.
*
* @param e the LDAP result to convert to a report result entry
* @return the report entry for the LDAP entry
*/
private ReportEntry convertLdapRecord(SearchResultEntry e) {
String creationValue = e.getAttributeValue("createTimestamp");
String uid = e.getAttributeValue("uid");
// lookup the user in the DB to find last session time
EvtLog lastSession = EvtLog.findLastLoginSession(uid);
return ReportEntry
.builder()
.setEmail(e.getAttributeValue("mail"))
.setId(e.getAttributeValue("uid"))
.setGroups(Arrays
.asList(e.getDN().split(","))
.stream()
.filter(p -> p.startsWith("ou=") || p.startsWith("cn="))
.map(p -> p.substring(3))
.filter(name -> !name.equalsIgnoreCase(uid))
.collect(Collectors.toList()))
.setCreationDate(StringUtils.isBlank(creationValue) ? null
: LocalDateTime.parse(creationValue, CREATE_TIMESTAMP_FORMATTER).atZone(ZoneId.of("UTC")))
.setLastActive(lastSession == null ? null : lastSession.getEvtDateTime().atZone(ZoneId.of("UTC")))
.build();
}
/**
* CSV output row. The JSON property order is how CSV header order is determined.
*
* @author Martin Lowe
*
*/
@AutoValue
@JsonPropertyOrder({ "id", "email", "groups", "creationDate", "lastActive" })
@JsonDeserialize(builder = AutoValue_SpamLDAPReport_ReportEntry.Builder.class)
public abstract static class ReportEntry {
public abstract String getId();
public abstract String getEmail();
public abstract List<String> getGroups();
@Nullable
public abstract ZonedDateTime getCreationDate();
@Nullable
public abstract ZonedDateTime getLastActive();
public static Builder builder() {
return new AutoValue_SpamLDAPReport_ReportEntry.Builder();
}
@AutoValue.Builder
@JsonPOJOBuilder(withPrefix = "set")
public abstract static class Builder {
public abstract Builder setId(String id);
public abstract Builder setEmail(String email);
public abstract Builder setGroups(List<String> groups);
public abstract Builder setCreationDate(@Nullable ZonedDateTime creationDate);
public abstract Builder setLastActive(@Nullable ZonedDateTime lastActive);
public abstract ReportEntry build();
}
}
}
src/main/java/org/eclipsefoundation/reports/config/LdapCLIOptions.java 0 → 100644
+ 71
− 0
View file @ faa74446
/**
* Copyright (c) 2023 Eclipse Foundation
*
* This program and the accompanying materials are made
* available under the terms of the Eclipse Public License 2.0
* which is available at https://www.eclipse.org/legal/epl-2.0/
*
* Author: Martin Lowe <martin.lowe@eclipse-foundation.org>
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.eclipsefoundation.reports.config;
import picocli.CommandLine;
/**
* Shared LDAP CLI options to be used as a mixin for picocli commands.
*
* @author Martin Lowe
*
*/
public class LdapCLIOptions {
// settings for LDAP connection
@CommandLine.Option(names = { "-h", "--host" }, description = "Host for the LDAP", defaultValue = "localhost")
String host;
@CommandLine.Option(names = { "-p", "--port" }, description = "Port for the LDAP connection", defaultValue = "389")
Integer port;
@CommandLine.Option(names = { "-d", "--basedn" }, description = "Base DN for LDAP query", defaultValue = "dc=eclipse,dc=org")
String baseDN;
@CommandLine.Option(names = { "-B", "--binddn" }, description = "Bind DN for LDAP query")
String bindDN;
@CommandLine.Option(names = { "-P", "--password" }, description = "Password to authenticate for LDAP query", interactive = true)
String password;
/**
* @return the host
*/
public String getHost() {
return host;
}
/**
* @return the port
*/
public Integer getPort() {
return port;
}
/**
* @return the baseDN
*/
public String getBaseDN() {
return baseDN;
}
/**
* @return the bindDN
*/
public String getBindDN() {
return bindDN;
}
/**
* @return the password
*/
public String getPassword() {
return password;
}
}
src/main/java/org/eclipsefoundation/reports/dtos/EvtLog.java 0 → 100644
+ 173
− 0
View file @ faa74446
/**
* Copyright (c) 2023 Eclipse Foundation
*
* This program and the accompanying materials are made
* available under the terms of the Eclipse Public License 2.0
* which is available at https://www.eclipse.org/legal/epl-2.0/
*
* Author: Martin Lowe <martin.lowe@eclipse-foundation.org>
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.eclipsefoundation.reports.dtos;
import java.time.LocalDateTime;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.Table;
import io.quarkus.hibernate.orm.panache.PanacheEntityBase;
import io.quarkus.panache.common.Sort;
/**
* Event log for EF to track system events.
*
* @author Martin Lowe
*
*/
@Entity
@Table(name = "SYS_EvtLog")
public class EvtLog extends PanacheEntityBase {
@Id
private int logId;
private String logTable;
@Column(name = "PK1")
private String pk1;
@Column(name = "PK2")
private String pk2;
private String logAction;
private String uid;
private LocalDateTime evtDateTime;
/**
* Finds the most recent login event session if it exists
* @param uid
* @return
*/
public static EvtLog findLastLoginSession(String uid) {
return find("logTable = 'sessions' AND uid = ?1", Sort.descending("evtDateTime"), uid).firstResult();
}
/**
* @return the logId
*/
public int getLogId() {
return logId;
}
/**
* @param logId the logId to set
*/
public void setLogId(int logId) {
this.logId = logId;
}
/**
* @return the logTable
*/
public String getLogTable() {
return logTable;
}
/**
* @param logTable the logTable to set
*/
public void setLogTable(String logTable) {
this.logTable = logTable;
}
/**
* @return the pk1
*/
public String getPk1() {
return pk1;
}
/**
* @param pk1 the pk1 to set
*/
public void setPk1(String pk1) {
this.pk1 = pk1;
}
/**
* @return the pk2
*/
public String getPk2() {
return pk2;
}
/**
* @param pk2 the pk2 to set
*/
public void setPk2(String pk2) {
this.pk2 = pk2;
}
/**
* @return the logAction
*/
public String getLogAction() {
return logAction;
}
/**
* @param logAction the logAction to set
*/
public void setLogAction(String logAction) {
this.logAction = logAction;
}
/**
* @return the uid
*/
public String getUid() {
return uid;
}
/**
* @param uid the uid to set
*/
public void setUid(String uid) {
this.uid = uid;
}
/**
* @return the evtDateTime
*/
public LocalDateTime getEvtDateTime() {
return evtDateTime;
}
/**
* @param evtDateTime the evtDateTime to set
*/
public void setEvtDateTime(LocalDateTime evtDateTime) {
this.evtDateTime = evtDateTime;
}
@Override
public String toString() {
StringBuilder builder = new StringBuilder();
builder.append("SysEventLog [logId=");
builder.append(logId);
builder.append(", logTable=");
builder.append(logTable);
builder.append(", pk1=");
builder.append(pk1);
builder.append(", pk2=");
builder.append(pk2);
builder.append(", logAction=");
builder.append(logAction);
builder.append(", uid=");
builder.append(uid);
builder.append(", evtDateTime=");
builder.append(evtDateTime);
builder.append("]");
return builder.toString();
}
}
src/main/java/org/eclipsefoundation/reports/helper/LDAPHelper.java 0 → 100644
+ 103
− 0
View file @ faa74446
/**
* Copyright (c) 2023 Eclipse Foundation
*
* This program and the accompanying materials are made
* available under the terms of the Eclipse Public License 2.0
* which is available at https://www.eclipse.org/legal/epl-2.0/
*
* Author: Martin Lowe <martin.lowe@eclipse-foundation.org>
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.eclipsefoundation.reports.helper;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import org.apache.commons.lang3.StringUtils;
import org.eclipsefoundation.reports.config.LdapCLIOptions;
import com.unboundid.ldap.sdk.LDAPConnection;
import com.unboundid.ldap.sdk.LDAPException;
import com.unboundid.ldap.sdk.SearchRequest;
import com.unboundid.ldap.sdk.SearchResult;
import com.unboundid.ldap.sdk.SearchResultEntry;
import com.unboundid.ldap.sdk.SearchScope;
/**
* Basic helper to performe LDAP queries and connections for the currently running service.
*
* @author Martin Lowe
*
*/
public class LDAPHelper implements AutoCloseable {
public static final List<String> BASE_SEARCH_ATTRIBUTES = Collections.unmodifiableList(Arrays.asList("uid", "mail"));
private String password;
private String host;
private Integer port;
private String baseDn;
private String bindDn;
private LDAPConnection connection;
/**
* @param password
* @param host
* @param port
* @param bindDN
*/
public LDAPHelper(LdapCLIOptions options) {
this.password = options.getPassword();
this.host = options.getHost();
this.port = options.getPort();
this.bindDn = options.getBindDN();
this.baseDn = options.getBaseDN();
this.connection = null;
}
public List<SearchResultEntry> search(Map<String, String> params, Optional<SearchScope> scope, Optional<List<String>> attrs)
throws LDAPException {
return search(params, "|", scope, attrs);
}
public List<SearchResultEntry> search(Map<String, String> params, String searchOp, Optional<SearchScope> scope,
Optional<List<String>> attrs) throws LDAPException {
SearchResult result = getConnection()
.search(new SearchRequest(baseDn, scope.orElse(SearchScope.SUB), buildSearchQuery(params, searchOp),
attrs.orElse(BASE_SEARCH_ATTRIBUTES).toArray(new String[] {})));
return result.getSearchEntries();
}
private String buildSearchQuery(Map<String, String> params, String searchOp) {
StringBuilder sb = new StringBuilder();
sb.append("(").append(searchOp);
params.entrySet().forEach(e -> sb.append('(').append(e.getKey()).append("=%s)"));
sb.append(')');
return String.format(sb.toString(), params.values().toArray());
}
private LDAPConnection getConnection() throws LDAPException {
if (this.connection == null || !this.connection.isConnected()) {
this.connection = StringUtils.isNotBlank(password) ? new LDAPConnection(host, port, bindDn, password)
: new LDAPConnection(host, port);
}
return this.connection;
}
private boolean isActive() {
return this.connection != null && this.connection.isConnected();
}
@Override
public void close() throws Exception {
if (isActive()) {
this.connection.close();
}
}
}
src/main/resources/application.properties
+ 4
− 0
View file @ faa74446
quarkus.rest-client."org.eclipsefoundation.reports.api.DisposableDomainGithubProjectRaw".url=https://api.github.com/repos/unkn0w/disposable-email-domain-list/contents/domains.json
quarkus.datasource.db-kind = mariadb
quarkus.datasource.jdbc.url = jdbc:postgresql://localhost:13306/mydatabase
\ No newline at end of file
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

Copyright © Eclipse Foundation, Inc. All Rights Reserved.     Privacy Policy | Terms of Use | Copyright Agent