Skip to content

HTTP-119 - Fix bug where default Java TrustStore was not used when no cert-related properties were used. #123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import java.net.http.HttpClient;
import java.net.http.HttpClient.Redirect;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
Expand All @@ -24,8 +25,8 @@
public class JavaNetHttpClientFactory {

/**
* Creates Java's {@link HttpClient} instance that will be using default, JVM shared {@link
* java.util.concurrent.ForkJoinPool} for async calls.
* Creates Java's {@link HttpClient} instance that will be using default, JVM shared
* {@link java.util.concurrent.ForkJoinPool} for async calls.
*
* @param properties properties used to build {@link SSLContext}
* @return new {@link HttpClient} instance.
Expand Down Expand Up @@ -73,7 +74,9 @@ public static HttpClient createClient(Properties properties, Executor executor)
* @return new {@link SSLContext} instance.
*/
private static SSLContext getSslContext(Properties properties) {
SecurityContext securityContext = createSecurityContext(properties);

String keyStorePath =
properties.getProperty(HttpConnectorConfigConstants.KEY_STORE_PATH, "");

boolean selfSignedCert = Boolean.parseBoolean(
properties.getProperty(HttpConnectorConfigConstants.ALLOW_SELF_SIGNED, "false"));
Expand All @@ -88,6 +91,21 @@ private static SSLContext getSslContext(Properties properties) {
String clientPrivateKey = properties
.getProperty(HttpConnectorConfigConstants.CLIENT_PRIVATE_KEY, "");

if (StringUtils.isNullOrWhitespaceOnly(keyStorePath)
&& !selfSignedCert
&& serverTrustedCerts.length == 0
&& StringUtils.isNullOrWhitespaceOnly(clientCert)
&& StringUtils.isNullOrWhitespaceOnly(clientPrivateKey)) {
Copy link
Contributor

@davidradl davidradl Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kristoffSC What happens if we only have client certs. In this case, we want the trusted public server certs and the client certs. The current fix does not deal with that combination. Can we get hold of the default public server certs and add them like securityContext.addCertToTrustStore(cert); then process the client certs?


try {
return SSLContext.getDefault();
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
}

SecurityContext securityContext = createSecurityContext(properties);
Copy link
Contributor

@davidradl davidradl Sep 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kristoffSC Talking with our team around this, in addition to the above. It would be good to have the ability to:
1- run with public certs and any supplied private server certs. i.e. merge the supplied certs with existing public ones.
2- run only with private server certs (which is the current functionality when they are supplied in the config)

This would require a new config option maybe 'gid.connector.http.security.cert.server.includeDefaultCerts' with a default of true.

WDYT?


for (String cert : serverTrustedCerts) {
if (!StringUtils.isNullOrWhitespaceOnly(cert)) {
securityContext.addCertToTrustStore(cert);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,24 @@

public class ElasticSearchLiteQueryCreatorTest {

@Test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these new tests are not related to the issue HTTP-119? Could you please add them in a separate PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - it would be good to have unit tests to test the various code paths around this change.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "new code" is already partially covered by existing tests, but I will try to add new ones to cover it fully.

The reason why I added those new tests was that we dropped with branch coverage below 90% and while inspecting test report I have found those two.

I will like to leave those tests in this PR, unless there will be a strong NO to it. The PR is small so IMO having those extra, unrelated tests is ok.

image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine for me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kristoffSC fine my me too - thanks for putting this together.

public void testWithEmptyLookup() {

// GIVEN
LookupRow lookupRow = new LookupRow();
lookupRow.setLookupPhysicalRowDataType(DataTypes.STRING());

GenericRowData lookupDataRow = GenericRowData.of(StringData.fromString("val1"));

// WHEN
var queryCreator = new ElasticSearchLiteQueryCreator(lookupRow);
var createdQuery = queryCreator.createLookupQuery(lookupDataRow);

// THEN
assertThat(createdQuery.getLookupQuery()).isEqualTo("");
assertThat(createdQuery.getBodyBasedUrlQueryParameters()).isEmpty();
}

@Test
public void testQueryCreationForSingleQueryStringParam() {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,23 @@ public void shouldSerializeToJson() {
row.setField(0, 11);
row.setField(1, StringData.fromString("myUuid"));

this.jsonQueryCreator.createLookupQuery(row);
LookupQueryInfo lookupQuery = this.jsonQueryCreator.createLookupQuery(row);
assertThat(lookupQuery.getBodyBasedUrlQueryParameters().isEmpty());
assertThat(lookupQuery.getLookupQuery()).isEqualTo("{\"id\":11,\"uuid\":\"myUuid\"}");
}

@Test
public void shouldSerializeToJsonTwice() {
GenericRowData row = new GenericRowData(2);
row.setField(0, 11);
row.setField(1, StringData.fromString("myUuid"));

this.jsonQueryCreator.createLookupQuery(row);

// Call createLookupQuery two times
// to check that serialization schema is not opened Two times.
this.jsonQueryCreator.createLookupQuery(row);
LookupQueryInfo lookupQuery = this.jsonQueryCreator.createLookupQuery(row);
assertThat(lookupQuery.getBodyBasedUrlQueryParameters().isEmpty());
assertThat(lookupQuery.getLookupQuery()).isEqualTo("{\"id\":11,\"uuid\":\"myUuid\"}");
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package com.getindata.connectors.http.internal.table.lookup.querycreators;

import java.util.Collections;
import java.util.Optional;

import org.apache.flink.configuration.ConfigOption;
import org.apache.flink.configuration.ConfigOptions;
import org.apache.flink.configuration.Configuration;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;

class QueryFormatAwareConfigurationTest {

private static final ConfigOption<String> configOption = ConfigOptions.key("key")
.stringType()
.noDefaultValue();

@Test
public void testWithDot() {
QueryFormatAwareConfiguration queryConfig = new QueryFormatAwareConfiguration(
"prefix.", Configuration.fromMap(Collections.singletonMap("prefix.key", "val"))
);

Optional<String> optional = queryConfig.getOptional(configOption);
assertThat(optional.get()).isEqualTo("val");
}

@Test
public void testWithoutDot() {
QueryFormatAwareConfiguration queryConfig = new QueryFormatAwareConfiguration(
"prefix", Configuration.fromMap(Collections.singletonMap("prefix.key", "val"))
);

Optional<String> optional = queryConfig.getOptional(configOption);
assertThat(optional.get()).isEqualTo("val");
}

}
Loading