-
Notifications
You must be signed in to change notification settings - Fork 43
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Add method in Datastore client to invoke rpc for aggregation query * Creating count aggregation and using it to populate Aggregation proto * Moving aggregation builder method to root level aggregation class * Introducing RecordQuery to represent queries which returns entity records when executed * Updating gitignore with patch extension * Setting up structure of Aggregation query and its builder * Introducing ProtoPreparer to populate the request protos * Delegating responsibility of preparing query proto to QueryPreparer * Populating aggregation query with nested structured query * Delegating responsibility of preparing query proto in GqlQuery to QueryPreparer * Removing RecordQuery from the query hierarchy and making it a standalone interface for now * Populating aggregation query with nested gql query * Removing deprecation warning by using assertThrows instead of ExpectedException rule * Making DatastoreRpc call aggregation query method on client * Creating response transformer to transform aggregation query response into domain objects * Implementing aggregation query executor to execute AggergationQuery * Adding missing assertion statements * Creating RetryExecutor to inject it as a dependency in other components * Making RetryExecutor accept RetrySettings when creating it * Revert "Making RetryExecutor accept RetrySettings when creating it" This reverts commit 1dfafb7. * Revert "Creating RetryExecutor to inject it as a dependency in other components" This reverts commit 8872a55. * Introducing RetryAndTraceDatastoreRpcDecorator to have retry and traceability logic on top of another DatastoreRpc * Extracting out the responsibility of preparing ReadOption in it's own ProtoPreparer * Making QueryExecutor to execute query with provided ReadOptions * Exposing readTime to the user * Ignoring runAggregationQuery method from clirr check * Making readTime final * Allowing namespace to be optional in AggregationQuery * Add capability to fetch aggregation result by passing alias * Implementing User facing datastore.runAggrgation method to run aggregation query * Add integration test for count aggregation * Add transaction Id support in ReadOptionsProtoPreparer * Supporting aggregation query with transactions * Allowing user to create Aggregation directly without involving its builder * Preventing creating duplicated aggregation when creating an aggregation query * Marking RecordQuery implemented method as InternalApi * Writing comments and JavaDoc for aggregation query related class * Adding a default implementation in the public interfaces to avoid compile time failures * covering a scenario to maintain consistent snapshot when executing aggregation query in a transaction * Creating emulator proxy to simulate AggregationQuery response from emulator * Integration test to execute an aggregation query in a read only transaction * Getting rid off limit operation on count aggregation as same behaviour can be achieved by using 'limit' operation on the underlying query * Removing import statement from javadoc and undo changes in .gitignore file * Using Optional instead of returning null from ReadOptionsProtoPreparer * using assertThat from Truth library * fixing unit test * Getting rid off Double braces initialization syntax * Fixing lint * Getting rid off emulator proxy and using easy mock to check the aggregationQuery triggered * Deleting a entity created locally in other test which is causing failure in other test * Deleting all keys in datastore in integration test so that new test can start fresh * Executing two read write transaction simultaneously and verifying their behaviour * Removing tests to verify serializability as it's an underlying implementation detail * Fixing lint * Adding runAggregationQuery method to reflect config so that it's available and accessible in native image through reflection * Fixing equals of CountAggregation * Fixing lint * Adding an integration test of using limit option with aggregation query * Adding BetaApi annotation to public surface to indicate that aggregation query / count is in preview * Fixing lint * Removing unused functiona and fixing javadoc * fixing variable name
- Loading branch information
Showing
38 changed files
with
2,410 additions
and
12 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
176 changes: 176 additions & 0 deletions
176
google-cloud-datastore/src/main/java/com/google/cloud/datastore/AggregationQuery.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,176 @@ | ||
| /* | ||
| * Copyright 2022 Google LLC | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * https://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package com.google.cloud.datastore; | ||
|
|
||
| import static com.google.cloud.datastore.AggregationQuery.Mode.GQL; | ||
| import static com.google.cloud.datastore.AggregationQuery.Mode.STRUCTURED; | ||
| import static com.google.common.base.Preconditions.checkArgument; | ||
|
|
||
| import com.google.api.core.BetaApi; | ||
| import com.google.cloud.datastore.aggregation.Aggregation; | ||
| import com.google.cloud.datastore.aggregation.AggregationBuilder; | ||
| import java.util.HashSet; | ||
| import java.util.Set; | ||
|
|
||
| /** | ||
| * An implementation of a Google Cloud Datastore Query that returns {@link AggregationResults}, It | ||
| * can be constructed by providing a nested query ({@link StructuredQuery} or {@link GqlQuery}) to | ||
| * run the aggregations on and a set of {@link Aggregation}. | ||
| * | ||
| * <p>{@link StructuredQuery} example: | ||
| * | ||
| * <pre>{@code | ||
| * EntityQuery selectAllQuery = Query.newEntityQueryBuilder() | ||
| * .setKind("Task") | ||
| * .build(); | ||
| * AggregationQuery aggregationQuery = Query.newAggregationQueryBuilder() | ||
| * .addAggregation(count().as("total_count")) | ||
| * .over(selectAllQuery) | ||
| * .build(); | ||
| * AggregationResults aggregationResults = datastore.runAggregation(aggregationQuery); | ||
| * for (AggregationResult aggregationResult : aggregationResults) { | ||
| * System.out.println(aggregationResult.get("total_count")); | ||
| * } | ||
| * }</pre> | ||
| * | ||
| * <h4>{@link GqlQuery} example:</h4> | ||
| * | ||
| * <pre>{@code | ||
| * GqlQuery<?> selectAllGqlQuery = Query.newGqlQueryBuilder( | ||
| * "AGGREGATE COUNT(*) AS total_count, COUNT_UP_TO(100) AS count_upto_100 OVER(SELECT * FROM Task)" | ||
| * ) | ||
| * .setAllowLiteral(true) | ||
| * .build(); | ||
| * AggregationQuery aggregationQuery = Query.newAggregationQueryBuilder() | ||
| * .over(selectAllGqlQuery) | ||
| * .build(); | ||
| * AggregationResults aggregationResults = datastore.runAggregation(aggregationQuery); | ||
| * for (AggregationResult aggregationResult : aggregationResults) { | ||
| * System.out.println(aggregationResult.get("total_count")); | ||
| * System.out.println(aggregationResult.get("count_upto_100")); | ||
| * } | ||
| * }</pre> | ||
| * | ||
| * @see <a href="https://cloud.google.com/appengine/docs/java/datastore/queries">Datastore | ||
| * queries</a> | ||
| */ | ||
| @BetaApi | ||
| public class AggregationQuery extends Query<AggregationResults> { | ||
|
|
||
| private Set<Aggregation> aggregations; | ||
| private StructuredQuery<?> nestedStructuredQuery; | ||
| private final Mode mode; | ||
| private GqlQuery<?> nestedGqlQuery; | ||
|
|
||
| AggregationQuery( | ||
| String namespace, Set<Aggregation> aggregations, StructuredQuery<?> nestedQuery) { | ||
| super(namespace); | ||
| checkArgument( | ||
| !aggregations.isEmpty(), | ||
| "At least one aggregation is required for an aggregation query to run"); | ||
| this.aggregations = aggregations; | ||
| this.nestedStructuredQuery = nestedQuery; | ||
| this.mode = STRUCTURED; | ||
| } | ||
|
|
||
| AggregationQuery(String namespace, GqlQuery<?> gqlQuery) { | ||
| super(namespace); | ||
| this.nestedGqlQuery = gqlQuery; | ||
| this.mode = GQL; | ||
| } | ||
|
|
||
| /** Returns the {@link Aggregation}(s) for this Query. */ | ||
| public Set<Aggregation> getAggregations() { | ||
| return aggregations; | ||
| } | ||
|
|
||
| /** | ||
| * Returns the underlying {@link StructuredQuery for this Query}. Returns null if created with | ||
| * {@link GqlQuery} | ||
| */ | ||
| public StructuredQuery<?> getNestedStructuredQuery() { | ||
| return nestedStructuredQuery; | ||
| } | ||
|
|
||
| /** | ||
| * Returns the underlying {@link GqlQuery for this Query}. Returns null if created with {@link | ||
| * StructuredQuery} | ||
| */ | ||
| public GqlQuery<?> getNestedGqlQuery() { | ||
| return nestedGqlQuery; | ||
| } | ||
|
|
||
| /** Returns the {@link Mode} for this query. */ | ||
| public Mode getMode() { | ||
| return mode; | ||
| } | ||
|
|
||
| public static class Builder { | ||
|
|
||
| private String namespace; | ||
| private Mode mode; | ||
| private final Set<Aggregation> aggregations; | ||
| private StructuredQuery<?> nestedStructuredQuery; | ||
| private GqlQuery<?> nestedGqlQuery; | ||
|
|
||
| public Builder() { | ||
| this.aggregations = new HashSet<>(); | ||
| } | ||
|
|
||
| public Builder setNamespace(String namespace) { | ||
| this.namespace = namespace; | ||
| return this; | ||
| } | ||
|
|
||
| public Builder addAggregation(AggregationBuilder<?> aggregationBuilder) { | ||
| this.aggregations.add(aggregationBuilder.build()); | ||
| return this; | ||
| } | ||
|
|
||
| public Builder addAggregation(Aggregation aggregation) { | ||
| this.aggregations.add(aggregation); | ||
| return this; | ||
| } | ||
|
|
||
| public Builder over(StructuredQuery<?> nestedQuery) { | ||
| this.nestedStructuredQuery = nestedQuery; | ||
| this.mode = STRUCTURED; | ||
| return this; | ||
| } | ||
|
|
||
| public Builder over(GqlQuery<?> nestedQuery) { | ||
| this.nestedGqlQuery = nestedQuery; | ||
| this.mode = GQL; | ||
| return this; | ||
| } | ||
|
|
||
| public AggregationQuery build() { | ||
| boolean nestedQueryProvided = nestedGqlQuery != null || nestedStructuredQuery != null; | ||
| checkArgument( | ||
| nestedQueryProvided, "Nested query is required for an aggregation query to run"); | ||
|
|
||
| if (mode == GQL) { | ||
| return new AggregationQuery(namespace, nestedGqlQuery); | ||
| } | ||
| return new AggregationQuery(namespace, aggregations, nestedStructuredQuery); | ||
| } | ||
| } | ||
|
|
||
| public enum Mode { | ||
| STRUCTURED, | ||
| GQL, | ||
| } | ||
| } |
71 changes: 71 additions & 0 deletions
71
google-cloud-datastore/src/main/java/com/google/cloud/datastore/AggregationResult.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| /* | ||
| * Copyright 2022 Google LLC | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * https://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package com.google.cloud.datastore; | ||
|
|
||
| import com.google.api.core.BetaApi; | ||
| import com.google.common.base.MoreObjects; | ||
| import com.google.common.base.MoreObjects.ToStringHelper; | ||
| import java.util.Map; | ||
| import java.util.Map.Entry; | ||
| import java.util.Objects; | ||
|
|
||
| /** Represents a result of an {@link AggregationQuery} query submission. */ | ||
| @BetaApi | ||
| public class AggregationResult { | ||
|
|
||
| private final Map<String, LongValue> properties; | ||
|
|
||
| public AggregationResult(Map<String, LongValue> properties) { | ||
| this.properties = properties; | ||
| } | ||
|
|
||
| /** | ||
| * Returns a result value for the given alias. | ||
| * | ||
| * @param alias A custom alias provided in the query or an autogenerated alias in the form of | ||
| * 'property_\d' | ||
| * @return An aggregation result value for the given alias. | ||
| */ | ||
| public Long get(String alias) { | ||
| return properties.get(alias).get(); | ||
| } | ||
|
|
||
| @Override | ||
| public boolean equals(Object o) { | ||
| if (this == o) { | ||
| return true; | ||
| } | ||
| if (o == null || getClass() != o.getClass()) { | ||
| return false; | ||
| } | ||
| AggregationResult that = (AggregationResult) o; | ||
| return properties.equals(that.properties); | ||
| } | ||
|
|
||
| @Override | ||
| public int hashCode() { | ||
| return Objects.hash(properties); | ||
| } | ||
|
|
||
| @Override | ||
| public String toString() { | ||
| ToStringHelper toStringHelper = MoreObjects.toStringHelper(this); | ||
| for (Entry<String, LongValue> entry : properties.entrySet()) { | ||
| toStringHelper.add(entry.getKey(), entry.getValue().get()); | ||
| } | ||
| return toStringHelper.toString(); | ||
| } | ||
| } |
Oops, something went wrong.