Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2112 commits
Select commit Hold shift + click to select a range
b5c00da
Add ability for minion nodes to download segments from servers during…
tibrewalpratik17 May 17, 2024
b86d1a0
Allow user configurable regex library for queries (#13005)
itschrispeck May 17, 2024
d4aa66c
Introduce retries while creating stream message decoder for more robu…
tibrewalpratik17 May 17, 2024
0107bc6
Bump com.azure:azure-storage-file-datalake from 12.18.4 to 12.19.0 (#…
dependabot[bot] May 18, 2024
d4bf8f2
SQL compliance: allow INT and FLOAT literal (#13078)
Jackie-Jiang May 18, 2024
e71d1c6
Clean up BrokerRequestHandler and BrokerResponse (#13179)
Jackie-Jiang May 19, 2024
777040b
Fix the ServletConfig loading issue with swagger. (#13122)
abhioncbr May 19, 2024
a385e28
[Minor bugfix] Decrypt file in minion task executor when downloading …
tibrewalpratik17 May 19, 2024
9e1246d
Enhance Kinesis consumer (#12806)
Jackie-Jiang May 20, 2024
5345866
Adding config for having suffix for client ID for realtime consumer (…
eaugene May 20, 2024
6beabbb
Bump pulsar.version from 3.2.2 to 3.2.3 (#13182)
dependabot[bot] May 20, 2024
f2dc911
Bump bouncycastle.version from 1.78 to 1.78.1 (#13183)
dependabot[bot] May 20, 2024
34a1fb7
[multistage] Make window operator more resilient (#13180)
xiangfu0 May 21, 2024
c797305
Add PodDisruptionBudgets to the Pinot Helm chart (#13153)
andimiller May 21, 2024
56fda6b
Bump io.airlift:aircompressor from 0.26 to 0.27 (#13190)
dependabot[bot] May 21, 2024
a1991bb
Bump net.openhft:chronicle-core from 2.25ea14 to 2.25ea15 (#13192)
dependabot[bot] May 21, 2024
7aa9aeb
Add back 'numRowsResultSet' to BrokerResponse, and retain it when res…
Jackie-Jiang May 21, 2024
1feddc4
Ensure brokerId and requestId are always set in BrokerResponse (#13200)
Jackie-Jiang May 21, 2024
661e352
fix fieldsToRead class not in decoder (#13186)
rseetham May 21, 2024
652bb6b
Improved metrics for server grpc query (#13177)
soumitra-st May 22, 2024
429bb7a
Add locking logic to get consistent table view for upsert tables (#12…
klsince May 22, 2024
253ede9
Support ListAgg WITHIN GROUP clause (#13146)
xiangfu0 May 22, 2024
51bd6b3
Bump io.netty:netty-bom from 4.1.109.Final to 4.1.110.Final (#13202)
dependabot[bot] May 22, 2024
9b1df5d
Bump software.amazon.awssdk:bom from 2.25.44 to 2.25.57 (#13203)
dependabot[bot] May 22, 2024
4eb0690
Bump org.codehaus.mojo:exec-maven-plugin from 3.2.0 to 3.3.0 (#13204)
dependabot[bot] May 22, 2024
3c4fe72
[spark-connector] Add option to fail read when there are invalid segm…
cbalci May 22, 2024
ca7ab24
Properly return response if SegmentCompletion is aborted (#13206)
harold-kfuse May 23, 2024
d4e2ee1
[feature] Support funnelMaxStep function (#13176)
xiangfu0 May 23, 2024
29c560f
Move offset validation logic to consumer classes (#13015)
KKcorps May 23, 2024
0dec8a9
Refactored CommonsConfigurationUtils for loading properties configura…
abhioncbr May 23, 2024
96ef43f
[HELM]: Added additional probes options and startup probe. (#13165)
abhioncbr May 23, 2024
be6dd7e
add instrumentation to json index getMatchingFlattenedDocsMap() (#13164)
itschrispeck May 23, 2024
1e42a67
release segment mgrs more reliably (#13216)
klsince May 24, 2024
6c803e2
Update consuming freshness field in query resp to be backed by the se…
priyen May 24, 2024
c5859a1
Bump commons-cli:commons-cli from 1.7.0 to 1.8.0 (#13220)
dependabot[bot] May 24, 2024
9380168
Bump org.apache.commons:commons-compress from 1.26.1 to 1.26.2 (#13218)
dependabot[bot] May 24, 2024
fed2d5f
update access control check error handling to catch throwable and log…
dang-stripe May 24, 2024
e84f3f5
ensure inverse And/OrFilterOperator implementations match the query (…
itschrispeck May 25, 2024
1950323
Allow all raw index config in star-tree index (#13225)
Jackie-Jiang May 25, 2024
b6e8135
Allow using 'serverReturnFinalResult' to optimize server partitioned …
Jackie-Jiang May 25, 2024
518fd18
Fix some bugs related to FunnelMaxStepAggregationFunction (#13228)
Jackie-Jiang May 25, 2024
2c51d42
Return clear error message when no common broker found for multi-stag…
yashmayya May 28, 2024
c2b1132
Fixed infer logical type name from avro union schema (#13224)
rajagopr May 28, 2024
c0fce5a
upgrade to maven 3.9.7 (#13233)
sullis May 28, 2024
67a7d36
Bump com.google.apis:google-api-services-storage (#13244)
dependabot[bot] May 28, 2024
daed730
Bump com.puppycrawl.tools:checkstyle from 10.16.0 to 10.17.0 (#13239)
dependabot[bot] May 28, 2024
5d9f651
Fix error message for insufficient number of untagged brokers during …
yashmayya May 28, 2024
106f0c5
Don't enforce -realTimeInstanceCount and -offlineInstanceCount option…
yashmayya May 28, 2024
1b16615
Fix controller host / port / protocol CLI option description for admi…
yashmayya May 28, 2024
be4f740
Fix the issue that map flatten shouldn't remove the map field from th…
xiangfu0 May 28, 2024
fdfae5e
get startTime outside the executor task to avoid flaky time checks (#…
klsince May 28, 2024
8a80b80
fixing swagger setup using localhost as host name (#13254)
xiangfu0 May 29, 2024
717220c
Publish helm 0.2.9 (#13230)
abhioncbr May 29, 2024
07bd663
Fix the NPE when metadataTTL is enabled without delete column (#13262)
Jackie-Jiang May 29, 2024
67cc15a
Bump software.amazon.awssdk:bom from 2.25.57 to 2.25.61 (#13261)
dependabot[bot] May 29, 2024
f1c6a2e
Bump commons-validator:commons-validator from 1.8.0 to 1.9.0 (#13260)
dependabot[bot] May 29, 2024
49d1af0
Helm: bug fix for chart rendering issue. (#13264)
abhioncbr May 29, 2024
5a1e57d
including the underlying exception in the logging output (#13248)
fr0stbyte May 29, 2024
256d0aa
Do not pause ingestion when upsert snapshot flow errors out (#13257)
tibrewalpratik17 May 29, 2024
2f4a412
Properly handle complex type transformer in segment processor framewo…
KKcorps May 30, 2024
9302f18
Fix null literal handling for null intolerant functions in multi-stag…
yashmayya May 30, 2024
3b57116
Serialize V2 Plan using Protobufs instead of reflection. (#13221)
vrajat May 30, 2024
bdb9191
Bugfix. Allow tenant rebalance with downtime as true (#13246)
shounakmk219 May 30, 2024
d4e10e5
set hadoop-mapreduce-client-core dependency scope as compile (#13269)
gortiz May 30, 2024
f54c77d
Add a plan version field to QueryRequest Protobuf Message (#13267)
vrajat May 31, 2024
a1f9b89
Improve warn logs for requesting validDocID snapshots (#13280)
tibrewalpratik17 May 31, 2024
fcd7856
Bump com.google.cloud:libraries-bom from 26.39.0 to 26.40.0 (#13272)
dependabot[bot] May 31, 2024
61ce07f
Bump org.apache.maven.plugins:maven-enforcer-plugin from 3.4.1 to 3.5…
dependabot[bot] May 31, 2024
bf84802
Bump software.amazon.awssdk:bom from 2.25.61 to 2.25.63 (#13287)
dependabot[bot] May 31, 2024
283363a
Make allowDownloadFromServer minion-cluster-level config (#13247)
tibrewalpratik17 May 31, 2024
f70ae2d
Bump org.apache.maven.plugins:maven-shade-plugin from 3.5.3 to 3.6.0 …
dependabot[bot] May 31, 2024
cb5596c
Bump com.nimbusds:nimbus-jose-jwt from 9.39.1 to 9.39.3 (#13289)
dependabot[bot] May 31, 2024
34bab22
Add support for raw JSON filter predicates (#13283)
yashmayya Jun 1, 2024
6c85de5
Bump org.webjars:swagger-ui from 5.17.0 to 5.17.14 (#13286)
dependabot[bot] Jun 1, 2024
3dd653a
[bugfix] Handling null value for kafka client id suffix (#13279)
tibrewalpratik17 Jun 1, 2024
e2c5e73
Pass literal within AggregateCall via rexList (#13282)
Jackie-Jiang Jun 1, 2024
26209ed
Updated PDB and reload checksum config annotation for reload. (#13276)
abhioncbr Jun 1, 2024
b0e3cc7
Introduce PinotLogicalAggregate and remove internal hint (#13291)
Jackie-Jiang Jun 1, 2024
a4950fe
Remove SqlKind from FunctionCall (#13293)
Jackie-Jiang Jun 2, 2024
68685dc
Add backward compatibility regression test suite for multi-stage quer…
yashmayya Jun 3, 2024
a5d2a35
Cleanup unused aggregate internal hint (#13295)
Jackie-Jiang Jun 3, 2024
1dd4d49
Bump commons-net:commons-net from 3.10.0 to 3.11.0 (#13302)
dependabot[bot] Jun 3, 2024
3a98ab5
Bump org.apache.maven.plugins:maven-javadoc-plugin from 3.6.3 to 3.7.…
dependabot[bot] Jun 3, 2024
4fcd869
Bump com.google.errorprone:error_prone_annotations from 2.27.1 to 2.2…
dependabot[bot] Jun 3, 2024
3dfadd8
Bump software.amazon.awssdk:bom from 2.25.63 to 2.25.64 (#13300)
dependabot[bot] Jun 3, 2024
4eeae11
Bump net.openhft:posix from 2.25ea0 to 2.26ea0 (#13273)
dependabot[bot] Jun 3, 2024
18afbb4
add metrics for IdeaState update (#13266)
pengding-stripe Jun 3, 2024
2719c5c
Returning tables names failing authorization in Exception of Multi St…
eaugene Jun 4, 2024
fc9f34f
lucene `IndexOutOfBounds` bugfix, and use NRTCachingDirectory for rea…
itschrispeck Jun 4, 2024
66bce71
Short circuit SubPlanFragmenter because we don't support multiple sub…
Jackie-Jiang Jun 4, 2024
db69d3d
Add FunnelMaxStepAggregationFunction and FunnelCompleteCountAggregati…
xiangfu0 Jun 4, 2024
9822100
Add config to skip record ingestion on string column length exceeding…
tibrewalpratik17 Jun 4, 2024
164cd81
report rebalance job status for the early returns like noops (#13281)
klsince Jun 4, 2024
970d9b9
Fix the NPE from IS update metrics (#13313)
Jackie-Jiang Jun 4, 2024
e53d887
Bump org.checkerframework:checker-qual from 3.43.0 to 3.44.0 (#13309)
dependabot[bot] Jun 4, 2024
93ebc32
Bump org.apache.maven.plugins:maven-jxr-plugin from 3.3.2 to 3.4.0 (#…
dependabot[bot] Jun 4, 2024
bdef83b
Add metrics to count joins and window functions (#13032)
gortiz Jun 5, 2024
065f325
Bump software.amazon.awssdk:bom from 2.25.64 to 2.25.66 (#13316)
dependabot[bot] Jun 5, 2024
3d07b90
Fix Logging Location for CPU-Based Query Killing (#13318)
PraveenVora Jun 5, 2024
f4a4bf3
Custom configuration property reader for segment metadata files (#12440)
abhioncbr Jun 5, 2024
6290bc0
Interning for OnHeapByteDictionary (#12342)
vvivekiyer Jun 6, 2024
ba2c373
Bump org.apache.maven.plugins:maven-checkstyle-plugin (#13321)
dependabot[bot] Jun 6, 2024
2c59fbe
Bump pulsar.version from 3.2.3 to 3.3.0 (#13322)
dependabot[bot] Jun 6, 2024
b602df5
Bump software.amazon.awssdk:bom from 2.25.66 to 2.25.67 (#13323)
dependabot[bot] Jun 6, 2024
1d1d25d
Update getValidDocIdsMetadataFromServer to make call in batches to se…
tibrewalpratik17 Jun 6, 2024
fb64ec2
Fix few metric rules which were affected by the database prefix handl…
shounakmk219 Jun 7, 2024
0f92742
Multi stage metrics (#13035)
gortiz Jun 7, 2024
fc358a8
Add offset based lag metrics (#13298)
KKcorps Jun 7, 2024
4356583
[Multi-stage] Clean up RelNode to Operator handling (#13325)
Jackie-Jiang Jun 7, 2024
1e0dc30
Bump com.nimbusds:nimbus-jose-jwt from 9.39.3 to 9.40 (#13334)
dependabot[bot] Jun 7, 2024
cae5e4e
Bump lucene.version from 9.10.0 to 9.11.0 (#13328)
dependabot[bot] Jun 7, 2024
dbeeaaf
Add a benchmark to measure multi-stage block serde cost (#13336)
gortiz Jun 7, 2024
b78de53
Bump io.github.hakky54:sslcontext-kickstart-for-netty (#13329)
dependabot[bot] Jun 7, 2024
0137e32
Bump software.amazon.awssdk:bom from 2.25.67 to 2.25.68 (#13330)
dependabot[bot] Jun 7, 2024
5d1a042
Bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 (#13331)
dependabot[bot] Jun 7, 2024
18a6a0d
Bump com.azure:azure-core-http-netty from 1.15.0 to 1.15.1 (#13332)
dependabot[bot] Jun 7, 2024
dae4fbd
Bump com.azure:azure-core from 1.49.0 to 1.49.1 (#13333)
dependabot[bot] Jun 7, 2024
082ef40
Update metadata.properties to have schema max-length (#13187)
tibrewalpratik17 Jun 7, 2024
c418442
Fix flakiness of ControllerPeriodicTasksIntegrationTest (#13337)
Jackie-Jiang Jun 7, 2024
e5decf3
Extend compatibility verifier operation timeout from 1m to 2m to redu…
Jackie-Jiang Jun 7, 2024
caf2523
support deleteColumn for compactionTask by extending the record reade…
klsince Jun 7, 2024
9b75bff
handle overflow for `MutableOffHeapByteArrayStore` buffer starting si…
itschrispeck Jun 8, 2024
0dcad92
Fix LEAD/LAG window function implementation (#13340)
xiangfu0 Jun 8, 2024
70412e6
allow to set segment when use SegmentProcessorFramework (#13341)
klsince Jun 8, 2024
61aa6ce
Reposition query submission spot for adaptive server selection (#13327)
vvivekiyer Jun 8, 2024
1229add
[Multi-stage] Fix literal handling (#13344)
Jackie-Jiang Jun 8, 2024
5da68d7
Improve Retention Manager Segment Lineage Clean Up (#13232)
aishikbh Jun 9, 2024
8e5e8d9
Fix array literal handling (#13345)
Jackie-Jiang Jun 9, 2024
e8db382
[Backfill] allow externally partitioned segment uploads for upsert ta…
rohityadav1993 Jun 10, 2024
e072542
Bump dropwizard-metrics.version from 4.2.25 to 4.2.26 (#13348)
dependabot[bot] Jun 10, 2024
30a7609
Bump software.amazon.awssdk:bom from 2.25.68 to 2.25.69 (#13349)
dependabot[bot] Jun 10, 2024
b0f356c
Bump com.azure:azure-storage-file-datalake from 12.19.0 to 12.19.1 (#…
dependabot[bot] Jun 10, 2024
c5dc885
update node version in the docs (#13294)
jayeshchoudhary Jun 10, 2024
4969817
do not fail on duplicate relaxed vars (#13214)
jadami10 Jun 10, 2024
5fc9bf6
Fix URI construction so that AddSchema command line tool works when o…
suyashpatel98 Jun 10, 2024
e825e13
Revert "support deleteColumn for compactionTask by extending the reco…
klsince Jun 10, 2024
425182f
Adding a cluster config to enable instance pool and replica group con…
soumitra-st Jun 10, 2024
b371feb
extend CompactedPinotSegmentRecordReader so that it can skip deleteRe…
klsince Jun 10, 2024
393f130
Clean Google Dependencies (#13297)
abhioncbr Jun 11, 2024
ad5ca34
Add list of collaborators to asf.yaml (#13346)
yashmayya Jun 11, 2024
d09cd0c
move shouldReplaceOnComparisonTie to base class to be more reusable (…
klsince Jun 11, 2024
5ca1d97
Fix [Type]ArrayList elements() method usage (#13354)
xiangfu0 Jun 11, 2024
5c097fc
Bump io.grpc:grpc-bom from 1.62.2 to 1.64.0 (#13370)
dependabot[bot] Jun 11, 2024
0ac6f19
Bump io.netty:netty-bom from 4.1.110.Final to 4.1.111.Final (#13367)
dependabot[bot] Jun 11, 2024
8431fa4
Bump io.projectreactor:reactor-core from 3.6.6 to 3.6.7 (#13366)
dependabot[bot] Jun 11, 2024
d85f422
Bump org.apache.commons:commons-configuration2 from 2.10.1 to 2.11.0 …
dependabot[bot] Jun 11, 2024
3752ee1
Bump commons-net:commons-net from 3.11.0 to 3.11.1 (#13363)
dependabot[bot] Jun 11, 2024
ec1d187
Exclude dimensions from star-tree index stored type check (#13355)
Jackie-Jiang Jun 11, 2024
64a2ba6
Bump software.amazon.awssdk:bom from 2.25.69 to 2.25.70 (#13369)
dependabot[bot] Jun 11, 2024
ae4e7de
Bump com.azure:azure-identity from 1.12.1 to 1.12.2 (#13368)
dependabot[bot] Jun 11, 2024
5c75b15
Fix NPE in ArrayAgg functions (#13358)
xiangfu0 Jun 11, 2024
d91ad73
Optimize snapshot flow to only snapshot segments which have updates (…
tibrewalpratik17 Jun 11, 2024
36ce140
fix race condition in `ScalingThreadPoolExecutor` (#13360)
itschrispeck Jun 11, 2024
0689175
Support array sum aggregation function (#13324)
xiangfu0 Jun 12, 2024
902d9d6
Bump software.amazon.awssdk:bom from 2.25.70 to 2.26.0 (#13374)
dependabot[bot] Jun 12, 2024
e8c6052
Bump org.apache.maven.plugins:maven-dependency-plugin (#13376)
dependabot[bot] Jun 12, 2024
80aaa9c
add url assertion to SegmentCompletionProtocolTest (#13373)
sullis Jun 12, 2024
6303658
update RewriterConstants so that expr min max would not collide with …
jasperjiaguo Jun 12, 2024
37608c1
Bump software.amazon.awssdk:bom from 2.26.0 to 2.26.1 (#13386)
dependabot[bot] Jun 13, 2024
a564b7f
Bump com.google.cloud:libraries-bom from 26.40.0 to 26.41.0 (#13385)
dependabot[bot] Jun 13, 2024
030f1b0
Add clickstream example table for funnel analysis (#13379)
xiangfu0 Jun 13, 2024
7571f71
Fix segment size check in OfflineClusterIntegrationTest (#13389)
Jackie-Jiang Jun 13, 2024
0dd78f1
Refactored compatibility-verifier module (#13359)
abhioncbr Jun 14, 2024
0a8efaa
add unit test for StringFunctions encodeUrl (#13391)
sullis Jun 14, 2024
6245aa4
Use a two server setup for multi-stage query engine backward compatib…
yashmayya Jun 14, 2024
830388d
track segments for snapshotting even if they lost all comparisons (#1…
klsince Jun 14, 2024
226e731
Remove app-assembler from integration tests pom.xml (#13397)
abhioncbr Jun 14, 2024
228fa75
Bump net.openhft:chronicle-core from 2.25ea15 to 2.26ea1 (#13395)
dependabot[bot] Jun 14, 2024
4a428ec
Avoid port conflict in integration tests (#13390)
Jackie-Jiang Jun 14, 2024
6891150
[partial-upsert] configure early release of _partitionGroupConsumerSe…
rohityadav1993 Jun 14, 2024
0ff43b5
Reset controller port when stopping the controller in ControllerTest …
Jackie-Jiang Jun 14, 2024
b151554
Do not hard code resource class in BaseClusterIntegrationTest (#13400)
Jackie-Jiang Jun 15, 2024
c341e06
Add mode to allow adding dummy events for non-matching steps (#13382)
xiangfu0 Jun 15, 2024
57d6486
In ClusterTest, make start port higher to avoid potential conflict wi…
Jackie-Jiang Jun 16, 2024
0e2646f
Bump flink.version from 1.19.0 to 1.19.1 (#13405)
dependabot[bot] Jun 17, 2024
5bd9f52
Bump surefire.version from 3.2.5 to 3.3.0 (#13407)
dependabot[bot] Jun 17, 2024
720d730
Bump software.amazon.awssdk:bom from 2.26.1 to 2.26.3 (#13406)
dependabot[bot] Jun 17, 2024
86decba
Bump org.apache.maven.plugins:maven-release-plugin from 3.0.1 to 3.1.…
dependabot[bot] Jun 17, 2024
7c49920
Bump org.jline:jline from 3.26.1 to 3.26.2 (#13411)
dependabot[bot] Jun 17, 2024
909b343
Bump org.apache.maven.plugins:maven-project-info-reports-plugin (#13410)
dependabot[bot] Jun 17, 2024
6831d21
Bump org.apache.parquet:parquet-avro from 1.14.0 to 1.14.1 (#13409)
dependabot[bot] Jun 17, 2024
1e08065
Add ExceptionMapper to convert Exception to Response Object for Broke…
eaugene Jun 17, 2024
0ca1ab6
Improve logging in range index handler for index updates (#13381)
yashmayya Jun 17, 2024
4615ce5
Allow configurable queryWorkerThreads in Pinot server side GrpcQueryS…
xiangfu0 Jun 17, 2024
54fefe7
Improve error message for max rows in join limit breach (#13394)
yashmayya Jun 18, 2024
5ec7028
handle segments not tracked by partition mgr and add skipUpsertView q…
klsince Jun 18, 2024
abaf3c8
Add retry policy to wait for job id to persist during rebalancing (#1…
tahaboty-stripe Jun 18, 2024
743d6b6
add `skipUnavailableServers` query option (#13387)
itschrispeck Jun 18, 2024
99f6934
Fix an overflow in PinotDataBuffer.readFrom (#13152)
gortiz Jun 18, 2024
a2aa00c
Move resolveComparisonTies from addOrReplaceSegment to base class (#1…
tibrewalpratik17 Jun 18, 2024
c2bee3e
Bump circe.version from 0.14.7 to 0.14.8 (#13422)
dependabot[bot] Jun 18, 2024
8f4ea55
Bump software.amazon.awssdk:bom from 2.26.3 to 2.26.4 (#13423)
dependabot[bot] Jun 18, 2024
10186f4
clean up jetbrain nullable annotation (#13427)
klsince Jun 18, 2024
bb42575
Query response opt (#13420)
soumitra-st Jun 18, 2024
55b6024
Fix literal handling in Window functions (#13428)
xiangfu0 Jun 19, 2024
ddce06f
Bug fix: Handle missing shade config overwrites for Kafka (#13437)
KKcorps Jun 19, 2024
3224b36
Make 'valueIn' MV transform function work with the multi-stage query …
yashmayya Jun 20, 2024
6852205
Bump org.roaringbitmap:RoaringBitmap from 1.0.6 to 1.1.0 (#13446)
dependabot[bot] Jun 20, 2024
eb76d56
Bump org.apache.maven.plugins:maven-jar-plugin from 3.4.1 to 3.4.2 (#…
dependabot[bot] Jun 20, 2024
093827b
Allow stop to interrupt the consumer thread and safely release the re…
Jackie-Jiang Jun 20, 2024
9dd2a10
Bump com.google.cloud:libraries-bom from 26.41.0 to 26.42.0 (#13438)
dependabot[bot] Jun 20, 2024
ac4e6d1
Bump software.amazon.awssdk:bom from 2.26.4 to 2.26.6 (#13445)
dependabot[bot] Jun 20, 2024
a30c0cb
fixing the single quoted literal in window function (#13441)
xiangfu0 Jun 20, 2024
6883a4d
Improve upsert compaction threshold validations (#13424)
tibrewalpratik17 Jun 20, 2024
74e1a14
[HELM]: Added namespace support in K8s deployment. (#13380)
abhioncbr Jun 20, 2024
743f5f2
Bump com.azure:azure-identity from 1.12.2 to 1.13.0 (#13458)
dependabot[bot] Jun 21, 2024
5760d10
Fix bug preventing numeric casts for MV columns in filters using the …
yashmayya Jun 22, 2024
8df722a
Add some missing geospatial scalar functions to support use in interm…
yashmayya Jun 22, 2024
66676d9
New buffers (#13304)
gortiz Jun 24, 2024
23a3267
Bump com.fasterxml.woodstox:woodstox-core from 6.6.2 to 7.0.0 (#13470)
dependabot[bot] Jun 24, 2024
246faab
Bump org.apache.maven.plugins:maven-dependency-plugin (#13469)
dependabot[bot] Jun 24, 2024
97d7c23
Bump net.openhft:posix from 2.26ea0 to 2.26ea1 (#13460)
dependabot[bot] Jun 24, 2024
4b0dd42
Bump software.amazon.awssdk:bom from 2.26.6 to 2.26.7 (#13459)
dependabot[bot] Jun 24, 2024
9ccd1be
Enable more integration tests to run on the v2 multi-stage query engi…
yashmayya Jun 24, 2024
8a9f558
Remove legacy commons-math dependency. (#13464)
abhioncbr Jun 24, 2024
b809ccd
Fix field spec data type for JSON predicate evaluator tests (#13448)
yashmayya Jun 24, 2024
cecfe51
Added kafka partition number to metadata. (#13447)
davizucon Jun 24, 2024
fe3411d
Fix time stats in SegmentIndexCreationDriverImpl (#13429)
yashmayya Jun 24, 2024
d1cf485
Cleanup: Fix grammar in error message, also improve readability. (#13…
mayankshriv Jun 24, 2024
83b0c14
add metrics tracking lucene near real-time refresh delay (#13307)
itschrispeck Jun 26, 2024
f52e651
Add scalar functions for geohash (#13471)
KKcorps Jun 26, 2024
465a290
Fix 'WEEK' transform function in multi-stage query engine (#13483)
yashmayya Jun 26, 2024
c872bf1
[Cleanup] Move classes in pinot-common to the correct package (#13478)
Jackie-Jiang Jun 26, 2024
7dbb05d
Adding registerExtraComponents to allow registering additional compon…
soumitra-st Jun 26, 2024
55f6a6f
Removing legacy commons-lang dependency (#13480)
abhioncbr Jun 27, 2024
0b326c6
Handle errors gracefully during multi-stage stats collection in the b…
yashmayya Jun 27, 2024
07136df
Remove commons-lang from License-binary. (#13500)
abhioncbr Jun 27, 2024
8dab53a
Remove legacy commons-collections dependency. (#13501)
abhioncbr Jun 27, 2024
76729f8
Refactoring removeSegment flow in upsert (#13449)
tibrewalpratik17 Jun 27, 2024
df577b7
Bump org.apache.maven.plugins:maven-project-info-reports-plugin (#13499)
dependabot[bot] Jun 27, 2024
83de288
Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 (#13498)
dependabot[bot] Jun 27, 2024
87f0c80
Bump software.amazon.awssdk:bom from 2.26.7 to 2.26.10 (#13497)
dependabot[bot] Jun 27, 2024
8747331
ControllerRequestClient accepts headers. Useful for authN tests (#13481)
vrajat Jun 28, 2024
d7fff77
Allow PintoHelixResourceManager subclasses to be used in the controll…
9aman Jun 28, 2024
892609a
Bump lucene.version from 9.11.0 to 9.11.1 (#13505)
dependabot[bot] Jun 28, 2024
0eb614e
Bump software.amazon.awssdk:bom from 2.26.10 to 2.26.11 (#13506)
dependabot[bot] Jun 28, 2024
2328d62
Bump io.grpc:grpc-bom from 1.64.0 to 1.65.0 (#13507)
dependabot[bot] Jun 28, 2024
a0e861e
Support 'EXTRACT' as a scalar function (#13463)
yashmayya Jun 28, 2024
d447fcf
fix json_extract_index out of bounds case (#13479)
itschrispeck Jun 28, 2024
7062133
Don't drop original field during flatten (#13490)
xiangfu0 Jun 29, 2024
24775c1
Update Notice and License for 1.2.0 Release (#13691)
ankitsultana Jul 26, 2024
3390946
Potential Fix for 1.2.0 Release Failure: Switch maven-source-plugin t…
ankitsultana Jul 29, 2024
75e297e
Set maven.compiler flags explicitly (#13710)
ankitsultana Jul 30, 2024
5b9a40f
Fix mvn javadoc:javadoc goal for Pinot Release (#13719)
ankitsultana Jul 30, 2024
81cf64f
Add hadoop-client-runtime dependency to fix quickstart NoClassDefFoun…
yashmayya Jul 8, 2024
ad284a6
Minor LICENSE-binary Correction for 1.2.0 Release (#13783)
Aug 8, 2024
6943c0c
Fix filtered aggregate with ordering (#13784)
Jackie-Jiang Aug 9, 2024
cc33ac5
[maven-release-plugin] prepare release release-1.2.0-rc2
ankitsultana Aug 10, 2024
102a000
Bug fix for 1.2.0: getSegments call in ConsistentDataPush mode isn't …
jonbasebl5 Aug 24, 2025
f364937
lets guess what the build error was
jonbasebl5 Aug 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
18 changes: 18 additions & 0 deletions .asf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,26 @@
# under the License.
#

# Please refer https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features

github:
description: "Apache Pinot - A realtime distributed OLAP datastore"
homepage: https://pinot.apache.org/
labels:
- java
del_branch_on_merge: true
enabled_merge_buttons:
squash: true
merge: false
rebase: false
collaborators:
- sullis
- shenyu0127
- tibrewalpratik17
- abhioncbr
- zhtaoxiang
- shounakmk219
- itschrispeck
- soumitra-st
- swaminathanmanish
- yashmayya
1 change: 1 addition & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ updates:
directory: "/"
schedule:
interval: "daily"
open-pull-requests-limit: 20

- package-ecosystem: "npm"
directory: "/pinot-controller/src/main/resources"
Expand Down
108 changes: 108 additions & 0 deletions .github/workflows/build-multi-arch-pinot-docker-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
name: Pinot Multi-Arch Platform Docker Image Build and Publish

on:
workflow_dispatch:
inputs:
gitUrl:
description: "The Pinot git repo to check out to build, use https."
default: "https://github.com/apache/pinot.git"
required: true
commit:
description: "The branch/commit to check out to build Pinot image."
default: "master"
required: true
dockerImageName:
description: "The docker image name, default to 'apachepinot/pinot'."
default: "apachepinot/pinot"
required: true
tags:
description: "Tags to push of the image, comma separated, e.g. tag1,tag2,tag3"
default: ""

jobs:
generate-build-info:
name: Generate Build Info
runs-on: ubuntu-latest
outputs:
commit-id: ${{ steps.generate-build-info.outputs.commit-id }}
tags: ${{ steps.generate-build-info.outputs.tags }}
steps:
- uses: actions/checkout@v4
- name: Generate Build Info
id: generate-build-info
env:
PINOT_GIT_URL: ${{ github.event.inputs.gitUrl }}
PINOT_BRANCH: ${{ github.event.inputs.commit }}
TAGS: ${{ github.event.inputs.tags }}
run: |
.github/workflows/scripts/docker/.pinot_build_info_gen.sh
build-pinot-docker-image:
runs-on: ubuntu-latest
strategy:
matrix:
arch: [ "amd64", "arm64" ]
base-image-tag: [ "11-amazoncorretto", "11-ms-openjdk" ]
name: Build Pinot Docker Image on ${{ matrix.arch }} with base image ${{ matrix.base-image-tag }}
needs: [ generate-build-info ]
steps:
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/setup-qemu-action@v3
name: Set up QEMU
with:
platforms: linux/${{ matrix.arch }}
- uses: docker/setup-buildx-action@v3
name: Set up Docker Buildx
- uses: actions/checkout@v4
- name: Build and push the Docker image
env:
DOCKER_FILE_BASE_DIR: "docker/images/pinot"
DOCKER_IMAGE_NAME: ${{ github.event.inputs.dockerImageName }}
BUILD_PLATFORM: "linux/${{ matrix.arch }}"
BASE_IMAGE_TAG: ${{ matrix.base-image-tag }}
PINOT_GIT_URL: ${{ github.event.inputs.gitUrl }}
PINOT_BRANCH: "${{needs.generate-build-info.outputs.commit-id}}"
TAGS: "${{needs.generate-build-info.outputs.tags}}"
run: .github/workflows/scripts/docker/.pinot_single_platform_docker_image_build.sh
create-multi-arch-manifest:
name: Create Multi-Arch Manifest
runs-on: ubuntu-latest
needs: [ generate-build-info, build-pinot-docker-image ]
steps:
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/setup-qemu-action@v3
name: Set up QEMU
- uses: docker/setup-buildx-action@v3
name: Set up Docker Buildx
- uses: actions/checkout@v4
- name: Create Multi-Arch Manifest
env:
TAGS: "${{needs.generate-build-info.outputs.tags}}"
BUILD_PLATFORM: "linux/arm64,linux/amd64"
BASE_IMAGE_TAGS: "11-amazoncorretto,11-ms-openjdk"
run: .github/workflows/scripts/docker/.pinot_multi_arch_docker_image_manifest_package.sh
50 changes: 50 additions & 0 deletions .github/workflows/build-pinot-base-docker-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
name: Pinot Base Docker Image Build and Publish

on:
workflow_dispatch:
inputs: { }

jobs:
build-pinot-build-docker-image:
name: Build Pinot Base Docker Image
runs-on: ubuntu-latest
strategy:
matrix:
baseImageType: [ "build", "runtime" ]
openJdkDist: [ "amazoncorretto", "ms-openjdk" ]
steps:
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/setup-qemu-action@v3
name: Set up QEMU
- uses: docker/setup-buildx-action@v3
name: Set up Docker Buildx
- uses: actions/checkout@v4
- name: Build and push the Docker image
env:
OPEN_JDK_DIST: ${{ matrix.openJdkDist }}
BASE_IMAGE_TYPE: ${{ matrix.baseImageType }}
BUILD_PLATFORM: "linux/amd64,linux/arm64"
TAG: "11-${{ matrix.openJdkDist }}"
run: .github/workflows/scripts/docker/.pinot_base_docker_image_build_and_push.sh
8 changes: 4 additions & 4 deletions .github/workflows/build-pinot-docker-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Login to DockerHub
uses: docker/login-action@v1
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/setup-qemu-action@v1
- uses: docker/setup-qemu-action@v3
name: Set up QEMU
- uses: docker/setup-buildx-action@v1
- uses: docker/setup-buildx-action@v3
name: Set up Docker Buildx
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Build and push the Docker image
env:
DOCKER_FILE_BASE_DIR: "docker/images/pinot"
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/build-presto-docker-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Login to DockerHub
uses: docker/login-action@v1
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/setup-qemu-action@v1
- uses: docker/setup-qemu-action@v3
name: Set up QEMU
- uses: docker/setup-buildx-action@v1
- uses: docker/setup-buildx-action@v3
name: Set up Docker Buildx
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Build and push the Docker image
env:
DOCKER_FILE_BASE_DIR: "docker/images/pinot-presto"
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/build-superset-docker-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Login to DockerHub
uses: docker/login-action@v1
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/setup-qemu-action@v1
- uses: docker/setup-qemu-action@v3
name: Set up QEMU
- uses: docker/setup-buildx-action@v1
- uses: docker/setup-buildx-action@v3
name: Set up Docker Buildx
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Build and push the Docker image
env:
DOCKER_FILE_BASE_DIR: "docker/images/pinot-superset"
Expand Down
20 changes: 16 additions & 4 deletions .github/workflows/pinot_compatibility_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,23 @@ jobs:
test_suite: [ "compatibility-verifier/sample-test-suite" ]
name: Pinot Compatibility Regression Testing against ${{ github.event.inputs.oldCommit }} and ${{ github.event.inputs.newCommit }} on ${{ matrix.test_suite }}
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Set up JDK 11
uses: actions/setup-java@v1
uses: actions/setup-java@v4
with:
java-version: 11
distribution: 'temurin'
cache: 'maven'
- name: Setup node
uses: actions/setup-node@v4
with:
node-version: v16.15.0
cache: 'npm'
cache-dependency-path: pinot-controller/src/main/resources/package-lock.json
- name: Install npm
run: |
npm install -g npm@8.5.5
npm --version
- name: Pinot Compatibility Regression Testing
if : ${{github.event_name == 'workflow_dispatch'}}
env:
Expand All @@ -50,7 +62,7 @@ jobs:
TEST_SUITE: ${{ matrix.test_suite }}
MAVEN_OPTS: >
-Xmx2G -DskipShade -DfailIfNoTests=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=25
-Dmaven.wagon.http.retryHandler.count=3 -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false
-Dmaven.wagon.http.retryHandler.count=30 -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false
-XX:+IgnoreUnrecognizedVMOptions
--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
Expand All @@ -62,7 +74,7 @@ jobs:
if: always()
run: |
zip -1 -r artifacts.zip /tmp/compatibility-verifier/*
- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v4
name: Store compatibility verifier work directory
if: always()
with:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
name: Pinot Multi-Stage Query Engine Compatibility Regression Test

on:
workflow_dispatch:
inputs:
oldCommit:
description: "Git hash (or tag) for old commit. (required)"
required: true
newCommit:
description: "Git hash (or tag) for new commit. (required)"
required: true

jobs:
compatibility-verifier:
runs-on: ubuntu-latest
strategy:
matrix:
test_suite: [ "compatibility-verifier/multi-stage-query-engine-test-suite" ]
name: Pinot Multi-Stage Query Engine Compatibility Regression Testing against ${{ github.event.inputs.oldCommit }} and ${{ github.event.inputs.newCommit }} on ${{ matrix.test_suite }}
steps:
- uses: actions/checkout@v4
- name: Set up JDK 11
uses: actions/setup-java@v4
with:
java-version: 11
distribution: 'temurin'
cache: 'maven'
- name: Setup node
uses: actions/setup-node@v4
with:
node-version: v16.15.0
cache: 'npm'
cache-dependency-path: pinot-controller/src/main/resources/package-lock.json
- name: Install npm
run: |
npm install -g npm@8.5.5
npm --version
- name: Pinot Multi-Stage Query Engine Compatibility Regression Testing
if : ${{github.event_name == 'workflow_dispatch'}}
env:
OLD_COMMIT: ${{ github.event.inputs.oldCommit }}
NEW_COMMIT: ${{ github.event.inputs.newCommit }}
WORKING_DIR: /tmp/multi-stage-compatibility-verifier
TEST_SUITE: ${{ matrix.test_suite }}
MAVEN_OPTS: >
-Xmx2G -DskipShade -DfailIfNoTests=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=25
-Dmaven.wagon.http.retryHandler.count=30 -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false
-XX:+IgnoreUnrecognizedVMOptions
--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
run: .github/workflows/scripts/.pinot_compatibility_verifier.sh
- name: Archive artifacts into zip
if: always()
run: |
zip -1 -r artifacts.zip /tmp/multi-stage-compatibility-verifier/*
- uses: actions/upload-artifact@v4
name: Store multi-stage compatibility verifier work directory
if: always()
with:
## TODO: currently matrix.test_suite cannot be used as part of name due to invalid path character.
name: multi_stage_compatibility_verifier_work_dir
retention-days: 3
path: artifacts.zip
Loading