Actiknow
Data Engineering

BigQuery Project Caps: How to Stop One Team From Consuming Shared Analytics Capacity

Learn how BigQuery project caps, reservations and concurrency controls help protect critical dashboards, govern shared slots and avoid workload contention.

Data platform architect presenting bigquery project caps and shared analytics capacity governance to business intelligence leaders
Contents hide

Executive summary

A shared analytics platform is economical until one team’s workload interferes with everyone else’s. A data science experiment starts hundreds of queries, a development pipeline reruns a large transformation, or an analyst launches a broad scan during the executive reporting window. Suddenly a dashboard that normally refreshes in minutes misses its deadline. The immediate reaction is often to buy more compute. That may solve a capacity shortage, but it does not solve the governance problem: who is allowed to consume shared resources, how much, and when?

BigQuery project caps provide an additional control for capacity-based environments. Administrators can set approximate maximum slot consumption and query admission concurrency for projects using a reservation. They can define reservation-wide defaults and override them for particular projects. The feature is currently in Preview, and its slot cap is not a hard ceiling. That distinction matters when executives expect a financial guardrail or an absolute performance guarantee.

The better approach is to start with business service levels, measure real contention, use reservations for meaningful isolation, apply project caps where shared capacity still makes sense, and monitor the effect on both protected and constrained workloads. The objective is not to make every team equally slow. It is to keep essential decisions and pipelines dependable without paying for a separate large capacity pool for every project.

1. The problem: fair scheduling is not the same as business priority

BigQuery allocates query processing through slots. A slot is a unit of compute capacity, not a dedicated CPU that can be permanently assigned to a single query. With capacity-based pricing, teams use reservations containing baseline capacity, optional autoscaling and, where enabled, eligible idle capacity borrowed from other reservations. Projects, folders and organizations can be assigned to reservations through the resource hierarchy.

Within a reservation, BigQuery uses fair scheduling. The scheduler shares capacity across active projects and then among jobs within each project. This prevents a project from indefinitely taking everything, but it does not understand that a board dashboard due at 8:30 a.m. is more important than an exploratory model. Equal access is a technical policy; criticality is a business decision.

Imagine three projects using the same reservation: finance reporting, marketing analysis and experimentation. At 8:00 a.m., finance needs a reliable refresh. Marketing runs its scheduled attribution workload. Experimentation starts a batch of high-parallelism queries. The platform can be behaving as designed while finance’s elapsed time becomes unacceptable. Increasing the reservation may help temporarily, yet the next unbounded workload can recreate the same competition.

Analytics platform manager reviewing competing finance, marketing and experimentation workloads with department teams

Before changing controls, distinguish three different problems: insufficient total capacity, unfair or undesirable distribution of adequate capacity, and inefficient queries consuming more slot time than the business value justifies. Project caps principally address the second problem. They can make the third problem visible, but do not optimize SQL or fix a fundamentally undersized platform.

2. What BigQuery project caps actually control

Maximum slots: a best-effort project consumption boundary

A project max-slots policy limits the approximate number of slots consumed by queries from the assigned project within a reservation. Google documents a minimum configured value of 100 slots. This is an approximate cap, not a strict upper bound; brief or workload-dependent deviations should not be treated as a product defect or an audit failure. It is a scheduler policy rather than a billing spending limit.

A max-slots value of 200 should therefore be interpreted as an operational intent to constrain that project’s share, not a guarantee that a graph will never show 201 slots. Nor does a slot cap mean that every query receives 200 slots. A query may need fewer, may share capacity with other jobs in the project, or may wait for available resources.

Maximum concurrency: control admission, not running-job termination

The concurrency setting limits how many queries from the project are admitted simultaneously into execution. Excess queries can queue. Lowering the cap does not cancel already running queries; it affects subsequent admissions. This is useful for a development project that generates bursts of many simultaneous jobs, but it can increase waiting time and queue timeouts if the producer continues submitting work faster than the constrained project can complete it.

Concurrency and slots solve different bottlenecks. A few complex queries may saturate many slots even at low concurrency. Hundreds of small queries may create admission pressure while each uses relatively little capacity. Applying only one control without understanding the workload shape is a common source of disappointing results.

Cloud platform engineers reviewing BigQuery query admission and project concurrency controls in an office

Defaults and overrides

Administrators can configure default project scheduling policies on a reservation. Those defaults apply to projects using that reservation unless a project-specific assignment rule overrides them. This allows a common baseline for noncritical consumers and a different policy for a known exception. Project-specific caps are assignment resources governing scheduling behavior; they do not replace the assignment that determines which reservation a project uses.

Project cap overrides require a Google Cloud project assignee, not a folder or organization. Google notes that changes can take up to a minute to apply, and an updated project max-slots override requires a new query to start before taking effect. These details belong in any change-management runbook, especially during an incident.

3. Choose the right control before introducing caps

Project caps are one layer of workload management. They should be evaluated against four alternatives rather than treated as a universal answer.

Separate reservations for hard business boundaries

If executive reporting must remain available even when experimentation spikes, a separate reservation with sufficient baseline capacity can provide stronger workload isolation. The trade-off is cost and potentially underused dedicated capacity. Idle-slot sharing and reservation configuration affect how much protection exists in practice. For strict deadlines, Google advises making production reservations self-reliant rather than depending on borrowable idle slots.

Reservation autoscaling for genuine aggregate demand

If all teams have legitimate simultaneous demand and the reservation is routinely saturated, autoscaling or additional baseline capacity may be necessary. A project cap cannot create processing capacity. A capped project might simply take longer while other projects improve. Compare the cost of extra capacity with the business cost of delayed reports, but avoid assuming that faster queries always produce proportional economic value.

Query optimization and workload design

Partition filters, clustering choices, incremental models, materialized results, caching opportunities and query-plan inspection can reduce unnecessary work. Scheduling large batch transformations outside the reporting window can be more effective than capping them. For repeated dashboard queries, a precomputed reporting table may make performance more predictable than an expensive ad hoc join.

Project structure and assignment discipline

A cap is project-scoped. If a business unit mixes production dashboards, development notebooks and batch backfills inside one project, a single project policy cannot express three different priorities. Splitting workloads into distinct execution projects may be a prerequisite for meaningful governance. Dataset ownership and job execution project are not necessarily the same thing; identify where queries actually run before drawing the boundary.

4. Design a business-first workload policy

Start by classifying workloads into service tiers, then map each tier to projects and reservations. A useful first draft distinguishes Tier 1 time-bound operational and executive reporting, Tier 2 routine production transformations, Tier 3 interactive departmental analysis, and Tier 4 experimentation or bulk backfills. The names matter less than a written agreement on deadlines, owners and acceptable delays.

Business and data platform stakeholders organizing analytics workloads into service priority tiers during a planning workshop

For Tier 1, define a concrete promise such as ‘the revenue dashboard’s 8:30 a.m. refresh finishes by 8:20 a.m. on 99% of business days.’ For Tier 2, define a pipeline completion deadline and the maximum acceptable recovery window. For Tier 3, set a user-facing latency objective. For Tier 4, agree that work may queue during peak periods and may be moved to a scheduled window.

Each tier needs an accountable owner. The platform team owns the scheduler policy and telemetry, while the business owner approves the consequences of delayed work. Without that separation, a technical team can optimize slot utilization while unintentionally missing the reports executives actually need.

Illustrative capacity allocation

Consider an illustrative 1,000-slot reservation serving four projects. The finance project supports time-bound dashboards; the operations project runs hourly transformations; marketing performs interactive analysis; and a sandbox handles experiments. One possible policy is to isolate finance into a different reservation and leave the remaining three projects sharing the original pool, with a conservative concurrency limit on the sandbox. Another is to keep them together and test project overrides. Neither design is automatically superior; the answer depends on observed contention, criticality and cost.

Suppose the sandbox previously averaged 300 slots during a 20-minute peak and the proposed approximate cap is 150 slots. If its total computational work were fixed at 300 slot-minutes, a simplified calculation suggests 2 minutes at 150 slots rather than 1 minute at 300 slots. Real BigQuery jobs do not scale perfectly: stage dependencies, skew, overhead, data movement and scheduling can change both elapsed time and total slot consumption. Treat such arithmetic as a planning intuition, not a performance prediction.

Likewise, if a team submits 60 queries per minute but a concurrency policy allows only 10 running queries, the system is sustainable only if those ten queries finish quickly enough. If average completion is 30 seconds, the idealized throughput is about 20 queries per minute. That would create a growing queue of roughly 40 queries per minute under continuous arrival. Actual performance depends on job duration distribution and available slots. Capacity policies must therefore be paired with admission-rate and queue-age monitoring.

5. Measure before making changes

A defensible policy begins with at least one representative business cycle of telemetry, including month-end, campaign peaks and known batch windows where relevant. For many teams, seven days is a minimum useful starting point, not proof that the platform has seen its worst case. Use BigQuery’s administrative charts, Cloud Monitoring and INFORMATION_SCHEMA views to understand reservation utilization and individual job behavior.

At the project level, capture submitted and completed query counts, median and high-percentile elapsed times, queue wait, slot milliseconds, failures, timeouts and concurrency. At the reservation level, capture allocated baseline slots, autoscaling behavior, borrowed idle capacity and periods of saturation. Tag business-critical jobs so that you can report service-level outcomes separately from generic workload averages.

The JOBS and JOBS_BY_ORGANIZATION information schema views can expose fields such as project_id, reservation_id, creation_time, start_time, end_time and total_slot_ms, subject to access permissions and regional scope. The reservation_id identifies the reservation a job used. Be careful with parent SCRIPT jobs: parent records can overlap their child jobs’ reported usage, so exclude or handle them explicitly when aggregating slot consumption.

A practical SQL baseline

The following example summarizes completed query activity by project over seven days in the US multi-region. Replace the region qualifier and verify access to organization-wide metadata. It measures consumption and latency, not exact instantaneous slot concurrency or the billed cost of autoscaling.

SELECT

  project_id,

  COUNT(*) AS query_jobs,

  ROUND(SUM(total_slot_ms) / 3600000, 1) AS slot_hours_used,

  ROUND(AVG(TIMESTAMP_DIFF(end_time, start_time, SECOND)), 1) AS avg_run_seconds,

  ROUND(AVG(TIMESTAMP_DIFF(start_time, creation_time, SECOND)), 1) AS avg_prestart_seconds

FROM `region-us`.INFORMATION_SCHEMA.JOBS_BY_ORGANIZATION

WHERE creation_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)

  AND job_type = ‘QUERY’

  AND statement_type != ‘SCRIPT’

  AND state = ‘DONE’

  AND start_time IS NOT NULL

  AND end_time IS NOT NULL

GROUP BY project_id

ORDER BY slot_hours_used DESC;

Data analyst monitoring bigquery project query activity, slot consumption and execution performance across analytics dashboards

Averages are insufficient for deadline-sensitive workflows. Review p95 or p99 completion time and prestart delay for the actual production jobs, preferably by job label or known query pattern. A query can have low average latency while missing the one window that matters most. Also compare slot usage to business outcomes: a high-slot project may be essential, while a low-slot project can still block a critical deadline through poor timing.

6. Configure project caps safely

Begin with a noncritical project in a representative reservation and document the existing reservation configuration. In the Google Cloud console, navigate to BigQuery, Administration, Workload management. Reservation defaults can be configured in the reservation settings. For a specific project, use Assignment rules, Create assignment rule, select Project cap, choose the project and reservation, and enter Max slots and/or Max concurrency. Google also documents SQL and bq command-line methods for these policies.

For an existing reservation, a project-specific SQL policy has this documented shape. Substitute actual identifiers, choose a maximum of at least 100 slots when specifying max slots, and confirm the Preview feature is available to your organization:

CREATE ASSIGNMENT

  `admin_project.region-us.shared_reservation.sandbox_cap`

OPTIONS (

  assignee = ‘projects/sandbox_project’,

  scheduling_policy_max_slots = 200,

  scheduling_policy_concurrency = 8

);

This is an illustrative template, not a command to run without review. Confirm the actual administration project, location, reservation, assignment name and project ID; ensure that the project already uses the intended reservation through its workload assignment. A scheduling policy assignment changes scheduling behavior, not which reservation executes the workload.

Roll out in stages. First, enable a conservative cap on a sandbox project during a low-risk window. Second, run representative query batches and compare queue time, slot usage, runtime and completion success against baseline. Third, increase demand to simulate a busy reporting period. Fourth, introduce production defaults only after stakeholders accept the impact. Keep a documented path to relax or remove the policy.

Test both directions. Verify that the capped project cannot routinely dominate capacity, and that the protected projects actually improve. Then check whether the constrained project develops an unacceptable backlog. A policy that protects one team by silently breaking another is not successful governance.

Cloud engineering and business teams reviewing a staged bigquery project caps rollout and production change approval plan

7. The metrics an executive dashboard should show

A workload governance dashboard should answer four questions:

  • Are critical reports on time?
  • Is shared capacity being used efficiently?
  • Which projects are creating contention?
  • And are controls causing unintended delays?

Keep the first screen focused on outcomes, with engineering detail available for investigation.

Show the percentage of Tier 1 jobs completed before deadline; p95 end-to-end refresh duration; failed or timed-out critical jobs; reservation saturation intervals; autoscaling slot-hours or related billed capacity; project slot-hour share; and p95 query admission delay by project. A trend showing ‘finance deadline met’ alongside ‘sandbox queue age’ tells a more useful story than a chart of total slots alone.

When this measurement layer requires combining BigQuery job telemetry, billing exports and business deadlines, the data integration work needs explicit source ownership, refresh expectations and validation rules. A reporting system is only as useful as the definitions and data pipelines behind it.

Add a change annotation whenever a cap, assignment or reservation configuration is modified. Without annotations, a faster dashboard after a policy change may be incorrectly credited to query optimization, or a slowdown may be blamed on the scheduler when the underlying data volume doubled.

Distinguish operational usage from invoices. Query total_slot_ms is valuable for workload attribution, but autoscaling is billed according to provisioned scaling behavior and billing rules rather than simply the sum of each query’s reported slot time. Use Cloud Billing export and reservation attribution information for finance-grade cost allocation.

8. Risks and failure modes to plan for

Mistaking an approximate cap for a hard budget

A project slot cap is not a monetary spending limit. It applies within a reservation and does not replace billing budgets, alerts or cost allocation. If an executive requires an enforceable spending ceiling, discuss separate cost controls and workload scheduling; do not represent project caps as a guarantee of maximum monthly charges.

Setting a concurrency cap without managing the queue

Constraining admitted queries can increase waiting jobs. Monitor queued-job counts, age and timeouts, and review producer behavior. Batch pipelines may need backpressure, fewer simultaneous submissions or a wider overnight processing window. In BigQuery, the overall reservation concurrency target and the project concurrency policy are different controls; document both.

Overconstraining shared production projects

A production project may contain both essential and discretionary work. A cap applies to the project, not to a particular dashboard. Separate execution projects or reservations if critical and noncritical jobs need different treatment. Query labels help measure work but do not by themselves make project-level scheduling policies selective.

Relying on idle slots for guaranteed performance

Borrowed idle capacity can disappear when its owning reservation needs it. A test performed on a quiet day may make a small reservation look sufficient. Rehearse the scenario where neighboring reservations become busy. For strict service levels, secure baseline or appropriate autoscaling capacity rather than counting on unused slots elsewhere.

Ignoring regional and organizational boundaries

Reservations and their assignments are location-specific, and available slot capacity is not a global pool. Confirm that project jobs and reservations operate in the expected region. Also review the edition, administration project, assignment inheritance and required permissions before adopting a cross-team policy.

Treating Preview behavior as a contractual SLA

Google currently identifies project caps and scheduling policies as Preview features under pre-GA terms. Document the product status, the operational owner and a fallback design. If strict isolation is mandatory, evaluate established reservation boundaries rather than making a critical production promise depend solely on a Preview scheduler feature.

9. A four-week implementation plan

Week 1: inventory and baseline

Identify every query-producing project and its reservation assignment. Map critical dashboards, pipelines and deadlines. Extract job history and capacity metrics. Establish an owner for each workload tier. Deliver a simple map of who uses which reservation, when, and with what business consequence if delayed.

Week 2: policy design and simulation

Choose between isolation, autoscaling, optimization and caps for each workload class. Propose initial project defaults and exceptions based on observed peaks, not organizational politics. Estimate the impact on queue age and batch completion. Agree on explicit rollback triggers, including missed deadlines and unacceptable timeouts.

Week 3: controlled pilot

Enable caps on one or two noncritical projects. Run normal and peak-like tests. Compare before-and-after metrics, including the protected project’s completion times and the constrained project’s queue and failure rates. Document policy propagation and the effect of new queries on updated overrides.

Week 4: production decision and governance

Review pilot evidence with finance, analytics and platform owners. Roll out only the policies that improve business outcomes. Add configuration change approval, dashboards, alert thresholds and quarterly capacity reviews. Reassess after major data growth, new departments, seasonal demand or changes in the BigQuery feature’s release status.

10. Executive acceptance checklist

Before approving rollout, require clear answers to these questions:

  • Which reporting deadlines are protected?
  • Which projects are allowed to slow down?
  • What data proves contention rather than inefficient SQL?
  • Are project boundaries aligned to workload criticality?
  • Are slot and concurrency settings appropriate for the observed workload shape?
  • Is the Preview status acceptable?
  • Can the team detect queue buildup and timeouts?
  • Is there a documented rollback?
  • Can finance distinguish usage attribution from billed capacity?
  • Who reviews exceptions and when?

If the answers are vague, do not start with a large organization-wide policy. Start with an observable, reversible pilot. The best governance change is one whose impact can be explained in business terms and measured in production.

Frequently asked questions

1. What are BigQuery project caps?

They are scheduling policies that constrain a project’s approximate slot consumption and/or simultaneous query admissions within a BigQuery reservation. They are currently in Preview.

2. Are project slot caps hard limits?

No. Google’s documentation describes the maximum-slot setting as approximate rather than strict. Use it as a workload scheduling control, not a compliance-grade ceiling.

3. What is the minimum project max-slots value?

Google currently documents 100 slots as the minimum allowed value for the maximum-slots policy.

4. Can project caps be applied to an entire folder?

Project-specific override assignment rules require a project assignee. Reservation-level defaults can affect projects using the reservation, but a project-cap override is not a folder-level assignment.

5. What happens to queries already running if concurrency is reduced?

They are not canceled by the policy change. The reduced concurrency affects admission of subsequent queries, which can wait in the queue.

6. Do project caps replace reservations?

No. Reservations determine the capacity pool and assignment context. Caps govern how individual projects share that pool. Separate reservations remain useful for stronger workload isolation.

7. Can caps reduce BigQuery spending?

They may help restrain demand that would otherwise drive capacity usage, but savings are not guaranteed. Billing depends on reservation configuration, baseline commitments, autoscaling and actual behavior. Measure cost using billing data.

8. Do caps work for on-demand query billing?

The documented project caps are policies for projects using a reservation. Do not assume they provide equivalent controls for the on-demand pricing model.

9. Which should I set first: max slots or max concurrency?

Use measured bottlenecks. A few resource-heavy queries suggest examining slot consumption; a burst of many simultaneous queries suggests concurrency. Some workloads need both, but either can worsen queues if set too low.

10. How quickly do policy changes take effect?

Google says changes can take up to one minute to apply. For a project-specific max-slots override, a new query must start before the updated policy takes effect.

11. How can we tell whether a cap is helping?

Compare business-critical completion deadlines, p95 query runtime, queue wait, project slot share and failure rates before and after. Look for improvement in protected workloads without unacceptable degradation elsewhere.

12. What should we do if a project repeatedly hits its cap?

Determine whether the work is necessary and time-sensitive. Optimize queries, reschedule jobs, split workload classes, increase its allocation or move it to a separate reservation. Repeated saturation is evidence for review, not automatically a reason to raise every limit.

Further reading and implementation references

For product behavior and limitations, consult Google’s BigQuery reservations and project caps documentation, the project-specific assignment rules guide, and the reservation monitoring guide. These sources are authoritative for Preview availability and API syntax and should be reviewed again before configuration changes.

Where Actiknow can help

If several departments share a data platform, the most useful first step is a workload inventory and a small, measurable capacity-governance pilot. Actiknow’s business intelligence consulting and data integration services can support the reporting, workload analysis and operational dashboards needed to make that decision. Explore our business intelligence services, or contact Actiknow to discuss the scope of a focused assessment. The goal is to protect important reporting without adding capacity or complexity that the business does not need.