Google Cloud Credit Grants

Status

Closed

Date

Call for proposals will open in Spring 2025

We have partnered with Google to offer researchers the chance to submit proposals and receive up to $100,000 of credits to be used on Google Cloud in support of the HAI research vision (AI: Human impact, Augmenting human capabilities, and studying Intelligence). This program provides Stanford researchers the unique opportunity to access more credits than generally offered through the Google for Higher Education programs.

Proposals of all sizes will be considered; however, small and medium proposals for initial exploration of cloud computing usability will be prioritized. We will give preference to proposals that distinctively take advantage of Google Cloud capabilities rather than other computing infrastructures, including but not limited to: particular Google services, dynamic scalability to high distributed peak computing availability, high-availability servers, serverless event-driven computing, evaluation and testing against a wide variety of machine architectures, and needing not widely available particular capabilities, such as high-end GPUs. Credits are intended to be used to advance promising, novel, or emerging research that requires advanced computational resources provided by the commercial cloud.

HAI will evaluate projects on (I) the fit of the work with HAI’s focus areas, (II) how exciting and impactful we believe the research project will be, (III) the likelihood to initiate or sustain meaningful interdisciplinary collaborations across the University, and (IV) its need for, use of, and sensible allocation of cloud infrastructure. HAI does not attempt a detailed technical evaluation of your research.

HAI focus areas:

Human Impact:
Guiding, forecasting, and studying the human and societal impact of AI, domestically and globally
Augmenting Human Capabilities:
Designing and creating AI applications that augment human capabilities.
Inspired Intelligence:
Developing AI technologies inspired by the versatility and depth of human intelligence.

Please note: This program is now quite competitive due to greatly increased demand. A PI may submit more than one proposal, but only one proposal is likely to receive funding.

Proposal Requirements

Reporting Requirements

Data Security

Award Selection

Terms & Conditions

Link copied to clipboard!

Understanding Social Reasoning in Language Models with Language Models
Noah Goodman, Tobias Gerstenberg, Jan-Philipp Fränken, Kanishk Gandhi
Sep 25
Research
As Large Language Models (LLMs) become increasingly integrated into our everyday lives, understanding their ability to comprehend human mental states becomes critical for ensuring effective interactions. However, despite the recent attempts to assess the Theory-of-Mind (ToM) reasoning capabilities of LLMs, the degree to which these models can align with human ToM remains a nuanced topic of exploration. This is primarily due to two distinct challenges: (1) the presence of inconsistent results from previous evaluations, and (2) concerns surrounding the validity of existing evaluation methodologies. To address these challenges, we present a novel framework for procedurally generating evaluations with LLMs by populating causal templates. Using our framework, we create a new social reasoning benchmark (BigToM) for LLMs which consists of 25 controls and 5,000 model-written evaluations. We find that human participants rate the quality of our benchmark higher than previous crowd-sourced evaluations and comparable to expert-written evaluations. Using BigToM, we evaluate the social reasoning capabilities of a variety of LLMs and compare model performances with human performance. Our results suggest that GPT4 has ToM capabilities that mirror human inference patterns, though less reliable, while other LLMs struggle.
Finding Monosemantic Subspaces and Human-Compatible Interpretations in Vision Transformers through Sparse Coding
Mykel Kochenderfer, Romeo Valentin, Summeet Singh, Vikas Sindhwan, Vincent Vanhoucke
Jan 01
Research
We present a new method of deconstructing class activation tokens of vision transformers into a new, overcomplete basis, where each basis vector is “monosemantic” and affiliated with a single, human-compatible conceptual description. We achieve this through the use of a highly optimized and customized version of the K-SVD algorithm, which we call Double-Batch K-SVD (DBK-SVD). We demonstrate the efficacy of our approach on the sbucaptions dataset, using CLIP embeddings and comparing our results to a Sparse Autoencoder (SAE) baseline. Our method significantly outperforms SAE in terms of reconstruction loss, recovering approximately 2/3 of the original signal compared to 1/6 for SAE. We introduce novel metrics for evaluating explanation faithfulness and specificity, showing that DBK-SVD produces more diverse and specific concept descriptions. We therefore show empirically for the first time that disentangling of concepts arising in Vision Transformers is possible, a statement that has previously been questioned when applying an additional sparsity constraint. Our research opens new avenues for model interpretability, failure mitigation, and downstream task domain transfer in vision transformer models. An interactive demo showcasing our results can be found at https://disentangling-sbucaptions.xyz, and we make our DBK-SVD implementation openly available at https://github.com/RomeoV/KSVD.jl.

researchGrant

Google Cloud Credit Grants

Status

Closed

Date

Call for proposals will open in Spring 2025

HAI focus areas:

Please note: This program is now quite competitive due to greatly increased demand. A PI may submit more than one proposal, but only one proposal is likely to receive funding.

Proposal Requirements

Proposal length requirements by tiers:

Small: ≤ $5,000 for 12 months - One paragraph of text
Medium: $5,001 – $20,000 for 12 months - Up to one page
Large: $20,001 – $100,000 for 12 months - Up to two pages, excluding references

Proposals should cover the below items, but budget details are only needed for Medium and Large proposals.

Scope of Work
- Brief overview of the project (e.g., Background, Objectives, Expected significance, Relation to longer-term goals, Relation to the present state of knowledge in the field, etc.)
- Statement of Computational Work: Description of experimental methods and computational work, including how you will use the cloud credits for this work
Expected results or deliverables and timeline
Budget: use Google Cloud Credit Pricing Calculator to estimate usage and produce a budget (for Medium and Large projects)
Budget Justification: Briefly justify how the requested machine type(s) and estimated usage is suitable for the proposed work

Period of Performance: One calendar year from the date the credits are deposited into the researcher’s account. Extensions are not permitted.

Applications will be accepted and considered three times a year. The Google Cloud Credit Program is becoming increasingly popular, so HAI may not be able to honor full credit requests and credits may be split across multiple deposits.

Proposals will be accepted from all Stanford researchers, however students or other researchers who do not have PI status will be required to secure endorsement of the proposal from their PI-eligible faculty mentor.

Grant recipients may be requested to participate in a short interview to describe their project and/or participate in a feedback survey to enable continuous improvement of the HAI-Google Cloud Credits Program.

Reporting Requirements

Timely and substantive reporting of the value derived from cloud credits grants is important to HAI’s ability to continue and expand our grant programs.

One year after account set up or upon exhaustion of the credits allocated, recipients must provide a final report of research results, credit usage, and a list of publications, articles, or conference talks emerging from the research. Projects receiving larger credit grants will be expected to provide more detailed reports.

Data Security

Google Cloud Credits are suitable for Low, Moderate and High Risk Data as classified by the Information Security Office. Learn more about comprehensive information on classification.

Google is authorized for use with High Risk Data after a Data Risk Assessment is completed. Applicant agrees to adhere to the Minimum Security Standards for Infrastructure-as-a-Service (IaaS) and Containerized Solutions and the Administrative Guide Section 6.3.1: Information Security.

Award Selection

These grants are made possible by a gift from Google. Award decisions will be made by Stanford HAI. Please see our statements on HAI’s commitment to independence, and to transparency around our values and our fundraising policy, as well as Stanford’s policy on openness in research.

Terms & Conditions

Please read the Google Promotional Credit Terms & Conditions

Questions? E-mail us at hai-cloud-credits@stanford.edu

Link copied to clipboard!

Understanding Social Reasoning in Language Models with Language Models
Noah Goodman, Tobias Gerstenberg, Jan-Philipp Fränken, Kanishk Gandhi
Sep 25
Research
As Large Language Models (LLMs) become increasingly integrated into our everyday lives, understanding their ability to comprehend human mental states becomes critical for ensuring effective interactions. However, despite the recent attempts to assess the Theory-of-Mind (ToM) reasoning capabilities of LLMs, the degree to which these models can align with human ToM remains a nuanced topic of exploration. This is primarily due to two distinct challenges: (1) the presence of inconsistent results from previous evaluations, and (2) concerns surrounding the validity of existing evaluation methodologies. To address these challenges, we present a novel framework for procedurally generating evaluations with LLMs by populating causal templates. Using our framework, we create a new social reasoning benchmark (BigToM) for LLMs which consists of 25 controls and 5,000 model-written evaluations. We find that human participants rate the quality of our benchmark higher than previous crowd-sourced evaluations and comparable to expert-written evaluations. Using BigToM, we evaluate the social reasoning capabilities of a variety of LLMs and compare model performances with human performance. Our results suggest that GPT4 has ToM capabilities that mirror human inference patterns, though less reliable, while other LLMs struggle.
Finding Monosemantic Subspaces and Human-Compatible Interpretations in Vision Transformers through Sparse Coding
Mykel Kochenderfer, Romeo Valentin, Summeet Singh, Vikas Sindhwan, Vincent Vanhoucke
Jan 01
Research
We present a new method of deconstructing class activation tokens of vision transformers into a new, overcomplete basis, where each basis vector is “monosemantic” and affiliated with a single, human-compatible conceptual description. We achieve this through the use of a highly optimized and customized version of the K-SVD algorithm, which we call Double-Batch K-SVD (DBK-SVD). We demonstrate the efficacy of our approach on the sbucaptions dataset, using CLIP embeddings and comparing our results to a Sparse Autoencoder (SAE) baseline. Our method significantly outperforms SAE in terms of reconstruction loss, recovering approximately 2/3 of the original signal compared to 1/6 for SAE. We introduce novel metrics for evaluating explanation faithfulness and specificity, showing that DBK-SVD produces more diverse and specific concept descriptions. We therefore show empirically for the first time that disentangling of concepts arising in Vision Transformers is possible, a statement that has previously been questioned when applying an additional sparsity constraint. Our research opens new avenues for model interpretability, failure mitigation, and downstream task domain transfer in vision transformer models. An interactive demo showcasing our results can be found at https://disentangling-sbucaptions.xyz, and we make our DBK-SVD implementation openly available at https://github.com/RomeoV/KSVD.jl.

Google Cloud Credit Grants

Proposal Requirements

Reporting Requirements

Data Security

Award Selection

Terms & Conditions

Understanding Social Reasoning in Language Models with Language Models

Finding Monosemantic Subspaces and Human-Compatible Interpretations in Vision Transformers through Sparse Coding

Stay Up To Date

Navigate

Participate

Google Cloud Credit Grants

Proposal Requirements

Reporting Requirements

Data Security

Award Selection

Terms & Conditions

Understanding Social Reasoning in Language Models with Language Models

Finding Monosemantic Subspaces and Human-Compatible Interpretations in Vision Transformers through Sparse Coding